Offline Folders

Offline folders use to go by the name of Client Side Cache. This is evident with the directory this information is stored in C:\Windows\CSC, this directory is still there even if you do not use offline folders. You will find Offline folders more in a corporate environment and mainly on laptops. The thoughts behind this is that you want to store your data on a network drive but also have access to it when you are not on the network. There is a synchronization process that happens between your computer and the network drive where your data is stored. Depending on what your settings are is when the synch will happen.

One of the interesting things about this is that if you login into a laptop that is not yours at your company, your files on the network drive will start to synch to that laptop. After the synch your files should now be on that laptop. Now lets say you are looking at leaving the company and decide to remove all your files from the network drive and then resynch on your laptop, all the data is then removed from the offline folder on your laptop and is gone. Now what about that other laptop you logged into, guess what your files are still on that one and they can be potentially harvested. Now all you E-Discovery folks should be drooling at the mouth right about now since files that were deleted may be found somewhere else (especially if the backup tapes of the network drive are no good, lost, etc..). You just have to find out where you logged into besides your own laptop.

Now one downside to this is that your cube mate is an idiot and stores his porn on the network drive. He decides to login to your laptop and his files are now on your laptop. There is an investigation and they take both yours and his laptops. Without understanding Offline folders you may get accused of having porn on your laptop when you never put it there, your idiot cube mate did.

Now lets take a high level look at the offline folders (I am still gathering information so there may be some holes in it). Under the C:\Windows\CSC directory you will the following:

Directories named d1 to d8 - these hold all the files used for offline folders, the file names are system generated.

file 00000001 - this points to the network drive that you will synch to

file 00000002 - this files holds all the references to what directories your files are stored in and what there names are.

file 00000003 - Don't know have not figured this out yet (I did say this was a work in progress and any help would be appreciated)

file csc1.tmp - this appears to be a copy of file 00000002

Now in each directory (d1..d8) you will find 2 types of files, ones that have a first character of 0 or 8. The ones with a first character of 8 are the actual files that you stored there. The files that start with 0 hold the information/cross reference between the generated name and what their actual names are as well as size of the file and the date that the file was created (this is another place where I am still figuring it out but I do have some of the information).

In the next post I will dive deeper into the format of the files that start with 0 and provide some Perl programs that will be able to read those files and provide some useful information.

Now hopefully I was clear in what I just stated if not hopefully you will let me know.

Questions/Thoughts/Comments????

It's been a while

It has been quite a while since I last posted something. I hope to soon rectify this and start to post a few things. Some of the things that I hope to talk about will be Offline folders, a few informational postings on different programs, and other things.

Comparing Large Hash sets Against NSRL.......

I recently saw a post on a list I belong to asking about DeDuplicating and DeNSRLing some files. He was trying to do this in a very popular forensic product and after 4 days he still had nothing. Someone replied (I had thought the same thing) about using a SQL Server database to do this. Now if you are not that familiar with using databases then this would not be an easy task. Thinking about this I thought it would make a good project. To start off you first need to accommodate a large amount of data and it should perform well (that is a bigger challenge then you may think).

The parameters for the project are:

1. The NSRL reference table will only hold 1 set of hash values (I chose MD5 to use but you could choose SHA1 or CRC).

2. Load the NSRL data in a timely manner.

3. Be able to add my own hash sets to compare against as well.

4. Use as much free software as possible.

5. Load my hashs to compare in a timely manner.

6. Compare my hashs in a timely manner.

7. Be able to easily report and extract knowns and unknown hash sets from what I loaded.

8. Work on both Windows and Linux (Sorry Mac)

I started off by using SQLite with a perl script to load the NSRL data. I was able to load the NSRL data in aprox 1 hour which for the amount of data and an embedded database I thought was pretty good as well as you would only do this task possibly once a quarter. The problem came next when I tried to create an index on the table and it went out to lunch. After a couple of hours I knew I would have to come up with a different database solution. I then looked at the free version of Oracle (I am pretty familiar with this database and it also has a Linux version, that is why I chose it over SQL Server), now here is where it starts to get hard since I am limited to only having 4GB of data in the free version. I installed it without a problem and started it up. It was using aprox 300M of memory so for anyone out there wanting to do this you should probably have 1gb of memory on your machine.

I next started to create some tablespaces, users and tables. I then used Oracle's SQL Loader product to load the data into the database and then indexed the table. This took about 3.5 GB between the index and table (40,000,000+ rows). I then created a list of hashs from a previous examination that using x-ways forensics version 13. I then loaded this data into the database (600,000+ rows) and then created a table of known and unknown hashs for the examination. After trying many different things to make it fast and small I finally came up with the following:

NSRL table is deduplicated from 40,000,000 rows down to 14,000,000+ rows and from 3.5 GB (table and index) down to 1.2gb (table and index) with a load time of aprox 36 minutes.

My hash set was smaller then 500m and took aprox 5 minutes to load the 660,000+ rows and create 2 tables (known hash set and unknown hash set). The known hashs table has aprox 46,000 rows with the unknown hashs tables having 604,000+ rows.

Now I have uploaded the scripts here (sql and sqlload) and batch files to run to create your own little hash comparison system. There is a install.txt file to help you get started. Once you install Oracle Express and download the NSRL data you should be able to get started.

If you don't want to use the MD5 that I did then just change the MD5 references to SHA1 or CRC and then the load cards to only load what you want. You can also change the hash set tables to what ever you want to load. Just use what I supplied as a template to make your modifications. With a little creativity you can also create your own list of knowns and unknowns and use these to compare against as well, just use the nsrl schema as a template.

Now looking back I feel I accomplished everything I set out to. It is fast, 41 minutes from start to finish if I do not have the NSRL already loaded, otherwise it takes roughly 5 minutes for 660,000+ rows. It is a free solution. I can now export the rows, create reports as well. Using Oracle Express I can run it on either Windows or Linux platform and since I do not use any gui tools there are not too many modifications to make it work on either platform. I would love to hear your experiences with using this and what timing's you get with your hash set comparisons.

Questions/Comments/Thoughts?

Thumbs DB Files

I received a email about a new product from InfinaDyne. It is called ThumbsDisplay and you can display the contents of the Thumbs.db file. It will also do the following:

Cut and paste the picture to another application

Print 3 types of report (Contacts Sheet with all the pictures displayed, Picture with date and time, Full Size picture with date and time).

Scan the drive for all thumbs.db files.

You can also call the program with a thumbs.db file as a parameter and it will load that file into the viewer. This is really nice since you can then use it to view thumbs.db files from within other forensics programs, ie: X-Ways Forensics. One of the best things about this program is the price, only $29.99. If you want to test drive it before you buy they also have a demo version you can download.

The only draw back I see right now is that you can only print the reports, you can't save them. You need someting installed like cutePDF to print the file to a PDF file. Maybe in a future release they will add this feature. Otherwise it seems like a great inexpensive tool to keep in the toolbox. And in case you are wondering I did pay for my own copy of the program I am not getting anything free here.

Thoughts/Comments/Questions.

Registry Files in the Restore Point.

Your in the middle of an examination of an Windows XP machine and your wondering what some registry settings were during a specific time and you think to yourself, why don't I look in the System Restore Point. As you navigate to the restore point directory all of a sudden you see 20+ restore points and you think "Oh ????? (insert word here)". As you look at all the restore points you start to think how are you going to get all that information out and not take forever. You only want to look at 5 different registry keys over some time period that resides within those 20+ restore points. Don't despair I have a solution.

What I have done is taken Harlan Carvey's regp.pl program and modified it to scan a directory and read the raw registry files and insert the entries into a SQLite database (of course). I then created a program to read the database and output registry keys in chronological order so you can see the dates and times of what the entries are along with the restore point they belong to in a comma separated file. For example here is a sample of the output looking at the following registry keys.

Registry File Name, Registry Key, Last Write Date Time, Registry Key Name, Data Type, Registry Value, Registry Value, File Location
_REGISTRY_MACHINE_SOFTWARE,\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Prefetcher,Wed Apr 18 20:55:01 2007,StartTime,2007/04/18-16:55:01, //-:U:,c:/mark/restore/RP603/snapshot,
_REGISTRY_MACHINE_SOFTWARE,\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Prefetcher,Tue Apr 17 23:29:36 2007,StartTime,2007/04/17-19:29:36, //-:):6,c:/mark/restore/RP602/snapshot,
_REGISTRY_MACHINE_SOFTWARE,\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Prefetcher,Mon Apr 16 22:57:56 2007,StartTime,2007/04/16-18:57:56, //-:W:V,c:/mark/restore/RP601/snapshot,
_REGISTRY_MACHINE_SOFTWARE,\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Prefetcher,Sat Apr 14 21:10:18 2007,StartTime,2007/04/13-13:41:27, //-:A:,c:/mark/restore/RP600/snapshot,


_REGISTRY_MACHINE_SOFTWARE,\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Prefetcher,Sat Apr 14 21:10:18 2007,ExitTime,2007/04/13-12:22:04, //-:":,c:/mark/restore/RP600/snapshot,
_REGISTRY_MACHINE_SOFTWARE,\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Prefetcher,Mon Apr 16 22:57:56 2007,ExitTime,2007/04/16-16:05:08, //-::,c:/mark/restore/RP601/snapshot,
_REGISTRY_MACHINE_SOFTWARE,\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Prefetcher,Tue Apr 17 23:29:36 2007,ExitTime,2007/04/17-16:33:14, //-:3:,c:/mark/restore/RP602/snapshot,
_REGISTRY_MACHINE_SOFTWARE,\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Prefetcher,Wed Apr 18 20:55:01 2007,ExitTime,2007/04/18-15:51:38, //-:Q:8,c:/mark/restore/RP603/snapshot,

Pretty cool.

Now for the gory details. The main program takes as input a directory (where you exported the restore point to) and a database file name that you want to create. It scans the directory recursively until it finds a file named _REGISTRY (the beginning name of all the registry files in the restore point) . It then opens that file and parses it and inserts the records into the database. As it inserts the records it will take anything with a record type of binary (with a length less then 2000 bytes) and convert it to ascii so it is potentially readable. The report program takes a database file name and output file name as parameters. It reads a txt file that specifies what registry entries will be outputted. I have also included a autoit gui front end for the command line averse folks. The gui front end end will ask for the restore point directory and database file name for reading the registry and the database file name and output directory for the report. You can specify a verbose mode which will tell you what files you are currently processing. There is one more option to choose on the gui and that is the file extension to run, I did this in case you want to run either the .pl (perl source) or the .exe (executable version of the perl source).

One small problem with the program is that reading the registry files is pretty slow. In my testing I had a total directory size of aprox 250M (only counting the registry file sizes) which included 4 restore points and it took about 20 minutes to parse all of them. I have looked at the program at most of the time is in reading the registry files themselves not inserting into the database. The report runs pretty quickly though. One thing to note I felt it was quicker to get everything versus looking for what I want since what you want may change during the exam or overtime and the only thing you would then have to change is the report ini file.

Hopefully I have not confused everyone. Some of the code is ugly and all the comments may not be there so I apologize for that. As always report problems and so forth back to me and hopefully it helps out, saves time and gets you the data you need.

The zip file with all the goodies can be found here.

Questions/Comments/Suggenstions?

U3 Smart Technology.......

Man do I love technology sometimes. What is great about U3 smart technology is that as long as autorun is enabled for the cd you can potentially tell when one of these USB devices has been plugged in. By looking in the windows prefetch directory all you have to look for are these files, Launchu3.exe, Launchpad.exe and cleanup.exe. The 2 launch programs are run when ever the USB drive is plugged in (assuming autorun is enabled). The cleanup program is run whenever the USB drive is ejected using the launchpad.

Now if you are lucky you may see multiple entries for these files in the prefetch or you can see different create and modified dates for them as well. Now you may also notice that these files may have multiple different dates and times. Here is an example from the prefetch directory of the multiple dates and times.

Filename Created Modified Accessed
LAUNCHU3.EXE-XXXXXXXX.pf 2/5/2007 13:56 2/13/2007 5:52 2/13/2007 5:52
LAUNCHPAD.EXE-XXXXXXXX.pf 2/5/2007 13:57 2/13/2007 5:52 2/13/2007 5:52
CLEANUP.EXE-XXXXXXXX.pf 2/12/2007 21:54 2/13/2007 7:01 2/13/2007 7:01

Looking at these entries in the prefetch it tells me that the USB drive was attached on February 5, 2007 and also February 13, 2007. The drive was then removed on February 12 2007 and February 13, 2007. Pretty cool that I can tie the USB device to being used on 3 separate occasions. Also by looking in the setupapi.log file ypu can see when the drive was first attached which potentially adds a 4th time the drive was attached. Now you see why I love technology sometimes.

Thoughts/Comments/Questions?

URL History.

I wrote this program back in December 2005, what it does is to read in a ie or mozilla history file and will output it to a comma delimited, tab delimited or html file. You can also open it in Excel or a browser and sort the records in ascending or descending order. I know there are many programs that will do this but this program has one special feature that I added, you can make it output url records between specific dates so you can narrow down your search of url records.

When I created this I modeled it after pasco. It is a gui program and that is why it is so large, this would probably be a good candidate for an autoit front end instead of perl. One thing I did find out about pasco is that it looks in the index.dat file for the size of the file and only reads until the file size. What I found is that file size stored in the index.dat files is not always kept up to date. My program just reads until the end of the file so it will always get all the records.

The code and executable can be found here. As always comments, suggestions, improvements to the program are always welcome.


Thoughts, Comments, Suggestions?