Ever need to know what words were in what emails? Ever need a cross reference for those words to what emails they came from. Don't want to spend a lot of money to get this done but want to be able to do this with many mailbox types and do it quickly? Well do I have some good news, with some Perl scripting, a sqlite database (I told you I love databases) and 2 programs from Fookes Software, Aid4mail and MailBag Assistant (both are also part of Paraben's email examiner).

So here is what you need to do. I will use a Outlook pst as an example. First open up Aid4Mail and export your pst file to a directory into eml format (make sure you recreate the directory structure of the mailbox). Next open up Mailbag Assistant and import all the eml files including the subdirectories. You will need to create the following script and template to use (I will put all the files in a zip archive and put them on my webserver for you).

Script: Save_Body_As_Text

IfEmpty End
MergeData Save_Body_As_Text

Template: Save_Body_As_Text

>>>Files ?\{Mailbox}\{Subject}.txt
{Body}

The script will take all the selected emails (alt-a) from the "Grid View - Main" and run the template unless no emails were selected. The template will save the text body of the eml file to a directory you will be prompted for with a structure of <Directory Specified>\<Mail Box, IE: Inbox, Deleted, etc..>\<subject line>.txt. Once all the files have been extracted, run the get-word.pl Perl program passing the top level directory of where the email bodies were extracted to you will extract all the words and put them into the database ( I am not include a listing of the program but will have it available for download). Now you can run sql against the database to find the keywords that you want, you can also run the following sql against the database to create copy statements for you so that you can copy the emails you want out to another directory (If you want to get even fancier then include a table with the keywords you are looking for and add a subselect to the query, if you don't know what that is email me and I will explain it further)

select 'copy "'||directory_found_in||'/'||filename_found_in||
'" "c:/stuff/test/test/'||filename_found_in||'"'
from word_file_xref a, words b
where b.word_seq_num = a.word_seq_num and word = 'Oracle';

You can also make a slight modification and add a table with words you do not want to see (IE: and, if, or, not, etc..).

I will package all the code and database create statements up and also include a exe of the Perl program in case you do not have Perl but still want to test out the program (I know the code is not the neatest but it is functional). It can be found here.

One interesting thing to note is that this could be the beginning of an open source e-discovery email production package. Any takers for a project like this?

Questions/comments/suggestions?

Incident Response Hash Set Creation....

I use x-ways forensics as my main tool and I am pretty impressed with the product and support you get from the vendor. One of the things that I have been doing is creating my own hash sets. X-ways allows you to create the hash sets using many different methods (sha1, md5, sha256, etc..). Since x-ways is very light I thought I would try a little experiment. Using version 13.0 I installed it on my Hard Drive (no registry settings needed and weighs just over 4M with the external viewer and hash database). I then RDP'd to a QA server and mapped a drive back to my machine. I then fired up x-ways and examined the drives on the QA machine. I was then able to create a sha256 hashset of each drive of the server (4 seperate hashsets at this point for 4 drives). I then exported the 4 hashsets into a directory and reimported the directory naming the hashset the same name as the server (aprox 78,000 hashs created). I then waited 4 hours and rehashed all the drives on the QA server and compared it to what I created earlier. I was left with aprox 150 files that I had to look at, makes life a lot easier during a incident response. This is one of the many features X-ways has that can be used to help during Incident response.

Posting of Sample Notes

As requested I am putting up a sample of the information I have (it has been sanitized) of some notes I recently took during an investigation. The file is here. In the future when you leave comments if you can let me know who you are I would greatly appreciate it. If you don't feel comfortable leaving your name then just shoot me an email Mark.McKinnon@sbcglobal.net, I like to know who is requesting things and commenting.

I know I have not blogged lately and I am getting some stuff ready to share with everyone so be patient. If anyone has something they want passed along let me know and I will pass it along. You can contact me at the above email address. Make sure you put something in the subject relating to the blog.

Anyone will to share any file hashs that they have built? I have some hashs that I am putting together and will try and get them out within the Month.

Sorry this is short but more will be comming.