Have you ever wondered what these values are? The two scenarios we run into most frequently are as follows:

1) LegacyExchangeDN & X.500 Addresses

In an Ms Exchange organization, internal e-mails are routed using X.500 addresses instead of SMTP addresses. The X.500 address of each mailbox is stored in the legacyExchangeDN attribute in Active Directory, which is set when a mailbox is created and includes the name of the Exchange Administrative Group where the mailbox belongs. LegacyExchangeDN values typically look as follows:

/O: Organization Name
/OU: Organizational Unit
/CN: Common Name

Starting with Exchange 2007, user-defined Exchange Administrative Groups were replaced by a single administrative group called “Exchange Administrative Group (FYDIBOHF23SPDLT)”. The value “FYDIBOHF23SPDLT” is actually an encoded version of the string “EXCHANGE12ROCKS” with each character replaced with the letter that follows it in the alphabet (E->F, X->Y etc.).

When an e-mail that was sent within the Exchange organization is taken outside (i.e. for e-Discovery processing or digital forensic analysis), the SMTP e-mail address for the user (e.g. abc@cde.com) can no longer be resolved and the only available address would be the legacyExchangeDN value. In this scenario, the e-Discovery processing output may look similar to the example image in Figure 1.

Figure 1 – E-mail with X.500 Addresses

2) IMCEA Encapsulation

Another common scenario is IMCEA encapsulated addresses. The sender and recipient addresses for each message are looked up in the Global Address List (GAL) before the message is sent. If the SMTP addresses cannot be resolved (e.g. they are hidden from the GAL), the look-up fails and Exchange is forced to encapsulate the only address available (the X.500 directory name) using Internet Mail Connector Encapsulated Addressing (IMCEA). Addresses encapsulated in this manner would look as follows:

The string “EX” at the end of “IMCEAEX” indicates that the encapsulated non-SMTP address was an Exchange address. The encapsulation process replaces each forward slash “/” with an underscore “_” and each symbol with a plus sign “+” followed by its two-digit hexadecimal ASCII code (e.g. +20 for the space character).

In this scenario, the e-Discovery processing output may look similar to the example image in Figure 2. Please note that “domain.com” represents the SMTP domain that is used to encapsulate the non-SMTP address, which may not be the same domain where the original address belonged.

Figure 2 – E-mail with IMCEA Encapsulated Addresses

Preservation of Metadata

One of the most common issues associated with compressing electronic evidence has to do with the preservation of metadata. Unless proper care is taken, compressing electronic files can result in loss of valuable file system metadata.

In our experience, the most common file compression tools used in e-Discovery are WinZip, WinRAR and 7-Zip. Surprisingly, some of these applications do not capture and restore file system timestamps by default. The following table summarizes which metadata timestamps can be preserved using each software:

Table 1 – Date Metadata Preserved by File Archive Software

Based on the table above, only WinZip and WinRAR support preserving file creation dates. WinRAR captures file creation timestamps only after the option is selected while WinZip captures them by default. Consequently, if files were compressed using 7-Zip, or using WinRAR without the correct date options, their file system creation timestamps would be stripped off. This means that even if the electronic evidence was collected properly and file system metadata was preserved, the compression process can prevent this information from being transmitted to the recipient.

The relevant options during compression and extraction in WinRAR are as follows:

Figure 1 – WinRAR Date Settings (Compression)

Figure 2 – WinRAR Date Settings (Extraction)

Encryption

Security is always a valid concern when evidence is transmitted electronically. When compressing files, we recommend using encrypted archives as an additional security measure. The encryption password should be strong, and different than the credentials required to access the file transfer system. WinZip and 7-Zip support AES-256 encryption while WinRAR uses a maximum AES key size of 128 bits. WinRAR and 7-Zip support encrypting file names while WinZip does not.

Long File Paths

Another common issue is compressing or decompressing files with very long paths. In most cases, you can work around this issue by mapping the source (if compressing) or destination (if decompressing) folder path as a network drive and accessing the files through that drive letter. For example, let’s assume that the files we would like to compress are in the following folder:

\\server\share\Case Documents\Client Name\Sources\Case Name\Date\Data Set 1

We can map this folder to a drive letter such as Z: using the following command:

We can now compress the contents of, or extract the files to the Z: drive. In this example, this would shave 73 characters off the file paths.

Conclusion

• It is important to choose the right file compression tool and familiarize yourself with all available options. We typically recommend WinRAR as it provides a combination of good performance, compression and security features
• Encrypting file archives before sending them over the internet can be a valuable additional security measure
• In most cases, file paths that are too long can be accessed by mapping the parent folder as a network drive

Q1: What do “horizontal de-duplication” and “vertical de-duplication” mean?

These terms are related to the scope of de-duplication. Horizontal de-duplication refers to de-duplication that is performed globally, across custodians while vertical de-duplication indicates that de-duplication scope will be limited to each custodian’s documents.

When documents are de-duplicated horizontally, all but one copy of a document is removed from the document universe regardless of which custodians had the same document. On the other hand, when de-duplication is performed vertically, multiple copies of the same document may be left in the document universe as long as each copy originated from a different custodian. This would allow the legal team to gain a greater understanding of the custodians at the expense of having more documents in the review database.

Q2: I chose to have my documents de-duplicated prior to processing. Now, how do I track down duplicate copies of a certain document?

When your documents are de-duplicated, duplicates are flagged in the service provider’s back-end database instead of being removed permanently. Every time de-duplication is performed, your service provider should include a de-duplication report that lists de-duplicated copies of each document and at a minimum their hashes, sizes, file names and folder paths.

Q3: How are attachment families handled during de-duplication?

De-duplication should normally be performed at the attachment family level rather than document level. In other words, an e-mail message and all of its attachments would have to be identical to another e-mail family in order for them to be considered duplicates. This would ensure that an e-mail attachment would not be de-duplicated against a loose electronic document and removed from its family.

Q4: What is a cryptographic hash?

A cryptographic hash is a fixed-size signature for an arbitrary block of data that represents its contents. Hash algorithms are designed such that even the slightest change to the original data changes the signature dramatically. These signatures can then be used to compare documents and identify duplicates. Message Digest 5 (MD5) and SHA-1 are two popular cryptographic hash functions used in e-Discovery.

Q5: Can two documents with different file names or dates be considered duplicates?

Yes. De-duplication is usually performed by comparing cryptographic hashes (e.g. MD5, SHA1 etc.) of documents to each other. The calculated hash values are based on the binary contents of documents and do not take into account external metadata that is stored in the file system. Therefore, two files with the same contents but different file names would produce the same hash value.

Most e-Discovery service providers would allow you to use a custom hash that includes your choice of metadata fields in addition to document contents for de-duplication. For example, you could choose to include the file name field in your custom hash if you would like to make sure that documents can be considered duplicates only when their file names are also identical.

Note: Some document types (i.e. Ms Office documents, Adobe PDF files etc.) contain internal metadata including dates. Since this information is stored inside the document, it would affect the calculated hash value.

Q6: Are e-mails hashed the same way as loose electronic documents?

An e-mail is essentially a set of fields stored in a container. This container can hold an individual message (e.g. an MSG file) or it can be a database-like structure containing multiple messages (e.g. PST, NSF etc.). Consequently, most e-Discovery software compute cryptographic hashes for e-mails based on the metadata values found in a predetermined set of fields. The following is a list of fields typically used for e-mail de-duplication: Author, Recipient(s), CC, BCC, Date Sent, Subject, Attachment Count, Attachment Names, Message Body

E-mail messages contain many more fields than the fields typically used for de-duplication and some of these fields can have variations among multiple copies of an e-mail message. For example, 4 copies of a message may be read and the fifth copy may be unread. Depending on the nature of the case, the contents of these additional fields may or may not be relevant. However, it is always important to clearly define what should be considered a duplicate at the onset of each project.

Q7: What are the chances of two different documents having the same MD5 hash?

Extremely small. In a set of 2^64 (18,446,744,073,709,551,616) documents, the chances of two different documents having the same MD5 hash is 50% (birthday paradox).

Q8: I heard that MD5 is now considered cryptographically broken. Is it good enough to identify duplicate documents?

The fact that MD5 is now cryptographically broken means that an attacker can create a pair of non-identical files that produce the same MD5 hash. Keep in mind that this is different than a preimage attack where an attacker produces a file that matches a specific, known hash value. There are no known preimage attacks against MD5 as of this writing.

Briefly, MD5 is currently considered suitable for identifying duplicate documents. However, our recommendation is to hash each document using more than one algorithm (e.g. MD5 and SHA-1) to alleviate security concerns. We anticipate that SHA-256 will be the new hash standard for e-Discovery in the near future.

What are Time Zones?

Due to the shape of the earth and its rotation around the sun, different parts of our planet observe different parts of the day at the same time. Time zones are used to ensure that the same clock time corresponds to the same part of the day throughout the world. Traditionally, all time zones were represented as an offset from the Greenwich Mean Time (GMT). As technology advanced and more accurate time keeping devices became available, Universal Coordinated Time (UTC) emerged as the new international time standard. UTC is a form of atomic time and is regularly modified (leap seconds) in order to account for the irregularities of the earth and sun’s movements.

How Date/Time Values are Stored

Different computer systems and software use different methods to store timestamps. The following are a few examples that we encounter frequently.

1. Ms Outlook Emails

Ms Outlook stores e-mail dates in UTC and displays them in the end user’s local time zone. The example e-mail in Figure 1 was sent on 04/16/2012 at 1:08 PM Pacific Daylight Time (PDT). You can see that Outlook displays the sent date in the local time zone (PDT) while it stores it internally as the PR_CLIENT_SUBMIT_TIME value in UTC. On the other hand, the message header shows the same value in Eastern Daylight Time (EDT) since the server through which the message was sent was in EDT.

Figure 1 – Dates in an Outlook E-mail

2. Ms Office Documents

Most Ms Office documents also store internal document metadata in UTC. For example, the Ms Word document in Figure 2 was created on 4/16/2012 at 1:22 PM (PDT). Looking at the docProps\core.xml file reveals that the internal metadata was stored as “2012-04-16T20:22:00Z”, which is the same as 04/16/2012 8:22 PM (UTC).

Figure 2 – Dates in a Word Document

3. File Systems

File systems (e.g. FAT32, NTFS etc.) also store valuable date/time information about files. However, the way they store this information varies. For example, NTFS stores timestamps in UTC, as the number of 100 nanoseconds since 1/1/1601 (UTC). On the other hand, FAT file systems store date/time values in local time without a time offset. This means that one would have to know the correct time zone in which a file was created and used in order to determine its actual local date/time. For example, imagine the following scenario:

• File 1: Created in a FAT32 file system on 06/01/2004 at 9:32:15 PM (PDT)
• File 2: Created in an NTFS file system on 06/01/2004 at 9:32:15 PM (PDT)

If both files were viewed today on a computer in EDT, File 1′s creation timestamp would appear as 06/01/2004 9:32:15 PM while File 2′s timestamp would be 06/02/2004 12:32:15 AM. The difference is caused by the fact that FAT32 only stores the date and time without the time offset and even if the file was later viewed in a different time zone such as EDT, its timestamp would not reflect the local time zone in which the file is viewed.

Potential Issues

1. Processing Output

Applications that store timestamps in UTC typically adjust the displayed value depending on the end user’s selected time zone. This means that the same document can produce different output during e-Discovery processing depending on the chosen processing time zone value. For example, the processing output for the test e-mail we mentioned earlier would be as follows:

Figure 3 – E-mail Output in Different Time Zones

As seen in Figure 3, the e-mail message was printed with a sent time of 4:08 PM when processed using the Eastern Daylight Time (EDT) zone. While this may not always be an issue, it is important to ensure that legal teams understand that the printed date/time values may or may not be the actual local date/time the e-mail was sent by its author or received by its recipient. In some cases, it may be possible to process e-mails in a way that the actual time offset is displayed in the output (see Figure 4).

Figure 4 – E-mail Printed with Time Offset

Additionally, making sure that the review database always contains fielded information regarding the selected processing time zone and the UTC timestamps for each document would help alleviate this issue.

2. De-Duplication

Most e-mail platforms create cryptographic hash values for e-mails based on a combination of several metadata fields such as sent date, author, e-mail subject etc. While calculating these hash values, e-Discovery software should use UTC timestamps rather than local time. Otherwise, e-mail hashes would be a function of local time and would change depending on the selected time zone. This means that two copies of the same e-mail processed in different time zones would not be treated as duplicates.

3. Date Restrictions

When documents are culled using date restrictions, the selected time zone may determine whether or not a document falls within the relevant date range. For example, imagine an e-mail that was dated 04/30/2009 9:05 PM (PDT). If a date restriction was performed using Eastern Time, this e-mail would not fall into the date range 1/1/2009 – 4/30/2009 since the e-mail’s local date would be interpreted as 5/1/2009 12:05 AM (EDT).

4. Daylight Saving Time

Daylight Saving Time (DST) is the practice of advancing clocks forward by one hour in spring and moving them again backwards in autumn so that afternoons have more day light and mornings have less. The start and end dates of DST have changed several times in the past, making it a challenge to determine the actual local timestamps of documents in the past. For example, in order to determine the correct local timestamp of a file dated 3/12/1991 12:45:31 PM (UTC) that originated from Sweden, one would have to know whether or not DST was in effect at that time in Sweden. The tz database is a valuable resource which contains the history of local time for many representative regions in the world.

Conclusion

We believe that taking the time to ensure that time zones are handled correctly in a project should be at the top of a project manager’s priority list. Briefly, the following key points should be considered:

• Legal teams should decide how time zones should be handled at the onset of each project and communicate their preference to the e-Discovery service provider as part of e-Discovery processing specifications. It is common practice to normalize all times to UTC and capture each custodian’s time offset in cases involving multiple time zones
• e-Discovery software should normalize document timestamps to UTC while performing de-duplication
• The time zone used during e-Discovery processing as well as the UTC timestamps of each file should be included as additional fields in the review database
• Effects of time zones and Daylight Savings Time should be considered while constructing date restrictions
• Legal teams should be aware of the fact that depending on the chosen time zone, timestamps printed as part of processed documents (e.g. e-mail dates) may or may not reflect the actual local time when the e-mail was sent or received

There are commercial off-the-shelf file copy tools which have this functionality built-in, but they usually lack the flexibility that Robocopy offers. If you are a Robocopy fan, and do not mind a little bit of command line work, follow along and we will show you how to validate Robocopy results using the freely available software package md5deep.

What is md5deep?

md5deep is a command line application in the public domain. It can be used to calculate cryptographic hashes (MD5, SHA-1, SHA-256, Tiger192 and Whirlpool) of files. It can walk through directories recursively and calculate the hashes of each encountered file or work off of a text-based file listing. We chose to use md5deep for this post because it is fast, robust and free.

Required Tools

• md5deep – Available for free at http://md5deep.sourceforge.net
• Your text editor of choice (e.g. UltraEdit, TextPad etc.)

When you download md5deep, remember to either copy md5deep.exe to your Windows\System32 folder or add its path to your path system variable so that it can be accessed from anywhere.

This post assumes that:

• You have copied a set of files using Robocopy as outlined in this post
• The copy operation completed successfully without any errors
• Your source and destination folder paths were as follows:
  
  Source: D:\MySourceFiles\
  Destination: E:\MyDestination\

Step 1: Calculate the Hashes of The Source Files

We will use md5deep to calculate the hashes of all files in our input folder (“D:\MySourceFiles\”). md5deep outputs the calculated MD5 Hash values to the console. In order to save the output, we will redirect it to a text file using the “>” symbol. The steps are as follows:

• As always, make sure that your source is write-protected before accessing it
• Open a command prompt at your source folder “D:\MySourceFiles”
• Issue the following command:
  
  md5deep -rel * > "C:\Temp\InputHashes.md5"

  The -rel switch instructs md5deep to enable recursive mode, display a progress indicator and use relative file paths.

This will create a list of MD5 hashes for each file contained in your source folder. The list should look as in the example below:

Figure 1 – Input Files MD5 Hash List

Note that the resultant file “InputHashes.md5″ is a UTF-8 encoded text file and non-English characters in the file names were preserved.

Step 2: Calculate the Hashes of The Output Files

Similarly, we can calculate the MD5 hashes of the output files using md5deep as follows:

• Open a command prompt at your destination folder “E:\MyDestination”
• Issue the following command:
  
  md5deep -rel * > "C:\Temp\OutputHashes.md5"

Step 3: Compare The Hash Lists

At this point, you should have two hash lists: “InputHashes.md5″, which contains a list of MD5 hashes for the source files, and “OutputHashes.md5″, which contains a list of MD5 hashes for the output files. Since we chose the relative file path option while using md5deep, both hash lists should contain the same folder paths. Consequently, if all files were copied correctly, both hash lists should be identical.

We can easily check whether or not this is the case by hashing the hash lists and comparing them. We will use the following commands:

Contents of “Comparison.txt” should be as follows:

Figure 2 – Comparison.txt Contents

If both hash values are identical, we can conclude that each output file has the same MD5 hash value as the corresponding source file. If you would like to confirm that the process is working correctly, you can edit “OutputHashes.md5″ and change one of the MD5 hashes. When you re-run Step 3, the MD5 hash of the output hash list should be different than that of the input hash list.

What If The Hashes Do Not Match?

In the event that the two hash lists turn out to be different, you may want to determine which files have different hash values. A quick and easy way to accomplish this is to use a file comparison tool such as WinMerge (open source) or UltraCompare (commercial). These tools allow two files to be opened side by side (“InputHashes.md5″ and “OutputHashes.md5″ in this case) and highlight the differences.

Couldn’t We Have Used The Negative Matching Mode In md5deep?

md5deep has an option that enables negative matching mode. In this mode, the program takes a list of known hashes and identifies files that are outside of that list. For example, opening a command prompt at your destination folder “E:\MyDestination” and issuing the following command would create a list of all files in your destination folder that have hashes outside of the input hash list:

This looks like a very efficient way to determine which files were not copied correctly. However, if we had used this method, we would have missed two scenarios:

1. Files completely missing from the destination
2. Files that do not match their source file by MD5 hash, but match another file in the source data set

The second scenario may sound far-fetched, but consider the following example: The input folder contains a number of 0-byte files. If a file, which was not originally a 0-byte file, does not get copied correctly and becomes a 0-byte file in the destination, it would not be identified using the negative matching method because its hash matches that of other files in the source data set.

We typically recommend extracting all compound documents. However, we feel it is important that what this really means is understood clearly and an informed decision is made based on case requirements. We have come up with a few points for you to consider when making such a decision that will hopefully help you determine which route you should take.

What are Embedded Objects?

Many file types, including Microsoft Office and Adobe Acrobat files, act as containers and allow other documents to be linked to them or embedded in them. For example, one can embed a file into an Ms Word document by simply dragging it into an open Word document.

Depending on the file type and method used, the embedded document may or may not be directly visible in its parent document. For example, the contents of a single-page Visio drawing inserted into an Excel spreadsheet can be visible when the spreadsheet is viewed, while a ZIP file or an MSG file inserted into a Word document would typically be displayed as an icon and its contents would not be directly visible.

Extracting embedded objects means that the e-Discovery software identifies each linked or embedded document and extracts it (and its children recursively) as separate records during processing. Additionally, a parent/child relationship is established between the container document and the files embedded in it.

Advantages of Extracting Embedded Objects

• Completeness:

  In some cases embedded documents would not be processed at all unless embedded objects are extracted. This could result in critical content being missing from your review database and production. Take a look at the following three examples:

  Example 1: Documents Displayed As Icons

  Contents of documents that are displayed as icons in their parent document would not be processed during e-Discovery unless embedded objects are extracted. For example, the embedded archive (Embedded File1.zip) and the embedded e-mail message (Embedded File2.msg) in Figure 1 below would not be extracted and processed unless embedded objects are extracted. Please note that the embedded archive contains several child documents, which also need to be extracted recursively.

  

  Figure 1 – Word Document with Embedded Objects

  Example 2: Charts in Ms Office Documents

  When a chart is created in or inserted into an Office document, the underlying data is typically stored as an Excel spreadsheet. While the parent document only displays the resultant chart, the underlying spreadsheet can contain much more data than what is represented in the chart.

  The example Powerpoint presentation in Figure 2 contains an embedded chart. When the underlying Excel workbook is opened, it becomes apparent that the workbook contains additional worksheets.

  

  Figure 2 – Powerpoint Document with Embedded Chart

  Example 3: PDF Portfolios

  PDF portfolios contain multiple documents combined into a single PDF file. Documents contained in a PDF portfolio can be in various formats such as spreadsheets, e-mails, Powerpoint presentations etc. For example, one can select certain e-mail messages in Ms Outlook and convert them – including their attachments – to a PDF portfolio using the Adobe Acrobat PDFMaker Outlook Addin.

  Figure 3 exemplifies a PDF portfolio created directly from Ms Outlook. One of the original e-mails contains an attachment (“Analysis.xls”), which was included in the portfolio as an embedded Excel spreadsheet.

  

  Figure 3 – Adobe PDF Portfolio

  Extracting embedded objects would ensure that each item in the PDF portfolio is extracted as a separate record and processed.

• Fully Searchable Database:

  Even though some documents may be visible in their parent document, they may not be fully searchable. Extracting embedded objects would ensure that each object is tested separately and OCR’ed if it is found to be missing extractable text.

  Example 1: Images Embedded Into Searchable Documents

  We often run into images embedded in otherwise searchable documents (for example, an excerpt from a scanned page embedded into the first page of an e-mail). This poses an interesting challenge. Most modern e-Discovery software detect non-searchable documents and run them through optical character recognition (OCR) on-the-fly to make them searchable. However, this analysis is usually performed at a document level, or page level. If a page contains some searchable content, it passes as searchable and is not run through OCR.

  In the scenario below (see figure 4), we have a scanned image inserted into an otherwise searchable page. Most e-Discovery software would treat this page as a searchable page and would not attempt to OCR it. This would result in the text that could be extracted from the embedded image via OCR being lost. On the other end of the spectrum, if the scanned image is detected and the entire page is OCR’ed, the accuracy of text that could be extracted from the searchable part of the page would be reduced. Extracting embedded objects is a good option to make such documents searchable. When extracted, the embedded image would become a separate document itself. It would easily be detected as a document without extracted text and OCR’ed.

  

  Figure 4 – E-mail with Embedded Inline Image

• Native Productions:

  Producing complex documents in native form is a common trend. One of the greatest risks of a native production is producing more than intended. For example, a spreadsheet or a complex file type can contain embedded documents that can be overlooked during review unless they were extracted as separate database records. Producing such a document in native format would mean giving your opponent data that you haven’t seen during review.

  Extracting embedded objects helps make sure documents can be fully reviewed and minimizes the possibility of missing hidden content during review.

Disadvantages of Extracting Embedded Objects

• More Documents to Review:

  When embedded objects are extracted, separate database records are created for each child document. This results in a more crowded database, and potentially more documents to be reviewed. Imagine a Word document with 20 embedded objects: When embedded objects are extracted, the number of database records for this document increases from 1 to 21. If the child documents in turn have objects embedded in them, the number of database records after extraction can be even larger.

• Duplicative Content:

  Some embedded objects may be entirely visible in their parent document, making the extracted version redundant. Keep in mind that, even though the embedded object may be visible, it may not be searchable (see “Images embedded into searchable documents” above). So, this is not always a disadvantage.

• Incomplete Objects:

  Some embedded objects may not be complete, self-sufficient documents. Consequently, some of the extracted documents may not be directly viewable using their native application. Most modern e-Discovery software can work around this issue and still extract text & metadata from these objects and convert them to TIFF.

• De-Duplication Issues:

  Until recently, some off-the-shelf e-Discovery processing tools had problems when embedded objects were extracted and de-duplication was performed at the attachment family level. The issue was that some embedded objects were having minute differences in content (and therefore MD5 hash) each time they were extracted. Consequently, two identical compound documents were not being de-duplicated against each other due to the fact that one of their children had a different hash. To our knowledge, most software vendors have resolved this issue by now. That said, it is still worth keeping in mind when working with a new tool or service provider.

Conclusion

We are a proponent of extracting all compound documents. It is true that you may end up with more database records and it may take longer to review, tag and endorse the documents, but at least you can be confident that every document in the data set has been made searchable and reviewed. Bear in mind that it is easier to exclude database records from a production than it is to insert new records between existing documents. If you have embedded objects extracted up front, you would have the option to easily exclude them from your production if needed. On the other hand, if you don’t, and you later find out that some embedded objects should have been separate records, it would be more tedious to extract those objects after the fact and insert them where they belong.

Most of us are aware of the fact that opening a file usually changes file metadata as well as, in some cases, file contents. However, did you know that the mere act of copying a file from one folder to another using Windows Explorer causes the following changes?

• The file system last accessed date of the source file is updated to the present date/time
• The copy (destination file) receives the present date/time as its file system creation and last accessed dates

File system date/time values are valuable information that can be captured during e-Discovery processing or a forensic examination and can be used to shed light on, among other things, when a document was created, accessed and modified. You can help preserve this information by utilizing Robocopy to copy files instead of Windows Explorer.

Robust File Copy (Robocopy) is a free command-line replication tool from Microsoft. It has been a part of Windows distributions since Windows Vista, and is available as a separate download for earlier versions of Windows such as Windows 2003 and Windows XP. Even though it can take a large number of parameters and can look intimidating to use at first, it only takes a few minutes to get the hang of it and figure out which parameters suit your purposes the best.

Versions of Robocopy

Using Robocopy version XP026 or higher is recommended as some of the options that we will refer to here were not available before that version. Robocopy XP026 can be downloaded as part of Robocopy GUI v.3.1.2.

Robocopy Usage

Syntax

The basic Robocopy syntax is a follows:

Notable Options

The “/ndl” and “/np” switches are used here to control how the log file is formatted so that a file listing with full directory paths can be obtained. This listing can be fed into other scripts or software for further processing, such as generating MD5 hash values.

For a full list of options and more detailed information, visit http://technet.microsoft.com/en-us/library/cc733145%28v=ws.10%29.aspx

Sample log file

-----------------------------------------------------------------------------
 ROBOCOPY     ::     Robust File Copy for Windows     ::     Version XP026
-----------------------------------------------------------------------------

Started : Mon Mar 05 14:39:25 2012

 Source : D:\MySourceFiles\
   Dest : E:\MyDestination\

  Files : *.*

Options : *.* /TS /NDL /TEE /S /E /COPY:DAT /DCOPY:T /NP /R:3 /W:2 

----------------------------------------------------------------------------

   New File  		      19 2012/03/05 20:44:40	D:\MySourceFiles\File1.txt
   New File  		      21 2012/03/05 20:44:43	D:\MySourceFiles\File2_日本語.txt
   New File  		       7 2012/03/05 20:44:34	D:\MySourceFiles\Folder1\File3.txt

----------------------------------------------------------------------------

              Total    Copied   Skipped  Mismatch    FAILED    Extras
   Dirs :         2         1         1         0         0         0
  Files :         3         3         0         0         0         0
  Bytes :        47        47         0         0         0         0
  Times :   0:00:00   0:00:00                       0:00:00   0:00:00

  Ended : Mon Mar 05 14:39:25 2012

Examples

The following command would copy all files/folders from the file path “D:\MySourceFiles” to the file path “E:\MyDestination” and create a copy log at “E:\CopyLogs\MyCopyLog.log”

The following command would copy only the files with “.txt”, “.jpg” and “.tif” extensions:

The following command would copy only the files with “.txt”, “.jpg” and “.tif” extensions that have last modification dates within the 02/01/2010 – 04/30/2010 date range (not inclusive):

Windows Vista, Windows 7 and NTFS Junction Points

NTFS Junction points are a feature of the New Technology File System (NTFS) and allow symbolic links to a directory to be created. These symbolic links act as an alias of that directory.

Starting with Windows Vista, Microsoft changed the way certain critical folders were stored on the hard drive. For backward compatibility, the old folder names were also retained as junction points. For example, the “C:\Documents and Settings” location does not actually exist in a Windows Vista system, but points to the actual “C:\Users” folder.

In certain instances, a junction point can redirect to a parent folder, causing Robocopy to fall into an infinite loop. To prevent this from happening, you can use the “/XJ” switch to prevent Robocopy from parsing NTFS junction points.

The following command would copy all files/folders from the C: drive of a Windows 7 system to the file path “E:\MyDestination” and create a copy log at “E:\CopyLogs\MyCopyLog.log”

Robocopy is a trademark of Microsoft. Windows is a registered trademark of Microsoft. Other products or services may be trademarks or registered trademarks of their respective companies.

Concordance Field Naming Requirements

Field names have to:

• Be no more than 12 characters long
• Start with a letter
• Continue with letters, numbers or the underscore (“_”) character in the middle

As of this writing, a Concordance database can contain no more than 250 fields.

Concordance Field Types

Concordance is a registered trademark of LexisNexis. Other products or services may be trademarks or registered trademarks of their respective companies.

The above scenario is usually not an issue as long as the load file contains only a few fields. However, manually creating an e-Discovery database to accommodate a 100+ field Concordance Load File can be a tedious task. To make things a bit easier, we have created a Concordance program called “Create_DCB” using the Concordance Programming Language (CPL). The CPL reads the header row of a Concordance load file (DAT), extracts the field names and creates a Concordance Database (DCB) with matching fields. It requires that your load file starts with a header row which contains the field names.

Instructions:

• Download the program using one of the hyperlinks below and save it to your computer.
• Launch Concordance and start the program using the “File/Begin program…” menu item. You do not need to have a database open.
• Choose the Concordance Load File that you would like to work with. Make sure it has a header row. The load file can be ANSI, UTF-8 or UTF-16 encoded depending on the Concordance version.
• If your load file is not using the standard Concordance delimiters, you will be prompted to specify the ASCII codes of the delimiters.
• The program will check each field name to make sure it complies with Concordance field name requirements and will prompt you if it runs into anything problematic.
• Specify the destination path where the new database should be saved.
• Finally, you will be taken to your new database in edit mode where you can make changes to the database structure. For example, you can change the types of certain fields or specify an Image field.

This program is available for download free of charge. Feel free to give it a try and let us know your thoughts.

For Concordance 8:

Create_DCB_v1.26.cpl for Concordance v8

For Concordance 9:

Create_DCB_v1.27.cpl for Concordance v9

For Concordance 10:

Create_DCB_v1.26.cpl for Concordance v10

Free Software Updates

Would you like to receive e-mail updates when a new version of this CPL becomes available? Leave us your e-mail address below and we will keep you updated.

Concordance is a registered trademark of LexisNexis. Other products or services may be trademarks or registered trademarks of their respective companies.