We have created a Concordance CPL called “Populate_Prod_Att” to help make things a bit easier. The CPL reads the existing review attachment ranges and production Bates numbers in your Concordance database, and calculates the production attachment ranges for you.

Concordance CPL Features

• Calculates production attachment ranges based on review attachment ranges and production Bates numbers
• Can automatically shrink attachment ranges when some of the documents from an attachment family are not produced
• Can clear production attachment range fields for single documents (i.e. documents that are not part of an attachment family)
• Can be used for both TIFF productions and native-only productions
• Can work on the entire database or active query depending on your preference

Input

As the input, the CPL takes the following 4 fields: BEGATTACH, ENDATTACH, PRODBEG, PRODEND (field names can be different depending on your database structure).

BEGATTACH: Should be populated with the beginning review attachment range Bates number.

ENDATTACH: Should be populated with the ending review attachment range Bates number.

PRODBEG: Should be populated with the beginning production Bates number.

PRODEND: Should be populated with the ending production Bates number.

Please see the following table for an example of how the input fields should look in your database. In this example, the legal team decided to not produce the document starting with REV00000013. Consequently, the production attachment range will need to be reduced from 6 pages (REV00000011 – REV00000016) to 5 pages (PROD0000001 – PROD0000005).

Figure 1 – Sample Input Fields

Output

The program outputs the calculated values to two fields: PRODBEGATTACH and PRODENDATTACH. Since these fields are used to store the output, they will be overwritten. The output for the sample documents in Figure 1 would be as follows:

Usage for Native-only Productions

If you are working with a native-only database (i.e. you wish to populate the production parent ID field instead of production attachment ranges), you can do so by pointing at the same field (the review PARENTID field) for both the BEGATTACH and ENDATTACH fields. When you do, the program will not prompt you for the PRODEND and PRODENDATTACH fields, and will not attempt to populate the ending production attachment range field. It will only populate the PRODBEGATTACH field, which will be your production parent ID.

This program is available for download free of charge. Feel free to give it a try and let us know your thoughts.

For Concordance 8:

Populate_Prod_Att CPL for Concordance v8 (Version 1.04)

For Concordance 9:

Populate_Prod_Att CPL for Concordance v9 (Version 1.04)

For Concordance 10:

Populate_Prod_Att CPL for Concordance v10 (Version 1.04)

If you like this CPL, you may also like our CPL to Create Database (DCB) from Load File.

Free Software Updates

Would you like to receive e-mail updates when a new version of this CPL becomes available? Leave us your e-mail address below and we will keep you updated.

Concordance is a registered trademark of LexisNexis. Other products or services may be trademarks or registered trademarks of their respective companies.

A frequently (and improperly) used tool for e-Discovery searches is the built-in search functionality in Ms Outlook. Specifically, instead of engaging an expert to identify and collect electronic evidence, some businesses choose to have their IT staff get in front of each custodian’s computer, run a search in Outlook and copy the responsive e-mails to a new PST.

Outlook Instant search relies on the Windows Search Service, and helps users perform quick searches in their mailboxes. This is a very convenient way to locate an e-mail you received from someone last week, or find that e-mail attachment with a specific name; but it is not, by any means, an e-Discovery search tool. Here are a few reasons why you should not use it to locate responsive e-mails for e-Discovery:

1. Incomplete e-Discovery Search Strategy

Relying on only the e-mails found on each custodian’s computer would be an incomplete strategy to begin with. In an Outlook/Exchange environment, additional key e-mails can be found on the server (e.g. e-mails that were not downloaded from the server, e-mails that were hard deleted within the deleted item retention period etc.) as well as on mobile devices, e-mail server back-ups, mobile device back-ups etc.

2. Lack of OCR

While most Ms Office documents are searchable, some file types such as non-searchable PDFs (e.g. an agreement scanned to PDF) or images (e.g. a fax message in TIFF format) do not contain extractable text. Since Windows Search does not perform optical character recognition (OCR) to make these documents searchable, many potential search hits could be missed. Computer forensics and e-Discovery tools have the ability to identify and OCR documents without extractable text before keyword searches are performed.

3. Limited File Type Support

Windows Search supports indexing the contents of most common file types such as e-mails, Ms Office files, PDFs etc. However, it does not support nearly as many file types as an e-Discovery search engine. For example, we have found that contents of many business documents such as WordPerfect files, E-transcript files, RAR and 7-Zip archives, CAD files etc. were not indexed out of the box without installing third party IFilters (interfaces that allow Windows Search Service to extract text and properties from documents). Furthermore, supported file types were not indexed when given an unsupported extension (e.g. a plain text file with the .xyz extension).

4. Lack of Accountability and Reporting

When the end user attempts to search a mailbox that has not yet been fully indexed, Outlook displays a warning that indicates that the results may be incomplete. However, no warning is shown for documents that the Windows Search Service could not index or search. In other words, the user interface gives you the impression that everything is working well, even if the data set contains file types that Windows Search cannot handle. On the other hand, performing searches for e-Discovery requires documenting the search process, logging and reporting search exceptions and handling them when possible.

5. Limited Search Syntax

Admittedly, the query syntax for Windows Search is quite detailed, and should be sufficient for its intended usage – casual, everyday search and filtering. However, it is missing some of the e-Discovery search staples such as proximity searches, granular control over wildcards, stemming, fuzzy searching, synonym searching etc. Consequently, most complex e-Discovery search queries cannot be directly translated to a query in Outlook.

Example Scenario

For the purposes of this article, I searched a small PST containing 7,331 documents using Outlook and an e-Discovery tool. Both searches consisted of a simple, three-keyword query and attachment families were included as part of the results.

The search performed directly in Outlook resulted in 4,575 responsive documents, while the search performed using the e-Discovery tool identified 5,585 documents as responsive. In other words, Outlook found 1,010 fewer responsive documents (approximately 18%) than the e-Discovery tool (see Figure 1).

Figure 1 – Example Search Results

Conclusions

I believe that the dramatic difference in search performance shown in the example scenario was mainly due to the graphics and PDF files that needed to be OCRed, and file types, such as WordPerfect files, which Outlook could not search. That being said, the possibility of missing almost 20% of the responsive documents (or even more, depending on the data set) when searching directly in Outlook is a scary thought.

Identification, preservation and collection stages of e-Discovery can be crucial for the outcome of your case, and should be handled expertly. If you are unable to work with an expert, you should at least obtain the right tools and training. Make sure that you are familiar with how the search engine works (e.g. search syntax, tokenization, foreign languages etc.), what it can and cannot search and how exceptions should be logged and handled. Test your tools against a known data set to make sure they are performing as expected, and review exception reports carefully to make sure you are searching everything that you should be.

1. Understand your discovery agreement

Surprisingly, many law firms do not honor the agreed upon production format, or change the production specifications unilaterally as they see fit. For example, if you agreed to produce processing exceptions as placeholders with the accompanied native file, do not remove them from the production. Or, if you agreed to produce in Concordance v10 format, do not send out Concordance v8 load files.

2. Normalize the images before endorsement

Normalize produced images before endorsing them so that the output has consistent dimensions, resolution and compression. Furthermore, normalized images allow the endorsements to be applied consistently in terms of size and location. You should not send out a mixture of LZW compressed TIFFs, JPGs; 200, 300 and 600 DPI images; portrait and landscape pages etc.

If the data set contains oversize images (e.g. technical drawings) whose legibility could be affected during normalization, those images should be handled with care. Depending on the scenario, they could be normalized to different specifications or left as is.

3. Use legible endorsements

You should ensure that the endorsement process pads the images so that endorsements do not overlap with the contents of the documents and cover the actual image, or disappear on dark backgrounds. Furthermore, endorsements should be performed using an easy to read font face and size (Arial 12 Bold is usually a good choice). Some smart endorsement tools can alert you if some of your designations are too long and will run off the page or overlap with other endorsements.

4. Do not leave Bates gaps

Take the time to ensure that the documents are Bates numbered and endorsed sequentially, without Bates gaps or overlapping Bates ranges. If gaps are unavoidable (e.g. you had to make last minute changes to a very large production and the deadline does not allow renumbering the entire data set), provide both image and text placeholders for each excluded page as opposed to providing a single placeholder for a Bates range.

5. Organize and label your files clearly

Each component of the production should be separated into clearly labeled folders such as “DATA”, “IMAGES”, “TEXT” etc. Furthermore, load files should be named so that they contain the volume ID as well as a description of what type of file each one is. For example, “ABC001 Concordance Loadfile.dat”.

6. Review and load test the production

Even though the documents may have been meticulously reviewed in the review database, produced documents should be loaded into a new production database and examined. This would serve as an additional quality control step to ensure that the load files work as intended, redactions and designations were applied correctly etc.

7. Use sanitized delivery media

If the production will be sent out on a hard drive, make sure that it is securely wiped before the production deliverable is copied onto it. Simply deleting the old data and copying a new deliverable usually means you will be sending your opponent unintended files which can often be recovered effortlessly.

For example, let’s assume that you had sent your e-Discovery service provider a hard drive with 20 PSTs for processing. They processed the PSTs, hosted the processed data, prepared the production deliverable after your review and delivered it to you on the same drive along with the source PSTs. If you simply delete the PSTs and send out the drive, you would essentially be sending your opponent the original native PSTs along with your production.

8. Encrypt the data

Whether you are sending the production deliverable via electronic file transfer, or on a physical medium, using strong encryption can go a long way towards making sure your sensitive data doesn’t fall into the wrong hands. Imagine that the courier company lost the package with the hard drive containing your production. Wouldn’t you be relieved if you knew that the drive was AES-256 encrypted and was virtually impossible to decrypt using today’s technology?

I am generally not a fan of PDF productions because I think they lack both the advantages of a native production (e.g. maintaining the metadata and functionality of complex electronic files) and the advantages of a TIFF production accompanied by load files (e.g. flexibility and ease of use with legal review platforms). In fact, our experience shows that upon receiving a searchable PDF production, most law firms hire an outside company, or engage their in-house litigation support team to have the documents converted to a TIFF production with load files so that they can be loaded into a legal review platform.

The workflow above effectively discards the more accurate, electronically extracted text and replaces it with OCR text with much less accuracy. For example, let’s assume that the original electronic document was the following Excel spreadsheet:

Figure 1 – Original Native File

The text electronically extracted from this file is as follows. As expected, formatting is not maintained but the text is 100% accurate.

Figure 2 – Text Electronically Extracted from Original Native File

Once the native file is converted to TIFF, OCRed and exported as a searchable PDF, the text embedded in the PDF is degraded. Take a look at the following example prepared using one of the most accurate OCR engines:

Figure 3 – Text Extracted from Searchable PDF Created Using OCR

As you can see in the screenshot above, even though the plain text in cell A1 and the formula in cell B1 were correctly recognized, the lack of contrast in cell A2 and the complex font in cell B2 prevented the OCR engine from correctly recognizing those characters. Needless to say, if you are on the receiving end of such a production and utilizing keyword searches, the lack of accuracy in the provided text can be quite frustrating. Please note that the issue described above is not caused by a deficiency of the PDF format, but by the unnecessary OCR process in the workflow.

Conclusion

During e-Discovery processing, text is typically extracted along with the coordinates of each character or word. This makes it possible to export searchable PDFs with embedded extracted text if desired. Instead of OCRing images, searchable PDFs should be created through the e-Discovery platform using the original, more accurate extracted text. OCR should be used on non-searchable documents such as image-only PDFs, scanned images etc. to complement the extracted text.

If some of the documents contain redactions, the redactions can be made in a platform that supports automatically mirroring the redactions in the extracted text. If this is not possible, OCR can be performed only on the redacted images.

While drafting discovery agreements, legal teams should consider the distinction between extracted text and OCR and request extracted text when available.

This property indicates the relative position of a message within a conversation thread and is typically populated by the e-mail client for each outgoing message. Information extracted from the PR_CONVERSATION_INDEX property can help answer key questions such as:

• Is the message in question a new message, or was it created by replying to or forwarding another message?
• If the message is part of an e-mail thread, when was the thread started?
• When were other messages in the e-mail thread created?

MSDN Documentation

Microsoft has an article titled Tracking Conversations on how the PR_CONVERSATION_INDEX value is calculated. According to the article (quoted directly from their website):

However, our experience differs from Microsoft’s documentation in three areas:

1. The first byte is not merely a fixed-value reserved byte, but is also a part of the FILETIME structure that follows it.
2. The 6-byte FILETIME value (including the reserved byte) is actually the 6 most significant bytes of the 8-byte FILTIME structure, and needs to be padded with 2 bytes of zeroes.
3. The time difference values found in the child blocks do not indicate the difference in time from the header date, but from the date of the previous child block.

Further research on the discrepancies above revealed that Joachim Metz of Hoffmann Investigations noted the same issues as #1 and #3 above in his article titled “E-mail and appointment falsification analysis”.

Example Calculation

Let’s take a look at an example and try to make sense of the PR_CONVERSATION_INDEX value.

PR_CONVERSATION_INDEX
01CDE90ABFE0D78F0E4280824120B2F1D0E3C07ED0070000CCBA300000114460 (32-bytes)

Based on the Microsoft documentation, the PR_CONVERSATION_INDEX is divided as follows:

Figure 1 – Breakdown of Example PR_CONVERSATION_INDEX

After the zero padding, the header timestamp value becomes 01CDE90ABFE00000. This value is expressed in FILETIME units, and therefore represents the number of 100-nanosecond units since the start of January 1, 1601. When 0x01CDE90ABFE00000 is converted to decimal, we find that the timestamp represents 130,016,196,641,685,504 100-nanosecond units since the start of January 1, 1601, which corresponds to January 2, 2013 17:01:04 (UTC). This matches the sent date (PR_CLIENT_SUBMIT_TIME) of the original e-mail in the e-mail thread.

The following 16-bytes represent the following GUID: d78f0e42-8082-4120-b2f1-d0e3c07ed007.

The following 5 bytes, 0000CCBA30, contain the data for the first child block. When we convert 0x0000CCBA30 to binary, we find the following 40-bit value:

0000000000000000110011001011101000110000.

According to Microsoft’s documentation, the bits are divided as follows:

Figure 2 – Breakdown of Example Child Block

Since the first bit is 0, we assume that the high 15 bits and the low 18 bits were discarded when the time difference value was being calculated. Once the discarded digits are added, the FILETIME value becomes the following 64-bit binary string:
0000000000000000000000000000001100110010111010000000000000000000

When converted to decimal, this value represents 13,738,967,040 100-nanosecond units, which is approximately a 22 minute and 53.897 second time span. When added to the header date, we find that the date of Child Block 1 was January 2, 2013 17:23:58 (UTC). After converting the random number and sequence count bits to decimal, we find that the values are 3 and 0 respectively.

Same calculations on Child Block 2 indicate that the time difference for this message is 1 minute and 55.868 seconds and the random number and sequence count values are 6 and 0 respectively.

The final results are as follows:

Download Conversation Index Parser

We realize that performing these calculations on multiple messages can be quite tedious. We have created a free utility called Conversation Index Parser which extracts the information from the PR_CONVERSATION_INDEX value. If you would like to download a copy, please leave us your e-mail address below and we will get back to you with the download link.

Figure 3 – Conversation Index Parser v02 Screenshot

[contact-form-7]

Observations

We have made the following observations while analyzing the PR_CONVERSATION_INDEX properties of numerous e-mails:

• When creating a new message, Outlook populates the header date in PR_CONVERSATION_INDEX with the local timestamp when the message is sent (i.e. PR_CLIENT_SUBMIT_TIME).
• When replying to or forwarding a message, Outlook updates the PR_CONVERSATION_INDEX property and sets the time difference in the child block based on when the new message is created, not when it is sent. For example, let’s assume that person A receives an e-mail from person B and hits the reply button in Outlook at 3:00:00 PM. She then takes 10 minutes to compose her answer and send the e-mail. The time difference value contained in PR_CONVERSATION_INDEX would reflect the difference in time between 3:00:00 PM (when person A created the new message) and the timestamp of the previous message.
• Outlook calculates the time difference values based on the user’s local time. When multiple users with slightly different computer times participate in an e-mail conversation, the calculated time difference values in PR_CONVERSATION_INDEX would reflect this discrepancy. For example, let’s assume that person C creates and sends a message to person D at precisely 4:00:00 PM. Person D’s computer time is 5 minutes ahead, and shows 4:05:00 PM at this moment. Person D creates a response message at 4:20:00 PM according to his computer (4:15:00 PM according to Person C’s computer). This would cause Person D’s Outlook to calculate a 20 minute time difference value when, in reality, only 15 minutes passed between the two events.
• The time difference values found in the child blocks are unsigned. While one would normally assume that the time differences in an e-mail conversation would be positive (i.e. a response would be created after the original message), it is possible that the original time difference might have been negative due to a large difference in local computer times or due to tampering. For example, let’s assume that person E sends person F a message at 5:00:00 PM, and person F’s computer time was set to 4:50:00 PM at this moment (10 minutes behind that of person E). If person F replies to this message at 4:54:00 PM according to his computer (5:04:00 PM according to person E’s computer), person F’s Outlook would record a time difference value of 6 minutes while the difference (according to the local computer times) was -6 minutes.

Conclusions

Combined with additional evidence from the e-mail server or internal e-mail metadata, the information contained in the PR_CONVERSATION_INDEX MAPI property can be very helpful in the forensic analysis of e-mails. We believe that forensic examiners would benefit from understanding how and when this property is populated and which factors (such as the accuracy of the local computer time) affect the reliability of this information.

Figure 1 – Files Sorted Alphabetically

In scenarios where the order of the files is crucial (e.g. in the legal industry), end users typically pad the file names with zeros so that they are ordered correctly when sorted alphabetically. For example:

Figure 2 – Files with Zero Padding

The Shell team at Microsoft at some point decided to improve things a bit and implemented a new way of comparing Unicode strings that contain numerals (see StrCmpLogicalW). The change took effect after Windows 2000, so operating systems such as Windows Server 2003, Windows XP, Windows Vista and Windows 7 sort numerals in folder and file names according to their numeric value. For example, our example folder would look as follows in Windows XP:

Figure 3 – Windows XP Natural Numeric Sort

Associated Issues

While this seems logical and may be helpful to most people, we believe that it brings new issues, especially in the legal industry.

1. Compatibility with e-Discovery and Computer Forensics Software:

Imagine a lawyer organizing exhibits to be processed to TIFF, endorsed and produced. Looking at the files in Windows Explorer, he would naturally assume that the files would be processed in the order as he sees them on his computer. However, computer forensics and e-discovery tools do not implement Microsoft’s sort algorithm, and treat the file and folder names as strings while sorting. Consequently, files would be processed and numbered in a different order than what the attorney had anticipated. Had Windows sorted the files without any special handling, the attorney or litigation support team would have noticed the incorrect sort order and compensated for it by correctly padding the file names or applying a custom sort order.

2. Consistency within the Operating System:

Even though Windows Explorer takes advantage of the StrCmpLogicalW API and sorts files and folders with names containing numerals in a logical manner, other areas of the operating system (such as the command line interface) still use the traditional sort method, causing inconsistencies in the way files are displayed in different parts of the same operating system. Please see Figure 5 below for a comparison of how Windows Explorer and the Command Line Interface (CLI) display the same set of files.

Figure 4 – Files As Displayed by Windows vs. CLI on The Same Computer

3. Consistency among Operating Systems:

Microsoft’s proprietary sort algorithm does not match how files are displayed in other operating systems such as Linux and Mac OS. Furthermore, Microsoft has changed the StrCmpLogicalW API in different versions of its operating systems such as Windows XP, Windows Vista and Windows 7. Consequently, the way files are displayed in Windows Explorer varies slightly among Microsoft’s own operating systems.

Potential Solution

Luckily, starting with Windows XP SP-1, Microsoft has made available a registry key that can suppress the use of StrCmpLogicalW API and revert Windows Explorer to treating file names as strings. The registry key is as follows:

The value of the NoStrCmpLogical (DWORD) key should be set to 1 to prevent Windows XP and later versions from sorting numerals in folder and file names according to their numeric value. The Microsoft Support Website provides additional details about this issue. Please note that the above change requires a restart or log off to take effect. Remember to back-up your registry before making any changes.

What are Exceptions?

e-Discovery exceptions are documents that cannot be correctly processed by the e-Discovery platform. For example, the e-Discovery software may be unable to open, extract text or metadata from, or image a document. Exceptions can be encountered during various stages of the e-Discovery process such as collection, processing, review and production. For the purposes of this article, we will focus on exceptions in the processing stage of e-Discovery.

Types of Exceptions

A. Corrupt Files

Corrupt files are files that have structural problems which prevent them from being opened or manipulated in even their native application. File corruption can be caused by numerous factors such as network transmission errors, errors in the medium where files were stored (e.g. bad sectors on a hard drive) or unexpected termination of the software that was being used to edit the file (e.g. a power failure).

When handling corrupt file exceptions, the first course of action usually is to investigate the possibility of obtaining a replacement. If a replacement copy is not available, depending on the nature of the case and how critical the corrupt file is, attempting to repair the file may be a viable option (e.g. recovering a corrupt mailbox). Alternatively, the corrupt file can be excluded from processing and delivered in native format. In any case, the exception should be logged and all steps taken should be thoroughly documented.

B. Unprocessable Files

Unprocessable files are files that do not support the common e-Discovery actions such as text and metadata extraction or conversion to image format. For example, system files such as executables and dynamic link libraries are typically unprocessable file types.

C. Unsupported Files (Processable Files That Are Not Supported by The e-Discovery Platform)

No e-Discovery software supports all processable file types. An e-Discovery project may contain unsupported files that could be searched or converted to an image format using external software. Some examples would be advanced CAD/CAM files such as Unigraphics, less popular archive container formats such as ARJ and ACE, some database formats such as SQLite, project management files such as Primavera files and BlackBerry back-up files etc.

Depending on the type and amount of unsupported files, these files may be manually processed or the e-Discovery service provider can develop a custom solution to handle the files in an automated manner. Some complex file types such as technical drawings and databases can be processed and produced in native format if the discovery agreement allows.

D. Audio & Video Files

Audio and video files cannot be directly processed by most e-Discovery software, but depending on the case, they can be a very good source of electronic evidence. For example, voicemail messages sent as e-mail attachments by the VoIP phone system in a corporation may contain information not found anywhere else in the organization.

In some cases, the legal team may choose to perform audio discovery on the audio and video files to make them searchable. Alternatively, the files can be delivered in native format so that the reviewers can listen to the audio recordings or watch the videos during review.

E. Encrypted Files

Encrypted files are files that were protected by a password, via digital rights management (DRM) or other encryption schemes. Encrypted files can be single documents such as Ms Office files or PDFs, or encrypted containers such as TrueCrypt volumes.

The legal team should be informed of any encrypted files before further action is taken. Depending on the case, attorneys may choose to exclude the encrypted files from processing due to privacy concerns. On the other hand, in some cases, encrypted files may be of particular interest.

Attorneys may occasionally be able to obtain the passwords for the encrypted files in the data set. If passwords are not available, they can often be discovered by strategically reviewing neighbor documents or by attempting to crack the passwords.

How Should Exceptions be Tracked, Handled and Reported?

A. e-Discovery Software

A well-designed e-Discovery software should provide the following mechanisms for exception tracking, handling and reporting:

• All encountered exceptions should be logged and displayed to the technician conspicuously. The log files should contain detailed information about the exceptions such as the full file path, file name, hash value and a description of the exception.
• The e-Discovery technician should be able to manually process documents that cannot be automatically processed. Large amounts of unsupported file types should be able to be batch processed using the native application (e.g. Shell Print).
• The e-Discovery software should be able to complete the processing of exceptions after the fact. For example, if the password of an encrypted container is discovered after the processing job was completed, or a replacement for a corrupt mailbox was obtained, the e-Discovery software should be able to add the new extracted documents to the data set where they belong, without having to re-process the documents.
• The e-Discovery software should have built-in mechanisms that facilitate the tracking and review of the exceptions during quality control. The technician should be able to add his comments for each file. This information can then be exported as part of the exceptions report.
• The e-Discovery software should have built-in password cracking functionality and should allow the technician to input a list of known passwords to be used for opening and processing encrypted files.

B. The e-Discovery Deliverable

In our experience, most legal teams prefer to leave processing exceptions in the review database as placeholders with native hyperlinks. The placeholder usually contains a few fields of metadata for the exception such as the file name, file path and hash value. This helps them see the exceptions in context, and be able to look at them in more detail if they wish to do so. If the exceptions are left in the database as placeholders, those records should be identified (e.g. with a tag in the database or by populating a field) so that they can easily be excluded from a subsequent production if required.

On the other hand, if the legal team would like to have the exceptions excluded from the review database, the exceptions should be exported separately and delivered in native format (e.g. in a folder called “\EXCEPTIONS”) along with the exceptions report as part of the deliverable. This would ensure that someone looking at the deliverable in the future can easily locate and review the exceptions associated with that deliverable.

C. The Exceptions Report

The exceptions report is an important part of the documentation of the ESI lifecycle. At a minimum, an exceptions report should contain the following elements:

• Relevant information about the case and project so that, if the report is reviewed separately from the deliverable, the viewer can still identify which project/batch the report pertains to.
• Identifying information (e.g. BegDoc #, internal Doc ID) about each exception so that the entries on the exception report can be linked to the placeholders in the database or native files in the “\EXCEPTIONS” folder.
• Technician’s comments and description of why the file is an exception.
• Crucial file metadata such as file name, file path, file size, file extension, hash value etc.

Conclusion

Processing exceptions are unavoidable in e-Discovery. We believe that being able to accurately identify exceptions, thoroughly documenting every step of action, and having a well-thought-out exception handling plan are essential components of an effective and defensible e-Discovery process. Legal teams should not only discuss exception handling policies with their in-house litigation support team and outside service providers, but also with the opposing counsel during “Meet and Confer”.

Image normalization (in the e-Discovery sense) is the process of transforming images to make them consistent in terms of dimensions, resolution, color depth and orientation. For example, larger images can be resized to 8.5″x11″, landscape pages can be rotated to portrait, images with different resolutions can be converted to 300 DPI etc.

In our experience, the most frequently used image specifications in e-Discovery processing today are 8.5″x11″ portrait, monochrome images with 300 DPI resolution and CCITT Group 4 compression. Landscape images are typically rotated 90° counter-clockwise to portrait. When color for color processing is performed, color images are usually exported as 300 DPI JPGs. See Figure 1 below for an example landscape page that was normalized and endorsed.

Figure 1 – Normalized and Endorsed Landscape Image

Advantages and Disadvantages of Image Normalization

Whether or not image normalization is required should be decided on a case by case basis. Some of the advantages and disadvantages of normalization are as follows:

Advantages:

• If images are normalized before endorsement, the size and location of the endorsements would be consistent among different pages in the data set. See Figure 2 for an example document processed using different page sizes and endorsed without normalization.



Figure 2 – Endorsement Size Difference without Normalization

• Endorsing normalized images ensures that each page has the same amount of space for the Bates endorsement and confidentiality legend. Having images of various sizes can result in unexpected overlaps in endorsements on the smaller pages. See Figure 3 below for an example.



Figure 3 – Overlapping Endorsements

• If images are printed, using normalized images would prevent printing problems due to changes in page size and orientation. For example, some printing applications do not automatically rotate landscape pages and print them on a portrait page scaled down. See Figure 4 below for an example landscape page printed on a portrait canvas.



Figure 4 – Landscape Page Printed on Portrait Canvas

Disadvantages:

• Image normalization without proper quality control can result in legibility problems. For example, a 36″x24″, 600 DPI technical drawing would most likely become illegible if normalized to 8.5″x11″ 300 DPI.
• In some review platforms, landscape pages rotated to portrait may require the reviewers to either rotate the page or turn their heads sideways. This can be prevented by exporting a rotation flag along with the image reference file so that the review platform auto-rotates the pages to correct page orientation for review (if supported by the review platform).
• Using poorly designed normalization software can result in degradation of overall image quality.
• Image normalization can be a time consuming process and can add a significant amount of time to the e-Discovery export process in large cases.

How Normalization Should Be Performed

Image normalization should be performed with minimal impact to the original image quality. A few things to watch out for:

• Images smaller than what the normalization specifications require should typically be placed on a white canvas instead of being enlarged. Enlarging small images usually causes pixelation. See Figure 5 below for an example small image resized two different ways.



Figure 5 – Resized Small Image

• Aspect ratio of the images should be preserved during normalization. For example, a 14″x8.5″ page can be resized to an 11″x6.68″ image, placed on an 11″x8.5″ canvas and then rotated instead of being resized directly. Resizing without preserving the aspect ratio can result in stretched, distorted images.
• If color images are being converted to B&W, proper dithering should be employed (e.g. the Floyd–Steinberg algorithm) to ensure that the B&W image is an acceptable representation of the original color image. See Figure 6 below for a color image converted to B&W with and without proper dithering.



Figure 6 – B&W Conversion and Dithering

Conclusion

We believe that complete output image specifications should be included with every e-Discovery processing project. Legal teams should be aware of the advantages and disadvantages of image normalization and should make informed decisions on a case by case basis. In some cases, requesting the review database without normalization and normalizing images later before endorsement & production or blowbacks may be a good approach.

When it comes to handling deleted e-mail messages, e-Discovery processing software typically fall into two camps:

• Software Solutions That Only Process Active E-mails:

  Some e-Discovery processing software solutions only extract and process active e-mails. For example, products that use Messaging Application Programming Interface (MAPI) to access Ms Outlook e-mails would typically only retrieve and process active e-mails and disregard permanently deleted (but not yet purged) messages. Please note that these software tools still process the contents of the “Deleted Items” folder since e-mails in this folder are not yet permanently deleted.

• Software Solutions That Process Both Active and Deleted E-mails:

  Some e-Discovery processing products process both active and permanently deleted (but not yet purged) e-mail messages by default. These tools usually employ their own processes to parse mailboxes.

As you can imagine, it is crucial to have a good understanding of how the e-Discovery software or service provider that you use handles deleted e-mails. Depending on the case, it may or may not be desirable to process deleted items. Consider the following example scenarios:

• Opposing counsel produces a PST containing responsive e-mails. As it turns out, they conducted a manual review of the mailbox within Outlook (bad idea), deleted e-mails that were privileged or non-responsive and produced the PST file without compacting it. In this scenario, whether or not deleted e-mails are included in e-Discovery processing would determine if the inadvertently produced privileged and non-responsive e-mails would make their way into your review database.
• Similar to the first example above, an attorney from your firm opened and reviewed a PST file within Outlook without consulting with the litigation support department (again, bad idea). He deleted the e-mails that were privileged and gave you the PST for processing and production. Unless the processed data set is reviewed one more time by the attorney before production, processing deleted e-mails in this scenario can result in inadvertent production of privileged documents.
• Mailboxes of multiple custodians were forensically collected from each custodian’s local computer. Some of the mailboxes contain deleted e-mails that are relevant to the case. In this scenario, you could miss critical information unless deleted e-mails were included during e-Discovery processing.

Conclusion

• It is critical to know how your e-Discovery software or service provider handles deleted e-mails. Lack of this information can result in missing critical documents or inadvertent production of privileged information.
• If possible, it is recommended to use software that provides the flexibility to specify how you would like to handle deleted e-mails on a case by case basis. Otherwise, you may find yourself looking for a different solution depending on case requirements.
• In-house litigation support teams as well as outside service providers should pay attention to mailboxes that produce much less content than their size implies. A big discrepancy usually points to a large amount of deleted e-mails that were not purged.
• When working with e-Discovery service providers, make sure to indicate your preference regarding how deleted e-mails should be handled.

The answers to these questions depend on the requirements of each case and should ultimately be determined by the legal team. That said, we have prepared the following field list as an example, with the hope that it will serve as a good starting point. Please note the following:

• The list below is for electronic documents only, and does not include fields for scanned paper documents. If you keep scanned paper documents and electronic documents together, you should add to the list paper specific fields such as BOXNO, BOXNAME, FOLDERNAME etc.
• Depending on the legal review platform, e-Discovery databases can also contain several additional administrative and/or user populated fields
• We recommend that the time fields be populated using 24-hour format with second precision (e.g. 23:14:05)

Feel free to contact us with any questions and share your thoughts and suggestions in the comments.

Download:

Download Concordance style header row below to create a Concordance database using our Create_DCB CPL:

Meridian Discovery Fields Header Row v1 (Version 1)

Table 1 – Example e-Discovery Database Field List v1