Situation

Our client was assisting a client in a large international arbitration. The parties agreed to provide category-based discovery as an alternative to general discovery.

Some categories were very narrow, allowing the client to identify small sets of potentially relevant documents using simple searching methods quickly and easily. However, some categories were broader, and simple search methodologies returned large volumes of potentially relevant documents for review.

Despite various efforts to reduce the review volume accurately and defensibly, our client was left with over 40,000 documents to review. Their client was conscious of the cost of this review, particularly given that there would be rolling tranches of discovery required.

Solution

Due to the parameters of this review, we were able to propose the use of continuous active learning (CAL). Several documents in the set were identified as being relevant to the category. These documents were used to train the CAL model and develop a queue of documents the model deemed relevant based on the documents provided and the results of the ongoing review.

Initial review reporting showed that just over 98% of the documents deemed relevant by the CAL model were confirmed as relevant by a human reviewer. The legal team continued to review until the CAL model deemed no relevant documents left to review. There were approximately 26,000 documents left deemed unlikely to be relevant to the category.

An elusion test was run with the team reviewing samples of this set which showed that approximately 0.34% of those documents left might be relevant. A commercial decision was made to conclude the review on the basis that the potential benefits of continuing were outweighed by the cost and time associated with doing so. Privilege review and quality checks were then performed before the exchange occurred.

Impact

Our work in identifying CAL as a viable workflow and overseeing the deployment of that workflow allowed our client to:

• Reduce the number of documents they had to review by over 55% compared to a traditional review workflow. This reduced the time it took to complete their review allowing the legal team to focus on other time-critical tasks

• Rely on the methods used regarding the detailed reporting on the workflow accuracy and reliability

• Accommodate the end client’s concerns in respect of costs whilst also maintaining a high level of integrity and accuracy in the review process

• Use the trained CAL model to review subsequent tranches of documents regarding the same category.