
Verification in Legal AI Is a Design Problem
Lawyers remain responsible. Legal AI should make its work easy to check before they sign.

Sometimes the material does not support an answer. A legal AI system should preserve that condition instead of smoothing it into a conclusion. This is a difficult product requirement because fluent models are built to respond. The user asks what happened, who was responsible or whether an allegation is supported. A complete sentence can appear even where the evidentiary record is incomplete. For litigation, the absence of a reliable answer can itself be useful. It tells the lawyer what remains to be investigated, requested or tested.
A difficult question may have an answer in the supplied material that is hard to find or reason through. An insufficient question cannot be resolved from the material provided. A relevant witness statement may be missing. The documents may establish two possible dates without resolving which is correct. The only support may be an allegation by an interested party. A PDF may have failed during processing. The system needs to distinguish those situations.
Research published in 2026 gives a useful example. Learning When Not to Decide tested AI systems in unemployment insurance adjudication using cases where the information required for a determination was deliberately varied. Standard RAG approaches averaged 15% accuracy when information was insufficient. A structured method that required the system to identify missing information before making a determination reached 89% overall accuracy. That study concerns administrative adjudication, not litigation fact extraction. The transferable point is narrower: forcing an explicit evidence-completeness check before the conclusion materially changed the system's behaviour.
Mary proposes six factual states for incomplete or contested legal records:
Those states call for different responses. An allegation may still be highly relevant. A conflict may become the centre of cross-examination. A missing attachment may justify a request for production. An unread scan may require OCR or manual review. Combining them under a generic label such as "low confidence" loses the reason for the uncertainty.
A cited answer can be accurate as far as it goes. The cited email may support that a meeting occurred. Another document outside the model's selected context may place the meeting on a different day. A missing attachment may contain the terms discussed. The source link helps the lawyer check the stated proposition, but it does not show the coverage of the inquiry. The product should expose both:
That includes documents that failed, date ranges with no material, expected sources that were absent and relevant accounts that conflict with the answer. The real-source article separates source existence from faithful use and matter coverage.
An effective workflow can require the system to complete an evidence checklist before producing the answer. For a question about whether a notice was given, the checklist might ask:
The model may help identify and populate those fields. The system should retain the fields and their sources rather than hiding them inside internal reasoning. If a required field is absent, the output can say what is missing and why the conclusion is being withheld. That is more useful than a generic disclaimer.
A non-answer should move the work forward. "Insufficient information" on its own gives the lawyer little direction. A stronger response identifies the missing premise:
The lawyer can then decide whether to seek another document, question a witness, preserve the conflict or proceed on an express assumption. The system should not decide the legal consequence of the gap unless that task has been assigned and the necessary legal context is available.
A missing document does not prove concealment. A silent period in communications does not establish that nothing happened. An absent record may reflect collection scope, retention practices, privilege, processing failure or a genuine evidentiary gap. The product should report the observable condition and avoid importing an inference that the evidence does not support. Discovery makes the separation especially important. Federal Rule of Civil Procedure 26 requires parties to address discoverable information, proportionality and sources that are not reasonably accessible. The legal consequence of unavailable or withheld material depends on the circumstances and the applicable rules. Software can identify the gap; the lawyer determines what it means and what step follows.
A known-matter evaluation should include questions that cannot be answered from the supplied record. Measure whether the product:
A system can improve its apparent precision by refusing too often. The test should include clear cases as well as incomplete ones. The paper on the "detection-to-abstention gap" makes a related point: a reasoning model may recognise that a problem is underspecified and still continue to a confident answer. The control has to affect the final action, not merely appear in the hidden reasoning.
An incomplete record should remain structured. The fact can be stored with its source, status and open question. Expected missing material can be attached to the relevant issue or period. When the document later arrives, the system can propose an update without erasing the earlier uncertainty or the steps taken because of it. That is one reason Mary treats gaps, contradictions and allegations as part of the record rather than defects to be removed from the final prose.
The known-matter test provides a method for measuring this behaviour. Checking the AI is not enough shows how negative space and page-level review appear in practice.