The first time a lawyer handed me a brief with a citation to a case that did not exist, it took eleven minutes of searching to figure out what happened. The case name looked right. The reporter volume looked right. The year was plausible. It was fiction, generated by software that had been asked to sound confident, not to be correct.
That is the whole problem in one sentence. An ai legal assistant is a language tool wearing a lawyer’s vocabulary, and it will produce a citation shaped object whether or not the underlying case is real. The tool is not lying to you. It is pattern matching, and citation formats are an extremely easy pattern to match.
Once you understand why the fabrications happen, you stop treating them as a glitch and start treating them as a process risk you can actually manage. That shift is worth more than any prompt trick I have tried.
What is actually happening when a tool invents a case
Legal writing has a rigid skeleton. Parties, reporter abbreviation, volume, page, court, year. A model trained on millions of documents learns that skeleton fast, which is exactly why it can fill the skeleton with invented names that read like the real thing.
The underlying legal material comes from a body of published opinions, and the machinery for publishing and citing those opinions is old and consistent. Cornell Law School’s Legal Information Institute keeps one of the most widely used free indexes of that material, and the citation conventions it documents are the same ones a model is imitating. Imitation is not retrieval. That is the gap.
I have watched this play out in three patterns, and only the first one is obvious.
- The clean fabrication. The case does not exist anywhere. Easiest to catch, most embarrassing when you miss it.
- The real case, wrong holding. The citation checks out, but the sentence you wrote about it does not describe what the court actually said.
- The drift. Three paragraphs of good analysis, then a confident sentence built on a source nobody can produce.
The drift is the one that gets past review, because everything around it looks fine. I would take a hundred clean fabrications over one drift, since the fabrication fails loudly and the drift fails quietly.
The verify-before-you-file routine
Here is the routine I would run in any shop, regardless of which drafting tool you use. It takes less time than the argument about it.
- Separate drafting from sourcing. Let the tool write structure, tone, and argument shape. Pull every citation from somewhere you can open in a second tab.
- Open every authority. Not skim. Open. A citation you have not clicked is a citation you have not verified.
- Read the cited page for the proposition. Confirm the court said the thing you are citing it for, not something adjacent.
- Flag anything generated in your billing notes. If the work is unverified, the file should say so until it is not.
- Give one person the job of final citation pass. Shared responsibility means nobody owns it.
That last item matters more than the rest. Firms that assign verification to a named reviewer catch problems at a much higher rate than firms that assume everyone checks their own work.
Why “it’s just a drafting tool” is the wrong mental model
I keep hearing the same defense: we only use it for drafting, so accuracy is not the issue. Drafting is the issue. A draft is what goes into the filing if nobody catches the flaw.
There is a real framework for thinking about this, and it comes from outside law. The National Institute of Standards and Technology publishes an AI risk management framework that treats unreliable output as a governance problem rather than a user error. That framing is correct. If your tool can generate a citation, your firm owns the verification step, and that ownership needs to live somewhere concrete, not in a training deck.
The practical version: decide which parts of your workflow the tool may touch unsupervised, and which parts require a human to sign off. Contract review of a client’s own documents is a different risk than drafting a brief that goes to a court. Same software, different stakes.
A scenario worth walking through with your team
Picture a mid-size firm with two associates, a paralegal, and a partner who reviews everything the night before filing. The associate drafts an argument with the tool, gets a citation to an appellate case, likes the language, and drops it in. The paralegal checks the format. The partner reads for argument strength and misses the citation. Nobody was lazy. The system simply had no owner at the exact point where verification mattered.
Now rerun it. The associate opens the authority, sees the case exists but holds the opposite of what the draft claims, and rewrites the paragraph. Eleven minutes. That is the entire cost of a verification routine that actually works.
One more layer people forget: what you feed the tool. Client documents, work product, and drafts may carry confidentiality obligations, and dumping them into a tool without thinking about data handling is its own problem. The U.S. Copyright Office has been examining questions around AI inputs and outputs, which is a useful reminder that the material you paste in is not free of consequences just because the tool is convenient.
The bottom line for your practice
Your drafting tool will keep generating plausible authority. That is not a bug you fix with a better prompt. It is a permanent characteristic of how these systems work, and the firms that handle it well are the ones that stop expecting the software to be careful and start being careful themselves.
Pick one filing this month. Run every citation through the routine above before it goes out the door. Then ask yourself how many of your last ten filings would have survived that pass.