Why AI Invents Cases That Don't Exist
AI hallucination, explained by someone who builds these systems — why chatbots fabricate authorities that look real, and a simple rule to never cite a fake one.
If you take one thing from this blog, make it this: an AI chatbot can hand you a case citation that is complete, correctly formatted, plausible-sounding — and entirely invented. Advocates have been sanctioned for filing exactly that. Here is why it happens and the one rule that makes you immune.
Why the machine makes things up
A language model generates text by predicting what should come next, based on patterns in its training data. When you ask for authority on a point, it knows what a Kenyan case citation looks like — party names, a year, a court, a neutral citation number. So it produces something in that shape.
The problem is that “looks like a real citation” and “is a real citation” are, to the model, almost the same task. It is not retrieving a case from a database and copying the reference. It is composing a reference that fits the pattern. Most of the time the pattern happens to match something real. Sometimes it doesn’t — and the fabrication is indistinguishable from the genuine article at a glance.
This is called hallucination, and it is not a bug that will be fully patched away. It is a direct consequence of how the technology works. A model that can fluently draft your letter is, by the same mechanism, a model that can fluently invent a judgment.
The rule that makes you immune
You do not need to understand transformer architecture to stay safe. You need one habit:
Never rely on any authority you have not opened yourself.
If the AI cites Republic v Somebody [2019] eKLR, your job is to find that case in a real source — Kenya Law, or a tool that links straight to it — and confirm two things: that it exists, and that it actually says what the AI claims. If you can’t open it, you can’t cite it. Full stop.
That single discipline converts AI from a liability into a genuine accelerant. The model does the fast, tedious first-draft work; you do the verification that only a source of truth can provide.
Why grounding beats trusting
This is the difference between a bare chatbot and a grounded tool. A grounded system doesn’t ask you to trust its memory — it retrieves the actual case law and legislation and shows you the source behind every statement, so verification takes a click instead of an afternoon.
That is the whole reason JuriBase exists: not to be a smarter guesser, but to make every answer traceable back to primary Kenyan authority. Fluency you can get anywhere. Verifiability is what keeps you out of trouble.
For more on why confident output isn’t the same as correct output, start with What Happens When You Ask an AI a Legal Question.
Frequently asked questions
What is an AI hallucination?
A hallucination is when an AI produces confident, fluent output that is simply false — a fabricated case name, a wrong citation, a statute section that doesn't say what the model claims. It happens because the model generates plausible text, not verified facts.
How do I stop AI from giving me fake cases?
You can't fully stop a bare chatbot from fabricating — but you can refuse to rely on anything you haven't opened yourself. Use tools that link every claim to a primary source, and click through to confirm the case exists and says what's claimed before you cite it.
Have lawyers really been sanctioned for citing fake AI cases?
Yes. Courts in several jurisdictions have penalised advocates who filed submissions citing authorities that turned out to be AI fabrications. The lesson is universal: verify before you file.
Alvin Otanga
AI engineer and researcher building grounded, verifiable legal AI for Kenyan advocates.
Alvin is the founder of JuriBase, where he builds AI that is grounded in Kenyan case law and legislation — so advocates get answers they can actually verify and cite. He writes here to teach lawyers how AI really works, from the inside.