What is an AI hallucination?
By Yassine Seghrouchni, founderUpdated
In short
A hallucination is output from an AI system that is phrased convincingly but is wrong or made up: a legal article that doesn’t exist, an incorrect figure, a quote nobody ever said. It isn’t a malfunction but a consequence of how language models work. So you can’t prevent it entirely, but you can keep it under control.
Also known as: hallucination
Why it happens
A language model predicts likely text. Ask it about a legal article it doesn’t know, and “Article 14, paragraph 3” is a plausible continuation: it looks like a legal article and sounds like one. Whether it exists isn’t something the model takes into account.
That explains a counter-intuitive pattern: the more specific and formal an answer sounds, the more careful you should be. Vague answers are rarely made up; precise ones sometimes are.
What reduces the risk
Provide the sources: a model that has the contract, the manual or the price list in front of it doesn’t need to invent anything. This is by far the most effective measure.
Ask where it found it: “quote the exact sentence you’re basing this on” shows straight away whether there’s anything behind the answer.
Make checking a rule: set out in your AI policy which kinds of output are always checked before they go out. Think of figures, legal articles, names, quotes and commitments.
Watch where the output ends up: a hallucination in an internal draft is a nuisance. The same hallucination in a quote or a legal document is a problem.
Why it matters in practice
Three situations keep coming back. Someone copies a figure into a client presentation without checking the source. Someone has AI write a legal or tax text and assumes the references are right. Or an agent that takes several steps on its own builds on an error from an earlier step.
That last one is the most underestimated: in a chain of steps, an error doesn’t stand out, it carries through. Practical measures are in preventing AI hallucinations.
Frequently asked questions
Can you prevent hallucinations entirely?
No. Language models generate likely text, not verified facts. You can reduce the risk considerably, though: give the model the sources instead of letting it draw on its memory, and always have factual statements checked before they go out.
Which output is most prone to hallucinations?
Anything specific and verifiable: figures, dates, legal articles, case law, names, quotes, references and product specifications. That’s exactly where a made-up answer sounds most credible, because the model imitates the format perfectly.
