How hallucination happens
A language model is built to produce the most plausible continuation of your text, not to check whether it is true. When it knows a subject well, plausible and correct usually coincide. When it does not, the machinery still runs, and what comes out is a well-formed sentence with nothing behind it. There is no internal signal that says this part I am making up, which is why the invented material reads exactly like the accurate material.
Certain questions provoke it reliably. Anything narrow and specific — a small company, a local regulation, a niche product — sits in the thin part of what the model saw during training. Requests for citations, page numbers, prices or statistics are especially risky, because the model has learned what a reference looks like far better than it has learned any particular reference.
Grounding the tool in real documents reduces it considerably. But retrieval is imperfect: if the right passage is not fetched, the model will often answer from its own patterns rather than admitting the material was not there.
Why hallucination matters
The risk is not that the tool is wrong. It is that it is wrong in a register that sounds authoritative, so mistakes pass review. Nobody double-checks a sentence that reads like it came from a well-informed colleague.
In marketing that lands in three expensive places. A published article carrying an invented statistic or a source that does not exist damages the credibility you spent years building. A chatbot that improvises a price, a policy or an eligibility rule creates an expectation your business is then held to. And a proposal with fabricated references is the kind of thing a prospect checks.
Where hallucination causes real damage
Customer-facing automation is the sharpest edge, because there is no editor between the model and the reader. A bot answering questions about refunds, stock, medical eligibility or immigration paperwork can do genuine harm, not merely embarrassment.
The quieter damage is in research. Ask for market figures, competitor claims or industry benchmarks and you will get an answer shaped exactly like the one you wanted. Decisions then get made on numbers that were composed rather than measured, and the error is discovered much later, if at all.
How to act on it
Change what you ask for. Give the model the source material and ask it to work within it, rather than asking it to recall something. Ask it to quote the passage it relied on, and treat an answer with no quotable source as unverified. When it says it does not know, that is the tool working correctly, not failing.
Then put a check where the cost is highest. Every claim that could be verified by a reader gets verified before publication; every customer-facing bot is grounded in your own documents and told to hand over rather than guess. A human approval step before anything reaches a customer is the single most effective control, and retrieval from your own content is the most effective technical one.