// ai

Hallucination

A hallucination is an answer a language model produces with complete fluency and confidence despite having no basis in fact — invented statistics, fabricated citations, policies that never existed. It happens because the model is trained to produce plausible text, not true text: when it lacks the answer, it completes the sentence anyway, and the completion reads exactly like knowledge.

It matters because the failure is invisible: a wrong database query errors, a hallucinated legal clause reads smoothly. The cost lands where the answer is trusted — a customer told a wrong price, an employee citing a policy that does not exist. Hallucination cannot be eliminated; it can be engineered down to a measured, low rate with the countermeasures that work: grounding, refusal rules, low temperature and a fixed evaluation set.

A concrete pattern: a support bot asked about a product feature its manual does not cover. Ungrounded, it describes the feature convincingly — wrong in every detail. Grounded with a refusal rule, it answers that the material says nothing and offers a human follow-up. One way to keep the honest behaviour honest: fifty to a hundred fixed test questions, including ten the material cannot answer, rerun automatically after every change to prompt, model or retrieval.