What is a hallucination (AI)

There is no single binding definition of this term. The most solid reference available today is the generative AI profile of the NIST risk management framework, published in July 2024, which avoids the word and speaks of confabulation: the production of confidently stated but erroneous or false content. That document also includes outputs that diverge from what was asked for and those that contradict something said earlier in the same conversation.

The same text records the objection. Several commenters noted that "hallucination" and "fabrication" anthropomorphise the system, and that attributing human traits to a machine is itself a risk. A human hallucination is a perception without a stimulus. A language model perceives nothing: it calculates which continuation is probable. Calling it hallucination suggests an isolated, almost medical fault, when the mechanism is exactly the one that also produces the correct answers.

Academic criticism points in the same direction. A 2024 article in the journal Ethics and Information Technology argues that the term diverts attention towards a supposed malfunction, when what characterises the system is its indifference to whether what it says is true. For practical use the word is already established, so what helps is knowing what it describes and what it covers up.

A language model generates text by choosing, step by step, the most probable continuation according to the patterns in its training data. It consults no fact base and checks nothing before writing. When the question falls in an area well covered by that data, the probable continuation coincides with the true one. When it does not, the system still produces a probable continuation, and that continuation can be false. The internal process is identical in both cases, which is why the confident tone does not change.

The usual patterns follow from that. The model composes a plausible product name your catalogue does not carry, assigns you a headquarters or a founding year that fit the sector, invents a reference with correct formatting and a credible author, or constructs a coherent line of reasoning that justifies a wrong answer. The invented justification is especially misleading, because it looks like evidence.

Adding retrieval of external documents reduces the problem, but does not remove it. If the search brings back the wrong source, if the source is out of date, or if the retrieved passage does not contain the figure, the model can fill the gap just as before, now with a citation beside it that gives it the appearance of being checked. The risk moves: from the model's memory to the quality of what is retrieved and to the real link between the citation and the statement.

On frequency, caution is warranted. Any percentage depends on the model, the task and the measurement method, and those three things are rarely disclosed together. A figure without all three tells you nothing, and it also ages within weeks.

The decision that depends on this is whether or not you monitor what AI systems say about your brand. A model can state that your company has closed, attribute to you a product you never sold, or give a customer a price, a return period or a warranty condition that are not yours. Whoever receives the answer does not distinguish between what the system read and what it composed.

The consequence falls on the brand owner. The user complains to your customer service, not to the model provider's. And if the statement concerns regulated information, such as contractual terms or the properties of a medical or financial product, the problem stops being merely reputational.

There is a part you do control. A good share of the inventions about a brand grow out of gaps: information that is published nowhere, or that is published inconsistently across your site, your company listing and your external profiles. Closing those gaps with explicit, dated and consistent data shrinks the surface on which the system improvises. What you do not control is the model's output, and that is why periodic monitoring is part of the work.

• Publish your company's verifiable data on your own domain (legal name, headquarters, founding year, current catalogue, terms) and keep the same version in listings and external profiles.
• Date the pages whose information expires, such as prices, deadlines and availability, so that an old version is not read as current.
• Check periodically what AI systems answer about your brand, using a fixed set of questions, and keep the full answer together with the model and the date.
• Record every false statement you find in a document with a screenshot, date and system, so you can demonstrate the pattern and how it develops.
• Treat any generated text destined for publication as a draft: verify figures, proper names, citations and links against the original source before publishing it.
• If you find a persistent error with real impact, use the provider's reporting channel and in parallel correct the public source that may have caused it.

• Confusing a confident tone with reliability: firm wording tells you nothing about whether the content is correct.
• Asking the model itself to confirm whether it has invented something, and accepting its answer as verification.
• Taking a fact as verified because a citation accompanies it, without opening the link and checking that the passage says that.
• Quoting hallucination rates without stating model, task and method, or comparing figures from studies that measured different things.
• Claiming that a particular model errs more often than another without a dated source, when versions change every few weeks.