Questions
Technology
Will AI always make things up?
Training and evaluation can encourage language models to make up answers rather than admit they do not know, according to a study published in Nature. A preprint presents a proof that no language model can eliminate these fabrications, but the result depends on the paper’s assumptions.

Why it matters
How AI is evaluated matters: rewarding guesses can favor made-up answers. Errors can also be measured in specific tasks, as Vectara’s leaderboard does by checking whether summaries include something that was not in the original text.
What the evidence shows
- A preprint presents a mathematical proof that, under the paper’s assumptions, no language model can eliminate hallucination.Hallucination is Inevitable: An Innate Limitation of Large Language Models · Ziwei Xu, Sanjay Jain, Mohan Kankanhalli, arXiv, 2024
- A study published in Nature argues that training and evaluation reward guessing, making a fabricated answer score better than admitting uncertainty.Evaluating large language models for accuracy incentivizes hallucinations · Adam Tauman Kalai et al., Nature 653, 1047–1051, 2026
- Vectara’s leaderboard applies the same test to every model to measure the share of summaries containing made-up information absent from the original text.Hallucination Leaderboard (HHEM-2.3) · Vectara, atualizado em 22 out. 2026, 2026
What I would do
I recommend, as you expand your use of AI, measuring errors, checking sources, and requiring it to admit when it does not know and refer the question to a person.
Why I think so:
- “AI without evaluation is a bet: measure before you promise.” where it comes from
- “You can trust AI, if you check it.”
- “A serious AI says only what it knows, shows where it got it, says when it does not know, and calls a person.” where it comes from
People also ask
- Why does AI hallucinate?
- Can you trust what AI tells you?
