How to Check AI Hallucinations Before You Trust an Answer

To check an AI answer, separate its important claims and verify each against evidence you can inspect. Confidence, fluent prose and a long list of citations are not substitutes for that check.

An AI hallucination is an output that presents unsupported or incorrect information as if it were established. It might be an invented paper, a nonexistent software option, an incorrect calculation, or a conclusion that the supplied document never made. Anthropic's guidance on reducing hallucinations recommends allowing uncertainty and grounding answers in source material. These measures reduce risk; they do not certify an answer.

The following is an editorial checking workflow you can use with a research summary, a tool comparison or a technical explanation.

1. Identify the claims worth checking first

Read the answer once and mark the statements that would change your decision. If you are choosing software, that could be a price, a privacy setting or an export limitation. If you are implementing code, it might be an API name or a supported runtime version.

Prioritize assertions that are precise, time-sensitive, surprising or consequential. You do not have to investigate every harmless introductory sentence before checking the claim on which the whole answer depends.

Turn a paragraph into a short claim list:

Generated statement	Verification task
“The free plan includes unlimited exports”	Find the current plan page and the meaning of export
“This function was introduced in version 3”	Find the official reference and release note
“The intervention improved results by 20%”	Read the study's measure, baseline and sample
“The document recommends replacing the system”	Locate the recommendation in the actual document

These are illustrative examples, not claims about any particular product or study.

2. Open the source and look for actual support

A citation can fail in three different ways: the source does not exist, the source exists but says something else, or it supports a narrower claim than the answer makes.

For each important citation, check:

1. Does the destination identify a real document and its publisher?
2. Can you locate the passage, table or data behind the claim?
3. Does that evidence support the full statement, including qualifications?
4. Is it the applicable product version, date or population?

For software, begin with the maintainer's documentation. For a research claim, inspect the paper rather than a summary of a summary. For a policy, read the issuing organization's original text.

Keep a small evidence note: the claim, the URL, the relevant passage and your conclusion. Write “not established” when the evidence is incomplete. That is a useful result, not a failed search.

3. Check dates, units and comparisons

Many misleading answers contain real numbers used in the wrong way. A monthly price may assume annual billing. A benchmark may describe an older version. An improvement from 10 to 12 is two units, or 20% relative growth; those are different descriptions of the same change.

Recalculate important arithmetic independently. Check whether percentages are absolute percentage-point changes or relative changes. Make sure both sides of a comparison use the same timeframe and measurement.

A timestamp on the AI response tells you when it was generated. It does not tell you when the underlying evidence was published or last verified.

4. Reproduce technical claims when possible

If an answer says that a command works, test it in a suitable disposable environment with non-sensitive inputs. Check the documented flags before running it. Record the version and exact error if it fails.

For generated code, separate three questions:

• Does the code run?
• Does it return the right result for a normal example?
• What happens with an empty input, missing value or invalid request?

A successful run answers only the first question. A readable explanation of code does not answer any of them by itself.

For document analysis, reproduction is often simpler: open the cited page and compare the model's conclusion with the original wording. The NotebookLM study workflow applies this idea to course material.

5. Ask for uncertainty without outsourcing the verdict

Use a request like this before accepting a factual summary:

Separate your answer into:
1. Claims directly supported by the supplied sources.
2. Inferences, with the reasoning behind each.
3. Unknowns or claims that need an additional source.

For every important factual claim, identify the source and relevant passage.
Do not invent a URL, quotation, paper title or missing number.
If a source is unavailable, mark the claim unverified.

This is a checking aid. You still need to open the sources. Asking “Are you sure?” or having a second assistant approve the first answer can produce agreement without independent evidence.

Why adding documents does not solve everything

Retrieval-augmented generation gives a model access to selected material when answering. That makes source checking possible, but it creates additional questions: was the right document retrieved, was it current, and was it interpreted correctly?

Google's discussion of RAG limitations notes that poor source material or incorrect interpretation can still produce misleading outputs. If a system keeps answering from the wrong documents, a more forceful prompt may not fix the retrieval problem.

A stopping rule for everyday work

Before sharing the answer, you should be able to identify the evidence for the conclusion, reproduce any decisive calculation, and state what remains uncertain. If the central claim cannot be verified, omit it or label the uncertainty clearly.

Keep this effort proportional to the decision. A brainstorming list needs a different level of checking from a factual report someone will rely on. The goal is an answer whose important claims you can explain and defend.