How to Fact-Check AI-Generated Answers

Imagine an assistant has just given you a clear, confident paragraph about a topic you need for tomorrow. Three statistics, two named studies, a date for a rule change. It reads like a briefing from an expert. What now?

If you have been following this course, you already know the paragraph could contain invented details, and that you cannot tell by reading. This article is the routine for finding out. It is the skill that underlies almost every other article in this course, because every module ends with a version of "check what matters before you rely on it."

• A nine-step verification routine.
• How to decide what is worth checking.
• A source hierarchy, and why primary sources come first.
• How to check dates, context and numbers.
• How to verify citations an AI gives you.
• Extra care for high-stakes topics.

CLAIM  -> IDENTIFY what needs verification  -> FIND the primary source  -> COMPARE independent sources  -> CHECK the date  -> CHECK the context  -> VERIFY the numbers  -> CITE the source  -> DECIDE how far to trust it

You will rarely run every step on every claim. The routine tells you what is available, and the stakes tell you how far to go.

A paragraph is a bundle of claims, and they differ in how risky they are. Pull them apart. You can ask the assistant to help, because this is a language task you can check against the text:

List every factual claim in the text below as a numbered list, one claim per line,using the original wording. Mark each as: fact (checkable), number or statistic,named source (study, law, person, document), prediction, opinion, or definition.Do not judge whether they are true.

You now have a list. Next, decide what to verify.

You cannot verify everything. Prioritise:

• High stakes: the answer will affect health, money, legal position, safety, someone's reputation, or a decision.
• Specific: exact numbers, dates, names, quotes, page numbers.
• Load-bearing: the argument falls apart if this claim is wrong.
• Surprising: it contradicts what you thought you knew. Surprising claims are more often wrong, though not always.
• Unsourced: the assistant gave no way to check it.
• Time-sensitive: prices, laws, versions, office-holders, "latest" anything.

Low-stakes, general, well-known claims that you can judge yourself can be skimmed.

A primary source is the original: the text of the law, the dataset, the published study, the company's own filing, the official documentation, the original speech or document. A secondary source reports on or interprets primary material (news articles, textbooks, explainers). A tertiary source summarises secondary ones (encyclopedias, overviews).

A habit that helps is lateral reading: when you meet an unfamiliar site, open new tabs and search for what other sources say about it, instead of reading the site's own "About" page. Who runs it? Who funds it? Do reliable sources cite it?

Search with the claim's specific words plus the likely source type: the name of the law and "official text", the study's title and "journal", the product name and "documentation".

One source can be wrong. For important claims, find at least one other source that is independent, meaning it did not just copy the first. Five articles repeating the same press release count as one source.

• If sources agree and at least one is primary, you can be fairly confident.
• If they disagree, work out why: different dates, definitions, populations or errors. Do not average them.
• If you can find no source at all, treat the claim as unverified, however plausible.

Many "false" claims were once true.

• When was the source published or last updated? Look for "last updated" or a version number.
• Does the claim have an "as of" date? Laws, prices, limits and policies change.
• Is there a newer version? Check for amendments, new editions, corrections.
• When was the AI's knowledge fixed? It may not know about recent changes.

A claim is only as good as its date. Write the date next to every fact you keep.

A true fact in the wrong context misleads.

• Read the whole sentence and the paragraph around the quote. Conditions and exceptions often follow ("except where", "subject to").
• Who and what was studied? A finding about adults in one country may not apply to teenagers elsewhere.
• Is it a correlation, an experiment or an opinion? The strength of the claim must match the evidence.
• What does the source actually say versus what the AI says it says? Compare the two side by side.

• Find the figure in the source, and not only in a summary.
• Check the unit and the base. Percent of what? Per year or total? Thousands or millions? Rupees or dollars?
• Recalculate any derived number yourself. Models are unreliable with arithmetic.
• Check the definition. "Unemployment", "inflation", "user" and "revenue" can be defined in different ways.
• Be wary of precision. A very exact figure with no source is a red flag.

Record each verified fact so that you (or a reader) can retrace your steps:

A claim without a source is a rumour. Include the exact page or section, since that makes your check repeatable.

Give each claim a label:

Then decide what to do. Use confirmed claims. Qualify probable ones ("according to...", "as of..."). Drop or flag unverified ones.

Claim from an AI answer (fictional scheme, illustrative numbers). "The State Skills Grant pays Rs 5,000 per month for six months to any student aged 18 to 25, according to a 2023 government order."

Step 1 and 2. Claims: amount (Rs 5,000), duration (six months), eligibility (any student, 18 to 25), source (a 2023 order). All are specific and a student might apply based on them, so check all four.

Step 3. Search the state government's official website for the scheme name and "order" or "guidelines". Suppose you find the order. The assistant gave no link, and its description of the order may not match.

Step 4. Look for a second source: the scheme's official portal or a reputable news report. Suppose the news report mentions a different amount.

Step 5. The order is dated 2023, and the news report is from this year. Look for amendments. Suppose an amendment raised the amount and added an income limit.

Step 6. Read the eligibility clause. Suppose it says "students enrolled in recognised institutions with family income below a limit". The AI's "any student" was wrong.

Step 7. Check the amount and the duration in the latest version of the order. Recalculate the total.

Step 8 and 9. Record the amendment as the source, with the clause number and date. Label the amount and the eligibility "confirmed (as amended)", and the AI's "any student" "contradicted".

Result. The assistant's summary was a reasonable guess based on an older version of the scheme, and acting on it could have wasted an application. You also learned how it was wrong: outdated and over-simplified, not invented from nothing, which is typical.

AI-supplied references need their own check, because they are among the most commonly invented content. Test each in three ways:

1. Does it exist? Search for the exact title in a library database, the publisher's site or a scholarly search engine. Do not rely on the AI's link, which may be invented.
2. Are the details right? Authors, year, journal, volume and pages must match what you find. Real-looking mismatches are a sign of a made-up or garbled citation.
3. Does it support the claim? Read the relevant part. A real paper can be cited for something it does not say.

If it fails any of the three, remove it. Do not "fix" a citation by guessing the correct details. Find the real source from scratch, or drop the claim.

You will not run all nine steps for every question. For everyday low-to-medium stakes, a quick routine:

1. Pick the one or two claims that matter most.
2. Search the key phrase with the name of an official or well-known source.
3. Open the page, find the sentence and read around it.
4. Note the date.
5. If you cannot confirm it in two minutes, label it unverified and do not state it as fact.

AI can speed up the process without being the judge:

• Generate search queries and lists of possible primary sources.
• Extract claims from a text.
• Compare a claim with a document you provide, quoting the supporting line. Then you check the quote.
• List what would prove the claim wrong, which tells you what to look for.
• Explain jargon in a source you are reading.

What it cannot do is serve as the independent source. Another assistant agreeing with the first is not confirmation. They may share the same training data and the same mistakes.

The model makers say as much. Anthropic's guidance on reducing hallucinations suggests asking a model to support each claim with a quote and to retract any claim it cannot support, and it states that such techniques reduce hallucinations without eliminating them, so critical information must still be validated. Research by Kalai and colleagues on why language models hallucinate argues that training and evaluation have tended to reward confident guessing over admitting uncertainty, which is one reason you should not treat a confident tone as a sign of reliability.

• Checking one claim and trusting the rest.
• Treating a link as proof. Open it and read.
• Stopping at the first source that agrees.
• Ignoring dates.
• Quoting a number you did not trace to its source.
• Trusting a summary of a source instead of the source.
• Using another AI as the checker.
• Fixing a fake citation by guessing its details.

• Next: An AI-Assisted Research Workflow: Primary Sources and Citation Checking builds a full research process on this routine.
• For the reasons errors happen, go back to Why AI Sounds Right and Still Gets Things Wrong.
• For checking in high-stakes decisions, see Brainstorming, Planning and Decision Support with AI.