Chapter 3. Hallucinations, False Confidence, and the Collapse of Verification

3.1 What a hallucination is

In AI discussions, “hallucination” commonly describes a generated claim that is false, fabricated, or not supported by the available evidence. “Confabulation” is also used because it avoids implying that a machine has human subjective experience. Examples include a nonexistent court case, a misquoted policy, an incorrect statistic, or a bibliography entry whose title, author, journal, or DOI is wrong (NIST, 2024).

Errors are particularly dangerous when they are specific enough to appear verifiable but are accepted without checking. A fabricated citation can enter a student paper, a business report, or a policy memo and then be repeated by others. Repetition does not turn an unsupported claim into evidence.

3.2 A verification workflow

Identify the claim: separate factual statements, interpretations, estimates, and recommendations.
Trace the source: open the original publication, dataset, law, or official record rather than relying only on a summary.
Check the exact detail: confirm the author, date, page, quotation, statistic, jurisdiction, and context.
Seek independent confirmation: use a second credible source where the claim matters.
Test calculations: reproduce the arithmetic or analysis and inspect assumptions.
Record uncertainty: state what is known, what is inferred, and what remains unresolved.
Escalate high-stakes claims: ask a qualified person to review them before action.

3.3 Source quality and citation integrity

Source quality varies. Primary research, official statistics, legislation, standards, and original institutional reports often provide stronger evidence for specific factual questions than anonymous posts or promotional material. Yet official sources can still be incomplete or out of date, and a research paper can have limitations. Read the method, date, scope, conflicts of interest, and caveats.

Never cite an article solely because an AI tool supplied a plausible-looking reference. Search for the publication independently and confirm that it exists and supports the statement. Do not invent page numbers, quotations, DOIs, or findings. If a source cannot be found, remove it or clearly state that the claim remains unverified.

3.4 When not to use AI as the source of truth

Do not rely on an unverified model answer as the sole basis for medical treatment, legal action, investment, safety-critical engineering, or disciplinary decisions.
Do not use generated summaries as a substitute for reading the decisive clause of a contract, regulation, consent form, or research method.
Do not assume that a model has checked current information unless its tool access and cited sources make that process clear.
Do not let a deadline turn an unverified draft into an authoritative final document.