How to check whether an AI made up a citation
Every reference is a claim about the world. Some of them are false in ways that are difficult to see and very easy to submit.
25 August 2026
If you have used an AI tool to help with a literature review, you have probably already been handed a reference that does not exist. It looked correct. It had plausible authors, a real journal, a sensible year, and a title that sounded exactly like a paper somebody would write. That is the problem. A fabricated citation is not obviously wrong. It is designed, by the way it is produced, to be indistinguishable from a real one until you go and look.
This is a practical guide to looking. It takes about a minute per reference, and it catches the failure that almost nobody checks for.
Why it happens at all
A language model does not look anything up. It produces the text that most plausibly comes next, one piece at a time, based on patterns in what it has read.
A citation has an extremely regular shape. Surname, initials, year, title, journal, volume, pages. That shape is easy to imitate, and imitating it is the whole of what the model was asked to do. So it produces something in the shape of a reference, assembled from fragments of thousands of real ones. It is not lying to you, and it is not malfunctioning. It was never checking in the first place.
Which is also why asking the model whether its own citation is real does not help. In 2023 a lawyer in New York filed a court brief citing six cases that did not exist. He had asked ChatGPT to confirm they were genuine, and it had said yes. He was sanctioned. Asking the process that produced the error to review the error is not a check.
The three ways a citation can be wrong
They are not equally common, equally easy to spot, or equally damaging. They get worse as you go down.
Kind one
The paper does not exist
Invented authors, invented title, or a real journal with a volume that was never published. This is the one everybody talks about, and it is the least dangerous, because it collapses the moment anyone looks for it. Ten seconds of searching ends it.
Kind two
The paper exists, but not as described
The title is real and the authors belong to a different paper. Or the year is wrong, or the journal is wrong, or the title has been spliced together from two genuine articles. Harder to catch, because searching the title finds something, and finding something feels like confirmation. It is not.
Kind three
The paper exists and says something else
Correct authors. Correct year. Correct journal. A DOI that resolves. Everything about the reference is true, and the claim attached to it is not in the paper.
This is the dangerous one. Nothing in your reference list looks wrong, because nothing in your reference list is wrong. Your supervisor cannot catch it by reading your bibliography. It survives every check that operates on the reference rather than on the sentence, which means it survives all the way to the viva, where somebody who has actually read that paper is sitting across the table from you.
Kind one embarrasses you. Kind three is the one that costs you your defence, because by the time it surfaces you have built an argument on top of it.
How to check, fastest first
Resolve the DOI
Put the DOI after doi.org/ in your browser. If it resolves to a real
article page, the paper exists. If it does not resolve, the paper does not exist
and you are finished. This takes about ten seconds.
If no DOI was given at all, treat that as information rather than an inconvenience. Anything pulled from a real database comes with an identifier attached.
Search the exact title, in quotation marks
Google Scholar, Crossref or your library catalogue. The quotation marks matter, because without them a search engine will helpfully find you something close and you will accept it.
Then check the authors against what you were given, not just the title. A real title attached to the wrong authors is kind two, and it is the single most common way a reference passes a careless check.
Find the sentence
Open the paper, or at the very least the abstract, and find the actual words that support the claim you are about to make. Not a section on roughly that topic. Not a paper that is broadly about the right thing. The sentence.
This is the step people skip, and it is the only one that catches kind three. It is also the step that takes real time, which is exactly why it gets skipped.
If you cannot find a sentence, you do not have a citation. You have an association.
When you find one
The instinct is to go and find a different paper that does support the sentence, and move on. That is reasonable, and it is not enough, because it treats a fabricated reference as a formatting problem.
The claim came from nowhere. Nothing generated it except the fact that it sounded like the sort of thing that would be true. So before you go looking for a paper to attach to it, decide whether you still believe the claim at all. Sometimes you will, and the literature will back you. Sometimes you will realise you were about to argue something you have no evidence for, and you have been saved.
Then check the others. Fabrication is rarely isolated. If one reference in a list was assembled rather than retrieved, the rest of the list came from the same place.
Three questions to ask of any tool you use
You should not have to run the checks above on every reference for the rest of your career. The tool should have done it. Whichever one you use, ask it three things.
- Where did this paper come from? A real index such as Crossref, OpenAlex, PubMed or Scopus, or the model's memory of what papers usually look like? These are completely different operations and only one of them can be wrong in kind one.
- Can it show me the sentence? Not the paper. The specific words the claim was drawn from. A tool that cannot produce them has not checked either, and has handed the work of checking back to you without saying so.
- What did it actually read? Most papers are behind a paywall, so most tools read the abstract and nothing else. That is often fine. What is not fine is a tool that reads an abstract and then writes as though it read the paper, because that is kind three, performed on your behalf, at scale.
Why we built Gapnostic this way
Gapnostic is a research workspace built around those three questions. Papers come from a real academic index rather than from a model's recollection. Every claim it makes carries the sentence it came from, so you can see the original wording without leaving the page. And it tells you plainly whether it read the full paper or only the abstract, because pretending otherwise is the failure that matters most.
It will not write your paper. It does the reading, the cross-checking and the organising. The thinking and the writing stay yours, which is the only arrangement that survives a viva.