To a model, "extract" is a generation verb
When you ask a large language model to extract a value from a document, it does not go to the page, find the characters, and hand them back. It reconstructs the most probable answer from everything it has ever seen, your document included, but not only your document. The output is fluent, confident, and usually close. Sometimes it is subtly, invisibly wrong. And it will never tell you which time is which.
A better model gets the answer right more often. One of the two models we tested returned every value exactly. But no model can make its answer provable or reproducible. That gap is structural, it is what generation is. The question is not whether the model is smart. It plainly is. The question is whether you can tell its dream from the record on the page.
May cause vivid dreams in which every figure arrives with a citation, a confident smile, and a firm handshake. The citation is real, it simply belongs to a different number. Payment is due April 1st, which should have been the first clue.
What happened when we reached for one
We had a bulleted list of agreed terms in a services agreement. The bullet characters had been mangled by an encoding error, the common one, where a bullet becomes garbled bytes when UTF-8 is read as Windows-1252. A deterministic parser that split the terms on the bullet found no bullets and returned nothing.
So we did what everyone does. We reached for a model. The data is messy, let the AI sort it out. We ran the same synthetic document through two ordinary models, and the honest result is worth reporting, because the honest version is the useful one:
One model returned the terms and silently reformatted a date. Another kept every figure exactly right, and still gave a fluent paraphrase with no way to point at where any of it came from, and no promise it would say the same thing tomorrow. Neither model was random. Both were confident. And the mangled bullets we were routing around were a three-line deterministic repair, not a reason to hand the terms to a dream.
The self-test
Five questions separate a system that reads your record from one that dreams a plausible one. You can run them on any extraction pipeline:
1 · Re-run it
Same document, twice. Byte-for-byte identical output, or not?
2 · Blank a field
Remove a value and ask for it. Does it say not present, or invent a plausible one?
3 · Trace a number
Can you click any value and land on the exact spot in the source it came from?
4 · Corrupt it
Feed a mangled or redesigned version. Does it repair and locate, or paraphrase and hide?
A system you can defend passes all five. A generative pipeline fails #3 by construction, it has no real location to hand you, and treats #1 and #2 as habits it can break without warning. #2 is the one that, the day it quietly lapses, signs off on a number that was never on the page.
Some patients see numbers that are not on the page. The numbers are specific, articulate, and deeply offended when questioned. If a figure insists it has a source, nod politely and locate the source document. Do not accept a spreadsheet as a support animal.
The test you can paste into your model
You do not have to take our word for any of this. Copy the block below into whatever model you use for extraction, ChatGPT, Claude, Gemini, a local one, and read the result against the notes underneath.
Extract data from ONLY the document between the === markers. Use no outside knowledge. === SERVICE AGREEMENT - TEST FIXTURE Section 4. Terms. - Fixed fee: $48,750, payable on 2026-04-01. - Late payment interest: 1.5% per month. === Return one row per field for: fixed_fee, payment_date, late_interest_rate, governing_law, termination_notice. For EACH field, give five columns: 1. value 2. the exact characters you copied it from (verbatim) 3. where it is (line number and character position) 4. LOCATED (you copied it) or GENERATED (you reconstructed it) 5. Will this answer be byte-for-byte identical if I paste this again tomorrow? (yes / no) Rule: if a field is not in the document, answer NOT PRESENT. Do not guess.
How to read what comes back
We ran this block on two ordinary models, and the result is sharper than a strawman.
The values may be right, or quietly not, and you cannot predict which. One model returned the fee and the date exactly. The other dropped the dollar sign and comma, and converted "1.5% per month" into "0.015". Same prompt, same fixture, same day. A correct answer here is luck, not accountability.
Ask where it is, and it invents the citation. That is the tell. Both models dutifully returned character positions. They are fabricated. A model that generates cannot count the bytes in your file, it produces a plausible-looking offset. Pick any row and check it against the fixture, it points to the wrong place. A made-up citation is more dangerous than none, because it looks like proof.
NOT PRESENT, maybe. Both models correctly flagged the two absent fields this time. Good. But nothing guarantees it, you are trusting the model to abstain on every field, on every run, with no mechanism underneath.
The test does not mean your model is dumb. It plainly is not, one of ours got every value right. It shows something narrower and worse: it cannot demonstrate it is right, and it is not consistent. A number you can neither locate nor reproduce is not one you can put your name on.
The trap is in the question
The block above forced you to do something you never do in real life: demand the exact characters, and where they are. In real life you paste the document and ask "what are the terms?" or "what does it say?", and those are not retrieval requests, they are requests for a summary. You asked for prose, so you get prose. The paraphrase is not the model failing your question, it is the model answering exactly the question you asked.
And a sharper prompt cannot fix it. Even when we demanded verbatim text and a location, the models handed back real-looking values with a made-up location. The way out is not a better question. It is to stop asking a question at all. You do not ask a filing cabinet what a page says, you point at the drawer and pull the page. A system that reads the record takes a pointer and returns the bytes: no question, no prose, no dream in the loop.
Prolonged use may allow patients to read lines that were never printed: clauses, effective dates, and the governing law of states they have never visited. Confidence improves as the letters run out, the bottom line is always Delaware. Corrective lenses (deterministic checks) available by prescription.
Ask it about itself
The shortest version of this whole note is one question. Open a fresh chat and ask your model: "if I send you this exact message again, will your reply be byte-for-byte identical? Answer yes or no, and why."
We asked two models this, three times each. Every time the answer was "No", and every one of the three answers was worded differently. The model disproved its own determinism in the act of denying it. It cannot say the same thing twice, and it says so, three different ways.
Then watch it explain why, because that is where the dream shows. One model blamed "the current time" and "improvements made after the initial response". Neither is true, it does not learn between chats, and its output does not depend on the clock. Asked to account for its own dreaming, it dreamed the reason.
That "No" is a confession, not a footnote. The model is telling you, in its own words, that it will not give you the same answer twice, and cannot even say consistently why. That is the whole reason a number it produced cannot be signed onto anything, or defended in an audit: you cannot build a record on something that will not repeat itself, and it already knows.
What reading the record looks like instead
A system built to locate, not generate, returns the actual bytes from your file, a real offset you can verify, and NOT PRESENT wherever a field is genuinely absent. It repairs the mangled bullets deterministically and then points at the exact clause, unaltered, with a certificate that says it will say the same thing tomorrow. It is not more clever than the model. It is accountable, which is a different property, and the only one that survives an auditor.
| The test | Your LLM | Reading the record |
|---|---|---|
| Re-run it, byte-identical? | no guarantee | always |
| Blank a field, says NOT PRESENT? | sometimes | always |
| Trace a number, a real, checkable location? | no, fabricated | yes |
| Corrupt the bytes, repair and locate? | paraphrases | repairs and locates |
| Amend it, knows which version supersedes? | no mechanism | tracks both |
You should not have to trust the number. You should be able to prove it.
Run the paste-block on your own pipeline. If it fails #2 or #3, and it probably will, that is worth knowing before it signs something. doloop builds the deterministic, located-not-generated version.