Confidence isn't a promise: how to trust extracted data

Every system that extracts data eventually shows you a number: the confidence. It looks reassuring. But where does that number come from? Often it's a self-assessment: the system says «I'm 92% sure» without that 92% corresponding to anything verifiable.

For someone who has to post an invoice or close a warehouse, an opaque score isn't enough. You need to know whether the total adds up, whether the VAT number is plausible, whether the date makes sense. For confidence to be useful, it has to translate into checks you can list — not a generic grade.

Self-declared versus proven

There's a huge difference between «I think I read it right» and «I verified that the numbers add up». The first is an opinion, the second is a fact. For an accounting figure, the only kind of reliability that counts is the kind you can prove with a deterministic rule.

Language models and computer vision can be brilliant at reading complex layouts, but they aren't infallible. An internal «92%» from the model doesn't tell you whether that 1 and that 4 in the total were swapped. That's why you need a verification layer separate from extraction: rules that don't «interpret», but check.

• Taxable amount plus tax must equal the total, line by line and in the closing total
• The VAT number must pass the validity check (checksum)
• The IBAN must have a correct check digit
• Dates must be consistent and plausible (document, due date, delivery)
• Duplicates: same supplier, same number, same amount within a suspicious window

When a document passes all these checks, reliability isn't an impression: it's a measurable property. And when it doesn't pass them, you know exactly why and where to look — you don't have to re-read everything from scratch.

A good confidence score should answer a simple question: what, exactly, has been verified?

Confidence per field, not just per document

An error on the total is different from an error on a line description. A mature system distinguishes: the invoice number may be high confidence because it's clearly visible, while a row at the bottom of a table may be more uncertain. That enables finer routing rules: auto-approve the document but flag the doubtful line, or block only the anomalous field.

In practice, the review dashboard shouldn't show only «84%» in green or red, but «total: verified», «IBAN: verified», «line 7 quantity: needs review». That way the reviewer spends seconds, not minutes.

Why it matters for routing

Confidence isn't a grade to pin on the wall: it's there to make a decision. Above a certain threshold the document carries on by itself; below it, it goes to a person. If the threshold is based on real checks, the system sends the right documents to review — not random ones.

Thresholds are calibrated to your risk: a company with high amounts and few suppliers can be more restrictive; a logistics operator with hundreds of delivery notes a day may prioritize throughput and review only structural anomalies. What matters is that the threshold is tied to explicit rules, not a magic number from the model.

What to ask in a demo or RFP

• List of deterministic checks included out of the box
• Ability to add custom rules (e.g. approved supplier, amount threshold)
• Log for every check: passed/failed, expected vs extracted value
• Separation between model score and business verification score

If the vendor can't explain what the score is made of, be skeptical. Trust in accounting data isn't bought with an elegant UI — it's built, one check at a time.