# 3-variant OCR cross-referencing experiment

Tested whether running Tesseract over three image variants of the same
receipt photo (original color photo, plain grayscale, and the existing
denoised/contrast-enhanced preprocessing) and asking the extraction LLM
to cross-reference all three per field would recover more correct data
than the single-OCR-pass baseline, particularly for blurry/low-quality
photos.

## Result

Mixed. One more receipt matched overall, two fields improved
meaningfully, but the original target field (`litres`) did not improve,
and `total_price` regressed.

| Metric | Baseline (1 OCR pass) | New (3-variant cross-ref) |
|---|---|---|
| Matched to ground truth | 15/17 | 16/17 |
| price_per_litre | 11/15 (73%) | 13/16 (81%) ↑ |
| litres | 9/15 (60%) | 9/16 (56%) — unchanged, still worst |
| discount | 11/15 (73%) | 15/16 (94%) ↑↑ |
| total_price | 12/15 (80%) | 11/16 (69%) ↓ |

## Root causes identified

1. **The litres/price-per-litre line is the hardest OCR target**, regardless
   of variant count. Noise characters (`x2p 02`, `*/p`) precede the real
   numbers and the litre symbol `l` reads as `1`/`]`/`|` in all three
   variants alike, so this field stays wrong at the same rate.
2. **Three OCR readings don't guarantee the model picks the right one.**
   On the blurry Jet receipt, the correct total `57,13` survives intact in
   the original-photo variant, but the model still output `37.13`. Extra
   (noisy) input sometimes confuses the small 3B model rather than helping
   it reconcile.
3. **`total_price` regressed** — cross-referencing conflicting reads
   apparently made the model second-guess fields it used to get right from
   a single clean reading.
4. **Severe performance/stability cost** on this 2-core, GPU-less VM:
   extraction latency became wildly inconsistent (1–19 minutes per receipt
   vs. seconds before), and one run hit a hard timeout before the request
   timeout was raised from 300s → 900s. The full 17-receipt batch that used
   to finish in ~11 minutes now takes well over an hour.

## Working examples (fully correct — all 4 fields + date matched)

- `IMG_20260726_095120207`, `095321692`, `095428941`, `095442114`,
  `095510481`, `095519964` — all Shell receipts with clean, high-contrast
  OCR text across all three variants.

## Not-working examples

- **`IMG_20260525_182252232`** (the blurry Jet receipt that prompted this
  experiment) — still no date match at all (extracted `13.05.2026` vs. true
  `15.05.2026`). All three OCR variants scramble the header; even though
  the correct total `57,13` is legible in the original-photo variant, the
  model output `37.13` instead.
- **`IMG_20260525_182144541`** — worst case, everything wrong except
  date/time/transaction number: extracted `litres=1.205` (should be ~23),
  `price_per_litre=1.389` (plausible alone, but paired with nonsense
  litres), `discount=-0.08` (sign flipped), `total_price=5.33`. All three
  OCR variants are near-unreadable for this receipt.
- **`IMG_20260525_182116766`** — `litres` read correctly (`20.24`, matches
  the OCR text `"20,24 1 1979 EUR/]"` cleanly) but `price_per_litre` came
  out as `6.01` instead of `~1.979` — a pure extraction/reconciliation
  error, not an OCR gap, since the right number is sitting right there in
  the text.
- **`IMG_20260726_095312079`, `095335197`, `095416655`, `095452747`,
  `095501803`** — all wrong only on `litres`, the recurring weak field.
- **`IMG_20260726_095215757`** — wrong on `litres` and `total_price`
  together; the OCR text for that line is heavily corrupted
  (`j 2 N N 4) ha 1: Hi A 1.869 EUR/`) in all three variants, so this one
  genuinely lacks clean signal anywhere.

## Conclusion / recommendation

The extra OCR variants help in a few places (`discount`, `price_per_litre`)
but don't fix the core problem (`litres` line noise), introduce a new
failure mode (the model mis-reconciling conflicting-but-sometimes-correct
reads), and cost a lot of latency/stability on this hardware. Leaning
toward reverting to single-OCR unless a confidence-triggered fallback
(only pull extra variants when the primary reading looks incomplete) is
tried instead.