fuelish/testing/2026_10_03_ocr_variant_experiment.md

3-variant OCR cross-referencing experiment

Tested whether running Tesseract over three image variants of the same receipt photo (original color photo, plain grayscale, and the existing denoised/contrast-enhanced preprocessing) and asking the extraction LLM to cross-reference all three per field would recover more correct data than the single-OCR-pass baseline, particularly for blurry/low-quality photos.

Result

Mixed. One more receipt matched overall, two fields improved meaningfully, but the original target field (litres) did not improve, and total_price regressed.

Metric	Baseline (1 OCR pass)	New (3-variant cross-ref)
Matched to ground truth	15/17	16/17
price_per_litre	11/15 (73%)	13/16 (81%) ↑
litres	9/15 (60%)	9/16 (56%) — unchanged, still worst
discount	11/15 (73%)	15/16 (94%) ↑↑
total_price	12/15 (80%)	11/16 (69%) ↓

Root causes identified

1. The litres/price-per-litre line is the hardest OCR target, regardless of variant count. Noise characters (x2p 02, */p) precede the real numbers and the litre symbol l reads as 1/]/| in all three variants alike, so this field stays wrong at the same rate.
2. Three OCR readings don't guarantee the model picks the right one. On the blurry Jet receipt, the correct total 57,13 survives intact in the original-photo variant, but the model still output 37.13. Extra (noisy) input sometimes confuses the small 3B model rather than helping it reconcile.
3. total_price regressed — cross-referencing conflicting reads apparently made the model second-guess fields it used to get right from a single clean reading.
4. Severe performance/stability cost on this 2-core, GPU-less VM: extraction latency became wildly inconsistent (1–19 minutes per receipt vs. seconds before), and one run hit a hard timeout before the request timeout was raised from 300s → 900s. The full 17-receipt batch that used to finish in ~11 minutes now takes well over an hour.

Working examples (fully correct — all 4 fields + date matched)

• IMG_20260726_095120207, 095321692, 095428941, 095442114, 095510481, 095519964 — all Shell receipts with clean, high-contrast OCR text across all three variants.

Not-working examples

• IMG_20260525_182252232 (the blurry Jet receipt that prompted this experiment) — still no date match at all (extracted 13.05.2026 vs. true 15.05.2026). All three OCR variants scramble the header; even though the correct total 57,13 is legible in the original-photo variant, the model output 37.13 instead.
• IMG_20260525_182144541 — worst case, everything wrong except date/time/transaction number: extracted litres=1.205 (should be ~23), price_per_litre=1.389 (plausible alone, but paired with nonsense litres), discount=-0.08 (sign flipped), total_price=5.33. All three OCR variants are near-unreadable for this receipt.
• IMG_20260525_182116766 — litres read correctly (20.24, matches the OCR text "20,24 1 1979 EUR/]" cleanly) but price_per_litre came out as 6.01 instead of ~1.979 — a pure extraction/reconciliation error, not an OCR gap, since the right number is sitting right there in the text.
• IMG_20260726_095312079, 095335197, 095416655, 095452747, 095501803 — all wrong only on litres, the recurring weak field.
• IMG_20260726_095215757 — wrong on litres and total_price together; the OCR text for that line is heavily corrupted (j 2 N N 4) ha 1: Hi A 1.869 EUR/) in all three variants, so this one genuinely lacks clean signal anywhere.

Conclusion / recommendation

The extra OCR variants help in a few places (discount, price_per_litre) but don't fix the core problem (litres line noise), introduce a new failure mode (the model mis-reconciling conflicting-but-sometimes-correct reads), and cost a lot of latency/stability on this hardware. Leaning toward reverting to single-OCR unless a confidence-triggered fallback (only pull extra variants when the primary reading looks incomplete) is tried instead.