Add adaptive preprocessing escalation and single-receipt test scripts

Preprocessing now tries progressively heavier variants per receipt
(plain -> high-pass -> edge-enhance -> both), stopping as soon as a
cheap LLM-free arithmetic check on the OCR text looks plausible, so
the extra processing only applies to receipts that need it. A flat
version of the same heavier processing regressed the full batch by
helping hard receipts but hurting easy ones; the adaptive version
improves date-matching 37/52 -> 40/52 with no regression (see
testing/2026_10_04_gimp_highpass_nlfilter_experiment.md).

Also adds test_ocr.sh and test_single.sh for checking a single
receipt's OCR text or full pipeline result without a batch run.

testing/2026_10_04_gimp_highpass_nlfilter_experiment.md

@@ -0,0 +1,182 @@
+# GIMP High Pass + NL Filter preprocessing experiment
+
+Tested whether a GIMP sharpening/denoising combo, applied to the photo
+*before* our existing `preprocess.py` (grayscale → bilateral filter →
+percentile autocontrast), recovers the compact "litres / EUR-per-litre"
+price line that's the pipeline's worst-performing OCR target (see
+`AGENT.md`'s "Known remaining failure modes"). Three stress-test receipts
+with known-bad baseline OCR on that line (`testdata/2026_stress_test/`,
+not checked in):
+
+- **A** `IMG_20260525_182252232` — ground truth 29,31 l / 1,949 EUR/l
+- **B** `IMG_20260525_182144541` — ground truth 17,88 l / 1,989 EUR/l
+- **C** `IMG_20260929_182218993` — ground truth 18,86 l / 2,129 EUR/l
+
+The recipe (discovered manually in GIMP's GUI, applied to the full
+uncropped color photo):
+
+1. **Gaussian Blur** (Size X=Y=4.15, Filter=Auto, Abyss=Clamp) with
+   **Blending Options: Mode=Grain Extract, Opacity=100** set directly in
+   the filter dialog — GIMP 2.10's GEGL-based "High Pass" filter has no
+   Script-Fu/PDB wrapper at all, so this is a Script-Fu-reachable
+   reproduction of it via the classic manual technique (blur, then
+   grain-extract-blend against the original).
+2. **Levels**: input low=58, gamma=0.96, high=135; output 0–255.
+3. **NL Filter**, filter-type=2 (edge-enhancement), alpha=0.69,
+   radius=0.52.
+
+## Result
+
+Every variant beat the pipeline's baseline (no GIMP step) on at least one
+field, for all three images — confirming the general direction helps.
+But no single step generalized on its own; only the full 2-step recipe
+was consistently good-to-exact across all three:
+
+| Image (ground truth) | Baseline | 1. Blur+GrainExtract | 2. +Levels | 3. NL Filter alone | 4. Full recipe (2+3) |
+|---|---|---|---|---|---|
+| **A** (29,31l / 1,949) | both unreadable | price≈missing, litres✗ | litres 29,91✗(1 digit) / price **1,949✓** | litres 28,31✗(1 digit) / price 1,983≈ | litres **29,31✓** / price 1,993≈ |
+| **B** (17,88l / 1,989) | litres+price both wrong | litres **17,88✓** / price 1,909≈ | litres **17,88✓** / price **1,989✓** | litres✗missing / price≈1989 (garbled) | litres 17,B8≈(1 char) / price **1,989✓** |
+| **C** (18,86l / 2,129) | litres wrong, price✗missing | litres **18,86✓** / price✗missing | litres **0.86✗** (broke it!) / price≈2,129 | litres **18,86✓** / price✗missing | litres **18,86✓** / price≈2,129 |
+
+Note step 2 (Levels) actively *breaks* C's litres reading that was fine
+after step 1 alone — a clear example of a step helping one receipt while
+hurting another, which is why no single step is a safe generalization on
+its own even though each one looks promising in isolation.
+
+## Root causes / methodology notes
+
+1. **GIMP's "High Pass" filter has no PDB/Script-Fu wrapper** — confirmed
+   by querying the full procedure database (`gimp-procedural-db-query`)
+   for anything matching `high.?pass`; only `plug-in-edge`,
+   `plug-in-unsharp-mask`, `plug-in-sharpen`, and `gimp-selection-sharpen`
+   exist. It's GEGL-only in this GIMP version (2.10.30). The manual
+   Gaussian-Blur-with-Grain-Extract-blend technique is the only
+   automatable equivalent.
+2. **GIMP's own "default" vs "legacy" grain-extract blend-mode variants
+   turned out not to matter** — tested both (`LAYER-MODE-GRAIN-EXTRACT`
+   = 46 and `LAYER-MODE-GRAIN-EXTRACT-LEGACY` = 20 against the same
+   input): numerically near-identical output either way.
+3. **A real, unresolved gap remains between GIMP's GUI-applied
+   Gaussian-Blur-with-in-dialog-blend and a classic two-layer Script-Fu
+   reproduction** (duplicate layer → blur → `gimp-layer-set-mode` →
+   flatten) of the *same nominal parameters*. The GUI version's grain-
+   extract output has roughly double the standard deviation (spread) of
+   the scripted version's, confirmed via direct pixel-value percentile
+   comparison (user's real `_5_1.jpg`: mean 118.5, std 40.4; scripted
+   equivalent: mean 128.0, std 8.7–23.2 across a wide blur-radius sweep
+   from 8 to 180 — never converges, ruling out a simple radius/scale
+   miscalibration). `plug-in-gauss`'s legacy-compat wrapper applies an
+   internal ~0.32× multiplier converting its "radius" input to GEGL's
+   `std-dev`, and the Grain Extract formula itself was confirmed correct
+   from GEGL source (`CLAMP(lower - upper + 0.5)` in normalized space).
+   The actual cause (most likely GEGL's abyss-policy/edge-handling or an
+   image-precision difference) was not pinned down — **GIMP batch
+   scripting cannot currently reproduce this step's real-world output
+   closely enough to trust for pipeline automation.**
+4. Separately (fixed along the way, not the cause of the above): GIMP
+   batch-mode has two sharp edges worth remembering if scripted again —
+   (a) `gimp-console -b <script> -b '(gimp-quit 0)'` as two separate
+   `-b` flags does not reliably terminate the process on error or even
+   on success; always pipe one script via stdin (`-b -`) with
+   `(gimp-quit 0)` as the last form *inside* that same stream.
+   (b) `gimp-image-insert-layer`'s position argument needs an explicit
+   `0` (top of stack) — `-1` does not reliably mean "top" and silently
+   produces a no-op-looking flatten (new layer ends up under the base
+   layer, which then opaquely covers it).
+
+## Native Python reimplementation and full-batch result — rejected
+
+Reimplemented the two steps natively (no GIMP dependency) in
+`tooling-container/src/preprocess.py`, inserted before the existing
+bilateral+autocontrast stage:
+
+- `highpass_substitute()`: `cv2.GaussianBlur` + the confirmed
+  `clip(gray - blurred + 128)` grain-extract formula + a linear
+  levels-stretch (low=58, high=135, gamma=0.96 — the same values as the
+  GIMP recipe).
+- `edge_enhance()`: a calibrated unsharp-mask-style boost
+  (`gray + 0.69 * (gray - GaussianBlur(gray, sigma=0.52))`) as the
+  OpenCV substitute for GIMP's NL Filter edge-enhancement mode (no direct
+  OpenCV equivalent exists).
+- Both steps' blur radii scale with image width (`/ 768`, the width A/B
+  were shot at) — necessary because C is 4096×3072, roughly 4-5x A/B's
+  linear resolution, and a fixed pixel radius calibrated at one scale
+  clearly under/over-shoots at the other (confirmed: before scaling, C's
+  litres read `0.86` instead of `18,86`; after scaling, `18,86✓`).
+
+On the A/B/C set, this **outperformed the GIMP-based matrix** — litres
+exact on all three images, price exact-or-close on all three (no GIMP
+dependency, no reproduction-gap uncertainty).
+
+**But it failed full-batch validation** against the real 52-receipt
+ground-truth set (`testdata/2026_benzin_kosten/`), the required next step
+per this project's evidence-based philosophy before touching the real
+pipeline:
+
+| Metric | Baseline (before) | With new preprocessing | Change |
+|---|---|---|---|
+| Date-matched | 37/52 | 30/52 | **-7** |
+| price_per_litre | 27/37 (73%) | 18/30 (60%) | ↓ |
+| litres (the targeted field) | 19/37 (51%) | 12/30 (40%) | ↓ worse |
+| discount | 29/37 (78%) | 22/30 (73%) | ↓ |
+| total_price | 23/37 (62%) | 18/30 (60%) | ↓ |
+
+Regression across every metric, including the exact field the change
+targeted — the same failure pattern as the earlier-rejected unsharp-
+masking attempt (see `AGENT.md`): it visibly helps the specific hard
+receipts it was designed for, but amplifies the dotted security-
+background pattern into OCR noise on receipts that were already reading
+fine, making the batch worse overall. **Reverted** from
+`preprocess.py`/the numbered scripts/docs; not going into the pipeline.
+
+## Adaptive per-receipt escalation — accepted
+
+This is now the second distinct sharpening-family approach (after naive
+unsharp masking - see `AGENT.md`) confirmed to fail *when applied
+globally*: strong locally on hard receipts, net-negative on the full
+batch, because it amplifies the security background pattern into noise
+on receipts the cheap baseline already reads fine. Both failures point
+at the same fix: apply it *selectively*, only to receipts the cheap pass
+already reads poorly, rather than to every photo.
+
+Implemented as `tooling-container/src/preprocess.py`'s four variants
+(0=baseline, 1=+`highpass_substitute`, 2=+`edge_enhance`, 3=+both) tried
+cheapest-first by `tooling-container/src/adaptive.py`, escalating only
+when the current variant's raw OCR text fails a cheap arithmetic-
+consistency check (`tooling-container/src/plausibility.py`:
+`price_per_litre * litres - discount ≈ total_price` within 0.05 EUR —
+the same identity and tolerance `android/.../ReceiptPlausibility.kt` and
+`compare_ground_truth.py` already use, applied here via regex-extracted
+candidate numbers straight from OCR text so the check needs no LLM call
+and runs once per variant instead of once per receipt). This keeps the
+expensive LLM structuring call to exactly one per receipt regardless of
+how many variants are tried, since tier selection happens entirely
+before that call.
+
+Full-batch result against `testdata/2026_benzin_kosten/` (same baseline
+as above):
+
+| Metric | Baseline | Adaptive | Change |
+|---|---|---|---|
+| Date-matched | 37/52 | 40/52 | **+3** |
+| price_per_litre | 27/37 (73%) | 34/40 (85%) | **↑** |
+| litres | 19/37 (51%) | 24/40 (60%) | **↑** |
+| discount | 29/37 (78%) | 34/40 (85%) | **↑** |
+| total_price | 23/37 (62%) | 24/40 (60%) | ≈ flat (+1 in count) |
+
+Improves every metric except total_price, which is essentially flat.
+**Accepted** — wired into `server.py`, `test_batch.sh`, `test_single.sh`,
+and the new `12_adaptive_preprocess_ocr.sh` (replacing the separate
+`1_preprocess_image.sh` + `2_ocr_image.sh` calls those two use;
+`1_preprocess_image.sh`/`2_ocr_image.sh` themselves are unchanged and
+still useful for inspecting one variant in isolation).
+
+## Conclusion
+
+The litres/price-per-litre line was the pipeline's hardest OCR target
+specifically because every fix tried so far has been a global tradeoff:
+good for hard receipts, bad for easy ones, on the same underlying cause
+(amplifying the security background pattern). Making the fix conditional
+on the cheap pass already looking wrong - judged cheaply, without an
+LLM call - was what actually let the hard-receipt win through without
+the easy-receipt cost.