Add adaptive preprocessing escalation and single-receipt test scripts
build / build (push) Successful in 2m57s

Preprocessing now tries progressively heavier variants per receipt
(plain -> high-pass -> edge-enhance -> both), stopping as soon as a
cheap LLM-free arithmetic check on the OCR text looks plausible, so
the extra processing only applies to receipts that need it. A flat
version of the same heavier processing regressed the full batch by
helping hard receipts but hurting easy ones; the adaptive version
improves date-matching 37/52 -> 40/52 with no regression (see
testing/2026_10_04_gimp_highpass_nlfilter_experiment.md).

Also adds test_ocr.sh and test_single.sh for checking a single
receipt's OCR text or full pipeline result without a batch run.
This commit is contained in:
ml
2026-10-04 10:25:41 +02:00
parent 9bb1844cf4
commit ca85530b5a
11 changed files with 694 additions and 52 deletions
+11 -9
View File
@@ -1,17 +1,20 @@
#!/usr/bin/env bash
# Test-only tool: runs the full container pipeline (1_preprocess_image.sh,
# 2_ocr_image.sh, 3_extract_data.sh) over every *.jpg in a test-data
# directory, then compares the results against that directory's
# ground-truth spreadsheet (benzin_kosten.ods). Everything -
# Test-only tool: runs the full container pipeline
# (12_adaptive_preprocess_ocr.sh, 3_extract_data.sh) over every *.jpg in
# a test-data directory, then compares the results against that
# directory's ground-truth spreadsheet (benzin_kosten.ods). Everything -
# preprocessing, OCR, structuring, and the comparison itself - runs
# inside the tooling container; nothing here depends on a host Python
# environment.
#
# For an input directory <path>/<name>, two sibling directories are
# created (mirroring <path>/<name>):
# <path>/<name>-scratch/<receipt>/ - intermediate step output
# (01_gray.png, 02_bilateral.png,
# 03_final.png, ocr.txt) for
# <path>/<name>-scratch/<receipt>/ - intermediate step output: one
# variant<N>/ subdir per
# preprocessing variant tried
# (01_gray.png, ..., 0X_final.png,
# ocr.txt each) plus ocr.txt for
# the winning variant, for
# inspecting each step on its own
# <path>/<name>-result/<receipt>/ - final extracted.json per receipt,
# plus summary.txt with the overall
@@ -56,8 +59,7 @@ for img in "${IMAGES[@]}"; do
receipt_result="$RESULT_DIR/$name"
mkdir -p "$receipt_scratch" "$receipt_result"
echo "--- $name ---"
./1_preprocess_image.sh -in "$img" -out "$receipt_scratch" >/dev/null
./2_ocr_image.sh -in "$receipt_scratch/03_final.png" -out "$receipt_scratch" >/dev/null
./12_adaptive_preprocess_ocr.sh -in "$img" -out "$receipt_scratch" >/dev/null
./3_extract_data.sh -in "$receipt_scratch/ocr.txt" -out "$receipt_result" >/dev/null
done