fuelish/README.md

fuelish

Extracts structured data from photos of German fuel/petrol receipts — petrol station, transaction number, date, time, price per litre, litres, discount, and total price — and lets you review, correct, save, and export it from an Android app.

The extraction pipeline is self-hosted (no cloud AI APIs): Tesseract OCR plus a small local LLM (qwen2.5:3b, served by Ollama), all running in containers. It's built to eventually run on a Raspberry Pi 4 cluster under k3s; see AGENT.md for the pipeline's architecture, preprocessing rationale, and current extraction-accuracy notes.

How it fits together

• Pipeline (tooling-container/): preprocess → OCR → LLM-structure a receipt photo into JSON. Runs either as three independently-inspectable host scripts (for development/testing) or as an HTTP job API (for the app).
• Server (docker-compose.yml): the same pipeline, served over HTTP (POST /jobs, GET /jobs/{id}) so the Android app can submit a photo and poll real per-stage progress, backed by an Ollama container.
• Android app (android/): take a photo, watch it get processed, correct any misread fields, save it locally, browse everything saved so far, and export to CSV or .ods (matching the layout of the ground-truth spreadsheets under testdata/).
• CI (.gitea/workflows/build.yml): builds the Android debug APK on every push and publishes it to dl.ladkau.de, using a custom build image (build-image/) so the runner needs nothing preinstalled.

Repository layout

tooling-container/     Pipeline source (preprocess.py, ocr.py, extract.py,
                        server.py, compare_ground_truth.py) and the
                        Dockerfile/VERSION for the fuelish/tools image
                        both the host scripts and docker-compose.yml run.
0-3_*.sh                Numbered host scripts driving one pipeline stage
                        each, for independent inspection during
                        development (0_run_ollama.sh starts Ollama).
test_batch.sh           Runs the full pipeline over a directory of test
                        receipts and compares against its ground-truth
                        .ods - see AGENT.md's "Current state".
test_single.sh          test_batch.sh for one receipt instead of a
                        directory - re-check a single known failure.
test_ocr.sh             Preprocessing + OCR only, prints the raw OCR
                        text to stdout - no files left behind.
docker-compose.yml      Runs the pipeline as a long-lived HTTP job API
                        (api + ollama services) for the Android app.
android/                The Android app: Kotlin + Jetpack Compose, Room
                        for local storage, OkHttp against the server API,
                        CSV/.ods export.
build-image/            Dockerfile for the Android CI/release build
                        environment (JDK + Android SDK, no NDK needed -
                        the app has no native code), used by CI and by
                        the build-image.sh/run-image.sh/upload-image.sh
                        scripts at the repo root (see "Building" below).
.gitea/workflows/       CI: builds the Android APK and publishes it to
                        dl.ladkau.de.
Makefile                Root Makefile - packages a versioned release APK
                        (`make android`).
testdata/               Real receipt photos + ground-truth .ods files
                        (gitignored, not checked in - personal data).

See AGENT.md for the pipeline's architecture, why the current preprocessing approach was chosen over heavier alternatives, and the latest batch-accuracy numbers and known failure modes.

Building

The pipeline image (tooling-container/), used by the numbered host scripts, test_batch.sh, and docker-compose.yml:

./build-image.sh   # builds fuelish/tools and the Android build-image
                    # (below), and pulls ollama/ollama

The server, once the tools image is built:

docker compose up -d   # starts ollama + api; api reachable at :8080

The Android app (android/) needs the Android SDK (ANDROID_HOME set). Locally, without installing it yourself:

cp registry.env.example registry.env   # fill in your registry credentials
./build-image.sh   # builds the android-builder image (JDK + Android SDK)
./run-image.sh     # runs `make android` inside it, against this
                    # checkout -> dist/fuelish-<version>.apk

With the SDK already installed, make android (or cd android && ./gradlew assembleDebug) works directly on the host.

CI builds the same way — .gitea/workflows/build.yml runs make android inside the build-image/ image (pulled from the registry, published via ./upload-image.sh after ./build-image.sh) and publishes the APK to dl.ladkau.de on every push to the default branch.

CI needs its own copy of credentials, separate from your local registry.env (which only build-image.sh/upload-image.sh/ run-image.sh on your machine read - Gitea Actions never sees it). Add these as repo Settings → Actions → Secrets:

• REGISTRY_USER / REGISTRY_PASSWORD - to pull the private build-image/ image (same values as registry.env)
• DL_SFTP_KEY - the private key for the SFTP publish step to dl.ladkau.de

Without them, the workflow fails early (pulling the build image, or publishing) rather than partway through the build.

Running

Inspect the pipeline stage by stage, for development/debugging one receipt at a time:

./0_run_ollama.sh                                          # once, starts Ollama
./1_preprocess_image.sh -in photo.jpg -out scratch/
./2_ocr_image.sh -in scratch/03_final.png -out scratch/
./3_extract_data.sh -in scratch/ocr.txt -out scratch/       # -> extracted.json

Batch-test against ground truth:

./test_batch.sh -in testdata/<some-set>   # needs a *.ods ground-truth
                                           # file in that directory

Re-check one receipt (ground truth optional - compared if a *.ods is found next to the photo):

./test_single.sh -in testdata/<some-set>/some-receipt.jpg

Just see what Tesseract reads, no LLM call, no files left behind:

./test_ocr.sh -in testdata/<some-set>/some-receipt.jpg

Serve it for the app:

docker compose up -d

Then, in the Android app's Settings tab, point it at http://<this-host>:8080 (both HTTP-only - see the app's network_security_config.xml - and LAN-only by design, no auth/TLS). Take a photo, watch the 4-stage progress (upload → preprocess → OCR → extract), correct anything the model got wrong, and save. The Receipts tab lists everything saved so far and exports to CSV or .ods.

License

MIT — see LICENSE.