fuelish/README.md

fuelish

Extracts structured data from photos of German fuel/petrol receipts — petrol station, transaction number, date, time, price per litre, litres, discount, and total price — and lets you review, correct, save, and export it from an Android app.

The extraction pipeline is self-hosted (no cloud AI APIs): Tesseract OCR plus a small local LLM (qwen2.5:3b, served by Ollama), all running in containers. It's built to eventually run on a Raspberry Pi 4 cluster under k3s; see AGENT.md for the pipeline's architecture, preprocessing rationale, and current extraction-accuracy notes.

How it fits together

• Pipeline (tooling-container/): preprocess → OCR → LLM-structure a receipt photo into JSON. Runs either as three independently-inspectable host scripts (for development/testing) or as an HTTP job API (for the app).
• Server (docker-compose.yml): the same pipeline, served over HTTP (POST /jobs, GET /jobs/{id}) so the Android app can submit a photo and poll real per-stage progress, backed by an Ollama container.
• Android app (android/): take a photo, watch it get processed, correct any misread fields, save it locally, browse everything saved so far, and export to CSV or .ods (matching the layout of the ground-truth spreadsheets under testdata/).
• CI (.gitea/workflows/build.yml): builds the Android debug APK on every push and publishes it to dl.ladkau.de, using a custom build image (build-image/) so the runner needs nothing preinstalled.

Repository layout

tooling-container/     Pipeline source (preprocess.py, ocr.py, extract.py,
                        server.py, compare_ground_truth.py) and the
                        Dockerfile/VERSION for the fuelish/tools image
                        both the host scripts and docker-compose.yml run.
0-3_*.sh                Numbered host scripts driving one pipeline stage
                        each, for independent inspection during
                        development (0_run_ollama.sh starts Ollama).
test_batch.sh           Runs the full pipeline over a directory of test
                        receipts and compares against its ground-truth
                        .ods - see AGENT.md's "Current state".
docker-compose.yml      Runs the pipeline as a long-lived HTTP job API
                        (api + ollama services) for the Android app.
android/                The Android app: Kotlin + Jetpack Compose, Room
                        for local storage, OkHttp against the server API,
                        CSV/.ods export.
build-image/            Dockerfile for the Android CI/release build
                        environment (JDK + Android SDK, no NDK needed -
                        the app has no native code), used by CI and by
                        the build-image.sh/run-image.sh/upload-image.sh
                        scripts at the repo root (see "Building" below).
.gitea/workflows/       CI: builds the Android APK and publishes it to
                        dl.ladkau.de.
Makefile                Root Makefile - packages a versioned release APK
                        (`make android`).
testdata/               Real receipt photos + ground-truth .ods files
                        (gitignored, not checked in - personal data).

See AGENT.md for the pipeline's architecture, why the current preprocessing approach was chosen over heavier alternatives, and the latest batch-accuracy numbers and known failure modes.

Building

The pipeline image (tooling-container/), used by the numbered host scripts, test_batch.sh, and docker-compose.yml:

./build-image.sh   # builds fuelish/tools and the Android build-image
                    # (below), and pulls ollama/ollama

The server, once the tools image is built:

docker compose up -d   # starts ollama + api; api reachable at :8080

The Android app (android/) needs the Android SDK (ANDROID_HOME set). Locally, without installing it yourself:

cp registry.env.example registry.env   # fill in your registry credentials
./build-image.sh   # builds the android-builder image (JDK + Android SDK)
./run-image.sh     # runs `make android` inside it, against this
                    # checkout -> dist/fuelish-<version>.apk

With the SDK already installed, make android (or cd android && ./gradlew assembleDebug) works directly on the host.

CI builds the same way — .gitea/workflows/build.yml runs make android inside the build-image/ image (pulled from the registry, published via ./upload-image.sh after ./build-image.sh) and publishes the APK to dl.ladkau.de on every push to the default branch.

Running

Inspect the pipeline stage by stage, for development/debugging one receipt at a time:

./0_run_ollama.sh                                          # once, starts Ollama
./1_preprocess_image.sh -in photo.jpg -out scratch/
./2_ocr_image.sh -in scratch/03_final.png -out scratch/
./3_extract_data.sh -in scratch/ocr.txt -out scratch/       # -> extracted.json

Batch-test against ground truth:

./test_batch.sh -in testdata/<some-set>   # needs a *.ods ground-truth
                                           # file in that directory

Serve it for the app:

docker compose up -d

Then, in the Android app's Settings tab, point it at http://<this-host>:8080 (both HTTP-only - see the app's network_security_config.xml - and LAN-only by design, no auth/TLS). Take a photo, watch the 4-stage progress (upload → preprocess → OCR → extract), correct anything the model got wrong, and save. The Receipts tab lists everything saved so far and exports to CSV or .ods.

License

MIT — see LICENSE.