Add a custom Android CI/release build image and packaging scripts
build / build (push) Failing after 0s
build / build (push) Failing after 0s
- build-image/: Dockerfile bundling JDK 17, the Android SDK, and Node.js (needed by Gitea's JS-based actions) - the app has no native code, so no NDK - build-image.sh/upload-image.sh: extended to also build/push this image alongside the existing OCR pipeline tools image - run-image.sh: runs `make android` inside the image against a live checkout, for a CI-equivalent local build - Makefile: `make android` builds and versions a release APK into dist/, from the current git tag - .gitea/workflows/build.yml: now pulls the custom image and just runs `make android`, replacing the ad-hoc Node/SDK install steps - README.md: describes the pipeline, server, app, and build tooling - registry.env.example: fix REGISTRY_IMAGE to be fully-qualified (an unqualified name was silently resolving pushes to Docker Hub instead of the configured registry)
This commit is contained in:
ml
@@ -1,3 +1,139 @@
|
||||
<p align="center">
|
||||
<img src="logo.png" width="280" alt="fuelish logo">
|
||||
</p>
|
||||
|
||||
# fuelish
|
||||
|
||||
Extracting and organising fuel data from fuel receipts.
|
||||
Extracts structured data from photos of German fuel/petrol receipts —
|
||||
petrol station, transaction number, date, time, price per litre, litres,
|
||||
discount, and total price — and lets you review, correct, save, and
|
||||
export it from an Android app.
|
||||
|
||||
The extraction pipeline is self-hosted (no cloud AI APIs): Tesseract OCR
|
||||
plus a small local LLM (`qwen2.5:3b`, served by Ollama), all running in
|
||||
containers. It's built to eventually run on a Raspberry Pi 4 cluster
|
||||
under k3s; see [`AGENT.md`](AGENT.md) for the pipeline's architecture,
|
||||
preprocessing rationale, and current extraction-accuracy notes.
|
||||
|
||||
## How it fits together
|
||||
|
||||
- **Pipeline** (`tooling-container/`): preprocess → OCR → LLM-structure a
|
||||
receipt photo into JSON. Runs either as three independently-inspectable
|
||||
host scripts (for development/testing) or as an HTTP job API (for the
|
||||
app).
|
||||
- **Server** (`docker-compose.yml`): the same pipeline, served over HTTP
|
||||
(`POST /jobs`, `GET /jobs/{id}`) so the Android app can submit a photo
|
||||
and poll real per-stage progress, backed by an Ollama container.
|
||||
- **Android app** (`android/`): take a photo, watch it get processed,
|
||||
correct any misread fields, save it locally, browse everything saved
|
||||
so far, and export to CSV or `.ods` (matching the layout of the
|
||||
ground-truth spreadsheets under `testdata/`).
|
||||
- **CI** (`.gitea/workflows/build.yml`): builds the Android debug APK on
|
||||
every push and publishes it to dl.ladkau.de, using a custom build image
|
||||
(`build-image/`) so the runner needs nothing preinstalled.
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
tooling-container/ Pipeline source (preprocess.py, ocr.py, extract.py,
|
||||
server.py, compare_ground_truth.py) and the
|
||||
Dockerfile/VERSION for the fuelish/tools image
|
||||
both the host scripts and docker-compose.yml run.
|
||||
0-3_*.sh Numbered host scripts driving one pipeline stage
|
||||
each, for independent inspection during
|
||||
development (0_run_ollama.sh starts Ollama).
|
||||
test_batch.sh Runs the full pipeline over a directory of test
|
||||
receipts and compares against its ground-truth
|
||||
.ods - see AGENT.md's "Current state".
|
||||
docker-compose.yml Runs the pipeline as a long-lived HTTP job API
|
||||
(api + ollama services) for the Android app.
|
||||
android/ The Android app: Kotlin + Jetpack Compose, Room
|
||||
for local storage, OkHttp against the server API,
|
||||
CSV/.ods export.
|
||||
build-image/ Dockerfile for the Android CI/release build
|
||||
environment (JDK + Android SDK, no NDK needed -
|
||||
the app has no native code), used by CI and by
|
||||
the build-image.sh/run-image.sh/upload-image.sh
|
||||
scripts at the repo root (see "Building" below).
|
||||
.gitea/workflows/ CI: builds the Android APK and publishes it to
|
||||
dl.ladkau.de.
|
||||
Makefile Root Makefile - packages a versioned release APK
|
||||
(`make android`).
|
||||
testdata/ Real receipt photos + ground-truth .ods files
|
||||
(gitignored, not checked in - personal data).
|
||||
```
|
||||
|
||||
See [`AGENT.md`](AGENT.md) for the pipeline's architecture, why the
|
||||
current preprocessing approach was chosen over heavier alternatives, and
|
||||
the latest batch-accuracy numbers and known failure modes.
|
||||
|
||||
## Building
|
||||
|
||||
**The pipeline image** (`tooling-container/`), used by the numbered host
|
||||
scripts, `test_batch.sh`, and `docker-compose.yml`:
|
||||
|
||||
```bash
|
||||
./build-image.sh # builds fuelish/tools and the Android build-image
|
||||
# (below), and pulls ollama/ollama
|
||||
```
|
||||
|
||||
**The server**, once the tools image is built:
|
||||
|
||||
```bash
|
||||
docker compose up -d # starts ollama + api; api reachable at :8080
|
||||
```
|
||||
|
||||
**The Android app** (`android/`) needs the Android SDK
|
||||
(`ANDROID_HOME` set). Locally, without installing it yourself:
|
||||
|
||||
```bash
|
||||
cp registry.env.example registry.env # fill in your registry credentials
|
||||
./build-image.sh # builds the android-builder image (JDK + Android SDK)
|
||||
./run-image.sh # runs `make android` inside it, against this
|
||||
# checkout -> dist/fuelish-<version>.apk
|
||||
```
|
||||
|
||||
With the SDK already installed, `make android` (or
|
||||
`cd android && ./gradlew assembleDebug`) works directly on the host.
|
||||
|
||||
**CI** builds the same way — `.gitea/workflows/build.yml` runs
|
||||
`make android` inside the `build-image/` image (pulled from the
|
||||
registry, published via `./upload-image.sh` after `./build-image.sh`)
|
||||
and publishes the APK to dl.ladkau.de on every push to the default
|
||||
branch.
|
||||
|
||||
## Running
|
||||
|
||||
**Inspect the pipeline stage by stage**, for development/debugging one
|
||||
receipt at a time:
|
||||
|
||||
```bash
|
||||
./0_run_ollama.sh # once, starts Ollama
|
||||
./1_preprocess_image.sh -in photo.jpg -out scratch/
|
||||
./2_ocr_image.sh -in scratch/03_final.png -out scratch/
|
||||
./3_extract_data.sh -in scratch/ocr.txt -out scratch/ # -> extracted.json
|
||||
```
|
||||
|
||||
**Batch-test against ground truth**:
|
||||
|
||||
```bash
|
||||
./test_batch.sh -in testdata/<some-set> # needs a *.ods ground-truth
|
||||
# file in that directory
|
||||
```
|
||||
|
||||
**Serve it for the app**:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Then, in the Android app's Settings tab, point it at
|
||||
`http://<this-host>:8080` (both HTTP-only - see the app's
|
||||
`network_security_config.xml` - and LAN-only by design, no auth/TLS).
|
||||
Take a photo, watch the 4-stage progress (upload → preprocess → OCR →
|
||||
extract), correct anything the model got wrong, and save. The Receipts
|
||||
tab lists everything saved so far and exports to CSV or `.ods`.
|
||||
|
||||
## License
|
||||
|
||||
MIT — see [`LICENSE`](LICENSE).
|
||||
|
||||
Reference in New Issue
Block a user