Rename project from gastronomy to fuelish
Renamed all image tags, container/volume names, and doc references.
This commit is contained in:
+2
-2
@@ -24,8 +24,8 @@ while [ $# -gt 0 ]; do
|
|||||||
esac
|
esac
|
||||||
done
|
done
|
||||||
|
|
||||||
CONTAINER=gastronomy-ollama
|
CONTAINER=fuelish-ollama
|
||||||
VOLUME=gastronomy-ollama-data
|
VOLUME=fuelish-ollama-data
|
||||||
PORT=11434
|
PORT=11434
|
||||||
|
|
||||||
if docker ps --format '{{.Names}}' | grep -qx "$CONTAINER"; then
|
if docker ps --format '{{.Names}}' | grep -qx "$CONTAINER"; then
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ done
|
|||||||
|
|
||||||
command -v docker >/dev/null 2>&1 || fail "docker not found in PATH"
|
command -v docker >/dev/null 2>&1 || fail "docker not found in PATH"
|
||||||
|
|
||||||
IMAGE="${REGISTRY_IMAGE:-gastronomy/tools}"
|
IMAGE="${REGISTRY_IMAGE:-fuelish/tools}"
|
||||||
if [ -f "$ROOT/registry.env" ]; then
|
if [ -f "$ROOT/registry.env" ]; then
|
||||||
# shellcheck disable=SC1091
|
# shellcheck disable=SC1091
|
||||||
source "$ROOT/registry.env"
|
source "$ROOT/registry.env"
|
||||||
|
|||||||
+1
-1
@@ -26,7 +26,7 @@ done
|
|||||||
|
|
||||||
command -v docker >/dev/null 2>&1 || fail "docker not found in PATH"
|
command -v docker >/dev/null 2>&1 || fail "docker not found in PATH"
|
||||||
|
|
||||||
IMAGE="${REGISTRY_IMAGE:-gastronomy/tools}"
|
IMAGE="${REGISTRY_IMAGE:-fuelish/tools}"
|
||||||
if [ -f "$ROOT/registry.env" ]; then
|
if [ -f "$ROOT/registry.env" ]; then
|
||||||
# shellcheck disable=SC1091
|
# shellcheck disable=SC1091
|
||||||
source "$ROOT/registry.env"
|
source "$ROOT/registry.env"
|
||||||
|
|||||||
+1
-1
@@ -28,7 +28,7 @@ done
|
|||||||
|
|
||||||
command -v docker >/dev/null 2>&1 || fail "docker not found in PATH"
|
command -v docker >/dev/null 2>&1 || fail "docker not found in PATH"
|
||||||
|
|
||||||
IMAGE="${REGISTRY_IMAGE:-gastronomy/tools}"
|
IMAGE="${REGISTRY_IMAGE:-fuelish/tools}"
|
||||||
if [ -f "$ROOT/registry.env" ]; then
|
if [ -f "$ROOT/registry.env" ]; then
|
||||||
# shellcheck disable=SC1091
|
# shellcheck disable=SC1091
|
||||||
source "$ROOT/registry.env"
|
source "$ROOT/registry.env"
|
||||||
|
|||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# fuelish
|
||||||
|
|
||||||
|
Extracts structured data from photos of German fuel/petrol receipts:
|
||||||
|
petrol station, transaction number, date, time, price per litre, litres,
|
||||||
|
discount, and total price.
|
||||||
|
|
||||||
|
The extraction pipeline is self-hosted (no cloud AI APIs) and is built to
|
||||||
|
eventually run on a Raspberry Pi 4 cluster under k3s. Everything runs in
|
||||||
|
containers; there is no host Python environment to set up.
|
||||||
|
|
||||||
|
## Pipeline
|
||||||
|
|
||||||
|
Three verifiable, independently-inspectable steps, each a thin host
|
||||||
|
script that runs the actual work inside the `fuelish/tools` container:
|
||||||
|
|
||||||
|
1. `1_preprocess_image.sh -in <photo> -out <dir>` - grayscale conversion,
|
||||||
|
a mild bilateral filter, and a percentile contrast stretch. Writes
|
||||||
|
every intermediate stage (`01_gray.png`, `02_bilateral.png`,
|
||||||
|
`03_final.png`) so each step can be checked on its own.
|
||||||
|
2. `2_ocr_image.sh -in <dir>/03_final.png -out <dir>` - Tesseract OCR
|
||||||
|
(German), writes `ocr.txt`.
|
||||||
|
3. `3_extract_data.sh -in <dir>/ocr.txt -out <dir>` - sends the raw OCR
|
||||||
|
text to a small local LLM (`qwen2.5:3b`, served by Ollama) with a JSON
|
||||||
|
schema, writes `extracted.json`.
|
||||||
|
|
||||||
|
Preprocessing choice is evidence-based, not assumed: heavier processing
|
||||||
|
(strong denoising, CLAHE, unsharp masking) was tried and measured worse -
|
||||||
|
it amplifies the receipt's dotted security-background pattern into OCR
|
||||||
|
noise. The lighter bilateral+autocontrast approach in
|
||||||
|
`tooling-container/src/preprocess.py` is what survived testing.
|
||||||
|
|
||||||
|
## Supporting scripts
|
||||||
|
|
||||||
|
- `0_run_ollama.sh [-model <name>]` - starts Ollama as its own long-lived
|
||||||
|
Docker container (`fuelish-ollama`, model weights in the named
|
||||||
|
volume `fuelish-ollama-data`), pulling the model if not already
|
||||||
|
present. Deliberately separate from the tooling container: Ollama needs
|
||||||
|
to stay resident in memory with the model loaded, while the pipeline
|
||||||
|
steps are short-lived and stateless (one `docker run` per receipt).
|
||||||
|
`3_extract_data.sh` reaches it via `--network host` at
|
||||||
|
`http://127.0.0.1:11434` by default (override with `OLLAMA_URL`/
|
||||||
|
`OLLAMA_MODEL`).
|
||||||
|
- `build-image.sh` / `upload-image.sh` - build/push the
|
||||||
|
`tooling-container/` image, tagged from `tooling-container/VERSION`.
|
||||||
|
Image name defaults to `fuelish/tools`, overridable via
|
||||||
|
`registry.env` (copy from `registry.env.example`; gitignored).
|
||||||
|
- `test_batch.sh -in <test-data-dir>` - runs the full pipeline over every
|
||||||
|
`*.jpg` in a directory and compares results against that directory's
|
||||||
|
ground-truth spreadsheet (`*.ods`). For an input dir `<path>/<name>` it
|
||||||
|
creates sibling `<path>/<name>-scratch/` (intermediate step output per
|
||||||
|
receipt) and `<path>/<name>-result/` (final `extracted.json` per
|
||||||
|
receipt, plus `summary.txt`). Both are gitignored.
|
||||||
|
|
||||||
|
## Current state
|
||||||
|
|
||||||
|
Last full run against the 17-receipt ground-truth test set
|
||||||
|
(`testdata/2026_new/`, not checked in - see `.gitignore`):
|
||||||
|
|
||||||
|
- 15/17 receipts date-matched to ground truth (up from 11/17 with the
|
||||||
|
original plain-grayscale preprocessing)
|
||||||
|
- price_per_litre: 11/15 correct, litres: 9/15, discount: 11/15,
|
||||||
|
total_price: 12/15
|
||||||
|
|
||||||
|
Known remaining failure modes, not expected to be fixable by further
|
||||||
|
preprocessing tuning:
|
||||||
|
|
||||||
|
- OCR garbling on the compact "litres EUR/l price" line on some receipts
|
||||||
|
(the raw text sometimes lacks recoverable digits)
|
||||||
|
- Occasional year-digit misreads in the date causing a false date-match
|
||||||
|
failure
|
||||||
|
- Low-resolution source photos (well below the ~4000x3000 the good
|
||||||
|
results come from) OCR poorly regardless of preprocessing
|
||||||
|
|
||||||
|
The structuring prompt in `tooling-container/src/extract.py` uses
|
||||||
|
deliberately non-colliding example values. An earlier version used real
|
||||||
|
receipt values as worked examples, which caused the small model to
|
||||||
|
copy/anchor on those exact numbers instead of reading the actual OCR
|
||||||
|
text - a subtle failure mode worth remembering if the prompt is revised.
|
||||||
|
|
||||||
|
## Not yet built
|
||||||
|
|
||||||
|
Per the original idea (see git history / project notes): an Android app
|
||||||
|
to capture and confirm receipts, a backend server to store raw + confirmed
|
||||||
|
submissions, a simple authenticated web UI to query/filter/export
|
||||||
|
(CSV/JSON), and cross-checking total_price against
|
||||||
|
price_per_litre * litres - discount. None of that exists yet - this repo
|
||||||
|
currently covers only the extraction pipeline.
|
||||||
@@ -1,3 +1,3 @@
|
|||||||
# gastronomy
|
# fuelish
|
||||||
|
|
||||||
Extracting and organising fuel data from fuel receipts.
|
Extracting and organising fuel data from fuel receipts.
|
||||||
+2
-2
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Builds the gastronomy tools image (tooling-container/Dockerfile) locally, tagged with
|
# Builds the fuelish tools image (tooling-container/Dockerfile) locally, tagged with
|
||||||
# tooling-container/VERSION and :latest. Does not push - test it with the numbered
|
# tooling-container/VERSION and :latest. Does not push - test it with the numbered
|
||||||
# pipeline scripts first, then publish with ./upload-image.sh.
|
# pipeline scripts first, then publish with ./upload-image.sh.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
@@ -12,7 +12,7 @@ fail() { echo "PREFLIGHT FAIL: $*" >&2; exit 1; }
|
|||||||
command -v docker >/dev/null 2>&1 \
|
command -v docker >/dev/null 2>&1 \
|
||||||
|| fail "docker not found in PATH"
|
|| fail "docker not found in PATH"
|
||||||
|
|
||||||
IMAGE="${REGISTRY_IMAGE:-gastronomy/tools}"
|
IMAGE="${REGISTRY_IMAGE:-fuelish/tools}"
|
||||||
if [ -f "$ROOT/registry.env" ]; then
|
if [ -f "$ROOT/registry.env" ]; then
|
||||||
# shellcheck disable=SC1091
|
# shellcheck disable=SC1091
|
||||||
source "$ROOT/registry.env"
|
source "$ROOT/registry.env"
|
||||||
|
|||||||
@@ -4,6 +4,6 @@
|
|||||||
# local builds/runs; REGISTRY/REGISTRY_USER/REGISTRY_PASSWORD are only
|
# local builds/runs; REGISTRY/REGISTRY_USER/REGISTRY_PASSWORD are only
|
||||||
# needed to push with upload-image.sh.
|
# needed to push with upload-image.sh.
|
||||||
REGISTRY=
|
REGISTRY=
|
||||||
REGISTRY_IMAGE=gastronomy/tools
|
REGISTRY_IMAGE=fuelish/tools
|
||||||
REGISTRY_USER=
|
REGISTRY_USER=
|
||||||
REGISTRY_PASSWORD=
|
REGISTRY_PASSWORD=
|
||||||
|
|||||||
+1
-1
@@ -64,7 +64,7 @@ done
|
|||||||
echo
|
echo
|
||||||
echo "== Comparing against ground truth =="
|
echo "== Comparing against ground truth =="
|
||||||
|
|
||||||
IMAGE="${REGISTRY_IMAGE:-gastronomy/tools}"
|
IMAGE="${REGISTRY_IMAGE:-fuelish/tools}"
|
||||||
if [ -f "$ROOT/registry.env" ]; then
|
if [ -f "$ROOT/registry.env" ]; then
|
||||||
# shellcheck disable=SC1091
|
# shellcheck disable=SC1091
|
||||||
source "$ROOT/registry.env"
|
source "$ROOT/registry.env"
|
||||||
|
|||||||
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Pushes the gastronomy tools image - already built locally with
|
# Pushes the fuelish tools image - already built locally with
|
||||||
# ./build-image.sh, and ideally verified with the numbered pipeline
|
# ./build-image.sh, and ideally verified with the numbered pipeline
|
||||||
# scripts - to a container registry, so the k3s cluster (or other
|
# scripts - to a container registry, so the k3s cluster (or other
|
||||||
# machines) can pull it without rebuilding it.
|
# machines) can pull it without rebuilding it.
|
||||||
|
|||||||
Reference in New Issue
Block a user