Add containerized OCR pipeline for fuel receipts
Self-hosted, verifiable pipeline for extracting structured data (station, date/time, price per litre, litres, discount, total) from German fuel receipt photos: grayscale + mild bilateral filter + contrast stretch, Tesseract OCR, then LLM structuring via a small local model (Qwen 2.5, served by Ollama). Each step is independently runnable and inspectable through numbered scripts against a shared tooling container. Ollama runs as its own long-lived container (0_run_ollama.sh), separate from the stateless per-receipt pipeline steps, so the model stays loaded in memory between runs. Includes a containerized ground-truth batch test (test_batch.sh) that reproduces the full pipeline over the test receipts and reports per-field accuracy. Targets eventual deployment on a Raspberry Pi 4 k3s cluster.
This commit is contained in:
@@ -0,0 +1,64 @@
|
||||
#!/usr/bin/env bash
|
||||
# Step 0: start the Ollama inference server that the pipeline's
|
||||
# structuring step (3_extract_data.sh) talks to. Runs the official
|
||||
# ollama/ollama image as a long-lived container with model weights kept
|
||||
# in a named Docker volume - deliberately separate from the
|
||||
# tooling-container (which is stateless and one-shot per receipt), so the
|
||||
# model stays loaded in memory between pipeline runs instead of being
|
||||
# reloaded on every receipt.
|
||||
#
|
||||
# Usage: ./0_run_ollama.sh [-model <name>]
|
||||
set -euo pipefail
|
||||
|
||||
cd "$(dirname "${BASH_SOURCE[0]}")"
|
||||
|
||||
fail() { echo "FAIL: $*" >&2; exit 1; }
|
||||
|
||||
command -v docker >/dev/null 2>&1 || fail "docker not found in PATH"
|
||||
|
||||
MODEL="qwen2.5:3b"
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
-model) MODEL="${2:-}"; shift 2 ;;
|
||||
*) fail "unknown argument: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
CONTAINER=gastronomy-ollama
|
||||
VOLUME=gastronomy-ollama-data
|
||||
PORT=11434
|
||||
|
||||
if docker ps --format '{{.Names}}' | grep -qx "$CONTAINER"; then
|
||||
echo "== $CONTAINER already running =="
|
||||
elif docker ps -a --format '{{.Names}}' | grep -qx "$CONTAINER"; then
|
||||
echo "== Starting existing $CONTAINER container =="
|
||||
docker start "$CONTAINER" >/dev/null
|
||||
else
|
||||
echo "== Creating $CONTAINER (model weights kept in volume $VOLUME) =="
|
||||
docker run -d \
|
||||
--name "$CONTAINER" \
|
||||
-p "127.0.0.1:$PORT:11434" \
|
||||
-v "$VOLUME:/root/.ollama" \
|
||||
--restart unless-stopped \
|
||||
ollama/ollama >/dev/null
|
||||
fi
|
||||
|
||||
echo "== Waiting for Ollama to become ready =="
|
||||
ready=0
|
||||
for _ in $(seq 1 30); do
|
||||
if curl -s --max-time 2 "http://127.0.0.1:$PORT/api/tags" >/dev/null 2>&1; then
|
||||
ready=1
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
[ "$ready" = "1" ] || fail "Ollama did not become ready on port $PORT"
|
||||
|
||||
if ! docker exec "$CONTAINER" ollama list | awk 'NR>1{print $1}' | grep -qx "$MODEL"; then
|
||||
echo "== Pulling model $MODEL =="
|
||||
docker exec "$CONTAINER" ollama pull "$MODEL"
|
||||
fi
|
||||
|
||||
echo "== Done =="
|
||||
echo "Ollama running at http://127.0.0.1:$PORT (model: $MODEL)"
|
||||
echo "Next: ./1_preprocess_image.sh -in <photo> -out <scratch-dir>"
|
||||
Reference in New Issue
Block a user