server_cloud_ladkau_de/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Purpose

Infrastructure-as-code for a Strato VPS at cloud.ladkau.de. Every config change must be committed — the git history is the change log. The goal is a fully reproducible reinstall from this repo alone.

Key commands

# Install Ansible collections (one-time, after cloning)
ansible-galaxy collection install -r ansible/requirements.yml

# Run the full playbook (most common operation)
ansible-playbook -i ansible/inventory.ini ansible/site.yml --ask-vault-pass

# Run a single role
ansible-playbook -i ansible/inventory.ini ansible/site.yml --tags mail --ask-vault-pass

# Bootstrap a fresh server (before Ansible can run — creates the deploy user)
bash scripts/run-bootstrap.sh

# Verify all vault secrets are present and valid
bash scripts/check-vault.sh

# Verify all services are reachable
bash scripts/check-services.sh

# Edit encrypted secrets
ansible-vault edit ansible/group_vars/all/vault.yml

# Connect to server
ssh -i keys/notroot_cloud_ladkau_de deploy@217.154.207.148

Local dependencies: pip install ansible passlib and openssl.

Architecture

Provisioning flow

1. scripts/run-bootstrap.sh — copies and runs bootstrap-deploy-user.sh on the server as root; creates the deploy user with the SSH public key from keys/notroot_cloud_ladkau_de.pub
2. ansible-playbook ansible/site.yml — runs all roles in order against the deploy user (passwordless sudo)

Host memory

The VPS has 8 GB RAM. The base role creates a swap file (/swapfile, base_swap_size, vm.swappiness = base_swappiness) so memory pressure slows the host instead of freezing it. The largest consumers are the Ollama model (about 4 GB while loaded; ollama_keep_alive controls how long it stays) and CI job containers (capped by act_runner_job_memory).

Role → service → data directory

Each role deploys one Docker Compose stack. The pattern is:

• Template lives in roles/<name>/templates/docker-compose.yml.j2
• Ansible renders it and writes it to /opt/<name>/docker-compose.yml on the server
• Data/config for each service lives under /opt/<name>/ on the server
• The role also writes any config files (e.g. dovecot.conf, passwd, fetchmailrc) to /opt/<name>/ before starting the stack

Networking

Two Docker networks:

• traefik_public (external, created once by the traefik role) — all services that need HTTPS exposure join this network alongside Traefik
• Per-service internal networks (e.g. mail_internal) — isolate backend containers (databases, Dovecot) from the internet

Traefik routes HTTP/HTTPS/IMAPS via Docker labels on each service container. TLS certificates are issued via Let's Encrypt ACME. IMAPS (993) is handled as a TCP passthrough — Traefik terminates TLS and forwards plain IMAP to Dovecot port 143.

Secrets

All secrets live in ansible/group_vars/all/vault.yml (Ansible Vault, AES-256). Plain variables are in ansible/group_vars/all/vars.yml — commented-out keys there show what each role expects from the vault. scripts/check-vault.sh validates all required keys are present.

Passwords that need bcrypt hashes (Traefik dashboard, container registry) are stored as plaintext in the vault and hashed in the Jinja2 templates at deploy time using password_hash('bcrypt', ...). The passlib Python package is required locally for this.

Mail stack specifics

The mail role deploys four containers on mail_internal: mail-db (PostgreSQL for Roundcube), dovecot (IMAP server), roundcube (webmail at mail.ladkau.de), and conditionally fetchmail (polls external POP3 accounts).

• Dovecot uses passwd-file auth: roles/mail/templates/passwd.j2 generates /opt/mail/passwd with 8-field format (user:hash::::::) — the trailing :::::: is required for userdb lookup
• Fetchmail delivers via LMTP to Dovecot port 24; the auth_username_format = %Ln in Dovecot's LMTP config strips the @dovecot domain that fetchmail appends
• Fetchmail 6.6.x rcfile syntax: ssl is a user-level option (inline after password), port is smtphost dovecot/24 (slash-separated), smtpport keyword does not exist
• mail.ladkau.de is the Roundcube webmail client, not an MTA — no outgoing SMTP server is configured

n8n (AI chat and scheduled digests)

n8n (agent.ladkau.de) runs the chat page and the scheduled workflows. The n8n role templates its compose file to /opt/n8n and imports the repo's workflows into it. Key details:

• Traefik basic auth (n8n-auth middleware) uses n8n_users from the vault; n8n has its own owner login behind it
• /chat is redirected by a Traefik redirectregex middleware to the hosted chat webhook /webhook/<n8n_chat_webhook_id>/chat; n8n_chat_webhook_id must match webhookId in roles/n8n/files/workflows/chat.json
• Workflows are roles/n8n/files/workflows/*.json; credentials are imported from roles/n8n/files/credentials/ollama.json and a template rendered from the vault (smtp_user, smtp_password). Import (credentials, workflows, publish "active": true ones, restart) runs as a handler only when those files change
• Imports replace workflows by id: edits made in the n8n UI are lost unless exported back into the repo
• Data and the SQLite database are in /opt/n8n/data; n8n_encryption_key protects stored credentials and must never change after the first deploy
• n8n joins ollama_internal and browserless_internal; neither backend has a public route
• The fetch_page tool and the digest workflow call browserless POST /function, which runs a small Puppeteer script and returns the page text
• The web_search tool (web-search.json) queries SearXNG's JSON API and returns the top 6 results
• n8n also joins searxng_internal
• Telegram: the Telegram chat workflow answers one Telegram user (telegram_user_id); other users and non-text messages are ignored
• Model routing (Route step): messages go to the Gemini chain (gemini_api_key from the vault); messages starting with /qwen go to the local Qwen agent. The Gemini chain is gemini-3.8-flash, then gemini-3.5-flash if the answer is empty, then Qwen. Free-tier Gemini limits are per model; the reply ends with a note when a fallback answered
• Model names in the workflow JSON are plain strings. __OLLAMA_MODEL__ is replaced by ollama_model and __TELEGRAM_USER_ID__ by telegram_user_id when Ansible copies the files; keep the placeholders when exporting workflows from the UI
• All agents have the web_search and fetch_page tools, 4 iterations at most, and the instruction to make one fetch_page call per web address or one web_search per question. The Gemini agents share one conversation memory; the Qwen agent has its own small memory
• Scheduled workflows (digest-telegram.json) post through the Post message entry point of Telegram chat, so the text lands in the chat and in its memory
• The Telegram chat workflow has a schedule (Stock price schedule, three cron expressions, weekdays) and a manual trigger. Both send /stocks through Route, which expands it to STOCK_QUESTION (names the two Yahoo quote pages); /stocks typed in Telegram does the same and /qwen /stocks sends it to Qwen
• Telegram's servers call n8n at /webhook/<n8n_telegram_webhook_id>/webhook. That route has no basic auth; Traefik only admits telegram_webhook_source_ranges and n8n checks Telegram's secret token header. n8n_telegram_webhook_id must match webhookId in telegram-chat.json
• Replies are converted from Markdown to Telegram HTML by Code nodes and sent with parse_mode HTML
• Chat memory (Window Buffer Memory) lives in n8n's process memory, per workflow, and is lost on every n8n restart (including every deploy that changes workflows)

n8n backend: SearXNG

The searxng role runs a metasearch engine (searxng/searxng) on the internal network searxng_internal (http://searxng:8080), used only by the n8n web_search tool. settings.yml is rendered from a template with searxng_secret_key from the vault; the JSON output format is enabled and the bot limiter is off because there is no public route. Some upstream engines (DuckDuckGo, Brave) answer with captchas or rate limits; SearXNG skips them and uses the others.

n8n backend: Browserless

The browserless role runs headless Chromium (ghcr.io/browserless/chromium) on the internal network browserless_internal (http://browserless:3000), one session at a time with a memory limit (browserless_mem_limit).

Fuelish (receipt OCR API)

The fuelish role runs the job API of the fuelish project (/home/ml/krotik/fuelish, image cr.ladkau.de/fuelish/tools) at fuelish.ladkau.de. Key details:

• The image is built and pushed from the fuelish project with ./build-image.sh and ./upload-image.sh; fuelish_version in vars.yml selects the tag. The role logs in to the registry with fuelish_registry_user / fuelish_registry_password (default: the first registry_users entry) and pulls on every run
• The version must be built from fuelish commit 7a91a37 or later: older images ignore FUELISH_API_KEY and accept every request
• fuelish_api_key (vault) is passed as FUELISH_API_KEY; the Android app sends it as the X-API-Key header. /health stays open
• The API uses the shared ollama container (ollama_internal) with fuelish_model (default ollama_model, the model n8n uses, so Ollama keeps a single model loaded; a different value is pulled by the API on first start). The role passes OLLAMA_NUM_CTX (fuelish_num_ctx) and OLLAMA_NUM_THREAD (fuelish_num_thread), equal to the n8n Ollama nodes (16384 / 4), because Ollama restarts its runner (about 7 s) when a request's context size or thread count differs from the loaded one. Images built before fuelish tooling-container/VERSION 0.1.1 ignore both variables
• Traefik routes fuelish.ladkau.de with TLS and the rate-limit-lax middleware; there is no basic auth because the app authenticates with the API key

Ollama (local LLM)

The ollama role runs ollama/ollama (CPU only) on the internal Docker network ollama_internal — no Traefik route, no public port. n8n and fuelish join that network (as external) and reach it at http://ollama:11434.

• Model, context length and memory limit come from ollama_* in vars.yml (default qwen3:4b-instruct-2507-q4_K_M, 16k context, 4.5 GB limit — the VPS has 8 GB shared with ~24 containers)
• Models persist in /opt/ollama/models; the role pulls ollama_model if it is missing
• ollama_keep_alive unloads the model from RAM when idle
• Ollama starts 2 threads on this 2-core/4-thread host; the Ollama chat model nodes in the workflows set numThread to 4
• Changing ollama_model and re-running --tags ollama pulls the new model; old models are removed manually with docker exec ollama ollama rm <model>

Gitea Actions runners

The act_runner role deploys three gitea/act_runner containers (Docker executor). Each runner handles one concurrent job. Key details:

• Runners connect to Gitea via the public HTTPS URL — no internal Docker network needed
• Each runner gets its own data dir (/opt/act_runner/runner-N/) for its .runner registration file
• All runners share a single config.yml at /opt/act_runner/config.yml
• /var/run/docker.sock is mounted — job containers are spawned directly on the host
• Job containers are limited to act_runner_job_memory of RAM with swap disabled (container.options in config.yml): a build that needs more is killed and the host stays up. Android builds use a Gradle heap of 2 GB (org.gradle.jvmargs) and need about 3–4 GB in total
• gitea_runner_registration_token is obtained from Gitea admin UI after Gitea is running; add it to the vault and then deploy with --tags act_runner

SSO

Keycloak at sso.ladkau.de is the identity provider. Gitea and Nextcloud are configured post-provisioning to use it (see docs/runbook-configuration.md).

Runbooks

• docs/runbook-provisioning.md — step-by-step reinstall from scratch
• docs/runbook-configuration.md — first-run service configuration (Keycloak realm, Gitea admin, Nextcloud OIDC, etc.)
• docs/runbook-mail-import.md — migrating mbox-format mail from an old Dovecot server to Maildir++