Replace Clay with n8n, Ollama, SearXNG and browserless; add Telegram assistant

Remove the Clay role (Claude Code web UI) and its Anthropic API key.

New roles:
- ollama: local CPU LLM (qwen3:4b-instruct), internal network only
- browserless: headless Chromium for JavaScript-heavy pages
- searxng: metasearch backend for the chat web_search tool
- n8n at agent.ladkau.de: web chat (/chat), scheduled workflows,
  Telegram bot; workflows and credentials are imported from the repo

Telegram bot: Gemini 3.8 -> Gemini 3.5 -> local Qwen fallback chain,
/qwen to force the local model, /stocks and a weekday schedule
(07:30, 13:00, 17:40) for the MTE.DE / PTX.F stock report.

Also: dashboard and check-services entries for the chat, vault checks
for the new secrets (n8n, SearXNG, Gemini, Telegram, SMTP), updated
CLAUDE.md, runbooks and README.
This commit is contained in:
ml
2026-10-03 17:10:33 +02:00
parent b3cf4e250a
commit 974486bc53
37 changed files with 2805 additions and 290 deletions
+38 -9
View File
@@ -75,17 +75,46 @@ The mail role deploys four containers on `mail_internal`: `mail-db` (PostgreSQL
- Fetchmail 6.6.x rcfile syntax: `ssl` is a user-level option (inline after `password`), port is `smtphost dovecot/24` (slash-separated), `smtpport` keyword does not exist
- `mail.ladkau.de` is the Roundcube webmail client, not an MTA — no outgoing SMTP server is configured
### Clay (Claude Code web interface)
### n8n (AI chat and scheduled digests)
Clay (`agent.ladkau.de`) provides a browser-based UI for Claude Code. The role
builds a Docker image from `ansible/roles/clay/files/Dockerfile` (Node 20 +
`@anthropic-ai/claude-code` + `clay-server`) at deploy time. Key details:
n8n (`agent.ladkau.de`) runs the chat page and the scheduled workflows. The `n8n` role templates its compose file to `/opt/n8n` and imports the repo's workflows into it. Key details:
- `ANTHROPIC_API_KEY` is passed as an environment variable from the vault
- Sessions and config are persisted at `/opt/clay/data` → `/root/.clay` in container
- Projects (git repos) are mounted from `/opt/clay/workspace` → `/workspace`
- Traefik basic auth (`clay-auth` middleware) uses `clay_users` from the vault
- Scheduled agent workflows use Gitea Actions `on.schedule` + the existing act_runners
- Traefik basic auth (`n8n-auth` middleware) uses `n8n_users` from the vault; n8n has its own owner login behind it
- `/chat` is redirected by a Traefik `redirectregex` middleware to the hosted chat webhook `/webhook/<n8n_chat_webhook_id>/chat`; `n8n_chat_webhook_id` must match `webhookId` in `roles/n8n/files/workflows/chat.json`
- Workflows are `roles/n8n/files/workflows/*.json`; credentials are imported from `roles/n8n/files/credentials/ollama.json` and a template rendered from the vault (`smtp_user`, `smtp_password`). Import (credentials, workflows, publish `"active": true` ones, restart) runs as a handler only when those files change
- Imports replace workflows by id: edits made in the n8n UI are lost unless exported back into the repo
- Data and the SQLite database are in `/opt/n8n/data`; `n8n_encryption_key` protects stored credentials and must never change after the first deploy
- n8n joins `ollama_internal` and `browserless_internal`; neither backend has a public route
- The `fetch_page` tool and the digest workflow call browserless `POST /function`, which runs a small Puppeteer script and returns the page text
- The `web_search` tool (`web-search.json`) queries SearXNG's JSON API and returns the top 6 results
- n8n also joins `searxng_internal`
- Telegram: the `Telegram chat` workflow answers one Telegram user (`telegram_user_id`); other users and non-text messages are ignored
- Model routing (`Route` step): messages go to the Gemini chain (`gemini_api_key` from the vault); messages starting with `/qwen` go to the local Qwen agent. The Gemini chain is `gemini-3.8-flash`, then `gemini-3.5-flash` if the answer is empty, then Qwen. Free-tier Gemini limits are per model; the reply ends with a note when a fallback answered
- Model names in the workflow JSON are plain strings. `__OLLAMA_MODEL__` is replaced by `ollama_model` and `__TELEGRAM_USER_ID__` by `telegram_user_id` when Ansible copies the files; keep the placeholders when exporting workflows from the UI
- All agents have the `web_search` and `fetch_page` tools, 4 iterations at most, and the instruction to make one `fetch_page` call per web address or one `web_search` per question. The Gemini agents share one conversation memory; the Qwen agent has its own small memory
- Scheduled workflows (`digest-telegram.json`) post through the `Post message` entry point of `Telegram chat`, so the text lands in the chat and in its memory
- The `Telegram chat` workflow has a schedule (`Stock price schedule`, three cron expressions, weekdays) and a manual trigger. Both send `/stocks` through `Route`, which expands it to `STOCK_QUESTION` (names the two Yahoo quote pages); `/stocks` typed in Telegram does the same and `/qwen /stocks` sends it to Qwen
- Telegram's servers call n8n at `/webhook/<n8n_telegram_webhook_id>/webhook`. That route has no basic auth; Traefik only admits `telegram_webhook_source_ranges` and n8n checks Telegram's secret token header. `n8n_telegram_webhook_id` must match `webhookId` in `telegram-chat.json`
- Replies are converted from Markdown to Telegram HTML by Code nodes and sent with `parse_mode` HTML
- Chat memory (Window Buffer Memory) lives in n8n's process memory, per workflow, and is lost on every n8n restart (including every deploy that changes workflows)
### SearXNG
The `searxng` role runs a metasearch engine (`searxng/searxng`) on the internal network `searxng_internal` (`http://searxng:8080`), used only by the n8n `web_search` tool. `settings.yml` is rendered from a template with `searxng_secret_key` from the vault; the JSON output format is enabled and the bot limiter is off because there is no public route. Some upstream engines (DuckDuckGo, Brave) answer with captchas or rate limits; SearXNG skips them and uses the others.
### Browserless
The `browserless` role runs headless Chromium (`ghcr.io/browserless/chromium`) on the internal network `browserless_internal` (`http://browserless:3000`), one session at a time with a memory limit (`browserless_mem_limit`).
### Ollama (local LLM)
The `ollama` role runs `ollama/ollama` (CPU only) on the internal Docker network `ollama_internal` — no Traefik route, no public port. n8n joins that network (as `external`) and reaches it at `http://ollama:11434`.
- Model, context length and memory limit come from `ollama_*` in `vars.yml` (default `qwen3:4b-instruct-2507-q4_K_M`, 16k context, 4.5 GB limit — the VPS has 8 GB shared with ~24 containers)
- Models persist in `/opt/ollama/models`; the role pulls `ollama_model` if it is missing
- `ollama_keep_alive` unloads the model from RAM when idle
- Ollama starts 2 threads on this 2-core/4-thread host; the Ollama chat model nodes in the workflows set `numThread` to 4
- Changing `ollama_model` and re-running `--tags ollama` pulls the new model; old models are removed manually with `docker exec ollama ollama rm <model>`
### Gitea Actions runners