diff --git a/CLAUDE.md b/CLAUDE.md index a6c81a3..f3513f9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -75,17 +75,46 @@ The mail role deploys four containers on `mail_internal`: `mail-db` (PostgreSQL - Fetchmail 6.6.x rcfile syntax: `ssl` is a user-level option (inline after `password`), port is `smtphost dovecot/24` (slash-separated), `smtpport` keyword does not exist - `mail.ladkau.de` is the Roundcube webmail client, not an MTA — no outgoing SMTP server is configured -### Clay (Claude Code web interface) +### n8n (AI chat and scheduled digests) -Clay (`agent.ladkau.de`) provides a browser-based UI for Claude Code. The role -builds a Docker image from `ansible/roles/clay/files/Dockerfile` (Node 20 + -`@anthropic-ai/claude-code` + `clay-server`) at deploy time. Key details: +n8n (`agent.ladkau.de`) runs the chat page and the scheduled workflows. The `n8n` role templates its compose file to `/opt/n8n` and imports the repo's workflows into it. Key details: -- `ANTHROPIC_API_KEY` is passed as an environment variable from the vault -- Sessions and config are persisted at `/opt/clay/data` → `/root/.clay` in container -- Projects (git repos) are mounted from `/opt/clay/workspace` → `/workspace` -- Traefik basic auth (`clay-auth` middleware) uses `clay_users` from the vault -- Scheduled agent workflows use Gitea Actions `on.schedule` + the existing act_runners +- Traefik basic auth (`n8n-auth` middleware) uses `n8n_users` from the vault; n8n has its own owner login behind it +- `/chat` is redirected by a Traefik `redirectregex` middleware to the hosted chat webhook `/webhook//chat`; `n8n_chat_webhook_id` must match `webhookId` in `roles/n8n/files/workflows/chat.json` +- Workflows are `roles/n8n/files/workflows/*.json`; credentials are imported from `roles/n8n/files/credentials/ollama.json` and a template rendered from the vault (`smtp_user`, `smtp_password`). Import (credentials, workflows, publish `"active": true` ones, restart) runs as a handler only when those files change +- Imports replace workflows by id: edits made in the n8n UI are lost unless exported back into the repo +- Data and the SQLite database are in `/opt/n8n/data`; `n8n_encryption_key` protects stored credentials and must never change after the first deploy +- n8n joins `ollama_internal` and `browserless_internal`; neither backend has a public route +- The `fetch_page` tool and the digest workflow call browserless `POST /function`, which runs a small Puppeteer script and returns the page text +- The `web_search` tool (`web-search.json`) queries SearXNG's JSON API and returns the top 6 results +- n8n also joins `searxng_internal` +- Telegram: the `Telegram chat` workflow answers one Telegram user (`telegram_user_id`); other users and non-text messages are ignored +- Model routing (`Route` step): messages go to the Gemini chain (`gemini_api_key` from the vault); messages starting with `/qwen` go to the local Qwen agent. The Gemini chain is `gemini-3.8-flash`, then `gemini-3.5-flash` if the answer is empty, then Qwen. Free-tier Gemini limits are per model; the reply ends with a note when a fallback answered +- Model names in the workflow JSON are plain strings. `__OLLAMA_MODEL__` is replaced by `ollama_model` and `__TELEGRAM_USER_ID__` by `telegram_user_id` when Ansible copies the files; keep the placeholders when exporting workflows from the UI +- All agents have the `web_search` and `fetch_page` tools, 4 iterations at most, and the instruction to make one `fetch_page` call per web address or one `web_search` per question. The Gemini agents share one conversation memory; the Qwen agent has its own small memory +- Scheduled workflows (`digest-telegram.json`) post through the `Post message` entry point of `Telegram chat`, so the text lands in the chat and in its memory +- The `Telegram chat` workflow has a schedule (`Stock price schedule`, three cron expressions, weekdays) and a manual trigger. Both send `/stocks` through `Route`, which expands it to `STOCK_QUESTION` (names the two Yahoo quote pages); `/stocks` typed in Telegram does the same and `/qwen /stocks` sends it to Qwen +- Telegram's servers call n8n at `/webhook//webhook`. That route has no basic auth; Traefik only admits `telegram_webhook_source_ranges` and n8n checks Telegram's secret token header. `n8n_telegram_webhook_id` must match `webhookId` in `telegram-chat.json` +- Replies are converted from Markdown to Telegram HTML by Code nodes and sent with `parse_mode` HTML +- Chat memory (Window Buffer Memory) lives in n8n's process memory, per workflow, and is lost on every n8n restart (including every deploy that changes workflows) + +### SearXNG + +The `searxng` role runs a metasearch engine (`searxng/searxng`) on the internal network `searxng_internal` (`http://searxng:8080`), used only by the n8n `web_search` tool. `settings.yml` is rendered from a template with `searxng_secret_key` from the vault; the JSON output format is enabled and the bot limiter is off because there is no public route. Some upstream engines (DuckDuckGo, Brave) answer with captchas or rate limits; SearXNG skips them and uses the others. + +### Browserless + +The `browserless` role runs headless Chromium (`ghcr.io/browserless/chromium`) on the internal network `browserless_internal` (`http://browserless:3000`), one session at a time with a memory limit (`browserless_mem_limit`). + +### Ollama (local LLM) + +The `ollama` role runs `ollama/ollama` (CPU only) on the internal Docker network `ollama_internal` — no Traefik route, no public port. n8n joins that network (as `external`) and reaches it at `http://ollama:11434`. + +- Model, context length and memory limit come from `ollama_*` in `vars.yml` (default `qwen3:4b-instruct-2507-q4_K_M`, 16k context, 4.5 GB limit — the VPS has 8 GB shared with ~24 containers) +- Models persist in `/opt/ollama/models`; the role pulls `ollama_model` if it is missing +- `ollama_keep_alive` unloads the model from RAM when idle +- Ollama starts 2 threads on this 2-core/4-thread host; the Ollama chat model nodes in the workflows set `numThread` to 4 +- Changing `ollama_model` and re-running `--tags ollama` pulls the new model; old models are removed manually with `docker exec ollama ollama rm ` ### Gitea Actions runners diff --git a/README.md b/README.md index 02816cb..ff07363 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,7 @@ +

+ cloud.ladkau.de logo +

+ Info: -- Server: @@ -20,6 +24,9 @@ DNS Names: mail.ladkau.de cr.ladkau.de k8s.ladkau.de + vault.ladkau.de + dl.ladkau.de + agent.ladkau.de Purpose: -- @@ -36,7 +43,7 @@ Provisioning: Ansible Service runtime: Docker Compose Each service runs as a Docker Compose stack. Compose files live inside - their respective Ansible roles (roles//files/docker-compose.yml). + their respective Ansible roles (roles//templates/docker-compose.yml.j2). This keeps service definition and deployment config together. Reverse proxy / TLS: Traefik @@ -49,7 +56,7 @@ Repo layout: ansible/ inventory.ini # host address and connection vars site.yml # master playbook — runs all roles in order - group_vars/all.yml # shared variables (domains, image versions, ...) + group_vars/all/ # vars.yml (shared variables) and vault.yml (encrypted secrets) roles/ base/ # OS hardening, non-root user, SSH, ufw firewall docker/ # Docker Engine + Compose plugin install @@ -57,16 +64,26 @@ Repo layout: gitea/ # self-hosted Git nextcloud/ # file storage and collaboration sso/ # Single Sign-On - mail/ # mail server + mail/ # Roundcube webmail, Dovecot, Fetchmail registry/ # container registry (cr.ladkau.de) - k8s/ # k3s Kubernetes node + k8s/ # placeholder page (k8s.ladkau.de) + act_runner/ # Gitea Actions runners + vaultwarden/ # password vault (vault.ladkau.de) + dl/ # download server (dl.ladkau.de) + dashboard/ # status dashboard (cloud.ladkau.de) + ollama/ # local LLM server (internal network only) + browserless/ # headless Chromium (internal network only) + searxng/ # metasearch for the chat web_search tool (internal network only) + n8n/ # AI chat and scheduled digests (agent.ladkau.de) docs/ runbook-provisioning.md # step-by-step reinstall instructions runbook-configuration.md # first-run service configuration + runbook-mail-import.md # migrating mbox mail to Maildir++ scripts/ run-bootstrap.sh # bootstrap deploy user on a fresh server bootstrap-deploy-user.sh # runs on server as root — creates deploy user check-vault.sh # verify vault.yml has all required secrets + check-services.sh # verify all service endpoints respond keys/ *.pub # public SSH keys (private keys are gitignored) diff --git a/ansible/group_vars/all/vars.yml b/ansible/group_vars/all/vars.yml index 7719295..5f92dd3 100644 --- a/ansible/group_vars/all/vars.yml +++ b/ansible/group_vars/all/vars.yml @@ -125,12 +125,56 @@ dl_sftp_port: 2223 # Status dashboard (public — cloud.ladkau.de root) dashboard_data_dir: /opt/dashboard -# Clay — Claude Code web interface (agent.ladkau.de) -domain_clay: "agent.{{ domain_base }}" -clay_data_dir: /opt/clay -clay_users: [] # override in vault with [{username: ..., password: ...}] +# Ollama — local LLM server, internal network only (no public route) +ollama_version: "0.35.1" +ollama_data_dir: /opt/ollama +ollama_model: "qwen3:4b-instruct-2507-q4_K_M" # non-thinking variant: the thinking model needs minutes per answer on this CPU +ollama_context_length: 16384 +ollama_mem_limit: 4608m +ollama_keep_alive: "1h" # unload the model after this idle time to free RAM + +# Browserless — headless Chromium for rendering JavaScript-heavy pages (internal only) +browserless_version: "v2.57.0" +browserless_data_dir: /opt/browserless +browserless_mem_limit: 1g + +# SearXNG — metasearch backend for the n8n web_search tool (internal only) +searxng_version: "2026.10.2-19ffbcd30" +searxng_data_dir: /opt/searxng +searxng_mem_limit: 384m # Secrets — store values in ansible/group_vars/all/vault.yml (Ansible Vault) -# anthropic_api_key: "" # from console.anthropic.com → API keys -# clay_users: +# searxng_secret_key: "" # generate: openssl rand -hex 32 + +# n8n — workflow automation, AI chat and scheduled digests (agent.ladkau.de) +domain_n8n: "agent.{{ domain_base }}" +n8n_version: "2.41.6" +n8n_data_dir: /opt/n8n +n8n_timezone: "Europe/Berlin" +n8n_mem_limit: 768m +# Fixed webhook id of the chat workflow: the chat page is served at +# https://agent./webhook//chat and https://agent./chat redirects there. +# Must match "webhookId" in roles/n8n/files/workflows/chat.json. +n8n_chat_webhook_id: "3f2b6c1e-8a4d-4e57-9b0a-6d1c2e7f5a10" +# Fixed webhook id of the Telegram trigger; must match "webhookId" in roles/n8n/files/workflows/telegram-chat.json. +n8n_telegram_webhook_id: "7c1d4e92-5b3a-4f08-a6e1-2d9b8c0f3a47" +# Source addresses allowed to call the Telegram webhook (Telegram's webhook servers, see +# core.telegram.org/bots/webhooks). Set to [] to disable the address filter. +telegram_webhook_source_ranges: + - 149.154.160.0/20 + - 91.108.4.0/22 +n8n_users: [] # override in vault with [{username: ..., password: ...}] — basic auth in front of n8n + +# Outgoing mail for n8n (Strato SMTP) +smtp_host: "smtp.strato.de" +smtp_port: 465 +smtp_secure: true # true = implicit TLS (port 465); false = STARTTLS (port 587) +# Secrets — store values in ansible/group_vars/all/vault.yml (Ansible Vault) +# n8n_encryption_key: "" # generate: openssl rand -hex 32 +# gemini_api_key: "" # Google AI Studio API key (default model of the Telegram chat) +# telegram_bot_token: "" # from @BotFather +# telegram_user_id: "" # your numeric Telegram user id (the only user the bot answers) +# smtp_user: "" # full Strato mailbox address +# smtp_password: "" +# n8n_users: # - username: admin # password: "your-password" diff --git a/ansible/roles/browserless/handlers/main.yml b/ansible/roles/browserless/handlers/main.yml new file mode 100644 index 0000000..c09a9d4 --- /dev/null +++ b/ansible/roles/browserless/handlers/main.yml @@ -0,0 +1,6 @@ +--- +- name: Restart browserless + community.docker.docker_compose_v2: + project_src: "{{ browserless_data_dir }}" + state: present + recreate: always diff --git a/ansible/roles/browserless/tasks/main.yml b/ansible/roles/browserless/tasks/main.yml new file mode 100644 index 0000000..feb700e --- /dev/null +++ b/ansible/roles/browserless/tasks/main.yml @@ -0,0 +1,26 @@ +--- +- name: Create Browserless directory + ansible.builtin.file: + path: "{{ browserless_data_dir }}" + state: directory + owner: root + group: root + mode: "0755" + tags: browserless + +- name: Deploy Docker Compose file + ansible.builtin.template: + src: docker-compose.yml.j2 + dest: "{{ browserless_data_dir }}/docker-compose.yml" + owner: root + group: root + mode: "0644" + notify: Restart browserless + tags: browserless + +- name: Start Browserless + community.docker.docker_compose_v2: + project_src: "{{ browserless_data_dir }}" + state: present + pull: missing + tags: browserless diff --git a/ansible/roles/browserless/templates/docker-compose.yml.j2 b/ansible/roles/browserless/templates/docker-compose.yml.j2 new file mode 100644 index 0000000..da1b0e0 --- /dev/null +++ b/ansible/roles/browserless/templates/docker-compose.yml.j2 @@ -0,0 +1,21 @@ +# Managed by Ansible — do not edit manually +services: + browserless: + image: ghcr.io/browserless/chromium:{{ browserless_version }} + container_name: browserless + restart: unless-stopped + environment: + # One browser session at a time, short queue — keeps RAM use predictable on the shared VPS + CONCURRENT: "1" + QUEUED: "4" + TIMEOUT: "60000" + shm_size: 512m + mem_limit: {{ browserless_mem_limit }} + networks: + - browserless_internal + +# Not exposed via Traefik. Containers that need the API join this network +# (declared as external there) and reach it at http://browserless:3000. +networks: + browserless_internal: + name: browserless_internal diff --git a/ansible/roles/clay/files/Dockerfile b/ansible/roles/clay/files/Dockerfile deleted file mode 100644 index 0f18dc5..0000000 --- a/ansible/roles/clay/files/Dockerfile +++ /dev/null @@ -1,20 +0,0 @@ -FROM node:20-alpine - -# Claude Code needs git and bash to operate on repositories -RUN apk add --no-cache git bash curl - -# Install Claude Code CLI -RUN npm install -g @anthropic-ai/claude-code - -# Install clay web server -RUN npm install -g clay-server - -# MCP servers and skills — add packages here to survive container rebuilds -# RUN npm install -g @clay-ai/clay-ralph -# RUN npm install -g @modelcontextprotocol/server-filesystem - -RUN mkdir -p /workspace /root/.clay - -WORKDIR /workspace - -EXPOSE 2633 diff --git a/ansible/roles/clay/handlers/main.yml b/ansible/roles/clay/handlers/main.yml deleted file mode 100644 index 00667b7..0000000 --- a/ansible/roles/clay/handlers/main.yml +++ /dev/null @@ -1,16 +0,0 @@ ---- -- name: Rebuild and restart clay - community.docker.docker_image: - name: clay:local - build: - path: "{{ clay_data_dir }}" - source: build - force_source: true - state: present - notify: Restart clay - -- name: Restart clay - community.docker.docker_compose_v2: - project_src: "{{ clay_data_dir }}" - state: present - recreate: always diff --git a/ansible/roles/clay/tasks/main.yml b/ansible/roles/clay/tasks/main.yml deleted file mode 100644 index c1fd99f..0000000 --- a/ansible/roles/clay/tasks/main.yml +++ /dev/null @@ -1,48 +0,0 @@ ---- -- name: Create clay directories - ansible.builtin.file: - path: "{{ item }}" - state: directory - owner: root - group: root - mode: "0755" - loop: - - "{{ clay_data_dir }}" - - "{{ clay_data_dir }}/data" - - "{{ clay_data_dir }}/workspace" - tags: clay - -- name: Deploy Dockerfile - ansible.builtin.copy: - src: Dockerfile - dest: "{{ clay_data_dir }}/Dockerfile" - owner: root - group: root - mode: "0644" - notify: Rebuild and restart clay - tags: clay - -- name: Deploy Docker Compose file - ansible.builtin.template: - src: docker-compose.yml.j2 - dest: "{{ clay_data_dir }}/docker-compose.yml" - owner: root - group: root - mode: "0644" - notify: Restart clay - tags: clay - -- name: Build clay image - community.docker.docker_image: - name: clay:local - build: - path: "{{ clay_data_dir }}" - source: build - state: present - tags: clay - -- name: Start clay - community.docker.docker_compose_v2: - project_src: "{{ clay_data_dir }}" - state: present - tags: clay diff --git a/ansible/roles/clay/templates/docker-compose.yml.j2 b/ansible/roles/clay/templates/docker-compose.yml.j2 deleted file mode 100644 index b39261a..0000000 --- a/ansible/roles/clay/templates/docker-compose.yml.j2 +++ /dev/null @@ -1,35 +0,0 @@ -# Managed by Ansible — do not edit manually -services: - clay: - image: clay:local - build: - context: {{ clay_data_dir }} - dockerfile: Dockerfile - container_name: clay - restart: unless-stopped - tty: true - environment: - - ANTHROPIC_API_KEY={{ anthropic_api_key }} - volumes: - - {{ clay_data_dir }}/data:/root/.clay - - {{ clay_data_dir }}/workspace:/workspace - command: ["clay-server", "--yes", "-p", "2633"] - networks: - - traefik_public - labels: - - "traefik.enable=true" - - "traefik.http.routers.clay.rule=Host(`{{ domain_clay }}`)" - - "traefik.http.routers.clay.entrypoints=websecure" - - "traefik.http.routers.clay.tls.certresolver=letsencrypt" - - "traefik.http.services.clay.loadbalancer.server.scheme=https" - - "traefik.http.services.clay.loadbalancer.server.port=2633" - - "traefik.http.routers.clay.middlewares=clay-auth" -{% set ns = namespace(entries=[]) %} -{% for user in clay_users %} -{% set ns.entries = ns.entries + [user.username + ':' + (user.password | password_hash('bcrypt', (user.username | hash('md5'))[:22]) | replace('$', '$$'))] %} -{% endfor %} - - "traefik.http.middlewares.clay-auth.basicauth.users={{ ns.entries | join(',') }}" - -networks: - traefik_public: - external: true diff --git a/ansible/roles/dashboard/templates/app.py.j2 b/ansible/roles/dashboard/templates/app.py.j2 index 627cac3..30c580c 100644 --- a/ansible/roles/dashboard/templates/app.py.j2 +++ b/ansible/roles/dashboard/templates/app.py.j2 @@ -25,7 +25,8 @@ SERVICES = [ ("k8s", "https://{{ domain_k8s }}", 200, "https://{{ domain_k8s }}"), ("Vaultwarden", "https://{{ domain_vault }}", 200, "https://{{ domain_vault }}"), ("Downloads", "https://{{ domain_dl }}", 200, "https://{{ domain_dl }}"), - ("Clay", "https://{{ domain_clay }}", 401, "https://{{ domain_clay }}"), + ("AI Chat", "https://{{ domain_n8n }}/chat", 401, "https://{{ domain_n8n }}/chat"), + ("n8n", "https://{{ domain_n8n }}", 401, "https://{{ domain_n8n }}"), ] {% raw %} diff --git a/ansible/roles/n8n/files/credentials/ollama.json b/ansible/roles/n8n/files/credentials/ollama.json new file mode 100644 index 0000000..042e4a6 --- /dev/null +++ b/ansible/roles/n8n/files/credentials/ollama.json @@ -0,0 +1,9 @@ +{ + "id": "ollama0000000001", + "name": "Ollama (local)", + "type": "ollamaApi", + "data": { + "baseUrl": "http://ollama:11434", + "apiKey": "" + } +} diff --git a/ansible/roles/n8n/files/workflows/chat.json b/ansible/roles/n8n/files/workflows/chat.json new file mode 100644 index 0000000..179c151 --- /dev/null +++ b/ansible/roles/n8n/files/workflows/chat.json @@ -0,0 +1,222 @@ +{ + "id": "chat00000000001", + "name": "Chat", + "active": true, + "nodes": [ + { + "id": "c1", + "name": "Chat", + "type": "@n8n/n8n-nodes-langchain.chatTrigger", + "typeVersion": 1.4, + "position": [ + 0, + 0 + ], + "webhookId": "3f2b6c1e-8a4d-4e57-9b0a-6d1c2e7f5a10", + "parameters": { + "public": true, + "mode": "hostedChat", + "authentication": "none", + "initialMessages": "Hi! Ask me anything or paste a link and I will read the page.", + "options": { + "loadPreviousSession": "notSupported", + "responseMode": "streaming", + "title": "Local AI chat" + } + } + }, + { + "id": "c2", + "name": "AI Agent", + "type": "@n8n/n8n-nodes-langchain.agent", + "typeVersion": 3.1, + "position": [ + 300, + 0 + ], + "parameters": { + "promptType": "auto", + "text": "={{ $json.chatInput }}", + "options": { + "systemMessage": "You are a concise assistant. Answer in the language of the user. For questions about current events, weather, news, prices or anything you cannot know for sure, call the web_search tool first with a short search query, then answer from the results and name the source URLs. When the user gives a web address, or a search result needs more detail, call the fetch_page tool with the full URL and base your answer only on what it returns. If a tool fails, say so. Do not invent facts or page content." + } + } + }, + { + "id": "c3", + "name": "Ollama Chat Model", + "type": "@n8n/n8n-nodes-langchain.lmChatOllama", + "typeVersion": 1, + "position": [ + 200, + 260 + ], + "parameters": { + "model": "__OLLAMA_MODEL__", + "options": { + "numCtx": 16384, + "temperature": 0.3, + "numThread": 4 + } + }, + "credentials": { + "ollamaApi": { + "id": "ollama0000000001", + "name": "Ollama (local)" + } + } + }, + { + "id": "c4", + "name": "Window Buffer Memory", + "type": "@n8n/n8n-nodes-langchain.memoryBufferWindow", + "typeVersion": 1.3, + "position": [ + 380, + 260 + ], + "parameters": { + "sessionIdType": "fromInput", + "contextWindowLength": 3 + } + }, + { + "id": "c5", + "name": "fetch_page", + "type": "@n8n/n8n-nodes-langchain.toolWorkflow", + "typeVersion": 2.2, + "position": [ + 560, + 260 + ], + "parameters": { + "name": "fetch_page", + "description": "Opens a web page in a real browser (JavaScript is executed) and returns its title and readable text, truncated to about 3000 characters. Input: the full URL.", + "workflowId": { + "__rl": true, + "value": "fetchpage0000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "url": "={{ $fromAI('url', 'The full URL of the page to open, including https://', 'string') }}" + }, + "matchingColumns": [], + "schema": [ + { + "id": "url", + "displayName": "url", + "required": false, + "defaultMatch": false, + "display": true, + "canBeUsedToMatch": true, + "type": "string" + } + ], + "attemptToConvertTypes": false, + "convertFieldsToString": false + } + } + }, + { + "id": "c6", + "name": "web_search", + "type": "@n8n/n8n-nodes-langchain.toolWorkflow", + "typeVersion": 2.2, + "position": [ + 740, + 260 + ], + "parameters": { + "name": "web_search", + "description": "Searches the web and returns the top results (title, URL, short snippet). Input: a short search query.", + "workflowId": { + "__rl": true, + "value": "websearch000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "query": "={{ $fromAI('query', 'A short web search query', 'string') }}" + }, + "matchingColumns": [], + "schema": [ + { + "id": "query", + "displayName": "query", + "required": false, + "defaultMatch": false, + "display": true, + "canBeUsedToMatch": true, + "type": "string" + } + ], + "attemptToConvertTypes": false, + "convertFieldsToString": false + } + } + } + ], + "connections": { + "Chat": { + "main": [ + [ + { + "node": "AI Agent", + "type": "main", + "index": 0 + } + ] + ] + }, + "Ollama Chat Model": { + "ai_languageModel": [ + [ + { + "node": "AI Agent", + "type": "ai_languageModel", + "index": 0 + } + ] + ] + }, + "Window Buffer Memory": { + "ai_memory": [ + [ + { + "node": "AI Agent", + "type": "ai_memory", + "index": 0 + } + ] + ] + }, + "fetch_page": { + "ai_tool": [ + [ + { + "node": "AI Agent", + "type": "ai_tool", + "index": 0 + } + ] + ] + }, + "web_search": { + "ai_tool": [ + [ + { + "node": "AI Agent", + "type": "ai_tool", + "index": 0 + } + ] + ] + } + }, + "settings": { + "executionOrder": "v1" + } +} diff --git a/ansible/roles/n8n/files/workflows/digest-telegram.json b/ansible/roles/n8n/files/workflows/digest-telegram.json new file mode 100644 index 0000000..0666675 --- /dev/null +++ b/ansible/roles/n8n/files/workflows/digest-telegram.json @@ -0,0 +1,234 @@ +{ + "id": "digesttg00000001", + "name": "Weekly page digest to Telegram (example)", + "active": false, + "nodes": [ + { + "id": "g1", + "name": "Every Monday 06:00", + "type": "n8n-nodes-base.scheduleTrigger", + "typeVersion": 1.2, + "position": [ + 0, + 0 + ], + "parameters": { + "rule": { + "interval": [ + { + "field": "cronExpression", + "expression": "0 6 * * 1" + } + ] + } + } + }, + { + "id": "g2", + "name": "Sites", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [ + 240, + 0 + ], + "parameters": { + "jsCode": "// One item per page to summarise. Edit url and prompt.\nreturn [\n { json: { url: 'https://example.com', prompt: 'Summarise this page in five bullet points.' } },\n];\n" + } + }, + { + "id": "g3", + "name": "Fetch page text", + "type": "n8n-nodes-base.executeWorkflow", + "typeVersion": 1.2, + "position": [ + 480, + 0 + ], + "parameters": { + "source": "database", + "workflowId": { + "__rl": true, + "value": "fetchpage0000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "url": "={{ $json.url }}" + }, + "matchingColumns": [], + "schema": [], + "attemptToConvertTypes": false, + "convertFieldsToString": false + }, + "options": { + "waitForSubWorkflow": true + } + } + }, + { + "id": "g4", + "name": "Summarise", + "type": "@n8n/n8n-nodes-langchain.chainLlm", + "typeVersion": 1.7, + "position": [ + 720, + 0 + ], + "parameters": { + "promptType": "define", + "text": "={{ $('Sites').item.json.prompt + '\\n\\n---\\n' + $json.response }}" + } + }, + { + "id": "g5", + "name": "Ollama Chat Model", + "type": "@n8n/n8n-nodes-langchain.lmChatOllama", + "typeVersion": 1, + "position": [ + 720, + 240 + ], + "parameters": { + "model": "__OLLAMA_MODEL__", + "options": { + "numCtx": 16384, + "temperature": 0.2, + "numThread": 4 + } + }, + "credentials": { + "ollamaApi": { + "id": "ollama0000000001", + "name": "Ollama (local)" + } + } + }, + { + "id": "g6", + "name": "Format message", + "type": "n8n-nodes-base.set", + "typeVersion": 3.4, + "position": [ + 1000, + 0 + ], + "parameters": { + "assignments": { + "assignments": [ + { + "id": "f1", + "name": "message", + "type": "string", + "value": "={{ ('Digest: ' + $('Sites').item.json.url + '\\n\\n' + $json.text).slice(0, 4000) }}" + } + ] + }, + "options": {} + } + }, + { + "id": "g7", + "name": "Post to Telegram chat", + "type": "n8n-nodes-base.executeWorkflow", + "typeVersion": 1.2, + "position": [ + 1240, + 0 + ], + "parameters": { + "source": "database", + "workflowId": { + "__rl": true, + "value": "telegramchat0001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "message": "={{ $json.message }}" + }, + "matchingColumns": [], + "schema": [], + "attemptToConvertTypes": false, + "convertFieldsToString": false + }, + "options": { + "waitForSubWorkflow": true + } + } + } + ], + "connections": { + "Every Monday 06:00": { + "main": [ + [ + { + "node": "Sites", + "type": "main", + "index": 0 + } + ] + ] + }, + "Sites": { + "main": [ + [ + { + "node": "Fetch page text", + "type": "main", + "index": 0 + } + ] + ] + }, + "Fetch page text": { + "main": [ + [ + { + "node": "Summarise", + "type": "main", + "index": 0 + } + ] + ] + }, + "Ollama Chat Model": { + "ai_languageModel": [ + [ + { + "node": "Summarise", + "type": "ai_languageModel", + "index": 0 + } + ] + ] + }, + "Summarise": { + "main": [ + [ + { + "node": "Format message", + "type": "main", + "index": 0 + } + ] + ] + }, + "Format message": { + "main": [ + [ + { + "node": "Post to Telegram chat", + "type": "main", + "index": 0 + } + ] + ] + } + }, + "settings": { + "executionOrder": "v1" + } +} diff --git a/ansible/roles/n8n/files/workflows/digest.json b/ansible/roles/n8n/files/workflows/digest.json new file mode 100644 index 0000000..23a6b76 --- /dev/null +++ b/ansible/roles/n8n/files/workflows/digest.json @@ -0,0 +1,193 @@ +{ + "id": "digest0000000001", + "name": "Weekly page digest (example)", + "active": false, + "nodes": [ + { + "id": "d1", + "name": "Every Monday 06:00", + "type": "n8n-nodes-base.scheduleTrigger", + "typeVersion": 1.2, + "position": [ + 0, + 0 + ], + "parameters": { + "rule": { + "interval": [ + { + "field": "cronExpression", + "expression": "0 6 * * 1" + } + ] + } + } + }, + { + "id": "d2", + "name": "Sites", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [ + 240, + 0 + ], + "parameters": { + "jsCode": "// One item per page to summarise. Edit url and prompt.\nreturn [\n { json: { url: 'https://example.com', prompt: 'Summarise this page in five bullet points.', to: 'you@example.com' } },\n];\n" + } + }, + { + "id": "d3", + "name": "Fetch page text", + "type": "n8n-nodes-base.executeWorkflow", + "typeVersion": 1.2, + "position": [ + 480, + 0 + ], + "parameters": { + "source": "database", + "workflowId": { + "__rl": true, + "value": "fetchpage0000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "url": "={{ $json.url }}" + }, + "matchingColumns": [], + "schema": [], + "attemptToConvertTypes": false, + "convertFieldsToString": false + }, + "options": { + "waitForSubWorkflow": true + } + } + }, + { + "id": "d4", + "name": "Summarise", + "type": "@n8n/n8n-nodes-langchain.chainLlm", + "typeVersion": 1.7, + "position": [ + 720, + 0 + ], + "parameters": { + "promptType": "define", + "text": "={{ $('Sites').item.json.prompt + '\\n\\n---\\n' + $json.response }}" + } + }, + { + "id": "d5", + "name": "Ollama Chat Model", + "type": "@n8n/n8n-nodes-langchain.lmChatOllama", + "typeVersion": 1, + "position": [ + 720, + 240 + ], + "parameters": { + "model": "__OLLAMA_MODEL__", + "options": { + "numCtx": 16384, + "temperature": 0.2, + "numThread": 4 + } + }, + "credentials": { + "ollamaApi": { + "id": "ollama0000000001", + "name": "Ollama (local)" + } + } + }, + { + "id": "d6", + "name": "Send mail", + "type": "n8n-nodes-base.emailSend", + "typeVersion": 2.1, + "position": [ + 1020, + 0 + ], + "parameters": { + "fromEmail": "={{ $credentials.user }}", + "toEmail": "={{ $('Sites').item.json.to }}", + "subject": "={{ 'Digest: ' + $('Sites').item.json.url }}", + "emailFormat": "text", + "text": "={{ $json.text }}", + "options": {} + }, + "credentials": { + "smtp": { + "id": "smtp00000000001", + "name": "Strato SMTP" + } + } + } + ], + "connections": { + "Every Monday 06:00": { + "main": [ + [ + { + "node": "Sites", + "type": "main", + "index": 0 + } + ] + ] + }, + "Sites": { + "main": [ + [ + { + "node": "Fetch page text", + "type": "main", + "index": 0 + } + ] + ] + }, + "Fetch page text": { + "main": [ + [ + { + "node": "Summarise", + "type": "main", + "index": 0 + } + ] + ] + }, + "Ollama Chat Model": { + "ai_languageModel": [ + [ + { + "node": "Summarise", + "type": "ai_languageModel", + "index": 0 + } + ] + ] + }, + "Summarise": { + "main": [ + [ + { + "node": "Send mail", + "type": "main", + "index": 0 + } + ] + ] + } + }, + "settings": { + "executionOrder": "v1" + } +} diff --git a/ansible/roles/n8n/files/workflows/fetch-page.json b/ansible/roles/n8n/files/workflows/fetch-page.json new file mode 100644 index 0000000..531ced5 --- /dev/null +++ b/ansible/roles/n8n/files/workflows/fetch-page.json @@ -0,0 +1,98 @@ +{ + "id": "fetchpage0000001", + "name": "Fetch page text", + "active": true, + "nodes": [ + { + "id": "n1", + "name": "When called by another workflow", + "type": "n8n-nodes-base.executeWorkflowTrigger", + "typeVersion": 1.1, + "position": [ + 0, + 0 + ], + "parameters": { + "inputSource": "workflowInputs", + "workflowInputs": { + "values": [ + { + "name": "url", + "type": "string" + } + ] + } + } + }, + { + "id": "n2", + "name": "Render page in browserless", + "type": "n8n-nodes-base.httpRequest", + "typeVersion": 4.2, + "position": [ + 260, + 0 + ], + "parameters": { + "method": "POST", + "url": "http://browserless:3000/function", + "sendBody": true, + "specifyBody": "json", + "jsonBody": "={{ JSON.stringify({ code: \"export default async function ({ page, context }) {\\n await page.goto(context.url, { waitUntil: 'networkidle2', timeout: 40000 });\\n const title = await page.title();\\n const text = await page.evaluate(() => {\\n document.querySelectorAll('script,style,noscript,nav,footer,header,iframe,svg').forEach(e => e.remove());\\n return (document.body.innerText || '').replace(/\\\\n{3,}/g, '\\\\n\\\\n').trim();\\n });\\n return { data: { title, text: text.slice(0, context.maxChars || 3000) }, type: 'application/json' };\\n}\\n\", context: { url: $json.url, maxChars: 3000 } }) }}", + "options": { + "timeout": 90000 + } + } + }, + { + "id": "n3", + "name": "Format result", + "type": "n8n-nodes-base.set", + "typeVersion": 3.4, + "position": [ + 520, + 0 + ], + "parameters": { + "assignments": { + "assignments": [ + { + "id": "a1", + "name": "response", + "type": "string", + "value": "={{ 'Title: ' + $json.data.title + '\\n\\n' + $json.data.text }}" + } + ] + }, + "options": {} + } + } + ], + "connections": { + "When called by another workflow": { + "main": [ + [ + { + "node": "Render page in browserless", + "type": "main", + "index": 0 + } + ] + ] + }, + "Render page in browserless": { + "main": [ + [ + { + "node": "Format result", + "type": "main", + "index": 0 + } + ] + ] + } + }, + "settings": { + "executionOrder": "v1" + } +} diff --git a/ansible/roles/n8n/files/workflows/telegram-chat.json b/ansible/roles/n8n/files/workflows/telegram-chat.json new file mode 100644 index 0000000..bd82018 --- /dev/null +++ b/ansible/roles/n8n/files/workflows/telegram-chat.json @@ -0,0 +1,1096 @@ +{ + "id": "telegramchat0001", + "name": "Telegram chat", + "active": true, + "nodes": [ + { + "id": "t1", + "name": "Telegram Trigger", + "type": "n8n-nodes-base.telegramTrigger", + "typeVersion": 1.2, + "position": [ + 0, + 0 + ], + "webhookId": "7c1d4e92-5b3a-4f08-a6e1-2d9b8c0f3a47", + "parameters": { + "updates": [ + "message" + ], + "additionalFields": {} + }, + "credentials": { + "telegramApi": { + "id": "telegram0000001", + "name": "Telegram bot" + } + } + }, + { + "id": "t26", + "name": "Stock price schedule", + "type": "n8n-nodes-base.scheduleTrigger", + "typeVersion": 1.2, + "position": [ + 0, + -420 + ], + "parameters": { + "rule": { + "interval": [ + { + "field": "cronExpression", + "expression": "30 7 * * 1-5" + }, + { + "field": "cronExpression", + "expression": "0 13 * * 1-5" + }, + { + "field": "cronExpression", + "expression": "40 17 * * 1-5" + } + ] + } + } + }, + { + "id": "t28", + "name": "Run stock price now", + "type": "n8n-nodes-base.manualTrigger", + "typeVersion": 1, + "position": [ + 0, + -260 + ], + "parameters": {} + }, + { + "id": "t27", + "name": "Scheduled question", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [ + 260, + -420 + ], + "parameters": { + "mode": "runOnceForEachItem", + "jsCode": "// Builds a Telegram-style update for /stocks, so it goes through Route and the model chain.\nconst id = Number('__TELEGRAM_USER_ID__');\nreturn { json: { message: { from: { id }, chat: { id }, text: '/stocks' } } };\n" + } + }, + { + "id": "t2", + "name": "Only my user", + "type": "n8n-nodes-base.if", + "typeVersion": 2.2, + "position": [ + 260, + 0 + ], + "parameters": { + "conditions": { + "options": { + "caseSensitive": true, + "leftValue": "", + "typeValidation": "loose", + "version": 2 + }, + "conditions": [ + { + "id": "k1", + "leftValue": "={{ $json.message.from.id }}", + "rightValue": "__TELEGRAM_USER_ID__", + "operator": { + "type": "string", + "operation": "equals" + } + }, + { + "id": "k2", + "leftValue": "={{ $json.message.text }}", + "rightValue": "", + "operator": { + "type": "string", + "operation": "notEmpty", + "singleValue": true + } + } + ], + "combinator": "and" + }, + "options": {} + } + }, + { + "id": "t12", + "name": "Route", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [ + 520, + -100 + ], + "parameters": { + "mode": "runOnceForEachItem", + "jsCode": "// Default model is Gemini; a message starting with /qwen goes to the local Qwen model.\n// /stocks (typed in Telegram or sent by the schedule / manual trigger) asks the stock price question;\n// the question names the pages explicitly because small models only reliably call a tool when given a URL.\nconst STOCK_QUESTION = 'Use the fetch_page tool on each of these two pages and report the current price, '\n + 'the change and the change in percent for each stock: '\n + 'https://de.finance.yahoo.com/quote/MTE.DE/ and https://de.finance.yahoo.com/quote/PTX.F/';\nconst m = $input.item.json.message;\nlet text = (m.text || '').trim();\nlet model = 'gemini';\nconst match = text.match(/^\\/qwen(@\\w+)?(\\s+|$)/i);\nif (match) { model = 'qwen'; text = text.slice(match[0].length).trim(); }\nif (/^\\/stocks(@\\w+)?$/i.test(text)) text = STOCK_QUESTION;\ntext = text || 'Hello';\nconst quoted = m.reply_to_message && m.reply_to_message.text;\nconst prompt = (quoted ? 'Context (the message being replied to):\\n' + quoted + '\\n\\n' : '') + text;\nreturn { json: { chat_id: m.chat.id, model, prompt } };\n" + } + }, + { + "id": "t13", + "name": "Use Qwen?", + "type": "n8n-nodes-base.if", + "typeVersion": 2.2, + "position": [ + 760, + -100 + ], + "parameters": { + "conditions": { + "options": { + "caseSensitive": true, + "leftValue": "", + "typeValidation": "strict", + "version": 2 + }, + "conditions": [ + { + "id": "q1", + "leftValue": "={{ $json.model }}", + "rightValue": "qwen", + "operator": { + "type": "string", + "operation": "equals" + } + } + ], + "combinator": "and" + }, + "options": {} + } + }, + { + "id": "t14", + "name": "Gemini 3.8 Agent", + "type": "@n8n/n8n-nodes-langchain.agent", + "typeVersion": 3.1, + "position": [ + 1000, + 200 + ], + "onError": "continueRegularOutput", + "parameters": { + "promptType": "define", + "text": "={{ $('Route').first().json.prompt }}", + "options": { + "systemMessage": "You are a concise assistant chatting with the user on a phone. Answer in the language of the user and keep answers short. For questions about current events, weather, news, prices or anything you cannot know for sure, call the web_search tool first with a short search query, then answer from the results and name the source URLs. When the user gives a web address, or a search result needs more detail, call the fetch_page tool with the full URL and base your answer only on what it returns. Earlier messages in this conversation (for example a digest you sent) are available as context for follow-up questions. If a tool fails, say so. Do not invent facts or page content. Call a tool only when needed: one fetch_page per web address, or one web_search per question; never repeat the same call. Always fetch every web address the user names.", + "maxIterations": 4 + } + }, + "retryOnFail": true, + "maxTries": 2, + "waitBetweenTries": 3000 + }, + { + "id": "t15", + "name": "Gemini 3.8", + "type": "@n8n/n8n-nodes-langchain.lmChatGoogleGemini", + "typeVersion": 1.2, + "position": [ + 900, + 520 + ], + "parameters": { + "modelName": "models/gemini-3.8-flash", + "options": { + "temperature": 0.3, + "maxOutputTokens": 2048 + } + }, + "credentials": { + "googlePalmApi": { + "id": "gemini00000001", + "name": "Google Gemini" + } + } + }, + { + "id": "t16", + "name": "fetch_page (gemini 3.8)", + "type": "@n8n/n8n-nodes-langchain.toolWorkflow", + "typeVersion": 2.2, + "position": [ + 1260, + 520 + ], + "parameters": { + "name": "fetch_page", + "description": "Opens a web page in a real browser (JavaScript is executed) and returns its title and readable text, truncated to about 3000 characters. Input: the full URL.", + "workflowId": { + "__rl": true, + "value": "fetchpage0000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "url": "={{ $fromAI('url', 'The full URL of the page to open, including https://', 'string') }}" + }, + "matchingColumns": [], + "schema": [ + { + "id": "url", + "displayName": "url", + "required": false, + "defaultMatch": false, + "display": true, + "canBeUsedToMatch": true, + "type": "string" + } + ], + "attemptToConvertTypes": false, + "convertFieldsToString": false + } + } + }, + { + "id": "t17", + "name": "web_search (gemini 3.8)", + "type": "@n8n/n8n-nodes-langchain.toolWorkflow", + "typeVersion": 2.2, + "position": [ + 1440, + 520 + ], + "parameters": { + "name": "web_search", + "description": "Searches the web and returns the top results (title, URL, short snippet). Input: a short search query.", + "workflowId": { + "__rl": true, + "value": "websearch000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "query": "={{ $fromAI('query', 'A short web search query', 'string') }}" + }, + "matchingColumns": [], + "schema": [ + { + "id": "query", + "displayName": "query", + "required": false, + "defaultMatch": false, + "display": true, + "canBeUsedToMatch": true, + "type": "string" + } + ], + "attemptToConvertTypes": false, + "convertFieldsToString": false + } + } + }, + { + "id": "t24", + "name": "Gemini 3.8 ok?", + "type": "n8n-nodes-base.if", + "typeVersion": 2.2, + "position": [ + 1300, + 200 + ], + "parameters": { + "conditions": { + "options": { + "caseSensitive": true, + "leftValue": "", + "typeValidation": "loose", + "version": 2 + }, + "conditions": [ + { + "id": "o1", + "leftValue": "={{ $json.output }}", + "rightValue": "", + "operator": { + "type": "string", + "operation": "notEmpty", + "singleValue": true + } + } + ], + "combinator": "and" + }, + "options": {} + } + }, + { + "id": "t20", + "name": "Gemini 3.5 Agent", + "type": "@n8n/n8n-nodes-langchain.agent", + "typeVersion": 3.1, + "position": [ + 1600, + 200 + ], + "onError": "continueRegularOutput", + "parameters": { + "promptType": "define", + "text": "={{ $('Route').first().json.prompt }}", + "options": { + "systemMessage": "You are a concise assistant chatting with the user on a phone. Answer in the language of the user and keep answers short. For questions about current events, weather, news, prices or anything you cannot know for sure, call the web_search tool first with a short search query, then answer from the results and name the source URLs. When the user gives a web address, or a search result needs more detail, call the fetch_page tool with the full URL and base your answer only on what it returns. Earlier messages in this conversation (for example a digest you sent) are available as context for follow-up questions. If a tool fails, say so. Do not invent facts or page content. Call a tool only when needed: one fetch_page per web address, or one web_search per question; never repeat the same call. Always fetch every web address the user names.", + "maxIterations": 4 + } + }, + "retryOnFail": true, + "maxTries": 2, + "waitBetweenTries": 3000 + }, + { + "id": "t21", + "name": "Gemini 3.5", + "type": "@n8n/n8n-nodes-langchain.lmChatGoogleGemini", + "typeVersion": 1.2, + "position": [ + 1500, + 520 + ], + "parameters": { + "modelName": "models/gemini-3.5-flash", + "options": { + "temperature": 0.3, + "maxOutputTokens": 2048 + } + }, + "credentials": { + "googlePalmApi": { + "id": "gemini00000001", + "name": "Google Gemini" + } + } + }, + { + "id": "t22", + "name": "fetch_page (gemini 3.5)", + "type": "@n8n/n8n-nodes-langchain.toolWorkflow", + "typeVersion": 2.2, + "position": [ + 1860, + 520 + ], + "parameters": { + "name": "fetch_page", + "description": "Opens a web page in a real browser (JavaScript is executed) and returns its title and readable text, truncated to about 3000 characters. Input: the full URL.", + "workflowId": { + "__rl": true, + "value": "fetchpage0000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "url": "={{ $fromAI('url', 'The full URL of the page to open, including https://', 'string') }}" + }, + "matchingColumns": [], + "schema": [ + { + "id": "url", + "displayName": "url", + "required": false, + "defaultMatch": false, + "display": true, + "canBeUsedToMatch": true, + "type": "string" + } + ], + "attemptToConvertTypes": false, + "convertFieldsToString": false + } + } + }, + { + "id": "t23", + "name": "web_search (gemini 3.5)", + "type": "@n8n/n8n-nodes-langchain.toolWorkflow", + "typeVersion": 2.2, + "position": [ + 2040, + 520 + ], + "parameters": { + "name": "web_search", + "description": "Searches the web and returns the top results (title, URL, short snippet). Input: a short search query.", + "workflowId": { + "__rl": true, + "value": "websearch000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "query": "={{ $fromAI('query', 'A short web search query', 'string') }}" + }, + "matchingColumns": [], + "schema": [ + { + "id": "query", + "displayName": "query", + "required": false, + "defaultMatch": false, + "display": true, + "canBeUsedToMatch": true, + "type": "string" + } + ], + "attemptToConvertTypes": false, + "convertFieldsToString": false + } + } + }, + { + "id": "t25", + "name": "Gemini 3.5 ok?", + "type": "n8n-nodes-base.if", + "typeVersion": 2.2, + "position": [ + 1900, + 200 + ], + "parameters": { + "conditions": { + "options": { + "caseSensitive": true, + "leftValue": "", + "typeValidation": "loose", + "version": 2 + }, + "conditions": [ + { + "id": "o1", + "leftValue": "={{ $json.output }}", + "rightValue": "", + "operator": { + "type": "string", + "operation": "notEmpty", + "singleValue": true + } + } + ], + "combinator": "and" + }, + "options": {} + } + }, + { + "id": "t3", + "name": "Qwen Agent", + "type": "@n8n/n8n-nodes-langchain.agent", + "typeVersion": 3.1, + "position": [ + 2200, + -200 + ], + "parameters": { + "promptType": "define", + "text": "={{ $('Route').first().json.prompt }}", + "options": { + "systemMessage": "You are a concise assistant chatting with the user on a phone. Answer in the language of the user and keep answers short. For questions about current events, weather, news, prices or anything you cannot know for sure, call the web_search tool first with a short search query, then answer from the results and name the source URLs. When the user gives a web address, or a search result needs more detail, call the fetch_page tool with the full URL and base your answer only on what it returns. Earlier messages in this conversation (for example a digest you sent) are available as context for follow-up questions. If a tool fails, say so. Do not invent facts or page content. Call a tool only when needed: one fetch_page per web address, or one web_search per question; never repeat the same call. Always fetch every web address the user names.", + "maxIterations": 4 + } + }, + "onError": "continueRegularOutput" + }, + { + "id": "t4", + "name": "Qwen (Ollama)", + "type": "@n8n/n8n-nodes-langchain.lmChatOllama", + "typeVersion": 1, + "position": [ + 2100, + 60 + ], + "parameters": { + "model": "__OLLAMA_MODEL__", + "options": { + "numCtx": 16384, + "temperature": 0.3, + "numThread": 4 + } + }, + "credentials": { + "ollamaApi": { + "id": "ollama0000000001", + "name": "Ollama (local)" + } + } + }, + { + "id": "t6", + "name": "fetch_page (qwen)", + "type": "@n8n/n8n-nodes-langchain.toolWorkflow", + "typeVersion": 2.2, + "position": [ + 2360, + 60 + ], + "parameters": { + "name": "fetch_page", + "description": "Opens a web page in a real browser (JavaScript is executed) and returns its title and readable text, truncated to about 3000 characters. Input: the full URL.", + "workflowId": { + "__rl": true, + "value": "fetchpage0000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "url": "={{ $fromAI('url', 'The full URL of the page to open, including https://', 'string') }}" + }, + "matchingColumns": [], + "schema": [ + { + "id": "url", + "displayName": "url", + "required": false, + "defaultMatch": false, + "display": true, + "canBeUsedToMatch": true, + "type": "string" + } + ], + "attemptToConvertTypes": false, + "convertFieldsToString": false + } + } + }, + { + "id": "t7", + "name": "web_search (qwen)", + "type": "@n8n/n8n-nodes-langchain.toolWorkflow", + "typeVersion": 2.2, + "position": [ + 2540, + 60 + ], + "parameters": { + "name": "web_search", + "description": "Searches the web and returns the top results (title, URL, short snippet). Input: a short search query.", + "workflowId": { + "__rl": true, + "value": "websearch000001", + "mode": "id" + }, + "workflowInputs": { + "mappingMode": "defineBelow", + "value": { + "query": "={{ $fromAI('query', 'A short web search query', 'string') }}" + }, + "matchingColumns": [], + "schema": [ + { + "id": "query", + "displayName": "query", + "required": false, + "defaultMatch": false, + "display": true, + "canBeUsedToMatch": true, + "type": "string" + } + ], + "attemptToConvertTypes": false, + "convertFieldsToString": false + } + } + }, + { + "id": "t5", + "name": "Window Buffer Memory", + "type": "@n8n/n8n-nodes-langchain.memoryBufferWindow", + "typeVersion": 1.3, + "position": [ + 1130, + 360 + ], + "parameters": { + "sessionIdType": "customKey", + "sessionKey": "telegram-main", + "contextWindowLength": 8 + } + }, + { + "id": "t29", + "name": "Qwen Memory", + "type": "@n8n/n8n-nodes-langchain.memoryBufferWindow", + "typeVersion": 1.3, + "position": [ + 2300, + 360 + ], + "parameters": { + "sessionIdType": "customKey", + "sessionKey": "telegram-qwen", + "contextWindowLength": 2 + } + }, + { + "id": "t18", + "name": "Format reply", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [ + 2800, + 0 + ], + "parameters": { + "mode": "runOnceForEachItem", + "jsCode": "// Telegram's Markdown parser rejects most model output; send HTML instead.\n// Everything is escaped first, then a few Markdown patterns are turned into allowed tags.\nlet raw = String($json.output || ('Error: ' + ($json.error || 'no answer')));\n// Tell the user when a fallback model answered\nif ($('Gemini 3.5 Agent').isExecuted) {\n raw += $('Qwen Agent').isExecuted\n ? '\\n\\n(Gemini was unavailable, answered by the local Qwen model)'\n : '\\n\\n(Gemini 3.8 was unavailable, answered by gemini-3.5-flash)';\n}\nraw = raw.slice(0, 3800);\nconst esc = (s) => s.replace(/&/g, '&').replace(//g, '>');\nlet t = esc(raw);\nt = t.replace(/```[a-zA-Z]*\\n?([\\s\\S]*?)```/g, '
$1
');\nt = t.replace(/`([^`\\n]+)`/g, '$1');\nt = t.replace(/\\*\\*([^*\\n]+)\\*\\*/g, '$1');\nt = t.replace(/(^|[\\s(])\\*(\\S(?:[^*\\n]*\\S)?)\\*(?=[\\s).,!?:;]|$)/gm, '$1$2');\nt = t.replace(/^#{1,6}\\s+(.*)$/gm, '$1');\nt = t.replace(/^[ \\t]*[*-][ \\t]+/gm, '\\u2022 ');\nt = t.replace(/\\[([^\\]\\n]+)\\]\\((https?:\\/\\/[^)\\s\"]+)\\)/g, '$1');\n// One Telegram update per execution, so the first Route item is the right one\nreturn { json: { text: t, chat_id: $('Route').first().json.chat_id } };\n" + } + }, + { + "id": "t8", + "name": "Send reply", + "type": "n8n-nodes-base.telegram", + "typeVersion": 1.2, + "position": [ + 3040, + 0 + ], + "parameters": { + "resource": "message", + "operation": "sendMessage", + "chatId": "={{ $json.chat_id }}", + "text": "={{ $json.text }}", + "additionalFields": { + "appendAttribution": false, + "parse_mode": "HTML" + } + }, + "credentials": { + "telegramApi": { + "id": "telegram0000001", + "name": "Telegram bot" + } + } + }, + { + "id": "t9", + "name": "Post message", + "type": "n8n-nodes-base.executeWorkflowTrigger", + "typeVersion": 1.1, + "position": [ + 0, + 500 + ], + "parameters": { + "inputSource": "workflowInputs", + "workflowInputs": { + "values": [ + { + "name": "message", + "type": "string" + } + ] + } + } + }, + { + "id": "t10", + "name": "Add to chat memory", + "type": "@n8n/n8n-nodes-langchain.memoryManager", + "typeVersion": 1.1, + "position": [ + 260, + 500 + ], + "parameters": { + "mode": "insert", + "insertMode": "insert", + "messages": { + "messageValues": [ + { + "type": "ai", + "message": "={{ $json.message }}", + "hideFromUI": false + } + ] + } + } + }, + { + "id": "t19", + "name": "Format posted message", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [ + 520, + 500 + ], + "parameters": { + "mode": "runOnceForEachItem", + "jsCode": "// Telegram's Markdown parser rejects most model output; send HTML instead.\n// Everything is escaped first, then a few Markdown patterns are turned into allowed tags.\nconst raw = String($('Post message').first().json.message).slice(0, 3800);\nconst esc = (s) => s.replace(/&/g, '&').replace(//g, '>');\nlet t = esc(raw);\nt = t.replace(/```[a-zA-Z]*\\n?([\\s\\S]*?)```/g, '
$1
');\nt = t.replace(/`([^`\\n]+)`/g, '$1');\nt = t.replace(/\\*\\*([^*\\n]+)\\*\\*/g, '$1');\nt = t.replace(/^#{1,6}\\s+(.*)$/gm, '$1');\nt = t.replace(/^[ \\t]*[*-][ \\t]+/gm, '\\u2022 ');\nt = t.replace(/\\[([^\\]\\n]+)\\]\\((https?:\\/\\/[^)\\s\"]+)\\)/g, '$1');\nreturn { json: { text: t } };\n" + } + }, + { + "id": "t11", + "name": "Send posted message", + "type": "n8n-nodes-base.telegram", + "typeVersion": 1.2, + "position": [ + 1000, + 500 + ], + "parameters": { + "resource": "message", + "operation": "sendMessage", + "chatId": "__TELEGRAM_USER_ID__", + "text": "={{ $json.text }}", + "additionalFields": { + "appendAttribution": false, + "parse_mode": "HTML" + } + }, + "credentials": { + "telegramApi": { + "id": "telegram0000001", + "name": "Telegram bot" + } + } + } + ], + "connections": { + "Telegram Trigger": { + "main": [ + [ + { + "node": "Only my user", + "type": "main", + "index": 0 + } + ] + ] + }, + "Only my user": { + "main": [ + [ + { + "node": "Route", + "type": "main", + "index": 0 + } + ], + [] + ] + }, + "Route": { + "main": [ + [ + { + "node": "Use Qwen?", + "type": "main", + "index": 0 + } + ] + ] + }, + "Use Qwen?": { + "main": [ + [ + { + "node": "Qwen Agent", + "type": "main", + "index": 0 + } + ], + [ + { + "node": "Gemini 3.8 Agent", + "type": "main", + "index": 0 + } + ] + ] + }, + "Gemini 3.8 Agent": { + "main": [ + [ + { + "node": "Gemini 3.8 ok?", + "type": "main", + "index": 0 + } + ] + ] + }, + "Gemini 3.8 ok?": { + "main": [ + [ + { + "node": "Format reply", + "type": "main", + "index": 0 + } + ], + [ + { + "node": "Gemini 3.5 Agent", + "type": "main", + "index": 0 + } + ] + ] + }, + "Gemini 3.5 Agent": { + "main": [ + [ + { + "node": "Gemini 3.5 ok?", + "type": "main", + "index": 0 + } + ] + ] + }, + "Gemini 3.5 ok?": { + "main": [ + [ + { + "node": "Format reply", + "type": "main", + "index": 0 + } + ], + [ + { + "node": "Qwen Agent", + "type": "main", + "index": 0 + } + ] + ] + }, + "Qwen Agent": { + "main": [ + [ + { + "node": "Format reply", + "type": "main", + "index": 0 + } + ] + ] + }, + "Format reply": { + "main": [ + [ + { + "node": "Send reply", + "type": "main", + "index": 0 + } + ] + ] + }, + "Post message": { + "main": [ + [ + { + "node": "Add to chat memory", + "type": "main", + "index": 0 + } + ] + ] + }, + "Add to chat memory": { + "main": [ + [ + { + "node": "Format posted message", + "type": "main", + "index": 0 + } + ] + ] + }, + "Format posted message": { + "main": [ + [ + { + "node": "Send posted message", + "type": "main", + "index": 0 + } + ] + ] + }, + "Gemini 3.8": { + "ai_languageModel": [ + [ + { + "node": "Gemini 3.8 Agent", + "type": "ai_languageModel", + "index": 0 + } + ] + ] + }, + "Gemini 3.5": { + "ai_languageModel": [ + [ + { + "node": "Gemini 3.5 Agent", + "type": "ai_languageModel", + "index": 0 + } + ] + ] + }, + "Qwen (Ollama)": { + "ai_languageModel": [ + [ + { + "node": "Qwen Agent", + "type": "ai_languageModel", + "index": 0 + } + ] + ] + }, + "fetch_page (gemini 3.8)": { + "ai_tool": [ + [ + { + "node": "Gemini 3.8 Agent", + "type": "ai_tool", + "index": 0 + } + ] + ] + }, + "web_search (gemini 3.8)": { + "ai_tool": [ + [ + { + "node": "Gemini 3.8 Agent", + "type": "ai_tool", + "index": 0 + } + ] + ] + }, + "fetch_page (gemini 3.5)": { + "ai_tool": [ + [ + { + "node": "Gemini 3.5 Agent", + "type": "ai_tool", + "index": 0 + } + ] + ] + }, + "web_search (gemini 3.5)": { + "ai_tool": [ + [ + { + "node": "Gemini 3.5 Agent", + "type": "ai_tool", + "index": 0 + } + ] + ] + }, + "fetch_page (qwen)": { + "ai_tool": [ + [ + { + "node": "Qwen Agent", + "type": "ai_tool", + "index": 0 + } + ] + ] + }, + "web_search (qwen)": { + "ai_tool": [ + [ + { + "node": "Qwen Agent", + "type": "ai_tool", + "index": 0 + } + ] + ] + }, + "Window Buffer Memory": { + "ai_memory": [ + [ + { + "node": "Gemini 3.8 Agent", + "type": "ai_memory", + "index": 0 + }, + { + "node": "Gemini 3.5 Agent", + "type": "ai_memory", + "index": 0 + }, + { + "node": "Add to chat memory", + "type": "ai_memory", + "index": 0 + } + ] + ] + }, + "Stock price schedule": { + "main": [ + [ + { + "node": "Scheduled question", + "type": "main", + "index": 0 + } + ] + ] + }, + "Scheduled question": { + "main": [ + [ + { + "node": "Route", + "type": "main", + "index": 0 + } + ] + ] + }, + "Run stock price now": { + "main": [ + [ + { + "node": "Scheduled question", + "type": "main", + "index": 0 + } + ] + ] + }, + "Qwen Memory": { + "ai_memory": [ + [ + { + "node": "Qwen Agent", + "type": "ai_memory", + "index": 0 + } + ] + ] + } + }, + "settings": { + "executionOrder": "v1" + } +} diff --git a/ansible/roles/n8n/files/workflows/web-search.json b/ansible/roles/n8n/files/workflows/web-search.json new file mode 100644 index 0000000..90c408c --- /dev/null +++ b/ansible/roles/n8n/files/workflows/web-search.json @@ -0,0 +1,98 @@ +{ + "id": "websearch000001", + "name": "Web search", + "active": true, + "nodes": [ + { + "id": "s1", + "name": "When called by another workflow", + "type": "n8n-nodes-base.executeWorkflowTrigger", + "typeVersion": 1.1, + "position": [ + 0, + 0 + ], + "parameters": { + "inputSource": "workflowInputs", + "workflowInputs": { + "values": [ + { + "name": "query", + "type": "string" + } + ] + } + } + }, + { + "id": "s2", + "name": "Query SearXNG", + "type": "n8n-nodes-base.httpRequest", + "typeVersion": 4.2, + "position": [ + 260, + 0 + ], + "parameters": { + "method": "GET", + "url": "http://searxng:8080/search", + "sendQuery": true, + "queryParameters": { + "parameters": [ + { + "name": "q", + "value": "={{ $json.query }}" + }, + { + "name": "format", + "value": "json" + } + ] + }, + "options": { + "timeout": 30000 + } + } + }, + { + "id": "s3", + "name": "Format results", + "type": "n8n-nodes-base.code", + "typeVersion": 2, + "position": [ + 520, + 0 + ], + "parameters": { + "jsCode": "const results = ($input.first().json.results || []).slice(0, 6);\nif (!results.length) return [{ json: { response: 'No search results found.' } }];\nconst text = results.map((r, i) => (i + 1) + '. ' + r.title + '\\n ' + r.url + '\\n ' + (r.content || '').slice(0, 300)).join('\\n\\n');\nreturn [{ json: { response: text.slice(0, 3500) } }];\n" + } + } + ], + "connections": { + "When called by another workflow": { + "main": [ + [ + { + "node": "Query SearXNG", + "type": "main", + "index": 0 + } + ] + ] + }, + "Query SearXNG": { + "main": [ + [ + { + "node": "Format results", + "type": "main", + "index": 0 + } + ] + ] + } + }, + "settings": { + "executionOrder": "v1" + } +} diff --git a/ansible/roles/n8n/handlers/main.yml b/ansible/roles/n8n/handlers/main.yml new file mode 100644 index 0000000..df39383 --- /dev/null +++ b/ansible/roles/n8n/handlers/main.yml @@ -0,0 +1,42 @@ +--- +- name: Restart n8n + community.docker.docker_compose_v2: + project_src: "{{ n8n_data_dir }}" + state: present + recreate: always + +# "Import n8n content" is a topic, not a single handler: the tasks below all listen +# on it and run in the order they are defined (handlers cannot be blocks). +# Credentials first (workflows reference them by id), then workflows. +# Imported workflows start inactive; the ones marked "active": true are published. +# n8n only registers triggers/webhooks at startup, so restart afterwards. +- name: Wait for n8n to be ready (database migrations finished) + ansible.builtin.command: docker exec n8n wget -qO- http://localhost:5678/healthz/readiness + listen: Import n8n content + register: n8n_ready + changed_when: false + retries: 40 + delay: 3 + until: n8n_ready.rc == 0 + +- name: Import n8n credentials + ansible.builtin.command: docker exec n8n n8n import:credentials --separate --input=/import/credentials + listen: Import n8n content + no_log: true + changed_when: true + +- name: Import n8n workflows + ansible.builtin.command: docker exec n8n n8n import:workflow --separate --input=/import/workflows + listen: Import n8n content + changed_when: true + +- name: Publish active n8n workflows + ansible.builtin.command: docker exec n8n n8n publish:workflow --id={{ item }} + listen: Import n8n content + loop: "{{ n8n_publish_ids | from_json }}" + changed_when: true + +- name: Restart n8n to register triggers + ansible.builtin.command: docker restart n8n + listen: Import n8n content + changed_when: true diff --git a/ansible/roles/n8n/tasks/main.yml b/ansible/roles/n8n/tasks/main.yml new file mode 100644 index 0000000..fa6838e --- /dev/null +++ b/ansible/roles/n8n/tasks/main.yml @@ -0,0 +1,106 @@ +--- +- name: Create n8n directories + ansible.builtin.file: + path: "{{ item.path }}" + state: directory + owner: "{{ item.owner }}" + group: "{{ item.owner }}" + mode: "{{ item.mode }}" + loop: + - { path: "{{ n8n_data_dir }}", owner: root, mode: "0755" } + # n8n runs as uid 1000 (node) inside the container + - { path: "{{ n8n_data_dir }}/data", owner: "1000", mode: "0700" } + - { path: "{{ n8n_data_dir }}/import", owner: "1000", mode: "0700" } + - { path: "{{ n8n_data_dir }}/import/workflows", owner: "1000", mode: "0700" } + - { path: "{{ n8n_data_dir }}/import/credentials", owner: "1000", mode: "0700" } + tags: n8n + +- name: Deploy Docker Compose file + ansible.builtin.template: + src: docker-compose.yml.j2 + dest: "{{ n8n_data_dir }}/docker-compose.yml" + owner: root + group: root + mode: "0644" + notify: Restart n8n + tags: n8n + +# __TELEGRAM_USER_ID__ and __OLLAMA_MODEL__ in the workflow files are replaced by telegram_user_id and ollama_model +- name: Deploy workflows + ansible.builtin.copy: + content: "{{ lookup('ansible.builtin.file', item) | replace('__TELEGRAM_USER_ID__', telegram_user_id | string) | replace('__OLLAMA_MODEL__', ollama_model) }}" + dest: "{{ n8n_data_dir }}/import/workflows/{{ item | basename }}" + owner: "1000" + group: "1000" + mode: "0600" + with_fileglob: + - "workflows/*.json" + notify: Import n8n content + tags: n8n + +- name: Deploy Ollama credential + ansible.builtin.copy: + src: credentials/ollama.json + dest: "{{ n8n_data_dir }}/import/credentials/ollama.json" + owner: "1000" + group: "1000" + mode: "0600" + notify: Import n8n content + tags: n8n + +- name: Deploy SMTP credential + ansible.builtin.template: + src: smtp-credential.json.j2 + dest: "{{ n8n_data_dir }}/import/credentials/smtp.json" + owner: "1000" + group: "1000" + mode: "0600" + no_log: true + notify: Import n8n content + tags: n8n + +# Workflows with "active": true in their JSON are published after import +- name: Collect workflow ids to publish + ansible.builtin.set_fact: + n8n_publish_ids: >- + {%- set ns = namespace(ids=[]) -%} + {%- for f in query('fileglob', role_path ~ '/files/workflows/*.json') -%} + {%- set wf = lookup('file', f) | from_json -%} + {%- if wf.active | default(false) -%}{%- set ns.ids = ns.ids + [wf.id] -%}{%- endif -%} + {%- endfor -%} + {{ ns.ids | to_json }} + tags: n8n + +- name: Deploy Telegram credential + ansible.builtin.template: + src: telegram-credential.json.j2 + dest: "{{ n8n_data_dir }}/import/credentials/telegram.json" + owner: "1000" + group: "1000" + mode: "0600" + no_log: true + notify: Import n8n content + tags: n8n + +- name: Deploy Gemini credential + ansible.builtin.template: + src: gemini-credential.json.j2 + dest: "{{ n8n_data_dir }}/import/credentials/gemini.json" + owner: "1000" + group: "1000" + mode: "0600" + no_log: true + notify: Import n8n content + tags: n8n + +- name: Start n8n + community.docker.docker_compose_v2: + project_src: "{{ n8n_data_dir }}" + state: present + pull: missing + tags: n8n + +# Handlers run at the end of the play; flush so the import happens before the role finishes +- name: Apply pending handlers + ansible.builtin.meta: flush_handlers + tags: n8n diff --git a/ansible/roles/n8n/templates/docker-compose.yml.j2 b/ansible/roles/n8n/templates/docker-compose.yml.j2 new file mode 100644 index 0000000..167a58f --- /dev/null +++ b/ansible/roles/n8n/templates/docker-compose.yml.j2 @@ -0,0 +1,77 @@ +# Managed by Ansible — do not edit manually +services: + n8n: + image: n8nio/n8n:{{ n8n_version }} + container_name: n8n + restart: unless-stopped + environment: + N8N_HOST: "{{ domain_n8n }}" + N8N_PROTOCOL: "https" + N8N_PORT: "5678" + # Public URL used to build webhook / chat links behind Traefik + WEBHOOK_URL: "https://{{ domain_n8n }}/" + N8N_PROXY_HOPS: "1" + N8N_ENCRYPTION_KEY: "{{ n8n_encryption_key }}" + GENERIC_TIMEZONE: "{{ n8n_timezone }}" + TZ: "{{ n8n_timezone }}" + N8N_DIAGNOSTICS_ENABLED: "false" + N8N_VERSION_NOTIFICATIONS_ENABLED: "false" + N8N_TEMPLATES_ENABLED: "false" + # Keep the SQLite database small: drop execution history after 7 days + EXECUTIONS_DATA_PRUNE: "true" + EXECUTIONS_DATA_MAX_AGE: "168" + volumes: + - {{ n8n_data_dir }}/data:/home/node/.n8n + # Workflows and credentials imported by Ansible (see roles/n8n/tasks/main.yml) + - {{ n8n_data_dir }}/import:/import:ro + mem_limit: {{ n8n_mem_limit }} + networks: + - traefik_public + - ollama_internal + - browserless_internal + - searxng_internal + labels: + - "traefik.enable=true" + - "traefik.http.services.n8n.loadbalancer.server.port=5678" + # Editor, webhooks and chat page — all behind basic auth + - "traefik.http.routers.n8n.rule=Host(`{{ domain_n8n }}`)" + - "traefik.http.routers.n8n.entrypoints=websecure" + - "traefik.http.routers.n8n.tls.certresolver=letsencrypt" + - "traefik.http.routers.n8n.service=n8n" + - "traefik.http.routers.n8n.middlewares=n8n-auth" + # Short link: /chat redirects to the hosted chat page of the chat workflow + - "traefik.http.routers.n8n-chat.rule=Host(`{{ domain_n8n }}`) && Path(`/chat`)" + - "traefik.http.routers.n8n-chat.priority=100" + - "traefik.http.routers.n8n-chat.entrypoints=websecure" + - "traefik.http.routers.n8n-chat.tls.certresolver=letsencrypt" + - "traefik.http.routers.n8n-chat.service=n8n" + - "traefik.http.routers.n8n-chat.middlewares=n8n-auth,n8n-chat-redirect" + - "traefik.http.middlewares.n8n-chat-redirect.redirectregex.regex=^https://{{ domain_n8n | replace('.', '\\\\.') }}/chat/?$$" + - "traefik.http.middlewares.n8n-chat-redirect.redirectregex.replacement=https://{{ domain_n8n }}/webhook/{{ n8n_chat_webhook_id }}/chat" + # Telegram bot webhook: called by Telegram's servers, so no basic auth. Restricted to Telegram's + # address ranges; n8n additionally checks Telegram's secret token header and the workflow + # only answers the user id in telegram_user_id. + - "traefik.http.routers.n8n-telegram.rule=Host(`{{ domain_n8n }}`) && Path(`/webhook/{{ n8n_telegram_webhook_id }}/webhook`) && Method(`POST`)" + - "traefik.http.routers.n8n-telegram.priority=200" + - "traefik.http.routers.n8n-telegram.entrypoints=websecure" + - "traefik.http.routers.n8n-telegram.tls.certresolver=letsencrypt" + - "traefik.http.routers.n8n-telegram.service=n8n" +{% if telegram_webhook_source_ranges | length > 0 %} + - "traefik.http.routers.n8n-telegram.middlewares=n8n-telegram-ips" + - "traefik.http.middlewares.n8n-telegram-ips.ipallowlist.sourcerange={{ telegram_webhook_source_ranges | join(',') }}" +{% endif %} +{% set ns = namespace(entries=[]) %} +{% for user in n8n_users %} +{% set ns.entries = ns.entries + [user.username + ':' + (user.password | password_hash('bcrypt', (user.username | hash('md5'))[:22]) | replace('$', '$$'))] %} +{% endfor %} + - "traefik.http.middlewares.n8n-auth.basicauth.users={{ ns.entries | join(',') }}" + +networks: + traefik_public: + external: true + ollama_internal: + external: true + browserless_internal: + external: true + searxng_internal: + external: true diff --git a/ansible/roles/n8n/templates/gemini-credential.json.j2 b/ansible/roles/n8n/templates/gemini-credential.json.j2 new file mode 100644 index 0000000..e88bd0d --- /dev/null +++ b/ansible/roles/n8n/templates/gemini-credential.json.j2 @@ -0,0 +1,9 @@ +{ + "id": "gemini00000001", + "name": "Google Gemini", + "type": "googlePalmApi", + "data": { + "host": "https://generativelanguage.googleapis.com", + "apiKey": {{ gemini_api_key | to_json }} + } +} diff --git a/ansible/roles/n8n/templates/smtp-credential.json.j2 b/ansible/roles/n8n/templates/smtp-credential.json.j2 new file mode 100644 index 0000000..4985e18 --- /dev/null +++ b/ansible/roles/n8n/templates/smtp-credential.json.j2 @@ -0,0 +1,14 @@ +{ + "id": "smtp00000000001", + "name": "Strato SMTP", + "type": "smtp", + "data": { + "user": {{ smtp_user | to_json }}, + "password": {{ smtp_password | to_json }}, + "host": {{ smtp_host | to_json }}, + "port": {{ smtp_port }}, + "secure": {{ smtp_secure | to_json }}, + "disableStartTls": false, + "hostName": "" + } +} diff --git a/ansible/roles/n8n/templates/telegram-credential.json.j2 b/ansible/roles/n8n/templates/telegram-credential.json.j2 new file mode 100644 index 0000000..39663c8 --- /dev/null +++ b/ansible/roles/n8n/templates/telegram-credential.json.j2 @@ -0,0 +1,9 @@ +{ + "id": "telegram0000001", + "name": "Telegram bot", + "type": "telegramApi", + "data": { + "accessToken": {{ telegram_bot_token | to_json }}, + "baseUrl": "https://api.telegram.org" + } +} diff --git a/ansible/roles/ollama/handlers/main.yml b/ansible/roles/ollama/handlers/main.yml new file mode 100644 index 0000000..45179c1 --- /dev/null +++ b/ansible/roles/ollama/handlers/main.yml @@ -0,0 +1,6 @@ +--- +- name: Restart ollama + community.docker.docker_compose_v2: + project_src: "{{ ollama_data_dir }}" + state: present + recreate: always diff --git a/ansible/roles/ollama/tasks/main.yml b/ansible/roles/ollama/tasks/main.yml new file mode 100644 index 0000000..690823e --- /dev/null +++ b/ansible/roles/ollama/tasks/main.yml @@ -0,0 +1,44 @@ +--- +- name: Create Ollama directories + ansible.builtin.file: + path: "{{ item }}" + state: directory + owner: root + group: root + mode: "0755" + loop: + - "{{ ollama_data_dir }}" + - "{{ ollama_data_dir }}/models" + tags: ollama + +- name: Deploy Docker Compose file + ansible.builtin.template: + src: docker-compose.yml.j2 + dest: "{{ ollama_data_dir }}/docker-compose.yml" + owner: root + group: root + mode: "0644" + notify: Restart ollama + tags: ollama + +- name: Start Ollama + community.docker.docker_compose_v2: + project_src: "{{ ollama_data_dir }}" + state: present + pull: missing + tags: ollama + +- name: List installed Ollama models + ansible.builtin.command: docker exec ollama ollama list + register: ollama_installed + changed_when: false + retries: 10 + delay: 3 + until: ollama_installed.rc == 0 + tags: ollama + +- name: Pull Ollama model + ansible.builtin.command: docker exec ollama ollama pull {{ ollama_model }} + when: ollama_model not in ollama_installed.stdout + changed_when: true + tags: ollama diff --git a/ansible/roles/ollama/templates/docker-compose.yml.j2 b/ansible/roles/ollama/templates/docker-compose.yml.j2 new file mode 100644 index 0000000..b6d961e --- /dev/null +++ b/ansible/roles/ollama/templates/docker-compose.yml.j2 @@ -0,0 +1,26 @@ +# Managed by Ansible — do not edit manually +services: + ollama: + image: ollama/ollama:{{ ollama_version }} + container_name: ollama + restart: unless-stopped + environment: + OLLAMA_CONTEXT_LENGTH: "{{ ollama_context_length }}" + OLLAMA_FLASH_ATTENTION: "1" + OLLAMA_KV_CACHE_TYPE: "q8_0" + # One model, one request at a time — keeps RAM use predictable on the shared VPS + OLLAMA_MAX_LOADED_MODELS: "1" + OLLAMA_NUM_PARALLEL: "1" + # Unload the model after this idle time to free RAM for the other services + OLLAMA_KEEP_ALIVE: "{{ ollama_keep_alive }}" + volumes: + - {{ ollama_data_dir }}/models:/root/.ollama + mem_limit: {{ ollama_mem_limit }} + networks: + - ollama_internal + +# Not exposed via Traefik. Containers that need the API join this network +# (declared as external there) and reach it at http://ollama:11434. +networks: + ollama_internal: + name: ollama_internal diff --git a/ansible/roles/searxng/handlers/main.yml b/ansible/roles/searxng/handlers/main.yml new file mode 100644 index 0000000..6e10550 --- /dev/null +++ b/ansible/roles/searxng/handlers/main.yml @@ -0,0 +1,6 @@ +--- +- name: Restart searxng + community.docker.docker_compose_v2: + project_src: "{{ searxng_data_dir }}" + state: present + recreate: always diff --git a/ansible/roles/searxng/tasks/main.yml b/ansible/roles/searxng/tasks/main.yml new file mode 100644 index 0000000..64bb7ad --- /dev/null +++ b/ansible/roles/searxng/tasks/main.yml @@ -0,0 +1,37 @@ +--- +- name: Create SearXNG directory + ansible.builtin.file: + path: "{{ searxng_data_dir }}" + state: directory + owner: root + group: root + mode: "0755" + tags: searxng + +- name: Deploy SearXNG settings + ansible.builtin.template: + src: settings.yml.j2 + dest: "{{ searxng_data_dir }}/settings.yml" + owner: "977" + group: "977" + mode: "0640" + no_log: true + notify: Restart searxng + tags: searxng + +- name: Deploy Docker Compose file + ansible.builtin.template: + src: docker-compose.yml.j2 + dest: "{{ searxng_data_dir }}/docker-compose.yml" + owner: root + group: root + mode: "0644" + notify: Restart searxng + tags: searxng + +- name: Start SearXNG + community.docker.docker_compose_v2: + project_src: "{{ searxng_data_dir }}" + state: present + pull: missing + tags: searxng diff --git a/ansible/roles/searxng/templates/docker-compose.yml.j2 b/ansible/roles/searxng/templates/docker-compose.yml.j2 new file mode 100644 index 0000000..91d67d0 --- /dev/null +++ b/ansible/roles/searxng/templates/docker-compose.yml.j2 @@ -0,0 +1,19 @@ +# Managed by Ansible — do not edit manually +services: + searxng: + image: searxng/searxng:{{ searxng_version }} + container_name: searxng + restart: unless-stopped + environment: + SEARXNG_BASE_URL: "http://searxng:8080/" + volumes: + - {{ searxng_data_dir }}/settings.yml:/etc/searxng/settings.yml:ro + mem_limit: {{ searxng_mem_limit }} + networks: + - searxng_internal + +# Not exposed via Traefik. Containers that need the API join this network +# (declared as external there) and reach it at http://searxng:8080. +networks: + searxng_internal: + name: searxng_internal diff --git a/ansible/roles/searxng/templates/settings.yml.j2 b/ansible/roles/searxng/templates/settings.yml.j2 new file mode 100644 index 0000000..6d6fc83 --- /dev/null +++ b/ansible/roles/searxng/templates/settings.yml.j2 @@ -0,0 +1,24 @@ +# Managed by Ansible — do not edit manually +use_default_settings: true + +general: + instance_name: "search" + enable_metrics: false + +server: + secret_key: "{{ searxng_secret_key }}" + # Internal use only (no public route), so the bot limiter is not needed + limiter: false + image_proxy: false + +search: + # json is required by the n8n web_search tool + formats: + - html + - json + safe_search: 0 + autocomplete: "" + +outgoing: + request_timeout: 6.0 + max_request_timeout: 10.0 diff --git a/ansible/roles/traefik/templates/traefik.yml.j2 b/ansible/roles/traefik/templates/traefik.yml.j2 index 0da0fde..10e7efb 100644 --- a/ansible/roles/traefik/templates/traefik.yml.j2 +++ b/ansible/roles/traefik/templates/traefik.yml.j2 @@ -30,9 +30,6 @@ entryPoints: imaps: address: ":993" -serversTransport: - insecureSkipVerify: true # allows Traefik to proxy clay's self-signed d.clay.studio cert on port 2633 - providers: docker: exposedByDefault: false diff --git a/ansible/site.yml b/ansible/site.yml index 9259bae..dc662cc 100644 --- a/ansible/site.yml +++ b/ansible/site.yml @@ -15,5 +15,8 @@ - k8s - vaultwarden - dl - - clay + - ollama + - browserless + - searxng + - n8n - dashboard diff --git a/docs/runbook-configuration.md b/docs/runbook-configuration.md index 01b2a4d..61e55f4 100644 --- a/docs/runbook-configuration.md +++ b/docs/runbook-configuration.md @@ -25,7 +25,8 @@ The script checks these endpoints and verifies the expected HTTP status code: | `https://k8s.ladkau.de` | 200 | Placeholder page | | `https://vault.ladkau.de` | 200 | Vaultwarden web vault | | `https://dl.ladkau.de` | 200 | Public download server | -| `https://agent.ladkau.de` | 401 | Clay — basic-auth prompt, correct without credentials | +| `https://agent.ladkau.de` | 401 | n8n — basic-auth prompt, correct without credentials | +| `https://agent.ladkau.de/chat` | 401 | AI chat — basic-auth prompt, correct without credentials | A `000` result means the connection was refused or timed out — a container that did not start. Keycloak and Nextcloud may return `502` for up to 90 seconds on @@ -470,163 +471,268 @@ Traefik routes `/v2/` (Docker API) to the registry container and everything else to the UI container — both on the same domain, so no CORS configuration is needed. `docker push` and `docker pull` work exactly as before. -## 9. Clay — Claude Code web interface +## 9. n8n — AI chat and scheduled digests -Clay is a browser-based front end for Claude Code, served at -`https://agent.ladkau.de`. Access is protected by HTTP basic auth using the -`clay_users` credentials from the vault. +n8n is served at `https://agent.ladkau.de` behind HTTP basic auth +(`n8n_users` from the vault). It provides: -### 9.1 Add projects +- an AI chat page at `https://agent.ladkau.de/chat` that can search the web and read web pages +- a Telegram bot that answers with Gemini (default) or the local Qwen model (`/qwen`), and that scheduled workflows can also post to +- scheduled workflows (scrape a page, summarise it with the local model, send the summary by mail) -The container mounts `/opt/clay/workspace` at `/workspace`. Clone Gitea -repositories there to make them available to Claude Code: +Components: `n8n` (workflows), `browserless` (headless Chromium for +JavaScript-heavy pages), `searxng` (web search) and `ollama` (local model, +section 9.5). Ollama, browserless and SearXNG are reachable only on internal +Docker networks. + +### 9.1 First login + +Open `https://agent.ladkau.de`, pass the basic-auth prompt and create the n8n +owner account. The Ollama credential ("Ollama (local)") and the SMTP +credential ("Strato SMTP") are imported by Ansible; nothing needs to be +entered in the UI. + +### 9.2 Chat + +`https://agent.ladkau.de/chat` redirects to the hosted chat page of the +`Chat` workflow (`/webhook//chat`). The chat is an AI +agent with two tools: + +- `web_search` — queries SearXNG and returns the top 6 results (title, URL, snippet) +- `fetch_page` — opens a URL in browserless and returns the readable text + +- Every page load starts a new conversation. The last 6 messages are kept as context; history is held in memory and is lost when n8n restarts. +- `fetch_page` returns at most about 3000 characters of page text (title plus visible text without navigation). Tool results are kept in the chat memory. + +### 9.3 Workflows in git + +Workflows live in `ansible/roles/n8n/files/workflows/*.json` and are imported +by Ansible on every deploy that changes them: + +| File | Workflow | Purpose | +|------|----------|---------| +| `chat.json` | Chat | Hosted chat page with the `fetch_page` tool | +| `fetch-page.json` | Fetch page text | Sub-workflow: renders a URL in browserless and returns title and text | +| `web-search.json` | Web search | Sub-workflow: queries SearXNG and returns the top results | +| `digest.json` | Weekly page digest (example) | Schedule → fetch page → summarise → mail; inactive until configured | +| `telegram-chat.json` | Telegram chat | Telegram bot: Gemini agent by default, Qwen agent for `/qwen`, both with the search and page tools; also has a `Post message` entry point used by other workflows | +| `digest-telegram.json` | Weekly page digest to Telegram (example) | Schedule → fetch page → summarise → post to the Telegram chat; inactive until configured | + +A workflow with `"active": true` in its JSON is published on import. Sub-workflows +that are used as tools (`fetch-page`, `web-search`) must be published too; n8n +refuses to run an unpublished workflow, even when another workflow calls it. The +import replaces workflows with the same id, so changes made in the editor must +be exported to the repo or they are overwritten by the next import: + +1. In the n8n editor open the workflow, then **… → Download** (or copy it with Ctrl+A, Ctrl+C). +2. Save the JSON as `ansible/roles/n8n/files/workflows/.json`. Keep the `id` and, for the chat workflow, the `webhookId`. +3. Set `"active": true` in the file if the workflow should be published. +4. Commit, then deploy with `--tags n8n`. + +Workflows call each other by id (`fetchpage0000001`, `websearch000001`, `telegramchat0001`). +They reference credentials by id: `ollama0000000001` ("Ollama (local)") +and `smtp00000000001` ("Strato SMTP"). + +To force a re-import without changing a file, remove `/opt/n8n/import/workflows` +on the server and run the role again. + +### 9.4 Scheduled digest + +Copy `digest.json` (mail) or `digest-telegram.json` (Telegram, section 9.8), then in the editor adjust: + +- **Every Monday 06:00** — cron expression (`n8n_timezone` applies) +- **Sites** — one item per page: `url`, `prompt` and recipient `to` +- **Send mail** (mail version) — sender is the `smtp_user` address + +Export the result to the repo (section 9.3) and set `"active": true`. Run it +once manually with **Execute workflow** before relying on the schedule. +A summary of a long page takes several minutes on the CPU-only model. + +### 9.5 Ollama — local LLM server + +The `ollama` role runs Ollama (CPU only). It is reachable only on the internal +Docker network `ollama_internal` at `http://ollama:11434`; there is no public +route. + +Settings in `ansible/group_vars/all/vars.yml`: + +| Variable | Default | Meaning | +|----------|---------|---------| +| `ollama_version` | `0.35.1` | Image tag | +| `ollama_model` | `qwen3:4b-instruct-2507-q4_K_M` | Model pulled on deploy; also written into the workflows (placeholder `__OLLAMA_MODEL__`) | +| `ollama_context_length` | `16384` | Context window in tokens | +| (workflow option) `numThread` | `4` | Threads used by the Ollama chat model nodes; Ollama defaults to 2 on this 2-core/4-thread host, 4 reads prompts about 60% faster | +| `ollama_mem_limit` | `4608m` | Container memory cap (VPS has 8 GB shared with ~25 containers) | +| `ollama_keep_alive` | `1h` | Idle time before the model is unloaded from RAM | + +Deploy and verify: ```bash -ssh -i keys/notroot_cloud_ladkau_de deploy@217.154.207.148 -cd /opt/clay/workspace -git clone https://gitea.ladkau.de//.git +ansible-playbook -i ansible/inventory.ini ansible/site.yml --tags ollama --ask-vault-pass +ssh -i keys/notroot_cloud_ladkau_de deploy@217.154.207.148 \ + "sudo docker exec ollama ollama list && sudo docker stats --no-stream ollama" ``` -Then in the clay web UI, use **Add project** and point it at `/workspace/`. +Changing `ollama_model` and redeploying pulls the new model; also change the +`model` field of the Ollama Chat Model nodes in the workflow JSON files. +Remove unused models with `docker exec ollama ollama rm `. A CPU-only 4B +model at 16k context uses most of the memory available on this VPS; check +`docker stats` after the first model load. -### 9.2 MCP servers and skills +### 9.6 Browserless -Clay configuration (MCP server entries, session history, project registrations) -is stored in `~/.clay`, which is mounted from `/opt/clay/data` on the host and -survives container rebuilds. - -MCP server **binaries** installed interactively inside a running container (e.g. -via `npm install -g` in a clay terminal) are part of the container filesystem -and are lost when the image is rebuilt. - -To make a package survive rebuilds, add it to -`ansible/roles/clay/files/Dockerfile` and redeploy: - -```dockerfile -# MCP servers and skills — add packages here to survive container rebuilds -RUN npm install -g @clay-ai/clay-ralph -RUN npm install -g @modelcontextprotocol/server-filesystem -``` - -```bash -ansible-playbook -i ansible/inventory.ini ansible/site.yml --tags clay --ask-vault-pass -``` - -Ansible detects the Dockerfile change and triggers a rebuild automatically. - -### 9.3 Rebuild after clay updates - -The Docker image is built from `ansible/roles/clay/files/Dockerfile` at deploy -time. To pull upstream changes to `clay-server` or `@anthropic-ai/claude-code`, -rebuild the image and restart the container: - -```bash -ansible-playbook -i ansible/inventory.ini ansible/site.yml --tags clay --ask-vault-pass -``` - -The Ansible role forces a rebuild whenever the Dockerfile changes; to force a -rebuild without a Dockerfile change, delete the cached image first: +The `browserless` role runs headless Chromium (one session at a time, 1 GB +memory limit) on the internal network `browserless_internal` at +`http://browserless:3000`. Check it with: ```bash ssh -i keys/notroot_cloud_ladkau_de deploy@217.154.207.148 \ - "sudo docker image rm clay:local && sudo docker compose -f /opt/clay/docker-compose.yml up -d" + "sudo docker run --rm --network browserless_internal curlimages/curl -s http://browserless:3000/pressure" ``` -### 9.4 Interactive sessions vs. scheduled runs +### 9.7 SearXNG -Clay and Gitea Actions serve different purposes and are independent of each -other: +The `searxng` role runs SearXNG on the internal network `searxng_internal` at +`http://searxng:8080`. Its settings (`/opt/searxng/settings.yml`) are rendered +from `roles/searxng/templates/settings.yml.j2`. The JSON format must stay +enabled; the `web_search` tool depends on it. Check it with: -| | Clay | Gitea Actions | -|---|---|---| -| **Trigger** | You, in the browser | Schedule or manual button click | -| **Use case** | Exploratory, iterative work | Recurring automated tasks | -| **Project** | Repos cloned into `/opt/clay/workspace/` | Checked out fresh per run | -| **Output** | Interactive UI | Logs + optional email | - -Clay has no scheduling API — there is no way to trigger a Clay session -programmatically. For recurring agent tasks, Gitea Actions runs Claude Code -directly on the act_runners, independent of Clay. If both operate on the same -git repository the context is identical, but the sessions are separate. - -### 9.5 Scheduled Claude Code runs via Gitea Actions - -The act_runner containers can run Claude Code on a recurring schedule using -Gitea's `on.schedule` trigger. Adding `workflow_dispatch` alongside it allows -the same workflow to be triggered manually from the Gitea UI without waiting -for the next scheduled run. - -**Set up secrets** - -In the repository go to **Settings → Secrets → Actions** (or -**Site Administration → Actions → Secrets** for org-wide secrets) and add: - -| Secret | Value | -|--------|-------| -| `ANTHROPIC_API_KEY` | API key from the vault | -| `SMTP_USERNAME` | Gmail address used as the SMTP sender | -| `SMTP_PASSWORD` | Gmail App Password — myaccount.google.com → Security → App passwords | - -**Workflow file** - -Add `.gitea/workflows/weekly-agent.yml` to the repository: - -```yaml -name: Weekly agent run -on: - schedule: - - cron: '0 6 * * 1' # Every Monday at 06:00 UTC - workflow_dispatch: # also runnable manually from the Gitea Actions UI - -jobs: - agent: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - - name: Install Claude Code - run: npm install -g @anthropic-ai/claude-code - - - name: Run agent - env: - ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} - run: | - claude --print "Your prompt here" \ - --output-format text \ - | tee agent-output.txt - - - name: Send report email - env: - SMTP_USERNAME: ${{ secrets.SMTP_USERNAME }} - SMTP_PASSWORD: ${{ secrets.SMTP_PASSWORD }} - run: | - python3 - <<'EOF' - import smtplib, os - from email.mime.text import MIMEText - - with open("agent-output.txt") as f: - body = f.read() - - msg = MIMEText(body) - msg["Subject"] = "Weekly agent report" - msg["From"] = os.environ["SMTP_USERNAME"] - msg["To"] = os.environ["SMTP_USERNAME"] - - with smtplib.SMTP("smtp.gmail.com", 587) as s: - s.starttls() - s.login(os.environ["SMTP_USERNAME"], os.environ["SMTP_PASSWORD"]) - s.send_message(msg) - EOF +```bash +ssh -i keys/notroot_cloud_ladkau_de deploy@217.154.207.148 \ + "sudo docker run --rm --network searxng_internal curlimages/curl -s 'http://searxng:8080/search?q=test&format=json' | head -c 300" ``` -Adjust the `cron` expression, prompt, and `msg["To"]` address as needed. +Some upstream engines (DuckDuckGo, Brave) may answer with captchas or rate +limits from the server's IP address; SearXNG then returns results from the +remaining engines. The search may return fewer results at times. -**Run manually** +### 9.8 Telegram bot -Go to **Repository → Actions**, select **Weekly agent run** from the workflow -list, and click **Run workflow**. The run appears immediately in the Actions log. +The `Telegram chat` workflow lets you chat with the assistant from the Telegram +app and lets scheduled workflows post to the same conversation. The bot answers +only the Telegram user id in `telegram_user_id`; messages from anyone else and +non-text messages are ignored. -**Verify the schedule is active** +**Models** -After pushing the workflow file, scheduled runs appear automatically in -**Repository → Actions** at the configured time. +| Message | Model | +|---------|-------| +| `what is the weather in Osnabrück?` | Google Gemini `gemini-3.8-flash`, then `gemini-3.5-flash` if that fails, then local Qwen | +| `/qwen what is the weather in Osnabrück?` | Local Qwen via Ollama (slow, free, private) | + +If a Gemini model fails (daily quota, overload, anything else), the same +question goes to the next stage, and the reply ends with a note saying which +model answered. The stages are separate agents in `telegram-chat.json` +(`Gemini 3.8 Agent`, `Gemini 3.5 Agent`, `Qwen Agent`); the checks +`Gemini 3.8 ok?` and `Gemini 3.5 ok?` look for an empty answer. + +All agents have the `web_search` and `fetch_page` tools. The Gemini agents share +one conversation memory (last 8 messages, session `telegram-main`), so the Gemini +models can be switched in the middle of a conversation. The Qwen agent has its own +memory (last 2 messages, session `telegram-qwen`). + +The Gemini models are set in the `Gemini 3.8` and `Gemini 3.5` nodes of +`telegram-chat.json`. The model name is a plain string such as +`models/gemini-3.8-flash`. List the models a key can use with +`curl "https://generativelanguage.googleapis.com/v1beta/models?key="`; a +listed model can still be unavailable, so check it with a direct +`generateContent` call before using it. + +Gemini API limits: + +- The free tier limits requests per model, per minute and per day. The limit that was hit is named in the API error message. A question that uses a tool costs at least two requests (tool call, then answer). +- When a model is over its limit or overloaded, the agent retries once after 3 seconds and then the next stage answers. +- The system prompt asks for one `fetch_page` per web address or one `web_search` per question, never the same call twice. Agents stop after 4 iterations (`maxIterations`). + +Gemini requests send messages, search results and page contents to Google; +check the terms of the API tier in use. If every stage fails the bot replies +with `Error: ...`. + +The Qwen model is the non-thinking `qwen3:4b-instruct-2507` variant. The web chat +at `/chat` and the digests use the local Qwen model only. + +When `ollama_model` changes, remove the old model on the server with +`docker exec ollama ollama rm `. + +**Message formatting** + +Replies are sent with `parse_mode` HTML (the n8n Telegram node otherwise uses +Telegram's legacy Markdown mode). The `Format reply` and `Format posted message` +steps escape the text and convert bold, italics, inline code, code blocks, +headings, bullets and `[text](https://…)` links to Telegram HTML. Tables are shown +as plain text. + +**Get a Gemini API key** + +Create a key in Google AI Studio (`aistudio.google.com`, **Get API key**) and +store it in the vault as `gemini_api_key`. + +**Create the bot (once)** + +1. In Telegram open `@BotFather`, send `/newbot` and follow the prompts. Copy the bot token. +2. Open a chat with your new bot and send it any message (for example `/start`). +3. Find your numeric user id. Before the first deploy (while no webhook is set) run: + + ```bash + curl -s "https://api.telegram.org/bot/getUpdates" + ``` + + and read `message.from.id` in the result. +4. Add `gemini_api_key`, `telegram_bot_token` and `telegram_user_id` to the vault + (`ansible-vault edit ansible/group_vars/all/vault.yml`), check with + `bash scripts/check-vault.sh`, and deploy: + + ```bash + ansible-playbook -i ansible/inventory.ini ansible/site.yml --tags n8n --ask-vault-pass + ``` + + On start n8n registers the webhook `https://agent.ladkau.de/webhook//webhook` with Telegram. + +**Use it** + +- Write to the bot like in the web chat. Replying to an earlier message (swipe on the message) adds the quoted text as context. +- Gemini answers within seconds; `/qwen` answers take a minute or more on the CPU-only model. +- Conversation memory holds the last 8 messages and is lost whenever n8n restarts; replying to a message restores its context. + +**Stock price report** + +The `Telegram chat` workflow asks for the stock prices of MTE.DE and PTX.F three +times a day on weekdays at 07:30, 13:00 and 17:40 (time zone +`n8n_timezone`). The answer arrives in your Telegram chat through the normal +model chain, so you can ask follow-up questions. It can also be started: + +- by sending `/stocks` to the bot (Gemini chain), or `/qwen /stocks` to test the local model with the same question +- from the n8n editor: open `Telegram chat` and run the `Run stock price now` node with **Execute step**; the answer is sent to Telegram + +The question is the constant `STOCK_QUESTION` in the `Route` node. It names the +two Yahoo quote pages and tells the model to call `fetch_page` for each; adding +a ticker means adding its URL to that text. The model chooses the tool calls; +there is no fixed fetch step. The times are the three cron expressions of the +`Stock price schedule` node (a standard 5-field cron expression, where `1-5` +means Monday to Friday). Change either in +the editor, export the workflow to the repo (section 9.3) and deploy. The +schedule only runs while the workflow is published; every deploy that changes +workflows restarts n8n and registers it again. + +**Scheduled posts** + +`digest-telegram.json` ends with the sub-workflow call `Post to Telegram chat`. +The summary is sent to your chat and added to the chat memory, so a follow-up +question such as "what about the second point?" has the context. Any workflow +can do the same by calling the `Telegram chat` workflow with a `message` +input. + +**Security notes** + +- The Telegram webhook route has no basic auth because Telegram's servers call it. Traefik admits only the address ranges in `telegram_webhook_source_ranges` (Telegram's webhook servers; set the list to `[]` to turn the filter off if Telegram's ranges change and the bot stops receiving messages). n8n verifies Telegram's secret token header and the workflow filters on `telegram_user_id`. +- Telegram bot messages are not end-to-end encrypted. Summaries and questions pass through Telegram's servers. +- If the bot token leaks, revoke it with `/revoke` in `@BotFather`, update the vault and redeploy. + +**Troubleshooting** + +```bash +ssh -i keys/notroot_cloud_ladkau_de deploy@217.154.207.148 "sudo docker logs --tail 50 n8n" +curl -s "https://api.telegram.org/bot/getWebhookInfo" +``` + +`getWebhookInfo` shows the registered URL and the last delivery error. diff --git a/docs/runbook-provisioning.md b/docs/runbook-provisioning.md index e2e4c21..393981f 100644 --- a/docs/runbook-provisioning.md +++ b/docs/runbook-provisioning.md @@ -122,12 +122,17 @@ registry_users: - username: alice password: "your-password" -# Clay — Claude Code web interface at agent.ladkau.de -# Generate API key at console.anthropic.com → API keys -anthropic_api_key: "" -clay_users: +# n8n — AI chat and scheduled digests at agent.ladkau.de +n8n_users: # basic auth in front of n8n - username: admin password: "your-password" +n8n_encryption_key: "" # generate: openssl rand -hex 32 — never change after first deploy +searxng_secret_key: "" # generate: openssl rand -hex 32 +smtp_user: "" # full Strato mailbox address +smtp_password: "" +gemini_api_key: "" # Google AI Studio API key, see runbook-configuration.md section 9.8 +telegram_bot_token: "" # from @BotFather, see runbook-configuration.md section 9.8 +telegram_user_id: "" # your numeric Telegram user id # Gitea Actions runners — token obtained after Gitea is running # See step 3.4 of runbook-configuration.md; replace after Gitea admin account is created @@ -229,8 +234,11 @@ This runs all roles in order: | 10 | `k8s` | Placeholder page at k8s.ladkau.de | | 11 | `vaultwarden` | Vaultwarden password vault at vault.ladkau.de | | 12 | `dl` | Public download server at dl.ladkau.de — nginx HTTPS + SFTP upload | -| 13 | `clay` | Claude Code web interface at agent.ladkau.de — basic auth, Anthropic API key required | -| 14 | `dashboard` | Public status dashboard at cloud.ladkau.de — service health and server stats | +| 13 | `ollama` | Local LLM server (Ollama, CPU) on the internal `ollama_internal` network — pulls `ollama_model` on first run | +| 14 | `browserless` | Headless Chromium for rendering JavaScript-heavy pages — internal network only | +| 15 | `searxng` | Metasearch engine for the chat web_search tool — internal network only | +| 16 | `n8n` | n8n at agent.ladkau.de — AI chat (`/chat`), scheduled digests, basic auth; imports the workflows in the repo | +| 17 | `dashboard` | Public status dashboard at cloud.ladkau.de — service health and server stats | > **Note:** The `act_runner` role requires `gitea_runner_registration_token` in the > vault, which can only be obtained after Gitea is running and an admin account has diff --git a/scripts/check-services.sh b/scripts/check-services.sh index 70d1bbd..e6e4f1a 100644 --- a/scripts/check-services.sh +++ b/scripts/check-services.sh @@ -19,6 +19,7 @@ CHECKS=( "https://vault.ladkau.de 200 10" "https://dl.ladkau.de 200 10" "https://agent.ladkau.de 401 10" + "https://agent.ladkau.de/chat 401 10" ) if ! command -v curl &>/dev/null; then diff --git a/scripts/check-vault.sh b/scripts/check-vault.sh index 9088e23..59d76c6 100755 --- a/scripts/check-vault.sh +++ b/scripts/check-vault.sh @@ -44,7 +44,13 @@ required_scalars = [ "roundcube_db_password", "roundcube_des_key", "vaultwarden_admin_token", - "anthropic_api_key", + "n8n_encryption_key", + "searxng_secret_key", + "smtp_user", + "smtp_password", + "gemini_api_key", + "telegram_bot_token", + "telegram_user_id", ] # Required non-empty lists (must contain at least one entry) @@ -52,7 +58,7 @@ required_lists = [ "traefik_dashboard_users", "registry_users", "dovecot_users", - "clay_users", + "n8n_users", ] # Optional scalars: validated only when present — (key, validator_fn, hint)