server_cloud_ladkau_de/docs/runbook-provisioning.md

Provisioning Runbook

Follow these steps to provision the server from scratch — whether setting it up for the first time or reinstalling after a wipe.

After provisioning completes, follow runbook-configuration.md for first-run setup of individual services.

Prerequisites

Local tools

• Ansible installed (pip install ansible)
• passlib installed (pip install passlib) — required for bcrypt/SHA-512 password hashing in templates
• openssl available (for generating secrets)

SSH keys

Two SSH key pairs are required. Place them in the keys/ directory (private keys are gitignored; public keys are committed).

File	Purpose
keys/root_cloud_ladkau_de	Root access — used only during initial bootstrap
keys/root_cloud_ladkau_de.pub	Public counterpart
keys/notroot_cloud_ladkau_de	Deploy user — used by Ansible for all subsequent runs
keys/notroot_cloud_ladkau_de.pub	Public counterpart — installed on server by bootstrap

Generate fresh key pairs if they do not already exist:

ssh-keygen -t ed25519 -f keys/root_cloud_ladkau_de -C "root@cloud.ladkau.de" -N ""
ssh-keygen -t ed25519 -f keys/notroot_cloud_ladkau_de -C "deploy@cloud.ladkau.de" -N ""

Upload the root public key to the server via the Strato control panel (or paste it during the initial OS install) so that root SSH access is available before running anything. The server IP is defined in ansible/inventory.ini (ansible_host).

Steps

1. Set the server IP

Open ansible/inventory.ini and set ansible_host to the server's public IP:

cloud.ladkau.de ansible_host=<server-ip> ansible_user=deploy ansible_ssh_private_key_file=keys/notroot_cloud_ladkau_de

This is the only place the IP needs to be set — the bootstrap script and all Ansible roles read it from here.

2. Install Ansible collections

From the repo root:

ansible-galaxy collection install -r ansible/requirements.yml

Required collections:

• community.general — ufw, timezone modules
• ansible.posix — authorized_key module
• community.docker — docker_network, docker_compose_v2 modules

3. Configure DNS

Ensure the following DNS A records all point to the server IP (ansible_host in ansible/inventory.ini) before running the playbook. Traefik requests Let's Encrypt certificates on first start and DNS must resolve at that point.

Domain	Record
cloud.ladkau.de	A → server IP
gitea.ladkau.de	A → server IP
nextcloud.ladkau.de	A → server IP
sso.ladkau.de	A → server IP
mail.ladkau.de	A → server IP
cr.ladkau.de	A → server IP
k8s.ladkau.de	A → server IP
vault.ladkau.de	A → server IP
dl.ladkau.de	A → server IP

4. Create the vault and populate secrets

Create ansible/group_vars/all/vault.yml (gitignored) and encrypt it with Ansible Vault:

ansible-vault create ansible/group_vars/all/vault.yml

Populate all required secrets. Ansible generates all password hashes at deploy time — store plaintext values here (the vault is encrypted).

# Traefik dashboard basic auth — generate with: openssl rand -hex 32
traefik_dashboard_users:
  - username: admin
    password: "your-password"

# Gitea — generate with: openssl rand -hex 32
gitea_db_password: ""
gitea_secret_key: ""
gitea_internal_token: ""

# Keycloak — generate with: openssl rand -hex 32
keycloak_db_password: ""
keycloak_admin_password: ""

# Nextcloud — generate with: openssl rand -hex 32
nextcloud_db_password: ""
nextcloud_admin_password: ""

# Roundcube — des_key must be exactly 24 characters: openssl rand -hex 12
roundcube_db_password: ""
roundcube_des_key: ""

# Container registry — generate with: openssl rand -hex 32
registry_users:
  - username: alice
    password: "your-password"

# Gitea Actions runners — token obtained after Gitea is running
# See step 3.4 of runbook-configuration.md; replace after Gitea admin account is created
gitea_runner_registration_token: "placeholder"

# Vaultwarden password vault
vaultwarden_admin_token: ""        # generate: openssl rand -hex 32
# vaultwarden_sso_client_secret is added after Keycloak is configured — see
# step 2.6 of runbook-configuration.md

# Download server — SFTP upload key
# Generate: ssh-keygen -t ed25519 -f dl_deploy_key -N "" -C "gitea-actions"
# Paste the contents of dl_deploy_key.pub here; store dl_deploy_key as a Gitea Actions secret
dl_sftp_authorized_keys: |
  ssh-ed25519 AAAA...

# Dovecot IMAP users
dovecot_users:
  - username: alice
    password: "your-password"

# Fetchmail — external POP3 accounts to pull from (omit section if not needed)
# local_user must match a username defined in dovecot_users above
# fetchmail_accounts:
#   - server: pop.gmail.com
#     username: user@gmail.com
#     password: app-password    # use a Gmail App Password, not your main password
#     local_user: alice         # must match a dovecot_users entry
#     ssl: true
#     keep: true                # set false to delete from source after fetching
#     poll_minutes: 10          # how often to poll this account (default: 10)

Verify all secrets are present and non-empty:

bash scripts/check-vault.sh

The script enforces three tiers:

• Required scalars — must be present and non-empty (all secrets above except vaultwarden_sso_client_secret and dl_sftp_authorized_keys)
• Required lists — must be present and contain at least one entry (traefik_dashboard_users, registry_users, dovecot_users)
• Optional but validated when present — if the key exists in the vault it must pass a format check:
  • vaultwarden_sso_client_secret — non-empty string (added after Keycloak is configured)
  • dl_sftp_authorized_keys — must begin with a recognised SSH public key prefix

To edit the vault later:

EDITOR=nano ansible-vault edit ansible/group_vars/all/vault.yml

5. Bootstrap the deploy user

bash scripts/run-bootstrap.sh

The script runs the following steps in order:

Preflight checks (local)

1. Verifies all four key files exist under keys/ (both root and deploy key pairs)
2. Verifies scripts/bootstrap-deploy-user.sh exists
3. Sets chmod 600 on the private key files (SSH refuses keys with open permissions)
4. Runs scripts/check-vault.sh — decrypts the vault, confirms all required secrets are present and non-empty, and validates optional secrets when present

Remote actions 5. Opens a test SSH connection as root to confirm the root key works 6. Copies bootstrap-deploy-user.sh to /root/ on the server via scp 7. Executes it as root — creates the deploy user, grants passwordless sudo, and installs keys/notroot_cloud_ladkau_de.pub as the only authorized key 8. Opens a test SSH connection as deploy to confirm the new user can log in

If any step fails the script exits immediately with a descriptive error message.

6. Run the Ansible master playbook

ansible-playbook -i ansible/inventory.ini ansible/site.yml --ask-vault-pass

This runs all roles in order:

#	Role	What it does
1	base	OS hardening, deploy user, SSH config, ufw firewall, fail2ban
2	docker	Docker Engine + Compose plugin, shared Traefik network
3	traefik	Reverse proxy, automatic TLS via Let's Encrypt
4	gitea	Self-hosted Git with PostgreSQL
5	act_runner	Three Gitea Actions runners with Docker executor
6	nextcloud	File storage with PostgreSQL, Redis, cron sidecar
7	sso	Keycloak single sign-on with PostgreSQL
8	mail	Roundcube webmail client with PostgreSQL
9	registry	Docker Registry v2 with htpasswd auth
10	k8s	Placeholder page at k8s.ladkau.de
11	vaultwarden	Vaultwarden password vault at vault.ladkau.de
12	dl	Public download server at dl.ladkau.de — nginx HTTPS + SFTP upload
13	dashboard	Public status dashboard at cloud.ladkau.de — service health and server stats

Note: The act_runner role requires gitea_runner_registration_token in the vault, which can only be obtained after Gitea is running and an admin account has been created. On a fresh provisioning run the role will fail if the token is absent. Either add a placeholder and redeploy with --tags act_runner after Gitea is configured (see step 3.4 of runbook-configuration.md), or skip the role on first run: --skip-tags act_runner.

To apply a single role:

ansible-playbook -i ansible/inventory.ini ansible/site.yml --tags <role> --ask-vault-pass

Scripts reference

Script	Purpose
scripts/run-bootstrap.sh	Copy and run the deploy user bootstrap on a fresh server
scripts/bootstrap-deploy-user.sh	Runs on the server as root — creates the deploy user
scripts/check-vault.sh	Verify required secrets are present; validate optional secrets (vaultwarden_sso_client_secret, dl_sftp_authorized_keys) when present
scripts/check-services.sh	Verify all service endpoints are reachable (run after provisioning)