# Toolshed production deployment — manual steps `playbook.yml` automates installing docker.io and nginx (plus certbot, and obtaining/renewing a TLS certificate with it, on hosts that manage their own — see `behind_tls_proxy` below), building the backend, frontend and wiki images, exporting the frontend/wiki static builds and the backend's `collectstatic` output for nginx to serve directly (nginx also serves user-uploaded files directly, via an X-Accel-Redirect Django issues after its own permission check — see `location /redirect_media/` in `playbook.yml`), writing the small `/local/domains` and `/local/dns` fixture files the frontend fetches directly (registration domain list and DoH resolver preference — see `toolshed_register_domains`/`toolshed_doh_resolvers` in `playbook.yml`), configuring nginx, and installing the `toolshed-backend` systemd service. It does **not** set up the target server or DNS. Those are manual, one-time steps and are covered here. Seeding the backend's shared reference data is also a manual, one-time step — see [First superuser & shared reference data](#5-first-superuser--shared-reference-data). ## 1. Server & firewall - A Debian/Ubuntu host reachable over SSH. - Copy `inventory.example.yml` to `inventory.yml` (git-ignored, since it holds real hostnames/IPs) and fill in your host(s) — see [Per-deployment configuration](#2-per-deployment-configuration). - Inbound TCP 80 open in the firewall/security group. Also open 443 unless `behind_tls_proxy: true` — and keep both open permanently, not just for the initial deploy: certbot's renewal timer needs 80 for the ACME HTTP-01 challenge and 443 for HTTPS traffic for as long as this host is live. ## 2. Per-deployment configuration Each entry under `hosts:` in `inventory.yml` is its own independent deployment (its own repo checkout, database, domain, systemd service and Django `SECRET_KEY` — nothing is shared between hosts). Set these as host_vars directly on each host entry, not via `-e` on the command line, so a single `inventory.yml` can hold several unrelated deployments safely: ```yaml toolshed: hosts: my-server: ansible_host: 203.0.113.10 ansible_user: deploy toolshed_domain: toolshed.webdomain.tld toolshed_handle_domain: yourtoolshed.tld # optional, see below toolshed_repo_url: git@example.com:your-org/toolshed.git behind_tls_proxy: false ``` - `toolshed_domain` — the **web domain**: the nginx `server_name`, Django `ALLOWED_HOSTS`, and the hostname you'll point a TLS cert at — e.g. `toolshed.webdomain.tld`. Required, no default. This is not necessarily the same as the **handle domain** your users log in with (the part after `@` in `user@yourtoolshed.tld`) — see [DNS](#3-dns) for how those two relate. - `toolshed_handle_domain` — the **handle domain**, only needed when it's different from `toolshed_domain`. Omit it when the two are the same (it then defaults to `toolshed_domain`). It doesn't affect nginx/Django at all (they only ever accept `toolshed_domain` as the `Host` header) — it's used solely to populate the `/local/domains` registration fixture (see `toolshed_register_domains` in `playbook.yml`); publishing the SRV record is a separate, manual DNS step either way. - `toolshed_repo_url` — the git remote the playbook checks out and builds from. Required, no default. - `toolshed_version` — the branch, tag or commit to check out and build. Optional, defaults to `stable`. - `behind_tls_proxy` — `true` if TLS for this host is already terminated by something in front of it (e.g. an external reverse proxy or load balancer) that forwards plain HTTP here; `false` if this nginx has to terminate TLS itself. This controls two things: - Whether nginx trusts an upstream `X-Forwarded-Proto` header or sets its own — get this wrong and Django's `SECURE_PROXY_SSL_HEADER` check (`backend/backend/settings.py`) will treat every request as insecure or, flipped the other way, treat plain HTTP as secure. - Whether the playbook manages TLS at all. When `false`, it automatically obtains a Let's Encrypt certificate via certbot and switches nginx over to it — nothing to do manually beyond DNS (below). certbot's own systemd timer keeps renewing it afterwards, independent of the playbook. - `toolshed_letsencrypt_email` — required whenever `behind_tls_proxy` is `false`; the account email certbot registers the certificate under (used only for renewal-failure notices). Ignored otherwise. - `http_port` — optional, defaults to `80`. Only relevant when `behind_tls_proxy: true` and whatever's in front of this host forwards to a nonstandard port instead of 80. - `doh_resolvers` — optional, defaults to `["1.1.1.1", "8.8.8.8"]` (the same hardcoded fallback the frontend itself uses, see `frontend/src/dns.js`). DNS-over-HTTPS resolvers the frontend uses to look up a handle domain's `_toolshed-server._tcp` SRV record before it has a cached preference. Written to `/local/dns` at deploy time; only worth overriding as a host_var (or `-e doh_resolvers='["9.9.9.9"]'`) if you want this deployment to prefer a specific resolver. ## 3. DNS There are two distinct domains at play here, and it's easy to conflate them: - **Web domain** — the machine's actual hostname: nginx `server_name`, Django `ALLOWED_HOSTS`, your TLS cert, what's in `toolshed_domain`. This is what an A/AAAA record has to resolve to the server's IP for. - **Handle domain** — the part after the `@` in a username, e.g. `user@yourtoolshed.tld`. Toolshed usernames don't encode a server address directly; the frontend resolves the handle domain to a server via an SRV record, `_toolshed-server._tcp..` (see `frontend/src/store.js`, `lookupServer`), which always points at the web domain — nginx/Django never see the handle domain as a `Host` header. `toolshed_handle_domain` (see [Per-deployment configuration](#2-per-deployment-configuration)) only feeds the `/local/domains` registration fixture; publishing the actual SRV record is still a separate, manual DNS step, covered below. The SRV lookup happens for every login, not just federation with other servers, so **every** deployment needs it published for its own handle domain — even a standalone server that only ever serves itself. These two domains can be **the same** or **completely different**, and that's exactly the choice between an A record and an SRV record: - **Same domain**: if `yourtoolshed.tld` is both the web domain and the handle domain, it needs both an A record (so the domain itself resolves to the server) and an SRV record that happens to point back at itself. - **Different domains**: the handle domain only needs the SRV record — no A record of its own — pointing at whatever web domain the server actually lives at. This is useful when the handle you give out (short, brandable, independent of hosting) shouldn't have to match wherever the box is actually deployed (a subdomain of a shared hosting provider, an internal service name, etc.). **a) A/AAAA record — web domain → server IP:** ```sh dig A ``` **b) SRV record — handle domain → web domain + port.** Use port 443: the federation protocol is HTTPS-only. ```sh dig _toolshed-server._tcp. SRV ``` For example, with a handle domain of `yourtoolshed.tld` and a web domain of `toolshed.webdomain.tld`: ``` $ dig _toolshed-server._tcp.yourtoolshed.tld srv _toolshed-server._tcp.yourtoolshed.tld. 300 IN SRV 10 10 443 toolshed.webdomain.tld. $ dig toolshed.webdomain.tld A toolshed.webdomain.tld. 300 IN A 203.0.113.10 ``` If you instead want `yourtoolshed.tld` itself to be the web domain too, its SRV record just points at itself (`... SRV 10 10 443 yourtoolshed.tld.`) and it additionally needs its own A record. ## 4. Secrets `toolshed_secret_key` is generated once per host by the playbook (via the `password` lookup, keyed by the host's inventory name) and stored as `.secrets/_secret_key` on the *control* machine, not on the target. Back these files up — losing one invalidates all sessions and signed cookies for that deployment on its next redeploy. They're git-ignored on purpose; never commit them. ## 5. First superuser & shared reference data The production backend image only runs `migrate` and `collectstatic` at startup (see `Dockerfile.backend`) — unlike the dev compose setup, it never runs the interactive `configure.py`. Two things dev gets "for free" from that script therefore need doing manually, once, after a host's backend container is first up (run these on the target host itself, or prefix with `ssh `): - **Superuser account:** ```sh docker exec -it toolshed-backend python manage.py createsuperuser ``` - **Shared reference data** (the standard categories/properties/tags shipped in `backend/shared_data/*.json` — tools, electrical, screws, IT, etc.): without this step a fresh deployment starts with none of them. Run `configure.py` interactively (the `-it` flags matter — the script's prompts only appear with a real tty) and answer "yes" when it asks to import them: ```sh docker exec -it toolshed-backend python configure.py ``` The other prompts it asks first (create `.env`, create a database) are harmless to answer "yes" to as well: the container already gets its real `SECRET_KEY`/`ALLOWED_HOSTS`/db path from the environment (the systemd unit passes them via `--env-file`, see the "Write backend environment file" task in `playbook.yml`), those checks just look for files at paths relative to `/app` that don't exist in this container, and re-running `migrate` against the real database is idempotent. You can say "no" to the superuser prompt here if you already created one above. ## 6. Running the playbook Always target one host at a time with `--limit` — running against the whole `toolshed` group in one invocation would apply every host's own `toolshed_domain`/`toolshed_repo_url` correctly (they're per-host vars, see [Per-deployment configuration](#2-per-deployment-configuration)), but rolls out all deployments back-to-back in one run, which is rarely what you want: ```sh ansible-playbook -i inventory.yml playbook.yml --limit my-server ``` Re-run it to roll out a new version to that host. It deploys whatever `toolshed_version` is set for that host (`stable` by default) — set the host_var for a persistent change, or pass `-e toolshed_version=` for a one-off deploy of something else.