toolshed/deploy/prod

Toolshed production deployment — manual steps

playbook.yml automates installing docker.io and nginx (plus certbot, and obtaining/renewing a TLS certificate with it, on hosts that manage their own — see behind_tls_proxy below), building the backend, frontend and wiki images, exporting the frontend and wiki static builds for nginx to serve, writing the small /local/domains and /local/dns fixture files the frontend fetches directly (registration domain list and DoH resolver preference — see toolshed_register_domains/toolshed_doh_resolvers in playbook.yml), configuring nginx, and installing the toolshed-backend systemd service. It does not set up the target server or DNS. Those are manual, one-time steps and are covered here. Seeding the backend's shared reference data is also a manual, one-time step — see First superuser & shared reference data.

1. Server & firewall

• A Debian/Ubuntu host reachable over SSH.
• Copy inventory.example.yml to inventory.yml (git-ignored, since it holds real hostnames/IPs) and fill in your host(s) — see Per-deployment configuration.
• Inbound TCP 80 open in the firewall/security group. Also open 443 unless behind_tls_proxy: true — and keep both open permanently, not just for the initial deploy: certbot's renewal timer needs 80 for the ACME HTTP-01 challenge and 443 for HTTPS traffic for as long as this host is live.

2. Per-deployment configuration

Each entry under hosts: in inventory.yml is its own independent deployment (its own repo checkout, database, domain, systemd service and Django SECRET_KEY — nothing is shared between hosts). Set these as host_vars directly on each host entry, not via -e on the command line, so a single inventory.yml can hold several unrelated deployments safely:

toolshed:
  hosts:
    my-server:
      ansible_host: 203.0.113.10
      ansible_user: deploy
      toolshed_domain: toolshed.webdomain.tld
      toolshed_handle_domain: yourtoolshed.tld  # optional, see below
      toolshed_repo_url: git@example.com:your-org/toolshed.git
      behind_tls_proxy: false

• toolshed_domain — the web domain: the nginx server_name, Django ALLOWED_HOSTS, and the hostname you'll point a TLS cert at — e.g. toolshed.webdomain.tld. Required, no default. This is not necessarily the same as the handle domain your users log in with (the part after @ in user@yourtoolshed.tld) — see DNS for how those two relate.
• toolshed_handle_domain — the handle domain, only needed when it's different from toolshed_domain. Omit it when the two are the same (it then defaults to toolshed_domain). Set so nginx/Django accept requests for either domain, whichever ends up as the Host header.
• toolshed_repo_url — the git remote the playbook checks out and builds from. Required, no default.
• toolshed_version — the branch, tag or commit to check out and build. Optional, defaults to stable.
• behind_tls_proxy — true if TLS for this host is already terminated by something in front of it (e.g. an external reverse proxy or load balancer) that forwards plain HTTP here; false if this nginx has to terminate TLS itself. This controls two things:
  • Whether nginx trusts an upstream X-Forwarded-Proto header or sets its own — get this wrong and Django's SECURE_PROXY_SSL_HEADER check (backend/backend/settings.py) will treat every request as insecure or, flipped the other way, treat plain HTTP as secure.
  • Whether the playbook manages TLS at all. When false, it automatically obtains a Let's Encrypt certificate via certbot and switches nginx over to it — nothing to do manually beyond DNS (below). certbot's own systemd timer keeps renewing it afterwards, independent of the playbook.
• toolshed_letsencrypt_email — required whenever behind_tls_proxy is false; the account email certbot registers the certificate under (used only for renewal-failure notices). Ignored otherwise.
• http_port — optional, defaults to 80. Only relevant when behind_tls_proxy: true and whatever's in front of this host forwards to a nonstandard port instead of 80.
• doh_resolvers — optional, defaults to ["1.1.1.1", "8.8.8.8"] (the same hardcoded fallback the frontend itself uses, see frontend/src/dns.js). DNS-over-HTTPS resolvers the frontend uses to look up a handle domain's _toolshed-server._tcp SRV record before it has a cached preference. Written to /local/dns at deploy time; only worth overriding as a host_var (or -e doh_resolvers='["9.9.9.9"]') if you want this deployment to prefer a specific resolver.

3. DNS

There are two distinct domains at play here, and it's easy to conflate them:

• Web domain — the machine's actual hostname: nginx server_name, Django ALLOWED_HOSTS, your TLS cert, what's in toolshed_domain. This is what an A/AAAA record has to resolve to the server's IP for.
• Handle domain — the part after the @ in a username, e.g. user@yourtoolshed.tld. Toolshed usernames don't encode a server address directly; the frontend resolves the handle domain to a server via an SRV record, _toolshed-server._tcp.<handle domain>. (see frontend/src/store.js, lookupServer). What's in toolshed_handle_domain (see Per-deployment configuration) only makes nginx/Django accept it as a Host header — publishing the actual SRV record is still a separate, manual DNS step, covered below.

The SRV lookup happens for every login, not just federation with other servers, so every deployment needs it published for its own handle domain — even a standalone server that only ever serves itself.

These two domains can be the same or completely different, and that's exactly the choice between an A record and an SRV record:

• Same domain: if yourtoolshed.tld is both the web domain and the handle domain, it needs both an A record (so the domain itself resolves to the server) and an SRV record that happens to point back at itself.
• Different domains: the handle domain only needs the SRV record — no A record of its own — pointing at whatever web domain the server actually lives at. This is useful when the handle you give out (short, brandable, independent of hosting) shouldn't have to match wherever the box is actually deployed (a subdomain of a shared hosting provider, an internal service name, etc.).

a) A/AAAA record — web domain → server IP:

dig <your-web-domain> A

b) SRV record — handle domain → web domain + port. Use port 443: the federation protocol is HTTPS-only.

dig _toolshed-server._tcp.<your-handle-domain> SRV

For example, with a handle domain of yourtoolshed.tld and a web domain of toolshed.webdomain.tld:

$ dig _toolshed-server._tcp.yourtoolshed.tld srv
_toolshed-server._tcp.yourtoolshed.tld.  300  IN  SRV  10 10 443 toolshed.webdomain.tld.

$ dig toolshed.webdomain.tld A
toolshed.webdomain.tld. 300  IN  A  203.0.113.10

If you instead want yourtoolshed.tld itself to be the web domain too, its SRV record just points at itself (... SRV 10 10 443 yourtoolshed.tld.) and it additionally needs its own A record.

4. Secrets

toolshed_secret_key is generated once per host by the playbook (via the password lookup, keyed by the host's inventory name) and stored as .secrets/<inventory-hostname>_secret_key on the control machine, not on the target. Back these files up — losing one invalidates all sessions and signed cookies for that deployment on its next redeploy. They're git-ignored on purpose; never commit them.

5. First superuser & shared reference data

The production backend image only runs migrate and collectstatic at startup (see Dockerfile.backend) — unlike the dev compose setup, it never runs the interactive configure.py. Two things dev gets "for free" from that script therefore need doing manually, once, after a host's backend container is first up (run these on the target host itself, or prefix with ssh <that-host>):

• Superuser account:

  docker exec -it toolshed-backend python manage.py createsuperuser
  

• Shared reference data (the standard categories/properties/tags shipped in backend/shared_data/*.json — tools, electrical, screws, IT, etc.): without this step a fresh deployment starts with none of them. Run configure.py interactively (the -it flags matter — the script's prompts only appear with a real tty) and answer "yes" when it asks to import them:

  docker exec -it toolshed-backend python configure.py
  

  The other prompts it asks first (create .env, create a database) are harmless to answer "yes" to as well: the container already gets its real SECRET_KEY/ALLOWED_HOSTS/db path from the environment (the systemd unit passes them via --env-file, see the "Write backend environment file" task in playbook.yml), those checks just look for files at paths relative to /app that don't exist in this container, and re-running migrate against the real database is idempotent. You can say "no" to the superuser prompt here if you already created one above.

6. Running the playbook

Always target one host at a time with --limit — running against the whole toolshed group in one invocation would apply every host's own toolshed_domain/toolshed_repo_url correctly (they're per-host vars, see Per-deployment configuration), but rolls out all deployments back-to-back in one run, which is rarely what you want:

ansible-playbook -i inventory.yml playbook.yml --limit my-server

Re-run it to roll out a new version to that host. It deploys whatever toolshed_version is set for that host (stable by default) — set the host_var for a persistent change, or pass -e toolshed_version=<branch/tag/commit> for a one-off deploy of something else.