toolshed/deploy/prod/README.md

# Toolshed production deployment — manual steps

`playbook.yml` automates installing docker.io and nginx (plus certbot, and
obtaining/renewing a TLS certificate with it, on hosts that manage their own
— see `behind_tls_proxy` below), building the backend, frontend and wiki
images, exporting the frontend and wiki static builds for nginx to serve,
writing the small `/local/domains` and `/local/dns` fixture files the
frontend fetches directly (registration domain list and DoH resolver
preference — see `toolshed_register_domains`/`toolshed_doh_resolvers` in
`playbook.yml`), configuring nginx, and installing the `toolshed-backend`
systemd service. It does **not** set up the target server or DNS. Those are
manual, one-time steps and are covered here. Seeding the backend's shared
reference data is also a manual, one-time step — see
[First superuser & shared reference data](#5-first-superuser--shared-reference-data).

## 1. Server & firewall

- A Debian/Ubuntu host reachable over SSH.
- Copy `inventory.example.yml` to `inventory.yml` (git-ignored, since it
  holds real hostnames/IPs) and fill in your host(s) — see
  [Per-deployment configuration](#2-per-deployment-configuration).
- Inbound TCP 80 open in the firewall/security group. Also open 443 unless
  `behind_tls_proxy: true` — and keep both open permanently, not just for the
  initial deploy: certbot's renewal timer needs 80 for the ACME HTTP-01
  challenge and 443 for HTTPS traffic for as long as this host is live.

## 2. Per-deployment configuration

Each entry under `hosts:` in `inventory.yml` is its own independent
deployment (its own repo checkout, database, domain, systemd service and
Django `SECRET_KEY` — nothing is shared between hosts). Set these as
host_vars directly on each host entry, not via `-e` on the command line,
so a single `inventory.yml` can hold several unrelated deployments safely:

```yaml
toolshed:
  hosts:
    my-server:
      ansible_host: 203.0.113.10
      ansible_user: deploy
      toolshed_domain: toolshed.webdomain.tld
      toolshed_handle_domain: yourtoolshed.tld  # optional, see below
      toolshed_repo_url: git@example.com:your-org/toolshed.git
      behind_tls_proxy: false
```

- `toolshed_domain` — the **web domain**: the nginx `server_name`, Django
  `ALLOWED_HOSTS`, and the hostname you'll point a TLS cert at — e.g.
  `toolshed.webdomain.tld`. Required, no default. This is not necessarily the
  same as the **handle domain** your users log in with (the part after `@`
  in `user@yourtoolshed.tld`) — see [DNS](#3-dns) for how those two relate.
- `toolshed_handle_domain` — the **handle domain**, only needed when it's
  different from `toolshed_domain`. Omit it when the two are the same (it
  then defaults to `toolshed_domain`). Set so nginx/Django accept requests
  for either domain, whichever ends up as the `Host` header.
- `toolshed_repo_url` — the git remote the playbook checks out and builds
  from. Required, no default.
- `toolshed_version` — the branch, tag or commit to check out and build.
  Optional, defaults to `stable`.
- `behind_tls_proxy` — `true` if TLS for this host is already terminated by
  something in front of it (e.g. an external reverse proxy or load
  balancer) that forwards plain HTTP here; `false` if this nginx has to
  terminate TLS itself. This controls two things:
  - Whether nginx trusts an upstream `X-Forwarded-Proto` header or sets its
    own — get this wrong and Django's `SECURE_PROXY_SSL_HEADER` check
    (`backend/backend/settings.py`) will treat every request as insecure or,
    flipped the other way, treat plain HTTP as secure.
  - Whether the playbook manages TLS at all. When `false`, it automatically
    obtains a Let's Encrypt certificate via certbot and switches nginx over
    to it — nothing to do manually beyond DNS (below). certbot's own systemd
    timer keeps renewing it afterwards, independent of the playbook.
- `toolshed_letsencrypt_email` — required whenever `behind_tls_proxy` is
  `false`; the account email certbot registers the certificate under
  (used only for renewal-failure notices). Ignored otherwise.
- `http_port` — optional, defaults to `80`. Only relevant when
  `behind_tls_proxy: true` and whatever's in front of this host forwards to
  a nonstandard port instead of 80.
- `doh_resolvers` — optional, defaults to `["1.1.1.1", "8.8.8.8"]` (the same
  hardcoded fallback the frontend itself uses, see `frontend/src/dns.js`).
  DNS-over-HTTPS resolvers the frontend uses to look up a handle domain's
  `_toolshed-server._tcp` SRV record before it has a cached preference.
  Written to `/local/dns` at deploy time; only worth overriding as a
  host_var (or `-e doh_resolvers='["9.9.9.9"]'`) if you want this
  deployment to prefer a specific resolver.

## 3. DNS

There are two distinct domains at play here, and it's easy to conflate them:

- **Web domain** — the machine's actual hostname: nginx `server_name`,
  Django `ALLOWED_HOSTS`, your TLS cert, what's in `toolshed_domain`. This is
  what an A/AAAA record has to resolve to the server's IP for.
- **Handle domain** — the part after the `@` in a username, e.g.
  `user@yourtoolshed.tld`. Toolshed usernames don't encode a server address
  directly; the frontend resolves the handle domain to a server via an SRV
  record, `_toolshed-server._tcp.<handle domain>.` (see
  `frontend/src/store.js`, `lookupServer`). What's in `toolshed_handle_domain`
  (see [Per-deployment configuration](#2-per-deployment-configuration)) only
  makes nginx/Django accept it as a `Host` header — publishing the actual SRV
  record is still a separate, manual DNS step, covered below.

The SRV lookup happens for every login, not just federation with other
servers, so **every** deployment needs it published for its own handle
domain — even a standalone server that only ever serves itself.

These two domains can be **the same** or **completely different**, and
that's exactly the choice between an A record and an SRV record:

- **Same domain**: if `yourtoolshed.tld` is both the web domain and the
  handle domain, it needs both an A record (so the domain itself resolves to
  the server) and an SRV record that happens to point back at itself.
- **Different domains**: the handle domain only needs the SRV record — no A
  record of its own — pointing at whatever web domain the server actually
  lives at. This is useful when the handle you give out (short, brandable,
  independent of hosting) shouldn't have to match wherever the box is
  actually deployed (a subdomain of a shared hosting provider, an internal
  service name, etc.).

**a) A/AAAA record — web domain → server IP:**

```sh
dig <your-web-domain> A
```

**b) SRV record — handle domain → web domain + port.** Use port 443: the
federation protocol is HTTPS-only.

```sh
dig _toolshed-server._tcp.<your-handle-domain> SRV
```

For example, with a handle domain of `yourtoolshed.tld` and a web domain of
`toolshed.webdomain.tld`:

```
$ dig _toolshed-server._tcp.yourtoolshed.tld srv
_toolshed-server._tcp.yourtoolshed.tld.  300  IN  SRV  10 10 443 toolshed.webdomain.tld.

$ dig toolshed.webdomain.tld A
toolshed.webdomain.tld. 300  IN  A  203.0.113.10
```

If you instead want `yourtoolshed.tld` itself to be the web domain too, its
SRV record just points at itself (`... SRV 10 10 443 yourtoolshed.tld.`) and
it additionally needs its own A record.

## 4. Secrets

`toolshed_secret_key` is generated once per host by the playbook (via the
`password` lookup, keyed by the host's inventory name) and stored as
`.secrets/<inventory-hostname>_secret_key` on the *control* machine, not on
the target. Back these files up — losing one invalidates all sessions and
signed cookies for that deployment on its next redeploy. They're git-ignored
on purpose; never commit them.

## 5. First superuser & shared reference data

The production backend image only runs `migrate` and `collectstatic` at
startup (see `Dockerfile.backend`) — unlike the dev compose setup, it never
runs the interactive `configure.py`. Two things dev gets "for free" from that
script therefore need doing manually, once, after a host's backend container
is first up (run these on the target host itself, or prefix with
`ssh <that-host>`):

- **Superuser account:**

  ```sh
  docker exec -it toolshed-backend python manage.py createsuperuser
  ```

- **Shared reference data** (the standard categories/properties/tags
  shipped in `backend/shared_data/*.json` — tools, electrical, screws, IT,
  etc.): without this step a fresh deployment starts with none of them.
  Run `configure.py` interactively (the `-it` flags matter — the script's
  prompts only appear with a real tty) and answer "yes" when it asks to
  import them:

  ```sh
  docker exec -it toolshed-backend python configure.py
  ```

  The other prompts it asks first (create `.env`, create a database) are
  harmless to answer "yes" to as well: the container already gets its real
  `SECRET_KEY`/`ALLOWED_HOSTS`/db path from the environment (the systemd unit
  passes them via `--env-file`, see the "Write backend environment file" task
  in `playbook.yml`), those checks just look for files at paths relative to
  `/app` that don't exist in this container, and re-running `migrate` against
  the real database is idempotent. You can say "no" to the superuser prompt
  here if you already created one above.

## 6. Running the playbook

Always target one host at a time with `--limit` — running against the whole
`toolshed` group in one invocation would apply every host's own
`toolshed_domain`/`toolshed_repo_url` correctly (they're per-host vars, see
[Per-deployment configuration](#2-per-deployment-configuration)), but rolls
out all deployments back-to-back in one run, which is rarely what you want:

```sh
ansible-playbook -i inventory.yml playbook.yml --limit my-server
```

Re-run it to roll out a new version to that host. It deploys whatever
`toolshed_version` is set for that host (`stable` by default) — set the
host_var for a persistent change, or pass `-e toolshed_version=<branch/tag/commit>`
for a one-off deploy of something else.