stash

docs/design-in-progress/image-caching.md

@@ -0,0 +1,166 @@
+# Authenticated image caching (Design in Progress)
+
+Status: not implemented. This document proposes a fix for a real performance gap: every
+authenticated image in the app is refetched, re-verified, and re-decoded from scratch on every page
+load, even though the backend already sends headers built for exactly the opposite.
+
+## Problem
+
+Images are served from `GET /media/<hash_path>` and `GET /media/<size>/<hash_path>/`
+(`backend/files/media_urls.py`), both gated behind `SignatureAuthentication`
+(`backend/authentication/signature_auth.py`): the client signs the full request URL with an Ed25519
+key and sends `Authorization: Signature <user>@<domain>:<sig>`. There's no cookie and no
+URL-embedded token — auth lives entirely in a request header that a browser has no way to attach to
+a plain `<img src="...">`. So `AuthenticatedImage.vue` does it by hand: `fetch()` with the header,
+`.blob()`, `URL.createObjectURL()`, assign that to `src` (`federation.js`'s `getRaw`,
+`fileCache.js`). `fileCache.js` is a module-level `Map` — it dedupes concurrent requests and holds
+decoded blobs for the life of the page, but it's memory-only. Reload the page (or just navigate
+between the SPA's route-based chunks in a way that re-mounts things) and it's gone; every image the
+user has already looked at gets fetched, signature-verified, and blob-decoded all over again.
+
+Meanwhile the backend response already carries `ETag`, `Cache-Control: max-age=31536000, private,
+immutable`, and a 365-day `Expires` (`_cache_headers`, `media_urls.py`) — because `src` is a
+SHA-256 hash-addressed path, the same URL can only ever mean the same bytes, forever. Those headers
+are correct and unused: nothing durable in the client ever consults them. This design closes that
+gap using the browser's own Cache Storage API, without touching the backend.
+
+## Goals
+
+- Make a previously-viewed image load instantly on the next page load / browser restart, not just
+  within the current tab's JS session.
+- Do it without weakening the authorization model: a signature is still required and verified
+  server-side for the *first* fetch of a given file by a given identity. Caching must not let one
+  identity's cached bytes leak to a different identity sharing the same browser.
+- Reuse the backend's existing headers rather than inventing a parallel freshness scheme — content
+  is immutable, so a cache hit needs zero revalidation, ever.
+- No backend changes. This is purely a client-side storage question.
+
+## Non-goals (for now)
+
+- **Revoking already-cached bytes when access changes** (e.g. an unfriend). The backend's own
+  1-year `Cache-Control` already accepts that risk today for anything an HTTP-compliant cache might
+  hold; a persistent client cache extends the shelf life of that same accepted risk, it doesn't
+  introduce a new one. Not solving revocation here.
+- **Prefetching / warming the cache ahead of navigation.** Real optimization, separate piece of
+  work; this document is about not throwing away work already done.
+- **A Service Worker that reinstates plain `<img src>`.** Sketched below as a follow-up because it's
+  the "real" fix for the root cause (no way to attach a header to an `<img>` request), but it's a
+  bigger lift (SW lifecycle, an extra message-passing bridge for signing) than the storage win alone
+  needs. Scoped out of this pass.
+
+## Design: persist `fileCache` with the Cache Storage API
+
+`window.caches` (the `CacheStorage` interface) is available to any page context, not just inside a
+Service Worker — `caches.open(name)` gives a store of real `Request`/`Response` pairs that survives
+reloads and browser restarts, backed by the browser's own disk quota. That's the missing tier;
+nothing else about `fileCache.js`'s existing shape needs to change.
+
+**Two tiers, not one:**
+
+- **L1 — in-memory `Map<key, objectURL>`** (what exists today). Kept as-is: within a single page
+  session, components just want the already-created object URL back without re-touching storage at
+  all. Same LRU/budget logic (`MAX_BYTES`), unchanged.
+- **L2 — `CacheStorage`**, consulted on an L1 miss, before falling back to the network. Holds raw
+  `Response` objects (not blobs), keyed by the same request used for the authenticated fetch.
+
+Revised `get(key, fetcher)` flow:
+
+1. L1 hit → return the object URL, as today.
+2. L1 miss → check `cache.match(request)`. Hit → `.blob()` the cached response, create the object
+   URL, populate L1, done. **No conditional GET, no revalidation** — the response is `immutable`,
+   so if it's in the cache it's still correct by construction.
+3. L2 miss → run the existing authenticated `getRaw()` fetch. On success, `cache.put(request,
+   response.clone())` before consuming the body, then proceed as today (`.blob()`, object URL,
+   populate L1).
+
+**Namespacing by identity, not one global cache.** `Cache-Control: private` on the response is the
+backend telling shared caches to stay out — correct, since access is per-requester
+(`_accessible_files`'s friends-or-self check). A single browser-wide `CacheStorage` bucket keyed
+only by URL would quietly turn into exactly the shared cache that header is warning off, *if* this
+browser ever holds more than one local identity (switching accounts, a shared machine). Concretely:
+open the cache as `images-${username}@${domain}` (derived from the active `state.keypair`, the same
+identity that produces the signature) rather than a single `"images"` name. Same-identity re-fetches
+get the full cache benefit; a different identity in the same browser starts with an empty bucket and
+goes through the normal authenticated-fetch-then-verify path, same as it does today. `invalidate()`
+and `clear()` already exist on `FileCache` but nothing calls them — wire `clear()` to also
+`caches.delete(currentNamespace)` and call it on logout/identity-switch, which is the natural,
+already-there hook for this.
+
+**Storage budget.** L2 doesn't need its own hard byte cap the way L1 does — `CacheStorage` is
+subject to the browser's own storage-pressure eviction, which is the right backstop for "durable but
+not sacred" data like this. Optionally call `navigator.storage.persist()` once at startup to ask the
+browser to exempt the origin from casual eviction under pressure; harmless to skip if declined.
+
+**Net effect:** a returning user's already-seen images (inventory thumbnails, profile pictures,
+friends' shared items) render from disk with zero network round-trips and zero re-verification,
+using exactly the durability guarantee (`immutable`, hash-addressed) the backend already asserts.
+First-time images are unaffected — same authenticated fetch as today, just now also written to L2 on
+the way through.
+
+## Follow-up worth flagging: a Service Worker to restore plain `<img>`
+
+The deeper cost isn't just the network round-trip — it's that every image, cached or not, is forced
+through manual `fetch → blob → createObjectURL`, so the browser's native image pipeline (off-main
+thread decode, `loading="lazy"`, `fetchpriority`, responsive `srcset`) is unavailable, and object
+URLs have to be manually revoked (`fileCache.js` already does this correctly, but every new call
+site is a chance to leak one). The reason the app can't use plain `<img src>` at all is that nothing
+can attach the `Authorization: Signature` header to a browser-initiated image request.
+
+A Service Worker can, because its `fetch` handler intercepts requests — including image loads —
+before they leave the page, and can substitute its own request in place of the original:
+
+- On a `fetch` event where `event.request.destination === 'image'` and the URL matches `/media/`,
+  check the (identity-namespaced) `CacheStorage` first; hit → respond straight from cache, no
+  network at all.
+- Miss → the SW doesn't have the signing key (it lives in page memory / `localStorage`, neither
+  reachable from a SW), so it asks the one controlled client (`self.clients.get(event.clientId)` —
+  the specific tab that issued the request, not "any open tab") for a signature over this exact URL
+  via `postMessage`/`MessageChannel` — an in-process round trip, not a network call — attaches the
+  returned header, performs the real fetch, stores the result in `CacheStorage`, and responds with
+  it.
+- Once this exists, `AuthenticatedImage.vue` can go back to `<img :src="mediaUrl" loading="lazy"
+  decoding="async">` directly; the SW is what makes that legal despite the custom auth scheme.
+
+### Scoping the signing bridge: a compromised SW must not become a "sign anything" oracle
+
+The message bridge above is the one new capability this design adds that doesn't exist today: a
+channel through which something can ask the page to sign a URL on its behalf. A Service Worker is a
+long-lived, network-interposing piece of code — exactly the kind of thing a supply-chain compromise
+or an XSS-planted `registration.update()` would target. If the page's message handler blindly signs
+whatever URL the request names, a compromised SW stops being "something that can read images this
+identity can already see" and becomes "something that can get a validly-signed request for *any*
+endpoint" — e.g. `POST /api/inventory/items/5/delete` or `POST /api/friends/accept` — and then just
+replay it directly against the real backend. That's a full account-takeover primitive smuggled in
+through what was supposed to be an image-caching optimization, and it's strictly worse than not
+having the bridge at all.
+
+The fix has to live on the page side of the channel, since the SW is the presumed-compromised
+component in this threat model and can't be trusted to police itself. Treat the message handler as a
+dedicated, narrow function — not a thin wrapper around the app's general-purpose signer
+(`createSignAuth` in `federation.js`, which is used for arbitrary API calls elsewhere in the app) —
+that:
+
+- **Ignores any method the request claims and always signs as `GET`.** The bridge never accepts a
+  body/`data` field from the SW at all, which closes off the entire class of mutating requests
+  (`POST`/`PUT`/`PATCH`) regardless of what path is named.
+- **Validates the path against a strict allowlist grammar before signing anything**, rather than a
+  loose "starts with `/media/`" check. `src` values are hash-addressed —
+  `/media/<hex>/<hex>/<64-hex-char-sha256>.<ext>` for originals, with an optional `/<32|64|256>/`
+  size prefix for thumbnails. Because the variable part is constrained to `[0-9a-f]`, a regex over
+  that exact shape is effectively a closed grammar: `.` and `/` (the characters path traversal or
+  extra-segment tricks would need) simply aren't in the hex alphabet, so there's no meaningfully
+  malformed input that still matches. Anything that doesn't match — a different endpoint, an
+  encoded traversal attempt, an extra query string — is refused, silently or with a logged warning,
+  never signed.
+- Optionally also checks the URL's host against the identity's home domain or its current friend
+  servers (belt-and-suspenders — a signature is bound to the exact signed URL string, so it can't be
+  replayed against a different host than the one named in it, but this catches a compromised SW
+  fishing for signatures against a host that happens to also trust this key for unrelated reasons).
+
+With this in place, the worst a fully compromised SW can do is obtain signed `GET`s for images the
+current identity is already authorized to fetch — the same blast radius as "can read the
+already-authorized image cache" — not an oracle for arbitrary authenticated mutation.
+
+Deferred because it adds real surface area (SW registration/update lifecycle, this scoped
+message-passing bridge, first-load-before-SW-is-active edge cases) beyond what the storage change
+alone needs. Worth doing as a second pass once the simpler win above is in and paying off.