stash
This commit is contained in:
parent
c345372382
commit
395a9b156a
9 changed files with 1434 additions and 0 deletions
113
docs/design-in-progress/groups-mvp.md
Normal file
113
docs/design-in-progress/groups-mvp.md
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
# Groups MVP (Design in Progress)
|
||||
|
||||
Status: not implemented. [groups.md](groups.md) works out the underlying model (what a group is,
|
||||
how "acting as a group" is authenticated across federation). This document doesn't re-derive any
|
||||
of that; it takes that design as given and asks a narrower question: what is the smallest set of
|
||||
screens and actions that makes groups usable, reusing the UI patterns Toolshed already has for
|
||||
friends and inventory rather than inventing new ones.
|
||||
|
||||
## Goals
|
||||
|
||||
- Ship something a user can actually click through: create a group, add/remove members, and have
|
||||
the group own items, end to end.
|
||||
- Reuse existing screens and interaction patterns wherever the shape already matches, instead of
|
||||
designing new ones (see below).
|
||||
- Leave every deferred piece as an explicit non-goal rather than a silent gap, so it's clear what
|
||||
MVP does and doesn't cover.
|
||||
|
||||
## Non-goals (for now)
|
||||
|
||||
- **Group-to-group friending, and letting outside users friend a group.** groups.md's model
|
||||
supports both, but the only audience for a group's items in MVP is the group's own members;
|
||||
"private" already covers that (see "Availability policy for group items" below). Extending
|
||||
visibility to non-members via the group's own friends list is real, valuable, and deferred as a
|
||||
fast-follow, not designed here.
|
||||
- **Transferring an existing personally-owned item's ownership to a group.** MVP only supports
|
||||
creating a *new* item directly owned by a group. Moving an already-existing item across owners is
|
||||
a separate piece of work (ownership transfer isn't something Toolshed supports for user-to-user
|
||||
either today).
|
||||
- **Deleting a group.** MVP has no explicit "delete this group" action. A group's lifecycle is just
|
||||
"created, membership changes over time"; the orphaning guard (a group can't be left with zero
|
||||
members, see groups.md's Known Gaps) means a group that's no longer wanted just sits unused rather
|
||||
than needing a teardown flow.
|
||||
- **Any governance, roles, or per-item permissions.** Same non-goal as groups.md: every member has
|
||||
equal, full privileges over the group and everything it owns.
|
||||
|
||||
## User-facing features
|
||||
|
||||
### Creating a group
|
||||
|
||||
A new "Groups" section, entry point styled like Inventory's "Add" button. The form is just a name;
|
||||
the handle (`#name@yourdomain`) is derived from it the same way a username becomes part of a user's
|
||||
handle. The creator becomes the group's first member automatically — there's no empty-group state
|
||||
to design for.
|
||||
|
||||
### My Groups list
|
||||
|
||||
A "Groups" nav entry/page, modeled directly on `Friends.vue`: a table of groups the current user is
|
||||
a member of (name/handle, member count), each row linking into that group's detail page. No
|
||||
separate "discover groups you're not in" browsing for MVP — you land in a group by being added to
|
||||
it, the same way you become friends with someone by request/accept, not by browsing a directory of
|
||||
all users.
|
||||
|
||||
Known limitation: this list only ever queries the member's own home backend, so it only shows
|
||||
groups actually hosted there (groups you created, or joined on your own domain). Membership itself
|
||||
works regardless of which backend hosts the group — a remote member can still be invited, accept,
|
||||
and fully edit/delete the group's items (see "Owning items as a group" below) — but a group hosted
|
||||
on someone else's backend won't show up in your own "My Groups" list, because unlike friendship
|
||||
(which both sides record), group membership is only ever recorded on the group's own home backend,
|
||||
and there's no index anywhere of "which other backends has this identity been added to." Making a
|
||||
remote membership discoverable would need a small personal pointer index (written by the client at
|
||||
join time) plus a handle-based group lookup on the group's own backend; deferred as a fast-follow
|
||||
alongside group-friending.
|
||||
|
||||
### Group detail page
|
||||
|
||||
One page per group, with two sections, each reusing an existing pattern wholesale:
|
||||
|
||||
- **Members** — the add/remove-row pattern from `Friends.vue`'s friend list: an inline "add member"
|
||||
field (enter a handle, `user@domain`), and a remove (trash icon) action per row. Any member can
|
||||
add or remove any other member — flat privilege, no confirmation step beyond the orphaning guard
|
||||
(removing the group's last member is blocked, with an error explaining why, rather than silently
|
||||
emptying the group).
|
||||
- **Group inventory** — the exact table/grid pattern from `Inventory.vue` (Name, Availability
|
||||
Policy, Amount, Edit/Delete), scoped to items owned by this group, with the same "Add" button
|
||||
leading into the existing item-creation form (see below).
|
||||
|
||||
A "Leave group" action removes the current user from Members; if they're the last member, it's
|
||||
blocked by the same orphaning guard.
|
||||
|
||||
### Owning items as a group
|
||||
|
||||
The existing item create/edit form (`InventoryNew.vue` / `InventoryEdit.vue`) gets one new field:
|
||||
an "Owner" selector, defaulting to "Myself," with the groups you belong to as the other options.
|
||||
Every other field on that form (tags, properties, availability policy, storage location, quantity)
|
||||
is unchanged, and a group-owned item behaves exactly like a personally-owned one everywhere else in
|
||||
the app (edit, delete, detail view) — any member can edit or delete it, the same way the owner can
|
||||
today.
|
||||
|
||||
### Where group-owned items show up
|
||||
|
||||
The main "Inventory" page stays scoped to items you personally own, unchanged from today. A group's
|
||||
items are visible and managed in exactly one place: that group's detail page. This keeps "my
|
||||
inventory" meaning one thing (what I personally own) and avoids merging two different item lists
|
||||
with different edit semantics into one view for MVP.
|
||||
|
||||
### Availability policy for group items
|
||||
|
||||
The item form's existing Availability Policy field (private/share/lend/rent/sell) is unchanged and
|
||||
applies to group-owned items the same way it does today. Since group-friending is out of scope for
|
||||
MVP (see Non-goals), "share/lend/rent/sell" have no wider audience to expand to yet, only "private"
|
||||
is fully meaningful right now — a private group item is visible to and editable by every group
|
||||
member, which is already the core value the Problem statement in groups.md is after (a shared
|
||||
workshop's members all having a say over shared equipment). The field stays as-is rather than being
|
||||
trimmed down to just "private," so nothing needs to change on it once group-friending ships.
|
||||
|
||||
## Open scoping call
|
||||
|
||||
Deferring group-friending is the one judgment call in this document worth flagging explicitly:
|
||||
it means an MVP group can't yet share an item with anyone outside its own membership, which is a
|
||||
real limitation, not just a simplification. It was scoped out because it pulls in a second piece of
|
||||
UI (a group's own Friends tab, and "who can accept a friend request on the group's behalf") that
|
||||
isn't needed for the core "shared ownership among members" use case to work end to end. Worth
|
||||
confirming this is the right line before building against it.
|
||||
233
docs/design-in-progress/groups.md
Normal file
233
docs/design-in-progress/groups.md
Normal file
|
|
@ -0,0 +1,233 @@
|
|||
# Groups (Design in Progress)
|
||||
|
||||
Status: not implemented. This document collects the problem, goals, and open design questions for
|
||||
adding groups to Toolshed. Nothing here is settled; it's a starting point for discussion.
|
||||
|
||||
## What a group is
|
||||
|
||||
A group exists to model shared ownership of items, the way a club, workshop, or company owns
|
||||
equipment collectively rather than any one person owning it. A group has members, and all members
|
||||
have equal privileges to edit the items the group owns; there's no owner-vs-member distinction
|
||||
within a group, membership itself is the privilege.
|
||||
|
||||
This is a sharper (and more consequential) definition than "a label you can share things with":
|
||||
it means a group needs to be able to *own* things, not just receive shared access to them the way
|
||||
a friend does. That has implications worked out below.
|
||||
|
||||
## Problem
|
||||
|
||||
The only relationship Toolshed currently models between users is pairwise friendship (see
|
||||
[federation.md](../federation.md)), and every item has exactly one owning user. That's enough for
|
||||
"I trust this one specific person" and "I personally own this thing," but it breaks down for
|
||||
anything collectively owned:
|
||||
|
||||
- A shared workshop, tool library, or team can't own equipment as a unit. Today it has to belong
|
||||
to one specific person's account, which is a poor fit and doesn't reflect who actually has a say
|
||||
over it.
|
||||
- There's no way for several equally-privileged people to edit the same item; edit rights today
|
||||
are entirely tied to the single `owner` field.
|
||||
- Adding or removing a member of an informal group currently means renegotiating friendships and
|
||||
re-sharing individually; there's no shared object whose membership can just be edited once.
|
||||
|
||||
## Goals
|
||||
|
||||
- Let a set of users collectively own items, with every member holding equal edit rights over
|
||||
those items.
|
||||
- Let membership be managed in one place instead of via N pairwise arrangements.
|
||||
- A request from a client to any backend may not depend on any other backend being online at the
|
||||
same time — not to construct the request, and not to verify it. Concretely: the receiving server
|
||||
must be able to verify the request using only the request itself plus keys it has already cached
|
||||
from prior trust (friend-accept), with the group's own authoritative backend and the requesting
|
||||
member's home server both unneeded and unreachable-safe at verification time; and the client must
|
||||
be able to send the request using only what it already has cached, with the group's authoritative
|
||||
backend unneeded and unreachable-safe at send time too. This is the same property plain user
|
||||
requests already have (see federation.md's Cryptography section); group requests must not regress
|
||||
it on either side.
|
||||
- Fit into the existing handle system (see federation.md's "Unique Handles" section): a group
|
||||
should be nameable and referenceable using the same handle shape a user is.
|
||||
- Stay optional and additive. Pairwise friendship and single-user ownership should keep working
|
||||
exactly as they do now for people who never touch groups.
|
||||
|
||||
## Non-goals (for now)
|
||||
|
||||
- Group governance beyond flat, equal membership (voting, roles, hierarchies). Equal privileges
|
||||
for all members is the whole model for now; anything more layers on top later if needed.
|
||||
- Fine-grained per-item permissions within a group (e.g. "this member can edit but not delete").
|
||||
Membership is the only privilege level.
|
||||
|
||||
## Open design questions
|
||||
|
||||
### Does a group need its own keypair?
|
||||
|
||||
No. A group's day-to-day existence is a membership roster maintained by whichever backend is
|
||||
authoritative for the group's handle (the same "authoritative backend" idea a user's domain already
|
||||
implies, see federation.md's Servers subsection). A request made "as the group" is an ordinary
|
||||
request, signed with an actual member's own personal private key, together with a claim of which
|
||||
group it's acting on behalf of (`acting_as`), signed as part of the same payload as the rest of the
|
||||
request.
|
||||
|
||||
A receiving server must be able to authenticate such a request using only what arrives in the
|
||||
request plus keys it already holds; no other server, including the group's own authoritative
|
||||
backend, needs to be reachable at verification time. This is accomplished with **membership
|
||||
certificates**: on request, a group's authoritative backend issues a current member a small
|
||||
signed statement of the shape "handle X, public key P, is a member of group #G, valid from T1 until
|
||||
T2," signed with the group's own private key. Issuance is on-demand (the member asks, rather than
|
||||
the backend pushing renewals on a schedule), but it is its own separate action, decoupled from
|
||||
sending any particular group request: a member fetches and refreshes this certificate from the
|
||||
group's backend whenever they happen to be online, caches it locally, and later attaches whichever
|
||||
certificate they currently hold to a request they make "as the group." Sending that request never
|
||||
itself triggers a live fetch from the group's backend — if the cached certificate has expired and
|
||||
the group's backend isn't reachable right then, the request simply can't be sent as the group yet;
|
||||
the client doesn't fall back to contacting the group's backend synchronously to get one.
|
||||
|
||||
A receiving server checks two signatures, using keys it already has cached, with no outgoing call
|
||||
to anyone: the member's signature over the request, using the public key embedded in the
|
||||
certificate itself, and the certificate's own signature, using the group's public key, learned and
|
||||
cached exactly the way any friend's key is, at the point the group was friended. If both check out
|
||||
and the certificate hasn't expired, the request is authorized.
|
||||
|
||||
The certificate is what lets a receiving server trust a specific member's public key at all, for
|
||||
members it has never individually friended: that trust is vouched for by the group's already-cached
|
||||
key, rather than requiring a separate key-exchange with every member of every group a user happens
|
||||
to be friends with. Equal privileges for all members falls out of this directly, since any member's
|
||||
own key plus a valid certificate is sufficient proof.
|
||||
|
||||
Removing a member takes effect once their existing certificate expires, not the moment the
|
||||
backend's roster is edited; certificate lifetime is the parameter that governs how quickly a
|
||||
removal actually takes hold (see "How long should a membership certificate be valid for?" below,
|
||||
and Security below).
|
||||
|
||||
### Are group handles different from user handles, or is a group just a special kind of user?
|
||||
|
||||
The handle should look almost exactly like a user handle, just prefixed with `#`: `#groupname@domain`
|
||||
instead of `groupname@domain`. It's resolved the same way and referenceable in the same places (e.g.
|
||||
as an item's owner, or as a friend-list entry), so the reuse of the existing federation model is
|
||||
unaffected. The prefix exists only to keep group and user handles from occupying the same namespace on
|
||||
a domain: without it, "is `groupname@domain` a user or a group" would depend on which one happened to
|
||||
register the name first, and the two could never be told apart just by looking at the handle. With the
|
||||
prefix, a domain can have both a `climbing@domain` user and a `#climbing@domain` group with no
|
||||
collision and no ambiguity about which is which, and any code path that resolves a handle can dispatch
|
||||
on the actor kind (user vs. group) from the handle's own shape, before it even needs to ask a server.
|
||||
|
||||
But underneath, a group isn't really "a special kind of user," it's backed by a fundamentally
|
||||
different kind of identity: a membership list instead of a single keypair (see above). A user
|
||||
handle answers "is this request authentically from this one identity," a group handle answers "is
|
||||
this request from someone currently entitled to act for this collective identity." Those are
|
||||
different questions even though the answer to both ends up being "yes, forward the request." Groups
|
||||
and users are probably best thought of as two kinds of actor that share a handle format and most of
|
||||
the surrounding plumbing (ownership, friendship, availability policy), rather than one being a
|
||||
special case of the other.
|
||||
|
||||
### Should a group be able to grant read access to non-members ("group friends")?
|
||||
|
||||
Yes, this should reuse the same mechanism a user's own sharing already uses. Since a group is an
|
||||
actor with a handle, it can have its own friends the same way a user does, and a group's owned
|
||||
items can go through the same availability-policy check (private vs. visible to the group's
|
||||
friends) that an individual's items already do. Nothing new needs to be invented here, it's the
|
||||
existing friendship and availability-policy machinery applied to a second kind of actor.
|
||||
|
||||
### Should groups be able to befriend other groups?
|
||||
|
||||
Yes, for the same reason: if a group is an actor with a handle and a friends list, there's no
|
||||
reason the other side of that friendship has to be a user specifically. Two clubs befriending each
|
||||
other so each can see the other's shared equipment is the same mechanism as two users befriending
|
||||
each other, just with both sides being groups instead of one or zero.
|
||||
|
||||
### Should anyone be able to share directly with a specific group, instead of with "my friends" generally?
|
||||
|
||||
This is the one piece that isn't just reuse of what exists today. Right now, an item's
|
||||
availability policy is all-or-nothing across *all* of the owner's friends, there's no way to share
|
||||
with a subset of friends, individually or as a named group. Letting an item be shared with one
|
||||
specific group (a user's own item, shared with a club they belong to, say) instead of with every
|
||||
friend equally would be a genuine generalization of the current sharing model, not something that
|
||||
falls out of adding groups as an actor.
|
||||
|
||||
Worth noting: this capability would be just as useful for individual users wanting to share with a
|
||||
subset of their friends, without a group being involved at all. It might make more sense to design
|
||||
"share with a specific target (user, group, or named subset)" as its own piece of work, rather than
|
||||
building it as a groups-only feature.
|
||||
|
||||
### How long should a membership certificate be valid for?
|
||||
|
||||
This is the tuning knob the certificate design introduces, and it's a real trade-off rather than a
|
||||
detail to defer. A short validity window (say, hours) keeps the staleness window after a removal
|
||||
small, but means a member who's offline for longer than that can't act as the group at all until
|
||||
they reconnect and refresh. A long window (days or weeks) is more forgiving of intermittent
|
||||
connectivity but leaves a removed member's old certificate usable for longer. Whatever default is
|
||||
picked, a member should be able to fetch a fresh certificate well before the old one expires while
|
||||
still online, so the common case isn't "offline for exactly the wrong amount of time."
|
||||
|
||||
## Interaction with availability policy
|
||||
|
||||
Items currently have an availability policy (private / share / lend / rent / sell) that's a
|
||||
property of the item, not a list of who it applies to, "friends" is implicit and applies equally to
|
||||
all of them. A group-owned item works the same way, just with the group's own friends as the
|
||||
implicit audience instead of an individual's. Targeted sharing (the question above) would extend
|
||||
this, not replace it.
|
||||
|
||||
## Known gaps in the design
|
||||
|
||||
None of these block the design, but they're real gaps that need an explicit answer before
|
||||
implementation.
|
||||
|
||||
### Ambiguity
|
||||
|
||||
- **What exactly is signed.** The `acting_as` claim and the certificate's identifying fields must be
|
||||
signed as part of the same payload the member's key signs, not as free-standing, unsigned data
|
||||
alongside it. If they aren't inside the signed bytes, `acting_as` can be swapped after signing,
|
||||
turning a personal request into a group one or vice versa, or one group's request into another's.
|
||||
- **Who can change membership, and how.** Equal privileges to edit items doesn't by itself say
|
||||
whether that equality extends to *requesting or renewing certificates for others, or editing the
|
||||
authoritative backend's issuance list itself*. Flat and unilateral (any member can add or remove
|
||||
any member) is the simplest reading of "no owner-vs-member distinction," but it's a materially
|
||||
different trust model from "equal edit rights over items" and deserves its own explicit decision.
|
||||
|
||||
### Security
|
||||
|
||||
- **Confused deputy on `acting_as`.** Trust in an `acting_as` claim reduces to trust in the group's
|
||||
authoritative backend's issuance decisions: a malicious or compromised backend can sign a
|
||||
certificate for a handle that was never really a member, and every receiving server that trusts
|
||||
the group's key will accept it. The backend's issuance discipline, and the security of its own
|
||||
private key, is a single point of failure for the group as a whole.
|
||||
- **Membership staleness window.** There is a window after a member is removed during which their
|
||||
existing certificate keeps working: exactly the certificate's remaining validity period. This is a
|
||||
strictly worse revocation story than individual friendship, where trust is keyed to a public key
|
||||
learned once with no expiry, but the window is a bounded, chosen parameter (see "How long should a
|
||||
membership certificate be valid for?" above) rather than open-ended.
|
||||
- **Blast radius of a single compromised member key.** Because membership is flat and unilaterally
|
||||
editable by any member, a compromised personal key doesn't just expose that person's own items,
|
||||
as with an ordinary account compromise, it exposes edit rights over everything the group owns for
|
||||
as long as that member's certificate remains valid, and can be used to obtain a certificate for an
|
||||
attacker-controlled handle as a permanent member before anyone notices. This risk is inherent to
|
||||
"equal privileges, no roles" as a model, worth flagging even though richer governance is a
|
||||
non-goal for now.
|
||||
- **Self-lockout / orphaning.** Nothing in the design stops a group's last member from leaving (or
|
||||
removing everyone else) from the backend's issuance list, which would leave group-owned items with
|
||||
no one able to obtain a valid certificate for that owner at all once existing certificates expire.
|
||||
The backend should guard against removing the last member, but that guard doesn't address a
|
||||
member unilaterally removing every *other* member, which the flat model otherwise permits.
|
||||
- **Audit trail depends on discipline.** Since any member's signature plus a valid certificate
|
||||
satisfies authorization, "the group edited this item" is never sufficient for an audit trail; the
|
||||
actual signer's handle (from the certificate's embedded public key) must always be logged
|
||||
alongside the group claim, or member-level accountability is lost entirely.
|
||||
|
||||
### Maintainability
|
||||
|
||||
- **Two actor kinds sharing one code path.** `Group` should have the same shape as `User` for the
|
||||
things that matter (a `.friends` set, a `.handle`), so ownership/friending/availability-policy
|
||||
code can stay actor-agnostic. That reuse only holds if future code is disciplined about not
|
||||
special-casing `User` in ways that assume a single, non-expiring keypair (e.g. "cache the owner's
|
||||
public key forever, no expiry check needed") — a shortcut that would silently break the moment
|
||||
the owner turns out to be a group, where the *acting member's* key is only good until its
|
||||
certificate expires.
|
||||
- **Certificate issuance and refresh is a client responsibility.** A member's client needs to
|
||||
refresh its certificate before it expires, handle in-flight group actions failing closed if it
|
||||
didn't (the same as any expired-credential error), and surface refresh failures to the user
|
||||
rather than as a confusing rejected request.
|
||||
- **Expanded federation test surface.** Every existing federation test implicitly assumes the
|
||||
request's signer and its authorized actor are the same handle. `acting_as` plus an embedded
|
||||
membership certificate means the whole request-verification path needs testing for the
|
||||
signer-vs-actor split and the certificate's own signature and expiry checks, including
|
||||
cross-domain cases (group hosted on one domain, member's key registered on another, item owned by
|
||||
the group sitting on a third) and expiry-boundary cases (certificate expires mid-flight, is
|
||||
refreshed concurrently with a request, etc).
|
||||
166
docs/design-in-progress/image-caching.md
Normal file
166
docs/design-in-progress/image-caching.md
Normal file
|
|
@ -0,0 +1,166 @@
|
|||
# Authenticated image caching (Design in Progress)
|
||||
|
||||
Status: not implemented. This document proposes a fix for a real performance gap: every
|
||||
authenticated image in the app is refetched, re-verified, and re-decoded from scratch on every page
|
||||
load, even though the backend already sends headers built for exactly the opposite.
|
||||
|
||||
## Problem
|
||||
|
||||
Images are served from `GET /media/<hash_path>` and `GET /media/<size>/<hash_path>/`
|
||||
(`backend/files/media_urls.py`), both gated behind `SignatureAuthentication`
|
||||
(`backend/authentication/signature_auth.py`): the client signs the full request URL with an Ed25519
|
||||
key and sends `Authorization: Signature <user>@<domain>:<sig>`. There's no cookie and no
|
||||
URL-embedded token — auth lives entirely in a request header that a browser has no way to attach to
|
||||
a plain `<img src="...">`. So `AuthenticatedImage.vue` does it by hand: `fetch()` with the header,
|
||||
`.blob()`, `URL.createObjectURL()`, assign that to `src` (`federation.js`'s `getRaw`,
|
||||
`fileCache.js`). `fileCache.js` is a module-level `Map` — it dedupes concurrent requests and holds
|
||||
decoded blobs for the life of the page, but it's memory-only. Reload the page (or just navigate
|
||||
between the SPA's route-based chunks in a way that re-mounts things) and it's gone; every image the
|
||||
user has already looked at gets fetched, signature-verified, and blob-decoded all over again.
|
||||
|
||||
Meanwhile the backend response already carries `ETag`, `Cache-Control: max-age=31536000, private,
|
||||
immutable`, and a 365-day `Expires` (`_cache_headers`, `media_urls.py`) — because `src` is a
|
||||
SHA-256 hash-addressed path, the same URL can only ever mean the same bytes, forever. Those headers
|
||||
are correct and unused: nothing durable in the client ever consults them. This design closes that
|
||||
gap using the browser's own Cache Storage API, without touching the backend.
|
||||
|
||||
## Goals
|
||||
|
||||
- Make a previously-viewed image load instantly on the next page load / browser restart, not just
|
||||
within the current tab's JS session.
|
||||
- Do it without weakening the authorization model: a signature is still required and verified
|
||||
server-side for the *first* fetch of a given file by a given identity. Caching must not let one
|
||||
identity's cached bytes leak to a different identity sharing the same browser.
|
||||
- Reuse the backend's existing headers rather than inventing a parallel freshness scheme — content
|
||||
is immutable, so a cache hit needs zero revalidation, ever.
|
||||
- No backend changes. This is purely a client-side storage question.
|
||||
|
||||
## Non-goals (for now)
|
||||
|
||||
- **Revoking already-cached bytes when access changes** (e.g. an unfriend). The backend's own
|
||||
1-year `Cache-Control` already accepts that risk today for anything an HTTP-compliant cache might
|
||||
hold; a persistent client cache extends the shelf life of that same accepted risk, it doesn't
|
||||
introduce a new one. Not solving revocation here.
|
||||
- **Prefetching / warming the cache ahead of navigation.** Real optimization, separate piece of
|
||||
work; this document is about not throwing away work already done.
|
||||
- **A Service Worker that reinstates plain `<img src>`.** Sketched below as a follow-up because it's
|
||||
the "real" fix for the root cause (no way to attach a header to an `<img>` request), but it's a
|
||||
bigger lift (SW lifecycle, an extra message-passing bridge for signing) than the storage win alone
|
||||
needs. Scoped out of this pass.
|
||||
|
||||
## Design: persist `fileCache` with the Cache Storage API
|
||||
|
||||
`window.caches` (the `CacheStorage` interface) is available to any page context, not just inside a
|
||||
Service Worker — `caches.open(name)` gives a store of real `Request`/`Response` pairs that survives
|
||||
reloads and browser restarts, backed by the browser's own disk quota. That's the missing tier;
|
||||
nothing else about `fileCache.js`'s existing shape needs to change.
|
||||
|
||||
**Two tiers, not one:**
|
||||
|
||||
- **L1 — in-memory `Map<key, objectURL>`** (what exists today). Kept as-is: within a single page
|
||||
session, components just want the already-created object URL back without re-touching storage at
|
||||
all. Same LRU/budget logic (`MAX_BYTES`), unchanged.
|
||||
- **L2 — `CacheStorage`**, consulted on an L1 miss, before falling back to the network. Holds raw
|
||||
`Response` objects (not blobs), keyed by the same request used for the authenticated fetch.
|
||||
|
||||
Revised `get(key, fetcher)` flow:
|
||||
|
||||
1. L1 hit → return the object URL, as today.
|
||||
2. L1 miss → check `cache.match(request)`. Hit → `.blob()` the cached response, create the object
|
||||
URL, populate L1, done. **No conditional GET, no revalidation** — the response is `immutable`,
|
||||
so if it's in the cache it's still correct by construction.
|
||||
3. L2 miss → run the existing authenticated `getRaw()` fetch. On success, `cache.put(request,
|
||||
response.clone())` before consuming the body, then proceed as today (`.blob()`, object URL,
|
||||
populate L1).
|
||||
|
||||
**Namespacing by identity, not one global cache.** `Cache-Control: private` on the response is the
|
||||
backend telling shared caches to stay out — correct, since access is per-requester
|
||||
(`_accessible_files`'s friends-or-self check). A single browser-wide `CacheStorage` bucket keyed
|
||||
only by URL would quietly turn into exactly the shared cache that header is warning off, *if* this
|
||||
browser ever holds more than one local identity (switching accounts, a shared machine). Concretely:
|
||||
open the cache as `images-${username}@${domain}` (derived from the active `state.keypair`, the same
|
||||
identity that produces the signature) rather than a single `"images"` name. Same-identity re-fetches
|
||||
get the full cache benefit; a different identity in the same browser starts with an empty bucket and
|
||||
goes through the normal authenticated-fetch-then-verify path, same as it does today. `invalidate()`
|
||||
and `clear()` already exist on `FileCache` but nothing calls them — wire `clear()` to also
|
||||
`caches.delete(currentNamespace)` and call it on logout/identity-switch, which is the natural,
|
||||
already-there hook for this.
|
||||
|
||||
**Storage budget.** L2 doesn't need its own hard byte cap the way L1 does — `CacheStorage` is
|
||||
subject to the browser's own storage-pressure eviction, which is the right backstop for "durable but
|
||||
not sacred" data like this. Optionally call `navigator.storage.persist()` once at startup to ask the
|
||||
browser to exempt the origin from casual eviction under pressure; harmless to skip if declined.
|
||||
|
||||
**Net effect:** a returning user's already-seen images (inventory thumbnails, profile pictures,
|
||||
friends' shared items) render from disk with zero network round-trips and zero re-verification,
|
||||
using exactly the durability guarantee (`immutable`, hash-addressed) the backend already asserts.
|
||||
First-time images are unaffected — same authenticated fetch as today, just now also written to L2 on
|
||||
the way through.
|
||||
|
||||
## Follow-up worth flagging: a Service Worker to restore plain `<img>`
|
||||
|
||||
The deeper cost isn't just the network round-trip — it's that every image, cached or not, is forced
|
||||
through manual `fetch → blob → createObjectURL`, so the browser's native image pipeline (off-main
|
||||
thread decode, `loading="lazy"`, `fetchpriority`, responsive `srcset`) is unavailable, and object
|
||||
URLs have to be manually revoked (`fileCache.js` already does this correctly, but every new call
|
||||
site is a chance to leak one). The reason the app can't use plain `<img src>` at all is that nothing
|
||||
can attach the `Authorization: Signature` header to a browser-initiated image request.
|
||||
|
||||
A Service Worker can, because its `fetch` handler intercepts requests — including image loads —
|
||||
before they leave the page, and can substitute its own request in place of the original:
|
||||
|
||||
- On a `fetch` event where `event.request.destination === 'image'` and the URL matches `/media/`,
|
||||
check the (identity-namespaced) `CacheStorage` first; hit → respond straight from cache, no
|
||||
network at all.
|
||||
- Miss → the SW doesn't have the signing key (it lives in page memory / `localStorage`, neither
|
||||
reachable from a SW), so it asks the one controlled client (`self.clients.get(event.clientId)` —
|
||||
the specific tab that issued the request, not "any open tab") for a signature over this exact URL
|
||||
via `postMessage`/`MessageChannel` — an in-process round trip, not a network call — attaches the
|
||||
returned header, performs the real fetch, stores the result in `CacheStorage`, and responds with
|
||||
it.
|
||||
- Once this exists, `AuthenticatedImage.vue` can go back to `<img :src="mediaUrl" loading="lazy"
|
||||
decoding="async">` directly; the SW is what makes that legal despite the custom auth scheme.
|
||||
|
||||
### Scoping the signing bridge: a compromised SW must not become a "sign anything" oracle
|
||||
|
||||
The message bridge above is the one new capability this design adds that doesn't exist today: a
|
||||
channel through which something can ask the page to sign a URL on its behalf. A Service Worker is a
|
||||
long-lived, network-interposing piece of code — exactly the kind of thing a supply-chain compromise
|
||||
or an XSS-planted `registration.update()` would target. If the page's message handler blindly signs
|
||||
whatever URL the request names, a compromised SW stops being "something that can read images this
|
||||
identity can already see" and becomes "something that can get a validly-signed request for *any*
|
||||
endpoint" — e.g. `POST /api/inventory/items/5/delete` or `POST /api/friends/accept` — and then just
|
||||
replay it directly against the real backend. That's a full account-takeover primitive smuggled in
|
||||
through what was supposed to be an image-caching optimization, and it's strictly worse than not
|
||||
having the bridge at all.
|
||||
|
||||
The fix has to live on the page side of the channel, since the SW is the presumed-compromised
|
||||
component in this threat model and can't be trusted to police itself. Treat the message handler as a
|
||||
dedicated, narrow function — not a thin wrapper around the app's general-purpose signer
|
||||
(`createSignAuth` in `federation.js`, which is used for arbitrary API calls elsewhere in the app) —
|
||||
that:
|
||||
|
||||
- **Ignores any method the request claims and always signs as `GET`.** The bridge never accepts a
|
||||
body/`data` field from the SW at all, which closes off the entire class of mutating requests
|
||||
(`POST`/`PUT`/`PATCH`) regardless of what path is named.
|
||||
- **Validates the path against a strict allowlist grammar before signing anything**, rather than a
|
||||
loose "starts with `/media/`" check. `src` values are hash-addressed —
|
||||
`/media/<hex>/<hex>/<64-hex-char-sha256>.<ext>` for originals, with an optional `/<32|64|256>/`
|
||||
size prefix for thumbnails. Because the variable part is constrained to `[0-9a-f]`, a regex over
|
||||
that exact shape is effectively a closed grammar: `.` and `/` (the characters path traversal or
|
||||
extra-segment tricks would need) simply aren't in the hex alphabet, so there's no meaningfully
|
||||
malformed input that still matches. Anything that doesn't match — a different endpoint, an
|
||||
encoded traversal attempt, an extra query string — is refused, silently or with a logged warning,
|
||||
never signed.
|
||||
- Optionally also checks the URL's host against the identity's home domain or its current friend
|
||||
servers (belt-and-suspenders — a signature is bound to the exact signed URL string, so it can't be
|
||||
replayed against a different host than the one named in it, but this catches a compromised SW
|
||||
fishing for signatures against a host that happens to also trust this key for unrelated reasons).
|
||||
|
||||
With this in place, the worst a fully compromised SW can do is obtain signed `GET`s for images the
|
||||
current identity is already authorized to fetch — the same blast radius as "can read the
|
||||
already-authorized image cache" — not an oracle for arbitrary authenticated mutation.
|
||||
|
||||
Deferred because it adds real surface area (SW registration/update lifecycle, this scoped
|
||||
message-passing bridge, first-load-before-SW-is-active edge cases) beyond what the storage change
|
||||
alone needs. Worth doing as a second pass once the simpler win above is in and paying off.
|
||||
131
docs/design-in-progress/items-labels.md
Normal file
131
docs/design-in-progress/items-labels.md
Normal file
|
|
@ -0,0 +1,131 @@
|
|||
# Item Handles & Physical Labels (Design in Progress)
|
||||
|
||||
Status: not implemented. This document collects the problem, goals, and open design questions for
|
||||
giving inventory items stable identifiers and physical (scannable) labels. Nothing here is
|
||||
settled.
|
||||
|
||||
## Problem
|
||||
|
||||
As described in federation.md's "Unique Handles" section, an item today is identified only by a
|
||||
local id scoped to its owner, it isn't given an explicit, portable handle the way a tag, property,
|
||||
or category is. That's fine as long as the only thing ever addressing an item is the owning user's
|
||||
own signed API traffic. It stops being fine the moment something *outside* that loop needs to
|
||||
refer to the item:
|
||||
|
||||
- A friend who borrowed a physical tool has no way to look it up other than finding it in the
|
||||
owner's shared inventory list by eye.
|
||||
There's nothing you could put on a sticker.
|
||||
- If items ever need to be referenced from outside their owner's own requests (a group's shared
|
||||
view, a lending record, a printed label), there's currently no stable identifier to reference
|
||||
that's meaningful outside the owner's own account.
|
||||
- A local database id isn't something we'd want to expose or rely on externally: it's an
|
||||
implementation detail of one backend's storage, not a handle with the same guarantees
|
||||
(uniqueness, meaning, longevity) the rest of the federation model gives every other kind of
|
||||
entity.
|
||||
|
||||
Put simply: every other kind of thing in Toolshed (users, tags, properties, categories) has a
|
||||
handle that means something outside of one database. Items don't, and physical labeling is the
|
||||
clearest case where that gap actually matters.
|
||||
|
||||
## Goals
|
||||
|
||||
- Give an item a handle that's meaningful and resolvable outside its owner's own account, without
|
||||
requiring items to become shared/reusable entities the way tags are (an item is still owned by
|
||||
exactly one person; see the "Items" subsection of federation.md for why that keeps things
|
||||
simple).
|
||||
- Support a physical label (QR code, barcode, or similar) that can be printed and stuck on a real
|
||||
object, such that scanning it gets you to the right item on the right backend.
|
||||
- Make the label survive the normal life of a physical object: it gets lent out, comes back,
|
||||
maybe changes which storage location it lives in, all without needing a new label printed.
|
||||
|
||||
## Non-goals (for now)
|
||||
|
||||
- Turning items into shareable/reusable entities across owners (that's what tags/categories are
|
||||
for; see [tags.md](tags.md)). An item handle identifies *this specific person's specific thing*,
|
||||
not a class of thing.
|
||||
- Solving inventory tracking/auditing (check-in/check-out logs) as a whole system; that can build
|
||||
on top of a stable item handle once one exists, but isn't the same problem.
|
||||
|
||||
## Open design questions
|
||||
|
||||
**What does the handle look like?**
|
||||
The natural extension of the existing scheme is owner handle + local id, that's enough to be
|
||||
globally unique (no two users share a handle, and ids are already unique within one user's
|
||||
inventory) without inventing a new namespace. Worth deciding whether the id should be the existing
|
||||
internal database id (simple, but leaks a little implementation detail and a rough count of
|
||||
someone's inventory) or a separate opaque id generated for exactly this purpose (see the
|
||||
unguessability question below).
|
||||
|
||||
Two distinct formats are needed, because "an item handle" is used in two different situations:
|
||||
|
||||
- *A compact handle, for use where context already makes clear it's a Toolshed item.* Inside the
|
||||
app, in exports, in logs, anywhere the reader already knows they're looking at Toolshed data,
|
||||
the handle doesn't need to spell that out or be openable on its own. This can be as short as
|
||||
`user@domain.tld:id`, the item's owner handle with `:id` appended, mirroring how a tag/category
|
||||
handle already appends `:name` after its origin (see federation.md's Unique Handles section).
|
||||
No new delimiter concept, just the same pattern applied to items.
|
||||
|
||||
- *A self-contained URL, for use with no context at all.* A physical label, a link shared outside
|
||||
the app, has to work without the reader already knowing what it is or which server it belongs
|
||||
to, so it needs to open directly to the right frontend, resolve the right backend, and land on
|
||||
the right item. That means it has to encode the same information (owner handle + item id) as a
|
||||
full URL, e.g. `https://toolshed.webdomain.tld/i/alice@example.com/42`, note the owner's handle
|
||||
can be embedded in a path segment as-is (`@` doesn't need escaping in a URL path), which keeps it
|
||||
one segment shorter than splitting the handle back into `domain/user`, and means the handle is
|
||||
visible unmodified inside the link rather than reassembled from separate parts. If the owner is
|
||||
ever a group rather than a user (see groups.md), its handle carries a leading `#`, which does need
|
||||
the `+`-for-`#` substitution described in [handles-and-shortids.md](../handles-and-shortids.md)'s
|
||||
Handle syntax section before it can sit in a path segment, e.g.
|
||||
`https://toolshed.webdomain.tld/i/+climbing@example.com/42`. The frontend host
|
||||
in this URL (`toolshed.webdomain.tld`) doesn't have to be, and generally won't be, the backend
|
||||
authoritative for `example.com`, any frontend can resolve any handle (see federation.md's Servers
|
||||
subsection), so this is just whichever frontend happens to be handling the link, not part of the
|
||||
item's identity.
|
||||
|
||||
Both formats should stay as short as the encoded information allows, this matters most for the URL
|
||||
form, since it's the one that ends up in a QR code or printed label where physical size is a real
|
||||
constraint (see the labels goal above).
|
||||
|
||||
**How does this fit with the frontend's existing routes?**
|
||||
There's already a `/inventory/shared/:user/:id` route (`InventoryDetailForeign`), but today
|
||||
`:user` is just a bare username with no domain, i.e. it only works for a friend on the viewer's own
|
||||
domain, and the view itself doesn't yet do anything domain-aware with that param. A resolvable
|
||||
global handle needs the full `user@domain.tld` and a lookup step this route doesn't have yet. Two
|
||||
ways to reconcile that: extend the existing route to take a full handle in the `:user` segment
|
||||
(`/inventory/shared/alice@example.com/42`, no new route shape needed, just a richer meaning for the
|
||||
param it already has), or treat the short `/i/...` URL as a dedicated, minimal entry point whose
|
||||
only job is to resolve a handle and then hand off into whatever the richer in-app view ends up
|
||||
being. The two aren't mutually exclusive: the short form is what needs to be small enough to print,
|
||||
the in-app route doesn't have the same constraint and can stay more descriptive.
|
||||
|
||||
**What does scanning a label actually do?**
|
||||
Probably: the label encodes a URL or handle-like string; scanning it opens the frontend, which
|
||||
resolves the owner's domain the same way it resolves any other handle (see federation.md), and
|
||||
lands on that item. This reuses the discovery mechanism that already exists for logging in as a
|
||||
handle, rather than inventing a second one.
|
||||
|
||||
**Does resolving a label require authorization?**
|
||||
An item's availability policy already controls who can see it (owner-only if private, friends if
|
||||
shared, etc.). A label should presumably respect the same policy rather than being a backdoor that
|
||||
makes a private item visible to literally anyone who finds the physical object and scans its code.
|
||||
That means resolving a label isn't a free public lookup, it goes through the same friend/signature
|
||||
checks as everything else, which has UX implications (an anonymous finder of a lost tool can't
|
||||
necessarily see who it belongs to).
|
||||
|
||||
**Does the label need to be opaque/unguessable?**
|
||||
If item ids are small sequential integers, a label built from a guessable id lets anyone enumerate
|
||||
a user's items by scanning or guessing nearby numbers, even if each individual lookup is
|
||||
authorization-checked. Probably wants some amount of unguessability even before authorization is
|
||||
considered, as a defense-in-depth measure.
|
||||
|
||||
**What survives item changes?**
|
||||
Storage location, availability policy, name, and description can all change over the life of an
|
||||
object without it becoming a "different" item. The label should point at the handle, not at any of
|
||||
that mutable data, so none of those changes require a new label. The one thing that probably *does*
|
||||
need a decision is deletion: does a handle ever get reused, or is it retired for good once an item
|
||||
is deleted (retiring seems safer, avoids an old label resolving to an unrelated new item later)?
|
||||
|
||||
**Relationship to lending/borrowing.**
|
||||
A scannable label is the obvious hook for a future "mark as borrowed / returned" flow. Not solving
|
||||
that now, but the handle scheme chosen here should be able to carry that later without a redesign,
|
||||
i.e. it should be able to identify the item independent of who currently physically has it.
|
||||
252
docs/design-in-progress/tags.md
Normal file
252
docs/design-in-progress/tags.md
Normal file
|
|
@ -0,0 +1,252 @@
|
|||
# Tags, Properties & Categories (Design in Progress)
|
||||
|
||||
Status: partially implemented (the handle scheme and strict resolution described in federation.md
|
||||
exist today); this document is about the rough edges and open questions in that design, not a
|
||||
proposal to build it from scratch.
|
||||
|
||||
## Problem
|
||||
|
||||
Tags, properties, and categories are handles of the form `origin#type:name` (see federation.md's
|
||||
"Unique Handles" section), and a reference to a handle a server doesn't recognize is left
|
||||
unresolved rather than guessed at or merged. That strictness is deliberate and avoids silent data
|
||||
corruption, but it pushes a real cost onto users and creates some open problems:
|
||||
|
||||
- **Discoverability.** There's no way today to search or browse "what origins/taxonomies already
|
||||
exist" before creating a new tag. In practice this likely means people invent their own local
|
||||
tags for things a shared, canonical origin already covers, simply because they didn't know it
|
||||
existed.
|
||||
- **Fragmentation.** Because uniqueness is scoped per origin, nothing stops semantically identical
|
||||
concepts from existing under different names or different origins at once (`drill` vs.
|
||||
`power-drill`, one server's `origin:mytools` vs. another's `origin:community-tools`). Nothing
|
||||
reconciles these; they just coexist.
|
||||
- **No aliasing/synonyms.** If a user starts out with a local tag and later wants to adopt a shared
|
||||
origin's equivalent, there's no supported way to merge or alias the two; existing items keep
|
||||
pointing at the old handle.
|
||||
- **Trust and quality of shared origins.** An origin is "whatever the classification is considered
|
||||
to have come from," which could be an imported reference dataset. Nothing in the current design
|
||||
addresses who maintains such a dataset, how it's kept correct, or what happens when it changes
|
||||
(a category gets renamed or split upstream, e.g.).
|
||||
- **Localization.** A tag/category name is a single string. There's no notion of the same concept
|
||||
having a different display name per language while still resolving to the same handle.
|
||||
- **Property values aren't typed.** A property's value on an item is a plain string. A property
|
||||
definition can carry a unit (`unit_symbol`/`unit_name`), but nothing declares that its values are
|
||||
actually numbers meant to be read in that unit, versus, say, free text that happens to mention a
|
||||
unit. This is already a problem for one server in isolation (see the worked example below), and
|
||||
federation doesn't create it, it just means it now has to be solved consistently across origins
|
||||
instead of once.
|
||||
- **Handle collisions.** `origin` is a free-text string with nothing enforcing that it's actually
|
||||
unique to one definition. Fragmentation (above) is two different strings for the same thing;
|
||||
this is the more dangerous mirror image, the same handle string ending up attached to two
|
||||
different definitions, which is exactly the scenario the strict-resolution design was supposed
|
||||
to make impossible (see the second worked example below).
|
||||
|
||||
### Worked example: filtering by a numeric range across origins
|
||||
|
||||
Say two independently-run servers both end up with a property called "voltage," each under its own
|
||||
origin: `git:base#property:voltage` and `some-other-origin#property:voltage`. A user who's friends
|
||||
with people on both wants to filter their combined view of shared items by, say, `0 < voltage <
|
||||
4`. Two separate problems stack up:
|
||||
|
||||
1. **Are these the same property?** Per the fragmentation problem above, the two handles are, and
|
||||
stay, unrelated as far as the system is concerned, same name, different origin, no connection.
|
||||
A range filter built against one handle simply won't match items tagged with the other, even
|
||||
though a person looking at both would probably call them "the same thing."
|
||||
2. **Even if they were recognized as the same thing, are the values comparable?** A range filter
|
||||
needs actual numbers in a known unit. If one server's items store `"3.7"` and the other's store
|
||||
`"3700"` (volts vs. millivolts), a numeric comparison across the two silently produces nonsense
|
||||
unless the unit is known and converted. If either side stores the value as loosely-formatted
|
||||
text (`"3.7V"`, `"~3.7"`) rather than a bare number, it may not be reliably parseable as a
|
||||
number at all.
|
||||
|
||||
So a cross-origin range filter needs both an aliasing/equivalence answer (are `voltage` and
|
||||
`voltage` the same concept) and a units/typing answer (are their values actually numbers, in units
|
||||
that convert cleanly into each other), and the first being solved doesn't imply the second is.
|
||||
|
||||
### Worked example: the same handle meaning two different things
|
||||
|
||||
Say `git:ee2` names a specific reference dataset that started life in one shared git history, and
|
||||
two servers each imported it, at different times, from what has since become two diverging
|
||||
branches (or forks) of that history. Both servers now have a property whose handle is the exact
|
||||
same string, `git:ee2#property:charging_voltage`, but whose actual definition, say, unit, or
|
||||
dimensions has since diverged between the two branches. Neither server did anything wrong; each
|
||||
one faithfully imported "`git:ee2`" as it existed at the time.
|
||||
|
||||
This is a materially worse problem than the fragmentation/voltage example above. Fragmentation is
|
||||
a missed opportunity, two things that should be linked aren't, and the failure is visible (the
|
||||
filter just doesn't match as much as a person would expect). A handle collision is silent: nothing
|
||||
about the two servers exchanging data suggests anything is wrong, both sides say
|
||||
`git:ee2#property:charging_voltage`, so anything that trusts equal-handle-means-equal-definition
|
||||
(exactly what the strict-resolution design promises, and exactly what a filter, an alias, or a
|
||||
plain item-detail display would rely on) can silently combine or display incompatible values as if
|
||||
they were the same thing. This is the specific failure mode the whole handle design exists to
|
||||
prevent, so a scheme where it can still happen is a real gap, not just an inconvenience.
|
||||
|
||||
The root cause is that `origin` is a free-text label describing where something came from, not an
|
||||
identifier that's actually bound to a specific, fixed piece of content. A name like `git:ee2` reads
|
||||
as if it points to something immutable, but nothing about the origin field enforces that, `ee2`
|
||||
could easily be a branch or tag name rather than a specific commit, i.e. a pointer that can keep
|
||||
moving, and two importers pinned to it at different times without ever taking on a different
|
||||
handle to show for it.
|
||||
|
||||
This isn't hypothetical, it's exactly how the reference data included in this repository already
|
||||
works. The files in `backend/shared_data/` (`base.json`, `ee.json`, etc.) are origin datasets:
|
||||
`configure.py`'s import step sets `origin = "git:" + filename` for everything a file defines
|
||||
(`configure.py:119`), and a file can declare a fixed dependency on another one by name, e.g.
|
||||
`ee.json`'s `"depends": ["git:base"]`. The intent is clearly that these files are immutable once
|
||||
committed, and that a `depends` entry is a pin to a specific, settled parent, not a moving target,
|
||||
but nothing in the code enforces that today, it's a convention people are expected to follow.
|
||||
|
||||
There's already a piece of the machinery needed to enforce it, though: import already computes a
|
||||
sha256 of each file's raw content and stores it (`configure.py:125`, saved onto
|
||||
`ImportedIdentifierSets.hash`, which is `unique=True` alongside `name`, `hostadmin/models.py:15`).
|
||||
It's just not used for the thing it would be useful for, the "already imported, skipping" check
|
||||
(`configure.py:129-130`) matches on `name` alone, it never recomputes the hash of the file being
|
||||
imported and compares it against the hash already on record for that name. So even on a single
|
||||
server, an edited `ee.json` re-imported under its old filename wouldn't be noticed as a change,
|
||||
let alone flagged as a conflict, and the hash never leaves that server's own bookkeeping to be
|
||||
compared against what a friend server has on record for the same name.
|
||||
|
||||
## Goals
|
||||
|
||||
- Make it easy to find and reuse an existing origin/handle before inventing a new local one, to
|
||||
reduce fragmentation without weakening the strict-resolution guarantee that already exists.
|
||||
- Give users a path to move a locally-invented tag onto a shared origin later, without losing or
|
||||
having to manually re-tag their existing items.
|
||||
- Keep the core guarantee intact: a handle always means one specific, traceable thing, nothing
|
||||
should be implicitly merged or reinterpreted across origins.
|
||||
- Make that guarantee actually hold, not just assumed: two servers that both use a given handle
|
||||
should either really mean the same definition, or have some way to find out they don't, rather
|
||||
than the collision staying silent.
|
||||
|
||||
## Non-goals (for now)
|
||||
|
||||
- Building a moderation/governance system for shared origins. Worth thinking about, but a bigger
|
||||
problem than this document is trying to scope.
|
||||
- Free-text/fuzzy tag matching in search. Search UX can layer on top of resolved handles without
|
||||
changing what a handle means.
|
||||
|
||||
## Open design ideas
|
||||
|
||||
**A small set of well-known, shipped origins.**
|
||||
Toolshed could ship with one or a few canonical origins covering common tool/inventory categories
|
||||
out of the box, so that a fresh server already has a sensible baseline vocabulary to reuse instead
|
||||
of every server reinventing "drill," "screwdriver," "power tools," etc. independently. Doesn't
|
||||
solve fragmentation for everything, but raises the floor.
|
||||
|
||||
**Explicit aliasing rather than merging.**
|
||||
Rather than trying to detect and merge "equivalent" tags automatically (risky, exactly the kind of
|
||||
implicit behavior the strict-resolution design intentionally avoids), a tag could carry an explicit,
|
||||
user-initiated "supersedes"/"alias of" pointer to another handle. Items already tagged with the old
|
||||
handle could then be offered a one-time, explicit re-tag rather than a silent change of meaning.
|
||||
|
||||
For properties specifically, an alias needs to claim more than "these mean the same thing," it
|
||||
needs to claim the values are comparable, which means recording a unit conversion (possibly just
|
||||
"identical unit, factor 1") alongside the alias, not just a bare pointer. An alias with no stated
|
||||
conversion should probably be treated as "same concept, values not (yet) comparable," a range
|
||||
filter has no business guessing a conversion on its own.
|
||||
|
||||
**Filters default to per-handle, and only widen on an explicit alias.**
|
||||
Following directly from the strict-resolution philosophy in federation.md: a range filter should
|
||||
only ever combine two distinct property handles into one filterable facet because of an explicit
|
||||
alias (see above) that also states the values are comparable, never because their names or units
|
||||
happen to match. Absent that, two same-named properties from different origins should just show up
|
||||
as two separate filters, visibly distinct, rather than the UI silently guessing they're the same
|
||||
and producing a filter result that mixes incomparable values.
|
||||
|
||||
**Typed property values.**
|
||||
Giving a property definition a declared value type (number, text, boolean, ...) in addition to its
|
||||
existing unit metadata would let both the frontend and the alias/conversion mechanism above know
|
||||
whether "range filter" even applies to a given property, and would close the gap where a value
|
||||
happens to look numeric but isn't guaranteed to parse as one. This is useful even without
|
||||
federation in the picture, cross-origin comparison just makes the gap load-bearing instead of
|
||||
cosmetic.
|
||||
|
||||
**Origin metadata/versioning.**
|
||||
If an origin represents an imported dataset, giving it its own version or changelog would let a
|
||||
server know when the upstream taxonomy it imported has moved on, and decide explicitly whether to
|
||||
re-import, rather than silently drifting from what other servers using the "same" origin now have.
|
||||
This only helps if everyone's still on one shared timeline, though, it doesn't by itself address
|
||||
diverging forks/branches ending up with the same name (see below).
|
||||
|
||||
**Pin origins to immutable content, not movable names.**
|
||||
The `git:ee2` collision happens because the origin string names something mutable (a branch/tag)
|
||||
rather than something fixed. If an origin string were derived from the content itself, e.g. a hash
|
||||
of the definition, or a specific immutable commit rather than a branch, two independent imports
|
||||
could never end up with the same string unless the content was actually identical at that point,
|
||||
collisions would become structurally impossible rather than just unlikely. This is a bigger change
|
||||
than the versioning idea above: it's not tracking change over time, it's making the identifier
|
||||
itself incapable of silently referring to different things.
|
||||
|
||||
A concrete version of this: use the git blob hash of the file a property/tag/category was defined
|
||||
in as (part of) its origin. If the reference dataset already lives in a git repo, this is free,
|
||||
git's already computed it, and it's independently checkable, any server holding or able to fetch
|
||||
the same repo can recompute the hash from the content and confirm for itself, rather than trusting
|
||||
a label. This is a real improvement over free-text `origin` strings, but it isn't a free lunch:
|
||||
|
||||
- *Granularity.* A blob hash identifies a whole file, not a single property. If a file defines
|
||||
several properties together, editing any one of them changes every other property's "identity"
|
||||
in the same file too, even though nothing about them changed. Either definitions need to be
|
||||
one-per-file for the hash to mean what's intended, or the hash needs to cover just the relevant
|
||||
entry rather than the literal git blob.
|
||||
- *It converts every edit into a fork.* Since any change, including a typo fix, changes the hash,
|
||||
a routine upstream correction mechanically fragments what's still the same property into two
|
||||
handles. That's consistent with "never silently reinterpret a handle," but it means the aliasing
|
||||
mechanism above stops being a nice-to-have and becomes the primary upgrade path, every legitimate
|
||||
edit needs an explicit "supersedes" link, or old items are stranded on a stale, now-orphaned hash.
|
||||
- *A hash alone is an identity, not a location or a label.* It proves two things are the same (or
|
||||
aren't), but doesn't say where to fetch the content from if you don't already have it, and isn't
|
||||
human-readable. Pairing it with a location (which repo) and a mnemonic (which release/name it
|
||||
corresponds to) alongside the hash keeps the discoverability goal intact instead of trading it
|
||||
away for collision-proofing.
|
||||
|
||||
**Detect collisions on contact, as a backstop.**
|
||||
Even with better-behaved identifiers going forward, existing data and human-typed origin strings
|
||||
mean collisions can't be ruled out entirely. Whenever two servers interact over a handle they both
|
||||
claim to know (e.g. as part of resolving an alias, or federated search), comparing a fingerprint of
|
||||
the full definition, not just the handle string, would let a mismatch surface as an explicit
|
||||
conflict to resolve, rather than being silently trusted. This is the same instinct as the "explicit
|
||||
alias must state whether things are comparable" idea above, applied in the opposite direction, here
|
||||
the handles already match and the system needs to actively check whether that trust is warranted.
|
||||
|
||||
The needed ingredient already exists locally and just isn't being used this way: the sha256 hash
|
||||
already computed and stored per import (see above) is exactly the kind of definition fingerprint
|
||||
this needs. Two changes would make it actually do the job: first, comparing it on every import
|
||||
(including a "re-import" of a name already on record), not just recording it once, so a locally
|
||||
edited file gets caught before it's ever presented to anyone else, and second, exchanging it as
|
||||
part of whatever federated interaction references a shared-origin handle, so two servers can
|
||||
compare hashes for the same name and find out they've diverged instead of assuming they haven't.
|
||||
|
||||
**What this means for the handle actually on the wire.**
|
||||
Putting the above together, the everyday handle shouldn't change shape at all. It stays
|
||||
`origin#type:name`, e.g. `git:ee2#property:charging_voltage`, exactly as it is today. The reason is
|
||||
redundancy: this string is what appears on every single reference (every item's tag list, every
|
||||
property assignment), potentially many times per item across many items, while a hash only ever
|
||||
needs to be known once per origin. Carrying a full hash on every occurrence would repeat the same
|
||||
value over and over for no benefit beyond what knowing it once already provides.
|
||||
|
||||
Instead, the hash stays where it already lives, attached to the origin as a whole (extending
|
||||
`ImportedIdentifierSets`, see above), and gets exchanged at the points where two servers actually
|
||||
need to agree on one, e.g. the first time a friend's item references an origin a server doesn't
|
||||
already have a hash on record for. First contact just records it, same as resolving any unfamiliar
|
||||
handle today; a later mismatch against what's on record is the collision, and that's the point
|
||||
where it needs to become visible rather than silently trusted.
|
||||
|
||||
Only once a collision has actually been found does the wire format need to say more than
|
||||
`origin#type:name`, because at that point there genuinely are two different things sharing a name
|
||||
and something has to distinguish them for a person sorting it out. A short, abbreviated hash
|
||||
appended to the origin, the same idea git itself relies on for short commit hashes, keeps that
|
||||
escape hatch usable: `git:ee2~0f3a9c1e#property:charging_voltage` versus
|
||||
`git:ee2~7bc82a04#property:charging_voltage`. This longer form is exception-path plumbing for
|
||||
resolving an already-detected conflict, not something that changes the size or shape of handles in
|
||||
the common case.
|
||||
|
||||
**Search across known origins.**
|
||||
Before creating a new tag/category, a creation flow could search across origins the local server
|
||||
already knows about (its own, plus any it's imported) and surface likely existing matches. This is
|
||||
a UX/workflow fix rather than a change to the handle model itself, it doesn't need to touch
|
||||
resolution semantics at all.
|
||||
|
||||
**Display name vs. handle.**
|
||||
Separating "the name that appears in the handle" (stable, part of the identity) from "the label
|
||||
shown to a user" (translatable, cosmetic) would allow localization without affecting resolution or
|
||||
uniqueness, since resolution would stay keyed on the handle, not the display string.
|
||||
141
docs/glossary-todo.md
Normal file
141
docs/glossary-todo.md
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
# Glossary terminology TODO
|
||||
|
||||
Working list from a repo-wide audit of where code/docs use a different word for a concept that
|
||||
[glossary.md](glossary.md) already gives a canonical name. Nothing here has been changed yet, this
|
||||
is a collection point before any renaming/edit work starts. Grouped by glossary term; only real
|
||||
inconsistencies are listed, not every correct usage that was checked and cleared.
|
||||
|
||||
## Backend
|
||||
|
||||
- `frontend/src/federation.js` — the whole module (`class ServerSet`, `add(server)`, every
|
||||
request method) talks about "server" throughout where the concept is a **Backend**.
|
||||
- `frontend/src/store.js` — `getHomeServers`, `getFriendServers`, `getAllKnownServers`,
|
||||
`setAllFriendsServers`/`all_friends_servers`, `home_servers`, `lookupServer` (~line 313, 325,
|
||||
332, 359, 76-77, 107-108, 334).
|
||||
- `frontend/src/views/Friends.vue:16,43` — user-visible table column labeled "Server".
|
||||
- `docs/design-in-progress/tags.md` (lines 10, 20, 45, 76-78, 117, 204, 228-229) — "server" used
|
||||
throughout for what the glossary calls Backend (same looseness federation.md already has, but
|
||||
worth normalizing here too since tags.md is in active editing).
|
||||
- `issues.md:58` — "federated home servers" / "user identity" conflation.
|
||||
- `deploy/dev/docker-compose.yml` (`instance_a`/`instance_b`) and `docs/development.md:106-108` —
|
||||
"instance"/"backend instance" as a near-synonym for Backend.
|
||||
- `cli-client/toolshed-client.py:11-70`, `README.md:92` — `--host`/`self.host` for "which backend
|
||||
to talk to".
|
||||
|
||||
## Discovery
|
||||
|
||||
- `frontend/src/store.js:313` — `lookupServer` action *is* the discovery operation, never named
|
||||
"discovery".
|
||||
- `frontend/src/store.js:347` — `could not resolve server for friend` — "resolve" used instead.
|
||||
- `docs/design-in-progress/items-labels.md:99`, `docs/development.md:108` — describe the
|
||||
discovery operation via "resolves"/"direct the frontend to the correct backend" without naming
|
||||
it (minor, but candidates for a one-word tightening).
|
||||
|
||||
## Handle / User handle
|
||||
|
||||
- ~~`backend/authentication/models.py` (`class KnownIdentity`...), `signature_auth.py`
|
||||
(`author_identity`...), `frontend/src/identity.js` (`serializeIdentityRecord`...)~~ — no longer a
|
||||
finding: the glossary now has an explicit **Identity** entry (handle + keypair, held together as
|
||||
the unit a backend trusts), and this is exactly what these already name. No renaming needed here;
|
||||
if anything, these are the parts of the codebase the new Identity entry should point to as
|
||||
reference implementations.
|
||||
- `backend/toolshed/serializers.py:49-57` (`FriendSerializer`) — API field is literally named
|
||||
`"username"` but its value is a full handle (`username + '@' + domain`). Already
|
||||
self-acknowledged in a comment at `frontend/src/store.js:404`. **Highest-value single fix** —
|
||||
it's a live API contract, not just an internal name.
|
||||
- `frontend/src/store.js` — several action params destructured as `{username}` that actually carry
|
||||
a full handle: `lookupServer` (313), `getFriendServers` (359), `fetchFriendProfile` (401-405),
|
||||
`login` (276-282).
|
||||
- `frontend/src/views/Login.vue` (lines 24, 27-28, 82-83, 102-105, 115-117) — form label/variable
|
||||
"Username" for a field that must be a full user handle (`user@domain`, per its own validation
|
||||
message at line 103).
|
||||
- `frontend/src/router.js:51` — route param `/inventory/shared/:user/:id` uses `:user` for what's
|
||||
meant to eventually be a full handle; contrast with the sibling route at line 61 which already
|
||||
correctly uses `:handle`. (Already called out by items-labels.md itself, so low-risk to leave
|
||||
as-is, but listed for completeness.)
|
||||
|
||||
## Availability policy, Friend/Friendship, Signature/Signing, Strict resolution, Actor, Targeted sharing
|
||||
|
||||
No real inconsistencies found — implemented code already uses the glossary's own terms
|
||||
consistently (`availability_policy` field name throughout backend+frontend; `friend`/`befriend`
|
||||
consistently; `Signature`/`sign`/`verify` consistently; `_HandleNotFound`/`_resolve_handle` in
|
||||
`backend/toolshed/offlinedata.py` implement strict resolution faithfully without needing to name
|
||||
it; Actor and Targeted sharing are unimplemented with no competing name anywhere).
|
||||
|
||||
- Checked and cleared, not a real conflict: `frontend/src/neigbors.js`'s `NeighborsCache`/
|
||||
"neighbor" vocabulary — refers to unreachable backend *domains* during discovery, not to
|
||||
friendship, despite reading like a synonym at a glance.
|
||||
|
||||
## Group / Group handle / Membership list
|
||||
|
||||
- `backend/backend/settings.py:36` — `django.contrib.auth` ships a built-in `Group` model, shown
|
||||
as "Groups" in the Django admin. Not unregistered anywhere. Will collide by name with the
|
||||
proposed actor-type Group once that's implemented — worth a decision now (unregister the
|
||||
built-in admin Group, or otherwise disambiguate) before the real feature lands.
|
||||
- `issues.md` (issue #3, "Group Concept", ~lines 62-156) — a standalone proposal that conflicts
|
||||
with the already-settled `docs/design-in-progress/groups.md` design on three points at once:
|
||||
- `Group.public_key`/`private_key` fields (contradicts "does a group need its own keypair? No").
|
||||
- `GroupMembership` backed by a signed `membership_certificate` rather than a plain membership
|
||||
list (contradicts the glossary's Membership list entry).
|
||||
- Bare `Group.handle` strings with no `#` prefix, e.g. `"makerspace-nord"`, and a
|
||||
`GroupInvitationIncoming.group_handle` field/API surface (`POST /api/groups/` etc., ~lines
|
||||
65-66, 117-121, 150-156) that never uses the `#groupname@domain` shape.
|
||||
This is a design-conflict issue, not a wording tweak — `issues.md` should be reconciled with (or
|
||||
explicitly marked superseded by) `groups.md` before anyone implements from it.
|
||||
|
||||
## Keypair / Private key / Public key
|
||||
|
||||
- Wire-format drift on the one field that actually crosses the network: `befriender_key` is used
|
||||
for a public key at `frontend/src/store.js:435,449` and `backend/toolshed/api/friend.py:107`,
|
||||
while the model field, serializer field, and UI all call the same value
|
||||
`befriender_public_key`/`public_key` (`backend/authentication/models.py:144`,
|
||||
`backend/toolshed/serializers.py:65`, `backend/toolshed/api/friend.py:118`,
|
||||
`frontend/src/views/Friends.vue:81`).
|
||||
- `cli-client/toolshed-client.py` (`--key`, `TOOLSHED_KEY`, `self.signing_key`, ~lines 12, 52, 57)
|
||||
and `README.md:92` — never say "private key," just "key"/"Toolshed key", even though it's
|
||||
specifically the private half.
|
||||
|
||||
## Origin
|
||||
|
||||
- `backend/configure.py:130` ("Identifier set {} already imported, skipping") and the model
|
||||
`ImportedIdentifierSets` (`backend/hostadmin/models.py:13-19`) — call an imported origin dataset
|
||||
an "identifier set".
|
||||
- `issues.md:206-210` — Instance Admin TODO list: "identifier-sets" for **Origin** and bare
|
||||
"identifiers" for **Classification handle**, both alternate terms not matching glossary names.
|
||||
|
||||
## Alias
|
||||
|
||||
- `backend/shared_data/ee_packages.json:40,64` — two tags already carry an `"alias"` field in real
|
||||
data (e.g. `SOT54` → `alias: "TO-92"`), but shaped as a bare name string, not an
|
||||
`origin#type:name` handle pointer, and with no unit-conversion concept. It's silently dropped on
|
||||
import today (`Tag`/`TagSerializer` have no `alias` field). Not a different-word issue, but the
|
||||
design doc (which calls Alias "Proposed") doesn't acknowledge this pre-existing, inert
|
||||
precedent — worth reconciling either the data or the doc.
|
||||
|
||||
## Tag / Property / Category
|
||||
|
||||
- `frontend/src/components/workflow/workflows/BulkItemImportWorkflow.vue:495` — CSV
|
||||
column-auto-mapping heuristic treats `"type"` as a synonym for Category:
|
||||
`lowerColumn.includes('category') || lowerColumn.includes('type')`.
|
||||
|
||||
## Item Handle
|
||||
|
||||
- `frontend/src/views/Search.vue:30,52,110` — a field literally named `handle` is computed here
|
||||
(`e.owner==this.user ? e.id : "shared/"+e.owner+"/"+e.id`), but it's a router-path fragment, not
|
||||
an Item Handle: no domain-qualified `user@domain.tld:id` shape, and `e.owner` is a bare username.
|
||||
Whoever implements the real Item Handle later is likely to collide with this existing variable.
|
||||
|
||||
## Item Label
|
||||
|
||||
- `frontend/src/components/workflow/workflows/FotoFirstBulkImportWorkflow.vue:567-572,822` and
|
||||
`FotoFirstBulkImportWorkflow2.vue:593-598,847` — checkbox "Generate QR codes for items" /
|
||||
`importOptions.generate_qr_codes` names exactly the Item Label concept but never uses that term
|
||||
(and the option is currently unwired — declared and defaulted `true` but never read elsewhere).
|
||||
|
||||
## Item URL / Local id / Domain / Frontend / Definition fingerprint / Fragmentation / Handle collision / Classification handle
|
||||
|
||||
No real inconsistencies found — each already uses consistent, glossary-matching vocabulary
|
||||
(`id`/`item_id` for Local id; `origin` kept cleanly separate from `domain` everywhere it's used;
|
||||
`get_handle()` consistently for Classification handle; no competing names found anywhere for
|
||||
Fragmentation, Handle collision, or Definition fingerprint, which also doesn't collide with the
|
||||
unrelated `File.hash` content-hash field despite both being called "hash").
|
||||
229
docs/glossary.md
Normal file
229
docs/glossary.md
Normal file
|
|
@ -0,0 +1,229 @@
|
|||
# Glossary
|
||||
|
||||
This page collects one unambiguous name for each concept discussed in [federation.md](federation.md)
|
||||
and the [design-in-progress](design-in-progress/) documents, so later writing can refer to them
|
||||
consistently instead of reinventing or subtly renaming them. Entries are grouped by topic, and
|
||||
alphabetical within each group. Each one notes whether the concept exists in Toolshed today or is
|
||||
still a design proposal, and links back to where it's discussed in full.
|
||||
|
||||
Two terms are easy to conflate and worth telling apart up front: a **domain** is who a handle
|
||||
belongs to; an **origin** is where a classification entry came from. They look similar in prose but
|
||||
are unrelated concepts, see their entries below.
|
||||
|
||||
---
|
||||
|
||||
## Federation Basics
|
||||
|
||||
**Backend** (Implemented)
|
||||
The server software that stores an [actor](#actor)'s data and is currently authoritative for a
|
||||
[domain](#domain). Deliberately decoupled from both the domain (a backend isn't tied to one domain,
|
||||
and can be authoritative for many at once) and the [frontend](#frontend) (the app a user loads isn't
|
||||
necessarily served by the backend that ends up handling their requests). Prefer "backend" over the
|
||||
looser word "server" when precision matters, "server" is used informally in places to mean either
|
||||
backend or domain interchangeably.
|
||||
*See: [federation.md](federation.md#servers)*
|
||||
|
||||
**Discovery** (Implemented)
|
||||
The lookup a [frontend](#frontend) performs to find which [backend](#backend) is currently
|
||||
authoritative for a [domain](#domain), given a [handle](#handle). Operational/DNS-level detail
|
||||
about how this lookup works belongs in the deployment docs, not here, this glossary only fixes the
|
||||
name for the concept.
|
||||
*See: [federation.md](federation.md#servers)*
|
||||
|
||||
**Domain** (Implemented)
|
||||
The half of a [handle](#handle) after the `@`, e.g. `toolsheddomain.tld`. An authority record, not
|
||||
a location or a piece of software: it says which [backend](#backend) currently vouches for handles
|
||||
under it, nothing more. Not the same thing as an [origin](#origin), see the note at the top of this
|
||||
page.
|
||||
*See: [federation.md](federation.md#servers)*
|
||||
|
||||
**Frontend** (Implemented)
|
||||
The client application a user interacts with. Independent of any one [domain](#domain) or
|
||||
[backend](#backend): given a handle, it performs [discovery](#discovery) to find the right backend
|
||||
and talks to it directly, regardless of where the frontend itself was loaded from.
|
||||
*See: [federation.md](federation.md#servers)*
|
||||
|
||||
**Handle** (Implemented)
|
||||
A name that's unique within its own scope and carries, as part of itself, enough information to say
|
||||
where it's authoritative, without needing a central registry to look it up. The general term
|
||||
covering [user handles](#user-handle), [group handles](#group-handle) (proposed),
|
||||
[classification handles](#classification-handle), and [item handles](#item-handle) (proposed).
|
||||
*See: [federation.md](federation.md#unique-handles)*
|
||||
|
||||
**Strict resolution** (Implemented)
|
||||
The rule that a reference to a [handle](#handle) a backend doesn't recognize is left unresolved
|
||||
rather than guessed at or silently merged into something that looks similar. The foundational
|
||||
guarantee the rest of the handle system, and most of the open problems in
|
||||
[tags.md](design-in-progress/tags.md), are trying to preserve or actually make hold.
|
||||
*See: [federation.md](federation.md#tags-properties-and-categories)*
|
||||
|
||||
## Actors, Identity & Sharing
|
||||
|
||||
**Actor** (Proposed)
|
||||
Umbrella term for anything that can hold a handle, have friends, and own items. Today this only
|
||||
means [user](#user). [Groups](#group) are a proposed second kind of actor, so that "friendship,"
|
||||
"ownership," and "handle" all mean the same thing regardless of which kind of actor is involved.
|
||||
*See: [groups.md](design-in-progress/groups.md#are-group-handles-different-from-user-handles-or-is-a-group-just-a-special-kind-of-user)*
|
||||
|
||||
**Availability policy** (Implemented)
|
||||
A setting on an item (`private` / `share` / `lend` / `rent` / `sell`) controlling who besides the
|
||||
owner can see it. Today the audience for any non-`private` policy is implicitly "all of the owner's
|
||||
[friends](#friend-friendship)," equally, there's no way to name a narrower audience. See [targeted
|
||||
sharing](#targeted-sharing) for the proposed alternative.
|
||||
*See: [groups.md](design-in-progress/groups.md#interaction-with-availability-policy)*
|
||||
|
||||
**Friend / Friendship** (Implemented for users; proposed for groups)
|
||||
A mutual, explicitly-established trust relationship between two [actors](#actor). Established by a
|
||||
friend-request/accept exchange, which is also the point a [public key](#keypair-private-key-public-key)
|
||||
is first trusted for that handle. Currently only exists between users; groups having friends, and
|
||||
groups befriending groups, are proposed extensions of the same mechanism, not a new one.
|
||||
*See: [federation.md](federation.md#cryptography), [groups.md](design-in-progress/groups.md#should-a-group-be-able-to-grant-read-access-to-non-members-group-friends)*
|
||||
|
||||
**Group** (Proposed)
|
||||
A second kind of [actor](#actor), modeling collective ownership (a club, workshop, or company)
|
||||
rather than any one person owning something. All members hold equal edit rights over what the group
|
||||
owns; membership itself is the privilege, there's no separate owner/member distinction within a
|
||||
group. Backed by a [membership list](#membership-list) rather than a [keypair](#keypair-private-key-public-key),
|
||||
and identified by a [group handle](#group-handle).
|
||||
*See: [groups.md](design-in-progress/groups.md#what-a-group-is)*
|
||||
|
||||
**Group handle** (Proposed)
|
||||
A [group](#group)'s handle: a name and [domain](#domain) written like a [user handle](#user-handle)
|
||||
but prefixed with `#`, e.g. `#groupname@toolsheddomain.tld`. The prefix keeps groups and users in
|
||||
disjoint namespaces on the same domain (no squatting collision between a user and a group wanting
|
||||
the same name) and lets an actor's kind be read directly off its handle, without a lookup.
|
||||
*See: [federation.md](federation.md#users-and-groups), [groups.md](design-in-progress/groups.md#are-group-handles-different-from-user-handles-or-is-a-group-just-a-special-kind-of-user)*
|
||||
|
||||
**Identity** (Implemented)
|
||||
A [user handle](#user-handle) paired with the [keypair](#keypair-private-key-public-key) that backs
|
||||
it, held together as the one unit a [backend](#backend) actually trusts: not just a name, and not
|
||||
just key material, but both at once. This is what gets established at registration for one's own
|
||||
handle, and what gets recorded on [friend](#friend-friendship)-accept for someone else's handle.
|
||||
Only [users](#user) have an identity in this sense, since a [group](#group) is deliberately backed
|
||||
by a [membership list](#membership-list) instead of a keypair, there's no key half for a group
|
||||
handle to pair with.
|
||||
*See: [federation.md](federation.md#cryptography)*
|
||||
|
||||
**Keypair / Private key / Public key** (Implemented, users only)
|
||||
The asymmetric keypair backing exactly one [user handle](#user-handle); together, a handle and the
|
||||
keypair backing it are what's called an [identity](#identity). The private key signs requests made
|
||||
as that user and never leaves their control; the public key is handed out to establish trust in the
|
||||
handle (at registration, or via a [friend](#friend-friendship) exchange) and is used to verify
|
||||
signatures. Only user handles carry a keypair, not groups, classification handles, or item handles.
|
||||
*See: [federation.md](federation.md#cryptography)*
|
||||
|
||||
**Membership list** (Proposed)
|
||||
The record of which [user handles](#user-handle) currently belong to a [group](#group), maintained
|
||||
by whichever backend is authoritative for the group's handle. What backs a group's identity in
|
||||
place of a keypair: a request "as the group" is a normal signed request from a current member, plus
|
||||
a check against this list, not a request signed by some shared group key.
|
||||
*See: [groups.md](design-in-progress/groups.md#does-a-group-need-its-own-keypair)*
|
||||
|
||||
**Signature / Signing** (Implemented)
|
||||
The act of authenticating a request as genuinely coming from a specific [user
|
||||
handle](#user-handle), unaltered, using that handle's private key. Establishes authenticity and
|
||||
integrity only, not confidentiality (that's HTTPS's job) and not, today, protection against replay.
|
||||
*See: [federation.md](federation.md#cryptography)*
|
||||
|
||||
**Targeted sharing** (Proposed)
|
||||
Sharing an item with one specific [actor](#actor) (a particular friend, or a particular group)
|
||||
instead of the current all-or-nothing [availability policy](#availability-policy) audience of every
|
||||
friend equally. Flagged as a generalization useful beyond groups specifically, not a groups-only
|
||||
feature.
|
||||
*See: [groups.md](design-in-progress/groups.md#should-anyone-be-able-to-share-directly-with-a-specific-group-instead-of-with-my-friends-generally)*
|
||||
|
||||
**User** (Implemented)
|
||||
The original, and currently only implemented, kind of [actor](#actor): backed by exactly one
|
||||
[keypair](#keypair-private-key-public-key) and identified by a [user handle](#user-handle).
|
||||
*See: [federation.md](federation.md#users-and-groups)*
|
||||
|
||||
**User handle** (Implemented)
|
||||
A user's username paired with its [domain](#domain), written like an email address, e.g.
|
||||
`user@toolsheddomain.tld`. Unique only within its domain, not across all of Toolshed. Contrast with
|
||||
a [group handle](#group-handle), which is the same shape but prefixed with `#`. Paired with its
|
||||
[keypair](#keypair-private-key-public-key), the two together are called an [identity](#identity).
|
||||
*See: [federation.md](federation.md#users-and-groups)*
|
||||
|
||||
## Classification: Tags, Properties & Categories
|
||||
|
||||
**Alias** (Proposed)
|
||||
An explicit, one-directional "supersedes"/"alias of" pointer from one [classification
|
||||
handle](#classification-handle) to another, asserting they mean the same thing. Never inferred
|
||||
automatically, always a deliberate act, in keeping with [strict resolution](#strict-resolution). For
|
||||
a property alias specifically, also states a unit conversion (even if it's "identical unit, factor
|
||||
1"); without one, the values behind the two handles aren't assumed to be comparable.
|
||||
*See: [tags.md](design-in-progress/tags.md#open-design-ideas)*
|
||||
|
||||
**Classification handle** (Implemented)
|
||||
Umbrella term for a [tag](#tag-property-category), [property](#tag-property-category), or
|
||||
[category](#tag-property-category) handle, of the form `origin#type:name`, e.g.
|
||||
`git:base#property:length`. Distinct from a [user handle](#user-handle) or [item
|
||||
handle](#item-handle): it names a reusable classification concept, not an actor or an owned thing.
|
||||
*See: [federation.md](federation.md#tags-properties-and-categories)*
|
||||
|
||||
**Definition fingerprint** (Partially implemented)
|
||||
A hash of a classification entry's full definition, used to tell whether two [actors](#actor)
|
||||
that both use the same handle actually mean the same thing. A sha256 of each `shared_data/*.json`
|
||||
file is already computed and stored per import (`ImportedIdentifierSets.hash`), but it's only ever
|
||||
recorded, not compared, so it doesn't yet catch a [handle collision](#handle-collision) in
|
||||
practice.
|
||||
*See: [tags.md](design-in-progress/tags.md#open-design-ideas)*
|
||||
|
||||
**Fragmentation** (Known problem)
|
||||
Two different [classification handles](#classification-handle) that mean, or were intended to
|
||||
mean, the same real-world concept (different origins, or a locally-invented tag versus a shared
|
||||
one). The opposite failure from a [handle collision](#handle-collision): visible and merely
|
||||
wasteful, rather than silent and dangerous.
|
||||
*See: [tags.md](design-in-progress/tags.md#problem)*
|
||||
|
||||
**Handle collision** (Known problem)
|
||||
Two [actors](#actor) independently ending up with the exact same [classification
|
||||
handle](#classification-handle) string backing two different definitions, e.g. two servers that
|
||||
each imported `git:ee2` from what has since become diverging branches. The dangerous mirror image
|
||||
of [fragmentation](#fragmentation): silent, because nothing about the interaction signals that
|
||||
anything's wrong, both sides just say the same string.
|
||||
*See: [tags.md](design-in-progress/tags.md#worked-example-the-same-handle-meaning-two-different-things)*
|
||||
|
||||
**Origin** (Implemented)
|
||||
The first component of a [classification handle](#classification-handle), naming where that tag,
|
||||
property, or category came from, e.g. `git:base` in `git:base#property:length`. Not necessarily a
|
||||
server or a domain, it can equally be a shared reference dataset (like the files in
|
||||
`backend/shared_data/`) or a server's own locally-invented taxonomy. Not the same thing as a
|
||||
[domain](#domain), see the note at the top of this page.
|
||||
*See: [federation.md](federation.md#tags-properties-and-categories)*
|
||||
|
||||
**Tag / Property / Category** (Implemented)
|
||||
The three kinds of classification entity an item can reference, each identified by a
|
||||
[classification handle](#classification-handle). A property additionally carries unit metadata
|
||||
(`unit_symbol`/`unit_name`), though property *values* on an item are plain, undeclared-type
|
||||
strings today.
|
||||
*See: [federation.md](federation.md#tags-properties-and-categories), [tags.md](design-in-progress/tags.md)*
|
||||
|
||||
## Items & Physical Labels
|
||||
|
||||
**Item Handle** (Proposed)
|
||||
The compact identifier for a specific item that's meaningful outside its owner's own account:
|
||||
`user@domain.tld:id`, the owner's [user handle](#user-handle) plus a [local id](#local-id). Used
|
||||
where it's already clear from context that it's a Toolshed item, e.g. inside the app, in exports,
|
||||
in logs, so it doesn't need to spell that out or be openable on its own. Contrast with [Item
|
||||
URL](#item-url), the self-contained form for when no such context can be assumed.
|
||||
*See: [items-labels.md](design-in-progress/items-labels.md#open-design-questions)*
|
||||
|
||||
**Item Label** (Proposed)
|
||||
A physical, scannable encoding (QR code, barcode, or similar) of an item's [Item URL](#item-url),
|
||||
meant to be printed and stuck on the physical object it refers to.
|
||||
*See: [items-labels.md](design-in-progress/items-labels.md#goals)*
|
||||
|
||||
**Item URL** (Proposed)
|
||||
The self-contained URL form of an [Item Handle](#item-handle), for use with no context at all, e.g.
|
||||
an [Item Label](#item-label): `https://<any frontend>/i/user@domain.tld/id`. Has to open directly
|
||||
to the right frontend and land on the right item on its own, since the reader can't be assumed to
|
||||
already know what it is or which server it belongs to. Any frontend can serve this URL, the host
|
||||
named in it isn't part of the item's identity, only the handle in its path is.
|
||||
*See: [items-labels.md](design-in-progress/items-labels.md#open-design-questions)*
|
||||
|
||||
**Local id** (Implemented)
|
||||
An item's identifier as it exists today: unique only within its owner's own inventory, not
|
||||
meaningful outside that owner's account. The starting point both the [Item
|
||||
Handle](#item-handle) and [Item URL](#item-url) build on.
|
||||
*See: [federation.md](federation.md#items), [items-labels.md](design-in-progress/items-labels.md#problem)*
|
||||
168
docs/handles-and-shortids.md
Normal file
168
docs/handles-and-shortids.md
Normal file
|
|
@ -0,0 +1,168 @@
|
|||
# Handles and Short IDs
|
||||
|
||||
This is the syntax-level reference for two related but separate naming schemes. [federation.md](federation.md)'s
|
||||
"Unique Handles" section covers *why* Toolshed hands out handles at all and what each kind (user,
|
||||
group, tag/property/category) means conceptually; this document covers the parsing rules those
|
||||
handles have to follow once they're written down or embedded somewhere - legal characters and
|
||||
escaping. It also covers short ids end to end: a separate, newer scheme for packing small integer
|
||||
id chains into a compact token, implemented in `frontend/src/short-id.js`.
|
||||
|
||||
## Handle syntax
|
||||
|
||||
### Reserved characters
|
||||
|
||||
A username ends up embedded, unescaped, in several composite formats beyond its own handle, so it
|
||||
can't contain any character that already means something else in one of those: `@` (the
|
||||
user/domain separator in a user handle), `#` (the group-handle prefix, and the origin/type
|
||||
separator in a classification handle, see federation.md's Tags, Properties, and Categories
|
||||
section), `:` (the id delimiter in a proposed Item Handle, the type/name delimiter in a
|
||||
classification handle, and the delimiter in a signed request's `Authorization` header), `+`
|
||||
(reserved as the URL-embedding escape for `#`, see below), `~` (the short-id token prefix, see
|
||||
Short IDs below, and a proposed collision-disambiguation suffix delimiter on a tag's origin), and
|
||||
`/` (the path-segment delimiter every handle and id ultimately sits next to once embedded in a
|
||||
URL). This has to be enforced by an explicit validator rather than left to Django's default
|
||||
`UnicodeUsernameValidator` (`^[\w.@+-]+\Z`), which currently permits both `@` and `+` (its own
|
||||
regex doesn't happen to allow `#`, `:`, or `/`, but that's incidental, not a designed restriction).
|
||||
|
||||
### Embedding a `#`-bearing handle in a URL
|
||||
|
||||
A literal `#` can't appear unescaped in a URL path segment: per RFC 3986, `#` starts the URI's
|
||||
fragment component, so any URL-parsing client (a browser, a QR scanner, a link preview) treats
|
||||
everything from the first unescaped `#` onward as a fragment and never sends it to the server at
|
||||
all, before a request is even made, not merely a server-side quirk to work around. The usual fix is
|
||||
to percent-encode it (`%23`), but that's exactly the encode/decode step a self-contained item URL
|
||||
(a physical label, a link shared outside the app) is designed to avoid for anything that sits
|
||||
directly in a path segment (`@` needs no such treatment). Instead, whenever a handle containing a
|
||||
`#` (a group handle, or a tag/property/category
|
||||
handle) has to appear as a raw URL path segment, substitute `+` for `#` in that rendering only:
|
||||
`#groupname@domain` becomes `+groupname@domain` in a URL, and `origin#type:name` becomes
|
||||
`origin+type:name`. This is a URL-embedding convention, not a second handle format: the canonical
|
||||
handle, the one used in the API, in signed requests, in the database, and everywhere else a handle
|
||||
is written or displayed, is unchanged and is still written `#groupname@domain`. Reversing the
|
||||
substitution when parsing a path segment back into a handle is unambiguous only because `+` is
|
||||
otherwise forbidden in every field a handle is built from (see Reserved characters above); if a
|
||||
group name or tag name could itself contain a literal `+`, it would be indistinguishable from an
|
||||
escaped `#` once decoded.
|
||||
|
||||
Implemented in `frontend/src/handle-url.js` (`encodeHandleForUrl`/`decodeHandleFromUrl`).
|
||||
|
||||
## Short IDs
|
||||
|
||||
A general encoding for turning a small, fixed-shape list of integers into a compact, URL-safe
|
||||
token, with no server-side lookup table involved: the code *is* the data, nothing is stored
|
||||
server-side to make it resolvable. Originally proposed to answer items-labels.md's open "what does
|
||||
the handle/URL actually look like" question, but the encoding itself isn't item-specific; anything
|
||||
currently addressed by a short chain of small integers is a candidate. Implemented and tested in
|
||||
`frontend/src/short-id.js`; try it live at `/~<token>` (`frontend/src/views/ShortId.vue`), which
|
||||
decodes whatever token is in the URL and also lists worked examples for every registered kind.
|
||||
|
||||
### Shape: a kind tag, then a fixed list of integers
|
||||
|
||||
Every short id starts with a small, fixed-width **kind** tag saying which schema the rest of the
|
||||
bits should be read against, followed by exactly the integer fields that kind's schema calls for,
|
||||
in a fixed order. `kind` is a small, closed, slow-growing set, so it doesn't need to be
|
||||
self-delimiting the way the integer fields do: 2 bits directly name kinds 0-2, and the all-ones
|
||||
value (3) is an escape meaning "the real kind follows as the next field, offset by this direct
|
||||
range" - so kind 3 is encoded as escape + chunked-int `0`, kind 4 as escape + `1`, and so on. This
|
||||
costs nothing for a kind that already fits in the direct range, and keeps the tag itself extensible
|
||||
forever without ever having to widen it out from under codes that were already printed. A narrow
|
||||
tag only pays off if kind usage is actually skewed the way id values are (a few kinds dominate),
|
||||
which is why the registry below is ordered by expected frequency, cheapest (most-used) kind first:
|
||||
|
||||
| kind | name | fields | notes |
|
||||
|---|---|---|---|
|
||||
| 0 | `item` | `owner_identity_id`, `item_local_id` | dominant case - the primary physical-label use case |
|
||||
| 1 | `storage_location` | `owner_identity_id`, `storage_location_id` | also label-printed |
|
||||
| 2 | `category` | `category_id` | label-adjacent (tagging); global, no owner |
|
||||
| 3 | `workflow` | `owner_identity_id`, `workflow_id` | shared in-app, not printed - first to pay the escape's cost |
|
||||
| 4 | `group` | `group_id` | shared even less often; global, no owner |
|
||||
| 5 | `file` | `file_id` | least often shared standalone; global, deduplicated by content hash |
|
||||
|
||||
`owner_identity_id` is `KnownIdentity.pk` (`backend/authentication/models.py`), not
|
||||
`ToolshedUser.pk`. Every local account already has exactly one stable `KnownIdentity` row
|
||||
(`ToolshedUser.public_identity`, created once at registration and never recreated), and every
|
||||
friend this backend knows about - local or remote - is represented by that same table, unique on
|
||||
`(username, domain)`. So one small integer already stands in for "this owner, as known by this
|
||||
backend" for both cases, with no separate local-vs-remote branching needed, and it's the same row
|
||||
federation.md's Cryptography section already treats as the trust anchor for a handle's public key.
|
||||
It appears on `item`, `storage_location`, and `workflow` because their backing models
|
||||
(`InventoryItem`, `StorageLocation`, `WorkflowInstance`) all FK `ToolshedUser` directly; `category`,
|
||||
`group`, and `file` skip it because their models are global/unscoped (`Group` has an unowned
|
||||
`members` M2M, `File` is deduplicated globally by content hash), so a bare row id is already
|
||||
everything needed to look them up.
|
||||
|
||||
A short id is inherently scoped to the backend that minted it (an "owner" field is a row that only
|
||||
exists in, and only means anything to, that one backend's database), not a portable replacement for
|
||||
a `user@domain.tld` handle, which stays the form to use anywhere cross-domain resolution actually
|
||||
matters. Resolving a short id still goes through the same friend/signature checks as everything
|
||||
else, unchanged; nothing about how the code looks grants any authority of its own (see Guessability
|
||||
below).
|
||||
|
||||
### Packing one integer: dynamic bit depth
|
||||
|
||||
Each integer field is made self-delimiting with **continuation chunking** (UTF-8/LEB128-style):
|
||||
split the value into fixed-size chunks (4 data bits each, most-significant chunk first), each
|
||||
preceded by one continuation bit meaning "another chunk follows" (`1`) or "this is the last chunk"
|
||||
(`0`). A value like `42` (`0b101010`) needs two 4-bit chunks, costing 10 bits total (2 × (1
|
||||
continuation + 4 data)); `7` fits in one chunk, costing 5 bits. This was chosen over an
|
||||
Elias-gamma-style unary/delimiter scheme (encode the value's bit-length in unary, then that many
|
||||
literal bits): unary is cheaper for single-digit values but its prefix grows every time the value's
|
||||
bit-length grows, so it never wins once ids pass single digits, which is the common case here (auto
|
||||
increment database ids realistically sitting in the tens through low-hundred-thousands over an
|
||||
installation's life). A 4-bit chunk width is a reasonable fixed default across that whole range;
|
||||
per-field tuning was checked against both a uniform and a skewed (geometric) distribution and never
|
||||
won by more than a fraction of a character, not enough to justify a tuning knob.
|
||||
|
||||
### From bits to text: base64 without the byte layover
|
||||
|
||||
Standard base64 assumes byte-aligned (8-bit) input, grouping 3 bytes into 4 output characters and
|
||||
padding to a byte boundary before encoding. Since there's no byte layer here to begin with, the
|
||||
bit-packed stream is instead packed directly into 6-bit groups and mapped straight onto the
|
||||
URL-safe base64 alphabet (RFC 4648 §5: `-` and `_` in place of `+` and `/`), with the final
|
||||
character's unused low bits padded with zeros. That padding is safe by construction: the decoder
|
||||
always knows exactly how many integers a given kind calls for, and a chunk's continuation bit is
|
||||
`1 = more follows`, so a run of zero-padding at the very end can never be misread as "one more
|
||||
chunk" - it decodes as a terminated chunk, at which point every field the schema called for has
|
||||
already been produced and decoding simply stops. No `=` padding characters are needed either; those
|
||||
exist in classic base64 purely to communicate trailing-byte padding, and there is no byte layer
|
||||
here to need that.
|
||||
|
||||
### The leading `~`
|
||||
|
||||
Every token is prefixed with a literal `~`, so a short id in a URL looks like `~DyU`. Its only job
|
||||
is to mark "everything after me decodes as one of these": URL-safe base64 never produces a `~`
|
||||
itself, so the prefix can never be confused with the payload, and none of Toolshed's other
|
||||
path-segment formats (bare usernames, `user@domain` handles, slugs, plain numeric ids) start with
|
||||
`~` either. `~` is one of RFC 3986's `unreserved` characters (§2.3, the same class as letters,
|
||||
digits, `-`, `.`, and `_`), a stronger guarantee than merely being legal in a path segment: it's
|
||||
never a target for percent-encoding and never carries special meaning in any URI component, so a
|
||||
short id can be handed to any part of the stack without first checking which encoding rules apply
|
||||
there.
|
||||
|
||||
|
||||
### Worked examples
|
||||
|
||||
Encoding `kind = item` (0), `owner_identity_id = 7`, `item_local_id = 42`:
|
||||
|
||||
- `kind`: 2 fixed bits → `00`
|
||||
- `owner_identity_id = 7`: fits in one 4-bit chunk → 5 bits (`00111`)
|
||||
- `item_local_id = 42`: needs two 4-bit chunks → 10 bits (`1001001010`)
|
||||
|
||||
Total: 17 meaningful bits, padded to the next multiple of 6 (18) with one zero bit, yielding 3
|
||||
base64 characters: **`~DyU`**.
|
||||
|
||||
The same worked-out form for one example of every registered kind - `Bits` is the same
|
||||
space-separated segmentation (kind tag, escape offset if present, each field, then padding) the
|
||||
Examples table on `/~<token>` (`frontend/src/views/ShortId.vue`) shows for every registered kind:
|
||||
|
||||
| Kind | Fields | Serialized | Bits | Token |
|
||||
|---|---|---|---|---|
|
||||
| `item` | `owner_identity_id: 7`, `item_local_id: 42` | `[0, 7, 42]` | `00 00111 1001001010 0` | `~DyU` |
|
||||
| `storage_location` | `owner_identity_id: 3`, `storage_location_id: 1000` | `[1, 3, 1000]` | `01 00011 100111111001000 00` | `~Rz8g` |
|
||||
| `category` | `category_id: 5` | `[2, 5]` | `10 00101 00000` | `~ig` |
|
||||
| `workflow` | `owner_identity_id: 2`, `workflow_id: 9` | `[3, 2, 9]` | `11 00000 00010 01001 0` | `~wCS` |
|
||||
| `group` | `group_id: 11` | `[4, 11]` | `11 00001 01011` | `~wr` |
|
||||
| `file` | `file_id: 123` | `[5, 123]` | `11 00010 1011101011 0` | `~xXW` |
|
||||
|
||||
`workflow`, `group`, and `file` are kinds 3-5, so their `Bits` column shows the escape tag (`11`)
|
||||
followed by its own offset segment payload fields.
|
||||
|
|
@ -9,5 +9,6 @@ This is the documentation for the Toolshed project. It is a work in progress.
|
|||
- [Deploying Toolshed](deployment.md)
|
||||
- [Development Setup](development.md)
|
||||
- [About Federation](federation.md)
|
||||
- [Handles and Short IDs](handles-and-shortids.md)
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue