# Handles and Short IDs

This is the syntax-level reference for two related but separate naming schemes. [federation.md](federation.md)'s
"Unique Handles" section covers *why* Toolshed hands out handles at all and what each kind (user,
group, tag/property/category) means conceptually; this document covers the parsing rules those
handles have to follow once they're written down or embedded somewhere - legal characters and
escaping. It also covers short ids end to end: a separate, newer scheme for packing small integer
id chains into a compact token, implemented in `frontend/src/short-id.js`.

## Handle syntax

### Reserved characters

A username ends up embedded, unescaped, in several composite formats beyond its own handle, so it
can't contain any character that already means something else in one of those: `@` (the
user/domain separator in a user handle), `#` (the group-handle prefix, and the origin/type
separator in a classification handle, see federation.md's Tags, Properties, and Categories
section), `:` (the id delimiter in a proposed Item Handle, the type/name delimiter in a
classification handle, and the delimiter in a signed request's `Authorization` header), `+`
(reserved as the URL-embedding escape for `#`, see below), `~` (the short-id token prefix, see
Short IDs below, and a proposed collision-disambiguation suffix delimiter on a tag's origin), and
`/` (the path-segment delimiter every handle and id ultimately sits next to once embedded in a
URL). This has to be enforced by an explicit validator rather than left to Django's default
`UnicodeUsernameValidator` (`^[\w.@+-]+\Z`), which currently permits both `@` and `+` (its own
regex doesn't happen to allow `#`, `:`, or `/`, but that's incidental, not a designed restriction).

### Embedding a `#`-bearing handle in a URL

A literal `#` can't appear unescaped in a URL path segment: per RFC 3986, `#` starts the URI's
fragment component, so any URL-parsing client (a browser, a QR scanner, a link preview) treats
everything from the first unescaped `#` onward as a fragment and never sends it to the server at
all, before a request is even made, not merely a server-side quirk to work around. The usual fix is
to percent-encode it (`%23`), but that's exactly the encode/decode step a self-contained item URL
(a physical label, a link shared outside the app) is designed to avoid for anything that sits
directly in a path segment (`@` needs no such treatment). Instead, whenever a handle containing a
`#` (a group handle, or a tag/property/category
handle) has to appear as a raw URL path segment, substitute `+` for `#` in that rendering only:
`#groupname@domain` becomes `+groupname@domain` in a URL, and `origin#type:name` becomes
`origin+type:name`. This is a URL-embedding convention, not a second handle format: the canonical
handle, the one used in the API, in signed requests, in the database, and everywhere else a handle
is written or displayed, is unchanged and is still written `#groupname@domain`. Reversing the
substitution when parsing a path segment back into a handle is unambiguous only because `+` is
otherwise forbidden in every field a handle is built from (see Reserved characters above); if a
group name or tag name could itself contain a literal `+`, it would be indistinguishable from an
escaped `#` once decoded.

Implemented in `frontend/src/handle-url.js` (`encodeHandleForUrl`/`decodeHandleFromUrl`).

## Short IDs

A general encoding for turning a small, fixed-shape list of integers into a compact, URL-safe
token, with no server-side lookup table involved: the code *is* the data, nothing is stored
server-side to make it resolvable. Originally proposed to answer items-labels.md's open "what does
the handle/URL actually look like" question, but the encoding itself isn't item-specific; anything
currently addressed by a short chain of small integers is a candidate. Implemented and tested in
`frontend/src/short-id.js`; try it live at `/~<token>` (`frontend/src/views/ShortId.vue`), which
decodes whatever token is in the URL and also lists worked examples for every registered kind.

### Shape: a kind tag, then a fixed list of integers

Every short id starts with a small, fixed-width **kind** tag saying which schema the rest of the
bits should be read against, followed by exactly the integer fields that kind's schema calls for,
in a fixed order. `kind` is a small, closed, slow-growing set, so it doesn't need to be
self-delimiting the way the integer fields do: 2 bits directly name kinds 0-2, and the all-ones
value (3) is an escape meaning "the real kind follows as the next field, offset by this direct
range" - so kind 3 is encoded as escape + chunked-int `0`, kind 4 as escape + `1`, and so on. This
costs nothing for a kind that already fits in the direct range, and keeps the tag itself extensible
forever without ever having to widen it out from under codes that were already printed. A narrow
tag only pays off if kind usage is actually skewed the way id values are (a few kinds dominate),
which is why the registry below is ordered by expected frequency, cheapest (most-used) kind first:

| kind | name | fields | notes |
|---|---|---|---|
| 0 | `item` | `owner_identity_id`, `item_local_id` | dominant case - the primary physical-label use case |
| 1 | `storage_location` | `owner_identity_id`, `storage_location_id` | also label-printed |
| 2 | `category` | `category_id` | label-adjacent (tagging); global, no owner |
| 3 | `workflow` | `owner_identity_id`, `workflow_id` | shared in-app, not printed - first to pay the escape's cost |
| 4 | `group` | `group_id` | shared even less often; global, no owner |
| 5 | `file` | `file_id` | least often shared standalone; global, deduplicated by content hash |

`owner_identity_id` is `KnownIdentity.pk` (`backend/authentication/models.py`), not
`ToolshedUser.pk`. Every local account already has exactly one stable `KnownIdentity` row
(`ToolshedUser.public_identity`, created once at registration and never recreated), and every
friend this backend knows about - local or remote - is represented by that same table, unique on
`(username, domain)`. So one small integer already stands in for "this owner, as known by this
backend" for both cases, with no separate local-vs-remote branching needed, and it's the same row
federation.md's Cryptography section already treats as the trust anchor for a handle's public key.
It appears on `item`, `storage_location`, and `workflow` because their backing models
(`InventoryItem`, `StorageLocation`, `WorkflowInstance`) all FK `ToolshedUser` directly; `category`,
`group`, and `file` skip it because their models are global/unscoped (`Group` has an unowned
`members` M2M, `File` is deduplicated globally by content hash), so a bare row id is already
everything needed to look them up.

A short id is inherently scoped to the backend that minted it (an "owner" field is a row that only
exists in, and only means anything to, that one backend's database), not a portable replacement for
a `user@domain.tld` handle, which stays the form to use anywhere cross-domain resolution actually
matters. Resolving a short id still goes through the same friend/signature checks as everything
else, unchanged; nothing about how the code looks grants any authority of its own (see Guessability
below).

### Packing one integer: dynamic bit depth

Each integer field is made self-delimiting with **continuation chunking** (UTF-8/LEB128-style):
split the value into fixed-size chunks (4 data bits each, most-significant chunk first), each
preceded by one continuation bit meaning "another chunk follows" (`1`) or "this is the last chunk"
(`0`). A value like `42` (`0b101010`) needs two 4-bit chunks, costing 10 bits total (2 × (1
continuation + 4 data)); `7` fits in one chunk, costing 5 bits. This was chosen over an
Elias-gamma-style unary/delimiter scheme (encode the value's bit-length in unary, then that many
literal bits): unary is cheaper for single-digit values but its prefix grows every time the value's
bit-length grows, so it never wins once ids pass single digits, which is the common case here (auto
increment database ids realistically sitting in the tens through low-hundred-thousands over an
installation's life). A 4-bit chunk width is a reasonable fixed default across that whole range;
per-field tuning was checked against both a uniform and a skewed (geometric) distribution and never
won by more than a fraction of a character, not enough to justify a tuning knob.

### From bits to text: base64 without the byte layover

Standard base64 assumes byte-aligned (8-bit) input, grouping 3 bytes into 4 output characters and
padding to a byte boundary before encoding. Since there's no byte layer here to begin with, the
bit-packed stream is instead packed directly into 6-bit groups and mapped straight onto the
URL-safe base64 alphabet (RFC 4648 §5: `-` and `_` in place of `+` and `/`), with the final
character's unused low bits padded with zeros. That padding is safe by construction: the decoder
always knows exactly how many integers a given kind calls for, and a chunk's continuation bit is
`1 = more follows`, so a run of zero-padding at the very end can never be misread as "one more
chunk" - it decodes as a terminated chunk, at which point every field the schema called for has
already been produced and decoding simply stops. No `=` padding characters are needed either; those
exist in classic base64 purely to communicate trailing-byte padding, and there is no byte layer
here to need that.

### The leading `~`

Every token is prefixed with a literal `~`, so a short id in a URL looks like `~DyU`. Its only job
is to mark "everything after me decodes as one of these": URL-safe base64 never produces a `~`
itself, so the prefix can never be confused with the payload, and none of Toolshed's other
path-segment formats (bare usernames, `user@domain` handles, slugs, plain numeric ids) start with
`~` either. `~` is one of RFC 3986's `unreserved` characters (§2.3, the same class as letters,
digits, `-`, `.`, and `_`), a stronger guarantee than merely being legal in a path segment: it's
never a target for percent-encoding and never carries special meaning in any URI component, so a
short id can be handed to any part of the stack without first checking which encoding rules apply
there.


### Worked examples

Encoding `kind = item` (0), `owner_identity_id = 7`, `item_local_id = 42`:

- `kind`: 2 fixed bits → `00`
- `owner_identity_id = 7`: fits in one 4-bit chunk → 5 bits (`00111`)
- `item_local_id = 42`: needs two 4-bit chunks → 10 bits (`1001001010`)

Total: 17 meaningful bits, padded to the next multiple of 6 (18) with one zero bit, yielding 3
base64 characters: **`~DyU`**.

The same worked-out form for one example of every registered kind - `Bits` is the same
space-separated segmentation (kind tag, escape offset if present, each field, then padding) the
Examples table on `/~<token>` (`frontend/src/views/ShortId.vue`) shows for every registered kind:

| Kind | Fields | Serialized | Bits | Token |
|---|---|---|---|---|
| `item` | `owner_identity_id: 7`, `item_local_id: 42` | `[0, 7, 42]` | `00 00111 1001001010 0` | `~DyU` |
| `storage_location` | `owner_identity_id: 3`, `storage_location_id: 1000` | `[1, 3, 1000]` | `01 00011 100111111001000 00` | `~Rz8g` |
| `category` | `category_id: 5` | `[2, 5]` | `10 00101 00000` | `~ig` |
| `workflow` | `owner_identity_id: 2`, `workflow_id: 9` | `[3, 2, 9]` | `11 00000 00010 01001 0` | `~wCS` |
| `group` | `group_id: 11` | `[4, 11]` | `11 00001 01011` | `~wr` |
| `file` | `file_id: 123` | `[5, 123]` | `11 00010 1011101011 0` | `~xXW` |

`workflow`, `group`, and `file` are kinds 3-5, so their `Bits` column shows the escape tag (`11`)
followed by its own offset segment payload fields.