Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ jobs:
- name: File-backend format verify (stdlib only)
run: python3 tools/file-backend-reference.py

- name: Cache-key path-encoding verify (stdlib only; mutation self-test first)
run: python3 tools/path-encoding-verify.py

- name: Python-frame JS cross-check (zero-dep independent reader, full round-trip)
run: node tools/frame-crosscheck.mjs

Expand Down
38 changes: 38 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,44 @@ All notable changes to the CacheKit Protocol Specification.

## [Unreleased]

### SaaS API — cache-key path encoding specified (LAB-2879)

- [`spec/saas-api.md`](spec/saas-api.md) gains a normative **Cache-Key Path
Encoding** section. The spec documented `/v1/cache/{key}` and its `/ttl` and
`/lock` sub-resources without saying how `{key}` is placed in the path — a
silence that cost three SDK tickets (cachekit-py shipped the raw key
unquoted, CWE-22, until
[#279](https://github.com/cachekit-io/cachekit-py/pull/279); cachekit-ts
LAB-2877 and cachekit-rs LAB-2878 carry the same latent `.`/`..` gap). Rules: the key is ONE percent-encoded segment with
only RFC 3986 unreserved characters raw (`! * ' ( )` tolerated); the server
URL-parses under the WHATWG URL Standard, splits on raw `/`, then decodes
exactly once and validates the decoded key; encoders may differ on the
sub-delims because interop is defined on the **decoded** key, and every
server-accepted key is byte-identical on the wire regardless.
- **Reserved segments must be rejected client-side; percent-encoding cannot
save a `.` or `..` key.** The filing premise — encode `.`/`..` as `%2E`/`%2E%2E`
— is false: the server parses the request URL under the WHATWG URL Standard,
which treats `%2e`, `%2e%2e`, `.%2e`, `%2e.` (any case) as dot segments.
Verified live: `GET api.cachekit.io/v1/cache/%2E%2E/health` returns the
`/v1/health` response, `/v1/cache/%2E%2E/ttl` routes as `/v1/ttl`. httpx
0.28.1 sends `%2E%2E` intact (RFC 3986 §5.2.4 removes only literal dots), so
cachekit-py's f000ba3 rewrite moves the collapse from client to server rather
than preventing it; Node 25 `new URL()` and rust-url 2.5.8 collapse it before
sending. Clients MUST reject a key whose encoded form is exactly `.`, `..`,
or one of the route tokens `health`, `ttl`, `lock`. Found by execution during
the expert-panel round on this change (bug-hunter and security agents
independently probed the live server).
- New [`test-vectors/path-encoding.json`](test-vectors/path-encoding.json)
(15 rows: 10 transmittable with `key → encoded → decoded`, 5 `reject: true`
reserved segments with no wire form, one `encoded_alternates` row for the
`encodeURIComponent` form), CI-verified by
[`tools/path-encoding-verify.py`](tools/path-encoding-verify.py) (stdlib;
9-mutation self-test first, each mutation tripping a distinct guard).
- [`sdk-feature-matrix.md`](sdk-feature-matrix.md) Compliance Status gains a
path-encoding row with actual state: all three SDKs percent-encode one
segment but none yet rejects the reserved segments — Python's v0.18.0 `%2E`
rewrite is insufficient (follow-up filed); Rust and TypeScript in progress.

### Wire format — compressed-byte reproducibility scoped per-vector (LAB-1751)

- LZ4 compressed bytes are **not canonical** across conforming block encoders.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ layer's own store/retrieve flows are specified in
| [spec/cache-key-format.md](spec/cache-key-format.md) | Cache key generation algorithm — Blake2b-256, argument normalization, cross-SDK key strategy |
| [spec/wire-format.md](spec/wire-format.md) | ByteStorage envelope — LZ4 block compression, xxHash3-64 integrity, decompression bomb protection |
| [spec/encryption.md](spec/encryption.md) | AES-256-GCM encryption, HKDF-SHA256 key derivation, AAD v0x03, counter-based nonces, key rotation |
| [spec/saas-api.md](spec/saas-api.md) | REST API endpoints, binary wire protocol, error codes, metrics headers |
| [spec/saas-api.md](spec/saas-api.md) | REST API endpoints, cache-key path encoding, binary wire protocol, error codes, metrics headers |
| [spec/interop-mode.md](spec/interop-mode.md) | Cross-SDK cache sharing — language-neutral key format, canonical argument normalization *(normative; shipped opt-in in all three SDKs — see the [feature matrix](sdk-feature-matrix.md#compliance-status) for per-SDK version floors)* |
| [spec/interop-v2.md](spec/interop-v2.md) | Interop v2 compressed-values profile — opt-in LZ4-block + AES-256-GCM cross-SDK values *(DRAFT; no SDK implements it yet)* |
| [spec/file-backend-format.md](spec/file-backend-format.md) | Shared local File backend filename, header, expiry, and fail-closed flag negotiation |
Expand Down Expand Up @@ -125,7 +125,7 @@ An SDK is protocol-compliant when:
1. [Interop-mode](spec/interop-mode.md) key generation produces identical keys for identical inputs across all languages (auto-mode keys embed language-specific function identity and are not cross-SDK by design)
2. Interop-mode values encode and decode per the canonical vectors; where the SDK implements the ByteStorage envelope, it deserializes any spec-conformant envelope
3. Encrypted interop-mode payloads can be decrypted by any SDK with the same master key and tenant ID
4. SaaS API integration follows the documented endpoint contracts
4. SaaS API integration follows the documented endpoint contracts, including [cache-key path encoding](spec/saas-api.md#cache-key-path-encoding) (`test-vectors/path-encoding.json`)

Test vectors are published in [`test-vectors/`](test-vectors/) as JSON files.

Expand Down
3 changes: 2 additions & 1 deletion sdk-feature-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,7 @@ its spec:
| Encryption (AES-256-GCM) | ✅ Compliant | ✅ Canonical (cachekit-core) | ✅ Compliant | ⚠️ Untested |
| AAD v0x03 | ✅ Compliant (5 components — every auto serializer appends `original_type`; interop mode is the sole 4-component path) | ✅ Compliant (4 components) | ✅ Compliant (4 components) | ❌ Not implemented |
| SaaS API | ✅ Compliant | ✅ Compliant (CachekitIO backend) | ✅ Compliant | ❌ Not implemented |
| SaaS API — cache-key path encoding ([spec](spec/saas-api.md#cache-key-path-encoding)) | ⚠️ Partial — percent-encodes one segment (`quote(safe="")`, rules 1/3/4) since `f000ba3` ([#279](https://github.com/cachekit-io/cachekit-py/pull/279), v0.18.0). Its `%2E` all-dot rewrite is collapsed by the server's WHATWG parse, so `.`/`..` still route-escape (rule 2); reserved-segment rejection is LAB-2880 | ⚠️ Partial — percent-encodes one segment (`urlencoding::encode`, rules 1/3/4) on `main` @ `42697bd`; no reserved-segment rejection (rule 2); LAB-2878 in progress | ⚠️ Partial — percent-encodes one segment (`encodeURIComponent`, rules 1/3/4) on `main` @ `4261f16`; no reserved-segment rejection (rule 2); LAB-2877 in progress | ❌ Not implemented |
| Test vectors in CI¹⁶ | ✅ interop/v1 (full set, incl. AAD + encryption through the real stack) | ✅ interop/v1 (full set) since [#33](https://github.com/cachekit-io/cachekit-rs/pull/33) | ✅ interop/v1 (full set, incl. its key vectors) + inline Python-generated AAD-construction and encryption (decrypt-Python-ciphertext) vectors | ⚠️ Pending |
| Interop mode ([spec](spec/interop-mode.md), opt-in) | ✅ Released — PyPI 0.14.0+¹⁷ ([#220](https://github.com/cachekit-io/cachekit-py/pull/220)) | ✅ Released — crates.io 0.4.0+ ([#33](https://github.com/cachekit-io/cachekit-rs/pull/33)) | ✅ Released — npm 0.1.3+ ([#71](https://github.com/cachekit-io/cachekit-ts/pull/71)) | ❌ Not implemented |

Expand All @@ -293,7 +294,7 @@ its spec:
>
> ¹⁵ Auto-mode **stored bytes** are SDK-internal and differ per SDK — see [wire-format.md → SDK Storage Containers](spec/wire-format.md#sdk-storage-containers-auto-mode). Python stores the ByteStorage envelope *inside* its CK v3 frame; `cachekit-rs` does not use the envelope for values at all (it uses `cachekit-core` only for encryption). Cross-SDK value compatibility is exclusively an [interop-mode](spec/interop-mode.md) property (protocol#11).
>
> ¹⁶ "Test vectors in CI" = vectors the SDK's own default CI executes. Beyond the SDKs, this repo's `verify.yml` CI-verifies `interop-mode.json`, `encryption.json`, `python-frame.json`, `file-backend.json` ([`tools/file-backend-reference.py`](tools/file-backend-reference.py)), and — since LAB-423 — `wire-format.json` ([`tools/wire-format-reference.py`](tools/wire-format-reference.py)) against reference implementations. `cache-keys.json` (regenerated by cachekit-py v0.12.0, byte-identical to the v0.5.0 originals) is vendored and CI-verified in cachekit-py since [cachekit-py#229](https://github.com/cachekit-io/cachekit-py/pull/229) (LAB-425).
> ¹⁶ "Test vectors in CI" = vectors the SDK's own default CI executes. Beyond the SDKs, this repo's `verify.yml` CI-verifies `interop-mode.json`, `encryption.json`, `python-frame.json`, `file-backend.json` ([`tools/file-backend-reference.py`](tools/file-backend-reference.py)), `wire-format.json` ([`tools/wire-format-reference.py`](tools/wire-format-reference.py); since LAB-423), and `path-encoding.json` ([`tools/path-encoding-verify.py`](tools/path-encoding-verify.py); since LAB-2879) against reference implementations. `cache-keys.json` (regenerated by cachekit-py v0.12.0, byte-identical to the v0.5.0 originals) is vendored and CI-verified in cachekit-py since [cachekit-py#229](https://github.com/cachekit-io/cachekit-py/pull/229) (LAB-425).
>
> ¹⁷ Version cells are **floors** (`X+`), not snapshots — they stay true as new versions publish; check the registry for the current release. Python's floor is the first *installable* one: interop merged under the `v0.13.0` tag, but neither `0.12.0` nor `0.13.0` was ever published to PyPI, so `0.14.0` is the earliest PyPI release containing interop mode. Do not "correct" this to 0.13.0 from the cachekit-py changelog alone.

Expand Down
28 changes: 28 additions & 0 deletions spec/saas-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
- [Overview](#overview)
- [Authentication](#authentication)
- [Content Type](#content-type)
- [Cache-Key Path Encoding](#cache-key-path-encoding)
- [Cache Endpoints](#cache-endpoints)
- [Stale-While-Revalidate](#stale-while-revalidate)
- [Lock Endpoints](#lock-endpoints)
Expand Down Expand Up @@ -66,6 +67,33 @@ Content-Type: application/octet-stream

---

## Cache-Key Path Encoding

Every endpoint below carries the cache key as a path segment — `/v1/cache/{key}`, `/v1/cache/{key}/ttl`, `/v1/cache/{key}/lock`. The key is caller-controlled (each SDK's `key=` escape hatch accepts an arbitrary string), so how it is placed in the path is a security boundary, not a formatting detail: an unencoded key can escape `/v1/cache/` and deliver the bearer token to a different route (CWE-22 — cachekit-py shipped exactly that until [cachekit-py#279](https://github.com/cachekit-io/cachekit-py/pull/279)). MUST, MUST NOT, SHOULD and MAY are used as in RFC 2119.

### Encoding rules

**1. One segment, percent-encoded.** `{key}` MUST be exactly one path segment. Clients MUST percent-encode the key's UTF-8 bytes (RFC 3986 §2.1) so that only unreserved characters — `ALPHA / DIGIT / "-" / "." / "_" / "~"` — appear raw. Every other byte MUST be sent as `%HH` — the delimiters `/ ? # %`, `:` (a canonical key carries six), space (`%20`, never `+`), every byte ≥ `0x80` — with one tolerance: the sub-delims `! * ' ( )` MAY be left raw (rule 4). Hex digits SHOULD be uppercase (RFC 3986 §2.1); the server decodes either case. Reference encoders: Python `urllib.parse.quote(key, safe="")`, Rust `urlencoding::encode`, JavaScript `encodeURIComponent`.

**2. Reserved segments MUST be rejected client-side.** A key of exactly `.` or `..` survives rule 1 unchanged (`.` is unreserved) and is a *dot segment*: URL parsers remove it before routing — `/v1/cache/..` becomes `/v1/`, `/v1/cache/../ttl` becomes `/v1/ttl` — so the request lands on a different route, still carrying `Authorization`, and never reaches the key validator. Percent-encoding the dots does not help. The server parses the request URL under the WHATWG URL Standard, which treats an ASCII-case-insensitive `%2e` as a single-dot segment and `%2e%2e`, `.%2e`, `%2e.` as double-dot segments (URL Standard §4.1), so `%2E%2E` is collapsed *server-side* even when the client's own parser (RFC 3986 §5.2.4, e.g. `httpx`) sent it intact; WHATWG clients (`fetch`/undici, browsers, the Workers runtime, rust-url and therefore `reqwest`) collapse it before sending. **No wire form of a `.` or `..` key reaches the validator from any client.** The literal segments `health`, `ttl` and `lock` are route tokens at this level — `/v1/cache/health` is the health endpoint, and a final `ttl` or `lock` segment selects the sub-resource — so a key encoding to one of those words is routed elsewhere or read as an empty key.

Therefore clients MUST reject a key whose encoded form is exactly `.`, `..`, `health`, `ttl` or `lock` before building the URL, surfacing a client-side error; servers MUST NOT be relied on to compensate. Only an *entirely*-dot segment is a dot segment: `a:..`, `..a`, `x..y` are inert and MUST be sent per rule 1 with their dots raw. Canonical and interop keys always contain `:` and never meet this rule. Conformance tests MUST assert on the *parsed* request path (`httpx.Request.url.raw_path`, `new URL(u).pathname`, `Url::parse(u)?.path()`), not on the un-parsed template string — a template-string test passes while the traversal ships.

> **Evidence (2026-09-04):** against `api.cachekit.io`, `GET /v1/cache/%2E%2E/health` returns the `/v1/health` response and `/v1/cache/%2E%2E/ttl` is routed as `/v1/ttl`, while `/v1/cache/a%3A..%2Fb/ttl` reaches the cache route. `httpx` 0.28.1 sends `%2E%2E` unchanged; Node 25 `new URL()` and rust-url 2.5.8 collapse it client-side. A fix proven on one parser is not proof for the other, and cachekit-py's `%2E` rewrite ([cachekit-py#279](https://github.com/cachekit-io/cachekit-py/pull/279), v0.18.0) moves the collapse from client to server rather than preventing it (LAB-2880).

**3. The server decodes exactly once.** After the WHATWG parse of rule 2, the server splits the path on raw `/`, then percent-decodes the key segment once (`decodeURIComponent`-equivalent; a malformed escape is `400`) and validates the *decoded* key: non-empty, within an implementation-defined maximum length (the deployed cap exceeds the 250-character SDK key limit in [cache-key-format.md](cache-key-format.md#key-length-limits)), drawn from `[A-Za-z0-9_.:-]`, free of the substring `..`, and — for `ns:` / `nsapi:` keys — of the shape `{prefix}:{namespace}:{rest}` with a 1–64-character namespace drawn from `[A-Za-z0-9_-]` and a non-empty `{rest}`. Anything else is `400 Bad Request`. Consequences clients MUST honour:

- Clients MUST NOT double-encode. A literal `%` in a key is sent as `%25` once; `%2525` decodes to `%25`, a different key.
- An encoded `%2F` never becomes a segment boundary: the split on raw `/` happens *before* decoding, so `a%2Fb` reaches the validator as `a/b` and is rejected by the charset rule. A conformant client can neither traverse nor store a key containing `/`.

**4. Interop is defined on the decoded key.** `encodeURIComponent` leaves the sub-delims `! * ' ( )` raw (legal `pchar` in a path segment; they decode to themselves); `quote(safe="")` and `urlencoding::encode` emit `%21 %2A %27 %28 %29`. Both forms are conformant because the server-side key is identical after the single decode. Cross-SDK key equality is therefore a property of the **decoded** key, not of the wire bytes in general — but every key the server accepts is drawn from `[A-Za-z0-9_.:-]`, on which all three reference encoders agree (`:` → `%3A`, the rest raw). Every canonical auto-mode key and every [interop-mode](interop-mode.md) key is thus byte-identical on the wire across SDKs; the variance set only ever appears in keys the server rejects.

### Test vectors

[`test-vectors/path-encoding.json`](../test-vectors/path-encoding.json) pins `key → encoded → decoded` in the reference form (`quote(safe="")`). Rows with `reject: true` are the reserved segments of rule 2 and carry no wire form; `encoded_alternates` lists the `encodeURIComponent` form where it differs (rule 4). Verified in this repo's CI by [`tools/path-encoding-verify.py`](../tools/path-encoding-verify.py).

---

## Cache Endpoints

All cache endpoints are prefixed with `/v1/cache/`.
Expand Down
Loading
Loading