apotheca — Specification

version 1.0 RC3

This document specifies the apotheca protocol: the operations a cella exposes, the integrity guarantees those operations make, the constraints on names, the atomicity rules backends must honour, the on-disk layout of the local backend, and the CLI surface that exposes the protocol to the shell.

The specification is language-agnostic. The reference implementation is in Rust, published as the apotheca crate (https://gitlab.com/pantheca/apotheca/apotheca-rs); other implementations bind to the same protocol.

For motivation, philosophy, and the relationship to neighbouring projects, see README.md. For the reasoning behind specific decisions, see RATIONALE.md.

The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, and MAY in this document are to be interpreted as described in RFC 2119.

1 Scope and terminology

1.1 Cella

A cella is the unit of storage. It holds an unbounded set of named deposita, each depositum consisting of a name and the bytes stored under that name. A cella is composed of one or more backends; this document specifies the single-backend case in full and defers multi-backend composition to a future revision (see §10).

1.2 Depositum

A depositum is a (name, bytes) pair durably held by a cella. The term is geological: like sedimentary deposita, each depositum is laid down once and is thereafter neither modified nor removed. Once a deposit operation returns Ok, the bytes under that name are immutable for the lifetime of the cella. A depositum is present in a cella once its deposit operation has returned Ok and the cella has not since been corrupted or destroyed.

"Entry" is permissible as an informal English synonym in prose; normative text in this specification uses depositum exclusively.

1.3 Name

A name is a caller-chosen identifier under which bytes are stored. Name constraints are defined in §4.

1.4 Bytes

Bytes denote an arbitrary octet sequence of length in [0, 2^63). The zero-length sequence is a valid value.

1.5 Digest

A digest in this specification is a SHA-256 hash, represented as 32 octets. SHA-256 is defined by FIPS 180-4.

1.6 Pinax

A pinax is a (name, bytes) pair held by a cella whose bytes MAY be replaced by a subsequent successful set_pinax. A cella’s pinakes are kept in a namespace disjoint from its deposita: the same name MAY be used for both a pinax and a depositum without collision (see §4.3). The term is borrowed from Greek πίναξ (pinax), tablet or list-board; the plural is pinakes. Where a depositum is sediment laid down once and never disturbed, a pinax is a surface designed to be overwritten.

A pinax is present in a cella once any set_pinax for that name has returned Ok and the cella has not since been corrupted or destroyed.

Pinakes are intended for small, frequently-updated values such as state pointers and history heads. The spec imposes no maximum size; implementations SHOULD support pinakes of at least 1 KiB and MAY reject larger pinax bytes with an implementation-defined error.

2 Operations

A cella exposes six operations: deposit, get, stat, and deposit_cas on the depositum namespace; get_pinax and set_pinax on the pinax namespace. Each operation is total: it terminates with one of the outcomes listed for it.

2.1 deposit(name, bytes)

Stores bytes under name.

The implementation MAY compute sha256(bytes) directly, or MAY compare the candidate bytes with the stored bytes by any means that yields the same outcome.

2.2 get(name)

Returns the bytes stored under name. Implementations MUST verify the returned bytes against the stored digest before returning them to the caller.

IntegrityError indicates backend corruption (silent disk error, partial write surviving recovery, tampering). It is a real condition, not a theoretical one; callers SHOULD propagate it as an error rather than retry.

2.3 stat(name)

Returns metadata for name without reading the bytes.

stat MUST be implementable without transferring the bytes. On backends where bytes and metadata are co-located (e.g. local filesystem), implementations MAY return metadata derived from on-disk state without re-hashing on every call.

2.4 get_pinax(name)

Returns the bytes stored under name in the pinax namespace. Implementations MUST verify the returned bytes against the stored digest before returning them to the caller (see §3).

2.5 set_pinax(name, bytes, expected)

Stores bytes under name in the pinax namespace, conditionally on the current value matching expected. The argument expected is Option<Digest>:

Outcomes:

The actual digest reported with Conflict MUST be observed by the same atomic check that determined the precondition failed. Implementations SHOULD return Conflict to the caller rather than retry internally; the standard compare-and-swap pattern is for the caller to retry with actual as the new expected.

Backend accommodation: on backends whose native conditional-write primitive is keyed by a token other than the digest (e.g. ETag-conditioned object stores), the same-atomic-check requirement on actual is relaxed: actual MAY be derived from a separate read issued immediately after the failed conditional write. On such backends actual is advisory under continued concurrent modification — the retry pattern above re-verifies it — and a digest cycle between the check and the write (ABA) can yield a Conflict whose actual equals expected. Callers MUST treat any Conflict as a failed precondition regardless of the reported actual. The Ok/Conflict decision itself remains bound to the backend’s atomic conditional write in all cases.

The implementation MAY compute sha256(bytes) directly, or MAY compare with the stored bytes by any means that yields the same outcome.

2.6 deposit_cas(name, bytes)

Stores bytes under name under a caller-asserted content-addressed precondition: the caller asserts that if a depositum is already present under name, its bytes equal bytes. The caller is responsible for choosing name such that this precondition holds — typically by deriving name from bytes via a hash function — but the implementation makes no assumption about how name was chosen.

Outcomes:

The implementation MUST return Collision whenever the stored sha256 disagrees with sha256(bytes); collision detection is on the same normative footing as for deposit (§2.1). What deposit_cas MAY skip is the pre-emptive existence read that deposit performs before writing. A conforming implementation may issue a single put-if-not-exists and defer the digest comparison to the rare conflict branch (e.g. on the backend’s AlreadyExists response), saving one round-trip on fresh deposits where the backend reports successful create without a prior read. The trade-off is paid by the idempotent-re-deposit case, which costs one extra metadata read compared to deposit.

The hash function used to derive name is opaque to apotheca. The stored sha256 required by §3.1 is computed from bytes independently of name’s derivation; the two MAY use different hash functions. For example, syntheca derives name via BLAKE3 while apotheca stores sha256(bytes) as the depositum’s digest, yielding independent verification across two hash families on the syntheca-on-apotheca stack.

Under a collision-resistant naming function and honest content-addressed callers, Collision is unreachable from honest inputs: it signals either backend corruption or a successful adversarial collision against the caller’s hash family. Callers MUST treat Collision as a hard error, MUST NOT overwrite the existing depositum, and SHOULD log the event — the same response required for deposit (§2.1, §3.2).

Atomicity (§5) applies identically to deposit_cas.

3 Integrity

3.1 Mandatory hashing

Every present depositum MUST have a stored sha256 digest. There is no mode in which deposita are stored without an associated digest.

3.2 Verification

get MUST verify; deposit MUST detect collisions by digest equivalence; stat MUST report the stored digest. The digest is the only quantity that defines "same bytes" or "different bytes" for the purposes of this specification.

3.3 Integrity field

Every present depositum’s stored sha256 (§3.1) MUST be carried alongside the depositum’s bytes in a backend-appropriate integrity field. The field is the canonical carrier for get verification (§2.2) and stat reporting (§2.3); it exists in apotheca’s protocol independently of any backend-native checksum mechanism.

Each backend kind realizes the integrity field in its conventional metadata mechanism. On backends that use HTTP-style user metadata, the canonical key name is apotheca-checksum:

In every case, the integrity field’s value is the 64-character lowercase hexadecimal encoding of the depositum’s sha256.

Implementations MAY additionally carry the digest in a backend-native checksum field (e.g. x-amz-checksum-sha256 on S3-compatible backends, whose server-side validation provides extra protection against in-transit corruption). Such native carriage is defense-in-depth and does not affect conformance; conformance keys on the integrity field above.

Backend-native ETag-like fields (e.g. S3’s ETag header, which on multipart uploads is an MD5-of-MD5s rather than sha256(bytes)) MUST NOT be used to satisfy this specification’s integrity requirements.

3.4 Hash function

The hash function is SHA-256 and is fixed by this specification. Changing the hash function is a protocol revision.

3.5 Pinakes

The integrity guarantees of §3.1–§3.4 apply equally to pinakes: every present pinax MUST have a stored sha256 digest; get_pinax MUST verify; set_pinax MUST detect equivalence by digest; the stored sha256 is carried in the same integrity field as for deposita, per the per-backend mapping in §3.3. The local backend MAY derive the stored digest from the stored bytes on demand rather than persisting it in a separate file, provided every observable read returns a digest identical to what would be returned by a backend that stores it separately.

4 Name policy

4.1 Names

A name is a single non-empty filesystem-safe component:

Names are octet sequences, not Unicode strings. Implementations MUST NOT apply Unicode normalisation. Two names are equal iff their octet sequences are equal.

4.2 Future names

Future revisions MAY admit names containing /, with each /-separated segment subject to the §4.1 component constraints. Implementations SHOULD treat the namespace as opaque: foo/bar and foo have no hierarchical relationship as far as the protocol is concerned.

4.3 Pinakes and deposita

Pinakes and deposita are kept in disjoint namespaces within a cella. The same name MAY refer to a pinax and a depositum simultaneously; operations on the depositum namespace (§2.1–§2.3) are independent of operations on the pinax namespace (§2.4–§2.5). Name constraints (§4.1) apply identically to both namespaces.

5 Atomicity

The guarantees in this section hold across concurrent callers regardless of their locus: threads within one process, multiple handles onto the same cella root, and separate processes all observe the same atomicity and linearisation behaviour.

5.1 All-or-nothing visibility

A successful deposit operation MUST be all-or-nothing with respect to readers: a concurrent get or stat MUST observe either the full prior state of the name (including absence) or the full new depositum (bytes plus digest). No intermediate state in which bytes are present without their digest, or vice versa, is observable.

5.2 Crash safety

After a process crash, host crash, or power loss, the cella MUST be in a state consistent with §5.1: every name that is observable as present MUST have its bytes and its stored digest both intact and matching. Implementations MAY discard partially-written deposita during recovery.

5.3 Concurrent writers

Concurrent deposit calls for the same name MUST resolve such that exactly one of the following holds:

5.4 Pinax atomicity

A successful set_pinax MUST be all-or-nothing with respect to readers: a concurrent get_pinax MUST observe either the full prior value (including absence) or the full new value. No intermediate state in which bytes and digest disagree, or in which the pinax transiently appears absent during a replacement, is observable.

The precondition check against expected and the write are a single atomic step. Concurrent set_pinax calls for the same name MUST be linearised: each call observes the result of all calls preceding it. If two calls would store differing bytes under the same expected, at most one MAY observe Ok; the others MUST observe Conflict whose actual reflects either the prior state or the state set by the call that succeeded.

After a process crash, host crash, or power loss, every pinax observable as present MUST have its bytes and stored digest both intact and matching. Implementations MAY discard partially-written pinax staging files during recovery (§6.5).

6 Local backend on-disk layout

This section specifies the on-disk layout of the local-filesystem backend.

6.1 Cella root

The cella root is a directory. The default cella root is ~/.apotheca/. The root MAY be overridden by configuration; the layout below is relative to it.

6.2 Layout

<root>/
  deposita/
    <name>/
      bytes
      meta
  tmp/
    <staging-id>/
      bytes
      meta

6.3 Meta file format

The meta file is UTF-8 text, exactly:

size <decimal>
sha256 <hex>

with one trailing newline after the sha256 line. <decimal> is the unsigned decimal representation of the depositum’s size in octets. <hex> is the depositum’s sha256 as 64 lowercase hexadecimal digits.

Implementations MUST reject meta files that do not match this grammar exactly.

6.4 Atomic deposit procedure

A conforming local-backend deposit(name, bytes) proceeds as follows:

  1. Compute d = sha256(bytes).

  2. If deposita/<name>/ exists, parse deposita/<name>/meta. If its sha256 field equals d, return Ok. Otherwise return Collision.

  3. Choose a fresh <staging-id> (e.g. a random component) and create tmp/<staging-id>/.

  4. Write bytes to tmp/<staging-id>/bytes; fsync the file.

  5. Write the meta file to tmp/<staging-id>/meta; fsync the file.

  6. Fsync the staging directory.

  7. Atomically rename tmp/<staging-id>/ to deposita/<name>/ (rename(2)).

  8. Fsync the parent of deposita/<name>/.

Step 7 is the linearisation point: before it, no reader observes the new depositum; after it, every reader observes both bytes and meta together.

If any step fails, the staging directory MAY be removed; if it is left behind, recovery (§6.5) handles it.

6.5 Recovery

On startup, or on demand, an implementation MAY scan tmp/ and remove staging directories older than an implementation-defined threshold. tmp/ content is never authoritative; deposita become authoritative only when renamed into deposita/.

6.6 Get and stat procedures

get(name):

  1. If deposita/<name>/ does not exist, return NotFound.

  2. Parse deposita/<name>/meta to obtain (size, sha256_stored).

  3. Read deposita/<name>/bytes.

  4. If the read length does not equal size, return IntegrityError.

  5. Compute sha256(bytes_read). If it does not equal sha256_stored, return IntegrityError.

  6. Return the bytes.

stat(name):

  1. If deposita/<name>/ does not exist, return NotFound.

  2. Parse deposita/<name>/meta to obtain (size, sha256_stored).

  3. Return { size, sha256_stored }.

stat does not re-hash and does not read bytes.

6.7 Pinax layout

Pinakes are stored under <root>/pinakes/, parallel to and disjoint from <root>/deposita/. Their lockfiles are stored in a separate <root>/pinax-locks/ directory. deposita/ and the shared tmp/ staging area are as defined in §6.2:

<root>/
  deposita/                       # depositum namespace, write-once
  pinakes/                        # pinax namespace, compare-and-swap
    <name>                        # one regular file per pinax; content = bytes
  pinax-locks/                    # per-pinax lockfiles (created on demand)
    <name>
  tmp/                            # staging area, shared
    <staging-id>

Each pinax is stored as a single regular file at <root>/pinakes/<name> whose content is the pinax’s bytes. The local backend recomputes the stored digest from the file content on each read (§3.5). No meta file is written for pinakes.

Lockfiles MUST NOT be placed inside <root>/pinakes/ itself: every file in that directory is a pinax, and an auxiliary file placed there shadows the pinax namespace. (A lockfile at pinakes/<name>.lock is observable as a phantom pinax named <name>.lock, and a legitimate set_pinax for a name ending in .lock would rename over a live lockfile and break the §6.8 mutual exclusion.) The lockfile reuses the pinax’s own name unsuffixed, which also preserves the full §4.1 name length range on filesystems with a 255-octet filename limit.

Migration note (non-normative): cellae written against drafts before v1.0-rc3 may contain stale zero-length <name>.lock files inside pinakes/; under this revision those read as pinakes. They were lockfiles, not data, and MAY be removed manually.

This layout permits atomic file-over-file replacement via rename(2), which is portable and crash-safe; the per-name-directory layout used for deposita (§6.2) does not admit a portable atomic-replace primitive on POSIX.

6.8 Atomic set_pinax procedure

A conforming local-backend set_pinax(name, bytes, expected) proceeds as follows. The procedure MUST hold an exclusive advisory lock on <root>/pinax-locks/<name> (created on demand) for the duration of steps 2–7.

  1. Compute d = sha256(bytes).

  2. Acquire the exclusive lock.

  3. Determine the current state of <root>/pinakes/<name>:

    • if it does not exist, set actual = None;

    • else read the file content and set actual = Some(sha256(content)).

  4. If actual != expected, release the lock and return Conflict { actual }.

  5. If actual == Some(d) (idempotent re-set with identical bytes), release the lock and return Ok without writing.

  6. Otherwise, choose a fresh <staging-id>, write bytes to tmp/<staging-id>, fsync the file.

  7. Atomically rename tmp/<staging-id> to <root>/pinakes/<name> (rename(2) over the existing file is atomic on POSIX). Fsync <root>/pinakes/.

  8. Release the lock and return Ok.

Step 7 is the linearisation point. Readers (get_pinax) do not take the lock and observe whichever rename is current.

If any step after step 6 fails, the staging file is orphaned and recovery (§6.5) handles it.

6.9 get_pinax procedure

get_pinax(name):

  1. If <root>/pinakes/<name> does not exist, return NotFound.

  2. Read the file content as bytes_read.

  3. Compute sha256(bytes_read) and return bytes_read together with this digest as the verification.

The local backend, having derived the digest from the bytes just read, cannot observe an integrity mismatch within this procedure. The IntegrityError outcome of §2.4 is reachable only on backends that store the digest separately from the bytes (e.g. S3-compatible, where the digest is a header) and observe a mismatch between the stored and recomputed digests.

7 CLI surface

The reference CLI binary is apo. It exposes the protocol operations one-for-one, against the default local cella (§6.1) unless configured otherwise.

7.1 apo deposit

apo deposit [--name <name>] <path>
apo deposit  --name <name> -

7.2 apo get

apo get <name>

7.3 apo stat

apo stat <name>

7.4 apo pinax get

apo pinax get <name>

7.5 apo pinax set

apo pinax set --name <name> (--expect-absent | --expect <hex>) <path>
apo pinax set --name <name> (--expect-absent | --expect <hex>) -

8 Errors

The error conditions defined by this specification are:

Implementations MAY surface additional implementation-defined errors (I/O failure, permission denied, malformed meta file, exhausted disk space, …). Such errors MUST be distinguishable from the protocol-defined errors above; in particular, an I/O failure MUST NOT be reported as NotFound, Collision, or Conflict.

9 Conformance

9.1 Depositum-surface conformance

An implementation conformant to the depositum surface MUST implement:

Such an implementation MAY omit:

9.2 Pinax-surface conformance

An implementation conformant to the pinax surface MUST implement every requirement of the depositum surface (§9.1), plus:

A depositum-surface implementation MAY ship pinax-surface features additively without claiming pinax-surface conformance, provided its depositum-surface conformance remains intact.

10 Future work (non-normative)

S3-compatible backends occupy a middle ground: they are implementable today — §3.3 pins their integrity field, §2’s outcomes and §5’s atomicity apply in full, and conditional writes (put-if-absent, put-if-match) supply the CAS primitive — and the reference implementation ships one. What this revision does not yet specify is their on-store layout (object key scheme, staging), so two independent implementations sharing one bucket are not guaranteed to interoperate, and names that are not valid UTF-8 (legal per §4.1) may be rejected by such backends with an implementation-defined error. A future revision will pin the S3 object layout; until then S3 key layout is implementation-defined. See also the backend accommodation on actual in §2.5.

The following are explicitly out of scope for v1.0-rc3 and will be specified in future revisions:

These are listed here so that implementations and consumers know not to rely on their absence.