zetetes — Specification

version 0.1

This document specifies the zetetes protocol: a hybrid-query coordinator over a metatheca vault that has been extended with semopsis, logopsis, or both. zetetes pins a (semopsis-state, logopsis-state, fusion-config) triple as a content-addressed blob, advances a small head pinax over the chain of such triples, and answers hybrid queries by issuing a query to each available extension at the pinned states and fusing the results.

The specification is language-agnostic. The reference implementation is in Rust (zetetes-rs/).

zetetes is structurally different from the layers it sits above. metatheca, semopsis, and logopsis each define a substantive content contract: metatheca defines the vault as a chain of versioned semantic states; semopsis defines a semantic view over that vault; logopsis defines a lexical view. zetetes defines no new content. It is a seeker (whence the name, ζητητής, "one who inquires") that consults the views built by semopsis and logopsis on the user’s behalf and returns a fused ranking. Its durable state is minimal: which views to consult, at which states, with which fusion config.

Like the extensions it consults, zetetes occupies its own dedicated cella under the metatheca vault root, at the fixed path <vault>/ext/zetetes/cella/, separate from metatheca’s source-of-truth cella and from each extension’s cella. The cella holds zetetes-state blobs and the zetetes-head pinax; it is wholly derivable (a fresh zet init followed by zet advance invocations restores any reachable configuration) and MAY be discarded at any time (§5.6).

For motivation, design choices, and the relationship to the other layers, see README.md. For the substrates and extensions this document builds on, see metatheca SPEC.md, ../../../syntheca/SPEC.md, ../../../apotheca/SPEC.md, ../semopsis/SPEC.md, and ../logopsis/SPEC.md.

The key words MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY in this document are to be interpreted as described in RFC 2119.

1 Scope and terminology

This document specifies zetetes Phase 1: a single coordinator over at most one semopsis chain and at most one logopsis chain on a single metatheca vault, with reciprocal rank fusion as the only specified fusion algorithm, and explicit-advance semantics (no auto-snap to extension heads at query time). Phase 1 is the v0.1 release surface; deferred items are listed in §10–§11.

Terminology not redefined here is inherited from metatheca SPEC §1, syntheca SPEC §1, apotheca SPEC §1, semopsis SPEC §1, and logopsis SPEC §1.

1.1 Source

A source is one of the two extensions zetetes can consult: the literal string "semopsis" or the literal string "logopsis". A zetetes-state references zero, one, or both sources.

1.2 Fusion config

A fusion config is the parameter set that determines how zetetes combines per-source ranked result lists into a single ranked output. Phase 1 specifies one fusion algorithm:

The RRF score for a document d under config (k, weights) over the set of pinned sources S is:

score(d) = Σ_{s ∈ S, d ∈ results(s)} weights[s] / (k + rank_s(d))

where rank_s(d) is the 1-based rank of d in source s’s result list and the sum is taken only over sources in which d appears.

Phase 1 zetetes MUST implement RRF and MUST NOT implement any other fusion algorithm. The algorithm name is recorded in the zetetes-state blob (§4.2) so that a future Phase 2 zetetes implementation can interoperate with Phase 1 chains by recognizing that fusion algorithms other than "rrf" are out of its support.

1.3 Zetetes-state

A zetetes-state is a moment in the coordinator’s history. Each zetetes-state is materialized as a content-addressed zetetes-state blob (§4.2) capturing the pinned semopsis search-state hash (or null), the pinned logopsis search-state hash (or null), the fusion config, the metatheca state hash that was current at advance time, and a back-pointer to the prior zetetes-state. The set of zetetes-states linked through these back-pointers is the zetetes-state chain.

The genesis zetetes-state is the unique zetetes-state with previous = null, created by zet init (§5.1).

A zetetes-state with both semopsis = null and logopsis = null is permitted but useless; zet query against such a state returns an empty result set with no error. This case exists so that zet init can be invoked before either extension has been initialized, with zet advance filling in the pins later.

1.4 Zetetes-head

The zetetes-head is the single mutable pointer identifying the current zetetes-state. It is realized as an apotheca pinax (§6) on the zetetes cella, holding the BLAKE3 hash of the current zetetes-state blob.

The zetetes-head pinax is disjoint from metatheca’s head, semopsis’s head, and logopsis’s head: each pinax lives on its own cella (<vault>/cella/, <vault>/ext/semopsis/cella/, <vault>/ext/logopsis/cella/, and <vault>/ext/zetetes/cella/ respectively) and the four advance independently.

1.5 Zetetes-state reference

A zetetes-state reference (or zetetes-stateref) follows the same grammar as metatheca’s stateref (metatheca SPEC §1.7), resolving against the zetetes-state chain:

A zetetes-stateref MUST resolve to exactly one zetetes-state or fail.

2 Coexistence with metatheca, semopsis, and logopsis

zetetes operates on the same vault root as the layers it consults but on its own dedicated cella at <vault>/ext/zetetes/cella/, separate from metatheca’s source-of-truth cella at <vault>/cella/ (metatheca SPEC §7) and from each extension’s own cella (<vault>/ext/semopsis/cella/, <vault>/ext/logopsis/cella/). All zetetes durable state lives as deposita and pinakes on its own cella; it introduces no new storage primitive.

zetetes MUST NOT write to metatheca’s cella or to any extension cella. Its writes are confined to:

zetetes does read from the other cellae: from metatheca’s cella to record metatheca state hashes for audit (§4.2), and from each pinned extension’s cella to issue per-source queries (§5.3).

A vault MAY contain a zetetes chain with at most one of semopsis / logopsis pinned (or neither, transitionally). A zetetes chain MUST NOT exist without a metatheca chain on the same vault; zet init (§5.1) requires metatheca’s head pinax to be present at <vault>/cella/.

A zetetes-state’s pinned semopsis hash MUST name a search-state present on the semopsis cella at advance time, and likewise for logopsis. zetetes does not verify ongoing reachability after advance — if a future operation removes a referenced search-state (deletion is currently out of scope across the stack, §10) or the extension’s cella is destroyed (semopsis SPEC §5.8 / logopsis SPEC §5.8), the zetetes-state remains valid as a record but zet query against it fails with NotFound for the missing source.

3 Identifiers and time

3.1 Hashes

Content-addressing throughout zetetes uses syntheca’s BLAKE3 hash. Wire form: 64 lowercase hex digits.

3.2 Time

All timestamps follow metatheca SPEC §2.3: nanoseconds since the Unix epoch as a signed 64-bit integer, monotonically non-decreasing within a single vault process.

4 Wire formats

4.1 Canonical JSON

The single JSON blob defined by this specification (zetetes-state §4.2) is serialized as canonical JSON per RFC 8785, identical to metatheca SPEC §3.1.

4.2 Zetetes-state blob

A zetetes-state blob is a canonical-JSON object:

{
  "type":          "zetetes/state",
  "version":       1,
  "previous":      "<blake3-hex>" | null,
  "metatheca":     "<blake3-hex>",
  "semopsis":      "<blake3-hex>" | null,
  "logopsis":      "<blake3-hex>" | null,
  "fusion": {
    "algorithm":   "rrf",
    "k":           <number>,
    "weights":     { "semopsis": <number>, "logopsis": <number> }
  },
  "created_at_ns": <integer>
}

Field constraints:

A zetetes-state’s identity is the BLAKE3 hash of its canonical-JSON bytes.

The triple (semopsis, logopsis, fusion) is recorded together because the fusion config is meaningful only relative to the set of pinned sources. Recording them in a single blob ensures they cannot be observed in inconsistent combinations.

5 Operations

A zetetes-equipped vault exposes the following protocol operations. Each is total: it terminates with one of the outcomes listed for it.

5.1 init

Creates a genesis zetetes-state on a vault that already has a metatheca chain.

  1. Verify metatheca’s head pinax is present on metatheca’s cella at <vault>/cella/ (§2); fail with NotFound otherwise.

  2. Verify no zetetes cella exists at <vault>/ext/zetetes/cella/; fail with Conflict otherwise.

  3. Open a fresh syntheca cella at <vault>/ext/zetetes/cella/, creating any missing parent directories. All subsequent deposits and the zetetes-head pinax target this cella.

  4. Resolve the caller-supplied initial pins: a. If a semopsis ref was supplied, resolve it per semopsis SPEC §5.6 against the semopsis cella to a search-state hash; otherwise pin null. b. If a logopsis ref was supplied, resolve it per logopsis SPEC §5.6 against the logopsis cella to a search-state hash; otherwise pin null.

  5. Construct the genesis zetetes-state blob: previous = null, metatheca = <current metatheca head>, semopsis = <as resolved>, logopsis = <as resolved>, fusion = <as supplied or default> (default per §1.2), created_at_ns = now(). The fusion weights map MUST be constructed to match the set of non-null pins per §4.2. Deposit the blob on the zetetes cella; let g be the returned hash.

  6. Set the zetetes-head pinax (§6) on the zetetes cella to g with expected = None.

A genesis zetetes-state MAY have both pins null. This permits initializing zetetes before either extension exists; the first zet advance populates one or both pins.

On any failure prior to step 6, the implementation MUST leave no partially-initialized zetetes cella. If the cella was created in step 3, it MUST be removed before returning the failure.

5.2 advance

Atomically advances zetetes-head from the current zetetes-state to a new zetetes-state with updated pins, updated fusion config, or both.

Inputs (all optional):

When an input is omitted, the corresponding field is inherited unchanged from the current zetetes-state. At least one of --semopsis, --logopsis, or --fusion MUST be supplied; advance with no inputs is Malformed.

  1. Read the current zetetes-state blob via the zetetes-head pinax (§6); compute the storage digest d of the head pinax bytes: sha256 over the bytes returned by syntheca.get_pinax, or remembered from the prior successful set_pinax (apotheca SPEC §2.5/§3.5 expected semantics).

  2. For each supplied pin input, resolve it:

    • current resolves to the corresponding extension’s current search-head (semopsis SPEC §6 / logopsis SPEC §6).

    • none resolves to null.

    • Any other value is treated as a search-stateref and resolved per the corresponding extension’s resolve operation (semopsis SPEC §5.6 / logopsis SPEC §5.6). Inputs that name a search-state not present in the cella fail with NotFound.

  3. Compute the new pin set: each field is the supplied value if the input was provided, else the current zetetes-state’s value for that field.

  4. Compute the new fusion config. If --fusion was supplied, use it; otherwise inherit from the current zetetes-state. The resulting weights map MUST match the set of non-null pins per §4.2: if a pin moves from null to non-null without a fusion override, the implementation MUST insert that source’s weight with the default value 1.0; if a pin moves from non-null to null, the implementation MUST remove that source’s weight.

  5. Construct the new zetetes-state blob: previous = <current zetetes-head>, metatheca = <current metatheca head>, semopsis = <as computed>, logopsis = <as computed>, fusion = <as computed>, created_at_ns = now(). Deposit it; let s be the returned hash.

  6. Call syntheca.set_pinax(name = "head", bytes = s_bytes, expected = Some(d)) against the zetetes cella.

The commit point is step 6. Steps 1–5 are pre-commit and produce only content-addressed deposita that are safe under interruption.

On Conflict from step 6, another writer advanced zetetes-head concurrently. The caller MAY retry from step 1.

5.3 query

Given a query string and a zetetes-stateref, return the top k documents by fused score.

  1. Resolve the zetetes-stateref per §1.5 to a zetetes-state hash z. Fetch and parse the zetetes-state blob; extract its pinned semopsis and logopsis search-state hashes and its fusion config.

  2. Determine the active source set S: the set of sources whose pin field is non-null.

  3. If S is empty, return an empty result set with no error.

  4. For each source s ∈ S, issue a query against s’s extension at its pinned search-state. The query string is passed unchanged to each extension; the extensions are responsible for their own analyzer (logopsis) or embedding (semopsis) processing per their respective specs. Implementations SHOULD issue the per-source queries in parallel.

    The two extensions interpret the query string under divergent semantics: logopsis parses it as a structured boolean expression with phrase delimiters (logopsis SPEC §5.5.1), while semopsis treats it as opaque bytes to be embedded under its model (semopsis SPEC §5.5). A query like foo AND bar therefore drives a logical AND in logopsis and an embedding of the literal string foo AND bar in semopsis. This asymmetry is intrinsic to passing the same input through two retrieval modalities; per-source query rewriting is deferred to a future phase (§10). Callers issuing structured queries should expect the semopsis side to contribute results based on embedding similarity to the literal query string.

    • For each source, request a result set of size at least k. Implementations SHOULD request max(k, k_internal) where k_internal is implementation-defined and typically larger than k (e.g. 4 * k or 100) to give RRF a wider candidate pool. Larger candidate pools improve fusion quality at modest cost.

  5. Compute fused scores per §1.2 using the zetetes-state’s fusion config. Per-source hits are joined across sources by entry: documents present in only one source’s result list contribute their single weighted reciprocal-rank term; documents present in both contribute the sum.

  6. Return the top k documents by descending fused score, each as (entry, blob, fused_score). Equal fused scores MUST break by ascending entry (bytewise UUID order), matching the per-source tie rules. The returned blob MUST be the one reported by the source with the larger weighted reciprocal-rank contribution weights[s] / (k + rank_s(d)); when the contributions are equal, the logopsis blob. (The two sources can report the same entry with different blobs, since the two pins may index different metatheca states.) Per-source ranks and scores MAY be exposed via an implementation-defined verbose mode but are not part of the protocol’s required output.

query is read-only: it MUST NOT modify any depositum or pinax.

If a per-source query fails with a protocol error (e.g. NotFound because the pinned search-state is no longer present on the cella, or AnalyzerMismatch / ModelMismatch), zet query MUST surface that error rather than silently dropping the source. Hybrid queries with a partially failed source set are not specified to fall back to the surviving source; this avoids producing results that disagree with the audit trail.

5.4 Resolve a zetetes-state reference

Given a zetetes-stateref (§1.5) and the current zetetes-head, return the unique zetetes-state hash it denotes. Algorithmically identical to metatheca SPEC §5.3 with the zetetes-state chain substituted.

5.5 Walk the zetetes-state chain

Algorithmically identical to metatheca SPEC §5.4 with the zetetes-state chain substituted.

5.6 Destroy

Removes the zetetes cella and all zetetes state from the vault.

Inputs: none.

  1. If <vault>/ext/zetetes/cella/ is absent, fail with NotFound.

  2. Recursively remove the directory <vault>/ext/zetetes/ (including the cella and any extension-private auxiliary state under it).

Destroy is total: on completion, no zetetes state remains on the vault. metatheca state and any extension state are unaffected. A subsequent zet init (§5.1) succeeds if and only if destroy completed.

Because zetetes-states are small and cheap to recreate (a fresh zet init followed by zet advance invocations restores any desired pin configuration), destroy is the supported recovery mechanism for a corrupted zetetes cella. The audit trail of prior zetetes-states is lost; this is the protocol’s stance that the audit trail is a convenience, not a load-bearing artifact.

Destroy MUST NOT touch <vault>/cella/, <vault>/index.db, or any other entry in <vault>/ext/.

6 Zetetes-head pinax

6.1 Name

The zetetes-head pinax MUST be stored on the zetetes cella under the apotheca pinax name head (four ASCII octets: h, e, a, d). This name is fixed by this specification.

The name is a valid apotheca name (apotheca SPEC §4.1): non-empty, length 4, no /, no NUL. zetetes therefore imposes no multi-segment-name requirement on the underlying apotheca implementation.

The pinax shares only its name with metatheca’s head (metatheca SPEC §6.1), semopsis’s head (semopsis SPEC §6.1), and logopsis’s head (logopsis SPEC §6.1); the four pinakes live in disjoint cellae and never collide.

6.2 Bytes

The zetetes-head pinax bytes follow metatheca SPEC §6.2 verbatim: the 64-octet ASCII encoding of the current zetetes-state’s BLAKE3 hash as 64 lowercase hex digits, with no trailing newline or whitespace.

6.3 Compare-and-swap protocol

All updates to zetetes-head MUST go through syntheca.set_pinax with expected set to the SHA-256 digest of the current zetetes-head bytes, identical in mechanism to metatheca SPEC §6.3. The genesis case (init, §5.1) sets zetetes-head with expected = None.

7 CLI surface

The reference CLI binary is zetetes with the recommended alias zet. It exposes the protocol against the vault rooted at the current working directory unless overridden with --vault <PATH>.

This section specifies the protocol-bearing subcommands. Diagnostic output formatting is implementation-defined unless otherwise noted.

7.1 zet init

zet init [--semopsis <ref>] [--logopsis <ref>]
        [--k <number>] [--weight semopsis=<number>]
                       [--weight logopsis=<number>]
        [<path>]

Initialize a zetetes chain on a vault per §5.1. The vault root is <path> if supplied, otherwise the current directory. The vault MUST already contain a metatheca chain. Exits non-zero if metatheca is absent or a zetetes chain already exists.

--semopsis and --logopsis accept a ref or the literal current to snap to the corresponding extension’s current head; both default to current if the extension is initialized on the vault, and default to omitted (null pin) otherwise.

--k defaults to 60. --weight semopsis and --weight logopsis default to 1.0 for whichever sources are pinned.

7.2 zet advance

zet advance [--semopsis <ref>|current|none]
           [--logopsis <ref>|current|none]
           [--k <number>]
           [--weight semopsis=<number>]
           [--weight logopsis=<number>]

Advance zetetes-head per §5.2. At least one of --semopsis, --logopsis, --k, or any --weight MUST be supplied. On success, prints the new zetetes-state hash to standard output.

A common shorthand zet advance --semopsis current --logopsis current snaps the zetetes-state to the current heads of both extensions, preserving the existing fusion config.

7.3 zet query

zet query [--k <int>] [--as-of <zetetes-stateref>] <query>
zet query [--k <int>] [--as-of <zetetes-stateref>] -

Run a hybrid query per §5.3. <query> is the query text; - reads from standard input. Default --k is 10. Default --as-of is current.

Output format per result, one line:

<entry-uuid>  <blob-hash>  <fused-score>

Implementations MAY support a --verbose flag that adds per-source ranks and scores to the output; the verbose format is implementation-defined.

7.4 zet state

zet state [<zetetes-stateref>]

Show metadata for a zetetes-state: hash, created_at_ns, previous, metatheca, semopsis pin, logopsis pin, fusion config. Default argument: current.

7.5 zet log

zet log [--since <iso8601>] [--until <iso8601>] [-n <count>]

Walk the zetetes-state chain from current zetetes-head backward, emitting one line per zetetes-state with its short hash, ISO-8601 timestamp, semopsis pin short hash (or -), logopsis pin short hash (or -), and fusion summary.

7.6 zet fsck

zet fsck

Verify: the zetetes-head pinax bytes encode a zetetes-state present in the cella; every zetetes-state in the chain parses; every non-null pin names a search-state blob that is present and parses on the corresponding extension’s own cella — presence as a blob, not reachability from that extension’s current head: an extension chain advancing past a pinned state does not invalidate the pin; every fusion config’s weights key set matches the set of non-null pins per §4.2; every metatheca field names a metatheca state that is present and parses on the host metatheca cella. Implementations MAY rely on syntheca and apotheca verification for byte-level integrity.

7.7 zet destroy

zet destroy [--force] [<path>]

Destroy the zetetes cella per §5.6. The vault root is <path> if supplied, otherwise the current directory. Unless --force is supplied, implementations SHOULD prompt for confirmation before proceeding, since destroy discards the audit trail of prior zetetes-states. Exits non-zero if no zetetes cella exists at the resolved path.

7.8 Reserved

The verb gc is reserved for future phases.

8 Errors

The error conditions defined by this specification are:

Implementations MAY surface additional implementation-defined errors (I/O failure, extension binary unavailable, …); such errors MUST be distinguishable from the protocol-defined errors above.

9 Conformance

A Phase 1 implementation MUST implement:

A Phase 1 implementation MAY omit:

10 Out of scope (Phase 1)

The following are deliberately out of Phase 1 scope. Their absence is load-bearing for the v0.1 surface and consumers MUST NOT rely on them.

11 Future work (non-normative)