diff --git a/docs/CACERT_TRANSPARENCY_LOG_REGISTRY.md b/docs/CACERT_TRANSPARENCY_LOG_REGISTRY.md new file mode 100644 index 00000000000..a9d12f60625 --- /dev/null +++ b/docs/CACERT_TRANSPARENCY_LOG_REGISTRY.md @@ -0,0 +1,480 @@ +# CACert Transparency Log + Registry (CEN-T) Architecture + +## 1) ARCHITECTURE + +### System model + +CEN-T is a certificate-transparency-style append-only log for CACerts. Every CACert issuance is accepted only if it is durably appended to the log and returns a Signed Tree Head (STH) plus inclusion proof reference. + +Core components: + +- **Log Append Service (`cent-log`)** + - Accepts canonical CACert payloads and metadata envelope. + - Computes deterministic `entry_hash` and appends to storage in strict sequence order. + - Rebuilds or incrementally updates Merkle tree state. + - Emits STH snapshots (`tree_size`, `root_hash`, `sth_signature`). +- **Transparency Registry API (`cent-registry`)** + - Public, read-only API for entry lookup, STH retrieval, inclusion proof, and consistency proof. + - Supports cache-friendly immutable routes by `tree_size` and `entry_index`. +- **CLI Verifier (`summitctl cacert transparency`)** + - Offline verification of inclusion and consistency proofs. + - Verifies STH signatures against pinned public keys. + - Detects equivocation via persisted STH history. +- **Monitors / Witnesses** + - Third-party pull STH and proofs. + - Compare observed STH sequence over time to detect split views. + - Optionally cosign witnessed STH checkpoints. + +### Append-only data structure + +- Merkle tree with RFC6962-style leaf/internal domain separation: + - `leaf_hash = H(0x00 || canonical_log_entry_bytes)` + - `node_hash = H(0x01 || left_child_hash || right_child_hash)` +- Hash algorithm: SHA-256. +- Leaf order: strict append index (`entry_index` monotonic, zero-based). +- No in-place mutation, no delete, no reorder. +- STH identity: tuple of `tree_size` + `root_hash` + `log_id` + `issued_at` (timestamp excluded from hash scope). + +### Inclusion proof flow + +1. Issuer submits CACert to `POST /api/cacert/log/entries`. +2. Service canonicalizes payload, computes `entry_hash`, appends leaf. +3. Service returns `entry_index`, current `tree_size`, and STH. +4. Client or auditor requests `GET /api/cacert/log/proof/inclusion?...`. +5. Verifier reconstructs root from leaf + audit path. +6. Verifier checks reconstructed root equals STH `root_hash` and verifies `sth_signature`. + +### Consistency proof flow + +1. Auditor stores prior STH at size `m` and sees new STH at size `n` (`n > m`). +2. Auditor requests `GET /api/cacert/log/proof/consistency?from_tree_size=m&to_tree_size=n`. +3. Verifier checks cryptographic append-only relation proving `STH(m)` is a prefix-consistent ancestor of `STH(n)`. +4. Failure indicates fork, corruption, or equivocation. + +### Equivocation detection model + +- CLI maintains a local checkpoint log of observed STHs. +- If two STHs share the same `tree_size` but different `root_hash`, emit `EQUIVOCATION_DETECTED`. +- If consistency proof fails between sequential observed STHs, emit `INCONSISTENT_TREE_HISTORY`. +- Witness cosigning makes equivocation publicly undeniable. + +### Storage model + +- **Local authoritative store (required):** + - `data/cacert-transparency/log-entries.ndjson` (append-only records) + - `data/cacert-transparency/tree-state/` (Merkle layers or incremental frontier) + - `data/cacert-transparency/sth/` (signed checkpoints) +- **Optional remote replication (recommended):** + - Object storage mirror of immutable segments by sequence range. + - Optional secondary log mirror for cross-region availability. + - Witness checkpoint replication (public bucket or OCI artifact). + +## 2) PR STACK (≤7 PRs TOTAL) + +### PR1 — `feat(cent): add append-only CACert log entry model and canonical hashing` + +- **Purpose:** Define deterministic, hash-stable entry format and append-only persistence. +- **Exact file paths:** + - `packages/cacert-transparency/src/canonicalize.ts` + - `packages/cacert-transparency/src/hash.ts` + - `packages/cacert-transparency/src/storage/append_log.ts` + - `packages/cacert-transparency/src/types/log_entry.ts` + - `packages/cacert-transparency/test/canonicalize.test.ts` +- **API routes:** none (library layer only). +- **CLI tooling:** none. +- **CI wiring:** add unit tests + deterministic fixture snapshots in package test job. + +### PR2 — `feat(cent): implement Merkle builder, inclusion proofs, and signed tree heads` + +- **Purpose:** Generate Merkle roots and inclusion proofs over append-only entries. +- **Exact file paths:** + - `packages/cacert-transparency/src/merkle/tree.ts` + - `packages/cacert-transparency/src/merkle/inclusion.ts` + - `packages/cacert-transparency/src/sth/sign_sth.ts` + - `packages/cacert-transparency/test/inclusion_proof.test.ts` +- **API routes:** none. +- **CLI tooling:** none. +- **CI wiring:** property tests for proof reconstruction and root determinism. + +### PR3 — `feat(server): expose transparency append/read API` + +- **Purpose:** Minimal public API for append, STH fetch, entry lookup, and inclusion proofs. +- **Exact file paths:** + - `server/src/routes/cacert/transparency.ts` + - `server/src/services/cacertTransparencyService.ts` + - `server/src/schema/cacertTransparency.openapi.json` + - `server/tests/cacert-transparency-api.test.ts` +- **API routes:** + - `POST /api/cacert/log/entries` + - `GET /api/cacert/log/sth/latest` + - `GET /api/cacert/log/entries/:entryIndex` + - `GET /api/cacert/log/proof/inclusion` +- **CLI tooling:** none. +- **CI wiring:** integration test lane for API contract + signature verification. + +### PR4 — `feat(cent): add consistency proof generator + verifier` + +- **Purpose:** Provide append-only proofs across tree versions and detect forks. +- **Exact file paths:** + - `packages/cacert-transparency/src/merkle/consistency.ts` + - `packages/cacert-transparency/src/verify/verify_consistency.ts` + - `packages/cacert-transparency/test/consistency_proof.test.ts` +- **API routes:** none. +- **CLI tooling:** none. +- **CI wiring:** regression vectors for valid/invalid consistency proofs. + +### PR5 — `feat(server): publish consistency proof and checkpoint history endpoints` + +- **Purpose:** Enable independent auditors to pull consistency data and historical checkpoints. +- **Exact file paths:** + - `server/src/routes/cacert/transparency.ts` + - `server/src/services/cacertTransparencyService.ts` + - `server/tests/cacert-transparency-consistency-api.test.ts` +- **API routes:** + - `GET /api/cacert/log/proof/consistency` + - `GET /api/cacert/log/sth/:treeSize` + - `GET /api/cacert/log/sth/history` +- **CLI tooling:** none. +- **CI wiring:** API schema tests + backward compatibility checks. + +### PR6 — `feat(cli): offline verify inclusion/consistency and detect equivocation` + +- **Purpose:** Remove server trust assumption with standalone verifier and local checkpoint db. +- **Exact file paths:** + - `tools/summitctl/src/commands/cacert/transparency-verify.ts` + - `tools/summitctl/src/commands/cacert/transparency-monitor.ts` + - `tools/summitctl/src/lib/checkpointStore.ts` + - `tools/summitctl/test/transparency-verify.test.ts` +- **API routes:** consumes routes from PR3+PR5. +- **CLI tooling:** + - `summitctl cacert transparency verify-inclusion --entry --proof --sth ` + - `summitctl cacert transparency verify-consistency --old-sth --new-sth --proof ` + - `summitctl cacert transparency monitor --from --checkpoint-db ` +- **CI wiring:** offline fixture verification job (no network). + +### PR7 — `ci(cent): enforce logged-before-valid CACert policy` + +- **Purpose:** Block any CACert artifact unless it includes valid inclusion proof bound to STH. +- **Exact file paths:** + - `scripts/ci/verify_cacert_transparency.ts` + - `.github/workflows/cacert-transparency-gate.yml` + - `docs/CACERT_TRANSPARENCY_LOG_REGISTRY.md` + - `docs/roadmap/STATUS.json` +- **API routes:** none. +- **CLI tooling:** CI invokes verifier command in non-interactive mode. +- **CI wiring:** required check `cacert-transparency-gate` with fail-closed semantics. + +## 3) SCHEMAS + +### `log_entry.json` + +```json +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "CACert Transparency Log Entry", + "type": "object", + "required": [ + "version", + "log_id", + "entry_index", + "leaf_hash", + "certificate", + "metadata" + ], + "properties": { + "version": { "type": "string", "const": "1" }, + "log_id": { "type": "string", "minLength": 1 }, + "entry_index": { "type": "integer", "minimum": 0 }, + "leaf_hash": { + "type": "string", + "pattern": "^[a-f0-9]{64}$" + }, + "certificate": { + "type": "object", + "required": ["cacert_id", "subject_digest", "payload_hash", "signature"], + "properties": { + "cacert_id": { "type": "string", "minLength": 1 }, + "subject_digest": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "payload_hash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "signature": { "type": "string", "minLength": 1 } + }, + "additionalProperties": false + }, + "metadata": { + "type": "object", + "required": ["issuer", "policy_version", "evidence_ref"], + "properties": { + "issuer": { "type": "string", "minLength": 1 }, + "policy_version": { "type": "string", "minLength": 1 }, + "evidence_ref": { "type": "string", "minLength": 1 }, + "logged_at": { "type": "string", "format": "date-time" } + }, + "additionalProperties": false + } + }, + "additionalProperties": false +} +``` + +Hashed scope rules: + +- Canonicalized fields in hash scope: `version`, `log_id`, `entry_index`, `certificate`, `metadata.issuer`, `metadata.policy_version`, `metadata.evidence_ref`. +- Explicitly out of hash scope: `metadata.logged_at`. +- Canonical encoding: RFC8785 JSON Canonicalization Scheme. + +### `inclusion_proof.json` + +```json +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "CACert Inclusion Proof", + "type": "object", + "required": [ + "version", + "log_id", + "tree_size", + "root_hash", + "entry_index", + "leaf_hash", + "audit_path", + "sth" + ], + "properties": { + "version": { "type": "string", "const": "1" }, + "log_id": { "type": "string" }, + "tree_size": { "type": "integer", "minimum": 1 }, + "root_hash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "entry_index": { "type": "integer", "minimum": 0 }, + "leaf_hash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "audit_path": { + "type": "array", + "items": { "type": "string", "pattern": "^[a-f0-9]{64}$" } + }, + "sth": { + "type": "object", + "required": ["tree_size", "root_hash", "sth_signature", "log_pubkey_id"], + "properties": { + "tree_size": { "type": "integer", "minimum": 1 }, + "root_hash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "sth_signature": { "type": "string" }, + "log_pubkey_id": { "type": "string" }, + "issued_at": { "type": "string", "format": "date-time" } + }, + "additionalProperties": false + } + }, + "additionalProperties": false +} +``` + +Determinism rules: + +- `audit_path` order is leaf-to-root sibling order. +- `root_hash` must match `sth.root_hash`. +- `issued_at` excluded from cryptographic root and leaf calculations. + +### `consistency_proof.json` + +```json +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "CACert Consistency Proof", + "type": "object", + "required": [ + "version", + "log_id", + "from_tree_size", + "to_tree_size", + "from_root_hash", + "to_root_hash", + "consistency_path", + "old_sth", + "new_sth" + ], + "properties": { + "version": { "type": "string", "const": "1" }, + "log_id": { "type": "string" }, + "from_tree_size": { "type": "integer", "minimum": 1 }, + "to_tree_size": { "type": "integer", "minimum": 2 }, + "from_root_hash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "to_root_hash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "consistency_path": { + "type": "array", + "items": { "type": "string", "pattern": "^[a-f0-9]{64}$" } + }, + "old_sth": { + "type": "object", + "required": ["tree_size", "root_hash", "sth_signature", "log_pubkey_id"], + "properties": { + "tree_size": { "type": "integer", "minimum": 1 }, + "root_hash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "sth_signature": { "type": "string" }, + "log_pubkey_id": { "type": "string" } + }, + "additionalProperties": false + }, + "new_sth": { + "type": "object", + "required": ["tree_size", "root_hash", "sth_signature", "log_pubkey_id"], + "properties": { + "tree_size": { "type": "integer", "minimum": 2 }, + "root_hash": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "sth_signature": { "type": "string" }, + "log_pubkey_id": { "type": "string" } + }, + "additionalProperties": false + } + }, + "additionalProperties": false +} +``` + +Determinism rules: + +- `from_tree_size < to_tree_size`. +- Proof valid only when `old_sth.tree_size == from_tree_size` and `new_sth.tree_size == to_tree_size`. +- Timestamps are optional and outside hash scope. + +## 4) ATTACK / FAILURE CASES (CRITICAL) + +### Case 1: Log fork / split-view attack + +- **Attack:** Log serves different roots for same `tree_size` to different clients. +- **Detection:** Compare signed STHs gossip/witness feeds. +- **Verification result:** fail. +- **Structured error:** + +```json +{ + "code": "EQUIVOCATION_DETECTED", + "message": "Conflicting STH roots for identical tree_size.", + "tree_size": 1048576, + "observed_roots": ["", ""], + "log_id": "cacert-prod" +} +``` + +### Case 2: Withheld CACert (certificate issued but not logged) + +- **Attack:** Issuer provides CACert without log entry/proof. +- **Detection:** CI gate + verifier requires valid inclusion proof bound to STH. +- **Verification result:** fail. +- **Structured error:** + +```json +{ + "code": "MISSING_INCLUSION_PROOF", + "message": "CACert presented without verifiable transparency inclusion.", + "cacert_id": "", + "required_min_tree_size": 1 +} +``` + +### Case 3: Tampered log entry payload + +- **Attack:** Entry content modified post-append. +- **Detection:** Recompute leaf hash from canonical entry; mismatch with recorded leaf/proof. +- **Verification result:** fail. +- **Structured error:** + +```json +{ + "code": "ENTRY_HASH_MISMATCH", + "message": "Canonicalized entry hash does not match claimed leaf hash.", + "entry_index": 412, + "expected_leaf_hash": "", + "actual_leaf_hash": "" +} +``` + +### Case 4: Invalid inclusion proof + +- **Attack:** Audit path forged or reordered. +- **Detection:** Reconstructed root differs from signed STH root. +- **Verification result:** fail. +- **Structured error:** + +```json +{ + "code": "INVALID_INCLUSION_PROOF", + "message": "Inclusion proof does not reconstruct signed tree root.", + "entry_index": 412, + "tree_size": 1024, + "reconstructed_root": "", + "signed_root": "" +} +``` + +### Case 5: Inconsistent tree state / broken append-only guarantee + +- **Attack:** New STH not prefix-consistent with prior STH. +- **Detection:** Consistency verification fails between `m` and `n`. +- **Verification result:** fail. +- **Structured error:** + +```json +{ + "code": "INCONSISTENT_TREE_HISTORY", + "message": "Consistency proof failed; append-only invariant violated.", + "from_tree_size": 8192, + "to_tree_size": 16384, + "from_root": "", + "to_root": "" +} +``` + +### Case 6: Replayed proof against different context + +- **Attack:** Old valid proof replayed with different entry/tree or stale STH. +- **Detection:** Bind proof to `(log_id, entry_index, tree_size, root_hash)` and enforce exact match. +- **Verification result:** fail. +- **Structured error:** + +```json +{ + "code": "PROOF_CONTEXT_MISMATCH", + "message": "Proof replay detected: context binding mismatch.", + "expected": { + "log_id": "cacert-prod", + "entry_index": 9001, + "tree_size": 20000 + }, + "received": { + "log_id": "cacert-prod", + "entry_index": 8999, + "tree_size": 19900 + } +} +``` + +### Case 7: Untrusted or rotated key without trust update + +- **Attack:** STH signed by unknown key id. +- **Detection:** Local trust store key mismatch / missing key pin. +- **Verification result:** fail. +- **Structured error:** + +```json +{ + "code": "UNTRUSTED_LOG_KEY", + "message": "STH signature key is not trusted for this log_id.", + "log_id": "cacert-prod", + "log_pubkey_id": "" +} +``` + +### Case 8: Entry index gap or duplicate index + +- **Attack:** Silent omission by skipping index or rewriting index. +- **Detection:** Monotonic index audit over append stream. +- **Verification result:** fail. +- **Structured error:** + +```json +{ + "code": "NON_MONOTONIC_INDEX", + "message": "Append-only sequence violated (gap or duplicate index).", + "previous_index": 411, + "current_index": 413 +} +``` diff --git a/docs/roadmap/STATUS.json b/docs/roadmap/STATUS.json index 26d6924bff2..a86b088ac92 100644 --- a/docs/roadmap/STATUS.json +++ b/docs/roadmap/STATUS.json @@ -1,6 +1,6 @@ { - "last_updated": "2026-04-03T00:00:00Z", - "revision_note": "Added the canonical Decision Object v1 schema package, example payload, and standards documentation to anchor CAC-bound decision interoperability and external verification workflows.", + "last_updated": "2026-03-31T00:00:00Z", + "revision_note": "Added the CACert transparency log + registry architecture baseline with append-only Merkle proof design, CI gate plan, and verifier requirements for non-repudiable admissibility auditing.", "initiatives": [ { "id": "one-verified-workflow-lane", @@ -60,7 +60,7 @@ "id": "provable-system-governance-provenance-unification", "status": "in_progress", "owner": "codex", - "notes": "Implementation-ready governance, provenance, isolation, sovereignty, and ATO-native evidence bundle specifications are published and awaiting narrowed execution through one golden workflow. Published C2PA-aligned CAC Decision Manifest profile and external verification contract for admissible cognition artifacts." + "notes": "Implementation-ready governance, provenance, isolation, sovereignty, and ATO-native evidence bundle specifications are published and awaiting narrowed execution through one golden workflow." }, { "id": "antigravity-multi-agent-ga-convergence", @@ -69,16 +69,16 @@ "notes": "Multi-agent prompt suites, bounded charters, and router activation are in place, but GA still depends on proving one deterministic closed loop rather than widening orchestration." }, { - "id": "decision-object-canonicalization", - "status": "completed", + "id": "cacert-transparency-log-registry", + "status": "in_progress", "owner": "codex", - "notes": "Published schemas/decision-object.schema.json plus a complete example and standards profile for CAC-bound deterministic verification." + "notes": "Defined CEN-T architecture, PR stack, schemas, and attack detection model to guarantee logged-before-valid CACerts and equivocation detection." } ], "summary": { "total_initiatives": 12, - "completed": 5, - "in_progress": 7, + "completed": 4, + "in_progress": 8, "at_risk": 0 } }