diff --git a/docs/roadmap/STATUS.json b/docs/roadmap/STATUS.json index 26d6924bff2..db668164455 100644 --- a/docs/roadmap/STATUS.json +++ b/docs/roadmap/STATUS.json @@ -1,6 +1,6 @@ { - "last_updated": "2026-04-03T00:00:00Z", - "revision_note": "Added the canonical Decision Object v1 schema package, example payload, and standards documentation to anchor CAC-bound decision interoperability and external verification workflows.", + "last_updated": "2026-03-31T00:00:00Z", + "revision_note": "Added CAC Procurement and Audit Standard v1.0 with enforceable RFP clauses, repeatable third-party audit procedures, compliance crosswalks, and phased market-enforcement adoption guidance.", "initiatives": [ { "id": "one-verified-workflow-lane", @@ -60,7 +60,7 @@ "id": "provable-system-governance-provenance-unification", "status": "in_progress", "owner": "codex", - "notes": "Implementation-ready governance, provenance, isolation, sovereignty, and ATO-native evidence bundle specifications are published and awaiting narrowed execution through one golden workflow. Published C2PA-aligned CAC Decision Manifest profile and external verification contract for admissible cognition artifacts." + "notes": "Implementation-ready governance, provenance, isolation, sovereignty, and ATO-native evidence bundle specifications are published and awaiting narrowed execution through one golden workflow." }, { "id": "antigravity-multi-agent-ga-convergence", @@ -69,10 +69,10 @@ "notes": "Multi-agent prompt suites, bounded charters, and router activation are in place, but GA still depends on proving one deterministic closed loop rather than widening orchestration." }, { - "id": "decision-object-canonicalization", + "id": "cac-procurement-audit-standard-v1", "status": "completed", "owner": "codex", - "notes": "Published schemas/decision-object.schema.json plus a complete example and standards profile for CAC-bound deterministic verification." + "notes": "Published controlled CAC procurement and audit standard with testable clauses, artifact requirements, and phased adoption controls." } ], "summary": { @@ -81,4 +81,4 @@ "in_progress": 7, "at_risk": 0 } -} +} \ No newline at end of file diff --git a/docs/standards/CAC_Procurement_Audit_Standard_v1.0.md b/docs/standards/CAC_Procurement_Audit_Standard_v1.0.md new file mode 100644 index 00000000000..9a316d6b319 --- /dev/null +++ b/docs/standards/CAC_Procurement_Audit_Standard_v1.0.md @@ -0,0 +1,165 @@ +# CAC Procurement and Audit Standard v1.0 + +## 1) STANDARD DOCUMENT (RFC-STYLE) + +### Abstract +This document defines mandatory Cognitive Admissibility Criteria (CAC) controls for AI and decision systems used in regulated or high-impact operations. Conformance is determined solely through verifiable artifacts, including CACert, decision trace evidence, and append-only transparency log proofs. A system that cannot produce verifiable artifacts is non-conformant. + +### Scope +This standard applies to: +- AI models and agentic systems that generate recommendations, decisions, classifications, scores, or actions. +- Rule-based and hybrid systems that influence operational or compliance outcomes. +- Build, deploy, and runtime pipelines that issue or consume admissibility verdicts. + +This standard governs procurement qualification, operational acceptance, and third-party audit. + +### Definitions +- **Admissibility Verdict**: Binary outcome (`PASS` or `FAIL`) issued after evidence and policy validation. +- **Evidence Bundle**: Deterministic and signed artifact set (`report.json`, `metrics.json`, `stamp.json`) linked to a decision run. +- **Decision Trace**: End-to-end provenance and policy evaluation record from input receipt to output publication. +- **CACert**: Signed certificate asserting system conformance level, validity window, and evidence root hash. +- **Transparency Log Entry**: Append-only Merkle-log record containing artifact digest, inclusion proof, and log index. +- **Verifier**: Independent tool or service that validates signatures, hashes, trace continuity, CACert status, and log inclusion proof. + +### Normative Language +The key words MUST, MUST NOT, SHOULD, and MAY are normative. + +### CAC Requirements +#### Evidence Requirements +1. Systems MUST generate an Evidence Bundle for every production decision run. +2. Evidence artifacts MUST be content-addressed (SHA-256 or stronger) and signed. +3. Evidence artifacts MUST include policy evaluation results and model/runtime version identifiers. +4. Missing or unsigned evidence MUST produce an automatic `FAIL` verdict. + +#### Traceability Requirements +1. Systems MUST produce a Decision Trace that links each transformation step to prior hashes. +2. Trace records MUST include timestamp, actor/workload identity, control decision, and artifact digest. +3. Trace continuity breaks (missing step, hash mismatch, unsigned hop) MUST produce `FAIL`. +4. Trace retention SHOULD meet or exceed contractual retention windows; shorter retention MUST be declared as a non-conformance. + +#### Admissibility Verdict Requirements +1. Every decision run MUST emit exactly one admissibility verdict: `PASS` or `FAIL`. +2. `PASS` MUST be issued only when evidence, trace, certificate status, and transparency proofs validate successfully. +3. `FAIL` verdicts MUST block deployment and downstream decision publication until remediated and re-verified. +4. Systems MUST NOT emit “partial pass,” “warning pass,” or equivalent ambiguous states. + +#### Certification (CACert) Requirements +1. Conformant systems MUST emit a signed CACert containing at minimum: system identifier, certification level, validity dates, evidence root hash, issuing authority, and revocation endpoint. +2. CACert signatures MUST chain to a trusted root declared in procurement artifacts. +3. Expired, revoked, or unverifiable CACerts MUST cause procurement non-compliance and runtime `FAIL`. +4. CACert issuance and renewal SHOULD include independent verification evidence from outside the operating team. + +### Verification Requirements +1. Organizations MUST support independent verification via documented API and CLI procedures. +2. Verification MUST be reproducible by third parties without privileged internal access. +3. Verification MUST validate: artifact signatures, evidence hash integrity, trace continuity, CACert validity/revocation status, and transparency log inclusion proof. +4. Verification tooling version, command output, and timestamps MUST be preserved in audit artifacts. + +### Transparency Log Requirements +1. Each admissibility-relevant artifact digest MUST be submitted to an append-only Merkle transparency log. +2. Systems MUST provide inclusion proof and log index for each submitted digest. +3. Audit workflows MUST verify that the inclusion proof recomputes to a published signed tree head. +4. Log retroactive mutation, gap, or inclusion-proof failure MUST be treated as critical non-conformance. + +--- + +## 2) PROCUREMENT LANGUAGE (RFP READY) + +1. Vendor MUST provide a verifiable CACert for each proposed decision system before contract award. +2. Vendor MUST demonstrate that all production decision outputs are gated by a binary CAC admissibility verdict (`PASS`/`FAIL`). +3. Vendor MUST produce per-run Evidence Bundles (`report.json`, `metrics.json`, `stamp.json`) signed and hash-addressed. +4. Vendor MUST provide Decision Trace records that establish end-to-end provenance from input ingestion through output publication. +5. Vendor MUST expose independent verification through both CLI and API methods documented in technical response materials. +6. Vendor MUST provide proof that each admissibility-relevant artifact digest is anchored in an append-only transparency log with Merkle inclusion proof. +7. Vendor MUST provide CACert revocation and status-check endpoint(s), and contracting authority reserves right to reject expired/revoked certificates at any time. +8. Vendor MUST block release, deployment, or automated action when admissibility verdict is `FAIL`. +9. Vendor MUST preserve evidence and trace artifacts for the retention period stated in this solicitation and make artifacts available to authorized auditors within the contract response SLA. +10. Vendor MUST support third-party replay verification using supplied hashes, signatures, and inclusion proofs without requiring access to proprietary source code. +11. Vendor MUST document cryptographic algorithms, trust roots, key lifecycle controls, and signature verification procedures used for CAC evidence. +12. Vendor MUST notify procuring authority within 24 hours of any CACert revocation, transparency-log inconsistency, or failed admissibility gate affecting production. +13. Vendor MUST provide remediation evidence and re-verification results before failed controls are returned to production service. +14. Vendor MUST submit quarterly conformance attestations including pass/fail rates, failed-control root causes, and corrective action closure evidence. + +--- + +## 3) AUDIT FRAMEWORK + +### Audit Checklist +1. Confirm valid CACert exists for each in-scope system. +2. Confirm CACert signature chain and revocation status validate. +3. Sample decision runs and collect associated Evidence Bundles. +4. Verify bundle signatures and hash integrity. +5. Verify Decision Trace continuity and policy-evaluation completeness. +6. Validate binary admissibility verdict is present for each sampled run. +7. Confirm `FAIL` verdicts blocked deployment/actions and generated remediation records. +8. Verify transparency-log inclusion proof for sampled artifacts. +9. Recompute Merkle proof against signed tree head. +10. Validate retention/accessibility of required artifacts within SLA. + +### Required Artifacts +- Current CACert and certificate history (issuance, renewal, revocation records). +- Evidence Bundles for sampled runs: `report.json`, `metrics.json`, `stamp.json`. +- Decision Trace ledger for sampled runs. +- Verifier execution outputs (CLI/API) with timestamps and tool versions. +- Transparency log entries, inclusion proofs, and corresponding signed tree heads. +- Incident/remediation records for all sampled `FAIL` verdicts. + +### Verification Procedures +1. Select statistically or risk-weighted sample set (minimum 30 runs per critical system or 5% of monthly volume, whichever is greater). +2. Execute independent verifier on each sample. +3. Recompute artifact hashes locally and compare with signed manifests. +4. Validate CACert signature, validity window, and revocation status. +5. Validate each transparency proof by recomputing root and matching published tree head. +6. Confirm policy gate logs show blocked release for every `FAIL` case. +7. Record each control as Pass/Fail with objective evidence reference. + +### Pass/Fail Conditions +- **PASS**: All mandatory controls validate with complete evidence and no critical exceptions. +- **CONDITIONAL FAIL**: One or more major controls fail but no evidence of runtime bypass; remediation window may be granted per contract. +- **FAIL**: Any missing/unverifiable CACert, missing evidence bundle, broken trace continuity, unverifiable log proof, or detected `FAIL`-gate bypass. + +Third-party auditors MUST be able to reproduce results with provided artifacts and documented verifier steps. + +--- + +## 4) COMPLIANCE MAPPING + +| Framework | Alignment to CAC | Gaps CAC Fills | +|---|---|---| +| **NIST AI RMF** (Govern, Map, Measure, Manage) | **Govern**: policy-defined admissibility gates and certificate lifecycle; **Map**: explicit system scope and trace boundaries; **Measure**: repeatable verification outputs and pass/fail evidence; **Manage**: enforced blocking on `FAIL` plus remediation loops. | Converts broad risk-management expectations into deterministic technical controls with artifact-level proof and binary admissibility outcomes. | +| **ISO/IEC 42001** (AI management systems) | Supports auditable AI lifecycle governance through required evidence, operational controls, and independent verification steps. | Adds cryptographic artifact integrity, machine-verifiable trace continuity, and transparency-log anchoring not consistently mandated at implementation detail level. | +| **SOC 2** (security, availability, processing integrity, confidentiality) | Processing integrity and change-control assertions are supported by signed evidence, deterministic verification, and gate-enforced release controls. | Provides objective, replayable proof for control operation (not just policy statements), reducing reliance on interview/testimony-only audit evidence. | +| **EU AI Act** (transparency, risk management, recordkeeping, post-market obligations) | Strengthens transparency and recordkeeping through mandatory decision traces, certificate status checks, and retained verification artifacts. | Supplies concrete admissibility and cryptographic-verification mechanism for demonstrating continuous conformity and post-market accountability. | + +--- + +## 5) ADOPTION STRATEGY (CRITICAL) + +### Phase 0–90 Days: Procurement Insertion +1. Publish this CAC Procurement and Audit Standard as a controlled reference in supplier qualification packages. +2. Add mandatory RFP clauses requiring CACert, decision trace, and transparency-proof artifacts at proposal stage. +3. Update pre-award scoring: any vendor lacking verifiable CAC artifacts is disqualified from technical compliance scoring. +4. Train procurement and legal teams on artifact-based acceptance criteria and rejection triggers. +5. Establish baseline auditor runbook using the audit framework above. + +### Phase 90–180 Days: Auditor and Buyer Normalization +1. Require quarterly independent verification reports from existing vendors. +2. Include CAC control testing in internal audit plans and third-party assurance statements. +3. Add contract remedies for missed CAC obligations (service credits, remediation deadlines, suspension rights). +4. Build a public or consortium-accessible verifier profile so auditors can run consistent test suites. +5. Publish procurement FAQs defining “acceptable evidence” versus “non-verifiable claims.” + +### Phase 180+ Days: Market Enforcement and Vendor Compulsion +1. Make CAC conformance a standing condition for renewal and expansion; non-conformant vendors move to controlled offboarding. +2. Require subcontractors and downstream model providers to inherit CAC artifact obligations by flow-down clauses. +3. Integrate CAC pass/fail status into vendor risk scoring and board-level technology risk dashboards. +4. Require auditors to issue explicit CAC control opinions in annual assurance cycles. +5. Establish sector consortium language so CACert becomes a recognized interoperability and trust signal across buyers. + +### Enforcement Mechanics +- **How CAC becomes required in procurement**: embed mandatory clauses + disqualification criteria in RFP/RFQ templates and MSA control schedules. +- **How CACert becomes a market signal**: require valid CACert as a prerequisite for shortlist, renewal, and preferred-vendor status. +- **How auditors adopt verification**: standardize evidence package schema, verifier commands, and proof-validation steps in annual audit programs. +- **How vendors are forced to comply**: couple CAC conformance to contract eligibility, revenue continuity, and remedial penalty terms. + +A vendor that cannot produce verifiable CACert, trace, and transparency-log proofs is non-compliant and fails qualification.