Audit trails for AI decisions
When an AI system approves a refund, runs a query or merges code, someone may later ask how that decision was checked. This guide covers the record QWED leaves for each check, how a reviewer can test it, and what it does not prove.
The short answer
An audit trail for AI decisions should say, for every output or action: what was checked, how, what came out and what was decided. QWED returns that as one record per check, with a commitment a reviewer can recompute. Storing the records and protecting the store is up to you.
What a QWED record holds
Every check returns a result with a status, a message and structured findings. Engines can also write it as a Verification Context record (spec v1.0, draft), which has four layers plus the verdict:
| Question | Field | Contents |
|---|---|---|
| What was checked | object.formal_statement | The exact statement the engine evaluated, for example an expression, a query or "this commit is safe to merge". |
| What it means | context.interpretation | The theory, logic or SQL dialect the statement is read under. |
| How it was checked | context.proof | The verifier, its version, its configuration, and how timeouts and errors are treated. |
| What came out | context.evidence | The engine result: status, message, findings and the evidence it kept. |
| Commitment | context.evidence.proof_ref | A SHA-256 commitment to everything above. Present only on VERIFIED. |
| What was decided | context.decision.admission | ADMIT or DENY, separate from whether the claim was true. |
| Verdict | verdict | VERIFIED, UNVERIFIABLE or BLOCKED. |
A result that is UNVERIFIABLE or BLOCKED has no proof_ref, and its findings say why. That is still a record: it shows what was held back or stopped, and for what reason. Fail-closed, explained
A real record
QWED Security posts a record like this on every pull request it scans. This one is from a QWED documentation pull request, shortened:
{ "spec_version": "1.0", "object": { "formal_statement": "QWED-AI/docs@851611b2 (PR #325) is safe to merge" }, "context": { "interpretation": { "theory": "deterministic security verification", "logic": "evidence-context-policy pipeline" }, "proof": { "verifier": "QWED Security", "verifier_version": "qwed-security-ruleset/12", "configuration": { "repo": "QWED-AI/docs", "head_sha": "851611b2ec703f7f75d2744f090af95a635af45e", "files_scanned": 1, "attestation_token": "eyJhbGciOiJSUzI1NiIs… (signed token, shortened here)", "attestation_jwks_url": "https://qwed-security-334760594829.us-central1.run.app/.well-known/jwks.json" }, "theory_scope": "PR security scan against QWED deterministic rule set", "outcome_treatment": "unknown/timeout/error resolve to UNVERIFIABLE or BLOCKED" }, "evidence": { "evidence": { "status": "VERIFIED", "developer_fields": { "total_findings": 0, "advisory_checks": [] }, "scan_evidence": { "rule_set": "qwed-security-ruleset/12", "engines": ["entropy_scan", "pattern_scan"] } }, "proof_ref": "sha256:d98d9979cd86754eb32a9258d08799a910dcfa489c5c6129a4eb229be62f92d3" }, "decision": { "admission": "ADMIT" } }, "verdict": "VERIFIED"}The record says which commit was checked, against which rule set, with which engines, what they found and what was decided. It does not say the code has no bugs: the message in the full record states that it covers known patterns for this rule set and commit only.
How a reviewer checks a record later
- Recompute the commitment. Remove
proof_ref, encode the formal statement and context as canonical JSON (RFC 8785) and take the SHA-256. If it differs from the stored value, the record was changed and counts as unverified. - Check the signature, where there is one. Some records carry a signed attestation: QWED Security signs its results and publishes the public keys, and QWED A2A and the agent verification path sign with ES256. A valid signature shows who issued the record and that it has not changed since.
- Run the check again. The engines are deterministic: the same formal statement, checked with the same verifier version and configuration, gives the same verdict. The record names all three. A solver timeout is the exception, and it never resolves to VERIFIED.
Step one in Python, with the core package (pip install qwed):
import jsonfrom qwed_new.core.verification_context import resolve_document_proof_refwith open("record.json", encoding="utf-8") as f: record = json.load(f)# True only for a VERIFIED record whose proof_ref matches its contents.print(resolve_document_proof_ref(record))Where records are kept
- In your own log store. The library and the API return a result with each check, and in the library each engine can also produce the full Verification Context record. Write it to the place you already keep audit logs: a database, object storage with retention, or your SIEM.
- On the self-hosted API. The API also stores each request for the organisation that made it: the query, the result, whether it was verified, the engine and the time. Admins can export it as CSV from
/admin/compliance/export/csv. - On the pull request. QWED Security leaves its record in the pull request comment, next to the code it checked.
Record-keeping rules
Some rules ask for exactly this kind of record. Article 12 of the EU AI Act, for example, requires high-risk AI systems to record events automatically over their lifetime. A QWED record can be one of the entries you keep about how AI output was checked before it was used. Whether your system meets a given rule is for your own assessment: QWED does not certify compliance with the EU AI Act, SOC 2 or any other framework.
What this is not
- Not tamper-proof. Anyone who can edit a record can also recompute its
proof_ref. A signed attestation, or a copy of theproof_refkept somewhere else, is what shows a record was not changed later. - Not a complete history on its own. A commitment shows a record you hold is intact. It cannot show that no records were deleted. Use append-only storage and retention for that.
- Not a guarded store. The API's request log is a table in your deployment's database. Access control, backups and retention are yours to set.
- Not a record of intent. When a model turns a request into a formal statement, that translation is shown in the record but not verified. Untrusted translator
- Not a certificate. A record shows how one output or action was checked. It is not an audit opinion or a compliance certificate.