How do I audit what an AI agent saw and did?

To audit what an AI agent saw and did, keep one record per turn that links five things: what the agent read, what was placed in its context window, which tools it called with which arguments, who approved each action, and what it wrote back. Model provider logs and chat transcripts cover only part of that, and the OpenTelemetry GenAI semantic conventions leave message content off by default. Write the record to append-only storage, hash-chain and sign it, and keep it out of reach of the agent and the people who run it day to day.

What should you log on every turn?

Log five event types per turn, each with session, turn, agent and user identifiers and a timestamp.

Event	What to record	Example source
Read	Every source retrieved: ID, query, score, version	OpenTelemetry retrieval span, opt-in gen_ai.retrieval.documents
Placement	What entered the context window, including rules files	Opt-in gen_ai.input.messages; Claude Code InstructionsLoaded hook
Tool call	Tool name, arguments, result, success or failure	OpenTelemetry execute_tool span; Claude Code PreToolUse, PostToolUse and PostToolUseFailure hooks
Approval	Who approved or denied which action, and when	Claude Code PermissionRequest hook; your approval system
Write-back	What was saved to memory or another system afterwards	OpenTelemetry memory operations such as create_memory; your own write path

Record what was dropped as well as what was kept: an investigation often asks why the agent did not see a fact.

Why are model provider logs and transcripts not enough?

Model-side logs show the final prompt and response, not where each piece of context came from, why it was chosen, what was left out, or who approved the action that followed. Harness logs have gaps too. The Claude Code hooks reference notes that PreToolUse "runs only when Claude calls a tool", and that files referenced with @ in a prompt are added "without any tool call", so a tool-only log misses them. The same reference says the transcript file "is written asynchronously and may lag the in-memory conversation". The OpenTelemetry GenAI conventions, still at Development status, say instrumentations "SHOULD NOT capture" instructions, inputs and outputs by default. For production they recommend storing content externally and recording references on spans: "Using external storage enables separate access controls."

How do you make agent logs tamper-evident?

A tamper-evident log does not stop an edit; it makes any edit detectable. OWASP LLM08 already asks for "detailed immutable logs of retrieval activities". Five steps:

1. Write each event once to append-only storage. Give the agent, the harness and everyday admin accounts no update or delete rights.
2. Chain entries: each entry carries the hash of the previous one, so a removed or edited entry breaks the chain.
3. Sign entries, or periodic checkpoints of the chain, with a key the agent cannot reach, and store checkpoints in a second location.
4. Verify the chain on a schedule and before every investigation.
5. Keep sensitive content in a separate store with its own access and retention, and log IDs and hashes in the main trail.

Certificate Transparency (RFC 9162) uses this pattern at Internet scale: its logs are append-only Merkle trees that "do not themselves prevent misissuance" but let interested parties "detect such misissuance".

How do you capture the record in practice?

Capture events where they happen: in the harness for tool calls and approvals, and inside the turn for reads and placement. Claude Code hooks run your command on lifecycle events and pass JSON such as session_id, prompt_id and tool_input on stdin. This setting appends every successful tool call to a file that you then ship to append-only storage:

{"hooks": {"PostToolUse": [{"matcher": "*", "hooks": [{"type": "command", "command": "jq -c . >> /var/log/agent-audit/claude-code.jsonl"}]}]}}

Put audit hooks in managed policy settings: the reference says disableAllHooks "can't disable managed hooks from outside managed settings". Reads and placement need a component inside the turn. elephant.broker, a pluggable runtime for AI agents, works at that point, and the elephant.broker website states that every read, placement, write-back and approval is recorded against the actions taken, in a signed, append-only audit log. Reconcile it with your harness logs, and see what security teams should check for the wider review.

Sources

• elephant.broker: features, security and deployment
• Anthropic: Claude Code hooks reference
• OpenTelemetry: GenAI semantic conventions (now maintained in the semantic-conventions-genai repository), model spans, agent spans
• OWASP: LLM08:2025 Vector and Embedding Weaknesses
• IETF: RFC 9162, Certificate Transparency Version 2.0

Product details come from each vendor's public documentation as viewed on 2026-10-28 and may have changed. Product names are trademarks of their respective owners; elephant.broker is not affiliated with them.