Your Agent Needs a Flight Recorder

An AI agent receives a support request, checks an account, reads a policy, calls a billing tool, and reports that the refund is complete.

The customer still has no refund.

What happened?

Maybe the agent chose the wrong account. Maybe it found an old policy. Maybe the billing tool rejected the request. Maybe the tool succeeded but the agent misread the result. Maybe the agent retried and created two operations, then summarized only one.

The final message cannot answer those questions. It is the last frame of a longer sequence.

Once an agent can act across tools and systems, it needs something like a flight recorder: a durable account of what the run saw, what it attempted, what happened, and where control changed hands.

In software, we usually call that account a trace.

A good trace is not a transcript of every internal detail. It is structured evidence that lets an operator reconstruct a run.

Logs Tell You Events. Traces Tell You the Path.

Traditional logs are often lines written by individual components:

• request received
• model call started
• tool returned 403
• retry scheduled
• run completed

Those events matter. The hard part is connecting them.

A trace groups the work under one run and breaks it into timed steps, often called spans. A top-level span might represent the agent run. Child spans might represent model calls, retrieval, tool execution, approval waits, retries, and handoffs to another agent.

That structure answers questions a pile of logs cannot answer easily:

• Which user request started this action?
• Which version of the agent and model handled it?
• What source did the agent retrieve before choosing a tool?
• Which tool arguments were sent?
• Did the tool fail, time out, or return a result the agent misunderstood?
• Was an approval requested and granted?
• Did a retry repeat an external action?
• Which step consumed the time and cost?

OpenTelemetry’s evolving generative-AI semantic conventions provide a vendor-neutral direction for recording model operations, token usage, and—when explicitly enabled—content and tool details. The important idea is not one particular backend. It is giving the run a consistent shape.

The Minimum Useful Record

An agent trace should start with identity and causality.

At minimum, record:

• a run ID and trace ID
• the task or request that started the run
• the agent, workflow, prompt/configuration, and model versions
• timestamps, duration, and outcome for each step
• the tools and sources used
• approval, policy, and permission decisions
• retry, timeout, cancellation, and recovery events
• token, cost, and resource summaries where available
• links to resulting artifacts, changes, or external transaction IDs

The exact fields depend on the job. A coding agent needs the repository, commit or worktree, commands, test results, and diff. A research agent needs source URLs, retrieval timestamps, and citations. A communication agent needs the target, draft, approval state, send result, and message ID.

This extends the idea in What It Means for AI to Have a Job: if the role defines tools, authority, failure behavior, and a stop condition, the trace should show how the run moved through those boundaries.

Record Decisions Without Pretending to Read the Model’s Mind

Teams sometimes say they want to log the agent’s “reasoning.” That phrase needs care.

A model’s hidden internal process is not a reliable audit record. A generated explanation can be useful, but it may be incomplete, post-hoc, or shaped for readability. Saving every hidden reasoning token is also unnecessary for most diagnosis and can create privacy and security problems.

The more dependable record is observable evidence:

• the instruction and relevant context presented to the model
• the tool options available at that moment
• the structured action it selected
• the policy or approval result
• the tool response
• the next observable action

If the system asks the model to produce a short decision note—“selected refund tool because policy version 12 permits refunds under $50”—store it as a model-generated note, not as ground truth about cognition.

The trace should distinguish facts, model output, and operator decisions.

Content Is Useful—and Dangerous

The easiest trace to debug is often the most invasive one.

Full prompts can contain personal data, private documents, source code, credentials pasted by mistake, or confidential business context. Tool arguments can include email addresses, account numbers, file paths, and message bodies. Tool results may be even more sensitive.

This creates a second system to secure. Data that was briefly available during one run can become searchable, exportable, and long-lived in an observability backend.

Good instrumentation starts with metadata and adds content deliberately.

Useful controls include:

1. Allowlist fields instead of collecting everything. Record tool name, status, duration, and a safe operation ID before recording full arguments.
2. Redact before export. Remove secrets, tokens, personal identifiers, and sensitive values as close to the source as possible. OpenTelemetry documents a Collector redaction processor for scrubbing attributes before a backend receives them.
3. Separate content from trace structure. Keep the durable span tree while storing sensitive payloads in a more restricted system—or not at all.
4. Use role-based access. Someone who can see latency charts does not automatically need to read customer messages.
5. Respect user and legal boundaries. Consent, data residency, deletion rights, and regulated-data rules apply to telemetry too.

Redaction is not a magic eraser. A field can reveal sensitive information through context even after obvious patterns are removed. Test redaction with realistic data, not only clean examples.

Retention Should Match the Question

Keeping every detailed trace forever feels safe because future debugging is unknown. It usually creates cost and risk without proportional value.

Different evidence deserves different lifetimes:

• aggregate metrics can remain useful for long-term trends
• routine successful traces may need only a short window or representative sample
• errors and policy violations may need longer retention for investigation
• approval records and external transaction IDs may follow business or compliance requirements
• raw prompts and tool payloads should usually have the shortest, most restricted retention

The policy should answer: who can see this data, why it is kept, when it is deleted, and whether deletion propagates to exports and backups.

Sampling can control volume. OpenTelemetry describes sampling as choosing representative traces for processing and export. But purely random sampling can discard the one failure you needed. A practical design may keep all errors and safety events, sample routine successes, and preserve aggregate metrics across the full population.

Sampling is a cost decision. Retention is a lifecycle decision. Redaction is a data-minimization decision. They solve different problems.

Replay Is Not Time Travel

A trace can support replay, but the word is easy to overpromise.

There are at least three kinds of replay:

• visual replay: reconstruct the recorded timeline for a human reviewer
• mock replay: feed saved tool responses through the workflow to reproduce later steps without touching live systems
• live replay: run the task again against current models, tools, and data

Visual replay can faithfully reproduce the recorded timeline. Mock replay can be repeatable only when model outputs and other nondeterministic inputs are recorded or mocked along with tool responses. Live replay often is not deterministic.

Models can produce different outputs. External data changes. Tools and policies evolve. Time passes. Randomness, concurrency, caches, and network behavior shift. A live replay can also repeat a payment, message, deletion, or deployment.

Safe replay therefore needs boundaries:

• default to read-only or mocked tools
• mark recorded responses versus new responses
• use idempotency keys where external actions support them
• require approval before consequential live calls
• pin model, prompt, tool, and policy versions when comparison matters
• label a replay as a new run linked to the original

A replay is evidence and an experiment. It is not proof that the original run would behave identically today.

A Trace Is Not an Evaluation

Observability tells you what happened. Evaluation tells you whether it was good.

A trace can show that an agent called the right API in 300 milliseconds. It cannot decide that the customer received a fair outcome. It can show that citations were retrieved. It cannot guarantee the answer used them correctly.

The two systems should connect. A failed evaluation should link to the trace that produced the output. A production trace can become a regression case after sensitive data is removed. That closes the gap between judging AI by results rather than vibes and diagnosing why the result changed.

Metrics such as latency, tokens, retries, error rates, approval frequency, and tool failures are useful. They are not substitutes for task-specific success criteria.

Build the Recorder Before the Crash

The worst time to decide what evidence matters is after an agent has taken the wrong action.

Before increasing an agent’s authority, walk through one failure and ask:

• Could we identify the exact run?
• Could we reconstruct the sequence across model and tool calls?
• Could we see which source, configuration, and policy versions were active?
• Could we separate a model mistake from a tool or permission failure?
• Could we find the external transaction and reverse it if appropriate?
• Would the trace expose more private data than the incident itself?
• Would the necessary evidence still exist when the problem is discovered?

Sandboxes limit where an agent can act. MCP standardizes how applications connect to tools and data. A flight recorder addresses the next question: when a run crosses those boundaries, can we explain what actually happened?

Useful agents will fail in ordinary, complicated ways. The goal is not to record everything.

The goal is to preserve enough trustworthy evidence to diagnose the failure—without creating a second, permanent copy of everything the agent touched.