Best Tools for Coding Agent Observability

Your team runs Claude Code and Codex every day, the bill arrives at the end of the month, and nobody can say which sessions produced it. The products sold as agent observability answer that question differently because they read different sources. One reads what the agent chooses to export, one reads what you instrument, and one reads the transcript the agent leaves on disk. Pick by the source, because the source decides what you will be able to see.

A coding agent session gets recorded in three places

A coding agent run produces prompts, tool calls, file reads and edits, model requests with token counts, compaction boundaries, and a final result. No single collection path carries all of that.

The agent's own telemetry is the first path. Claude Code ships an OpenTelemetry export that emits counters and structured events, with prompt and tool content left out unless you turn it on. The second path is a tracing platform you already run, which stores whatever spans reach it over OTLP or an SDK. The third path is the session transcript the agent writes to your filesystem, which is the richest of the three because it is what the agent reads back to itself, within the limits of what that agent chose to record.

OpenTelemetry's GenAI semantic conventions give these paths a shared vocabulary. The conventions moved into their own repository, separate from the core semantic conventions, and they carry Development status rather than stable. They define spans for creating and invoking an agent, running a workflow, planning, executing a tool, loading a skill, and running a command, along with attributes such as gen_ai.tool.name and gen_ai.usage.cache_read.input_tokens, plus a separate set of conventions for Model Context Protocol activity. A shared vocabulary is useful, and it still does not decide which of your sessions is worth looking at.

What Claude Code exports on its own

Claude Code has a built-in OpenTelemetry export, documented by Anthropic, that you enable with CLAUDE_CODE_ENABLE_TELEMETRY=1 plus a metrics or logs exporter and an endpoint. Metrics cover claude_code.session.count, token.usage, cost.usage in USD, lines_of_code.count, commit.count, pull_request.count, code_edit_tool.decision, and active_time.total. Metrics export every 60 seconds with delta temporality, and logs every 5 seconds.

The events carry more of the shape of a run than the counters do. claude_code.api_request records model, duration, cost, and the input, output, cache-read and cache-creation token counts for each call. claude_code.tool_result records the tool name, whether it succeeded, how long it took, and the size of its input and output in bytes. claude_code.compaction records duration with approximate token counts before and after. claude_code.subagent_completed records the subagent type, how many tool calls it made, its run time, and whether the model changed mid-run. There are also events for skill activation, hook execution, MCP server connections, permission-mode changes, and API errors.

Content is withheld by default, which matters when your prompts contain your source code. Prompt text is controlled by OTEL_LOG_USER_PROMPTS. Assistant responses follow OTEL_LOG_ASSISTANT_RESPONSES, which inherits the prompt setting when it is unset, so turning prompt logging on also exports the model's responses and any code in them until you set OTEL_LOG_ASSISTANT_RESPONSES=0. Tool parameters and Bash commands require OTEL_LOG_TOOL_DETAILS, tool output requires OTEL_LOG_TOOL_CONTENT, and full request and response bodies require OTEL_LOG_RAW_API_BODIES. Content-bearing attributes are truncated at 60 KB. Distributed tracing is beta and needs CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 on top of telemetry being enabled.

Redacting content does not make the stream anonymous. Standard attributes ride on every event, and they carry the signed-in user's identity: user.email when the session authenticated through OAuth, plus user.account_uuid, user.account_id, organization.id, an installation-scoped user.id, and the per-session session.id. Anthropic's documentation notes that these go only to the endpoint you configure and never to Anthropic, and it suggests filtering or redacting the email field at your telemetry backend if that matters to your organization.

For a fleet, administrators put the same keys in the env block of managed settings. Claude Code ignores OTLP exporter variables set in a repository's .claude/settings.json and .claude/settings.local.json, so a cloned repository cannot switch telemetry on or redirect it somewhere else. When managed settings specify an endpoint, protocol, or credential, Claude Code drops conflicting developer-set variables at startup.

What you give up is the body of the work. A tool-result event tells you that Read ran, succeeded, and returned 48 KB. It does not tell you which file, unless you enable tool details and accept the file paths and commands that come with them. Anthropic publishes no separate price for the export, and you still operate the collector and backend that receive it.

Tracing platforms: it depends on what the integration captures

Langfuse, LangSmith, Helicone, Braintrust, Datadog, and Sentry were built to trace LLM calls inside an application you wrote and instrumented. Teams now point coding-agent telemetry at them so agent runs land beside product traces. The mechanism is span ingestion, over OTLP or through an SDK, followed by the platform's existing trace, cost, and evaluation views.

Three of these publish enough in their own repositories to characterize fairly. Langfuse is MIT licensed apart from its ee folders, ingests OTLP, and self-hosts through Docker Compose, a Helm chart, or Terraform on AWS, Azure, and GCP. Worth knowing before a security review: a self-hosted Langfuse reports basic usage statistics to a central server by default, and its documentation states that those statistics exclude raw traces, prompts, observations, and scores. Helicone is Apache 2.0, works as a gateway in front of model providers as well as an async logger, and self-hosts with Docker Compose. Arize Phoenix builds on OpenTelemetry and its OpenInference instrumentation, self-hosts through Docker or Helm, and ships under the Elastic License 2.0 rather than an OSI-approved license. Check the current documentation for LangSmith, Braintrust, Datadog, and Sentry before assuming any of them ingests Claude Code's stream the same way.

The limit to watch for is the bare model-call span. On its own it says a call consumed 180,000 input tokens and cost $0.54, not which repository, which pull request, or which five files the agent read three times. Agent-aware integrations close some of that gap without work on your side. Langfuse documents a Claude Code integration built on a Stop hook that reads the generated conversation transcripts and captures tool inputs and outputs. Claude Code can also tag its own metrics with repository attributes through the opt-in OTEL_METRICS_INCLUDE_REPOSITORY. So the question is not whether a platform can see repository work, but what the specific integration captures and where it sends that content.

Session logs keep what the exporters drop

A transcript preserves the conversation the agent recorded, and reading it needs no collector, no admin console, and no change to how anyone works. It is not an automatic guarantee of completeness. Persistence can be off, since Claude Code's --no-session-persistence flag and the CLAUDE_CODE_SKIP_PROMPT_HISTORY environment variable both stop sessions being saved to disk. What gets recorded also varies by agent, and a reader cannot recover what the agent never wrote.

Decant is Dosu's open source reader for those files, Apache 2.0, covering Claude Code, Codex, Gemini CLI, and OpenCode from ~/.claude/projects, ~/.codex, ~/.gemini/tmp, and ~/.local/share/opencode. It runs through npx @dosu/decant@latest, Homebrew, or Docker with read-only mounts, and makes no outbound network calls at runtime.

Because it reads the record rather than a counter, it can compute things an exported metric cannot. decant economics splits generation, tool calls, estimated cost, and active time into four buckets: context, planning, code, and communicating. The shell classification behind that split is deliberately conservative, so rg, cat, and git diff count as context while a mutating or unrecognized command counts as code, judged across every part of a compound command rather than its first word. Its analysis also marks the orientation boundary at the first detected file edit, which is how the orienting share below is derived, though that split is not part of what the economics command prints. Context-window occupancy is computed per call as input plus cache-read plus cache-creation tokens, and compactions come from the provider's own boundary records. decant files --group ext, decant tool stats, and decant mcp stats show where the work landed, and full-text search covers prompts and tool arguments.

Dosu used exactly this reading to measure its own agent spend. Replaying 60 pull requests from a public repository as Claude Code and Codex sessions, 46% of the estimated dollars went to orienting, meaning everything before the first line of code, at 43% for Claude Code and 52% for Codex. Costs there are modeled at standard API rates, and the study measured spend rather than whether either agent did the job well.

What a transcript holds varies by agent, and Decant's own methodology is explicit about the gaps. OpenCode 2.x counts title generation that never appears in a message row, so Decant's per-session totals can read slightly lower than OpenCode's, and an inner call inside a code-mode program has no result, output size, or duration of its own because OpenCode records only the program's combined output. Read the per-agent notes before comparing two agents on one number.

Two further tradeoffs come with reading transcripts. Decant keeps its own copy in an unencrypted SQLite archive at ~/.decant/decant.db, holding prompts, tool inputs, tool output, and whatever source code the agent read. The file is created owner-only, and the project's documentation says plainly that your retention policy for agent transcripts applies to it. Its Docker instructions keep the port bound to 127.0.0.1 for the same reason, since publishing it exposes an unauthenticated archive API. The second tradeoff is reach. A local archive covers one machine, so an organization-wide view needs a separate way to gather sessions.

Pick the category by the question you are asking

Three questions map cleanly onto the three sources.

If you need cost and usage across the team, use the native export. Managed settings roll it out centrally and pin the destination, and a counter stream is cheap to retain for a year. If you need to understand one expensive or failed session, read its transcript, because the counters were never going to tell you which file the agent reopened six times. If you already run a tracing platform for product LLM calls, route the agent's stream into it, and check whether it offers an agent-aware integration rather than assuming you get bare spans. Turning on OTEL_METRICS_INCLUDE_REPOSITORY costs nothing and tags the metrics you already export.

Most teams end up with the first and the second. Counters answer how much, transcripts answer why, and the two are collected by different people for different reasons.

Data residency decides the rest, in two steps rather than one. Claude Code's defaults keep prompts, responses, tool inputs, and tool output out of the export, so your source code stays put until you enable a content flag, and each flag you enable sends more of that code to whoever receives the telemetry. The counters themselves still identify who ran the session, down to the email address on an OAuth login, so the destination needs to be one approved for employee identity data even when no content flag is on. A transcript reader keeps both the content and the identity on the developer's own disk instead. Decide which of those you can defend before you decide which product you like.

FAQ

Does Claude Code send my prompts to my observability backend?

Not by default. With telemetry enabled, prompt text is redacted unless you set OTEL_LOG_USER_PROMPTS. Assistant responses are the catch: OTEL_LOG_ASSISTANT_RESPONSES inherits that prompt setting when unset, so enabling prompt logging exports responses too unless you set it to 0. Tool inputs and tool output have their own flags. The identity attributes are a separate question from content, since every event carries the signed-in user's account and, on an OAuth login, their email. Local session transcripts are a separate surface again, written unless persistence is turned off, so check all three when you are working out what leaves the machine.

What is the difference between OTel events and reading transcript files?

The events are a stream Claude Code emits while it runs, shaped for counters and dashboards, with content redacted unless you opt in. The transcript is the conversation the agent saved for itself, including tool arguments and output. A transcript reader can reconstruct a session after the fact without any exporter configured, while the event stream gives you the fleet-wide rollup a local file cannot. Neither is complete on its own: persistence can be turned off, and what a transcript records varies by agent.

Can I get an organization-wide rollup without a SaaS vendor?

Yes. Deploy the Claude Code telemetry variables through managed settings and point OTLP at a collector and backend you run. You own the pipeline and the storage, and you decide whether any content flags are on.

Does this work the same way for Codex or Cursor?

For Codex, yes. Codex, Gemini CLI, and OpenCode all write local session logs that Decant parses, so the transcript route covers Codex today. Cursor is not in that set, and using it this way would need a separately verified integration. The export route is written here against Claude Code's documented event names, which are specific to Claude Code, and each vendor runs its own telemetry and admin reporting on its own schedule. Check the current administrative documentation for every agent you run before planning a rollout.

Look at what your own context bucket holds

Start with whichever source answers the question in front of you. If that is where your sessions spend their budget, run decant economics over your indexed sessions. It takes no arguments and reports tokens, estimated cost, and active time split across context, planning, code, and communicating for the whole archive, which gives you your own repository's split rather than someone else's average.

Then read what sits inside that context bucket. It covers every read, search, and retrieval a task needed, so a large share is not by itself a finding. What is worth looking for is repetition: the same files opened across sessions, the same conventions or decisions worked out again. decant files --group path and the full-text search over tool arguments are how you check. Connect a repository to Dosu to make that knowledge available to the agent, then compare matched tasks before and after to measure whether anything changed.