- Home
- AI agents
- Observability and cost
- Tracing AI agents
Operate · Observability and cost
How do you trace an AI agent?
Tracing an AI agent means recording each run as a tree of timed steps, called spans: one for the agent, one per model call, one per tool call and one per hand-off to another agent, each with its tokens, latency, errors and, by opt-in, its content. The OpenTelemetry GenAI conventions give these spans standard names and attributes.
In this guide: 83 pages
Key takeaways
- Three span types carry most of the value: invoke_agent for the run, chat for each model call and execute_tool for each tool call. A hand-off is a nested invoke_agent span, which is how a multi-agent run stays one trace.
- Frameworks emit the spans for you. Microsoft Agent Framework, LangChain, LangGraph, the OpenAI Agents SDK, Strands Agents and Google’s Agent Development Kit all have instrumentation, and each cloud’s platform collects it once its telemetry is switched on.
- Two switches decide what you see: the content-capture flag (off by default everywhere) and the convention version flag, since the attribute names were still changing on 2 Nov 2026.
- Context propagation is the hard part. MCP carries the trace id in params._meta and A2A has its own attributes, but hosted or provider-managed tools break the chain, so document the gaps.
What does an agent trace hold?
A trace holds one span per step the agent took, nested under a root span for the run, with timing and attributes on each. The OpenTelemetry GenAI conventions define the span names and the attributes, and on 2 Nov 2026 every one of them carried Development status, which means the names can still change.
Span types in the OpenTelemetry GenAI conventions, as the repository’s main branch showed on 2 Nov 2026.
| Span | Emitted when | Attributes worth knowing | Notes |
|---|---|---|---|
| invoke_agent {agent name} | An agent run starts; nested for each sub-agent | gen_ai.agent.name, id and version; gen_ai.conversation.id; token totals | CLIENT kind when you call a remote agent, INTERNAL when it runs in your process |
| chat {model} (and other inference operations) | Each model call | gen_ai.provider.name, gen_ai.request.model, gen_ai.usage.input_tokens and output_tokens, error.type; content by opt-in | The older name gen_ai.system is replaced by gen_ai.provider.name in the current conventions |
| execute_tool {tool name} | Each tool call the agent makes | gen_ai.tool.name, call id; gen_ai.tool.call.arguments and result by opt-in | Arguments are where a prompt injection or an over-broad permission becomes visible |
| invoke_workflow, plan | A workflow runs; a planning step runs | Workflow and plan identifiers | Added with the multi-agent conventions Microsoft says it contributed with Cisco Outshift |
| create_memory, search_memory and other memory operations | The agent writes or reads long-term memory | Memory operation names | Lets you see what the agent remembered and when |
| {mcp.method.name} {target}, for example tools/call get-weather | An MCP client or server handles a request | mcp.method.name (required), mcp.session.id, mcp.protocol.version, jsonrpc.request.id | CLIENT and SERVER kinds; duration metrics for operations and sessions |
| A2A request spans | One agent calls another over A2A | a2a.method.name, a2a.task.id, a2a.task.state; the A2A context id maps to gen_ai.conversation.id | Separate from the agent spans; Development status, like the rest |
Metrics come with the spans: gen_ai.client.operation.duration, token counters for input, output, cache-read and reasoning tokens, and gen_ai.invoke_agent.duration, inference_calls and tool_calls. The last two are the earliest signal of a loop.
How do you instrument an agent, and which switches matter?
You rarely write spans by hand. The framework emits them, the platform collects them, and your job is to set two switches: whether content is captured and which version of the conventions the SDK speaks.
Microsoft Agent Framework emits invoke_agent, chat and execute_tool spans plus the metrics gen_ai.client.operation.duration, gen_ai.client.token.usage and agent_framework.function.invocation.duration. Sensitive data is off by default; the Python switch is ENABLE_SENSITIVE_DATA and the .NET switch is EnableSensitiveData, and the guidance is to enable it only in development or testing environments. When no source name is set, the .NET default source is Experimental.Microsoft.Agents.AI. Foundry documents the same collection path for LangChain, LangGraph and the OpenAI Agents SDK. On AWS, CloudWatch generative AI observability works with Strands Agents, LangChain and LangGraph. On Google, Agent Runtime instruments the Agent Development Kit. MLflow Tracing adds one-line auto-tracing for OpenAI, LangChain, LlamaIndex, DSPy and others and natively supports the GenAI conventions.
The content switch in the spec is OTEL_INSTRUMENTATION_GENAI_CAPTURE_CONTENT with the values NO_CONTENT (the default), SPAN_ONLY, EVENT_ONLY and SPAN_AND_EVENT. Google Agent Runtime and the Agent Framework .NET harness use a different name, OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, and the accepted values differ by SDK and version. Name the SDK whenever you quote a variable in a runbook, because the two names look alike and behave differently. For large content the spec defines hooks that upload the payload to external storage and leave a reference attribute (gen_ai.input.messages_ref) on the span.
The version switch is OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental. Many SDKs still read it to move from the older attribute names (gen_ai.system) to the current ones (gen_ai.provider.name). Datadog ingests GenAI traces at semantic conventions 1.37 and later and asks frameworks that default to older versions to set it. Pin it, write it down and test each dashboard after an SDK upgrade, because a query keyed on the old name goes blank without an error.
One warning from the Agent Framework docs: instrumenting both the chat client and the agent records the same content twice. Instrument once, at the agent, and let the model-call spans nest under it.
How does the trace cross tool servers and other agents?
The trace crosses a boundary when the caller passes its trace context and the callee continues it, and each protocol has a place for that context. Where the provider makes the call for you, the context is not passed, and the trace shows a gap.
For MCP, the client puts the W3C headers traceparent, tracestate and baggage inside params._meta of the JSON-RPC request. The server reads them and opens a child span, so a tool call made through an MCP server appears under the agent’s span in your backend even though the server runs elsewhere. Agent Framework does this automatically for MCP connections the agent opens itself. It does not for hosted or provider-managed MCP tools, where the provider’s service makes the call, so those steps are missing from your trace and the run looks shorter and cheaper than it was.
For agent-to-agent calls over A2A, the conventions define request spans with a2a.method.name, a2a.task.id and a2a.task.state and map the A2A context id onto gen_ai.conversation.id, so a conversation that moves between two agents can be joined on that id; the remote agent’s own invoke_agent span needs its own instrumentation. A hand-off inside one framework is simpler: it is a nested invoke_agent span under the caller.
Write the gaps into the runbook: which tools are hosted, which agents are external, and what each gap hides. Bedrock model invocation logging has a similar blind spot on the data side: it captures the bedrock-runtime endpoint and not the bedrock-mantle endpoint used by the Responses and Chat Completions APIs.
How do you switch tracing on in each cloud?
In each cloud, tracing is a configuration step rather than a coding step for agents built on the platform. The steps below are what the vendor pages described on 2 Nov 2026.
- Microsoft Foundry: connect the project to an Application Insights resource. Server-side tracing then turns on automatically, with no code change, and the portal shows traces for the past 90 days with conversation-level results. Tracing was generally available for prompt and hosted agents; workflow and external agents were in preview. Agents running elsewhere can be registered through the Foundry Control Plane AI Gateway and send OpenTelemetry GenAI telemetry to the same Application Insights resource, where the Agents view (preview) lists them next to Foundry and Copilot Studio agents.
- Amazon Bedrock AgentCore: AgentCore Observability stores metrics, spans and logs in CloudWatch and records session count, latency, duration, token usage and error rates, with CloudWatch generative AI observability covering Runtime, Built-in Tools, Gateways, Memory and Identity (both GA 13 Oct 2025). AWS says there is no extra charge beyond standard CloudWatch pricing. CloudTrail covers Gateway and Agent Registry calls. For model calls outside AgentCore, Bedrock model invocation logging (disabled by default) writes full requests and responses to CloudWatch Logs or S3, with bodies over 100 KB going to S3.
- Google Agent Runtime (the former Vertex AI Agent Engine): enable the Telemetry API and set GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY, which turns on traces, logs and metrics in Cloud Trace without prompt or response data. OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT adds prompts, responses and user ids, and Google says to have end-user consents, notices and data handling policies in place first. The page describes its span format as experimental and subject to change.
- Independent backends: Datadog, Langfuse, Arize Phoenix, MLflow and Dynatrace accept or recommend OpenTelemetry GenAI spans, Datadog also takes OpenInference and OpenLLMetry, and LangSmith, W&B Weave and New Relic have their own SDKs. Point the exporter at the backend and apply the same access control and retention you use for logs.
What do you do with the traces once you have them?
Four things: debug a run, alert on the metrics, feed the evaluation loop and answer an auditor. The third is where traces pay for themselves, because a failed production run is a regression test nobody had to invent.
Debugging starts in the tree: the Application Insights Agents view lets you sort traces by most tokens used and open a simple view of the end-to-end transaction, and LangSmith and Langfuse show the same tree with the prompts inline where content capture is on. Alerting uses the agent metrics (tool calls, inference calls, duration per run) and error.type, the signals a model cannot fake.
Evaluation is the third use. AgentCore Evaluations converts Strands and LangGraph traces, in OpenTelemetry or OpenInference format, to a unified format and scores them with a model as judge. Foundry continuous evaluation samples live runs as they complete, with Random or Intelligent sampling and an hourly cap that defaults to 100 runs. DKB’s banking assistant, publicly reported by Langfuse on 31 Aug 2026, traces every interaction and runs a second instance that screens each answer for compliance, with flagged sessions routed to human annotation queues and curated into regression tests before each release.
Audit is the fourth. The CoSAI AI Incident Response Framework V1.0 (27 Oct 2025) recommends logging prompt and response pairs, tool execution traces, system prompts and model versions so incidents can be reconstructed. A trace with content capture under a privileged role is that record. The Evaluation and testing topic covers how traces become test cases.
How do Microsoft, AWS and Google implement it?
How the three clouds collect agent traces, with statuses as the vendor pages showed on 2 Nov 2026.
Capability, Microsoft, AWS, Google
| Capability | Microsoft | AWS | |
|---|---|---|---|
| Tracing product and status | Foundry tracing to Application Insights; GA for prompt and hosted agents, workflow and external agents preview | AgentCore Observability and CloudWatch generative AI observability, GA 13 Oct 2025 | Agent Runtime tracing to Cloud Trace; span format marked experimental |
| Frameworks instrumented | Agent Framework, LangChain, LangGraph, OpenAI Agents SDK | Strands Agents, LangChain, LangGraph | Agent Development Kit on Agent Runtime |
| Content switch | ENABLE_SENSITIVE_DATA (Python), EnableSensitiveData (.NET), default off | Model invocation logging, disabled by default | OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, with consents and notices first |
| Retention and access | Portal traces 90 days; Application Insights retention settings; content in the protected AppGenAIContent table | CloudWatch log group and S3 bucket retention and policies | Cloud Trace and Cloud Logging retention and IAM |
Google renamed Vertex AI Agent Engine to Agent Runtime on 22 Apr 2026, so older tutorials use the old name for the same tracing setup.
Frequently asked questions
What is the difference between a trace, a span and a log for an agent?
A span is one timed step with attributes: a model call, a tool call, an agent run. A trace is the tree of spans for one run, joined by a trace id. A log is a line of text with a timestamp. Agent debugging needs the tree, because the failure is usually in the relationship between steps.
Do we need OpenTelemetry if the platform traces our agents automatically?
The platforms use OpenTelemetry underneath, so you are already on it. You only touch it directly when an agent runs outside the platform, when you add a second backend, or when you set the content and convention-version switches. Knowing the span names helps you read any of the dashboards.
How do we trace a tool call that goes through a hosted MCP server?
You often cannot, end to end. When the provider makes the MCP call on your behalf, your trace context is not forwarded and the step shows as a gap. Record the gap in the runbook, log the tool call on your side with the request id, and prefer MCP connections the agent opens itself where the trace matters.
How long should we keep agent traces?
Keep metadata traces as long as your incident and audit processes need, often the same period as application logs. Keep content traces only as long as a named process needs them, behind a privileged role, with a deletion date. Foundry shows 90 days in the portal; longer retention is an Application Insights setting.
More in Observability and cost
Sources
Each source was read on 2 November 2026. The date shown is the date on the page, or the page’s ms.date for Microsoft Learn; an undated page shows the date it was read.
- Semantic conventions for GenAI agent spans
- Semantic conventions for GenAI client spans and content capture
- Semantic conventions for GenAI metrics
- Semantic conventions for Model Context Protocol
- Semantic conventions for A2A
- Tracing agents: concepts
- Set up tracing for agents in Microsoft Foundry
- Observability in Microsoft Agent Framework
- Agents view in Application Insights
- Agent Monitoring Dashboard in Microsoft Foundry
- Observability for Amazon Bedrock AgentCore
- Amazon CloudWatch generative AI observability is now generally available
- Amazon Bedrock model invocation logging
- Evaluations for Amazon Bedrock AgentCore
- Tracing for Agent Runtime
- Gemini Enterprise Agent Platform release notes
- OpenTelemetry instrumentation for LLM Observability
- MLflow Tracing for GenAI
- How DKB runs its AI banking assistant on Langfuse
- AI Incident Response Framework V1.0
Planning an AI program?
Talk to the team that builds it and runs it after launch.
- Built to SOC 2, HIPAA and GDPR standards
- EU AI Act-aligned delivery
- Microsoft Solutions Partner