Cutting AI agent telemetry costs with OpenTelemetry Collector

Adnan Rahic from Bindplane by Dynatrace published a practical guide on October 22, 2026, detailing how to optimize OpenTelemetry pipelines for AI agents. The article addresses the common production pain point of excessive telemetry data generated by tools like Claude Code, which leads to high storage costs and unmanageable cardinality.

What happened

Turning on telemetry for an AI coding agent often results in a massive volume of data that is expensive to store and difficult to interpret. In a test case where Claude Code fixed two failing tests and added boundary tests, a single run produced 181 telemetry records and 289 KB of JSON. This data includes full prompts, file bodies, and diffs, most of which are unnecessary for ongoing operational monitoring.

The core issue is that default instrumentation is designed for debugging individual developer sessions, not for managing a fleet of agents. At scale, such as with 200 sessions a day, this approach becomes unsustainable. Metrics carry high-cardity attributes like session ID and user email, creating hundreds of unique time series that never expire. Additionally, tool content is duplicated across traces and logs, further inflating storage requirements.

Rahic demonstrates that the goal is not to collect less data arbitrarily, but to keep only what answers specific operational questions. By shaping the data in the OpenTelemetry Collector before it reaches the backend, teams can significantly reduce costs without losing visibility into critical performance and cost metrics.

How it works

The solution involves configuring the OpenTelemetry Collector to fan out every signal into two separate pipelines: a raw pipeline and an optimized pipeline. Both pipelines receive byte-identical input from a single OTLP receiver, ensuring an honest comparison. The raw pipeline acts as a control group, containing only a service name and a batch processor, while the optimized pipeline applies specific filters and transformations.

In the optimized pipeline, processors remove high-cardinality dimensions and redundant content. For example, full file contents from tool outputs are stripped from spans and logs, leaving only essential metadata. Metrics are pre-summed or adjusted to remove per-session identifiers that cause cardinality explosions. This ensures that the data sent to the backend is lean and focused on actionable insights.

The setup uses Grafana LGTM for both pipelines to visualize the difference. Dashboards compare cardinality, bytes per minute, and record counts side by side. In a production environment, the raw pipeline might be directed to object storage for occasional deep debugging, or dropped entirely, while the optimized pipeline feeds the primary monitoring backend.

Key details

• A single Claude Code run generated 181 telemetry records and 289 KB of JSON with all content flags enabled.
• Default metrics include high-cardinality attributes like session.id and user.email, creating 300 series for just eight metrics across 25 sessions.
• Tool content is duplicated in both traces and logs, with file contents stored as span events and log attributes.
• The optimized pipeline reduces metric series from 300 to 12 by removing per-session dimensions.
• Six key questions drive the optimization: cost per model, token usage, tool failure rates, edit rejections, latency, and errors.
• The otelcol_exporter_enqueue_size_bytes metric exists only at the detailed level with the sending queue enabled.

Why it matters

For engineering teams building with AI agents, telemetry costs can quickly spiral out of control if left unmanaged. High cardinality in metrics leads to increased storage and query costs in observability platforms, while large trace and log payloads consume bandwidth and indexing resources. By filtering this data at the collector level, organizations can maintain comprehensive visibility without the prohibitive expense of storing every prompt and file diff.

Moreover, focusing on specific operational questions improves the signal-to-noise ratio for developers and SREs. Instead of sifting through walls of text, teams can monitor clear indicators of agent performance, such as token usage by type, tool success rates, and latency. This shift from raw data collection to targeted observability enables faster troubleshooting and more effective cost management.

What you can do

• Identify the six key questions your team needs answered about AI agent performance, such as cost per model and tool failure rates.
• Configure your OpenTelemetry Collector to create separate raw and optimized pipelines for AI agent telemetry.
• Remove high-cardinality attributes like session IDs and user emails from metrics to prevent time series explosion.
• Strip full file contents and prompts from trace spans and logs, retaining only essential metadata and error information.
• Use pre-summed metrics where possible to reduce the number of data points sent to your backend.
• Visualize the difference between raw and optimized streams to validate storage savings and ensure no critical data is lost.