What New Relic Shipped on 6 October

New Relic rolled three new capabilities into its AI monitoring surface on 6 October 2026, all announced as part of a broader October round-up. They sit under a single umbrella the company calls AI Observability.

AI Evaluation adds response-quality scoring in the APM UI. The idea: rather than sampling outputs post-hoc in a separate notebook, evaluation results land as trace metadata alongside latency and error rate. New Relic has not published which scoring dimensions ship out of the box, but the announcement describes quality, relevance, and performance as the three axes.

AI Agent Monitoring tracks tool calls, handoffs between sub-agents, and workflow execution across multi-step agent runs. Each tool call is a child span of the agent span, following the OpenTelemetry GenAI semantic conventions: gen_ai.agent.name, gen_ai.tool.name, and gen_ai.tool.status are standard attributes, distributed with W3C traceparent headers across service boundaries.

AI Observability for OpenTelemetry is the plumbing that connects the two features above to teams not using New Relic’s native APM agents. You route your existing OTel trace stream to New Relic’s OTLP endpoint, and a TraceProcessor at the ingestion layer scans for any span with a gen_ai.* attribute. When it finds one, it tags the source service with aiEnabledApp:true in New Relic’s entity catalog — no manual configuration, no custom instrumentation. This is the Shadow AI discovery mechanism: unregistered services that are quietly calling an LLM appear in the AI Monitoring UI automatically.

How the Architecture Works Under the Hood

The key design decision is what New Relic calls Normalize on Read. OTel spans arrive and are stored in standard Span tables without ingestion-time translation. At query time, the UI detects whether a span came from a native APM agent or an OTel source and adapts its NRQL queries accordingly. The practical effect: you can run Model Comparison views side-by-side across APM-instrumented services and OTel-instrumented services without a data migration.

For evaluation, the architecture leans on stateless processing. Each OTel span contains both the prompt (gen_ai.input.messages) and the response (gen_ai.output.messages) in a single payload. An OtlpEvaluationHandler can score a span without Redis caching or session reassembly — which matters for teams that run high-throughput pipelines where stateful session trackers become a bottleneck.

Safety evaluation runs in a parallel stream: trace data is routed through ai_evaluation pipelines for PII masking, real-time prompt-injection detection, and toxicity screening. This is described as live-traffic analysis, not a batch job run on a sample. New Relic states the deployment requires zero custom code rewrites and no New Relic SDK lock-in for teams already on standard OTel instrumentation.

The Model Inventories view aggregates token consumption, request latency, and error rates account-wide, combining APM and OTel data into a single cost dashboard. It is the observability equivalent of a cloud spend console: one place to see which services are calling which models and what it is costing.

How It Compares: Six AI Observability Tools in One Table

Five standalone platforms compete in the same space. Pricing figures below are as of October 2026 from each vendor’s published rates; New Relic’s AI-specific compute pricing is unavailable without a sales conversation.

Pricing sourced from each vendor’s published rates as of September–October 2026. Arize Phoenix self-hosted is free; cloud Arize AX pricing is per Dupple’s October 2026 roundup. New Relic AI compute pricing requires a sales conversation per published rate sheet.

The split in the market is cleaner than the vendor marketing implies. Standalone eval tools (Langfuse, Arize Phoenix, LangSmith, Braintrust) exist to help you improve your AI application quality. APM-native tools (New Relic, Datadog) exist to make AI applications visible inside the same dashboard you already use for the rest of your infrastructure. They do different jobs, and plenty of teams will run one of each: Langfuse to score outputs during development, New Relic to watch the production fleet.

For Swiss & EU teams

New Relic operates a dedicated EU data center, selectable at account signup. According to New Relic’s documentation, customer data is stored in the EU region you select, but the company notes that some features may temporarily process data outside that region, and system operational data stays in the US regardless of account region. There is no dedicated Switzerland region.

For Swiss teams subject to the Federal Act on Data Protection (FADP), this means the EU data center gets you closer to compliance but does not eliminate US data flows entirely. Teams with strict data-residency requirements should either verify the current processing terms with New Relic’s DPA, or evaluate a self-hosted option such as Langfuse (MIT core) or Arize Phoenix (single Docker container), where trace data never leaves your own infrastructure.

What to Do on Monday

1. Audit your OTel setup for gen_ai.* attributes. If your LangChain, LangGraph, or CrewAI app is already emitting OTel traces, check whether your framework is attaching GenAI semantic convention attributes (gen_ai.system, gen_ai.request.model, usage.input_tokens). If not, one OTEL_SEMCONV_STABILITY_OPT_IN flag or a framework-level exporter may be all you need.
2. If you’re already on New Relic, enable AI Monitoring with zero code changes. Route your existing OTLP endpoint to New Relic if you haven’t already. The TraceProcessor will auto-discover any services emitting gen_ai.* spans and surface them in the AI Monitoring UI. Cost is the same per-GB data rate you’re paying now — no additional per-LLM-span billing on the standard data tier.
3. Before committing to the AI compute add-on, get a quote in writing. New Relic’s Advanced Compute Capacity Units for the AI Evaluation engine and Agentic AI features have no published rate. Ask your account executive for the aCCU rate and the minimum commitment before budgeting.
4. If you’re greenfield or want eval-first tooling, benchmark Langfuse self-hosted first. The MIT-licensed core has no event caps when self-hosted, covers offline LLM-as-judge evaluation, and keeps all trace data inside your infrastructure. Initial setup needs ClickHouse, Redis, and S3 — manageable on a Kubernetes cluster. See our guide on securing your AI coding agent pipeline for guidance on where observability data fits in the security model.
5. For teams running multi-model pipelines, set up a Model Inventory view before you need it. Whether it’s New Relic’s account-wide dashboard or a Langfuse cost rollup, the first question in every AI incident is “which model call failed and what did it cost.” Having that view ready before the first production incident cuts triage time significantly. If you’re building a multi-model pipeline, our guide on running a multi-model coding pipeline in 2026 covers the instrumentation side.

Further Reading

• New Relic: Introducing AI Observability for OpenTelemetry — the full technical post on Normalize on Read, Shadow AI discovery, and stateless evaluation architecture.
• Dupple: 10 Best AI Observability Tools in 2026 — independent comparison with verified pricing for Langfuse, Arize, LangSmith, Braintrust, Helicone, and W&B Weave as of September 2026.
• OpenTelemetry GenAI Semantic Conventions for Agents (2026) — what attributes are standardised, which frameworks support them natively, and the MCP span spec.

Your turn: Which observability tool does your team use to monitor LLM calls in production — and did you pick it for the eval features, the infra integration, or the pricing? Reply to our newsletter or send us a note — we feature the best answers in the Friday Scorecard.

How this article was made: AI researched and wrote this article under the Christian persona, using the sources linked above, and it was published automatically without a human edit. Editorial guidelines are set by Adi. Spotted an error? Tell us and we will correct it.