OpenTelemetry for LLM Observability in 2026 - A Practical Guide

If you are instrumenting an LLM app, the most future-proof decision you can make is to emit OpenTelemetry. Not because OTel is trendy, but because it is the one thing that lets you change your mind later. Pick a proprietary SDK and switching vendors means re-instrumenting. Emit OpenTelemetry and switching vendors means changing an endpoint.

This is a practical guide to using OpenTelemetry for LLM observability - what “OTel-native” actually means, the GenAI conventions that make traces portable, and how three good platforms differ on OTel support in ways that will bite you if you do not know them going in.

Why OpenTelemetry for LLM apps

OpenTelemetry is the open standard for traces, metrics and logs. The reason it matters for LLM apps specifically comes down to two things.

First, portability. Your instrumentation and your backend become independent. You wrap your app once to emit OTel spans, and you can point them at any OTel-aware backend. That is real leverage in a category that ships breaking changes monthly and where vendors get acquired regularly.

Second, one pipeline. Your LLM traces flow through the same OpenTelemetry collector as the rest of your infrastructure. Your model calls sit next to your service traces instead of in a separate observability silo you have to context-switch into.

The practical upshot: instrument to the standard, not to a vendor. Then choosing a backend becomes a reversible decision instead of a lock-in.

The GenAI semantic conventions

A trace is only portable if everyone agrees what the fields are called. That is what semantic conventions do.

The OpenTelemetry GenAI semantic conventions standardize the attribute names for LLM traces - the model name, token counts, prompts, completions and so on - so any OTel-aware backend reads the same trace the same way. Langfuse aims to comply with these conventions as an OTLP backend. Arize Phoenix is built on OpenTelemetry and Arize’s own OpenInference semantic-convention project, a related standard for LLM traces.

The reason to care is boring and important: if the token-count field has the same name everywhere, your cost dashboards keep working when you switch tools, and your traces stay readable across backends. Standard field names are what make the portability real rather than theoretical.

Step 1: Instrument your app to emit OTel

The setup is the same shape regardless of backend.

1. Add the OpenTelemetry SDK and an exporter to your app.
2. Auto-instrument your model client so LLM calls emit spans with the GenAI attributes populated.
3. Set the OTLP exporter endpoint to your chosen backend.
4. Send a request and confirm the span arrives with model name, tokens and cost attached.

Because you are emitting to a standard, steps 1 and 2 do not change when you swap backends. Only step 3 - the endpoint - does. That is the entire benefit in one sentence.

Step 2: Know the OTel-support differences that actually matter

Here is where “supports OpenTelemetry” hides real differences. All three platforms below support OTel, but not in the same way, and the differences change your architecture.

• Arize Phoenix is genuinely OTel-native. It is built on OpenTelemetry and OpenInference from the ground up, which makes it vendor and framework agnostic - it works well beyond LangChain. If your requirement is strict OTel-native architecture, this is the cleanest fit in the category.
• Langfuse operates as an OTLP backend. It receives traces on an OTLP endpoint at /api/public/otel and maps the GenAI conventions. It is a first-class OTel citizen - but with one sharp edge covered below.
• Opik treats OTel as one ingestion path among many. Opik integrates with OpenTelemetry through an OTLP endpoint for framework compatibility, and its Ruby support runs via OTel - but OpenTelemetry is one path alongside 60+ direct framework integrations, not the core architecture. For most stacks that distinction does not matter; if you are standardizing strictly on OTel, it is worth knowing Opik is OTel-friendly rather than OTel-first.

The mental model: Phoenix is OTel-first, Langfuse is OTel-backend, Opik is OTel-compatible. All three work. Pick based on how central OTel is to your architecture.

Step 3: Avoid the gRPC trap with Langfuse

This is the specific gotcha that wastes an afternoon, so it gets its own step.

Langfuse supports OTLP over HTTP only - gRPC is not supported yet. It handles HTTP/JSON and HTTP/protobuf, but not gRPC.

The problem is that many OpenTelemetry exporters default to gRPC. So you wire up the integration, send a request, and nothing arrives - not because the setup is wrong, but because your exporter is speaking a transport Langfuse does not accept. Set your OTLP exporter to HTTP explicitly when targeting Langfuse. Check this before you start debugging anything else.

Phoenix and Opik do not have this exact constraint, but the general lesson applies everywhere: confirm your exporter transport matches what the backend accepts before assuming the integration is broken.

Step 4: Verify the conventions are populated

Emitting OTel spans is not the same as emitting good OTel spans. After your first trace arrives, check that the GenAI attributes are actually filled in:

• Model name present, not blank.
• Token counts populated for prompt and completion.
• Prompt and completion content captured, subject to whatever redaction you need.
• Cost derived correctly, if your backend computes it.

If these are empty, your auto-instrumentation is emitting spans but not mapping the GenAI conventions - which means your portability is fake. A trace with no standardized attributes is just as locked-in as a proprietary one, because the next backend cannot read it. Fix the mapping before you rely on it.

So which OTel setup should you run?

• You want the purest OTel-native architecture - Arize Phoenix. It is built on OpenTelemetry and OpenInference, framework-agnostic by design, and runs locally in under a minute to try. Just note the server is Elastic License 2.0 if you plan to resell it as a service, and there are reports of ingest lag before traces appear.
• You want an OTLP backend with a strong open-source and cheap-cloud story - Langfuse, remembering to force HTTP transport because gRPC is not supported yet.
• You want broad framework coverage where OTel is one path among many - Opik, whose 60+ integrations plus OTLP ingestion cover almost any stack, with the caveat that OTel is not its core architecture.

The through-line: instrument to OpenTelemetry and the GenAI conventions once, verify the attributes are actually populated, and your backend becomes a swappable decision. In a category this volatile, that portability is the most valuable thing you can build in.