OpenTelemetry

What it means

OpenTelemetry is a Cloud Native Computing Foundation project that standardizes how applications emit telemetry - traces, metrics and logs. A trace is made of spans, and each span records a unit of work with timing, attributes and status. For LLM systems, a growing set of semantic conventions (the GenAI conventions) describe how to record the model name, prompt, completion, token usage and cost on those spans so that any compliant backend can read them.

Why it matters

The value of OTel is that instrumentation is decoupled from the tool that stores and displays it. You instrument your code once against the standard, then send that data to whichever observability backend you choose - and switch backends later without rewriting instrumentation. For LLM applications, which often chain retrieval, tool calls and multiple model invocations, having a common trace format avoids lock-in and lets you correlate model behavior with the rest of your stack.

In practice

Most modern LLM observability platforms accept OpenTelemetry data. Langfuse exposes an OTel-compatible endpoint so traces from standard SDKs land alongside its own, and Helicone and Arize Phoenix likewise build on or interoperate with the standard. In a typical setup you add an OTel SDK to your service, wrap model calls so each one becomes a span with token and latency attributes, and export to a collector that fans the data out to your chosen backend. See OpenTelemetry for LLM observability for a walkthrough of the GenAI conventions and how to wire them up.