---

title: “07 AI Agent Observability with Langfuse: Setup, Costs and Limits” slug: “07-ai-agent-observability-langfuse” description: “07 AI agent observability with Langfuse: 10 observation types, 7 self-host components, a 120,000-observation cost model and 10 open issues, snapshot 9 August 2026.”

07 AI Agent Observability with Langfuse: Setup, Costs and Limits

The “07” is a month segment from a URL. The canonical page lives at langfuse.com/blog/2024-07-ai-agent-observability-with-langfuse, published July 2024 and revised since. It is not a version number, a chapter, or a module in a course. People search 07 ai agent observability with langfuse because that is the string that gets pasted into Slack threads and syllabi.

07 AI agent observability with Langfuse means capturing every step an agent takes, from LLM calls and tool invocations to retrievals and control-flow decisions, as structured queryable traces instead of log lines. Langfuse does this through 10 observation types, ingests OpenTelemetry over OTLP, and runs either self-hosted or on 4 regional clouds.

Jump: the runnable Python example, framework tiers, the billing-unit model, or where Langfuse breaks.

Evidence policy. Every limitation below links to a public GitHub issue with its number and reaction count as read on 9 August 2026. Every comparative claim is labelled by who made it: the vendor about itself, a competitor about a rival, or a third party with no product in the category. Nothing here is a benchmark. No latency overhead figures, no ingestion throughput numbers, no resource baselines, because that needs a controlled test nobody ran for this article. The publisher has no commercial relationship with any platform named here.

What 07 AI agent observability with Langfuse captures

A working setup captures 6 classes of data. Miss one and you are guessing during an incident.

1. LLM calls. Prompt, completion, model parameters, token counts, computed cost.
2. Tool calls. Which tools were offered, which one the model chose, what arguments it passed.
3. Control flow. Subagent spawns, handoffs, loop iterations and their counts.
4. Context. Prompt version, retrieved documents, full message history at each step.
5. Grouping. Session and user identifiers, so a 14-turn conversation reads as one thing.
6. Quality signals. User feedback and evaluation scores attached back onto the trace.

Classical APM answers a different question. A Datadog or New Relic APM view tells you /chat returned 200 in 4.2 seconds. Agent observability tells you the agent called search_docs 11 times because the tool returned an empty list and the model never gave up. Both describe the same request. Only one tells you what to fix.

The second distinction matters more. Single-call LLM observability treats the completion as the unit: prompt in, tokens out, cost recorded. Agent observability treats the loop as the unit. A well-instrumented call inside a badly instrumented loop tells you nothing about why the agent spent $2.40 on a question that should have cost $0.04.

Sixty times the price. Same answer.

Visual needed: Schematic nested tree of one multi-agent run, with callouts on the root trace, agent, tool (available-vs-called), retriever and generation observations, plus a trace-level score. Flag where issues #8780 and #3961 fragment the tree.

Why agents break differently from ordinary software

The defining failure looks like success. An agent returns a well-formed, confident, wrong answer. Nothing throws a 500, no latency threshold trips, every binary health check stays green. Aryan Kargwal, in the Digital Applied tracing and monitoring stack guide, frames this as the reason status-code monitoring is blind to agent failure. The system behaves exactly as designed while producing garbage.

Non-determinism compounds it. The same prompt takes a different tool path on consecutive runs. One happy-path trace proves nothing about the 99 runs you did not open, which makes agent debugging an aggregation problem before it is a trace-viewing problem. How often does the agent take the 4-step path rather than the 11-step path, and what distinguishes the runs where it loops?

Then spend. Agents decide their own budget. The model chooses how many calls a task takes, so cost is emergent, not a unit price you multiply by request volume. Attribute it per trace, per user and per model, or you find out on the invoice.

Traces get large. Plan-act-observe loops with subagent delegation produce hundreds to thousands of observations per run, and Langfuse’s own post claims teams running long-lived agents see traces reaching hundreds of thousands. That number arrives with no customer name, no date and no linked source. It is a vendor assertion about its own product, and it should not enter a capacity plan until someone measures it.

The consequence for tool selection is direct. Judge Langfuse, or any rival, on its search, filter and aggregate views. Not on how attractively the trace tree renders.

The anatomy of a Langfuse agent trace

The data model has 2 levels. A trace contains observations, and observations nest arbitrarily deep.

Oracle’s write-up on multi-agent observability states the enterprise rule cleanly: one user request maps to exactly 1 trace, with the orchestrator owning the root span and downstream agents contributing child spans. Adopt this before your second service exists. Retrofitting trace identity across 4 services that each create their own root is a week of work.

Langfuse documents 10 observation types. The type controls what the platform can do with the observation downstream.

Observation type	What it represents	What the type unlocks
event	A point-in-time occurrence, no duration	Timeline markers
span	A unit of work with start and end	Generic nesting, duration
generation	An LLM call	Token counts, cost, model filters, playground replay
agent	An agent’s reasoning step or turn	Graph node rendering, agent-level filters
tool	A tool execution	Available-vs-called analysis, tool-name grouping
chain	A composed sequence	Structural grouping in the graph
retriever	A retrieval step	Retrieval filters and eval access
evaluator	A scoring step run inside the trace	Separates judge calls from product calls
embedding	An embedding call	Embedding cost separation
guardrail	A safety or policy check	Guardrail pass/fail views

Source: Langfuse documentation and its agent observability post. Vendor self-description.

Setting types in custom code is a one-argument change:

from langfuse import observe

@observe(as_type="agent")
def research_agent(question: str) -> str:
    docs = retrieve(question)
    return answer(question, docs)

@observe(as_type="retriever")
def retrieve(question: str) -> list[str]:
    ...

@observe(as_type="tool")
def check_inventory(sku: str) -> dict:
    ...

Framework integrations set these types for you. Custom instrumentation does not. That is the most common reason an agent graph renders as a flat list of identical grey boxes: everything defaulted to span, so there is no structure to draw.

The available-tools versus called-tools distinction makes 3 queries possible that raw JSON payloads do not support.

• Looping agents. Observations with 30 or more tool calls against a single available tool. Almost always a retry loop against a failing dependency.
• Dead tools. Tools present in the available list across thousands of traces and never once invoked. Each one is prompt tokens you pay for on every call, forever.
• Sensitive scope. Was delete_record even offered to this agent, and was it ever called?

The structured tool_calls field carries 5 keys: id, name, arguments, type and index. It is exposed to code and to LLM-as-a-judge evaluators alike. any(c["name"] == "search" for c in tool_calls) answers “did the agent search before answering” with no model in the loop.

Aggregated vs expanded graph views

	Aggregated	Expanded
Repeated calls	Collapsed into 1 node with a counter	Each call gets its own node
Loops	Drawn as a cycle	Unrolled into a DAG in execution order
Answers	”What shape is this agent?"	"Where did run #4471 go wrong?”
Best for	Architecture review	Incident forensics on 1 trace

The graph appears automatically for any trace containing an observation type other than span, event or generation. No types, no graph. Langfuse called the graph view beta in its July 2026 post, so check the changelog before building a workflow on it.

Mapping OpenTelemetry GenAI conventions to Langfuse observations

The question underneath “should we use Langfuse” is “how locked in am I”. The answer runs through the OpenTelemetry GenAI semantic conventions, which define the vendor-neutral vocabulary, and Langfuse’s typed model, which is richer in places and lossy in others.

OTel GenAI operation	Span kind	Key attributes	Nearest Langfuse type	What is lost
create_agent	INTERNAL	gen_ai.agent.name, gen_ai.agent.id	agent	No distinct lifecycle concept; creation and invocation collapse
invoke_agent (remote)	CLIENT	gen_ai.system, server.address	agent	Remote-vs-local is not preserved as a property
invoke_agent (in-process)	INTERNAL	gen_ai.agent.name	agent	Same name, different semantics, no encoding of the difference
invoke_workflow	INTERNAL	workflow identifiers	chain	Approximate match, not a defined mapping
execute_tool	INTERNAL	gen_ai.tool.name, gen_ai.tool.call.id	tool	Cleanest mapping in the set
chat / text completion	CLIENT	gen_ai.request.model, gen_ai.usage.input_tokens	generation	Clean; token and cost fields map
n/a	n/a	n/a	evaluator, guardrail, embedding, retriever	4 Langfuse-side concepts with no OTel counterpart; they degrade to generic spans on export

Built from the OTel GenAI semantic conventions and Langfuse’s observation-type docs. Check operation names and attribute keys against the spec before you build against this table, for the reason in the next paragraph.

Visual needed: Two node columns, OTel operations left and the 10 Langfuse types right. Solid connectors for clean mappings, dashed for approximate, unconnected right-hand nodes flagged as Langfuse-only.

Span kind is the subtlety most integration guides skip. invoke_agent is CLIENT when the agent executes remotely, as with OpenAI Assistants and Amazon Bedrock Agents. It is INTERNAL when the framework runs in your process, as LangChain and CrewAI do. Same operation name, different meaning for anyone writing alerting rules on span kind.

The stability trap. At semconv v1.41 the GenAI conventions sit at Development status, not Stable. Nearly every gen_ai.* attribute carries a Development badge; the exceptions are error.type, server.address and server.port. A key like gen_ai.usage.input_tokens can be renamed without a major version bump. Setting OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental dual-emits legacy and current names during a transition, which is how you survive a rename without a synchronised deploy across 6 services. That summary reaches this page through the Digital Applied guide rather than the spec, and the status line moves with each semconv release, so the spec page is the authority on the day you read it.

MCP conventions arrived in v1.39 with 3 attributes: mcp.method.name, mcp.session.id and mcp.protocol.version. The design decision is enrich-don’t-duplicate. MCP instrumentation decorates the existing execute_tool span instead of creating a parallel one, so an MCP-backed tool call stays 1 node rather than 2.

Two histogram metrics are the floor for production: gen_ai.client.operation.duration and gen_ai.client.token.usage. Without those in Grafana, Prometheus or whatever you run, you have traces and no telemetry.

The vendor conflict, adjudicated. MLflow’s comparison page marks Langfuse’s OpenTelemetry support “Partial (ingest)” and its own “Full”. Langfuse markets itself as OTel native. Ingest-only means you can point an OTLP exporter at Langfuse and data arrives, but the Langfuse SDKs stay the first-class path, and attributes outside the recognised set land in an observation’s metadata object rather than the typed model. Langfuse’s own Vercel AI SDK docs acknowledge that metadata fallback. “Partial” fairly describes mapping fidelity. “Native” fairly describes the ingestion path. Neither page says which it means.

Quickstart: instrumenting a Python agent end to end

The shortest complete path in Python is about 30 lines, 4 decorators and 1 flush.

# Written against Langfuse Python SDK v3 (langfuse >= 3.x) and re-read
# against release v4.6.0 (2026-08-06). Decorator import paths moved
# between v2 and v3; confirm against the release you install.

import os
from langfuse import Langfuse, observe, get_client

os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
os.environ["LANGFUSE_BASE_URL"]   = "https://cloud.langfuse.com"  # EU region

langfuse = Langfuse()

@observe(as_type="retriever")
def search_docs(query: str) -> list[str]:
    return ["doc-141: refund window is 30 days", "doc-903: exceptions"]

@observe(as_type="tool")
def lookup_order(order_id: str) -> dict:
    return {"order_id": order_id, "shipped": True, "days_since": 12}

@observe(as_type="generation")
def answer(question: str, context: list[str]) -> str:
    # your model call; usage is captured when the provider SDK is wrapped
    return "You are inside the 30-day window, so a refund applies."

@observe(as_type="agent")
def support_agent(question: str, order_id: str, user_id: str) -> str:
    client = get_client()
    client.update_current_trace(user_id=user_id, session_id=f"sess-{user_id}")

    order = lookup_order(order_id)
    docs = search_docs(question)
    reply = answer(question, docs + [str(order)])

    client.create_score(name="tool_before_answer", value=1.0)
    return reply

if __name__ == "__main__":
    print(support_agent("Can I still return this?", "A-4471", "user-88"))
    langfuse.flush()          # never skip this in short-lived processes

Pick your base URL deliberately. Four data regions exist, selected purely by LANGFUSE_BASE_URL: EU at https://cloud.langfuse.com, US at https://us.cloud.langfuse.com, Japan at https://jp.cloud.langfuse.com, HIPAA at https://hipaa.cloud.langfuse.com. Source: Langfuse’s Vercel AI SDK integration docs.

The TypeScript path has a version split that will cost you an afternoon. AI SDK 7 uses callback telemetry through @langfuse/vercel-ai-sdk and needs Node.js 22 or later. AI SDK v6 uses the older experimental_telemetry: { isEnabled: true } option. Pinned to Node 20? You need the v6 path. Follow the AI SDK 7 docs on Node 20 and you get an install that looks fine and emits nothing.

Two things silently break traces.

• Serverless termination. Call forceFlush() before the function returns. A Lambda or Cloud Run instance that returns while the exporter still has a queue drops the trace, invisibly.
• Streaming responses. Set endOnExit: false when the response streams, or the span closes at first byte and the rest of the generation lands outside it.

When traces do not show up, set LANGFUSE_LOG_LEVEL=DEBUG and read the fork. Spans in the logs but nothing in the UI points at credentials, project routing or flushing. No spans at all means the instrumentation never ran, usually a decorator sitting on a function the framework calls through a path the SDK cannot see.

One practitioner data point, labelled. A founder building a competing tool wrote on Hacker News that when evaluating LangSmith and Langfuse for agent cost and performance visibility, “they felt like they are overcomplicated to integrate” (HN 47247631). One opinion, from someone with a product in the same category.

Framework and harness coverage: what’s first-class, what’s OTel-only

Langfuse advertises 100+ integrations. MLflow’s comparison page credits it with 60+. The gap is 40 integrations, two-fifths of the larger claim, and both counts use self-serving definitions of the word.

Tier	Meaning	Frameworks
A. Dedicated integration and docs page	Langfuse ships and documents the adapter; observation types set automatically	LangGraph, OpenAI Agents SDK, Claude Agent SDK, CrewAI, Vercel AI SDK (7 and v6), Pydantic AI, Hugging Face smolagents, Strands Agents, LangChain
B. Framework-native OTel, Langfuse ingests	Framework emits OTel over OTLP; typed mapping is partial	Google ADK, Microsoft Agent Framework, Semantic Kernel, OpenLIT, Traceloop OpenLLMetry, OpenInference-instrumented stacks, GitHub Copilot
C. Roll your own	@observe decorators and manual span types	Bespoke orchestrators, in-house agent loops

The index also lists LlamaIndex, Haystack, DSPy, LiteLLM, Ollama, IBM watsonx Orchestrate, Amazon Bedrock, Azure OpenAI, Google Vertex AI and the no-code builders Flowise, Langflow, Dify and n8n. It changed twice in the 30 days before the snapshot, so count it yourself on the day you decide.

The coding-agent class deserves separate treatment. It is the fastest-growing use of agent tracing, and only 2 pages in the current top 10 mention it. Claude Code and OpenAI Codex integrate through lifecycle hooks. GitHub Copilot exports OTel natively. Cursor, Kiro, OpenCode and Augment Code also appear in Langfuse’s coverage.

Oracle’s A-Team documented the Codex plugin path in 5 steps: add the marketplace, add the plugin, set plugin_hooks = true, supply credentials, restart.

Their privacy warning belongs alongside it. The Codex observability plugin can upload transcript data including prompts, assistant messages, reasoning summaries, tool-call inputs and outputs, model metadata and token usage. Do not enable it on sessions containing data you would not deliberately store in Langfuse.

A coding agent’s transcript contains your source code.

Two status notes. Langfuse’s post states AutoGen is in maintenance mode, with Microsoft steering new projects to the Agent Framework. That is a vendor’s claim about a third party; check Microsoft’s repository before planning a migration on it. And the gap users surfaced themselves: issue #6588 requests an official first-party Google ADK integration guide and carries 64 reactions. ADK tracing works over OTel today. The guide people keep asking for is a documentation gap that has stayed open long enough to gather that much signal.

Visual needed: The 3-tier table as a graphic, with AutoGen flagged maintenance mode and issue #6588 as the ADK footnote.

Multi-agent, distributed and MCP tracing

The invariant: separate services become 1 trace only if they share a trace ID. Nothing else joins them. Three mechanisms produce that shared ID.

1. OTel context propagation across service boundaries. Headers carry the context, and the receiving service continues the trace instead of starting one.
2. Deterministic trace IDs derived from a seed, such as an external request ID. The agent service and an unrelated batch job both compute the same ID from the same seed.
3. Sessions, for multi-turn conversations where each turn is legitimately its own trace and you want them grouped rather than merged.

The second is more useful than it sounds. An offline evaluation pipeline running 6 hours after the request computes the trace ID from the request ID it already has, then attaches scores without ever holding a trace handle or passing one through a queue. That is the difference between an eval pipeline that plumbs through every layer and one that needs a single string.

MCP client and server linking goes through the MCP _meta field. Skip it and you get 2 disconnected traces: a client-side tool call that goes nowhere, server-side work with no parent. Most common cause of “my tool calls vanished”.

For supervisor and subagent architectures, OpenAI Agents SDK handoffs and LangChain DeepAgents subagent spawning both nest as typed observations under the parent, which keeps the delegation legible in the tree and the graph.

Where this breaks in practice is async context propagation. Not architecture. The Python contextvars mechanism itself. See the failure-mode catalogue.

Working with traces that have thousands of observations

Three features form a workflow, and treating them as a feature list is how people end up scrolling a 3,000-node tree at 2am. The Observations table makes every LLM call, tool execution and agent step a queryable row: filter by type, sort generations by cost, pull every ERROR-level observation for one user, save the view. Full-text search across inputs, outputs and metadata finds the 7 traces mentioning a specific SKU. The trace log view renders the whole trace as one scrollable document you can Ctrl+F, which beats tree navigation once a trace passes roughly 200 observations.

Three saved views worth building on day 1: highest-cost generations in the last 24 hours; observations with 30 or more tool calls, your loop detector; ERROR-level observations grouped by tool name, your dependency-failure board.

Two recent changelog additions, both landed within 14 days of the snapshot: Pulse, a chart strip above the Observations table for spotting count, cost and latency spikes and drag-selecting the window, and table-to-chart toggling (changelog).

Before you build alerting on saved filters, read the score-filtering bug in the catalogue below.

Features the comparison posts skip

Six capabilities sit outside tracing and rarely appear in the ranking pages, though they decide the daily experience.

• Prompt management. Versioned prompts with deployment labels, fetched by the SDK at runtime and cached locally, so a prompt change ships without a deploy and every generation records the version that produced it.
• Annotation queues. Human reviewers score traces in a queue rather than by hunting through the table. This is the mechanism that makes the 50-trace review below repeatable rather than heroic.
• Datasets and Experiments. Curated inputs with expected outputs, run against a prompt or model variant, with results comparable across runs.
• Alerts and webhooks. Threshold-triggered notifications into Slack or an HTTP endpoint, for error rate, cost or a score dropping below a bound.
• Batch exports to S3 or blob storage. Scheduled dumps of traces, observations and scores in JSONL or CSV, which is also 1 of the 3 exit routes below.
• Environments and custom model pricing. An environment attribute separates staging from production in every view, and custom per-token prices let self-hosted or fine-tuned models on Bedrock, Vertex AI or Ollama get costed alongside GPT and Claude calls.

MLflow claims SSO, RBAC and audit logs sit behind paid plans. That is a competitor’s claim about a rival, and open-core products usually do gate exactly those, which is a reason to check the pricing page rather than to accept it.

Evaluating agents, not just watching them

Sequencing beats tooling here. Review roughly 50 real production traces by hand before writing a single automated scorer. Automate first and you build scorers that measure what you assumed would go wrong. Those 50 traces tell you the actual failure is the agent calling the retrieval tool with the user’s raw question instead of a reformulated query, which no off-the-shelf scorer was going to catch.

	Offline	Online
Runs against	Curated datasets	Live production traffic
Timing	Before shipping	After shipping
Catches	Regressions	Drift, live quality drops
Writes to	Score objects on traces	Score objects on traces

Both write to the same objects, which is what makes the two comparable at all.

Trajectory evaluation is what final-answer scoring misses. An agent can reach a correct answer through a path costing 8 times what it should, or touching a tool it had no business touching. Four assertions against the structured tool_calls field:

def trajectory_checks(tool_calls: list[dict], step_budget: int = 12) -> dict:
    names = [c["name"] for c in tool_calls]
    counts = {n: names.count(n) for n in set(names)}
    allowed = {"search", "lookup_order", "check_inventory"}
    return {
        "searched_before_answering": "search" in names,
        "no_tool_over_n_times":      max(counts.values(), default=0) <= 5,
        "no_out_of_scope_tool":      set(names) <= allowed,
        "within_step_budget":        len(tool_calls) <= step_budget,
    }

Four booleans. No judge model, no API cost. Run them on every production trace.

Turning production failures into dataset items is what makes observability spend compound. A failure you saw once becomes a dataset row, the row becomes a regression test, the test means that failure never ships twice. Without the loop you are paying for an expensive log viewer.

The mature pattern, which the Digital Applied guide names correctly, is the eval gate: scorers wired into CI that block a merge when a score regresses past a threshold. Prompt changes get treated like code changes.

One changelog entry fixes a mistake teams make constantly. Consistent evaluator sampling compares evaluators on the same sample of matching observations. Comparing 2 LLM judges on 2 different samples and concluding one is better is a category error.

The ceiling, labelled. MLflow’s page claims Langfuse offers only “basic LLM-as-judge scoring” and gates advanced evaluation behind paid plans. Direct competitor, page ranks itself first, no date and no source attached. Check Langfuse’s own pricing and docs before assuming either parity or a gap.

Where Langfuse actually breaks: a failure-mode catalogue from the issue tracker

Frame this before reading it. On 9 August 2026, langfuse/langfuse showed 32,775 stars and 761 open issues, 1 open issue per 43 stars. For an actively maintained open-source project of that size, that ratio is normal. What matters is which issues share a root cause, how long they stay open, and whether they sit on your critical path.

Nothing here was reproduced in-house. Each row reports what the issue says, links it, and gives its reaction count on the snapshot date. Some may have been fixed in v4.6.0, released 3 days before, or since.

Sorted by reaction count. The 10 rows carry 657 reactions between them, a mean of 66 and a median of 56.

Symptom	Issue	Signal	Who it affects	Workaround
Incompatible with Python 3.14; attributed to Pydantic v1 usage, which Pydantic states is unsupported on 3.14	#9618	179	Anyone on current Python	Pin to 3.13 or earlier
Open request for an official Google ADK integration guide	#6588	64	ADK teams wanting first-party docs	Generic OTel ingestion
Repeated “Failed to detach context” with langfuse.langchain.CallbackHandler on async methods like ainvoke	#8780	62	Async LangChain/LangGraph	See async cluster below
Experiments and Evaluators fail to extract thinking and text blocks from Anthropic reasoning models (claude-haiku-4-5-20251001)	#11109	62	LLM-as-judge stacks using reasoning models	Judge with a non-reasoning model
import langfuse in a Jest test throws “A dynamic import callback was invoked without —experimental-vm-modules”	#5704	57	TS/JS teams testing with Jest	Run Jest with --experimental-vm-modules
Python SDK is fully typed but ships no py.typed marker, so it is not PEP 561-compliant	#2169	51	Downstream projects running mypy	Local stub package or mypy ignore
Filtering traces by numeric score value does not return all matching traces (reported filtering judge output by score < 1)	#11874	50	Alerting built on score filters	Verify against a known small dataset
FastAPI StreamingResponse resets ContextVars after the first yielded chunk, recording 1 run as several fragmented traces	#3961	49	Streaming Python APIs	See async cluster below
ClickHouse consumes high CPU frequently after the V3 upgrade, reportedly continuing after langfuse-web and langfuse-worker are shut down	#6572	42	Self-hosted deployments	Budget for a warm ClickHouse
Images uploaded via LangfuseMedia reach the S3 bucket but render as a button rather than inline	#4555	41	Multimodal agents	Cosmetic; data is stored correctly

Visual needed: The catalogue as the page’s linkable asset: symptom, issue link, reactions, affected stack, open/closed status, workaround. Snapshot date in the caption; re-open each issue at publish time.

The synthesis nobody has made. Issues #8780 and #3961 are 2 reports of 1 root cause. Python contextvars do not survive the transitions async streaming frameworks make. #8780 surfaces as “Failed to detach context” when a LangChain callback handler crosses an ainvoke boundary. #3961 surfaces as one agent run recorded as several disconnected traces, because FastAPI’s StreamingResponse resets the context after the first yielded chunk.

Between them they describe the stack most production Python agents run on. An async FastAPI service streaming responses from a LangGraph agent. Not exotic. The default. Combined signal is 111 reactions, 17% of the catalogue total, across 2 of 10 rows.

If that is your architecture, test context propagation on a streaming endpoint before designing anything else around Langfuse traces. Twenty minutes. Highest-value check in this article.

Three more deserve a sentence.

Python 3.14 (#9618, 179 reactions, 27% of all signal in the catalogue) is a runtime-version gate, a different class of problem from a bug. It blocks adoption outright. Check its status first.

Score filtering (#11874, 50 reactions) matters disproportionately because of how it presents. A filter returning some matching traces looks identical to one returning all of them. Clean result set, no error. Anyone alerting on “traces where the judge scored below 1” should confirm the filter behaves on a 20-row dataset with a known answer.

Reasoning-model evaluation (#11109, 62 reactions) is the one to check before you design. If your eval architecture assumes a reasoning model as judge and Experiments cannot parse that model’s thinking blocks, you find out after building the whole thing.

Self-hosting Langfuse: the real component footprint

Oracle deployed self-hosted Langfuse and documented the components independently of Langfuse’s marketing. Seven pieces.

Component	Responsibility
Web server	UI, API, SDK ingestion traffic
Async worker	Background event processing
Redis / Valkey	Cache and queue
PostgreSQL	Transactional state (users, projects, prompts, config)
ClickHouse	Traces, observations and scores at volume
S3 / MinIO / Azure Blob	Raw events, exports, multimodal attachments
LLM gateway (optional)	Playground and evaluation flows

Source: Oracle’s multi-agent observability write-up, deployed both on a VM with Docker Compose and on Oracle Kubernetes Engine.

The split has reasons you can evaluate. Postgres holds system state where transactional guarantees matter and volume is low. ClickHouse holds trace data where volume is enormous and queries are analytical aggregations over columns. Redis absorbs ingestion bursts so a spike does not become dropped spans. The worker keeps event processing off the web tier so a heavy ingest does not freeze the UI. Judge whether your team can operate that, rather than counting containers.

Visual needed: Component diagram with data-flow arrows labelled by payload (SDK ingest, queued events, transactional state, analytical queries, attachments). Annotate ClickHouse with issue #6572.

The vendor dispute, adjudicated. MLflow’s page calls this “5+ services requiring ClickHouse expertise” with “steep operational overhead and frequent architecture changes in the past”. Competitor, undated, unsourced. Oracle, an infrastructure vendor with no LLM observability product in this category, documents the same architecture and the same count. The component count is corroborated. The adjective is not.

MLflow’s FAQ also states you cannot substitute PostgreSQL or another analytical backend for ClickHouse trace storage. Competitor claim, consistent with the architecture Oracle describes, worth confirming in the self-hosting docs before planning around it.

The operational risk nobody else mentions: #6572, 42 reactions, reports ClickHouse burning CPU after the V3 upgrade with no Langfuse usage, reportedly even after both langfuse-web and langfuse-worker are stopped. Budget consequence: your self-hosted baseline may not scale down to idle. Budget for a warm ClickHouse, not a sleeping one.

Deployment paths are Docker Compose on a VM for simple setups, Kubernetes with Helm or Terraform for production. Oracle ran both.

The decision rule. Self-host if trace payloads cannot leave your infrastructure, or if you need a residency guarantee no cloud region gives you. Use the cloud if your team cannot commit to ClickHouse retention policy, patching, backups, access control and audit logging as ongoing work. That is not a setup cost. It is a permanent line in someone’s job description.

What this section cannot tell you is the resource baseline at 10 million observations a month. That needs a controlled deployment and sustained measurement, which nobody in the current top 10 has published either.

What observability actually costs: a billing-unit model

The whole section turns on one fact. Platforms meter different units. LangSmith bills traces. Braintrust and Datadog bill spans. Helicone bills requests. Langfuse bills observations. The same workload prices completely differently across platforms, and the cheapest sticker price is routinely the most expensive bill. Budget 07 AI agent observability with Langfuse by unit, not by headline number.

Platform	Billing unit	Free tier	Paid entry
Langfuse	Observations	50,000/month, no card	Per vendor pricing page
LangSmith	Traces	5,000/month, 14-day retention	Plus $39/seat/month; overage ~$2.50 per 1,000 traces
Braintrust	Spans	1,000,000/month (Starter)	Pro $249/month
Helicone	Requests	10,000/month	Per vendor pricing page
Datadog LLM Observability	LLM spans only	40,000/month	Pro $160/month with 100,000 LLM spans

Langfuse’s figure is its own claim. The rest are compiled from the Digital Applied guide and MLflow’s comparison page. All published-pricing snapshots read 9 August 2026 and none of it is a quote.

Work the arithmetic on a shape you can substitute your own numbers into. One agent task produces roughly 1 LLM call, 8 tool calls and 2 retrievals, plus the agent span. Call it 12 observations per task.

At 10,000 tasks per month, the same workload is:

• 10,000 traces. Twice LangSmith’s free tier, so about $12.50 a month in overage before seats.
• 120,000 observations. Two and a half times Langfuse’s free tier, leaving 70,000 billable.
• 120,000 spans. Twelve per cent of Braintrust’s Starter allowance.
• 10,000 LLM spans. A quarter of Datadog’s free tier, because the other 110,000 tool, retrieval and agent spans are not billed at all.

Same agents. Same month. Four different answers about whether you are still on the free tier.

The Datadog detail inverts the ranking for tool-heavy agents. Billing LLM spans only means tool, embedding, retrieval and agent spans cost nothing, so a system with 11 non-model spans per model call is dramatically cheaper on a span-class-aware model than on a flat per-span one. Flip the ratio to 1 tool call per 4 model calls and the advantage disappears.

Visual needed: Grouped bar chart pricing 10,000 tasks/month × 12 observations against each platform’s unit. Caption states the assumptions and the snapshot date, and that this is a model, not a measurement.

Cutting volume before it bills

Three levers reduce billable units, all configuration rather than code.

Head sampling. OTEL_TRACES_SAMPLER=parentbased_traceidratio with OTEL_TRACES_SAMPLER_ARG=0.1 keeps 1 trace in 10, whole. The modelled workload drops from 120,000 observations to 12,000, back inside the free tier with 38,000 to spare. You also lose 9 incidents in 10, so sample production and leave staging at 1.0.

Span filtering. Drop spans emitted by unrelated libraries in the same process before they reach the exporter. Langfuse’s troubleshooting docs name these as a cause of unexplained billable volume.

Payload trimming. Store a reference or a hash instead of the full retrieved document. The structure survives. The bytes do not.

Retention is the hidden dimension. MLflow’s compiled table lists Langfuse at 30 days free rising to 3 years on pro, LangSmith at 14 days free with a 400-day paid add-on, Arize Phoenix at 7 days free and 15 on pro, Braintrust at 14 days on starter and 30 on pro, MLflow unlimited when self-hosted. Competitor-compiled, and retention figures go stale faster than any other number in this category.

Self-hosting is not free. It is ClickHouse plus Postgres plus Redis plus object storage plus an on-call engineer who understands all 4. Price that list against your cloud bill before treating self-hosted as the zero-cost option.

Governance: masking, PII and what your traces now store

State it plainly. A trace store holding full prompts, retrieved documents and tool payloads is a second copy of your most sensitive data, in a system with looser access controls than the primary one. Your production database has row-level security and an audit log. Your trace viewer has a project-level role.

Oracle’s A-Team reference architecture is the only page in the top 10 treating data protection as its own layer, and its framing is right. Shift left. Mask before export. Keep raw records in the governed systems that already hold them, and store references, hashes, row IDs or summaries in telemetry wherever the debugging value survives the substitution.

Langfuse offers 2 control points, per that write-up: client-side masking, so sensitive fields never leave the application process, and self-hosted enterprise ingestion masking, so policy is enforced centrally instead of reimplemented by every team in every service.

The OTel-native alternative generalises across backends. Content capture is opt-in in the GenAI conventions, with 3 modes: not recorded, stored on span attributes, or stored externally with only a reference URL on the span. Mode 3 is the pattern for PII systems. Full trace structure for debugging, no customer data in telemetry, and the same spans stay readable in Grafana Tempo, Jaeger, SigNoz or any other OTLP backend.

Region selection is a compliance decision, not a latency one. EU, US, Japan and HIPAA endpoints are chosen by 1 line of configuration, which means that line belongs in code review rather than in an environment variable somebody set once.

One hygiene point from Langfuse’s troubleshooting docs is a privacy issue and a cost issue at once. Unrelated OTel spans from other libraries in your process land in Langfuse and count toward billable units. Span filtering is noise control and cost control together.

Langfuse versus MLflow, LangSmith, Arize Phoenix, Braintrust and Helicone

The methodological point first. The most-cited comparison table for this keyword is published by MLflow, a direct competitor, and it ranks MLflow first. That does not make any row wrong. It does mean every claim about a rival is an interested party’s characterisation, and no page in the top 10 tells the reader which.

Dimension	Langfuse	MLflow	LangSmith	Arize Phoenix	Braintrust	Provenance
Licence	MIT core, separately licensed ee/	Apache 2.0, Linux Foundation	Proprietary	Elastic License 2.0	Proprietary	GitHub API + MLflow’s table
Self-host	Yes	Yes	Limited	Yes	Limited	MLflow’s table (competitor)
Components to run	6–7	Fewer	n/a	Fewer	n/a	Oracle (neutral) corroborating
OTel support	”Native” vs “Partial (ingest)”, disputed	Full (claimed)	Partial	OpenInference	Partial	Both vendors, contradictory
Free tier and unit	50k observations/mo	Self-host unlimited	5k traces/mo	Published tier	1M spans/mo	Vendor pricing pages
Retention (free)	30 days	Unlimited self-host	14 days	7 days	14 days	MLflow’s table (competitor)
Corporate status	ClickHouse Inc., acquired Jan 2026	Linux Foundation	LangChain Inc.	Arize AI	Independent, $80M Series B	Public reporting

Licensing, precisely. Langfuse’s core is MIT with a separately licensed ee/ directory, which is why GitHub’s API reports NOASSERTION: the machine-readable consequence of a split licence, not a red flag. MLflow is Apache 2.0 under Linux Foundation governance. Arize Phoenix uses Elastic License 2.0, restricting managed-service resale. LangSmith and Braintrust are proprietary.

PyPI download figures deserve a warning. MLflow’s table lists MLflow at 30M+/month, Langfuse 15M+, LangSmith 65M+, Phoenix 1M+, Braintrust 3M+. MLflow itself footnotes that LangSmith’s number is inflated by being an automatic dependency of langchain. The objection applies unevenly elsewhere, since anything pulled in transitively inherits its parent’s install count. Download counts are a poor proxy for production adoption whoever publishes them.

Six more products sit in this category and are not benchmarked here: Weights & Biases Weave, Comet Opik, LangWatch, Portkey, PostHog LLM analytics and Arize AX. So do the general-purpose APM vendors: New Relic, Dynatrace, Honeycomb, Grafana Cloud, Splunk, Elastic and Sentry. If your spans already flow over OTLP, several will accept the same data without reinstrumentation.

Recommendation by constraint, not by winner:

• LangChain/LangGraph-only stack, want first-party depth → LangSmith
• Must self-host, already operate ClickHouse → Langfuse
• Want Apache 2.0 with foundation governance → MLflow
• Want research-backed built-in evaluation metrics → Arize Phoenix
• Want gateway-level cost control across providers → Helicone
• Non-technical stakeholders reviewing traces → Braintrust

The closest thing to a disinterested read comes from Oracle’s A-Team, whose table characterises Langfuse’s best fit as teams wanting open-source control plus broad coverage. Not neutral in the absolute, since the same page steers toward OCI Log Analytics elsewhere. On Langfuse specifically, they have nothing to gain.

What the ClickHouse acquisition changes

ClickHouse acquired Langfuse on 16 January 2026, inside a $400M Series D valuing ClickHouse at $15B. At acquisition Langfuse reported more than 2,000 paying customers and tens of millions of SDK installs a month, and stated that open-source licensing and self-hosting stay unchanged. Both figures are the company’s own numbers, announced by the party with an interest in them, and neither has been audited or independently confirmed.

The structural read: the acquirer is the database Langfuse already depended on. That tightens the ClickHouse dependency in self-hosted deployments. Good for ingestion performance, good for the odds that ClickHouse-specific bugs get fixed by people who wrote ClickHouse. Bad for backend optionality if you hoped to swap the analytical store. CEO Marc Klingen has framed the deal around that alignment, which is a chief executive describing his own company.

MLflow’s FAQ answers the acquisition question with “the long-term product roadmap and investment level remain to be seen”. A competitor speculating, no evidence attached.

Checkable evidence against abandonment risk, all clickable:

• Repository last push 2026-08-09 (github.com/langfuse/langfuse)
• Release v4.6.0 on 2026-08-06, 3 days earlier
• CTO and co-founder Max posting in Ask HN: Who is hiring (August 2026), recruiting Product and Backend Engineers in Berlin and EU-remote at €70–130k plus ClickHouse equity, describing Langfuse as “now part of ClickHouse” (HN 49180088)

Releases 3 days before the snapshot and independent hiring 7 months after the acquisition is the strongest available signal. Speculation loses to a commit log.

The exit path: how to get your traces back out

The lock-in question is not “is it open source”. It is this: what is the procedure for moving 6 months of traces to another backend, and how lossy is it?

1. Instrument through vendor-neutral OTel. Re-pointing an OTLP exporter is a configuration change. Cheapest exit, and it has to be decided at instrumentation time.
2. Own the storage. Self-hosted deployments hold the ClickHouse tables and the S3 event store directly.
3. Public API, SDKs and scheduled batch exports for programmatic extraction.

The lossy part, stated honestly. OTel gives you span portability, not evaluation-layer portability. The 10 observation types, scores, dataset items, prompt-management links and session groupings have no guaranteed equivalent in another backend’s model. Your spans move. Your 6 months of judge scores, your curated regression datasets and your prompt-version-to-quality links get rebuilt against whatever the next platform calls them.

Portability of traces is not portability of the evaluation layer. Every platform in this comparison shares that property. Treat vendor-SDK-only features as priced lock-in you accept knowingly.

The diligence rule applies to all 6 platforms, not only Langfuse. Write down the export procedure and confirm each step in the documentation. A vendor claiming portability without documenting the procedure is making a marketing claim.

A 30/60/90 rollout plan

Days 1–30. Instrument one agent properly. Pick a single agent and trace it end to end through OTel-compliant paths. Set observation types explicitly on every piece of custom code. Verify traces arrive in the correct data region before anything sensitive flows. Confirm forceFlush behaviour in your serverless or streaming path rather than assuming it. Establish one-request-one-trace as an architectural rule while there is still only 1 service.

Days 31–60. Look at the data before automating anything. Review 50 real production traces by hand and write down every failure in plain language in a shared document. Build the 3 saved views: cost outliers, observations with 30 or more tool calls, ERROR-level observations grouped by tool name. Turn the top 5 recurring failures into dataset items. Configure masking now, before the trace store accumulates 90 days of unredacted customer data you then have to delete.

Days 61–90. Close the loop. Encode the reviewed failures as scorers, including the 4 trajectory assertions above. Wire an eval gate into CI so a scored regression blocks a merge. Add per-user and per-model cost attribution. Run the billing-unit arithmetic against observed volume rather than forecast, and check it against your contract. Document the export procedure where the next engineer will find it.

Visual needed: Three horizontal bands for days 1–30, 31–60 and 61–90 with their 5 actions each, and the 4 pre-adoption checks pinned as a gate before day 1, labelled with issue numbers and reaction counts.

Pre-adoption checklist. Do these before committing.

• Confirm your Python version against #9618 (Python 3.14, 179 reactions).
• Test async streaming context propagation on your actual FastAPI plus LangGraph stack, against #3961 and #8780.
• Verify score-based filtering on a small dataset with a known answer, against #11874, before building alerting.
• If your judge is a reasoning model, read #11109 before designing the eval architecture.

Frequently Asked Questions

What does the “07” in “07 ai agent observability with langfuse” mean?

It is the month segment of Langfuse’s blog URL, langfuse.com/blog/2024-07-ai-agent-observability-with-langfuse. Not a version, a chapter or a course module. The post dates from July 2024 and has been revised repeatedly since, so the date in the slug tells you nothing about how current the content is.

What is AI agent observability?

It captures every step an agent takes, from LLM calls and tool invocations to retrievals and control-flow decisions, as structured traces you can inspect, filter and score. It extends single-completion LLM observability to multi-step, non-deterministic workflows where failures hide in intermediate steps rather than in the final answer.

Is Langfuse fully open source?

The core is MIT-licensed with an ee/ directory under separate enterprise licensing, which is why GitHub’s API reports NOASSERTION. MLflow, a competitor, claims SSO, RBAC and advanced evaluation sit behind paid plans. Check Langfuse’s pricing page rather than taking either vendor’s word.

Does Langfuse work with Python 3.14?

Issue #9618 reports incompatibility, attributed to Pydantic v1, which Pydantic states it does not support on 3.14. At 179 reactions it was the highest-signal open issue on 9 August 2026, 27% of the catalogue total. Check its current status, because this class of blocker gets fixed.

What frameworks does Langfuse support for agent tracing?

First-class integrations cover LangGraph, OpenAI Agents SDK, Claude Agent SDK, Pydantic AI, CrewAI, Vercel AI SDK, smolagents, Strands Agents and LangChain. Anything emitting OpenTelemetry is ingested, which is how Google ADK, Microsoft Agent Framework and Semantic Kernel work. Langfuse claims 100+ integrations, MLflow credits 60+, and the 2 counts use different definitions.

How hard is it to self-host Langfuse?

You operate 6 components plus an optional LLM gateway: web server, async worker, Redis or Valkey, PostgreSQL, ClickHouse and object storage, per Oracle’s independent deployment. ClickHouse is a hard dependency with no documented substitute. Docker Compose and Kubernetes with Helm both work. Note that #6572 suggests the baseline may not idle cheaply.

Langfuse vs LangSmith vs MLflow, which should I use?

Choose by constraint. A LangChain-only stack gets most from LangSmith’s first-party depth. A team that must self-host for residency and can operate ClickHouse should pick Langfuse. A team wanting Apache 2.0 under Linux Foundation governance should pick MLflow. Disclosure: the most-cited comparison for this query is published by MLflow and ranks MLflow first.

Why are my Langfuse traces splitting into multiple traces?

Two documented bugs and one design cause. FastAPI StreamingResponse resets ContextVars after the first yielded chunk (#3961, 49 reactions). LangChain and LangGraph async methods like ainvoke raise “Failed to detach context” (#8780, 62 reactions). Separately, services that do not propagate a shared trace ID produce separate traces by design; fix that with OTel context propagation or deterministic seeded trace IDs.

Is there a GitHub example or PDF for Langfuse agent observability?

Langfuse maintains public cookbook notebooks and integration examples in its GitHub organisation, and IBM published its watsonx Orchestrate tutorial code on GitHub. No official Langfuse PDF of this guide exists; the docs and blog are HTML-only. The Python example above is copy-pasteable as-is.

How much does agent observability actually cost?

Platforms meter different units, so one workload prices very differently. Langfuse gives 50,000 observations a month free, LangSmith 5,000 traces, Braintrust 1,000,000 spans, Datadog 40,000 LLM spans and nothing charged for tool or retrieval spans. All figures are snapshot readings from 9 August 2026 and need re-checking. Read the exit path before committing to any of them.

---

Compiled 9 August 2026. Issue counts, reaction totals, release versions and pricing were read from the linked public sources that day and will move. Check the issue tracker, the changelog, the releases page and each vendor’s own pricing page rather than trusting this snapshot. No product was installed, configured or measured for this article, and where a question needs hands-on measurement, such as span processor overhead or self-hosted resource baselines, that is stated rather than estimated. Treat this as a dated snapshot of 07 AI agent observability with Langfuse: the 10 observation types and the 7 self-host components will outlast the 657 reactions and the $39, $160 and $249 price points.