Promptfoo vs Langfuse in 2026 - Which One You Actually Need

People line up Promptfoo against Langfuse a lot, and the comparison is slightly the wrong shape. These are not two versions of the same tool. Promptfoo is a test runner for LLMs. Langfuse is a trace store. One tells you whether a change is safe to ship. The other tells you what happened after you shipped it. You can absolutely pick one over the other, but the real question is usually “which job am I trying to do,” not “which product is better.”

I have run both in production stacks. Here is the honest split, where each one wins, and when you should stop choosing and just run both.

The short version

	Promptfoo	Langfuse
What it is	Config-driven eval and red-teaming CLI	Self-hostable observability backend
Primary job	Pre-ship testing and security probes	Production tracing and monitoring
Config style	Declarative YAML in version control	SDK instrumentation plus OTLP
License	MIT	MIT
Self-host	Free, local or self-hosted	Free, but four-service stack
Starting price	$0 (10k probe cap)	Free / $29/mo
Owner	OpenAI (acquired March 2026)	ClickHouse (acquired January 2026)

What Promptfoo is for: gating merges and red-teaming

Promptfoo is the de-facto open-source CLI for LLM eval and red-teaming, and its whole shape is built for CI. You describe your tests in declarative YAML that lives in version control next to your prompts, point it at your models, and it runs comparisons, assertions and vulnerability scans from the command line. Because the config is a file a reviewer can read in a pull request, evals stop being scattered Python scripts and start being reviewable artifacts.

The signature specialty is red-teaming, and it is the strongest in the category. Promptfoo ships 50+ attack plugins with built-in presets for the OWASP LLM Top 10, OWASP Agentic and NIST, probing for prompt injection, jailbreaks, PII leakage and more. If security testing is part of your eval story, nothing else in the open-source category is this complete.

The gotchas are ownership and caps. Promptfoo was acquired by OpenAI on 9 March 2026 and folded into OpenAI Frontier. Both sides state it stays open source under MIT, and that commitment is on the record - but long-term governance of an OSS project under a single large owner is a fair thing to weigh. The free tier is also capped at 10k red-team probes per month, and Enterprise pricing is contact-sales only.

What Langfuse is for: storing and querying production traces

Langfuse does the opposite job. It is an observability backend - you instrument your app, it captures traces, and you query them later. It covers tracing, evals, prompt management and human annotation, all framework-agnostic, and it runs as an OpenTelemetry backend receiving traces on an OTLP endpoint. Where Promptfoo answers “is this release good enough to ship,” Langfuse answers “what is my live app actually doing, and where did that bad response come from.”

Its real edge is self-hosting that does not cripple you. Only three features are enterprise-gated in the self-hosted MIT build - tracing, evals, prompt management, human annotation and RBAC are all free. And it is dramatically cheaper than the closed alternatives at scale, running about $101/mo at 1M events where LangSmith runs roughly $2,514/mo for comparable volume.

The catch is operational. Langfuse v3 needs Postgres plus ClickHouse, Redis and S3-compatible storage - four services to stand up, and the migration to that architecture is where self-hosters get stuck. It is also a ClickHouse subsidiary now, after the January 2026 acquisition. If you cannot spare the ops time, the $29/mo Core cloud tier sidesteps the whole stack.

Where DeepEval fits in this

If you are weighing Promptfoo against Langfuse because you want programmatic evals in CI, DeepEval is the third name you should have on the table. It is the pytest for LLM apps - SDK-first, Python-native, and it drops deepeval test run straight into an existing test suite. Promptfoo is config-driven YAML and red-teaming-led; DeepEval is code-driven and metrics-led, with 50+ research-backed metrics including G-Eval. The two are the real head-to-head for eval frameworks, and we cover that in DeepEval vs Promptfoo. Langfuse is not in that fight - it is the trace store you point either of them at.

Promptfoo vs Langfuse: which should you pick?

• You want to gate merges and red-team every release - Promptfoo. It is built for CI, the YAML lives in git, and the OWASP mapping is the best available. The free tier is the real product.
• You need a production trace store you query weeks later - Langfuse. Persistent retention, dashboards and framework-agnostic tracing are its whole job, and self-host is genuinely feature-complete.
• You want programmatic Python evals more than YAML or security probes - look at DeepEval instead, then still send its traces to Langfuse.
• You want a permissive license and no vendor lock-in on either - both are MIT, so this is a wash. Weigh the OpenAI and ClickHouse ownership questions equally.
• You are a serious team shipping to production - run both. Promptfoo before you ship, Langfuse after. They are complementary, not competing.

If you are still deciding on the observability half specifically, the Langfuse alternatives rundown and our best open-source LLM observability tools list go deeper on the trade-offs. For the eval half, best LLM eval frameworks is the place to start.

Every price and date here was read from each vendor’s own materials on 26 July 2026. This category ships breaking changes monthly, so we re-verify every 30 days. The one thing that will not change on you - Promptfoo tests before you ship and Langfuse watches after. Pick by the job, not the logo.