Braintrust vs Arize Phoenix in 2026 - Eval Platform or OSS Tracer?

These two get pitted against each other, but they are answering different questions. Braintrust asks “did this change make the model worse, and can I block the merge if it did.” Arize Phoenix asks “how do I trace and score my app for free, especially the RAG.” One is an eval-first commercial platform with a billing meter to watch. The other is free open-source tracing with a license footnote. Pick based on which question is keeping you up at night.

I have added Langfuse as the third option, because if your priority is open-source self-hosting with a genuinely permissive license, it splits the difference between these two.

The short version

Tool	Best for	Starting price	Self-host	License
Braintrust	Turnkey evals and CI regression gates	Free / $249/mo	Hybrid, Enterprise	SDKs open, backend closed
Arize Phoenix	Fast OSS tracing and RAG eval	Free (OSS)	Free, runs locally	ELv2 server
Langfuse	Open-source default, cheap at scale	Free / $29/mo	Free, near-complete	MIT

Braintrust: the turnkey eval and regression platform

Most tools in this space start as observability and add evals later. Braintrust started with evals and built observability around them, and it shows. The distinctive thing is that human review, automated scorers, LLM-as-judge, tracing, datasets and CI/CD quality gates all live in one system. The autoevals library ships working scorers - exact match, embedding similarity, LLM-as-judge factuality - and the CI gates can block a merge on a statistically significant regression, not just log that quality dropped. There is no per-seat charge - users are unlimited on every tier. It is well-backed too, with an $80M Series B in February 2026 at a roughly $800M valuation.

The gotcha is the billing meter. Braintrust bills “processed data” in GB, and it counts every byte - inputs, outputs, prompts, metadata, traces, spans, attachments. There is no hard spending cap, and the $0 Starter jumps straight to $249/mo with nothing in between. The trap is that verbose multi-step agents and large RAG contexts - the workloads that most need observability - burn the GB allowance fastest. The $249 is a floor, not a ceiling. Set billing alerts on day one, or the meter watches you.

Arize Phoenix: fast OSS tracing, the best RAG eval

Phoenix is the open-source pick that starts fastest - a working trace UI on your laptop in under a minute. It is genuinely OpenTelemetry-native, built on OTel and Arize’s own OpenInference conventions, so it is framework-agnostic rather than tied to one ecosystem. And its 50+ pre-built eval metrics include what reviewers call the best RAG evaluation in the category - serious retrieval and answer scoring without writing your own judge prompts. Phoenix OSS is free with no usage caps and no feature gates on the actual features. Arize the company is well-funded, with a $70M Series C in February 2025, so it is not going anywhere.

The gotcha is the license, and it is a real one. Arize markets Phoenix as “fully open source, no feature gates,” but the main server repo is Elastic License 2.0 - source-available, not OSI-approved open source. Only the client and eval subpackages are Apache-2.0. ELv2 forbids offering Phoenix as a hosted service to third parties. For internal use it behaves like open source and the features are not gated - but if your plan is to resell it as a service, read the license first. There are also reports of ingest lag before traces appear.

Langfuse: the middle path

If you want open-source self-hosting without either the closed Brainstore backend or the ELv2 restriction, Langfuse is the cleanest license of the three - MIT, genuinely OSI open source, self-hosts free with only three features gated. It is also roughly 25x cheaper than LangSmith at scale and runs about $101/mo at 1M events managed.

Its catch is operational. Langfuse v3 needs four services - Postgres plus ClickHouse, Redis and S3-compatible storage - and the migration bites. Its evals are less turnkey than Braintrust’s out of the box, and its RAG eval is not as deep as Phoenix’s. It is the balanced default, not the specialist.

Braintrust vs Arize Phoenix: which should you pick?

• Regression testing and blocking bad merges in CI is the job - Braintrust, with billing alerts on from day one. Nothing else is this complete out of the box.
• You want free open-source tracing you can run today, with strong RAG eval - Arize Phoenix, as long as you are not reselling it.
• RAG evaluation depth specifically is your priority - Arize Phoenix. Its RAG eval is the best around.
• You need OTel-native architecture - Arize Phoenix is built on OpenTelemetry from the ground up; Braintrust is an OTLP receiver.
• You want a permissive open-source license with no ELv2 or closed-backend asterisk - Langfuse if you can run the self-host stack.

The honest case for each: Braintrust genuinely earns the premium if CI quality gates are what you came for - just watch the uncapped meter. Phoenix is free, fast and capable for internal use, and the license only bites the narrow group who wanted to resell it.

For the wider field, see our Braintrust alternatives and Arize Phoenix alternatives breakdowns, or the head-to-head Langfuse vs Arize Phoenix. Every price and date here was read from each vendor’s own materials on 26 July 2026. This category ships breaking changes monthly, so we re-verify every 30 days.