MLflow Review (2026)

What changed with MLflow 3

MLflow spent years as the default open-source experiment tracker for classical machine learning. If your mental model is “the thing that logs training runs,” it is out of date.

MLflow 3 is a genuine GenAI platform. It unifies tracking, evaluation and observability across the development and production lifecycle - real-time trace logging, built-in and custom scorers, human feedback collection, and version tracking.

MLflow Tracing gives end-to-end observability into GenAI applications including complex agent systems. A single line of code captures prompts, retrievals, tool calls, responses, latency and token counts. It automatically instruments popular GenAI libraries or ingests traces directly, and offers interactive timeline views and side-by-side comparison.

Critically, tracing is OpenTelemetry-compatible with no vendor lock-in. Your trace data is standard telemetry, not a proprietary format you would have to re-instrument to escape.

The evaluation workflow is the underrated part

Most comparisons stop at “MLflow has evals.” The workflow around them is what makes it interesting.

You get built-in and custom LLM judges and scorers, so you can define what quality means for your use case rather than accepting generic metrics. Fine, everyone has that.

The differentiated piece is review apps. Domain experts review outputs and provide feedback. That feedback builds labelled evaluation datasets. Those datasets are then used to align the automated judges against expert judgment.

That loop - expert opinion becoming labelled data becoming a calibrated automatic judge - is the genuinely hard problem in LLM evaluation, and it is the difference between an LLM-as-judge score you trust and one you do not. Few competitors handle it this coherently.

You can also build evaluation datasets directly from production traces, which is the correct shape for continuous improvement: the failures your users actually hit become the regression suite.

Free, and genuinely complete

This needs saying plainly because the category is full of misleading self-hosting claims.

MLflow’s open-source build is a complete platform, not a gated teaser. Apache 2.0, full tracing, full evaluation, prompt registry, run comparison. Several commercial vendors advertise self-hosting and then withhold the useful half behind their cloud tier. MLflow does not.

What Databricks sells on top is operational and governance value - managed hosting, production-scale infrastructure, lakehouse integration, Unity Catalog governance. That is a legitimate commercial line to draw, and it means the free version is genuinely usable in production rather than a trial.

The honest cost of free is operational. You own the tracking server, the backing database, the artifact store, scaling, upgrades, backups and availability. At small scale that is a container and a Postgres instance. At high trace volume it is a system someone has to own, and trace data grows quickly. Budget engineering time, not licence fees - and be honest about whether that ownership will survive the person who set it up moving teams.

Where open source and Databricks diverge

This is the detail most comparisons miss, and it matters because the shared name hides it.

Databricks now recommends storing traces in Unity Catalog for new and production workloads. Traces land in OpenTelemetry Delta tables, which gives you three real things:

• No storage cap
• SQL queryability over raw trace data
• Unity Catalog-governed access control

That is arguably the best trace storage architecture in the entire category. It is also a Databricks capability. Self-hosted open-source MLflow does not get it.

So “MLflow” describes two meaningfully different experiences depending on where you run it, and the gap is in governance and scale rather than core features. If you are evaluating on the strength of the Unity Catalog story, be clear that you are evaluating Databricks.

Two smaller version notes worth knowing: open-source telemetry collection was introduced in MLflow 3.2.0 and is disabled on Databricks by default, and in MLflow 3 the Agent Evaluation SDK methods were integrated with Databricks-managed MLflow, which matters if you are migrating from the older SDK.

Where it is weak

The UI. It is functional rather than pleasant, and it looks dated next to Braintrust or Logfire. MLflow was built for ML engineers reading experiment tables, and application developers used to modern observability tooling notice the difference immediately.

The lineage shows. Some GenAI workflows still feel grafted onto an experiment-tracking substrate rather than designed for LLM applications from scratch. This is improving but it is visible.

Databricks stewardship is a soft dependency. The license is permissive and you can genuinely run independently, but roadmap priorities reflect a commercial sponsor. Compare with Langfuse, where the open-source project and the paid cloud are the same feature set.

Should you use it?

Use MLflow if you already run it for classical ML, you are a Databricks customer, or you want a complete free self-hosted platform and have the operational capacity to run it.

Don’t use it if you have no infrastructure capacity, UI quality matters a lot to your team’s adoption, or you only do LLM application work and would prefer a purpose-built tool.

Bottom line: the strongest genuinely-free option in the category, and unusually honest about what the open-source build includes. The evaluation loop - expert review to labelled dataset to aligned judge - is better than its reputation suggests. Pay for it in operational time rather than licence fees, and understand that the best-governed version of it lives on Databricks.

---

Features and architecture verified against MLflow and Databricks documentation on 31 July 2026. Managed pricing is bundled into Databricks consumption and is not published as a standalone rate. This is a researched directory entry - we have not yet instrumented this platform with our reference application.