Promptfoo Review (2026)

What Promptfoo actually is

Promptfoo is an open-source command-line tool for two jobs - evaluating LLM apps and red-teaming them for security. You describe your tests in declarative YAML, point it at your prompts and models, and it runs comparisons, assertions and vulnerability scans from the CLI or in CI. It is MIT-licensed, it has roughly 23.5k GitHub stars - the most of the major eval tools - and OpenAI/vendor figures put it at 350k+ developers.

The headline you need first, though, is the ownership. OpenAI announced the acquisition of Promptfoo on 9 March 2026. Both OpenAI and Promptfoo state the project stays open source under its current MIT license, existing customers keep being supported, and the technology folds into OpenAI’s “Frontier” work on agentic security testing. That commitment is on the record. What it is worth to you is a judgment call - you are now betting on OpenAI’s stewardship of an OSS project, and long-term governance under a single large owner is a fair thing to weigh before you build a multi-year workflow on it.

Promptfoo was co-founded in 2024 by Ian Webster and Michael D’Angelo. It raised a $5M seed from a16z, then an $18.4M Series A led by Insight Partners in July 2025 - roughly $23.4M in total before the acquisition.

The distinctive part: YAML config plus red-teaming

Two things set Promptfoo apart from the pytest-style crowd.

It is config-driven, not SDK-first. Your evals are declarative YAML that lives in version control next to your prompts. You are not scattering assertions through Python test files - you are describing what “good” looks like in one place a reviewer can read in a pull request. Security and CI teams tend to love this shape; engineers who want programmatic Python test suites tend to prefer DeepEval’s approach. It is a genuine fork in the road, not a better-or-worse thing.

Red-teaming is the signature specialty. This is where Promptfoo pulls ahead of general eval tools. It ships 50+ attack plugins and built-in presets for the OWASP LLM Top 10, plus OWASP API Top 10, OWASP Agentic and NIST. It probes for harmful content across 20+ subcategories, PII leakage, prompt injection, jailbreaks, excessive agency, hallucination and competitor mentions, and you can write custom policy probes. If security testing is part of your eval story, this is the tool built for it.

There is a third detail worth flagging: Promptfoo acts as its own OTLP receiver with a built-in trace viewer. You send OpenTelemetry traces and Promptfoo displays them - no separate Jaeger or Tempo to run. One caveat from the tracker: issue #7333 reports some agent-SDK providers do not export traces to the receiver, so verify your stack works before you rely on it.

Pricing: free where it counts, opaque above it

Tier	Price	What you get	Cap
Community	$0	All core eval, every provider, red-teaming, self-host	10k red-team probes/mo
Enterprise	Contact sales	Dashboards, team controls, monitoring, SSO, compliance, SLA	Custom
On-Premise	Contact sales	Enterprise features inside your own network, air-gapped	Custom

The free tier is the real product for most people. Community is MIT-licensed and free forever, with all core eval and red-teaming included - the only hard limit is 10k red-team probes per month. Gated behind Enterprise are the things bigger teams need: centralized dashboards, team management, continuous monitoring, custom plugins, compliance frameworks and a dedicated runner.

The honest gap: Enterprise and On-Premise have no public pricing - both route to “contact sales.” We could not find a figure, so we are not going to invent one. If you are budgeting, assume a sales conversation.

Promptfoo versus DeepEval

These are the two names people compare, and the split is clean. Promptfoo is YAML-config, red-teaming-led, and free with no seat cost. DeepEval is pytest-style, metrics-led, and pushes you toward the Confident AI cloud at $200/mo for real collaboration. If your team thinks in security tests and CI gates, Promptfoo fits the way you already work. If your team writes Python and wants research-backed scoring metrics like G-Eval, DeepEval is the more natural home.

One more difference that matters post-2026: Promptfoo is now an OpenAI company, while DeepEval’s Confident AI is an independent YC startup. If vendor independence is part of your calculus, that is a point on the board for DeepEval.

Should you use it?

Use Promptfoo if red-teaming or security testing is part of your eval work, you want your evals in version control as declarative config, or you want a genuinely capable CLI with no seat cost. For OWASP-mapped LLM security scanning specifically, nothing else in the open-source category is this complete.

Think twice if you want programmatic Python test suites - DeepEval is more ergonomic for that - or if long-term OSS governance under OpenAI is a dealbreaker for your organization. Neither is a reason not to try it, but both are worth naming before you commit.

Bottom line: it is the strongest open-source eval-plus-red-teaming tool available, the free tier is the real thing, and the OpenAI acquisition is a fact to weigh rather than a reason to walk away. Start with Community and see how far the probe cap gets you.

---

Pricing and features verified against promptfoo.dev on 23 July 2026. This category ships breaking changes monthly - we re-verify every 30 days.