AI Agent Testing - A Practical Engineering Playbook (2026)

Most guides on testing AI agents stop at “write some evals.” That is not a playbook, it is a slogan. An agent is non-deterministic and multi-step - the same input can take three different paths and produce three different-but-valid answers - so the testing discipline is genuinely different from normal software, and the tooling has to match. Here is the playbook I actually use, in the order I run it.

The core mindset: you are not asserting that output equals a golden string. You are testing a system in layers, running each case several times to see the distribution, and scoring quality with a rubric where there is no single right answer. Get that framing right and the tools slot in easily.

Step 1: Separate the layers you are testing

Do not test the agent as one black box. Break it into four layers and test each on its own, cheapest and most deterministic first:

1. Tool correctness. Did the agent call the right tool with the right arguments? This is deterministic - assert on the tool name and the parsed arguments. No LLM judge needed, so it is fast and free. Test it in isolation.
2. Trajectory. Did the agent take a sensible path - not loop, not stall, not call the same tool ten times? Score the sequence of steps, not just the end.
3. Output quality. Is the final answer correct, faithful to its sources, and safe? This is where you need a rubric and a judge, because there is no golden string.
4. Regression. Did this change make any of the above worse than the last version? This is the layer that decides whether you ship.

The mistake is jumping straight to layer three and running an expensive LLM-as-judge over everything. Most bugs live in layer one, and layer one costs nothing to test.

Step 2: Build the test set from real failures

Your test cases should come from your production traces and actual failure reports, not from your imagination. The happy path is not where agents break - the weird multi-turn conversation, the ambiguous request, the tool that returned an empty result, that is where they break.

So the raw material is your traces. Capture every real run with an observability layer, mine it for the paths that failed or looked wrong, and turn those into fixed test cases. Langfuse is the open-source default for this - tracing, datasets and human annotation queues, self-hosted free under MIT, so your failure cases and the data behind them never leave your infrastructure. It runs as an OpenTelemetry backend, so it ingests trajectories from any agent framework. The catch is operational: the v3 self-host is four services (Postgres, ClickHouse, Redis, S3-compatible storage), and the migration is where people get stuck - the $29/mo cloud tier sidesteps that. Every case you promote from a real trace into your dataset is a bug that cannot silently come back.

Step 3: Simulate multi-turn users before production

Real failures are a great seed set, but you do not want to discover every failure in production. This is where pre-release simulation earns its place. Instead of waiting for a live user to find the conversation path that breaks your agent, you generate that path synthetically and test against it first.

Maxim is built for exactly this. Its agent simulation generates realistic multi-turn user interactions across thousands of scenarios and personas, so you stress-test a multi-turn agent before it touches live traffic - a genuinely different workflow from “ship it and watch.” It also carries four SDKs (Python, TypeScript, Java, Go) and takes OpenTelemetry. The honest gotcha is the cost model: Maxim charges per seat and caps logs, so both meters run at once - Professional is $29/seat/mo with a 100k-log cap then $1 per 10k over, and a five-engineer team is $145/mo in seats before a single log. Self-host is Enterprise-only, no open-source build. DeepEval also offers multi-turn conversation simulation inside its framework if you would rather keep simulation in code - see Step 4.

Step 4: Score with the right method, in code

Now the scoring. Use the cheapest method the task allows, and only reach for an LLM judge when you have to.

For a Python team that thinks in tests, DeepEval is the most natural fit - it is pytest for LLM apps. You write test cases, use assert_test, and run deepeval test run from the CLI or CI. Under it sits a library of 50-plus research-backed metrics including G-Eval (LLM-as-judge with custom criteria), plus hallucination, faithfulness, answer-relevancy and agent metrics, and DeepTeam for red-teaming across 50-plus vulnerabilities. The OSS framework is Apache-2.0 and fully usable with no account. The gotcha you must plan around: nearly all the metrics are LLM-as-judge, so every case fires another model call - runs take minutes not seconds, and big suites rack up real API bills. Keep deterministic checks (tool-argument assertions, exact match, regex) for everything they cover, and reserve the judge for genuinely open-ended quality. The judge mechanics are in our LLM-as-a-judge guide.

Step 5: Gate it in CI

Testing that does not block a bad merge is just logging. The point of the suite is to stop a regression from shipping.

Braintrust is the most turnkey option here - its CI/CD quality gates can block a merge on a statistically significant regression, not just record that quality dropped. Its autoevals library ships working scorers out of the box (exact match, embedding similarity, LLM-judge factuality), and evals, tracing, datasets and human review share one system, with no per-seat charge. The gotcha is the billing meter: it counts “processed data” in GB - every byte of inputs, outputs, prompts and metadata - with no hard cap, and verbose multi-step agents are exactly the workloads that burn it fastest. Set billing alerts on day one. If you would rather keep the gate in your own pipeline, DeepEval’s deepeval test run slots into an existing CI job directly. Either way, keep the CI suite tight and run the exhaustive suite nightly - the mechanics are in how to run LLM regression tests.

Step 6: Keep testing in production

Offline tests catch what you thought to test for. Production catches what you did not. Trace every real run, run online evaluation on a sample of live traffic, and feed the failures back into the Step 2 dataset so the suite compounds. Langfuse and Opik both run online evaluation over production traces; the loop closes when today’s production failure becomes tomorrow’s fixed test case.

The short version

• Layer your tests - tool calls (deterministic), trajectory, output quality (judge), regression (the gate).
• Seed the suite from real failures, capture them with Langfuse.
• Simulate multi-turn users before prod with Maxim, watching the double meter.
• Score in code with DeepEval, deterministic checks first, judge only where needed.
• Gate the merge with Braintrust, billing alerts on.
• Keep testing live and feed failures back into the set.

For the tool landscape behind this, see best LLM eval frameworks, best AI agent observability tools, and if you are building the scoring harness itself, how to benchmark AI agents.

Every price and date here was read from each vendor’s own pages on 26 July 2026, and this category ships breaking changes monthly - we re-verify every 30 days.