How-To
Step-by-step implementation guides.
30 posts
How to Detect Prompt Injection in 2026 - Guardrails, Eval Tests and Red-Teaming
A practical guide to detecting direct and indirect prompt injection - input and output guardrails at the gateway, eval tests that catch it in CI, and red-teaming that finds the attacks you did not think of. With the tools that fit and their trade-offs.
How to Evaluate AutoGen Agents in 2026 - Multi-Turn Runs, Loop Convergence and Termination
A practical guide to evaluating Microsoft AutoGen conversational multi-agent runs - score whether the conversation converged, terminated cleanly, stayed on task, and produced a correct result, so you can tell a productive loop from an infinite one.
How to Evaluate CrewAI Agents in 2026 - Task Completion, Handoffs and Per-Agent Scoring
A practical guide to evaluating CrewAI multi-agent crews - score the crew's final output, each agent's task completion, the handoffs between them, and the tool calls, so you know which agent to fix. With the tools that fit and their trade-offs.
How to Evaluate LLM Summarization in 2026 - A Practical Guide
A good summary is faithful, complete and concise all at once - and ROUGE measures none of that well. Here is how to build a real summarization eval with coverage, conciseness and faithfulness scorers, why n-gram metrics fail, and the tools that ship the judges.
How to Evaluate Multi-Turn Conversations in LLM Apps (2026)
Single-turn evals miss the failures that only show up over a dialogue - lost context, forgotten constraints, goals that never close. Here is how to score a whole conversation, turn by turn and end to end, and build multi-turn test cases with the tools that fit.
How to Evaluate RAG Chunking in 2026 - Test Chunk Size and Strategy
Chunk size, overlap and strategy quietly decide whether your RAG system retrieves the right context - and most teams tune them by vibes. Here is how to measure chunking impact with retrieval metrics, run a proper sweep, and pick settings on evidence instead of guesswork.
How to Generate Synthetic Data for LLM Evaluation in 2026
No labeled eval set is the most common reason teams never start evaluating. Synthetic data fixes that - generate golden test cases from your own documents with an LLM. Here is how to do it well, how to avoid the quality traps, and the tools that ship a synthesizer.
How to Measure Tool-Calling Accuracy in AI Agents (2026)
Tool-calling accuracy is not one number - it is three questions. Did the agent pick the right tool, pass the right arguments, and call them in the right order? Here is how to decompose it, score each part deterministically, and wire it into CI, with the tools that fit each step.
How to Red-Team an LLM in 2026 - A Step-by-Step Workflow
Red-teaming an LLM is not random prompt-poking - it is a repeatable pipeline of an attack taxonomy, an adversarial dataset and automated scans you rerun on every change. Here is the exact workflow, plus the two OSS tools that ship the attacks so you are not inventing jailbreaks by hand.
How to Reduce LLM Hallucinations in 2026 - Techniques That Actually Work
Measuring hallucination tells you how bad it is - reducing it is a different job. Here are the grounding, retrieval, decoding and guardrail techniques that actually lower the rate, ranked by impact, plus how to prove each change worked with an eval.
How to Trace the Anthropic Claude API in 2026 - Three Ways to Add Observability and Cost Tracking
Add tracing, token accounting and cost tracking to Anthropic Claude API calls three ways - a decorator around your call, a proxy gateway, and OpenTelemetry - with working setup for each and which one to pick.
How to Trace LangGraph Agents in 2026 - Node-Level Spans, Loops and Failure Debugging
A practical guide to tracing LangGraph state machines - get one span per node, see the state at every edge, catch runaway loops, and pin down which node actually failed. With the tools that fit and their honest trade-offs.
How to Trace a LlamaIndex RAG App in 2026 - Three Ways, With Setup
Trace a LlamaIndex pipeline end-to-end - retrieval, reranking, and generation as nested spans - three ways - a native callback handler, OpenTelemetry via OpenInference, and eval hooks that attach RAG scores to spans.
AI Agent Testing - A Practical Engineering Playbook (2026)
A real, step-by-step playbook for testing AI agents - separate the layers, build a test set from actual failures, simulate multi-turn users before prod, score with the right method, gate it in CI, and keep testing in production. With the tools that fit each step, and the honest gotcha for each.
How to Build an LLM Eval Pipeline in 2026 - A Practical Guide
A working LLM eval pipeline is datasets, scorers, a CI gate and production traces feeding back in - not a one-off notebook. Here is how to build each piece, with the tools that ship the parts so you assemble less from scratch.
How to Evaluate AI Agents in 2026 - A Vendor-Neutral Guide
A practical, tool-inclusive guide to evaluating AI agents - define what good means before you touch a metric, pick the right scoring method, build a labeled eval set, calibrate your LLM judge against humans, and measure the distribution not one lucky run. With the tools that fit each step and their honest trade-offs.
How to Evaluate LLM Applications in 2026 - A Practical Guide
A working playbook for evaluating LLM apps - build a dataset, pick metrics that match the failure mode, run evals in CI, and watch production. With the tools that fit each step, and the traps that make eval scores lie.
How to Evaluate a RAG System in 2026 - A Practical Step-by-Step Guide
RAG breaks in two places - retrieval and generation - and you have to measure them separately. Here is the exact workflow I use to score a RAG pipeline, the metrics that matter, and the three tools I reach for.
How to Measure LLM Hallucination in 2026 - A Practical Guide
Hallucination is not one metric - it is faithfulness, answer relevancy and factuality, each measured differently. Here is how to actually score it, with the eval tools that ship the metrics so you do not write judge prompts from scratch.
How to Monitor an LLM in Production in 2026 - The Full Workflow
Production LLM monitoring is not just uptime and latency - you have to score output quality on live traffic too. Here is the end-to-end workflow, the metrics that matter, and the three tools I trust for it.
How to Reduce LLM Costs in 2026 - 6 Levers That Actually Move the Bill
Most LLM bills are 30 to 70 percent waste - the wrong model on easy calls, no caching, and bloated context. Here are the six levers that actually cut spend, in the order I pull them, plus the tools for each.
LLM Regression Testing in 2026 - How to Catch Quality Drops Before They Ship
LLM outputs are non-deterministic, so classic regression testing does not work out of the box. Here is how to build a regression suite that catches quality drops before they ship - a fixed test set, the right scorers, and a CI gate - with the three tools I use.
How to Self-Host Langfuse in 2026 - The Honest Setup Guide
Langfuse self-hosting is free under MIT and genuinely feature-complete - but v3 is four services now, and the ClickHouse migration is where people get stuck. Here is the real setup path, what breaks, and when to just pay for cloud instead.
How to Set Up LLM Tracing in 2026 - A Practical Guide
A step-by-step guide to instrumenting your LLM app with tracing - what a trace actually captures, how to wire up Langfuse, Opik or Arize Phoenix in an afternoon, and the mistakes that make traces useless.
How to Trace OpenAI API Calls in 2026 - Three Ways, Ranked
The three ways to trace OpenAI SDK calls - a drop-in SDK wrapper, a proxy base-URL swap, and OpenTelemetry - with working setup for each, and which tool to use for which. One of them is a dead end in 2026.
How to Version Prompts in 2026 - A Practical Guide for LLM Teams
A prompt is code, and a one-word change can wreck your outputs. Here is how to version prompts properly - decouple them from deploys, tie every version to eval scores, and roll back in seconds - plus the three tools that make it easy.
How to Integrate Langfuse with LangChain in 2026 - A Practical Guide
Wire Langfuse tracing into a LangChain or LangGraph app with a callback handler, see every chain and tool call in the dashboard, and add evals - plus the self-host gotcha and when Opik is the cheaper managed pick.
LLM as a Judge in 2026 - A Practical Guide That Actually Works
LLM-as-a-judge is how most teams score AI output at scale, but naive judges are unreliable and expensive. Here is how to write a judge prompt, calibrate it against humans, control the cost, and the tools that ship working judges out of the box.
LLM Observability Best Practices for 2026 - 8 Rules That Save You a Rewrite
The eight LLM observability practices I wish I had followed on day one - trace the whole request, standardize on OpenTelemetry, score quality instead of logging it, and watch the retention meter before it watches you.
OpenTelemetry for LLM Observability in 2026 - A Practical Guide
How to use OpenTelemetry for LLM apps without locking yourself to one vendor - what OTel-native actually means, the GenAI semantic conventions, and how Phoenix, Langfuse and Opik differ on OTel support in ways that matter.