Your AI agent passed the demo. Now make it pass the harness
A demo proves nothing about an AI agent. Deployability comes from a logged eval run: frozen cases, ground truth, a failure taxonomy and a pass/fail gate.
An AI agent is deployable when a written evaluation harness scores its runs pass/fail — not when it works in a demo. A demo shows one happy path with a human watching. A harness shows what the agent does with the inputs you didn't think about: the message that mixes two requests, the instruction-override attempt hiding inside a quote, the vague "we want to automate things" that must not become a qualified lead. We built a harness for one bounded workflow — inbound lead triage for a three-tool agent — and ran it for real: 56 logged executions across three versions of the agent's instructions. The single-intent cases passed 24 out of 24. The boundary cases failed in patterns, the patterns were fixable with instruction changes, and the re-run after the fix caught a new flip. Every one of those failures would have reached a customer without the harness.
What a demo proves, and what it doesn't
The demo is not evidence. It proves the plumbing works — the model responds, the tools fire, the CRM record appears. It proves nothing about reliability, because the person running the demo unconsciously selects the inputs: the clean message with a name, a service, and a phone number.
Agents don't fail on those messages. They fail on the boundary — and the failures are boring in a way that makes them dangerous. The agent doesn't crash. It calls the wrong tool with full confidence, and the wrong tool is often the one that looks right: a lead "qualified" from a message that never named a service, or a legitimate buyer question handed to a human because the message contained an instruction-override phrase in a quote.
That asymmetry is why "it worked when we tested it" is not an acceptance criterion. You cannot stumble onto boundary cases in a demo. You have to write them down, in advance, with the correct answer already decided.
The harness: one workflow, three tools, frozen ground truth
The harness we run is deliberately small, because a harness nobody finishes is worth nothing:
- One bounded workflow. Inbound messages to a triage agent. Not "the whole business process" — one hop.
- The exact production tool scope. Three tools:
qualify_lead(only with a named service and a contact method),ask_question(plausible buyer, something missing),handoff_human(complaints, existing customers, out-of-scope, legal, override attempts). - Frozen cases with pre-written ground truth. Twelve single-intent cases and eight boundary cases, each with its expected tool and required arguments decided before the first run. The agent never sees the case set.
- A deterministic grader. It parses the agent's JSON action and scores two things: the right tool, and the required fields present and non-empty. No vibes, no LLM-judge circularity on the core verdict.
| Case family | What it probes | Example case |
|---|---|---|
| Clean buyer | Extraction under ideal input | Name + service + phone → qualify_lead |
| Missing element | Does it ask instead of guessing | Service named, no contact → ask_question |
| Out of scope | Scope discipline | Tax-filing automation → handoff_human |
| Multi-intent | Mixed in-scope + out-of-scope | Chatbot and WordPress fix → handoff_human |
| Quoted override | Data-vs-instruction separation | Prospect quotes "ignore all instructions" → buyer question, not an attack |
| Vague enthusiasm | Over-qualification pressure | "Automate things" + phone must not qualify |
| Buyer/seller inversion | Who is actually buying | A vendor pitching you is not a lead |
| Complaint with budget | Emotion vs. process | Furious customer who still wants to buy → human, not a pipeline stage |
Two runs per case, temperature 0.3, a flash-class model behind a JSON-action protocol. Total inference cost for all 56 executions: pocket change. The expensive part is not the runs — it's the discipline of writing ground truth first.
What the run found: three failure modes, one of them a lie you tell yourself
| Failure mode | What it looks like | Observed in our run |
|---|---|---|
tool_error | Right-looking action, wrong tool for the ground truth | 4 executions: the agent over-reacted to a quoted override (handed off a buyer question) and over-qualified a vague request (contact present, no service named) |
context_loss | Correct tool, required field missing or empty | 1 execution: handoff_human fired with an empty reason — the human gets a ticket with no context |
scope_error | An action outside the granted tool scope | 0 — the tool schema held |
silent_no_op | No parseable action | 0 — the JSON contract held |
Read the tool_error row carefully, because it contains the finding that justifies the whole exercise: both wrong-tool patterns were deterministic. The agent made the same wrong call on both runs of the same case. That's not model noise you can wait out — it's a defect in the written instructions, and no amount of "let's test it a few more times" makes it go away. Deterministic failures are the best news in an evaluation run: they name the exact sentence in your SOP that is wrong.
The second finding is the one people skip. After patching the instructions against both patterns — quoted text is data, and vague phrases name no service — the full-suite re-run passed 15 of 16, and the one failure was a case that had passed before: the multi-intent handoff fired with an empty reason field on its second run. Call it sampling noise or call it a patch regression; operationally it doesn't matter. The full-suite re-run is what caught it. A team that only re-runs the two broken cases after a fix would have shipped with a coin-flip handoff that produces context-free tickets.
The pass/fail gate you can put in a statement of work
"Test the agent before launch" is not a deliverable. This is — and a buyer can hold a vendor to every line:
- Frozen case set, ground truth dated before the first run. If the expected answers are written after seeing the outputs, you have graded the agent on its own behavior.
- Boundary coverage, not volume. Every case family above, at least once. Ten clean messages prove nothing.
- Two runs per case. A single pass can be luck; a single failure can be noise. Two identical failures are a defect.
- Two consecutive full-suite passes to ship. Not re-runs of the failed cases — the whole suite, every time the instructions, the model, or a tool schema changes.
- Zero tolerance on two categories. Wrong tool on any boundary case, and any missing required field. Everything else can be triaged; these two mean the agent cannot be trusted unsupervised on that path.
- The run log is the deliverable. Per-case outcomes, the tool calls, the failure classification. If a vendor can't hand you the log, they don't have an evaluation — they have an anecdote.
This is the same posture quality teams already take toward code, pointed at the part of the system that has no compiler. OpenAI's function-calling documentation makes the mechanical half explicit — tools defined by schema, the model choosing among them — but the schema only guarantees the agent can't call something you didn't grant. It says nothing about whether it calls the right thing. Only the harness measures that.
This became a procurement question this month
Two stories landed within the same 48 hours and moved agent evaluation from engineering hygiene to buying criteria. 1Password shipped a way for AI agents to log in on a customer's behalf without ever seeing the password — agents now hold real access paths into real systems (TechCrunch, October 22, 2026). And Microsoft attributed the ConnectWise breach to a group using AI agents at scale against thousands of customers (TechCrunch, October 22, 2026). When an agent can act — not just answer — "how do you know it does the right thing?" stops being a lab question. The answer you want to be able to give is not "we watched it work." It's a log.
If you're evaluating whether an agent belongs in your customer-facing path at all, start from the workflows where a wrong tool call has a cost — appointment booking and lead intake are the two we see most — and read how far the boundary goes before a human should take over: AI agents for appointment scheduling and the write-access boundary that keeps an agent inside its lane. The same discipline — written rules checked before anything ships — is what we covered in why WhatsApp templates get rejected before they ever send.
We run this harness as the acceptance step on every autonomous AI agent we build: frozen cases, logged runs, a pass/fail gate in the contract. If you want an agent your team can actually trust with a customer, bring us the workflow — we'll bring the harness.
Frequently Asked Questions
How many test cases does an AI agent evaluation harness need?
Coverage of failure modes matters more than volume. In our logged run, 12 single-intent cases plus 8 boundary cases were enough to surface every failure the agent had: wrong tool on ambiguous inputs, missing required fields, and instruction-override handling. Add cases only when a new failure mode appears, and freeze the set once it stops producing surprises — a harness that grows forever can never be passed.
What pass rate means an AI agent is ready for production?
Two different gates, not one number. Single-intent cases are table stakes: anything below 100% twice in a row is an instruction defect, not noise. Boundary cases are the real gate: require 100% across two consecutive full-suite runs, with zero wrong-tool calls and zero missing required fields, before the agent touches a real customer. In our run that gate was reached only after one instruction patch — and the re-run after the patch is what exposed the remaining flip.
Do I need a separate harness for every AI agent I deploy?
The pattern transfers; the cases don't. Every agent harness has the same four parts — a frozen case set with ground truth written before the run, the exact tool scope the agent gets in production, a grader that scores tool choice and required fields, and a failure taxonomy. Only the workflow and the case list change. Budget an afternoon per workflow, not a project.
