What an AI Agent Audit Covers, and What It Should Cost

An AI agent audit is a short, focused engagement that answers one question with evidence: is your agent actually working in production? A real one covers four things. First, a review of the architecture, prompts, and tool definitions, because most agent failures are designed in, not random. Second, failure analysis on real production traffic, not on a demo dataset. Third, a small set of evaluations left running after the auditor leaves, so the findings become a regression safety net instead of a snapshot. Fourth, a written findings document plus a working session with your team. For a single production agent, that is days of work, not months, and it should be priced accordingly: fixed, in the low thousands of euros, at the bottom of the wider AI-audit market rather than the top.

What exactly gets audited?

The design: architecture, prompts, tool calls. The auditor reads the system the way a security reviewer reads code: what the agent is told, which tools it can call, what the tool schemas allow, how results flow back into context. This is where the structural risks live, an over-permissive tool, a prompt that quietly accumulates history, a retry policy that can loop. My essay on the five recurring failure patterns, Your AI Agent Has the Same Red Flags as Your Ex, is essentially a catalog of what this review looks for.

The behavior: failure analysis on real traffic. This is the part that separates an audit from a code review. The auditor pulls real production runs and reads them end to end: where answers were not grounded in what the tools returned, where the wrong tool fired, where token spend grew without anyone noticing. If a vendor proposes to audit your agent without looking at production traces, you are buying a slide deck.

The safety net: evaluations that stay. Findings age fast; the next model upgrade or prompt tweak can undo a fix silently. A useful audit converts its top findings into three to five automated evaluations that keep scoring production behavior after the engagement ends. I wrote about the checks that matter most in How to Tell If Your AI Agent Is Actually Working.

The transfer: findings and a team session. A written document your leadership can read, and a working session where the team sees the traces behind each finding. The goal is that your own engineers can keep doing this without the auditor.

How is this different from an AI readiness audit?

They share a word and not much else. A readiness audit is an adoption-planning exercise for companies deciding where AI should go: workflow inventory, opportunity scoring, data and team readiness, as guides to general AI audits describe. An agent audit is a technical trust exercise for something you have already shipped. If you have no AI in production yet, you want the first kind. If you have an agent answering customers today and nobody can prove it behaves, you want the second.

What should it cost?

Published benchmarks for AI audits put narrow small-business engagements at roughly $2,000 to $8,000, mid-market audits at $5,000 to $15,000, and enterprise assessments at $15,000 to $50,000 or more. Those ranges describe broad, whole-organization assessments. A focused audit of one production agent is a smaller, sharper job, and it should sit at or below the bottom of that market: low thousands, fixed price, delivered in weeks.

Two pricing red flags. Open-ended day rates without a defined deliverable tend to grow into the engagement the consultant needs rather than the one you need. And a price that looks like the enterprise end of the range for a single agent usually means you are paying for compliance framework theater rather than trace-level analysis.

Worth naming plainly, since this is also what I sell: my Agent Trust Sprint is exactly this audit, 2 weeks at about 4 working days, EUR 2,400 fixed.

What should you demand in the deliverable?

Ask every prospective auditor four questions. Will you analyze real production traffic, and what access do you need for that? Which evaluations will still be running for us after you leave? Can we replay the failing runs you find, ourselves, afterwards? And is the price fixed against a written scope? A "no" on any of these is a sign you are buying advice rather than evidence.

Access-wise, a serious auditor needs read access to your traces or logs, your prompts and tool definitions, and a sample of production conversations with anything sensitive redacted. They do not need write access to your codebase, and they do not need your customer database.

FAQ

Can't our own engineers just do this? The mechanics, yes, and a good audit should leave them able to. What an outside auditor adds is pattern recognition across many agents, no attachment to the design decisions being reviewed, and the political cover of a neutral finding when the news is bad.

How long should it take? For one production agent: one to two weeks of calendar time, a handful of working days of effort. Longer usually means broader scope, an organization-wide assessment, or padding.

Is it worth auditing before launch? A lighter version, yes: the design review and the evaluations can be built pre-launch. The failure analysis needs real traffic, so plan a follow-up pass a few weeks after going live.

What if the audit finds nothing serious? Then you bought proof, which is the product. "We looked at real traffic and the agent behaves, here are the evals that will tell us if that changes" is exactly the sentence your leadership wants to be able to say.