Before You Ship Your AI Agent

Did you build for the process that actually exists, or the one that's documented?

Check 1: The Documentation Gap

Pull your process SOP and walk through it with someone who does this work every day. Ask them to flag every step where they actually do something different from what the document says.

Red flag: They find deviations immediately. The workarounds are numerous, unwritten, and load-bearing. Your agent will encounter all of them.

Check 2: The Exception Rate

For every 100 transactions through your process, how many follow the standard path versus some variant?

Red flag: More than 20% hit an exception path. Agents designed only for the standard flow will fail roughly 1 in 5 times.

Check 3: The Stress Test User

Who runs this process under the worst conditions they face regularly? Not the power user in UAT. The person at hour 11 of a 14-hour shift, managing three other things simultaneously. Did you observe the process from their position, in the environment where the work actually happens?

Red flag: You don't know who this person is, or the process was never observed under real conditions.

Check 4: The Handoff Map

List every informal handoff in the process. The Slack messages, the phone calls, the "grab whoever's on tonight" moments that aren't in any workflow diagram.

Red flag: These handoffs are numerous, undocumented, and you don't know how the agent handles them. Salesforce's 2026 Connectivity Report found 50% of deployed enterprise agents operating in complete isolation, unable to share context across systems. This is usually why.

Did you design for accountability, or just completion?

Check 5: The Explainability Test

Pick 10 actions your agent is designed to take. For each one: can a human reconstruct what the agent saw, what it considered, and why it chose that action?

Red flag: More than two of those 10 are effectively black boxes. When something goes wrong in production, you won't be able to explain it to whoever asks.

Check 6: The Override Path

In your agent's interface, how does a human intervene, override, or correct an action in progress? Walk through the flow and time it.

Red flag: The override path takes more than 30 seconds, or doesn't exist for certain action types.

Check 7: The Uncertainty Behavior

When the agent hits missing data, ambiguous context, or an edge case, what does it do? Does it communicate uncertainty, escalate to a human, or proceed?

Red flag: It proceeds confidently in cases where a competent person would pause and check.

Check 8: The Audit Trail

After the agent completes an action, what record exists? Could you pull a complete log six months from now and answer: what did the agent see, what did it decide, and when?

Red flag: The log exists for debugging but couldn't support a compliance review or a client-facing explanation of why something happened.

Are you measuring what matters in production, or what's easy to measure in UAT?

Check 9: The Right Metric

Write down your success metric for the agent. Be specific.

Red flag: The metric is task completion rate, response time, or anything generated in a controlled test environment. These tell you nothing about whether the agent is actually helping the humans who work alongside it.

Check 10: The Incident Baseline

Before deployment, have you defined what counts as an incident? When the agent acts incorrectly, how will you know? Who decides? What's the escalation path?

Red flag: You haven't defined this. Incidents will get classified as "edge cases" or "working as intended" in ways that obscure the pattern until it's expensive.

Check 11: The Six-Week Review

Do you have a structured review planned for six weeks post-deployment, with specific metrics you'll measure that you can't measure today?

Red flag: Monitoring is continuous but unstructured. You'll have data and no frame for interpreting it.

0 to 2 red flags: You're probably ready. Watch checks 9 and 11 closely in the first month.

3 to 5 red flags: Go back to whichever phase the red flags cluster in. That phase needs work before you ship.

6 or more: You're in demo mode. This agent is not production-ready.

The current wave of enterprise agent development is genuinely impressive. Teams using Claude Code are shipping internal tooling in days that used to take months. MCP servers connecting Claude to internal systems, agents automating repetitive engineering work, customer-facing bots handling tier-1 support with actual reasoning instead of decision trees. The capability jump is real.

What they're skipping: checks 5 through 8 of this audit. Almost universally.

The interface for humans to work alongside the agent gets zero design attention. The agent does the work. The human experience of monitoring what the agent is doing, understanding why, overriding when needed, and auditing afterward is treated as an afterthought. You end up with technically impressive agents that the people supposed to use them don't trust, can't properly oversee, and quietly route around.

This is the same pattern we documented in the clinical decision support case two editions ago.

That system had 11% physician adoption (full case at thirtytwonorth.com/case-study/clinical-decision-support-system). The AI was good. The interface gave physicians no way to understand the AI's reasoning in 90 seconds under pressure, no clear override path, no decision trail they could explain to a review board. After the redesign: 68% adoption in four months. Same AI. Checks 5 through 8 done properly.

The same gap is showing up now in every agentic deployment that treats those four checks as optional.

One thing has changed. In advisory AI, the misread shows as low adoption. Someone notices, the team goes back in, the system gets fixed. In agentic AI, the agent has already acted before anyone knows something went wrong. The message was sent, the record was modified, the downstream process was triggered. The gap between the documented process and the actual one doesn't produce low adoption numbers anymore. It produces incidents.

Two comments from the Edition 5 thread worth putting in the record here.

Ricky Jones:

"The important move is that stability stops coming from internal repair cycles and starts coming from the constraint surface itself. Same ocean, different control logic."

In advisory AI, the physician who ignored the CDSS recommendation was an internal repair cycle. A human correcting the system's misread in real time. Agentic systems mostly don't have that moment. The repair has to move upstream, into checks 1 through 4, before deployment. Leave the constraint surface wrong and you're not running repair cycles. You're running incident response.

Richard Lynes:

"If the production reality is misread at the human layer, agentic systems do not fix that. They scale the misread. That is why the spreadsheet, the workaround, the quiet pause in the room, and the 3am user all matter more than the brief."

The workarounds that signal adoption problems in advisory AI are the same things that map the exception paths your agent will hit autonomously. If checks 1 through 4 don't surface them, the agent will in production.

Run it on the agentic deployment you're planning or the one already in progress. Score it honestly. Share it with whoever owns the deployment decision.

If you hit 5 or more red flags: reply to this email with REALITY. I'll send a deeper version of the audit with specific guidance on each red flag and how to work through it before you ship. No pitch, no follow-up sequence.

If you want someone to run the audit with your team: thirtytwonorth.com/work shows the kind of work we've done in this space.

Six editions. Started with a question about why AI adoption fails and ended at the same question one layer deeper, with higher stakes on the answer.

The audit is the practical version of everything this series covered. Save it. Run it. Share it with the person on your team about to ship an agent.