The human isn't a failure of automation

There's an assumption buried in most conversations about AI at work: that a human in the process is a sign the automation didn't quite succeed. A temporary scaffold. Something to remove in version two.

Every production system we've built or studied says the opposite. The humans are still there, on purpose, in specific places — and the systems that work are the ones that were deliberate about where.

The question was never whether. It's where.

Three places a person belongs

In practice there are only three, and they're easy to tell apart because they answer different worries.

1. Before publication — the approval queue

The pattern comes from systems that generate content at volume. A model produces material in bulk, offline. Domain experts review it, correct it, accept or reject it. Only accepted material is promoted into the pool the live system is allowed to serve.

The crucial detail is the staging area. Generated material does not go straight into production — it lands in a holding area that only a human decision moves it out of. The live system can only ever serve from the approved pool.

This buys something a reviewer values more than accuracy: predictability. Whatever a customer sees was looked at by someone whose job it is to care.

It works when generation can happen ahead of time. It doesn't apply to a live conversation, which is where the second place comes in.

2. Before a consequential action — the approval gate

When an agent is about to do something in the real world — send the email, raise the invoice, move the money — the person approves the specific action before it happens.

We've described the mechanics of this before in letting an agent act as you, safely: gate writes and spend rather than reads, bind the approval to the person who gave it, and re-check it at the moment it runs. The design rule that makes it survivable is restraint. Gate everything and people rubber-stamp, which is worse than no gate at all, because now you have the illusion of oversight plus a signature.

3. After an uncertain answer — the escalation path

The third is the one most teams underbuild. When the system's own confidence is low, or a user reacts badly, the case goes to a person — and the handover carries the conversation so far, so the human doesn't start from nothing.

What makes this a system rather than a safety valve is measuring it. Escalation rate is a first-class metric: a sudden rise means the agent has started failing at something it used to handle, usually before any individual complaint arrives.

It's the rare metric that's useful in both directions. Too high and the agent isn't earning its place. Too low — with unhappy users — and it's confidently answering things it shouldn't.

The case for humans that isn't sentimental

We have a concrete reason to believe in this, and it isn't a principle. It's an incident.

One of our agents produced a document in which a closing section degenerated into four thousand words of repeated adverbs. Three automated checks examined that document and passed it. The third of those checks was another model, asked to review the first model's work, and it approved the wall of text because — from the inside — fluent and positive is what good writing looks like.

A person caught it.

We wrote up what we built so that arithmetic catches it next time, and that detector now works. But the sequence is the point: every automated check reported success, and the only thing standing between a broken document and a customer was someone reading it.

Automated checks catch the failures you anticipated. People catch the ones you didn't. You need both, and you will never finish moving things from the second category into the first.

Designing the human's job properly

Putting a person in the loop is easy. Putting them there in a way that still works in six months is not. Four things decide it.

Review has to be faster than doing the work. If checking the model's draft takes as long as writing it, the reviewer will start approving without reading. Show the diff, highlight what's uncertain, make accept-and-move-on one key.

Only interrupt for things that matter. Every unnecessary approval request spends attention you'll need later for a real one.

Give them what they need to decide. "Approve this?" is not a question anyone can answer. "Approve sending this email — drafted after reading an inbound message from an external address" is.

Feed the decisions back. Rejections are the highest-quality training signal you will ever get, and most teams throw them away. They tell you exactly where the system is weak, in the words of the person who noticed.

What this means commercially

Teams sometimes hide the human, on the theory that buyers want full automation. In our experience the opposite is true — and the people who ask the hardest questions are the ones who find a deliberate answer most reassuring.

"Nothing reaches your client without someone approving it" is a stronger sentence than "it's fully automated." It tells a buyer you've thought about being wrong, which is the thing they're actually trying to find out.

It also reframes what the agent is for. The win was never removing the expert. It's that the expert stops producing the first draft and starts reviewing the twentieth — doing the part that needs their judgement, at a volume they couldn't have reached by hand.

That's the version that survives contact with a real business. Not an agent nobody checks, but an expert whose attention has been moved to where it's worth the most.