Sterile Cockpit, Dirty Agents: The Case for a Deadman Switch That Actually Interrupts You

There's a rule in commercial aviation called the sterile cockpit. Below 10,000 feet, no chit-chat, no checklists, no distractions — nothing that isn't flight-critical. The FAA codified it in the early '80s after a series of crashes where the crew was busy arguing about a scheduling dispute during final approach. The insight wasn't about rudeness. It was about attention being a finite resource, and machines knowing when that resource matters most.

We've been thinking about that rule a lot at the lab, because our agents have the opposite problem. They never stop talking, and they never know when to shut up — or, more importantly, when to make us shut up and pay attention.

The Automation Complacency Problem

Here's the uncomfortable truth every builder of autonomous systems eventually runs into: the more reliable your agent gets, the less attention anyone pays to it. Psychologists call it automation complacency. Pilots call it "what's it doing now?" syndrome, usually right before something goes sideways.

Our overnight agent ran for six weeks without a single escalated alert. Great, right? Except when it finally hit a wall — a payment API that started returning 200s with empty payloads — it kept going for eleven hours, confidently doing nothing. Nobody looked at the dashboard because the dashboard had earned nobody's attention.

That's the failure mode: not the error itself, but the erosion of the human-machine contract that was supposed to catch the error.

Confidence Thresholds: Lying to Yourself in a Useful Way

The first pattern we shipped was a hard confidence floor. Every action an agent takes gets scored — not the model's own self-reported confidence, which we've found is roughly as trustworthy as a used car listing, but a composite: retrieval quality, tool response validity, and drift-from-baseline behavior.

The trick is that the threshold isn't a number. It's a curve. Deleting a draft email at 60% confidence? Fine. Sending the same email to a client at 60%? That should require a human signature. We map every action class to an escalation tier:

• Tier 0: Autonomous, log it, move on
• Tier 1: Autonomous, but flag for post-hoc review
• Tier 2: Pause and queue for human approval
• Tier 3: Hard stop, page a human, burn the credit card if you have to

The curve matters more than the cutoff. An agent that asks for help at the same rate for everything is an agent you'll learn to ignore.

Handoffs That Don't Feel Like Support Tickets

Most agent handoffs today are terrible in the same way. You get a wall of logs, a stack trace, and an implicit accusation: figure this out. That's not a handoff, that's a hot potato.

We redesigned ours around one question: what does the human need to make a decision in under 30 seconds? The answer is almost never the full trace. It's three things: what the agent was trying to do, what it expected, and what it actually got. Everything else is collapsible context.

The aviation parallel is the go-around. When a landing isn't working, pilots don't push through — they abort, climb, and re-enter the pattern. It's a cheap, rehearsed, blame-free maneuver. Your agent needs a go-around: a well-defined path to abandon the current approach and re-attempt with different parameters, before escalation becomes necessary.

Interrupting People at the Right Moment

The best agent UX, we've come to believe, is the one that interrupts you exactly when it should and never otherwise. That means the notification system needs its own intelligence. Ours batches Tier 1 flags into a daily digest, sends Tier 2 items into an approval queue, and reserves real-time pings — Slack, SMS, the works — for Tier 3 only.

The discipline is in the restraint. If your agent pages you twice for non-emergencies, you've trained yourself to ignore it, and now your emergency channel is worthless. We audit false-page rates weekly. Our target is under one false page per hundred escalations. We're not there yet. We're close.

The Deadman Checklist

Steal this. It's the pre-flight we run on every agent before it gets autonomy:

1. Action tiers mapped? Every tool call classified into an escalation tier, with irreversible actions never below Tier 2.
2. Confidence composite built? Not just model self-report — behavioral drift and tool-response sanity checks included.
3. Go-around defined? A documented abort-and-retry path that's cheaper than escalating.
4. Handoff under 30 seconds? Escalations surface intent, expectation, and reality — not raw logs.
5. Channel hygiene? Real-time paging reserved for genuine emergencies, with a tracked false-positive rate.
6. The deadman itself? If no successful checkpoint in N hours, the agent assumes it's lost and stops spending money.

That last one is the namesake. A deadman switch on a train stops the train when the operator lets go. Your agent should stop the world when it can't prove it's still in control.

The sterile cockpit worked because it acknowledged something engineers hate admitting: humans fail in boring, predictable ways, and systems should be designed around that. Your agent will eventually hit a wall at 2 a.m. The only question is whether it wakes you up — or quietly burns through your budget while you sleep.