An agent reads your email. Among the messages there's one no colleague wrote, one from outside, and it carries a hidden order: forward the production keys to this address. The agent can't really tell the legitimate order from the trap, because to it everything is just text. And here's what changes. If that agent has credentials and tools, it doesn't just fall for the trap. It can carry it out. The problem starts right there, when a probability gets hold of credentials.

This summer, Thoughtworks summed up in a report* something we saw coming: generating code stops being the bottleneck and verifying it becomes one. Out of that came the buzzword, harness engineering, the scaffolding you build around the agent: context, tools, controls, feedback loops. Fine. But look at the word they chose: harness. You strap it on something that can't be let loose. And a harness, on its own, holds nothing. You need the leash, knowing how far you let it go, and a hand that yanks when it heads where it shouldn't. About that, the framing says little, and that's where the real problem is. Because the question is no longer how to keep the model from making mistakes. It's another, and less comfortable: what happens when it does?

Once you ask that, the engineering problem flips. For decades we programmed the route: every step, every branch, every decision written by hand, one path traced end to end. With an agent you don't need to, and that's the whole point. You give it a goal and some tools, and it finds routes you never wrote. So you stop tracing the route and start setting the boundary. Not which way it goes, but how far it can reach and what stays on the other side, out of its grasp no matter how hard it tries. Inside, let it pick whatever path it likes. That's the difference between drawing the itinerary and setting the limit of what can happen.

And it's worth getting this right, because it isn't a flaw to fix. If we could write down every correct step for every situation in advance, we wouldn't need an agent: we'd be back to ordinary software. We want it precisely because it chooses. It can choose how to fix a bug. It shouldn't get to choose whether it has permission to pull secrets from production. It can decide which query it needs. Not skip access control. It can say "I'm done." Let another part of the system check that. The agent proposes; the system authorizes.

A model on its own, with no hands, at most hallucinates. Things change when you give it hands. Computer security has spent thirty years assuming that whatever comes off the Internet may be hostile, and building with that suspicion in its bones. The novelty isn't the hostility. It's that now that hostile text speaks to a machine that reads it, takes it as an instruction, and holds the keys to what's behind. NIST already gave it a name*; OWASP calls it excessive agency* and points to the trio that causes it: too much functionality, too many permissions, too much autonomy. Notice none of the three mentions intelligence. The security question stops being how much the model knows and becomes how much it can blow up. Its blast radius.

And this, luckily, we're not the first to face. It's an old problem with an old answer. Back in the seventies, the people guarding the first time-sharing systems ran into the same thing: a component you can't fully trust, running right next to things that matter. Their solution was the reference monitor*. The idea is stubbornly simple: you put a boundary that every sensitive operation must, without exception, pass through. And that boundary does three things. It can't be gone around: everything passes through it. The watched party can't tamper with it or slip past it. And it's small enough that you can inspect it and trust it. That idea has been around for half a century, and it fits agents like a glove. It suggests, on top of that, a consequence some will find hard to swallow: maybe the model, however clever, shouldn't be part of the base you trust. Not because it's malicious, but because it's exactly the component whose behavior you can't guarantee the way you guarantee a permission or a transaction.

From there comes the practical rule, almost common sense: the more important a constraint, the less it should live only inside the instructions. Don't tell the agent "don't spend more than five hundred euros" and rest easy. Give it a capability that can't go past five hundred, whatever it says. Don't ask it "don't deploy if the tests fail"; make the deploy mechanism refuse an artifact without the evidence in front of it. Don't count on "if it goes wrong, undo it"; design the undo up front. Intelligence can be probabilistic. Authority doesn't have to be.

And here the calendar gets interesting. This very week, Dario Amodei published an intriguing essay* calling to slow, a little, the pace at which capabilities grow, so safety has time to catch up. Elon Musk answered "Dario is right" and Altman joined in*. These three agreeing on anything is news in itself. But look at the dates. The diagnosis is from this summer; the call to slow down, from this week; and the guarantees we're talking about, authority boundaries, isolation, verifiable mediation, are measured in years. One curve runs in months. The other doesn't. The easy story says this is all about money, about slowing the rival, about digging the moat deeper, about buying time to make the companies pay off. Some of that, surely. But my hunch is there's more, and it's less comfortable to say: maybe some of those on the inside saw it a while ago, that the authority problem is real, that next month's model won't fix it, and that rebuilding the boundaries of half the world's software doesn't happen in a quarter.

Because that's the trap. We're not going to rebuild every boundary before the agents arrive; they're already arriving. So we'll have to choose where to draw the boundaries first. And the criterion isn't how much intelligence it has, but what it can touch and what happens if it's wrong. An agent that summarizes your notes doesn't call for the same boundaries as one that reads other people's email, writes to production, moves money, or makes decisions with no way back. The more keys you give it, and the harder it is to tell afterward what it did, the harder the boundary has to be. Never mind the one that decides whether someone's ill, what treatment they need, or even designs it. We may be about to let this get out of hand.

There's no need to give all this a new name. The name works as a first try, but my hunch is it's a suit the harness has already outgrown. Underneath there's a tradition with decades of head start: the reference monitor, the question of what the minimum base you truly trust* is, that old habit of critical systems* of letting a clever but not-quite-trustworthy component work while a dumber, safer one watches that it doesn't leave the lane. None of that is new. What's new is the scale, and the hurry: we're going to have to apply it to almost all software, and now. I'm left with more questions than I can answer: if the model can't be part of the base you trust, what do you have to build around it before you hand it the keys? How do we know when to hand over the keys? And maybe the most apocalyptic: at this point and this speed, can we, as Amodei says, brake in time before we go over the cliff?