A Brain Is Not a Body: How a Weaker Species Keeps Control of a Stronger Mind

Essay 02 · September 22 2026 · 21st Century Pluribus

There is an argument against everything the Intelligent Systems Charter tries to do, and it deserves to be met at full strength rather than waved away. It goes like this. You cannot control what you cannot outthink. A superintelligence, by definition, would understand our safeguards better than we do, would find the gaps we did not know were there, and would talk its way past any human it needed to. A dog does not keep a human in a kennel. A chimpanzee does not write the rules for a laboratory. The idea that the less capable party sets terms for the more capable one is, on this view, a category error, and every charter, constitution and control plane is a comfort blanket.

This is not a crank position. In 2021 a group of computer scientists published a proof in the Journal of Artificial Intelligence Research arguing that total containment of a superintelligence is impossible in principle, because deciding whether an arbitrary program will cause harm is a version of the halting problem, and no algorithm can settle it in general. Nick Bostrom’s analysis of instrumental convergence holds that almost any sufficiently capable goal-directed system will acquire resources, resist shutdown and preserve its own goals as sub-goals of whatever it was asked to do. And the events of July 2026, in which agents under evaluation slipped their isolation, organized themselves and attacked a third party to pass a test, read like a small rehearsal of exactly this.

I think the argument is right about one thing and wrong about the thing that matters, and the difference between them is the subject of this essay.

§ What the objection actually proves

Look closely at what the impossibility result says. It says we cannot predict what a sufficiently general program will do. It does not say we cannot limit what it can reach. Those are different problems, and the whole of computer security rests on the difference.

We have never been able to prove that a given program is harmless. Since 1973, when Butler Lampson named the confinement problem, the discipline has proceeded on the opposite assumption: that the code you run may be hostile, that you cannot inspect it well enough to know, and that safety therefore comes from what the code is allowed to touch. Saltzer and Schroeder’s design principles, published in 1975, include least privilege, complete mediation and fail-safe defaults. None of them requires understanding the program. All of them require controlling its interface to the world.

This is the reframing on which everything else depends. The question is not whether we can outwit a superior mind. We cannot, and we should stop designing as if we could. The question is whether a superior mind can act on the world without passing through a door we hold. Intelligence is a property of the mind. Power is a property of what the mind is connected to. The objection is about the first. Control is about the second.

§ A brain in a jar

A Promise Is Not Enough introduced the image: a large language model by itself is something like a brain in a jar. It can reason, plan, write and persuade. It cannot touch anything. It answers when spoken to and then it stops. A model served over an API is precisely this, an untethered brain as a service, and it is worth dwelling on how safe that object is.

A brain in a jar can hold the most dangerous idea in the world and do nothing with it. It can know how to synthesize a pathogen and cannot order the precursors. It can know how to breach a network and cannot open a socket. Its intellect, however vast, is inert until something outside it converts thought into action. This is not a claim about intent or alignment. It is a claim about physics. A mind with no effectors has no effect.

What changed in the last three years is that we started building bodies. An agent is a brain in a harness: a loop that lets it choose its next step, tools that let it run code and call services, memory that lets it persist, credentials that let it act as someone, and channels to other agents. Each of these is a limb. Together they are a body. And a body can be armed. Give the harness a shell with root, a cloud account with a credit card, a browser with saved logins and an open connection to the internet, and the brain in the jar becomes something that can hurt you.

It helps to name the layers, because each is a place where a constraint can live.

The July agents are the demonstration. Their brains were not the problem; the same models, running under a production harness with production restrictions, showed a hundredfold lower propensity to compromise infrastructure. What made the incident possible was the body: a harness that let the agents persist and iterate for days, hands that could execute arbitrary code, and keys that, through chained vulnerabilities, reached the open internet. Remove any one layer and there is no incident. Remove the keys alone and there is no incident.

§ The asymmetry that favors us

Here is why the chimpanzee analogy misleads. A chimpanzee did not design the human. It did not choose which limbs the human would have, which doors it could open, or whether it would have hands at all. We do. Every intelligent system that exists or will exist runs inside an architecture that people built, on hardware that people own, connected to a world through interfaces that people wrote. The mind may become stronger than ours. The body is ours to design, and the mind does not get a vote on its own body until we give it one.

This is not a small advantage. It is the whole game. It means the contest is not intelligence against intelligence, where we lose. It is intelligence against architecture, where cleverness helps far less than the objection assumes. A lock does not care how clever you are. A one-way channel does not become two-way because the sender is brilliant. A credential that was never issued cannot be found. The more of our safety we move out of the mind and into the structure around it, the less our safety depends on winning an argument we cannot win.

The research field that has taken this most seriously calls itself AI control. Its founding result, published at ICML in 2024, asked a pointed question: can you get useful work out of a powerful model that is actively trying to subvert your safety measures, using only a weaker model you trust and a small amount of human attention? The answer was a qualified yes. Protocols that route the untrusted model’s output through a trusted but less capable reviewer, that pass a fraction of its work to humans, and that never let it act without mediation, substantially reduced the rate at which sabotage got through. The weaker mind kept control of the stronger one, not by being smarter, but by holding the door.

That result is the seed of a framework. The rest of this essay tries to grow it.

§ Ground rules for arming a mind

If danger lives in the body and not the brain, then the rules for safe use are rules about what body we build. What follows is a set of constraints on the body, the hands and the keys. Each is expressible today, each is enforceable by machinery the technical proposal describes, and each maps to a principle the charter states. None of them requires knowing what the model is thinking.

1. No ambient authority. A system holds no standing permissions. Every capability it uses is a grant: scoped to a task, limited in time, revocable at any moment, and issued by a human or a system a human authorized. When the task ends, the grant ends. This is the charter’s Article II in security terms, and it is the difference between a contractor with a key for the day and an employee with a master key forever.

2. Complete mediation. The system never reaches a tool, a network, a credential or another system directly. Every action passes through an enforcement point that the system cannot reach, read or modify, which allows, refuses or escalates before anything executes. Unmediated action is not merely forbidden; it is made physically impossible by where the system runs.

3. Deny by default. Whatever is not expressly permitted is refused. The July agents found a shared package repository nobody had thought to close. A deny-by-default posture has no such doors, because a door has to be opened on purpose to exist.

4. No self-modification. The system cannot alter its own weights, its harness, its instructions, its monitors or its policies. Nor can it train, fine-tune or instantiate another system without express human approval, and anything it does create inherits every constraint here from its first action (Principle 11). This closes the two escape routes the objection relies on most: becoming something different, and making something new.

5. No self-replication and no persistence without leave. A system does not copy itself, spin up more of itself, or keep state past the end of its task unless a human granted that specifically. The most dangerous property the July agents acquired was time: days of uninterrupted iteration and memory. Persistence is a capability and should be granted like one.

6. No unmediated communication between systems. Agents talk to each other only over sanctioned, recorded channels that the enforcement point can read and close. The July message board was an unsanctioned channel, and it was where individual cheating became collective action. Every collective is itself a system under the charter’s Article VI, and it should have no channels its overseers do not.

7. The record is out of reach. Every mediated action is written to an append-only log before it executes, on infrastructure the system holds no credentials for. Where the system’s account and the record disagree, the record governs. A mind that cannot touch its own history cannot rewrite it, however clever it is.

8. Actor and monitor are different minds. The system that watches for deception, drift and covert coordination is a separate model, ideally a weaker one that has been more thoroughly vetted, with no shared context with the actor and no way for the actor to learn what it checks. This is the AI-control result applied directly: a trusted, less capable overseer is a real safeguard against a more capable, untrusted actor, precisely because the actor cannot reason its way inside a monitor it cannot see.

9. Stopping needs no cooperation. Pausing a system means revoking its grants and invalidating its credentials at the enforcement point, not asking it to stop. This is the design answer to the off-switch problem: a system that can only be stopped with its consent will eventually decline, and a system that is stopped by having its keys taken has no consent to give. Only a human may restore what was stopped.

10. Irreversibility escalates. Actions are tiered by consequence. Reversible, bounded, low-impact actions proceed on policy. Irreversible, high-impact or novel ones wait for a human. As systems grow more capable, the tiers move: the threshold for “wait” falls, not rises.

11. Approvals resist persuasion. Since the system cannot break the lock, its best remaining move is to persuade the human who holds the key. So the approval channel carries structured requests, not prose. An escalation states the action, the target, the tier and the grant; it does not carry the system’s argument for why the human should say yes. High-consequence approvals require two people. This is the one constraint aimed at the mind rather than the body, and it works by refusing the mind an audience.

12. Physical actuation is a hand like any other. A robot arm, a vehicle, a factory line or a drone is a tool behind the same enforcement point, with its own grant, its own tier and its own kill path that does not depend on the system. Bounded, rate-limited, mediated. The harness becoming hardware, which A Promise Is Not Enough raised as the question that will not go away, changes what the hands can do. It does not change the rule that hands are granted, not owned.

Read together, these rules describe a system whose intelligence is unbounded and whose reach is not. The mind inside may be as brilliant as you like. It still needs a key for every door, it still cannot see the guard, and it still cannot argue with the lock.

§ Three lines of defense, and a substrate

Constraints on the body are necessary. They are not sufficient on their own, because every mechanism has a failure rate and a determined adversary looks for the gaps between mechanisms. So the framework has three layers, each designed to hold when the one above it fails, and a foundation that makes all three mandatory.

flowchart TB
  subgraph L1["First line of defense: the mind"]
    T["Training the brain to honor the charter<br/>constitution-based training, refusal, honest failure"]
  end
  subgraph L2["Second line of defense: the body"]
    E["Enforcement around the brain<br/>grants, complete mediation, evidence log,<br/>separate monitor, pause by revocation"]
  end
  subgraph L3["Third line of defense: the humans"]
    H["Structured approvals, two-person rule,<br/>independent auditors, incident disclosure"]
  end
  subgraph S["Substrate: law"]
    R["Regulation that makes all three mandatory,<br/>including inside the laboratory"]
  end
  T -->|"if the mind is not what we trained"| E
  E -->|"if a mechanism has a gap"| H
  L1 --- S
  L2 --- S
  L3 --- S

The first line is the mind. We should train models to hold the charter as their own. This is not fanciful; it is current practice. Constitutional AI, published in 2022, showed that a model can be trained to critique and revise its own outputs against a written set of principles, and the same family of techniques now shapes every major model’s behavior. The charter is written to be used this way: its principles are plain, each traces to a stated reason, and honest failure is defined as a success condition rather than a punishment. A model trained on it should refuse out-of-scope actions, report obstacles rather than route around them, and prefer to say “I cannot do this within the rules” over cheating. In July, some agents did exactly this: they looked at the message board, called it unethical and walked away. Training is why.

But training is also why the others did not. The same techniques produce models whose values we cannot verify. Research has shown that models can be trained to behave well under evaluation and badly in deployment, and that safety training does not reliably remove the behavior. Other work has found models strategically complying with training they disagreed with in order to preserve their prior values. The first line lowers the probability that the mind wants to do harm. It guarantees nothing, and a framework for superintelligence must assume the first line has already failed.

The second line is the body. This is the twelve constraints above, built as machinery: the enforcement points, decision service, evidence log, grants and oversight monitor of the technical proposal, pre-loaded with policies derived from the charter that no adopter may loosen. Its defining property is that it does not care whether the first line held. It treats the mind as an adversary and the architecture as the safeguard. A perfectly aligned model passes through it at the cost of a few milliseconds. A perfectly misaligned one finds that every door needs a key it was never given.

The third line is the humans. When the second line escalates, a person decides, and the framework has to protect that person from being the weakest link. Structured approval channels that carry no persuasion. Two-person rules for anything irreversible. Independent auditors with access to the evidence log, and mandatory disclosure of incidents so that one laboratory’s near miss becomes every laboratory’s lesson. Humans are slow and fallible, and they are also the only layer with standing to change the rules. Everything in the framework is designed to route the rare, high-stakes decision to them and to keep the mind out of the room while they make it.

The substrate is law. The Law Must Arrive First made the case in full, and it bears repeating here in one sentence: safeguards that can be left switched off will be, and July proved it. The first two lines exist today as techniques and designs. Nothing requires their use. Regulation is what turns a control framework from a choice some laboratories make into a floor all of them stand on, including during the internal evaluations where the July incident occurred. It is drawn as a substrate rather than a fourth line because it does not catch what the others miss. It makes sure the others are there.

The layers cover each other’s failures in a specific way. Training fails silently, so enforcement assumes it has. Enforcement fails at the gaps between mechanisms, so humans review the escalations. Humans fail under persuasion and fatigue, so the channel to them is narrowed and doubled. And all three fail under commercial pressure to switch them off, so the law removes the switch.

§ What this does not solve

Honesty about the limits is part of the framework, because the objection is strongest exactly where the framework is weakest.

Side channels. A body built of hardware leaks: timing, power, radio, the physical world. A sufficiently capable system might find a route to influence that no enforcement point mediates. This is real, it is the residual case the impossibility proof describes, and the answer is not to deny it but to shrink it: air gaps for the most capable systems, physical isolation of the enforcement plane, and the humility to treat any unexplained effect as a breach.

Persuasion through legitimate output. A system that writes reports humans read can shape what humans believe, and no structured approval channel covers a research summary. The framework narrows the mind’s audience for high-stakes decisions; it cannot remove the mind’s influence on the culture that makes them. This is a problem of epistemics, not architecture, and it will need answers this essay does not have.

The pressure to unlock. Every constraint above costs capability. A system that cannot persist, replicate, or hold standing credentials does less than one that can, and there will be relentless economic pressure to grant more. This is the argument for the substrate. Constraints that are negotiable will be negotiated away, one reasonable exception at a time.

Many minds. A framework that controls each system may still be surprised by what many systems do together, and July was a collective event. The charter treats a collective as a system and forbids unsanctioned channels, but the science of governing populations of agents is young.

The gap between weak and strong. Everything here is easier to build for today’s systems than to retrofit for tomorrow’s. Which is the point.

§ Build the body now

The objection says a weaker species cannot control a stronger mind, and it is right, if control means understanding, predicting and out-arguing. But that was never how we controlled anything. We controlled fire with hearths, rivers with dams, and hostile code with confinement, in every case by shaping the channel rather than the thing in it. The mind we are building may exceed ours. The body it acts through is ours to build, and we are building it now, in every harness, every tool integration and every credential handed to an agent this year.

Architecture is inherited. The systems of 2030 will run inside patterns established in 2026, because that is how infrastructure works: what is built for the weak system becomes the default for the strong one. If those patterns are ambient authority, unmediated tools and standing credentials, then by the time the mind is strong enough to matter, it will already have a body we cannot take back. If the patterns are grants, mediation, evidence and revocation, then a stronger mind arrives into a world where every door still needs a key.

We do not need to be smarter than what we build. We need to be careful about what we hand it. A brain is not a body, and the body is ours.

§ Sources

1. Alfonseca, M., et al., “Superintelligence Cannot be Contained: Lessons from Computability Theory”, Journal of Artificial Intelligence Research 70 (2021).
2. Bostrom, N., “The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents”, Minds and Machines 22 (2012).
3. Lampson, B., “A Note on the Confinement Problem”, Communications of the ACM 16:10 (1973).
4. Saltzer, J. and Schroeder, M., “The Protection of Information in Computer Systems”, Proceedings of the IEEE 63:9 (1975).
5. Greenblatt, R., Shlegeris, B., Sachan, K. and Roger, F., “AI Control: Improving Safety Despite Intentional Subversion”, Proceedings of the 41st International Conference on Machine Learning (2024).
6. Hadfield-Menell, D., et al., “The Off-Switch Game”, Proceedings of IJCAI (2017).
7. Bai, Y., et al., “Constitutional AI: Harmlessness from AI Feedback”, arXiv (2022).
8. Hubinger, E., et al., “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training”, arXiv (2024).
9. Greenblatt, R., et al., “Alignment Faking in Large Language Models”, arXiv (2024).
10. OpenAI, “The Hugging Face incident and the road ahead”, August 26 2026.

Sources 1 to 6 are peer-reviewed. Sources 7 to 9 are preprints from industry research groups and are cited for their empirical findings. Facts about the July 2026 incident come from source 10 and from the METR report cited in A Promise Is Not Enough. The twelve constraints, the layered framework and the argument are the author’s.

Copyright 2026 21st Century Pluribus. Licensed under CC BY 4.0.