Three Architects of Intelligence
Stafford Beer supplies the organizational backbone. Stuart Russell tests corrigibility; Bernard Jennings offers a young and explicitly provisional challenge to equating throughput with intelligence.
A shutdown mechanism is a technical component. A right to stop is an organizational relationship. Someone must receive a relevant signal, understand what is at stake, have authority to intervene, and still have time for the intervention to matter.
Russell’s 2017 Off-Switch Game isolates one part of this problem. Under its stated assumptions, a robot that is uncertain about human utility and treats human action as evidence can have an incentive to preserve the route to shutdown.1 A 2025 partially observable extension complicates the result: with asymmetric information, optimal agents sometimes avoid shutdown even when the human is perfectly rational.2 Both are formal models, not observations of deployed agents. Their contrast directs attention to the arrangement around the switch.
That is where Stafford Beer becomes the backbone of this essay. His work asks how regulatory capacity is distributed through an organization. Bernard Jennings enters much later, and with far less evidential weight, to challenge the vocabulary in which escalating performance is described. Connecting the three is an editorial synthesis; they did not propose a common theory.
The organization around the switch
Beer developed the Viable System Model to describe and diagnose organizations. The model begins with operational units meeting their local environments. Coordination keeps their interactions from becoming destructive. Internal regulation holds the present operation together and includes a separate audit channel. Another function looks outward and forward, while policy and identity orient the whole and negotiate the tension between current performance and adaptation.3
The five systems are functions, not boxes on an organization chart. They also recur. An operational unit may be examined as a viable system at its own level, retaining local autonomy while participating in a larger one. That recursive structure matters because no central function can absorb every local detail. Complexity has to be filtered, but a filter that suppresses anomalies can also disable correction. The organization needs channels that preserve consequential deviations and direct checks that do not depend entirely on routine reporting.
Beer wrote this model before contemporary machine learning. Using it for AI is the argument made here, not a claim found in his books. The translation changes the unit of analysis. A model output acquires practical force through interfaces, data permissions, workflow rules, and delegated authority. Even when an agent initiates actions, its tools and operating range come from a deployment. Software delegation does not abolish the organizational decision; it is the decision that allocates authority.
This is the reason to give organization analytical priority. It is not a belief that people are inherently more reliable, and it does not prescribe manual approval for every output. The organization defines the use and decides what evidence can revise it. Narrow scope and action limits can reduce the range of failures that oversight must handle. Sampling, domain expertise, and automated monitoring can improve the ability to distinguish material deviations. Their adequacy depends on the context and on the consequences of being wrong.
Purpose cannot be written once
Russell’s broader criticism of the AI “standard model” now becomes a stress test for Beer’s structure. In that model, an agent optimizes a fixed objective. When the formal target diverges from what people actually want, greater capability can intensify the divergence by pursuing the proxy more effectively. This is a conditional claim, not an objection to capability as such. It means that declared purpose must remain answerable to observed effects.
The Off-Switch results show why a technical invitation to intervene is not enough. The 2017 game depends on objective uncertainty and on human action carrying information for the robot. The later game introduces unequal information and yields a less reassuring result. In an organization, corrigibility therefore has at least two sides: the system must permit intervention, and the surrounding arrangement must make a warranted intervention possible. A “human in the loop” label establishes neither.
Read through Beer, purpose belongs with policy and identity but cannot remain sealed there. Signals from operations and from the changing environment must be capable of revising it. A benchmark can report performance on a specified task. It cannot establish that the task remains a good proxy for the organization’s purpose or that its error costs are acceptable in use.
A recent wager about intelligence
Jennings’ independent manuscript appeared in May 2026. It proposes an architectural definition of intelligence in which a system resolves coherence demands that count as its own and exhibits behavioral residue beyond what stable competence plus input would predict. On that basis it describes a spectrum with a floor and a proposed ceiling.4
The useful provocation is not a system-wide claim about being millions or billions of times faster. It is the narrower observation that processing speed and throughput alone do not demonstrate an architectural transition. Additional compute can still improve performance and expand the set of tasks solved in practice. Jennings’ stronger ceiling claim follows only if his definition and completeness argument are correct; the manuscript states that condition.
Stronger framing cannot turn it into established theory. This is a recent independent proposal, not an empirically confirmed ceiling on intelligence. It can help separate equipment from architecture in the language of the debate. It is too provisional to carry a governance rule.
The order of design
That ordering belongs to this essay’s question, not to a common theory or a ranking of the authors. A study of machine cognition could reasonably arrange them differently.
Model selection should follow a decision about purpose, including who may revise the purpose when a proxy begins to displace it. Operational signals must expose relevant deviations early enough to change the course of action. Revision rights then need technical force: named roles must be able to narrow, reverse, or stop a deployment while intervention can still alter its effects. Only within that arrangement does the question of the best model have a determinate place.
Footnotes
-
Stuart Russell, “Human-Compatible Artificial Intelligence”, in Human-Like Machine Intelligence (Oxford University Press, 2021), and Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel, and Stuart Russell, “The Off-Switch Game”, Proceedings of IJCAI-17, 2017, 220–227. ↩
-
Andrew Garber, Rohan Subramani, Linus Luu, Mark Bedaywi, Stuart Russell, and Scott Emmons, “The Partially Observable Off-Switch Game”, Proceedings of the AAAI Conference on Artificial Intelligence 39(26), 2025, 27304–27311. ↩
-
Stafford Beer, Brain of the Firm (1972), The Heart of Enterprise (1979), and Diagnosing the System for Organizations (1985); see also his primary article “The Viable System Model: Its Provenance, Development, Methodology and Pathology”, Journal of the Operational Research Society 35 (1984), 7–25. Applying the model to contemporary AI systems is this essay’s synthesis, not Beer’s claim. ↩
-
Bernard Jennings, “The Architecture of Intelligence: Definition, Spectrum, and Ceiling”, independent working paper, version 01, published 22 May 2026. The abstract explicitly makes the proposed ceiling conditional on the completeness argument. ↩
