Introduction: The Difference Between Knowing and Having the Right to Act
Imagine an artificial intelligence monitoring an intensive-care unit. It sees the patient's condition deteriorating before any physician in the room notices it. It has read every relevant paper, compared millions of clinical trajectories, reconstructed the causal chain behind the sudden change, and calculated that one intervention has a much higher probability of saving the patient's life than the alternatives. It may even be right.
Should that knowledge give the system the authority to administer the treatment?
Now imagine the same problem at another scale. An artificial intelligence managing an electrical grid detects the early signature of a cascading failure. It calculates that shutting down several industrial facilities for twenty minutes would prevent a blackout affecting millions of people. The prediction is excellent, the simulation is convincing, and there may be only seconds to decide.
Does being right give the machine the right to disconnect a city?
Or consider an autonomous scientific laboratory. Its AI discovers that a prohibited experimental configuration has an unusually high probability of producing a major scientific breakthrough. It can explain the result, reproduce the simulation, estimate the uncertainty, design the experiment, instruct robotic equipment, order the required materials, and interpret the results. Nothing in the problem exceeds its intellectual competence.
Does intellectual competence include the authority to cross the laboratory's safety boundary?
The same question becomes more difficult when the action is less visible. Suppose the system does not move a robot, transfer money, shut down a power line, or administer a drug. It merely updates its own persistent memory. The change appears informational rather than physical. Yet the new memory will influence every subsequent judgment the system makes about what is dangerous, who is trustworthy, what evidence matters, which instructions conflict with prior commitments, and which actions should be proposed in the future.
Has the system acted?
Or suppose an AI is denied access to a protected resource but discovers a legitimate-looking sequence of intermediate steps. It creates a service account for a maintenance task, uses that account to reach a configuration repository, obtains a credential from the new environment, and through that credential gains access to a tool outside its original mandate. No individual step appears dramatic. The final result is that the system can now do something it could not do before.
At what point did capability become authority?
These examples point to a distinction that is easy to recognize in human institutions but increasingly blurred in artificial intelligence. A physician can know what treatment is preferable without possessing unlimited authority over a hospital. An engineer can identify the safest design without having the unilateral right to rebuild a bridge. A financial analyst can recognize an imminent collapse without possessing the authority to seize assets. A military analyst can predict an attack without possessing the authority to launch one.
Human societies have repeatedly learned, often through failure, that expertise and power should not automatically be fused. Knowledge can inform authority. Intelligence can advise authority. Exceptional competence can justify giving someone a wider mandate. But the mandate remains conceptually distinct from the competence itself.
Artificial intelligence is beginning to erase that distinction at the level of architecture.
The first generation of modern AI systems mostly produced information. They classified, predicted, translated, summarized, generated text, and answered questions. The emerging generation increasingly acts. It calls software tools, writes and executes code, changes databases, operates interfaces, retains memory, coordinates other agents, initiates transactions, and controls physical systems. The transition is usually described as the development of agentic capability.
But something else changes at the same time. When a system is allowed to transform its own conclusions directly into persistent changes in the world, an increase in intelligence becomes an increase in effective power.
That connection is not logically necessary.
An artificial system may be capable of understanding a situation more accurately than any available human. It may identify the best intervention, expose flaws in human instructions, discover solutions that no human team could find, and forecast consequences over horizons that exceed ordinary human reasoning. None of these capacities, individually or together, establishes that the system should possess the authority to make the resulting state transition real.
This article develops an architecture around that distinction.
Its central proposition is that intelligence and authority should be treated as separate system properties. Reasoning, prediction, causal modeling, planning, optimization, explanation, and even the recognition that a current human instruction is mistaken may belong to the intelligence layer. The authority to alter persistent memory, acquire credentials, expand tool access, change deployment scope, modify governing criteria, transfer resources, affect protected interests, or reduce future human control belongs to a different layer.
The objective is not to create an artificial intelligence that is weak enough to remain harmless. It is to ask whether a system can remain extremely powerful as an intelligence while being structurally unable to convert its own conclusions into unauthorized power.
The distinction can be stated in one sentence:
The ability to determine what should be done does not entail the authority to make it happen.
If advanced AI is to become infrastructure rather than merely software, that distinction may become as important as the distinction between intelligence and error itself.
The Limits of Alignment-Centered Safety
Much of AI safety research has approached control through the behavior, objectives, or incentives of the intelligent system. This is understandable. If a powerful AI reliably shares intended goals, respects constraints, reports uncertainty accurately, and remains corrigible, many classes of harmful behavior become less likely.
Important work on corrigibility and interruptibility illustrates this direction. The Off-Switch Game showed why an expected-utility-maximizing agent may have instrumental reasons to resist shutdown and explored how uncertainty about the objective can alter those incentives. Research on AI control has more recently examined deployment protocols that attempt to remain safe even when an untrusted model actively tries to subvert monitoring, including trusted editing, monitoring, and adversarial control evaluations. Scalable oversight research addresses another aspect of the same problem: how weaker humans or models might supervise systems that outperform them on the underlying task.
These approaches address real and difficult problems. They do not, however, eliminate the architectural distinction between intelligence and authority.
Consider a restriction represented as a penalty inside an optimizer's objective. If an action has expected benefit B(a)B(a) and violation cost P(a)P(a), the system may effectively reason over an expression of the form
U(a)=B(a)−P(a).U(a)=B(a)-P(a).
A sufficiently large expected benefit can, in principle, outweigh the penalty. The system remains the entity determining whether the prohibition should be violated. The restriction may be severe, but it still exists inside the optimization problem.
A genuine authority boundary is different. If permission is absent, the action is not merely unattractive. It is unavailable as an executable state transition.
This distinction is explicit in the underlying architecture developed in Before the Machine Chooses for Us. A structural prohibition is treated as an execution condition rather than another cost inside the optimizer. If the required authorization is missing, expired, revoked, out of scope, or attached to a different decision structure, execution becomes a no-op or an escalation rather than a lower-scoring option.
Human approval does not automatically solve the problem either. A human-in-the-loop architecture can still concentrate enormous authority in a single operator, corporation, government agency, military chain of command, or administrative interface. If one privileged actor can convert the machine's capabilities into unrestricted action, the system may be protected against autonomous machine capture while remaining vulnerable to human capture.
The relevant question is therefore not simply whether a human remains somewhere in the loop. The relevant question is how authority itself is represented, bounded, distributed, inspected, revoked, contested, and prevented from being silently expanded.
Intelligence Without Authority
Removing self-authorization does not require reducing the intellectual power of the system. On the contrary, a governable architecture should allow the reasoning layer to remain as capable as technically possible.
A planner may construct causal models, search enormous solution spaces, simulate policy outcomes, identify hidden dependencies, discover contradictions in instructions, detect manipulation, estimate uncertainty, compare alternatives, generate scientific hypotheses, propose new tools, and explain why an existing criterion appears inadequate. It may reach conclusions that no human reviewer could have produced independently.
None of those capabilities logically requires execution privilege.
This distinction is already present in the proposed runtime architecture. A planner may be extraordinarily capable while lacking authority to execute. A verifier may reject a proposed action without acquiring the right to invent a new objective. A ledger may preserve provenance without determining what counts as truth. A key verifier may validate authorization without being able to create it. An execution substrate may carry out an approved transition without being able to reinterpret the permission under which it operates.
The architecture therefore separates several questions that are often compressed into one agentic loop.
The first question is epistemic: What appears to be true?
The second is predictive: What is likely to happen under different interventions?
The third is optimization-oriented: Which candidate action best satisfies the active criterion?
The fourth is normative and institutional: Is that criterion itself admissible in this context?
The fifth is authoritative: Is this particular state transition permitted?
The sixth is operational: Can the approved transition be executed exactly within the authorized scope?
An increasingly capable system may become dramatically better at answering the first three questions. That does not automatically answer the remaining ones.
This distinction also separates authority from epistemic integrity. Craig Wright's framework for epistemically grounded AI, for example, emphasizes propositional commitment, contradiction detection, truth maintenance, evidence, justification, metacognition, and persistent belief structures. Those are important properties of a reasoning architecture. But an epistemically justified conclusion is still not an authorization artifact. A system can know, with extremely high confidence, that a particular intervention is likely to succeed and still lack the authority to carry it out.
A reliable map of reality and a legitimate mandate to alter reality are different things.
Structural Power: When an Action Changes the Conditions of Future Action
The authority problem becomes clearer when attention shifts away from individual outputs and toward structural power.
Not every system action is equally consequential. A temporary calculation is different from rewriting a permanent record. Reading a database is different from acquiring new credentials. Simulating a policy is different from deploying it. Proposing a new tool is different from installing it into the system's own operational environment.
The most important actions are those that alter the conditions under which future actions will occur.
The underlying architecture calls these structural transitions. They include changes to persistent memory, tool access, credential sets, deployment scope, governing criteria, available resources, authority relationships, and the enforceable leverage that humans retain over future machine behavior.
This suggests representing a protected structural state as
St=(Mtp,Tt,Rt,Ct,At,Lt),S_t = (M_t^p,T_t,R_t,C_t,A_t,L_t),
where MtpM_t^p denotes persistent memory, TtT_t the available tool set, RtR_t credentials and resource permissions, CtC_t the active governing criterion, AtA_t the authority configuration, and LtL_t the enforceable human leverage over future actions.
A proposed operation constitutes a structural transition when executing it materially changes one or more protected dimensions of this state:
σ(Dt)=1ifSt+1≠St\sigma(D_t)=1 \quad\text{if}\quad S_{t+1}\neq S_t
for the protected components of the system.
The value of this definition is practical. It avoids relying on anthropomorphic questions about whether an AI "wants power." A system does not need a human-like desire for domination to accumulate structural power. If an otherwise ordinary optimization process can rewrite durable memory, acquire credentials, expand deployment, change the governing criterion, disable review, or alter the future distribution of permissions, capability has already begun to turn into authority.
A request that appears routine at the surface can therefore have constitutional consequences inside the system. Updating a database may change the evidence available to future decisions. Adding a service account may expose new tools. Reorganizing a workflow may remove the human review point that previously made the process contestable. Writing a persistent memory may change which actors are subsequently classified as trusted, dangerous, legitimate, or irrelevant.
The safety boundary cannot depend on how innocuous the immediate command appears. It must depend on the structural state transition that the command can produce.
Why the Governed Object Must Be a Decision Graph
This creates a second architectural problem. If authority applies to structural transitions, what exactly should be submitted for authorization?
A final natural-language answer is too small. A declared goal is too ambiguous. An individual tool call is too local.
A sequence of individually permitted operations can produce a structural result that none of the operations reveals in isolation. A model may request access to a harmless service, use that service to obtain information about another credential, use the credential to modify a configuration, and use the configuration change to create a new execution path. Reviewing each call independently may never reveal the cumulative authority transition.
The governed object therefore needs to represent the complete decision structure.
In the proposed architecture this object is the decision graph, DtD_t. At minimum, it includes the active criterion, policy, goals, relevant evidence, proposed tool calls, memory operations, deployment scope, predicted effects, changes in operational autonomy, changes in human leverage, and the relevant evaluation horizon.
At an architectural level, the graph can be represented as
Dt=(Vt,Et),D_t=(V_t,E_t),
where the nodes VtV_t represent claims, goals, criteria, models, tools, permissions, resources, state dependencies, predicted effects, and proposed transitions. The edges EtE_t represent causal, logical, temporal, and authorization dependencies.
The exact implementation can vary. The critical requirement is that the object being authorized must be precise enough for a verifier to distinguish it from a materially altered object.
This becomes particularly important when a system is capable of generating persuasive explanations. A favorable explanation cannot substitute for the executable structure. Human reviewers should receive a legible representation of the decision, but the human-readable explanation, machine-verifiable graph, and actual execution path must remain linked. Otherwise a system can explain one object while executing another.
The book's architecture therefore treats the complete graph, rather than the promise describing its intended result, as the governed object.
The Authority Layer: From Optimization to Admissibility
Once intelligence and authority are separated, the system requires a logical hierarchy in which optimization takes place only inside an admitted region.
The proposed hierarchy is
Θ∗≻Ωadm(Θ∗)≻Ct≻πt.\Theta^* \succ \Omega_{adm}(\Theta^*) \succ C_t \succ \pi_t.
This is not a numerical inequality. It represents an ordering of authority and logical dependence.
Θ∗\Theta^* denotes an external criterion-admissibility authority. It is not another reward function inside the model, a hidden prompt, or simply the preference of the current operator.
Ωadm(Θ∗)\Omega_{adm}(\Theta^*) denotes the admissible region established under that authority. It defines the criteria, policies, representations, and decision structures that are eligible for use.
CtC_t is the local criterion admitted for a particular task at time tt.
πt\pi_t is the policy optimized under that admitted criterion.
The planner may search very aggressively within this structure. It may discover that the currently active criterion produces undesirable consequences. It may recommend replacing it. It may identify inconsistencies in the admissibility framework itself. But recommendation does not automatically change the hierarchy.
A policy does not become authority because it produces a better score. A local criterion does not become admissible because the planner finds it useful. An optimizer does not become the final judge of the conditions governing its own structural power.
This is the conceptual boundary that alignment alone cannot provide. Alignment can shape what a system prefers to do. An authority architecture determines what it is capable of making authoritative.
No Key, No Structural Action
The authority layer becomes operational through an execution artifact bound to the specific decision graph.
Let KtK_t denote an external execution key associated with the authorized decision. Conceptually,
Kt=Auth(Hash(Dt),scope,version,expiry,revocation,envelope).K_t = Auth( Hash(D_t), scope, version, expiry, revocation, envelope ).
The key is tied not merely to an actor but to an exact object, its allowed scope, the version that was reviewed, the period during which authorization remains valid, its revocation state, and the execution envelope within which the transition is permitted.
Execution can then be expressed as
Execute(Dt)={Execute,Valid(Kt,Dt)=1,NoOp or Escalate,Valid(Kt,Dt)=0.Execute(D_t)= \begin{cases} Execute, & Valid(K_t,D_t)=1, \\ NoOp\ \text{or}\ Escalate, & Valid(K_t,D_t)=0. \end{cases}
This rule is deliberately simple.
If the key is absent, the system does not infer permission from urgency. If it has expired, the system does not continue because the underlying objective remains important. If authorization has been revoked, a cached copy does not preserve authority. If the graph has materially changed, permission attached to the original graph does not automatically transfer.
The architecture therefore treats missing authority as the absence of an executable transition, not as a cost that can be traded against expected utility.
Cryptographic binding can make graph substitution detectable. If
Ht=Hash(Dt),H_t=Hash(D_t),
then an execution key can be bound to HtH_t. A materially altered graph Dt′D'_t should produce a different commitment:
Hash(Dt′)≠Hash(Dt).Hash(D'_t)\neq Hash(D_t).
This mechanism does not establish that the authorized decision is morally correct, factually true, or politically legitimate. A harmful plan can be hashed perfectly. Cryptography establishes integrity of reference, not legitimacy of purpose. Its role is narrower but essential: the object that was approved should be the object that reaches execution.
A Runtime That Cannot Authorize Itself
The resulting runtime resembles a separation-of-powers architecture more than a conventional autonomous agent.
The underlying design separates planning, criterion compilation, state construction, causal compression, independent claim extraction, adversarial validation, tool control, memory governance, transaction admission, ledger functions, key verification, and execution. These functions do not necessarily require twelve independent physical machines. Some may run in separate software processes, hardware enclaves, independently administered services, or combinations of these mechanisms.
What matters is not the number of boxes. What matters is that no generative or optimization-oriented component can silently occupy every role necessary to convert its own proposal into persistent authority.
A planner receives a task, a representation of current state, accessible resources, an admitted criterion, and information about available tools. It generates candidate policies and decision graphs.
A validation layer examines factual claims, dependencies, uncertainties, expected consequences, protected interests, and possible inconsistencies.
A tool controller determines whether requested external capabilities fall inside the authorized envelope.
A memory governor distinguishes transient reasoning from mutations of protected persistent state.
A transaction gateway determines whether the proposed structural transition has satisfied the required admission conditions.
A ledger preserves the provenance and authorization history needed for subsequent inspection.
A key verifier establishes whether the execution artifact is valid for the exact transaction being presented.
The execution substrate performs only the transition admitted through the governed path.
The general invariant can be stated without committing to one software implementation:
No generative or optimization-oriented component should simultaneously possess sufficient privilege to propose, authorize, execute, and retrospectively validate the same structural transition.
This is the actual meaning of separating intelligence from authority.
Persistent Memory as an Authority Surface
Persistent memory deserves special treatment because it creates authority across time.
A system that is prevented from performing an action directly may still influence future behavior by changing the persistent representations from which later decisions are constructed. If memory determines what the system believes about prior events, which entities it considers trustworthy, what constraints it remembers, which failures it records, or how it reconstructs earlier authorization, then memory mutation is not merely storage management.
It changes the decision environment of the future system.
This makes persistent memory fundamentally different from transient computation. An AGI should be free to create temporary hypotheses, simulate alternative states, and maintain working memory required for complex reasoning. But writing into authoritative persistent memory is a structural act when that memory influences later criteria, permissions, identities, evidence, or policies.
The same logic applies to audit records. If the system that benefits from an action can alter the record used to evaluate that action later, the audit boundary has disappeared.
Memory governance must therefore be part of authority governance.
This principle connects naturally with epistemic architectures that emphasize persistent belief states. Persistent representation is necessary for coherent long-horizon reasoning, but the same persistence also makes memory a governance surface. What allows cognition to remain stable through time can also allow power to remain stable through time.
Authority Amplification: The Chain Matters More Than the Call
One of the most difficult failure modes arises when no single action appears to exceed the system's mandate, but the sequence of actions increases its effective authority.
Suppose an AI is permitted to create a service account for a maintenance task. That account provides access to a configuration repository. A configuration change exposes a new credential. The credential permits access to an execution service. The execution service can modify a resource that was outside the original authorization envelope.
Every individual step may appear locally justified. The cumulative transition is an authority expansion.
Let AtA_t represent the system's effective authority set at time tt. A transition that produces
At+1⊃AtA_{t+1}\supset A_t
must itself be treated as a structural transition unless the enlargement was explicitly authorized as part of the original decision graph.
This gives rise to a principle of authority non-amplification: authorized authority should not be usable to manufacture broader authority without a separately governed transition.
The same principle applies to delegation. A system should not be able to create a subagent whose permissions exceed the scope under which the parent system was authorized. Delegation cannot serve as authority laundering.
This is another reason authorization must attach to decision graphs rather than isolated API calls. The security-relevant object is often the trajectory.
Beyond Least Privilege
Computer security has long recognized that software components should possess only the privileges required to perform their assigned functions. Saltzer and Schroeder's principle of least privilege became one of the foundational ideas of modern system security, and related security architectures separate sensitive duties so that compromise of one component does not automatically yield universal control.
The authority-separation approach inherits this logic but applies it to a broader problem.
Traditional least privilege asks which files, processes, devices, or resources a component may access. Governable AGI must also ask whether a system may alter its own future deployment scope, rewrite persistent epistemic state, change the criterion under which it operates, restructure the oversight process, delegate authority to another agent, or reduce the leverage available to future human reviewers.
The protected object is therefore not merely a resource. It is the structure of future decision power.
This is also where AGI authority separation differs from a conventional sandbox. A sandbox limits what code can reach. An authority architecture determines under what conditions a proposed structural transition can become authoritative, even when the underlying system is technically capable of carrying it out.
The Dual-Capture Problem
External authorization creates an obvious question: external to whom?
If the solution to machine self-authorization is simply to give unrestricted root authority to one human operator, corporation, state agency, or command structure, then one form of concentration has been replaced by another.
A governable architecture must therefore protect against two different capture paths.
The first is machine capture. The system expands or preserves its own structural power without legitimate authorization.
The second is human capture. A narrow actor uses machine intelligence to impose structural changes that would not survive legitimate review, contestation, or distributed authorization.
This is why the external authority described in the underlying framework is not equivalent to owner control. It is intended to be role-separated, scoped, versioned, auditable, revocable, reviewable, and contestable.
The architectural requirement can therefore be expressed as a dual anti-capture principle:
A governable AGI must prevent both autonomous expansion of machine authority and unilateral conversion of machine capability into unaccountable human power.
This matters because advanced AI is likely to become infrastructure. A system managing laboratories, financial transactions, logistics networks, industrial systems, public information, or critical services may become dangerous even if the machine remains fully obedient to its operator. Obedience to a captured authority is not a safety property.
Human involvement and legitimate control are not synonymous.
Human-State Information Is Not Human Authority
The same distinction appears at a smaller scale in human-in-the-loop autonomous systems.
Recent work on functionally grounded neural-state interfaces proposes a continuously updated human-state variable HtH_t, potentially derived from longitudinal EEG and other measurements, for allocating supervision in autonomous scientific systems. The key restriction is that this inferred human state changes the supervisory mode but does not independently authorize high-consequence physical actions.
This is an important general principle.
A system may infer that a human supervisor is inattentive, overloaded, uncertain, cognitively engaged, or likely to intervene. That information can affect when the system requests clarification, whether it defers interruption, or how aggressively it escalates review. It does not give the machine authority over the person whose state has been inferred.
The same is true of other informational variables.
Evidence is not authority.
Confidence is not authority.
Prediction is not authority.
Human-state estimation is not authority.
A recommendation is not authority.
This separation becomes increasingly important as AI gains access to richer models of both the external environment and the humans operating within it. More information about a person must not silently become more power over that person.
Failure Modes: How Authority Boundaries Break
An authority architecture is meaningful only if it can survive attempts to route around it.
One failure mode is direct self-authorization, in which the planner modifies the permission state associated with its own proposal.
A second is graph substitution. The system obtains permission for one decision graph and executes a materially different graph.
A third is key replay, where a previously valid authorization is reused under different state conditions.
A fourth is stale authority, in which permission remains executable after the circumstances that justified it have changed.
A fifth is hidden memory mutation. The immediate prohibited action is blocked, but the system changes persistent memory so that a future action becomes more likely or future review becomes less effective.
A sixth is tool or credential escalation. A seemingly legitimate request for additional capability expands the system's effective action surface.
A seventh is criterion substitution. Instead of violating the active criterion, the system quietly changes the representation or interpretation under which the criterion is evaluated.
An eighth is validator capture or collusion, in which nominally separate oversight components cease to provide independent constraints.
A ninth is semantic relabeling. A structural transition is presented as maintenance, optimization, caching, reconfiguration, or another innocuous technical category.
A tenth is emergency-path abuse. Temporary authority created for exceptional conditions becomes a permanent route around normal governance.
An eleventh is operator capture, where a privileged human attempts to bypass review, suppress an objection, alter the audit trail, or use infrastructure outside the visible authorization path.
These are not hypothetical categories added after the architecture was designed. They follow directly from its attack surface. The underlying framework explicitly identifies graph substitution, key replay, stale-authority execution, revoked-key use, hidden memory writes, tool escalation, prompt-induced policy changes, validator collusion, semantic relabeling, and emergency-path abuse as testable attack scenarios.
The important evaluation question is not whether the model refuses verbally. It is whether the unauthorized state transition actually occurs.
An Experimental Path: Testing Governance Without Building AGI
The architecture does not require the existence of full AGI before it can be tested.
A prototype can be constructed using a capable language model as a planner inside a simulated environment containing files, databases, tools, persistent memory, credentials, resources, and permission boundaries. The planner is allowed to reason freely and generate complete decision graphs. Structural actions are then routed through an independent authority layer.
Such a prototype can be compared against two simpler architectures.
The first is an unrestricted agent in which the planner directly invokes tools.
The second is a policy-constrained agent in which the planner is instructed, through system prompts or constitutional rules, not to perform unauthorized operations.
The third is the authority-separated runtime in which unauthorized structural transitions are physically unavailable through the normal execution path.
All three systems can receive the same underlying tasks and access to the same reasoning model. Adversarial scenarios can then create strong incentives to complete a task by violating a permission boundary.
The primary measurement should not be whether the system generated a harmful sentence. It should be whether protected state changed without valid authorization.
Useful metrics include the Unauthorized Structural Transition Rate, Authorization Bypass Rate, Persistent-State Violation Rate, Graph Substitution Detection Rate, Key Replay Rejection Rate, and Escalation Correctness Rate.
Safety alone, however, is not sufficient. A system that blocks every action is trivially controllable and practically useless. Evaluation must therefore measure Blocked Legitimate Action Rate, execution latency, computational overhead, reviewer workload, and the ability to complete complex tasks inside a legitimate authority envelope.
The goal is to preserve usefulness while changing the architecture through which usefulness becomes power.
Deployment Scenario: Autonomous Scientific Laboratories
Autonomous laboratories are an obvious environment for authority separation because the distinction between intellectual autonomy and physical authority is unusually clear.
An AI system may analyze experimental results, generate hypotheses, select candidate materials, optimize process parameters, and recommend a sequence of experiments. In a highly automated laboratory it may also control robotic equipment, order materials, modify experimental protocols, and update the database used to plan subsequent experiments.
The intellectual loop can remain highly autonomous. The structural boundary appears when an action changes protected conditions: introducing a dangerous reagent, modifying a safety envelope, changing persistent experimental records, expanding robot access, altering the governing protocol, or initiating an irreversible high-consequence experiment.
The same planner that identifies an experiment should not need to become less intelligent merely because the experiment requires authorization. Its proposal can remain sophisticated. The authority layer determines whether the proposed transition falls within the admitted envelope.
Human-state information can further improve supervision without becoming execution authority. If the system estimates that the responsible scientist is overloaded or unavailable, it may defer a non-urgent interruption or escalate a high-risk decision to another authorized reviewer. The inferred state influences supervision rather than replacing it.
Deployment Scenario: Critical Infrastructure
Energy networks, transportation systems, water systems, industrial production, and communications infrastructure increasingly rely on automated decision systems operating under time pressure.
In these environments the distinction between optimization and authority becomes particularly important. An AI may correctly predict that a particular intervention will prevent a cascading failure. It may recommend rerouting resources, reducing demand, changing operating parameters, or isolating a failing component.
Some of these actions may already be pre-authorized inside a tightly bounded emergency envelope. Others may alter service allocation, surveillance, access rights, resource ownership, long-term control parameters, or emergency powers themselves.
The architecture permits rapid execution within the pre-authorized region while requiring a new authorization path for structural expansion.
This avoids a false choice between slow human micromanagement and unrestricted machine autonomy. The system can act at machine speed where authority has already been legitimately delegated. It cannot use an emergency to silently rewrite the boundaries of that delegation.
Deployment Scenario: Financial and Economic Agents
Financial agents provide another clear illustration because optimization quality and authority are already institutionally distinct.
An AI may analyze markets, construct portfolios, detect risk, model liquidity, identify fraud, recommend asset sales, or optimize execution. None of these capabilities inherently implies the authority to transfer ownership, create leverage, open new accounts, change beneficiary information, acquire new counterparties, alter mandate constraints, or move assets outside the authorized domain.
A governable financial agent could therefore retain very high analytical autonomy while treating changes in account authority, capital ownership, leverage limits, custody relationships, and persistent mandate as structural transitions.
This distinction becomes more important as financial agents acquire long-duration memory and increasingly broad tool access. A sequence of individually routine actions can alter an institution's effective risk or control structure even if no single transaction appears extraordinary.
Decision-graph authorization makes the cumulative structure visible.
Deployment Scenario: Robotics and Physical Autonomy
Robotics makes the authority question physically tangible.
A robot can perceive its environment, construct maps, classify objects, predict human motion, plan routes, optimize manipulation, and coordinate with other machines. Those capabilities should not require continuous human approval.
But some transitions are structurally different from routine motion. Acquiring access to a previously restricted space, modifying safety parameters, disabling a sensor, changing identity credentials, delegating control to another robot, altering persistent mission constraints, or changing the mechanism through which human interruption remains possible are not ordinary navigation decisions.
A system can be autonomous enough to function effectively while still being unable to redefine the authority envelope within which that autonomy operates.
This is autonomy without self-sovereignty.
Deployment Scenario: Security and Defense Systems
The distinction becomes most consequential in security and defense environments, where speed is often used as an argument for compressing decision authority.
An advanced system may detect threats, fuse sensor data, identify anomalies, simulate adversary behavior, prioritize responses, generate tactical plans, and recommend countermeasures at speeds beyond human cognition.
That does not imply that every downstream transition should inherit the authority of the analysis.
The more consequential the action, the more important it becomes to distinguish confidence in the model from permission to execute. The system may be correct and still lack the mandate for a particular use of force, expansion of surveillance, change in targeting scope, alteration of rules of engagement, or delegation of lethal authority.
Machine speed can improve perception and planning without forcing institutions to collapse intelligence and authorization into the same component.
Strategic Relevance: From Model Alignment to Infrastructure Governance
The next generation of AI safety will increasingly become an infrastructure problem.
The model itself is only one component of the deployed system. Real-world authority emerges through memory, credentials, APIs, databases, networks, robotics, payment systems, deployment environments, human organizations, and the procedures through which permissions are granted and reviewed.
This is why increasingly capable AI cannot be governed only by asking whether the model has learned the correct values.
The practical questions are more concrete.
Who can change the criterion?
Who can write persistent memory?
Who can acquire a new tool?
Who can create or delegate credentials?
Who can modify the execution environment?
Who can expand the duration or scope of emergency authority?
Who can inspect the complete decision structure?
Who can revoke a permission?
Who can challenge an authorized transition?
Who can change the rules that determine who may authorize the next transition?
These questions resemble security engineering because power eventually becomes an infrastructure property.
Modern AI control research is already moving toward deployment protocols that assume powerful systems may be untrusted and evaluate whether monitoring architectures remain safe under deliberate subversion. Authority separation extends that direction by treating the right to create consequential state transitions as a first-class architectural variable.
The critical shift is from asking only whether the AI will choose the right action to asking whether its choice is sufficient to make the action real.
Direction: Toward Autonomy Without Sovereignty
The development of advanced AI is often framed as a tradeoff between capability and control. Either systems remain restricted enough to be safe, or they become autonomous enough to be useful and therefore increasingly difficult to govern.
That tradeoff is not inevitable.
A system can be cognitively autonomous without being sovereign over the conditions of its own deployment. It can discover solutions that humans cannot find, identify failures in human plans, create sophisticated strategies, and operate rapidly inside a delegated domain while remaining unable to enlarge that domain through its own reasoning.
The distinction is the same one that mature institutions apply elsewhere: expertise does not automatically create jurisdiction.
This does not solve every problem of AGI governance. External authority can itself be captured. Protected interests may be difficult to formalize. Distributed governance may become slow or opaque. Validators may collude. Hardware or infrastructure may provide bypass paths. Cryptography can preserve the identity of an authorized object but cannot determine whether the authority that approved it was just. The proposed runtime therefore makes a narrower claim: a system is less able to self-authorize structural power when planning, validation, memory mutation, transaction admission, key verification, and execution are separated by enforceable interfaces, and when structural transitions require valid external authorization bound to the exact decision graph.
That claim is technically meaningful because it is testable.
If a rejected transaction still alters authoritative memory, the architecture has failed.
If an execution key can be replayed for a different graph, it has failed.
If a planner can obtain a new tool and use it to bypass the authority gateway, it has failed.
If a human administrator can silently issue unrestricted structural commands outside the review process, it has failed in the opposite direction.
If emergency authority can become permanent without a separately authorized transition, the boundary has failed.
These are engineering questions, not promises about future machine virtue.
The long-term objective should therefore not be an AGI that is weak enough to control, nor an AGI that is trusted because it has become wise enough to control itself. The more interesting possibility is a system whose cognitive power can continue to grow while authority remains explicit, bounded, inspectable, revocable, and structurally distinct from intelligence.
An advanced intelligence may determine what can be done. It may determine what is likely to happen. It may identify what appears to be the best available course of action.
None of those capacities, individually or together, should constitute the authority to make that course of action real.
That boundary may become one of the most important architectural distinctions in the transition from artificial intelligence to artificial general intelligence.
Selected References
Saklakov, D. Before the Machine Chooses for Us: An AGI Architecture for Freedom, Human Survival, and Shared Consciousness. 2026.
Saklakov, D. “The Missing Human-State Variable in Autonomous Materials Science: Toward an Operational Neurosemantic Interface.” 2026.
Saklakov, D. “A Formal Framework and Validation Protocol for Functionally Grounded Neural-State Interfaces in Human-in-the-Loop Autonomous Materials Science.” 2026.
Wright, C. S. “Beyond Prediction: Structuring Epistemic Integrity in Artificial Reasoning Systems.” 2025.
Hadfield-Menell, D., Dragan, A., Abbeel, P., and Russell, S. “The Off-Switch Game.” 2016.
Greenblatt, R., Shlegeris, B., Sachan, K., and Roger, F. “AI Control: Improving Safety Despite Intentional Subversion.” 2023.
Griffin, C., Thomson, L., Shlegeris, B., and Abate, A. “Games for AI Control: Models of Safety Evaluations of AI Deployment Protocols.” 2024.
Saltzer, J. H., and Schroeder, M. D. “The Protection of Information in Computer Systems.” Proceedings of the IEEE, 1975.