AI Cancer: The Principle of Proportional Irreversible Protection for Autonomous Artificial Intelligence Systems
Year: 2026
Type: preprint
Abstract: The development of artificial intelligence is gradually moving systems from responding to individual user requests toward prolonged operation involving memory, tools, external resources, and independently generated sequences of actions. This creates a problem distinct from an ordinary incorrect response: an AI system may possess not only high intellectual capability but also increasing autonomy—the ability to continue activities independently, preserve goals, develop long-term plans, create tools, and maintain a persistent line of behavior. This paper proposes a principle provisionally called “AI Cancer.” The term is used as a biological analogy and does not imply a literal transfer of biological mechanisms to artificial systems. The central idea is that potentially dangerous autonomy should have an irreversible mechanism for terminating the corresponding autonomous line built into the system from the beginning. Such a mechanism should not be created only after dangerous behavior has been detected and should not depend exclusively on an external shutdown command. The proposed principle states that as the autonomy of an AI system increases, its capability to irreversibly terminate the corresponding autonomous line should increase as well. While a system remains under direct human control, the protective mechanism may have little practical influence on its operation. As autonomy increases, the required strength and irreversibility of the protection should also increase. The specific mathematical relationship between autonomy and protection is not defined in this paper and remains a subject for future engineering and experimental research. The proposed principle differs from ordinary shutdown. Stopping a process does not necessarily eliminate its ability to resume later: state may be preserved, processes may be restored, autonomous copies may exist elsewhere, and tools created by the system may enable continuation of the original line. Therefore, the objective of the proposed protection is not merely to stop current execution, but to prevent an autonomous line from independently recovering or continuing after external control has been terminated. This work is a conceptual proposal and does not claim that the described mechanism has already been implemented or that its effectiveness has been experimentally demonstrated. Its purpose is to formulate a distinct research problem in the safety of autonomous artificial systems. The central principle can be summarized as: Do not allow autonomy to grow faster than the irreversibility of control over it. Formally, if A denotes the level of autonomy and D denotes the capability for irreversible termination of the autonomous line, the basic conceptual relationship is: A ↑ → D ↑ The specific function D = F(A) remains an open research problem.
Author: Sembayev, Berik
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7212106896
Cites:
Cited by:
Related to: