The Holistic Concept and Challenges of Containment

Key Insight

Containment represents humanity's paramount challenge in the twenty-first century: fostering sufficient legitimate political power, wisdom, technical mastery, and robust norms to constrain technologies, thereby ensuring they consistently produce more benefits than harms. This endeavor, described as a 'new kind of grand bargain', aims to simultaneously harness and control the exponential technological wave, facilitating the creation of sustainable and flourishing societies while averting severe catastrophes without inadvertently creating a dystopia through overly intrusive controls. Fundamentally, containment means possessing the capability to drastically curtail or entirely stop the negative impacts of technology, ranging from localized incidents to planetary and existential threats. This encompasses both vigorous enforcement against the misuse of proliferated technologies and proactively guiding the development, direction, and governance of nascent ones.

Contained technology is characterized by known, managed, and mitigated modes of failure, where the mechanisms to shape and govern it advance in parallel with its capabilities. Instead of a 'magic box' to seal away technology, containment is better understood as a system of 'guardrails' operating at various levels and through diverse implementations, including AI alignment research, laboratory design protocols, and international treaties. These guardrails must be robust enough, in theory, to prevent a runaway catastrophe. The strategy for containment must be adapted to the specific nature of each technology, channeling its development towards more controllable directions. This involves posing critical questions about its characteristics: Is it omni-use or general-purpose, making it harder to contain, or is it specific? Is it becoming dematerialized into 'bits', leading to hyper-evolution that is difficult to track? Are its price and complexity rapidly decreasing, thereby increasing proliferation risks? Are viable, safer alternatives readily available to facilitate phasing out harmful uses? Does it enable asymmetric impacts? Does it possess autonomous characteristics that reduce the need for human oversight? Does it confer an outsized geopolitical strategic advantage, making restrictions difficult? Does it favor offense or defense, with a preference for defense aiding containment? Are there resource or engineering constraints that inherently limit its invention, development, and deployment in the short term?

Achieving containment is not feasible under current global conditions; it necessitates 'something dramatically new' – an all-encompassing program integrating safety, ethics, regulation, and control that currently lacks a definitive name or established framework. A significant obstacle lies in the psychological difficulty humans face when confronting diffuse, uncertain, and temporally distant threats, akin to climate change, which only recently gained clear focus through quantifiable metrics such as atmospheric carbon at 420 ppm in 2022. Unlike climate change, technological risk lacks a universal, objective metric or widespread consensus among scientists and the public, impeding the formation of a popular movement or a shared understanding across national capitals and boardrooms. Existing elites often exhibit 'pessimism aversion', preferring to avoid honest discussions about dangers they cannot control using traditional levers. However, the 'coming wave' has not yet fully arrived, presenting a unique opportunity to influence its final form. The path to containment involves 'marginal gains'—a continuous accumulation of small efforts aimed at buying time, slowing down development, building alliances, and advancing technical work, ultimately altering the underlying conditions to make containment and human flourishing possible, despite inherent uncertainties and the absence of quick fixes.