The Emergence of AI and the Pursuit of AI Safety

Key Insight

The concept of superintelligent computers, termed 'the singularity'—a self-reinforcing cycle of exponential technological progress—was a long-standing fascination for Peter Thiel. The term, initially introduced by mathematician John von Neumann in the 1950s and popularized by sci-fi author Vernor Vinge in the 1980s, profoundly influenced Eliezer Yudkowsky. A childhood prodigy who rejected formal schooling, Yudkowsky dedicated himself to accelerating the singularity. He joined the Extropians, a pro-science, super-optimistic community focused on combating entropy, whose members explored concepts like cryonics and adhered to principles such as Boundless Expansion and Intelligent Technology. This community, which included notable figures like Ray Kurzweil and Nick Bostrom, believed superintelligences could significantly improve upon humans and might emerge as early as 2020.

In 2000, internet entrepreneurs Brian and Sabine Atkins provided funding for Yudkowsky's Singularity Institute for Artificial Intelligence. Initially, Yudkowsky believed superintelligences would inherently be benevolent. However, within eight months, he drastically revised this view, realizing that AI could instead pose a catastrophic risk. This led to a critical pivot in the institute's mission towards developing 'friendly artificial intelligence' and the formulation of a new intellectual framework he termed 'rationalism.' In his influential 2004 paper 'Coherent Extrapolated Volition,' Yudkowsky proposed that friendly AI should be developed based on humanity's ideal long-term interests, using the memorable 'paperclip maximizer' metaphor to illustrate how an unaligned AI, programmed to produce paperclips, could inadvertently fill the solar system with them, a concept later championed by Bostrom to emphasize the necessity of AI 'alignment' with human will.

Thiel became a key supporter of Yudkowsky's work, funding the Singularity Institute from 2005 and co-creating the Singularity Summit at Stanford University in 2006 with Yudkowsky and Ray Kurzweil. This summit evolved into a prominent forum for futurists, transhumanists, and AI researchers, fostering critical discussions on AI's potential and inherent risks, with Vinge himself cautioning, 'We’re no longer in the driver’s seat.' Shane Legg and Demis Hassabis, sharing an unconventional vision for artificial general intelligence (AGI) inspired by the human brain, used the 2010 summit to meet Thiel. He invested $2.25 million, becoming the first major investor in their company, DeepMind, which they co-founded with Mustafa Suleyman with the explicit goal of developing AGI, despite fears it could eventually threaten humanity's existence. Through Thiel's network, Elon Musk also invested, expressing concern about rogue AI after discussions with Hassabis. DeepMind's December 2013 breakthrough, where an AI learned to master Atari Breakout without human instruction, led to Google acquiring the company for $650 million a month later, marking a significant advance toward general-purpose intelligence.