OpenAI's Evolution, Funding Challenges, and Technological Breakthroughs

Key Insight

By 2018, OpenAI faced significant internal struggles, floundering with projects like applying AI to videogames and a robotic hand for Rubik's Cube. Elon Musk, critical of OpenAI falling 'hopelessly behind Google,' which had seen successes like DeepMind's AlphaGo beating a Go champion and WaveNet emulating human speech, grew dissatisfied. Musk proposed taking a majority stake or integrating OpenAI into Tesla, but Altman rejected this, fearing the loss of key personnel. Musk subsequently withdrew his support, prompting Hoffman to commit another $10 million and assist in raising additional funds. Altman then stepped down as Y Combinator president to become OpenAI's CEO, with Hoffman joining its board of directors.

The cost of AI development proved immense, reversing the trend of declining startup barriers. Top AI researchers commanded salaries of $1 million or more annually, with OpenAI reportedly offering Ilya Sutskever $1.9 million a year plus stock to lure him from Google. Even more substantial was the 'compute' cost for training and running models, requiring weeks or months of continuous computer time. A pivotal breakthrough came with Google's 2017 'Transformer' paper, offering a new model for teaching neural nets to deduce human meaning and respond naturally. This architecture allowed OpenAI to make significant progress with large language models (LLMs) by evaluating chunks of words for context, but it necessitated vast, expensive-to-run models.

To address the escalating financial demands, Altman devised a solution in 2019: a 'capped profit' subsidiary, answerable to the nonprofit board. This allowed OpenAI to seek new investors while prioritizing its charter over profit, capping investor returns at 100 times their original investment. Hoffman proved pivotal again, leading this first commercial round with another substantial check, despite the absence of a traditional business or product plan, calling it 'a bet on them being able to do something magical with AI.' This enabled the development of groundbreaking models like GPT-1 (2018), which could understand and respond to plain English queries. GPT-2 (2019) advanced, generating coherent multi-paragraph text, and by 2020, GPT-3 processed 175 billion parameters, trained on 300 billion words, demonstrating advanced creativity, intuition, and even sarcasm detection, solidifying Hoffman's 'all in' commitment to AI.

Microsoft invested $1 billion in OpenAI in 2019, further solidifying its financial position.