The Rise and Ubiquitous Impact of Large Language Models

Key Insight

Processing natural language was long considered too complex for AI, until November 2022 when OpenAI released ChatGPT, a large language model (LLM) that gained over 1 million users in a week. ChatGPT, a highly powerful and versatile chatbot, generates instantaneous, fluent prose, capable of writing essays, business plans, syllabi, and even Python scripts in various styles. This capability stems from its ability to predict what comes next in a sequence of information, building on research from Google in 2017 on how AI systems focus on important parts of data series.

LLMs, also known as transformers, operate by ingesting vast amounts of language data, learning abstract representations, and then predicting subsequent elements. They construct an 'attention map' by organizing frequently occurring letter groups or punctuation into 'tokens'—a custom vocabulary the model creates—and determine the importance of each token within a sentence. For instance, in the sentence 'There is going to be a fairly major storm tomorrow in Brazil,' the model identifies 'storm,' 'tomorrow,' and 'Brazil' as key features to logically autocomplete the next sequence.

The scale and capability of LLMs have grown exponentially, with GPT-2 (1.5 billion parameters) followed by GPT-3 (175 billion parameters) in 2020, and GPT-4 in 2023. GPT-4 can compose poetry, imitate literary styles, generate business plans, and ace standardized tests like the bar exam and GRE. It also handles images and code, creating 3D games, smartphone apps, debugging, identifying contract weaknesses, and suggesting drug compounds. These models are trained on trillions of words, equivalent to six orders of magnitude more than a human reads in a lifetime, enabling them to achieve remarkable performance in tasks like medical licensing exams. This rapid advancement ensures AI will become as ubiquitous and consequential as the internet, but at a much faster adoption rate.