Mustafa Suleyman frames modern Artificial Intelligence progress as an exponential phenomenon that runs counter to human intuition about linear change. From the time he began work on Artificial Intelligence in 2010 to now, the amount of training data that goes into frontier Artificial Intelligence models has grown by a staggering 1 trillion times, from roughly 10¹⁴ flops for early systems to over 10²⁶ flops for today’s largest models. He presents that jump as the central force behind recent advances and rejects recurring claims that development is close to hitting a wall.
He argues that the acceleration is coming from several layers of the computing stack at once. Nvidia’s chips have delivered an over sevenfold increase in raw performance in just six years, from 312 teraflops in 2020 to 2,250 teraflops today. Microsoft’s Maia 200 chip, launched this January, delivers 30% better performance per dollar than any other hardware in the company’s fleet. HBM3 triples the bandwidth of its predecessor, while interconnect technologies such as NVLink and InfiniBand link hundreds of thousands of GPUs into warehouse-size supercomputers. Where training a language model took 167 minutes on eight GPUs in 2020, it now takes under four minutes on equivalent modern hardware. Moore’s Law would predict only about a 5x improvement over this period. He says the industry saw 50x instead, and notes a shift from two GPUs training AlexNet in 2012 to over 100,000 GPUs in today’s largest clusters.
Suleyman also points to software gains that are making models cheaper to train and serve. Research from Epoch AI suggests that the compute required to reach a fixed performance level halves approximately every eight months, much faster than the traditional 18-to-24-month doubling of Moore’s Law. The costs of serving some recent models have collapsed by a factor of up to 900 on an annualized basis. He says those trends indicate that Artificial Intelligence is becoming radically cheaper to deploy even as capabilities improve.
Looking ahead, he describes a continued surge rather than a slowdown. Leading labs are growing capacity at nearly 4x annually, and since 2020, the compute used to train frontier models has grown 5x every year. Global Artificial Intelligence-relevant compute is forecast to hit 100 million H100-equivalents by 2027, a tenfold increase in three years. He says that could amount to another 1,000x in effective compute by the end of 2028, and that by 2030 an additional 200 gigawatts of compute could come online every year. He links that scale to a transition from chatbots to nearly human-level agents capable of carrying out extended, semiautonomous work across industries, while acknowledging energy as the clearest constraint. A single refrigerator-size Artificial Intelligence rack consumes 120 kilowatts, equivalent to 100 homes, but he argues that falling solar and battery costs create a path for cleaner scaling.