Editor’s note: This work is part of AI watchdog, The Atlantic‘s ongoing investigation into the generative AI industry.
As they struggle to keep their systems online, AI companies are making it expensive for the rest of us. Large language models like ChatGPT and Claude are so resource hungry that tech companies may buy 70 percent of the world’s supply of high-end computer memory, creating a shortage. As a result, computer memory and data storage prices are skyrocketing: Hard drives I bought for my reporting two years ago for $350 apiece cost $800 when I checked two weeks ago and are now sold out. Prices of some laptops have risen by up to 50 percent, and budget computers have been hit the hardest. Affordable entry-level computers could “disappear by 2028,” according to one forecast. And the storage shortage is expected to continue for years.
The storage will be housed in data centers that technology companies are expanding at an incredible rate. They plan to multiply the total capacity of US data centers eightfold in the next few years. The demand for electricity at these locations is already so great that some companies are converting jet engines to power them.
The problem isn’t just that AI is being deployed so widely or so quickly. Other computing technologies have experienced similarly massive growth without such a large surge in power or shortages of computing components: videos and music are now streamed around the globe, accounting for many terabytes of Internet traffic every day; The smartphone boom required the production of billions of devices that now transmit massive amounts of data; Billions of household devices are now part of the Internet of Things; And entire industries have shifted their operations to cloud software, hosted in data centers rather than in the air.
The problem with generative AI, as the industry jargon goes, is that it is not scalable. The cost of growing from, say, a thousand users to a million is a key factor that venture capitalists consider when evaluating startups. They want to ensure that the cost of adding each new user decreases over time, allowing the company to support millions of users and generate increasing profits. This is achieved in part by carefully developing computer systems that can efficiently handle more users who want to post photos, hail Ubers or stream music.
With generative AI, the work of building efficient, scalable systems isn’t done. And the problem is compounded by ever-larger generative AI models, which independent estimates have grown from 175 billion parameters in 2020 to over 1 trillion today (the actual sizes of the models that power products like Claude and ChatGPT are secret). The large In large language model shouldn’t be a selling point. But the industry’s observation that larger models tend to outperform smaller ones has led to a totemic belief in “scaling laws” that suggest any problem can be solved by simply making models larger. “Maybe AI with 10 gigawatts of computing power can figure out how to cure cancer,” OpenAI CEO Sam Altman wrote on his blog in September.
Nevertheless, returns are declining. The larger an AI model is, the less it improves with each parameter added. Therefore, it needs to scale up faster to maintain steady progress. I asked a few AI researchers if they could name any other real-world software that scales so poorly. None of them could imagine anything. Even outside the world of software, it’s difficult to find a comparable example, since economies of scale are the principle that made light bulbs, cars, and clothing so affordable. From an economic and technical standpoint, generative AI might be the worst technology ever deployed.
But given the huge investment behind the current bloated approach, there may not be much will to change. Ilya Sutskever, co-founder and former chief scientist at OpenAI, said in a November interview that companies use the brute force approach “because it gives you a very low-risk way to invest your resources.” It would be harder, he argued, to invest in research that would overhaul a product that currently commands trillion-dollar valuations. Those who suspect we are in an AI-driven bubble economy have pointed out that the profitability of these companies remains an open question, largely due to the technology’s high costs and inefficiency.
Efficiency is a core principle of computer science. One of the first things students learn is that it is easy to write a program that sorts a list of 50 words. But if you give this program 50 million In other words, it will probably run out of memory or take hours to finish. Much of computer science is about learning clever coding techniques that prevent this from happening. Many of these techniques exploit repeating patterns in the data so that as more input comes in, the program requires less time or memory to process each additional bit. This efficiency is one of the reasons modern smartphones and computers are so powerful and affordable. That’s what it’s called logarithmic Scaling, and if you graph it, it looks like this:
Large language models cannot be scaled logarithmically. When they are given more words to process, they become slower and use more memory – time and resources increase faster as input increases. Technically, LLMs scale squarely. Every computer science student knows that this is very bad.
Epoch AI, an organization that tries to determine the cost of running AI models, published a graphic last year that is reproduced here with permission. It shows the exponentially increasing cost of providing more “tokens” – the words users type into chatbots – with multiple public AI models.
AI doesn’t have to be structured like that. Traditionally, the goal of AI has been to solve problems in a way that simulates human mental processes. The researchers observed their own thinking and tried to translate their mental habits into code. This approach was largely abandoned, partly due to the difficulty of recognizing and articulating the rules of human thought, but it had the advantage of using far fewer resources and data.
Today’s approach to AI does not attempt to describe the rules of human thought; Instead, it gives a computer millions of examples to imitate. The large number of examples is one reason that large models perform better at generating speech, images, and music than small ones – they have more material to draw from. Some researchers want to bring back the old, more efficient approach and combine it with the modern approach, but so far these projects have not attracted nearly as much attention or funding as the models that power chatbots.
Chatbot companies are aware that their products are inefficient. Some have found techniques to improve performance, but they have not provided significant benefits. Occasionally, companies claim to have achieved breakthroughs – Anthropic CEO Dario Amodei has called them “compute multipliers” – but they are usually vaguely described and there is no evidence that the fundamental problems of quadratic scaling and exploding model size have been overcome. (Anthropic declined to comment on the record when I inquired about it.)
Some researchers are working on extremely small models that require less data and less computing power. I spoke to Alexia Jolicoeur-Martineau, an AI researcher at Microsoft who independently designed one, and asked her about the industry’s brute force approach. “It’s a little crazy,” she told me. “At some point you have to learn to be a little more efficient.”
Last year, Jolicoeur-Martineau won a $50,000 prize for her work on a “tiny recursive model” that doesn’t consume large amounts of computing resources. “The idea that you have to rely on massive base models trained by a large company for millions of dollars to succeed at difficult tasks is a trap,” she wrote. Their model isn’t a replacement for an LLM – it’s intended to solve logical problems in areas like biology and electrical engineering, not generate language – but it can do some of the tasks that much larger AI models are currently used for.
Yet we seem to be stuck with LLMs, perhaps because they have been marketed so aggressively. They are now added to everything, whether you want them or not. In 2024 and 2025, they were integrated into both Windows and MacOS, meaning more computing power is now required to run a basic personal computer. Smartphones are also being sold with improved hardware as companies anticipate new AI features. Inefficient AI is also being built into popular programs like Adobe Photoshop and Microsoft Word, meaning computers need to be more powerful to run this software.
This is all particularly bad because computers aren’t improving as quickly as they used to. Since the 1950s, manufacturers have learned to make microchips faster, smaller and cheaper, a trend known colloquially as Moore’s Law. But in recent years, components have become so small that manufacturers have reached the limits of further shrinking at the molecular level, significantly slowing progress.
Instead of downsizing components, manufacturers have focused on developing new hardware tailored for AI. This has led to occasional incremental performance improvements, but none have come close to keeping up with the exponential curve of increasing demands on AI.
Ultimately, inefficiency may be of little concern to people in the technology industry who believe they are reproducing intelligence themselves. Many in Silicon Valley are almost religiously convinced that LLMs, which are ultimately just statistical language-generating software, could produce something resembling a mind—despite the software’s inability to remember basic facts, its lack of common sense, and its complete dissimilarity to a biological brain. Even Yann LeCun, one of the “godfathers” of AI, told it The New York Times recently that “LLMs are not a path to superintelligence or even human-level intelligence.” But the mythological lure of AI is so strong that many engineers believe nothing should stand in their way. Not even the basic task of writing efficient software.
https://www.theatlantic.com/technology/2026/07/generative-ai-engineering-disaster/687901/
