Sam Altman Criticises Local AI Models Amid Growing Global Memory Shortage
OpenAI CEO Sam Altman believes running AI models locally on personal computers could worsen the global memory shortage, arguing that cloud data centres offer better memory utilisation.
OpenAI CEO Sam Altman has voiced his fears about the rise of the practice of performing AI calculations locally on personal computers, saying that it might place extra strain on the already scarce supply of memory worldwide. Despite the fact that running AI locally on one's computer is being widely touted by big tech firms as the future of personal computing, Altman says that spreading out memory in individual devices might not be the best way to go about it.
In the course of an interview conducted on the Cheeky Pint podcast recently, Altman talked about the memory shortage situation and explained why he thinks it would be more effective to perform AI computations in centralised data centres. As per Altman, in a situation of constrained supply, maximum use of memory capacity must be made, which might be easier through cloud infrastructure.
“You want maximum utilization in the cloud if you're trying to deal with this very supply-constrained environment. And somehow, many people's intuition is the opposite of that,” Altman said during the discussion.
His comments come at a time when local AI computing is gaining considerable attention across the technology industry. Companies such as Nvidia and Microsoft have been highlighting the capabilities of modern PC hardware designed to run AI models directly on users' devices. These systems come with larger memory capacities and specialised processors, allowing users to perform certain AI tasks without constantly relying on cloud-based services.
One of the benefits of using local AI is associated with privacy, offline capability, and less reliance on cloud subscriptions. The user could even have more control over their data since all operations can be performed right on the user's device. Still, according to Altman, this technology can lead to an issue on a worldwide scale.
As was mentioned during the podcast, Altman challenged the idea that the rising availability of open-source AI models would make the world less dependent on large data centers in the future. According to Altman, if millions of people begin to use AI models on their own computers, then much of the DRAM will not be used.
“A lot of people want local inference. And somehow, they assume that the problem is solved now because we are going to have open-source models. We're going to be able to run them on our computers. And so we're not going to need all the stuff in the data center,” Altman explained.
Moreover, he stated that having plenty of memory within each computer is not necessarily an optimal solution since there might be cases when this memory can be used only infrequently. “From the perspective of allocating memory around the world, actually it's quite awful to have everybody have a bunch of the world's very limited supply of DRAM in their own computer. And being used 5% of the time,” he said.
On the other hand, it might also seem that Altman’s claim reflects the commercial interests of companies that provide AI via cloud-based platforms. Providers of AI such as OpenAI are highly dependent on the infrastructure of data centres. If people start using the open source models in their own computers, this can lead to a significant decrease in the usage of paid cloud services.
Also, during the discussion, the severity of the shortage of memory was raised. When asked for possible solutions to the problem, Altman responded quite negatively with just one word, “Bad”. Indeed, the shortage is expected to remain a serious problem in the coming years until 2030.
Another interesting part of Altman's argument focused on the memory industry's ability to respond to increasing demand. In a typical market situation, rising demand and higher prices would encourage manufacturers to expand production capacity. This is often explained through economist Adam Smith's concept of the invisible hand, where market forces help balance supply and demand.
However, Altman believes the memory industry is not responding as quickly as expected. “I think the invisible hand is failing right now,” he said, questioning whether normal market forces are sufficient to address the current supply challenges.
He also gave his reason why memory producers might be unwilling to venture into new production facilities. Establishing semiconductor facilities is a very expensive process, and an expansion of production too quickly might bring yet another challenge if at a certain moment the demand for chips becomes lower. This will cause an oversupply of chips and will drive the price down.
According to Altman, memory producers had been through similar experiences before and thus are cautious about making such mistakes again. Altman stated that memory producers remember how close they were to going broke because of too many fabrication facilities that were built under exaggerated forecasts of demand.
At the same time, the lack of memory has already become an issue impacting the availability and cost of advanced computing hardware, as the new-age AI-oriented PCs and workstations for running bigger models locally need much high-speed memory. For example, Nvidia RTX Spark solutions reveal the growing trend of using hardware specifically oriented towards local AI, yet their high cost might prevent them from reaching wide adoption.
All in all, Altman's statement reflects the crucial dilemma related to the future of artificial intelligence. While cloud data centers provide users with shared infrastructure and resources, local AI allows for more privacy, independence and flexibility. There are certain pros and cons for both alternatives, and the industry's future depends on the ability to find the right compromise between them.
For consumers, the local AI solution will get increasingly appealing as technology advances and open-source models become more advanced. However, as long as memory supplies remain low and its price high, running powerful AI models on local computing devices will remain costly.
Comments