Ironwood is Google’s latest tensor processing unit
Nvidia’s position as the dominant supplier of AI chips could be threatened by a specialized chip launched by Google, with reports suggesting that companies like Meta and Anthropic are looking to spend billions on Google’s tensor processing units.
What is a TPU?
Much of the success of the artificial intelligence industry relies on graphics processing units (GPUs), a type of computer chip capable of performing many parallel calculations at the same time, rather than one after the other like the computer processing units (CPUs) that power most computers.
GPUs were originally developed to facilitate computer graphics, as the name suggests, and gaming. “If I have a lot of pixels in a space and I need to rotate them to calculate a new camera view, this is an operation that can be done in parallel, for many different pixels,” explains Francesco Conti of the University of Bologna in Italy.
This ability to perform calculations in parallel has proven useful for training and running AI models, which often use calculations involving large grids of numbers performed at the same time, called matrix multiplication. “GPUs are a very general architecture, but they are extremely suited to applications with a high degree of parallelism,” explains Conti.
However, because they weren’t originally designed for AI, there can be inefficiencies in how GPUs translate calculations done on the chips. According to Conti, tensor processing units (TPUs), initially developed by Google in 2016, are designed solely around matrix multiplication, which are the main calculations needed to train and run large AI models.
This year, Google launched the seventh generation of its TPU, called Ironwood, which powers many of the company’s AI models like Gemini and AlphaFold, a protein modeling model.
Are TPUs much better than GPUs for AI?
Technologically, TPUs are more of a subset of GPUs than an entirely different chip, says Simon McIntosh-Smith of the University of Bristol, UK. “They focus on things that GPUs do more specifically for training and inference for AI, but in reality they are in some ways more like GPUs than you might think.” But because TPUs are designed for certain AI applications, they can be much more efficient at those tasks and potentially save tens or even hundreds of millions of dollars, he says.
However, this specialization also has its drawbacks and can make TPUs inflexible if AI models change significantly from one generation to the next, Conti says. “If you don’t have the flexibility of your [TPU]you have to do [calculations] on the CPU of your node in the data center, which will slow you down enormously,” says Conti.
One of the traditional advantages of Nvidia GPUs is that there is simple software that can help AI designers run their code on Nvidia chips. This didn’t exist in the same way for TPUs when they first appeared, but the chips are now at a stage where they are simpler to use, Conti says. “With TPU, you can now do the same thing [as GPUs]”, he says. “Now that you’ve enabled this, it’s clear that availability becomes a major factor.”
Who builds TPUs?
Although Google first launched the TPU, many of the largest AI companies (known as hyperscalers), as well as smaller start-ups, have now begun developing their own specialized TPUs, including Amazon, which uses its own Trainium chips to train its AI models.
“Most hyperscalers have their own internal programs, and part of that is because GPUs became so expensive because demand outstripped supply, and it might be cheaper to design and build your own,” McIntosh-Smith says.
How will TPUs affect the AI industry?
Google has been developing its TPUs for over a decade, but it mainly uses these chips for its own AI models. What appears to be changing now is that other major companies, like Meta and Anthropic, are making significant purchases of computing power from Google’s TPUs. “What we haven’t heard about is big customers switching, and maybe that’s what’s starting to happen now,” McIntosh-Smith says. “They have matured enough and there are enough of them.”
In addition to giving larger companies more choice, it could make financial sense for them to diversify, he believes. “It might even mean you’ll get a better deal from Nvidia in the future,” he says.
Topics:
