Even though a GPU in a data center should only require 700 watts to run a large language model, it may actually need 1,700 watts due to the inefficiency of how the electricity gets to it. It’s a problem that Peng Zou and his team at start-up PowerLattice say they have solved by miniaturizing and repackaging high-voltage regulators.
The company says its new chips can reduce power consumption by up to 50% and double performance per watt by reducing the voltage conversion process and bringing it significantly closer to processors.
Shrinkage and displacement of the power delivered
Traditional systems power AI chips by converting alternating current from the grid into direct current, which is then transformed again into low-voltage direct current (about one volt), usable by the GPU. With this voltage drop, the current must increase to save energy.
This exchange occurs close to the processor, but the current still travels a significant distance in its low-voltage state. High current traveling any distance is bad news, because the system loses power as heat proportional to the square of the current. “The closer you get to the processor, the less distance the high current has to travel, and so we can reduce the power loss,” says Hanh-Phuc Le, a power electronics researcher at the University of California, San Diego and no relation to PowerLattice.
Given the ever-increasing power consumption of AI data centers, “it’s almost become a major problem today,” says PowerLattice’s Zou.
Zou believes he and his colleagues have found a way to avoid this huge loss of power. Instead of dropping the voltage a few centimeters from the processor, they figured out how to do it a few millimeters, within the processor case. PowerLattice designed tiny power delivery chipsets: shrinkable inductors, voltage control circuits, and software-programmable logic in an integrated circuit about twice the size of a pencil eraser. The chipsets are located under the substrate of the processor case, to which they are connected.
One of the challenges the minds at PowerLattice faced was how to reduce the size of the inductors without altering their capabilities. Inductors temporarily store energy and then release it smoothly, helping regulators maintain stable outputs. Their physical size directly influences the amount of energy they can handle, so reducing them weakens their effect.
The startup solved this problem by building its inductors from a specialized magnetic alloy that “allows us to operate the inductor very efficiently at high frequencies,” Zou says. “We can operate at a frequency a hundred times higher than the traditional solution. » At higher operating frequencies, circuits can be designed to use an inductor with much lower inductance, meaning the component itself can be made with less physical material. The alloy is unique because it maintains better magnetic properties than comparable materials at these high frequencies.
The resulting chipsets are less than 1/20th the area of current voltage regulators, Zou says. And each is only 100 micrometers thick, or about the thickness of a strand of hair. Their reduced size allows chipsets to fit as close as possible to the processor, and the space savings provide valuable space for other components.
PowerLattice’s chipsets would sit under a GPU’s case to provide power from below.Power Network
Even with their small size, the proprietary technology is “highly configurable and scalable,” says Zou. Customers can use multiple chiplets for a more comprehensive patch or less if their architecture does not require it. “It’s a key differentiator” of the PowerLattice solution to the voltage regulation problem, according to Zou.
Using the chipsets can reduce an operator’s power requirements by 50%, doubling performance, the company claims. But this figure seems ambitious to Le. He says power savings of 50% “could be achievable, but that means PowerLattice must have direct control over the load, which also includes the processor.” The only way he sees this as realistic is if the company has the ability to manage power in real time based on a processor’s workload (a technique called dynamic voltage and frequency scaling), which PowerLattice doesn’t offer.
Facing the competition
Currently, PowerLattice is in the process of reliability and validation testing before releasing its first product to customers, approximately two years from now. But bringing the chipsets to market will not be easy because PowerLattice has renowned competitors. Intel, for example, is developing a fully integrated voltage regulator, a device partially dedicated to solving the same problem.
Zou does not consider competition from Intel because, apart from the fact that the products differ in their approaches to the power supply problem, he does not believe that Intel will provide its technology to its competitors. “In terms of our market position, we are quite different,” says Zou.
A decade ago, PowerLattice didn’t have a chance to succeed, Le says, because companies that sold processors only ensured the reliability of their chips if customers also bought their power supplies. “Qualcomm, for example, can sell its processor chip and the vast majority of its customers also have to buy their proprietary Qualcomm power management chip, because otherwise they would say, ‘We don’t guarantee reliable operation of the entire system.’ »
But now there may be hope. “There’s a trend toward what we call chipset implementation, so it’s heterogeneous integration,” says Le. Customers mix and match components from different companies to achieve better system optimization, he explains.
And while notable vendors like Intel and Qualcomm may continue to have the upper hand with notable customers, smaller companies — mostly startups — that build processors and AI infrastructure will also be power hungry. These groups will need to look for a source of electrical power, and that’s where PowerLattice and similar companies could come in, Le says. “That’s the way the market is. We have a startup working with a startup doing something that competes, and even competes, with some big companies.”
From the articles on your site
Related articles on the web
