Qualcomm spends a lot of time during its new product unveilings talking about its Hexagon NPUs. Keen observers may recall that this branding was repurposed from the company’s line of digital signal processors (DSPs), and there’s good reason for that.
“Our journey in AI processing probably started 15 or 20 years ago, with our first foothold being signal processing,” said Vinesh Sukumar, head of AI products at Qualcomm. DSPs have a similar architecture to NPUs, but they are much simpler and focus on processing audio (e.g., voice recognition) and modem signals.
The NPU is one of the multiple components of modern SoCs.
Credit: Qualcomm
The NPU is one of the multiple components of modern SoCs.
Credit: Qualcomm
As the set of technologies we call “artificial intelligence” developed, engineers began using DSPs for more types of parallel processing, such as long-short-term memory (LSTM). Sukumar explained that as the industry has become more passionate about convolutional neural networks (CNNs), the technology underlying applications such as computer vision, DSPs have focused on matrix functions, which are also key to generative AI processing.
While there is an architectural lineage here, it’s not entirely accurate to say that NPUs are just fancy DSPs. “If you’re talking about DSP in the general sense, yes, [an NPU] is a digital signal processor,” said Mark Odani, assistant vice president of MediaTek. “But everything has come a long way and it is much more optimized for parallelism, the operation of transformers and the storage of a large number of parameters for processing.
Although they are so important in new chips, NPUs are not strictly necessary for running AI workloads at the “edge,” a term that differentiates local AI processing from cloud-based systems. CPUs are slower than NPUs but can handle some light workloads without consuming as much power. Meanwhile, GPUs can often process more data than an NPU, but they use more power to do so. And there are times when you might want to, according to Qualcomm’s Sukumar. For example, running AI workloads while a game is running could favor the GPU.
