NPU

Infrastructure
About 1 min read

A dedicated AI processor that accelerates large-scale matrix operations—the core of deep learning—at the hardware level, helping improve on-device AI performance with low power consumption.

Also known as
Neural Processing UnitNeural Processing UnitAI AcceleratorNeural Engine

Detailed explanation

An NPU is special-purpose hardware designed to perform AI inference tasks with less energy than a CPU or GPU by optimizing the data flow of artificial neural network models. Equipped with hardware blocks specialized for repetitive matrix operations and convolution processing, it reduces data bottlenecks and increases processing speed. In particular, it serves as the core engine for implementing real-time features while conserving battery life in 'on-device AI' environments where data is processed locally without a cloud connection. Recently, it has become a standard metric for measuring local AI execution capability, as seen in Microsoft's Copilot+ PC requirements (40 TOPS or more), supporting various AI features like image recognition, real-time translation, and background blur at the hardware level.

Why it matters when choosing tools

NPUs allow high-performance AI tools to run quickly on personal PCs or mobile devices without an internet connection. In particular, when choosing a laptop, a higher NPU performance reduces battery consumption while smoothly handling tasks such as video conference filtering, running local Large Language Models (LLMs), and detecting security threats.

What to check

  • TOPS (Tera Operations Per Second): The number of operations per second; check if it meets the Copilot+ PC standard of 40 TOPS or higher.
  • Software ecosystem: Compatibility with major AI frameworks such as OpenVINO, ONNX, and Core ML.
  • Power efficiency: Lower power consumption compared to CPUs/GPUs for the same operation (performance per watt).

Key products

Examples include Apple's Neural Engine (M-series), Qualcomm's Hexagon NPU (Snapdragon X Elite), Intel's AI Boost (Core Ultra), and Google's TPU (Pixel series).

Related terms

GPUOn-device AILLM