A neural processing unit (NPU) is a dedicated processor for AI workloads, built to run the maths of neural networks in parallel using less energy than a CPU or GPU, and chip makers rate it in TOPS, trillions of operations per second. Every new laptop chip from AMD, Apple, Intel and Qualcomm now carries an NPU, and Microsoft uses a 40+ TOPS threshold to define a Copilot+ PC, so the number appears on spec sheets, shop shelves and procurement forms. This page explains what an NPU does, how TOPS figures are produced, why NPU TOPS and platform TOPS are different numbers, what each chip maker publishes, and what a buyer or IT team should check instead of comparing one headline figure. It is for buyers, IT and endpoint teams and anyone deciding whether a device is "AI-ready". Chip makers are listed alphabetically (AMD, Apple, Intel, Qualcomm) everywhere on the page and order is never a ranking.
Last checked: 2026-11-15.
This is an independent reference page from NHI Mgmt Group based on public provider documentation; it is not sponsored or endorsed by any provider. Chip specifications are vendor-reported marketing figures; confirm them on each vendor's current pages.
Key takeaways
- An NPU is a third processor. Microsoft describes it as a specialised chip for AI-intensive processes that works alongside the CPU and GPU and uses less energy on AI tasks, which helps battery life.
- TOPS is a peak, not a result. It counts how many simple operations the hardware could do per second at full load; real speed also depends on memory bandwidth, software support and the model.
- NPU TOPS and platform TOPS differ. Intel quotes up to 180 platform TOPS for Core Ultra series 3, which adds CPU, GPU and NPU together; Copilot+ PC eligibility is set on NPU TOPS alone, at more than 40.
- Vendor figures are not like for like. AMD quotes up to 60 NPU TOPS for Ryzen AI 400, Apple quoted 38 TOPS for the M4 Neural Engine and gave no TOPS figure for M5, and Qualcomm quotes 80 TOPS for the Snapdragon X2 Elite Hexagon NPU; precision and test conditions vary.
- Software decides whether the NPU is used. On Windows, apps reach the NPU through Windows ML and ONNX Runtime execution providers; an app that does not target the NPU runs on the CPU or GPU however many TOPS the chip has.
What is an NPU?
A neural processing unit is a block of silicon designed for the operations neural networks spend most of their time on: large numbers of multiply and add steps on matrices of numbers. Microsoft's Copilot+ PC developer guide defines an NPU as "a specialized computer chip for AI-intensive processes like real-time translations and image generation" and says NPUs process large amounts of data in parallel while using less energy on AI tasks than a CPU or GPU.
Vendors use their own names for the same idea. AMD calls its NPU architecture XDNA (the Ryzen AI 400 series uses second-generation XDNA 2). Apple calls its NPU the Neural Engine and, from M5, also adds a Neural Accelerator to each GPU core. Intel uses the plain term NPU. Qualcomm calls its NPU the Hexagon NPU.
Why it matters on a laptop or phone: sustained AI features such as live captions, background blur, on-device transcription and local language models would drain a battery quickly on the CPU or GPU. The NPU lets these run continuously at lower power. The privacy side of running models locally is covered in On-Device vs Cloud AI.
CPU, GPU and NPU compared
| Processor | Built for | Typical AI role | Trade-off |
|---|---|---|---|
| CPU | General-purpose code, few powerful cores | Small models, pre and post-processing, fallback when no accelerator is supported | Flexible but slow and power-hungry for large matrix maths |
| GPU | Thousands of parallel cores for graphics and compute | Heavy generative workloads, image generation, larger local models | High throughput, higher power draw; discrete GPUs bring their own memory |
| NPU | Fixed-function and programmable units for neural network operations | Always-on and background AI features, efficient inference of supported models | Most efficient per watt for supported models; depends on driver and framework support |
NVIDIA's AI Decoded explainer makes the same point from the GPU side: it calls TOPS "only the beginning of the story" and says language model performance is better measured in tokens per second, with batch size and memory capacity as further factors.
What does TOPS mean?
TOPS stands for trillions of operations per second, the term Microsoft uses for its Copilot+ PC benchmark. Qualcomm describes TOPS as a measure of potential peak AI inferencing performance, based on the architecture and frequency of the NPU (Qualcomm), so it is a theoretical ceiling rather than a measured result.
Three details change the number without changing the hardware:
- Precision: vendors typically quote TOPS at low-precision integer maths, often 8-bit (INT8). The same chip quoted at 4-bit precision would show a higher number, and at 16-bit floating point a lower one. Check the footnote for the precision used.
- Scope: "NPU TOPS" covers the NPU only. "Platform TOPS" or "total TOPS" adds the CPU and GPU, so the two cannot be compared with each other.
- Sustained versus peak: TOPS is a potential peak figure, so it does not describe how a device performs under a sustained real workload.
A useful way to read a spec sheet is to treat TOPS like engine horsepower, the analogy NVIDIA uses: a rough indicator of potential, not a measure of how fast a given journey will be.
What does each chip maker publish?
The table quotes each vendor's own announcement. Figures are vendor-reported and use each vendor's own test conditions, so they are listed for reference, not comparison. As of 2026-11-15.
| Vendor and chip family | NPU name | Figure published by the vendor | Source and date |
|---|---|---|---|
| AMD Ryzen AI 400 series | XDNA 2 (second generation) | Up to 60 NPU TOPS (Ryzen AI 9 HX 475 and HX PRO 475); other models 50 to 55 NPU TOPS; Ryzen AI Max+ 392 and 388 at 50 NPU TOPS | AMD CES 2026 press release, 5 Jan 2026 |
| Apple M4 | 16-core Neural Engine | Up to 38 trillion operations per second | Apple Newsroom, 7 May 2024 |
| Apple M5 | 16-core Neural Engine plus a Neural Accelerator in each GPU core | No TOPS figure; over 4x the peak GPU compute for AI compared with M4; 153GB/s unified memory bandwidth | Apple Newsroom, 15 Oct 2025 |
| Intel Core Ultra series 3 (Panther Lake) | NPU | Up to 180 platform TOPS (CPU, GPU and NPU combined); NPU-only figure not stated on the Intel pages checked | Intel Newsroom, 14 Oct 2025 |
| Intel Core series 3 | NPU | Up to 40 platform TOPS | Intel Newsroom, 16 Apr 2026 |
| Qualcomm Snapdragon X2 Elite and X2 Elite Extreme | Hexagon NPU | 80 TOPS; devices expected in the first half of 2026 | Qualcomm press release, 24 Sep 2025 |
Reading the table: the Intel Core series 3 figure of 40 is platform TOPS, while the Copilot+ threshold is NPU TOPS, so the two numbers are not the same test. Apple did not publish a TOPS figure for M5 and instead described GPU AI compute relative to earlier chips. AMD states that every Ryzen AI 400 processor exceeds the Copilot+ PC requirements.
What is the 40 TOPS Copilot+ PC requirement?
Microsoft defines Copilot+ PCs as Windows 11 devices with an NPU that can perform more than 40 trillion operations per second. Its developer guide says "many" new Windows AI features need an NPU at that level, not all of them. The guide names the first eligible chip families as Qualcomm Snapdragon X Elite, AMD Ryzen AI 300 series and Intel Core Ultra 200V series, and lists devices from Acer, ASUS, Dell, HP, Lenovo, Microsoft Surface and Samsung.
The requirement is why "40 TOPS" became a common reference point: it is a feature gate for Windows, not a performance score. A chip at 50 or 80 NPU TOPS meets the same gate as one just over 40. Full requirements and the exclusive features are on Copilot+ PCs Explained.
How do apps actually use the NPU?
An NPU only helps if software sends work to it. On Windows, Microsoft's guide recommends Windows ML, which detects available hardware, downloads the right execution provider and falls back to the GPU or CPU if the preferred provider fails. Windows ML uses ONNX Runtime underneath. For Qualcomm NPUs the QNN execution provider is selected automatically and delivered with Windows or through Windows Update. Microsoft says the recommended path for NPU and GPU acceleration has moved from DirectML to Windows ML.
| Layer (Windows, as of 2026-11-15) | What it does |
|---|---|
| Application | Calls a model through a framework; decides whether to target NPU, GPU or CPU |
| Windows ML | Recommended runtime; detects hardware, fetches execution providers, falls back if one fails |
| ONNX Runtime | Open-source inference engine used by Windows ML |
| Execution provider | Vendor-specific bridge to the hardware, for example QNNExecutionProvider for Qualcomm NPUs |
| Driver and NPU | Runs the operations the model is compiled into |
Checking the NPU is working: on devices with an NPU, Windows Task Manager shows NPU utilisation, driver version and other details alongside CPU, GPU, memory and disk. An NPU sitting at zero during an "AI" feature means the work is running elsewhere.
On Apple devices the Neural Engine is used by Apple's frameworks and Apple Intelligence; see Apple Intelligence Explained. On Android phones, Gemini Nano runs through the AICore system service; see Gemini Nano Explained.
What matters more than the headline number?
| Factor | Why it matters | What to check |
|---|---|---|
| Memory capacity | A local model must fit in memory alongside the operating system and apps | RAM size; unified memory on Apple and some other designs |
| Memory bandwidth | Language model speed is often limited by how fast weights can be read | Vendor bandwidth figure, for example Apple's 153GB/s for M5 |
| Software support | Unsupported models or operators fall back to CPU or GPU | Whether your apps list NPU support; execution provider availability |
| Feature gates | Operating system features can require a minimum NPU | Copilot+ eligibility; vendor device lists for phone AI features |
| Sustained performance | Thin devices throttle under long loads | Independent sustained tests rather than peak figures |
| Measured task speed | Users notice response time, not TOPS | Tokens per second or seconds per task on the models you will run |
How to choose: factual criteria
- Start from the features you need. If staff need Copilot+ PC features, the device must be on Microsoft's Copilot+ list; extra NPU TOPS above the gate do not unlock more of them.
- List the apps that will use the NPU. Ask each software vendor whether it targets the NPU, GPU or CPU on your chosen chip family.
- Size memory for local models. If teams will run local language models, memory capacity and bandwidth usually matter more than NPU TOPS.
- Compare like with like. Use NPU TOPS against NPU TOPS, at the same stated precision, and never platform TOPS against NPU TOPS.
- Pilot on real workloads. Measure the tasks you care about on a small set of devices before standardising.
Small organisations can usually rely on the Copilot+ list and vendor device lists; large estates benefit from a short pilot across two or more chip families, since software support differs by execution provider.
Common failure patterns
- Comparing platform TOPS with NPU TOPS: a 180 platform TOPS figure and an 80 NPU TOPS figure measure different things.
- Buying on TOPS alone: an NPU that no installed app targets adds nothing to day-to-day performance.
- Ignoring memory: local models that do not fit in RAM are slow or fail regardless of NPU rating.
- Assuming Apple publishes the same metric: Apple gave a TOPS figure for M4 but described M5 by GPU AI compute and bandwidth instead.
- Treating "AI PC" as a standard: only Copilot+ PC has a published NPU threshold; other "AI PC" labels are vendor marketing terms.
Where identity and security fit
An NPU makes local models practical, which moves AI processing, prompts and outputs onto endpoints that IT may not monitor. Local model runtimes, model files and any API keys used for cloud fallback need the same inventory and control as other software and credentials. The security side is covered in the On-Device AI Security Guide, and policy choices for unsanctioned tools in Block vs Sanction vs Coach.
Practitioner checklist
- Record NPU TOPS, platform TOPS and the stated precision separately for every device model you evaluate.
- Confirm Copilot+ PC eligibility from Microsoft's list rather than from a TOPS figure.
- Ask software vendors which execution providers or frameworks their AI features use on each chip family.
- Specify memory capacity and bandwidth for any role that will run local models.
- Check NPU utilisation in Task Manager during a pilot to confirm features use the NPU.
- Keep NPU drivers and execution providers on the vendor update channel.
- Inventory local model runtimes and model files on endpoints.
- Recheck vendor chip pages each quarter, since new generations change the figures.
Standards and references
- Qualcomm: A guide to AI TOPS and NPU performance metrics
- Microsoft Learn: Copilot+ PCs developer guide (NPU devices)
- AMD: Ryzen AI 400 series at CES 2026
- Apple Newsroom: Apple introduces M4 chip
- Apple Newsroom: Apple unleashes M5
- Intel Newsroom: Introducing Panther Lake by the numbers
- Intel Newsroom: Intel launches Intel Core series 3 processors
- Qualcomm: Snapdragon X2 Elite Extreme and Snapdragon X2 Elite announcement
- NVIDIA Blog: AI Decoded, what TOPS means
Related NHI Mgmt Group resources: On-Device AI and AI PCs Hub · Copilot+ PCs Explained · AI PC Chips Compared · On-Device AI Security Guide