If there's one buzzword that just about anyone can point to as taking over in 2024, it's AI. Just about every major tech firm is employing AI in its new products in some way or other, whether it actually makes sense or not.
Beyond that, businesses are scrambling to get AI working for them, and even individuals are looking for ways to improve productivity and get an edge on the competition by employing AI.
Of course, PC makers have a whole lot to gain, as their products are a primary gateway to the emerging world of AI. Everyone is looking to cash in on the new wave, and you're no doubt seeing the term "AI PC" thrown around quite frequently.
What is an AI PC? What is an NPU? Is it necessary to buy an AI PC if you're currently shopping for one of the best laptops? Let's answer some of these questions right here to give you a better understanding of what goes into an AI PC.
Recent updates
April 9, 2024: I've added a section on TOPS and what it means when it comes to measuring NPU performance. — Cale Hunt
What is an AI PC?
Accurately answering this question isn't exactly easy. At least not yet. It really depends on who you ask, with companies attempting to define the term in their own light.
Intel is one of the leading companies pushing AI tech in PCs, with its latest Core Ultra processors (CPU) packing a Neural Processing Unit (NPU) designed and optimized specifically for AI tasks.
AMD and Qualcomm are also working hard on AI-enhanced CPUs. AMD's Ryzen 8040 series chips, announced in late 2023, feature an XDNA NPU that's all about AI. Qualcomm's Snapdragon X Elite CPU should give AMD and Intel some strong competition, though PCs with the chip aren't expected until later this year.
Generally speaking, an AI PC is a computer that has hardware inside — an NPU — specifically designed to boost the performance of AI and machine learning tasks without having to rely on cloud computing.
While the applications aren't yet widespread, we're no doubt going to see AI enhancements creep into a lot more areas. It doesn't even have to be that complicated. Something like Windows Studio Effects can already lean on an NPU rather than your PC's CPU and GPU to improve your camera feed, freeing up the latter components to keep your system running smoothly.
AI PCs according to Microsoft and Intel
Microsoft has laid out its core requirements of what makes an AI PC, which was shared at an AI PC Acceleration Program event hosted by Intel at the end of March 2024.
Here's what a computer must have in order to be considered an AI PC, at least according to Microsoft:
- A new CPU, GPU, and NPU.
- Onboard Copilot software.
- A Copilot keyboard button.
Microsoft's key features differ a bit from what Intel claims is an AI PC, with the chipmaker notably dropping the Copilot and Copilot key requirements and instead focusing on having a CPU, GPU, and NPU with AI acceleration capabilities.
What is an NPU?
CPUs and GPUs have been around for a long time, and "NPU" is now often mentioned in the same breath.
As I briefly mentioned above, an NPU is a new PC component that is optimized specifically to better handle AI tasks. Not only is the NPU optimized for specific workloads, it can also take some of the stress off of your CPU and GPU, allowing your system to run smoother while AI tasks are completed.
An NPU isn't necessarily better than a CPU or a GPU; it's best to consider it as a new piece of hardware designed to work in tandem with the other pieces to improve overall system performance as AI-powered features become more prevalent in day-to-day computing.
Our own Rebecca Spear wrote an informative guide on how NPUs differ from GPUs, which does a great job of explaining in more detail how these components work together and what they do differently.
What does TOPS mean?
Measuring a PC’s performance is usually split into synthetic benchmark results, real-world results, and a general vibe on how it performs. For example, when I review a laptop I’ll put it through a bunch of standardized benchmark tests, but I also like to give a general idea of how it performs in day-to-day operations. Does it lag behind in certain tasks? Does it have enough RAM to keep up with multitasking? You get the idea.
With AI PCs and their new NPUs, judging performance becomes more abstract. One term that keeps popping up when talking about AI performance is “TOPS.” It stands for “Tera Operations Per Second,” with Tera meaning trillions.
That’s kind of an absurd calculation to wrap your head around, especially when you consider that something like NVIDIA’s RTX 4090 graphics card is capable altogether of about 1,321 TOPS. Those numbers are enormous.
Bringing things back to AI PCs and their NPUs, Qualcomm’s Snapdragon X Elite chip includes an NPU running at 45 TOPS. That’s quite impressive considering the first run of Intel Core Ultra chips offer up to 34 TOPS with the CPU, GPU, and NPU pooled together, with about 10 TOPS for the NPU alone. AMD’s Ryzen 9 8945HS chip touts a total package limit of 39 TOPS, with 16 TOPS specifically from the NPU.
Does that mean that the Snapdragon X Elite is going to be about four times better at AI tasks? In a general sense, it’s safe to assume that being able to perform four times as many tasks per second is going to offer a noticeable bump in performance. However, that’s not really accounting for any nuance below the surface. Some NPUs might be better geared toward certain AI tasks, completing them faster even if TOPS are not quite equal.
As reported by Tom’s Hardware, one of the requirements for running Copilot locally on an AI PC will be an NPU capable of 40 TOPS. Even if TOPS aren’t the end-all way to measure NPU performance, counting TOPS can give buyers an idea of what they need to look out for if they want their AI PC to be able to handle certain tasks. And it's also safe to bet that these numbers will be prominently advertised by manufacturers to separate their hardware from the competition.
Do you need an AI PC?
Whether or not you really need an AI PC really comes down to how you work and how you plan to work in the future (at least through the lifespan of your PC). We've been getting by just fine without AI PCs for decades, and a majority of users are still going to get by just fine without an NPU or Copilot key on their laptops.
An NPU can already boost some tasks that rely on AI, but those tasks are still fairly limited. Unless you want to be at the forefront of AI, rushing out and buying an AI PC when your current laptop is still fresh really isn't something I'd recommend.
Of course, if you already plan on upgrading to a new laptop it won't really hurt to pick up a device classified as an AI PC. Just because an NPU's usefulness isn't particularly impressive in day-to-day computing right now doesn't mean it won't be necessary in even the near future.
AI advancements are moving at an incredible pace, and the usefulness of an NPU could become more apparent sooner than you think.
What does AI mean for the future of Windows?
As revealed in an exclusive late 2023 report by Senior Editor Zac Bowden, Microsoft's next major Windows release is going big on AI. The release, named "Hudson Valley," is still under construction and is expected in Fall 2024.
As stated by Bowden, "the big focus for Hudson Valley is on next-generation AI experiences that are being woven and integrated throughout the OS, much of which will likely require new NPU hardware to function."
The biggest feature is expected to be a new AI-powered Windows Shell that will run in the background "to enhance search, jumpstart projects or workflows, understand context, and much more." Think of it sort of as a Copilot that's always on and ready to help out in many different ways. If you want to take advantage of these improvements, you will most likely need an AI PC.
Furthermore, Intel revealed at its AI summit hosted late-March 2024 that Copilot will run locally on AI PCs in the near future. Why is that important? There are a number of reasons, ranging from cost reduction on Microsoft's side, to better security, always-enabled access, and improved personalization on the user's side.
It's very clear that the usefulness of an AI PC is only going to grow, and Microsoft looks to be a main player in making AI PCs make a lot of sense for users.
