On-device AI processing is a privacy play, a classic Apple strategy. But it’s not the first to make this move. Microsoft was. Its Copilot+ standard is a similar concept, but based solely around laptops. Copilot+ laptops have dedicated AI hardware designed to allow for on-device AI processing. The two even rely heavily on the same core artificial intelligence, that of ChatGPT creator OpenAI.
Which leads us to this completely justified question: Why the hell can’t my $900 iPhone 15 do this stuff too?
Neural Thinking
NPU is the term: the tech under the spotlight that separates older “non-AI” hardware from the new stuff our purses are meant to quiver in anticipation for. It stands for “neural processing unit.”
The latest phones and tablets have one of these in addition to the CPU (central processing unit) and the GPU (graphics processing unit). Its specialization is in carrying out an awful lot of operations simultaneously—and in the context of phones and laptops, doing so while using minimal power is key.
Just as generative AI has pebble-dashed the internet and wider society with what might charitably be called “stuff,” the creation of that stuff also requires a wide and shallow type of work.
For those who want to get slightly more technical, NPUs excel at matrix manipulation. This is the core form of work that powers chatbot LLMs and other generative AI. In each case, an AI starts off with a prompt and gradually homes in on a final result—be it a picture or a sentence of prose—through matrix manipulation.
Beyond the NPU
Case closed? Are we simply in an NPU world now? Sure, but we have been since at least 2017. Apple introduced the Apple ANE (Apple Neural Engine) in 2017, seen in the iPhone 8, iPhone 8 Plus, and iPhone X. It’s an NPU.
It was needed back then, years before AI was as much of a buzzword as it is today, because Apple has actually used forms of AI and machine learning in iOS for ages.
Perhaps Apple Intelligence isn’t all that different—there’s just more of it. And the software scene has already proved some of these new “AI” features are being artificially gated, paywalled, behind new hardware.
A hobbyist software historian who goes by the name of Albacore online has got some of Microsoft’s Copilot+ features working on hardware that shouldn’t, in theory, be able to run it—a cheap Samsung Galaxy Book2 Go notebook. It costs a fifth to a quarter of the price of a Copilot+ laptop.
“Getting Recall working was moderately challenging, I suppose. I’ve been reverse engineering Windows bits for close to 10 years, so tracking down checks and restrictions is practically second nature to me,” Albacore tells WIRED.
How does it work? “Track down what guards the features, devise a plan for altering the check so it’s always happy no matter the real requirements, and then wrap it in a nice installer. Some testing and edge case hunting later, the app was ready,” Albacore says.
