▲ 220 points
back
317 comments
Apple has totally failed to deliver interesting AI experiences so far ... and I still think they're going to be the dominant provider of AI in 5 years. We're just one or two advances in chips / models / both away from being able to run very good local models for free on mid-tier Apple devices. The privacy, cost, and latency story there will be too much for OpenAI/Anthropic/Google to beat.
Just writing this down so I can be praised/mocked in 5 years.
I'm perfectly happy with Apple not becoming an "everything we do is AI-centric" business.
I'm fatigued by it all at this point. It's streamlining the interesting and fun parts out of my job (by practical necessity of use there), and if I used it half as much outside of work I'm sure it'd do the same there too.
Here’s the two main reasons why local inference won’t compete any time soon with the cloud:
1. Most useful LLM work is done in parallel. A Mac Mini can run one LLM inference thread at a time. The cloud can spool up dozens and spread that inference across efficiently batched operations over a fleet of hardware.
2. Faster inference hardware such as the chips from Cerebras and Groq cannot be run locally. But the advantages of running >5x the token throughput per thread can’t be overstated. Add in the multi-threading advantage and it’s a knock-out punch for local LLMs.
Local inference has a role: if you’re working with extremely private matters or you want an uncapped model that will talk dirty or generate NSFW photos, local is the only option. I think Apple and others will continue to also run a lot of useful workloads locally such as text editing suggestions, speech to text, text to speech, and image manipulation. As local hardware improves, these capabilities will get better too.
But, for most LLM work, the cloud will continue to dominate for a long time to come, if not forever.
I don't mind the subtle ML integrations that they have put in the photos app: plant ID, recognizing faces, removing background, OCR text search (even for handwriting!), etc.
I suspect we'll see a hybrid before an all or nothing. Local models for computer control or delegating, online models for things that need strong reasoning, planning, and knowledge access. Again, I'd be more than happy to be wrong. I just see models growing faster than the hardware can.
I'm also of this opinion, but also that it doesn't have to be Apple (but they are well positioned). What I've seen with running local models on my 48GB M4 MBP is really impressive - it's not the same level as hosted stuff, but it's better than what I was using a year or two ago.
I wrote: "we should all be buying a fully loaded Mac Studio (128GB of ram, 20 CPU cores, a lot of GPU and Neural cores.)" April, 2023
We are both late and early.
I'm not sure that's what this article is about.
Apple is doing something very different. Their AI experience for end users definitely has been a little behind.
Apple Silicon, however, has been quite unique for the last 4-6 years and it's increasing overlap with LLMS.
The model/chip optimizations are definitely improvements, the thing that is really standing out the past 2 years is how much the open source model community has been making possible, especially when you know a group of use cases.
There is common wisdom: during gold rush, sell shovels.
Apple's shovel (ahem, Mac mini) is the highest quality.with Companies burning money left, right and center, Apple can dispense with advertising altogether
Apple has failed to live up to the Steve Jobs era and initial iPhone hype. It's not just "AI experiences", it's computing in general. Maybe the consumer sector is just dead/dying. Maybe consumers are just running out of cash because filthy VCs are destroying communities and forcing the 99% into poverty.
The one thing that is marginally exciting: the Apple SoC or M series chips.
It's unfortunate they are locked behind crappy macOS and other proprietary apple crap.
Its a shame RAM prices are getting in the way of Apple. The 512 GB RAM Mac Studios would have been worthwhile.
"Apple has totally failed to deliver interesting AI experiences so far ..."
You think this is a mistake...
Did you mean 5 months? :)
Apple could probably sell a machine starting at $10,000 if they architected it as the sole place one’s Private Cloud Compute [1] ran.
It would need a path to a $2,500 machine, I think. But this is a niche I don’t think another consumer-facing brand could do like Apple.
The out-of-stock $6000 M3 Ultra Mac Studios with 800GB+ memory bandwidth are going for $24,000 on eBay, so yes definitely
I asked an Apple (via a sales rep who visited our company to showcase in internal iPad healthcare app) to please do this for iCloud when iCloud Drive was in-development. We would have easily paid $50,000 for a rack-able Mac Pro you could point "managed" devices at.
Apple simply cannot comprehend the ask.
Apple is quite hostile to professional users, so no.
Apples on stage use cases for their hardware and software makes me wonder if they actually use computers over there, or what a "job" at apple entails.
I am unsure that apple themselves understand why their hardware (top end & bottom end) has been so successful, without this understanding leaning into these use cases isn't really going to be possible.
Ages ago, back when the Macs would come out, my co-workers and I would take a bit of time to configure the most expensive possible configuration --- time was, it was pretty easy to hit six figures, but over time, that has gradually come down.
How many of these 'consumer' customers would they get for $10,000 and how would they reach their Apple-typical gigantic margin with that?
Our company would buy at least 10 of those instantly.
But that would be more for enterprise usage. I cannot imagine a personal computing usage which can justify a 10k machine.
Bring back xServe!
Who would be the market for such a thing?
100%
Very few brands have such a distortion field.
If I had the capital I’d make an household inference appliance.
No peripherals except Ethernet, integrated compute (cpu+gpu+mem) and secondary storage (+mobo, psu). No accoutrements, just the minimum amount of hardware to run a model as a utility.
Even the appliance faceplate would be a display showing stats like an old HiFi stereo.
Edit: something like a series of modules consisting of a RISC-V CPU + Vortex GPGPU + memory
You're describing the mac mini/studio with some facelift.
What I’m not sure to understand is that if you want to just run Claude code or openclaw type software with llm apis or subscriptions (and not run local models) to benefit from a local file system and always-on capability for ‘second brain’ type of workflows, I guess you don’t need a Mac mini but can run it on a raspberry pi or an old laptop ? Does anyone have experience with that ?
Running models on-device on a Mac is immensely annoying though. Figuring out what will work out of BF16, FP8, BF16+FP8, NVFP4, INT8, GGUF ... the list goes on ... is 'non-obvious' at best. Apple do little to support with tooling. There's MLX, but unless you're happy to transform a model to that format yourself you'll be lagging a long way behind.
Apps like LMStudio, Ollama, Draw Things, etc do a great job of simplifying it but it's still a pain.
It’s not for the AI inference, it’s for the tool calls and desktop GUI app workloads and browser. There aren’t any on-device models capable enough of real work that can run on lower end Mac Minis. But for running a few browsers and GUI apps, you’re much better off buying a Mac Mini than paying for a more expensive and worse-performing container in the cloud. Browsers were not designed to run in Linux containers but they run optimally on baremetal desktop OSes. An M4 Mac Mini beats the single core performance of any VM you might rent in the cloud, in terms of raw compute per dollar (Geekbench scores).
> "The speed of AI development right now is just crazy," Brooks said. "I can't imagine where we're going to be a year from now, three months from now, or even a month from now," he added.
I don't think I'm taking this out of context when I say this is unintentionally correct. Apple still doesn't know what to do about AI.
Luckily, it doesn't matter because it's a solution in search of a problem. Most consumers aren't using AI apart from google search.
Everyone else is using it as a content scraper and praying nobody will step in to end the piracy/fraud.
Apple could dominate this niche if they decided, for a while until prices fall, to eat some margin and bump up RAM in high end models. Couple that with a new M series chip with even faster AI performance.
It’s not a huge niche but it’s an influential one. They’d get the engineers and CXOs of AI ventures and a lot of academics and hobbyists.
For the platform it would keep them cemented as the high end vendor. In the long term it would position them to take advantage of any software or training breakthroughs that deliver frontier model performance at that scale.
> Apple's Mac mini and Mac Studio have become the machines of choice for running AI agents, according to Doug Brooks, Apple's senior product manager of Apple silicon.
This is mostly an US phenomenon, no Mac mini nor Mac Studio around here.
Only Thinkpads and Macbooks laptops talking to hyperscalers.
People bought diesel cars as miles per gallon was higher than petrol.
People are buying apple unified as electricity costs in many countries are very high, so cheaper to run than Nvidia setup.
As non-apple unified memory options increase, many people will have more choose those
This is a paraphrase of https://www.thedeepview.com/articles/how-apple-s-decade-long...
Being a Mac switcher since 2003 I am as much of a fanboy as anybody else but this quote from the article caught my attention, and smells like PR.
> Many AI tools are also Mac-first or Mac-only
I fail to recall AI tools Mac-only general purpose AI or agentic tools. Most of the claws, harnesses, studios and inference engines seem to be multiplatform. You can say you can run then in a Mac with a nicer UI wrapper or whatever, but "Mac-first" or "Mac-only"?
On-device isn't only cheaper/faster - for health data it's the whole point! The self-sovereign community desperately needs an affordable AI appliance and Apple is still the most likely candidate to deliver on the privacy promise.
When you don't limit on-device AI to "can be used to run coding model", Mac minis are great. My M4/16gb been working on a long term research project using Qwen 2.5 8b for months now. The performance is good enough for processing a lot of small text prompts.
The submission link should have been the original editorial piece at
https://www.thedeepview.com/articles/how-apple-s-decade-long...
I don’t understand why they are not reintroducing the Apple Xserve?
Apple doesn’t have an ads business, and it is inevitable that Google/openai/Anthropic are going to seek to monetise consumer ai via ads.
So the ad free Apple on device experience will be welcome.
Oh please the neural engine is mostly useless for LLMs. Siri in iOS 27 is laughably pathetic and slow compared to GPT Live DESPITE sending personal context to their (attested) cloud to execute anything but the most basic queries. Still years behind.
Except that Mac ultra M3 they talk about is now only being sold in the 96 GB configuration. It’s no longer being sold in larger Ram configurations by Apple Apple because of the global RAM shortage. And you can not add memory after purchase because it’s integrated/soldered on.
Now just imagine they'd kept making the Mac Pro and enabled compute offload to GPUs. Or even just passthrough to Linux VMs. Would've been quite the AI machine.
The issue with the Apple is that they didn’t really develop any competitive local AI machine. Their strategy/marketing falls flat when you ask them how exactly they implement AI: they buy it from Google cloud. In the future local AI may become a thing but that’s 4-5 years away. I count the 2q-4q and atrocious performance as “local ai” only for the enthusiasts crowd not for people doing competitive work.
> "people often want a system that's under their control, isolated from their primary machine, and capable of running 24 hours a day, seven days a week," said Brooks. "A Mac mini is an amazing system for that," he added.
These execs are so out of touch they believe Apple hardware to be "a system that's under their control", how does it come to this? Besides, a VM without bi-directional sharing of data gives you pretty much the exact same thing.
Did hundreds/thousands of developers really go out there and bought Mac Minis just because one prominent technology semi-celebrity happens to have used a Mac Mini for the development of their thing? Seems bananas people would spend hundreds on monies on something they barely grasp how it works.
Ok, I didn’t want to take the bait but this one’s just too much.
> “He also described a shift toward running AI locally rather than in the cloud – a move motivated by privacy, security, and the rising cost of inference as agents consume more tokens.”
Classic Apple. No more just beating the “security and privacy” drum, now its “tokens are expensive!”
<neanderthal voice/> Cloud scary. Cloud expensive. Mac good. Buy Mac!
> “He also singled out what he calls ‘transparent AI’ on iPhone and iPad, referring to features scattered throughout the operating system and third-party apps that work quietly without announcing themselves as AI.”
<neanderthal voice/> Apple use AI, Apple just not say it. Apple smart, not lagging behind industry! Buy iPhone!
How about you invest in developing your own models, correctly? And provide a secure and private inference cloud service on your fancy Apple silicon? And integrate that into your platform so Siri gets smarter without you farming queries out to Google Gemini? Bill me for it in iCloud+ I’ll probably pay for those tokens.
Was that so hard?