▲ 491 points
back
283 comments
My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.
To throw out some real and up-to-date numbers from [1] for FHE at "128-bit security level", to sort 8x 8-bit unsigned integers on the most ordinary of desktop PCs, wait 3 seconds for the result. Want to sort 32x 8-bit unsigned integers instead? Come back 34 seconds later for the result.
update: also see [2] for some primitive unsigned 64-bit integer operation benchmarks with the TFHE-rs library (winner in the sorting performance comparison of [1]). Equality at 80ms, addition and subtraction at 100ms, division at 8 seconds, etc.
[1] https://eprint.iacr.org/2026/1495.pdf Oblivious Sorting under Fully Homomorphic Encryption: A Comprehensive Survey and Performance Analysis, Omar Ahmed and Rostin Shokri and Nektarios Georgios Tsoutsos, 2026
[2] https://docs.zama.org/tfhe-rs/tfhe-rs/1.0/get-started/benchm...
The article conspicuously fails to go into much detail about that. I poked around with an AI a bit (to rapidly cover all the linked pages) and it seems the best numbers we can get are from this arxiv paper: https://arxiv.org/html/2506.18150v4 Which says:
"We evaluate HE-LRM on UCI (health prediction) and Criteo (click prediction), achieving inference latencies of 24 seconds on UCI and 228 to 489 seconds, respectively, on a single-threaded CPU."
There don't seem to be any direct comparisons available, probably because nobody else has any reason to limit themselves to one single-threaded CPU with normal techniques, but for reference the AI seems to expect that normal times for conventional setups are in the milliseconds range, fairly comfortably, even on CPU. I didn't find a clean primary source to link to for this claim, but clicking through various things that don't cleanly state the situation it did seem plausible. So we seem to still be in the range of single-digit orders of magnitude slower, possibly as much as 5 or 6, which is to say, we're still talking the range where we need to take the log of the difference to get sensible numbers, we're not using percentages.
(To run it yourself, I basically just fed the URL from the HN link, mentioned that FHE is known to be slow, and asked if anything linked in the blog post gave concrete times.)
1000x slowdown is bad but not a complete deal breaker. Do you have a sense of what a reasonable achievable factor is? Do you have sense for how long before we get to that achievable factor?
Commercially viable for Google boils down to can they attribute ads behaviors to it or not.
Then there’s a second tier of things that just make those wheels turn and if they do or don’t make ads revenue is nominally immaterial.
The teams doing this stuff at Google are purely for show, none of this makes it into any real products.
There’s the narrow exception of stuff like gboard, that does use privacy preserving ML/fed learning, but this stuff isn’t in the same zone.
I find it a bit embarrassing when Google publishes this stuff to be honest.
I saw a paper about this in early 2020 (pre-COVID shutdowns) at the ScaledML conference. I looked into it and had the same conclusions. At some point, running your own models in the clear is just more practical.
Only 1000x overhead would make some image classification tasks go from 1ms to 1s. That’s viable for some applications!
One of the biggest problems IMHO is that they aren't trying to usefully accelerate it on anything other than specialty hardware or 64+ core EYPCs so nobody gets to play with it at home.
ex: A 7900XTX barely gets 0.5 TOPS of u/i64 naively w/ hip-direct, 5-10s just to bootstrap!
I needed more throughput for non-crypto i64 diff eqs so I slopped up a lib that uses RNS & CRT w/ Int8 GEMM... it's good for ~3.9 TOPS (~90% theoretical peak of the RDNA3) at prod relevant FHE sizes (2048/4096). This lowers bootstrap time to 200-500ms. It was basically free real estate lol
It isn't done yet (not worth the heat in the summer), going to finish it in the fall. Have been accumulating cloud credits to do CDNA3/4 validation in the meantime (If anyone has some to offer do let me know!)
It's neat but very dry, uses semantic contracts so you tell it what kind of mult you need and it chooses the validated best backend. If you're doing lots of smaller ops (512, 1024) it will use custom WMMA/MFMA kernels, dual issue, and grouped dispatch to land >70x over hip-direct.
That's the reason for HEIR like optimization and parameter selection.
It narrows the 10^3 - 10^6 penalty to 10x - 100x.
The primary path to speed ups appear to be in custom ASICs by startups like Niobium. Combined with the recent Taalas acquisition by AMD, I think I see where this is going.
But yeah, for hot path traffic it's probably going to be swamped by the input data rate. But I expected identity tables and cached lookup data will need to be a core component so duplicate checks is avoided in every way available.
Great, private AI, at the cost of >1000x the resource usage. Because apparently AI companies weren't already using quite enough energy to cook the planet.
The most private AI is the one running on my own hardware, not in some giant data center.
So much inefficiency just to run it on someone else's untrusted hardware. Private AI is already possible today with local open-weight models running on hardware you control.
Homomorphic encryption is cool technology, but I'm really not sure what problem it solves.
I think you folks are reading too much into it. I think the people working on FHE need to publish an AI-oriented pitch to retain funding from AI-pilled execs. Must be the same case with the golang post few days ago.
This is the same Google that doesn't have e2ee on their password manager by default. Like WTF, it's a password manager.
Google is making private AI practical with Gemma4 something that you can run without an Internet connection.
All of the proofs of privacy rely on us getting the math right. All of the privacy from unplugging your internet cable is there by default.
"hotword" detection, is this basically sniffing encrypted data for words the authorities don't want you to speak?
Seems counterproductive to their stated aim of privacy.
https://fhetextbook.github.io/ which i believe was linked on this website previous is what i used to learn about how this stuff works.
I see a lot of comments here about how private AI is much cheaper than doing this, and that is currently true.
But I am curious if anyone can think of any use cases for this as it currently stands? For every idea I can think of, I can think of another solution that would probably be a better solution at current slowdown in cost from using homomorphic encryption.
For their two examples...
1) For the "will they click the add" one, how would anyone verify that the model I am uploading of myself is remotely accurate? And can't they learn my characteristics by running them against very specific ads that pinpoint my preferences?
2) For the "Do you have heart disease" one, why not just generating a billion datapoints (or more). The user can download the compressed data of this huge sample of points and then just enter their data on their own side and find the point that is closest (if not an exact match) to their own data?
I'm not pulling apart the idea of homorphic encryption, I'm just curious if anyone can think of a really compelling use case for it given it 10^3 to 10^6 greater compute cost?
Has FHE really progressed so far that it's now so efficient that doing computation on an encrypted prompt is feasible? I thought even basic operations like FHE addition were still thousands of times more complex. The only mention in the article I see is:
> But while homomorphic encryption has a nontrivial cost overhead, it shifts the capability/privacy trade-off to a question of cost. And the cost of homomorphic encryption is rapidly decreasing.
Which doesn't spell out exactly hon "nontrivial" the cost overhead still is.
I've published two papers on using HE for ML and it's nice, but also alarming, to see big players like Google and Microsoft making decent tools for performing HE. Although the technology is still much farther out from being commercially viable, it does pose an interesting problem about how these data aggregation companies will utilise a tech that is inherently private.
It seems to me this tech still presumes the data sits in data warehouses, which i don't like to start with. The tech succeeds in packing the data in identical black boxes, so they all seem equal to the map-reduce function that runs over them. A separate identification layer knows which of the boxes is yours. But who knows, maybe from the results you can be fingerprinted anyway. As in the game, how many questions before you can guess the thing I'm thinking of? Answer: not that many.
A homomorphic encryption blogpost without any mention of the computational overhead...
I did some amount of research into the feasibility of PHE and FHE about 20 years ago, and my conclusion at the time was that the space overhead of the encrypted output was a massive bottleneck, which meant that while it was potentially useful in a small number of niche cases it wasn't ever going to be practical for general-purpose computations without a major breakthrough.
The gist was I could do an encrypted (int)x + (int)y = (int)z computation, I could encrypt the inputs and then get a result back that was correct, secure, and decryptable, but was like 1MB in size.
So, for someone whose knowledge is 20 years outdated and is about Pallier crypto, has that major breakthrough happened?
Maybe I'm not understanding this, but how is it that you can know enough about the data to process it without undermining the fundamental concept of encryption? Isn't encrypted data supposed to be just random noise without the key? The more you know about the underlying data the easier it gets to decrypt? Does this mean someone can just steal your encrypted data and use that to steal your identity without even needing to decrypt it anymore?
I think this could be an effective privacy preserving approach:
Run the encoding and first and last layer of the LLM on the users machine. They could still get some idea of the content but really couldn’t easily get the actual text back.
Anyone know if that’s viable?
Does this also solve provable compute as a side effect? Since the compute is fully encrypted end-to-end, the host wouldn’t be able to modify the program and still give valid results?
So a caller could run a program on untrusted hardware and trust that the hardware actually ran the program?
I‘m new to this, but mildly skeptical. In complex agentic engineering, there are lots of intermediate steps that depend on each other. Even if you disregard the significant encryption cost, this makes it impractical. So seems like this will stay a niche.
It's "private" "cloud" computing all over again, this time brought to you by jensen et al.
One flaw with FHE is that it guarantees only that you need the key to see the inputs or outputs of the computation, but not necessarily that the computation is the one you want. For example, the computation could be adversarial for certain inputs, or an adversary could insert their own computation first (or last).
I really want nobody to waste cycles on my AI than me. This is useless because now I need to pay for wasted inference AND encryption cycles on top of wire encryption.
Funny to read this, but google is currently number one anti-privacy big tech company. No other company makes it as difficult for its users using anonymization/privacy software to access their services
Ideally: we will have private AI
Reality: our computers will be used as distributed AI calculators
I think at 1000x it's probably way cheaper to do an "AWS outpost" style thing. Although not sure if it's a fundamental constraint or something that could be optimised.
Why is the bold letter n lighter than the rest in this page's font?
It sounds neat, but I do wonder how viable this is commercially. How high do we rate the chances that governments around the world will step in before another kind of E2E is rolled out.
The only question is do you trust Google? No…
Quoted from the blog post:
> user-data can be protected from data breaches, but then the service provider cannot provide features that depend on the data, such as spam or virus detection
I think they forgot "or advertising" at the end.
I don't trust Google. I would much prefer to use on-prem or - at most - one of the secure-enclave providers like Tinfoil[1] or Private Mode[2]
Encryption or not, if it's on somebody else's server, it isn't yours. I don't believe Google has my best interest.
Does this rely on the Trust Me Bro model, or is there some way for the client to verify that the provider actually isn't able to see your inputs?
I want to read a whitepaper but all I can find is the tl;dw conference presentation
It's a step to provide targeted advertising with mathematically provable "no sensitive info stored" approach. The google must fight really hard because this is the only source of income that makes sense for their position. And it /is/ quite evil tbh
Have all the skeptics in this thread somehow forgot about Moore's law?
Man oh man did I read that title wrong and was completely confused
Correct. I appreciate the theoretical technology here, but I believe a great deal of harm is done by the fact that people are not likely to understand exactly what this means.
Which is to say, I believe that google is strongly implying the falsehood of "no one at Google can read your stuff."
> Google’s history of innovations in privacy technology
Ah yes, their history of innovations! Thanks to Gemini for making a list for me over at https://share.gemini.google/vx9lE5dEJ1L7
The list contains more than 6 chapters with multiple entries each.
Google has truly been very innovative in destroying privacy online.
Zama.ai is also a player in this space
Private AI is practical by running the model locally, every much more so than any homomorphic encryption scheme.
So essentially the headline sells this as work to keep your data private, but really it's work to keep the AI-- which was trained on your code and your writing-- private.
all we need now is indistinguishable obfuscation