back
177 comments
For those who are lazy like me—or maybe you'd call it efficient—here's the key points summary from Kagi:

- Google Search has evolved significantly from its early days as a simple list of blue links to becoming an encyclopedia, predictive engine, image repository, shopping mall and more that is overloaded with information.

- It has become more difficult to find authoritative answers on Google Search due to an overabundance of sponsored content, prompts, and low-quality keyword-stuffed pages.

- Google is currently undergoing an antitrust trial regarding whether it maintains its search engine monopoly through anticompetitive means such as exclusivity deals rather than having truly superior technology.

- Access to large amounts of user data is very important for powering search engine algorithms through personalization and improving the user experience.

- While Google argues diminishing returns to scale for user data, internal emails show Google engineers acknowledging that scale remains highly important for the company.

- Some feel Google has lost its way and become conservative due to its success, with its corporate bureaucracy stifling acquired companies.

- Google Search has become bloated with advertisements and prioritization of its own services over organic results.

- Search quality may be declining as its algorithms are gamed by low-quality sites and search engine optimization techniques.

- Google's trajectory of scaling up its mission to organize the world's information through acquisitions and exclusivity deals has put pressure on it to keep growing.

- No company can likely grow indefinitely to keep up with the ever-expanding internet without being overwhelmed, as Google Search now demonstrates.

Thanks for the excellent summary. Though I think the article and summary miss the root cause: Google Search has two masters, users and advertisers. These stakeholders’ interests are not aligned, and by taking one middle road after another, Google has deteriorated into near-uselessness for end users, relying on scale rather than quality.
> Google is currently undergoing an antitrust trial regarding whether it maintains its search engine monopoly through anticompetitive means such as exclusivity deals rather than having truly superior technology.

I'm not really sure how they lose that case -- Firefox switched to Yahoo after MS outbid them, and switched back to Google the second they could legally do so and keep the money.

I'm still not sold on the idea that "personalization" improves the "user experience" of search.

When I think about what I actually use search for... it's highly situational. It depends as much on day-to-day and moment-to-moment context as it does any long-term trend specific to myself.

Whatever "personalization" could be realized over search results, would it really offer that much benefit over improving the search function to sift through and find results that are just more useful generally?

What I mean is there seems to far, far more to be gained from making search results better fit the query than to fit me personally.

That’s a pretty great summary. Can you elaborate a bit on how this works with Kagi? Is it just a wrapper over gpt-4, do you feed it an article and ask for a summary, or does it summarize in the search results?
Why is it that when I use Google search, the links are always some intermediary google.com/** links which when clicked takes me to the original site. Is this some kind of user tracking?
I would add, as advertiser this time, that you don't know what you are really paying in AdWords. I used AdWords from the beginning and there were business leads for super niche spaces, now it is very difficult to get leads with a moderated budget and you cannot explain clearly why.
Apparently (since this is the top-ranked comment) I'm alone in thinking that this sort of "summary" is dumb and pointless. Just skim over the article. You cannot separate form and content. What's next -- a PowerPoint summary?
As the article mentions, the web has changed. It's very hard to return a list of good results when there are none. I've yet to see side-by-side comparisons where another search engine is doing a better job.

I've had "Search Generative Experience" (AI/LLM synthesized result/summary above the standard web results) turned on for awhile. It is definitely the future. To survive the flood of revenue optimized content (recipes being a familiar example), you need an active [software] agent working on your behalf to read content and make one straight thing out of the crooked timber of the web.

The problem isn't just that the internet has become more SEO-oriented and encumbered with low-value clickbait websites -- it's also that Google doesn't listen to instructions the way it used to.

Google started slipping when it began assuming that it knows better than you do. For instance, when the exact search operator (" ") stopped working.

I'd take the Google of 2014 over the Google of 2023 without thinking twice, and I think that I'm not alone in this.

I'll also had that a huge portion of genuinely good content is now inaccessible to Google in walled gardens like Discord, Facebook, Twitter, and TikTok.
So, how paid search engines like Kagi manages to bring useful results every time?

Its results are superior, with way less spam and SEO optimized content.

"To survive the flood of revenue optimized content (recipes being a familiar example), you need an active [software] agent working on your behalf to read content and make one straight thing out of the crooked timber of the web."

So what do you think, will be the goal of all advertisers?

To get their recommendations baked into that AI. Direct bribing/revenue sharing with the manufactur of that AI. Or sneaking data into training sets via bribed employes and so on.

I don't trust black boxes and never will.

I don't think the problem, in general, is that no good results exist for any given query, more often the problem is that the good results are rarely well search engine optimized, and thus stand zero chance of making the search results.

Some people admittedly expect pretty weird websites to exist. Like who would even publish a good objective comparison between products in a specific category. But people make those queries looking for those types of results, and spammers generate such content to make bucks off the traffic. 20 years ago you'd have gotten nothing in most cases.

> To survive the flood of revenue optimized content..., you need an active [software] agent working on your behalf

How long do you think it will be before that software agent's output will be just as riddled with commercialized content? I think we might have only a short window where we get to enjoy the fairly unadulterated output of the base LLMs by the large providers (yes, I'm aware that they're screening for safety already, but there could be a lot more coming). Imagine a dialogue like this: Q: What should I watch out for when buying new car tires?

A current answer might be something like: Here's a list of things to watch out for when buying car tires: A, B, C, D, E. Remember to always consult with specialists. Shall I explain more about the topic?

A future answer might look something like: According to experts like the professionals at [tire shop near you], here are the things you need to watch out for when buying car tires: A, B, C. You can use this coupon to get a 10% discount off your next purchase at [tire shop near you]. Shall I book an appointment for a free consultation at [tire shop near you] for you?

I don't believe that anything Google does now - summarization included - will avoid being gamed just as effectively as their current search results have.

It's a negative-sum arms race, but if Google (or whoever) wasn't spending the money to fuel the arms race by passing on ad money, who would pay for Google to exist in the first place?

> you need an active [software] agent working on your behalf

I'd love to have one, but do any exist? SGE works on google's behalf, collecting data for google who will use what they learn from it against me. I'm guessing that until we can use common hardware to train/host an AI agent ourselves, we're stuck with terrible search results unless we're willing to hand over more and more data to others who want to exploit us, while pretending we can trust them to return results that are in our best interest instead of theirs. That's not a great future to look forward to.

I am convinced that a lot of the reason why Google search went downhill is that they started using more AI. Specifically, using it to try to interpret what I'm searching for rather than taking my word for it, which leads to irrelevant search results.

That's just my hypothesis, of course. I don't know for certain. But it does make me very skeptical about using AI to help search the web.

I want a robust set of search modifiers instead.

I liked StartPage/Ixquick results much better than what Google used to show. Maybe they still do but wouldn't know as I stopped using it after they got acquired. Didn't/don't they rely on Google?
It certainly seems like Google Search prefers to return ad-ridden monetized sites rather than more usable sites, with the obvious explanation being that they earn more money from sites that show ads.

But this feels like too obvious a conspiracy theory and it’s probably not directly designed to act that way - more likely it is an unintended consequence of something else. Unintended, but tolerated all the way to the bank.

I guarantee you no experiments that negatively impact revenue but improve the search experience ship
Unfortunately, the sad truth is likely that Google has hard telemetry evidence that users in aggregate prefer this SEO junk, engage with it, and move on feeling that Google successfully satisfied their query.

If Google could drive higher levels of engagement and satisfaction by screening out the dross I think they would, it’s clearly within their abilities.

Hopefully AI gets good enough and scales efficiently enough so they can disaggregate this kind of ranking decision.

I would guess they monitor tons of metrics for any change to search and any change that reduces ad revenue gets rolled back or reviewed.
A more likely explanation is that ad-ridden monetized sites are able to invest more in SEO since a visitor is worth more to them than an ad-free or low-ad site.
> no company or product can grow alongside the internet forever without, eventually, being swallowed up.

Even not Google. I wonder if they can still turn things around, but I have the feeling MS caught up finally at high speed and passed them.

PS I was reading (sceptically) 'google search is broken' already for a year or so in HN comments and have to admit I believe it now.

https://archive.ph/X3gHI
In case anyone is interested: the RSS feed for The Atlantic is still pretty good, able to read most articles in full text.
The other day I asked Google "Are US drivers licenses forgery-proof?" and "Can US Real-ID drivers licenses be forged?"

The #1 or #2 hit is utterly irrelevant: nothing to do with US or Real-ID (zero mentions), it's about New South Wales, Australia (lots of mentions of "NSW"):

> ‘Tough to Forge’ Digital Driver’s Licenses Are—Yep—Easy to Forge. Researchers found a litany of security flaws that allow simple, quick, and cheap forgeries in Australia. https://www.wired.com/story/digital-drivers-license-forgery-...

Kagi needs a free tier.

even if its only 10 searches a day, it needs something. I am never going to use it unless I can try it out for a while.

Google became unusable for me. I have to suffix every search by "reddit" in order to get something useful. Same with stack overflow, I now mainly look at GitHub issues to find what I'm looking for. Man I wish we could just ban specific domains in order to get less shitty results from Google. I tried brave Search and duck duck go, they are just as bad imo.
Anecdote: I’m not sure what changed with Google last week, but search on mobile is almost unusable now. I see maybe six real results and the more I scroll, the more ads and recommendations that I see. The more I scroll, the less relevant the recommendations become to the point of absurdity. It’s ridiculous and feels like pinterest/Facebook/TikTok.
There is a logical issue here, people are accusing Google with the line of attack that large scale gives them benefits. I've seen variants of this before:

> “It’s absolutely not true that scale is not important,” Manber wrote. “We make very good use of everything we get.”

I think that is true, but then the obvious conclusion - if one party doesn't have 90% of the market then the quality of search results will be lower. Successful anti-trust prosecutions against the party with 90% of the market will have a likely outcome of:

1) Reducing the quality of service for everyone.

2) Pushing the 90% market company to be one domiciled somewhere with more reasonable monopoly laws.

Successfully proving that more data -> more scale -> better results is actually implies that one company with 90% market share is desirable. Although I'm still not going to use Google search, I don't think that argument is correct. But the line of attack is not clever.

Seems like the complaints that have been showing up here a for a while have reached the mainstream. Any chance of an upstart competitor taking the opportunity?
Buyer beware: Sensation sells (as is portrayed in this news article). Competitors are one click or one setting away. Last I tried switching to DuckDuckGo things didn't go so well -- just overall subpar experience. Bing arguably is a bit better, but that too didn't satisfy, despite the super cool chat feature (deserves kudos for that!).
Good to see non techies start to care about this
I'm a little tired of all these "Google search used to be better" feel-pieces that don't do anything to substantiate the claim beyond relying on the author's vague sentiment.
I know there are other search engines out there, but I haven't really given any a shot. Do any of them feel like the Google Search of the past?
The Orion browser does not seem able to render Kagi as a search engine when something is searched. It straight up doesn't load anything.

What do?

I wish they had an option for me to always exclude certain sites from the resulrs
Google search quality started dropping when they dropped the plus operator
To read the full story start your free trial today sigh
The Tragedy of Google Analytics…
I won't even read it.

Google remains the best search engine, period.

Nothing is even close.

It’s not a story the Jedi would tell you.
I have always speculated, what if Google simply turn off shop.

Just turned off its servers and said "we're closing".

Then some would realise what a huge impact it serves on a daily basis.

Pop culture will never last.

To whom it may concern,

We're building a new search engine. Of course it's AI driven, but at its core, it's all about human dignity. When you use it, you'll have a voice. If you'd like to try it, please email me: josh@sirch.org

Thanks! Josh

PS. If you'd like to join our team, we have 16 people now (mostly Snapchat/Instagram/Twitter folks), and we'd like to go up to 50 before we raise capital. It's going to be a wild ride.