you are viewing a single comment's thread
view the rest of the comments
[–] 33 points 3 months ago (4 children)

Information-gathering agents are an evolution of Google Alerts. Beyond spotting changes, they can make sense of them, too.

… Links will become an afterthought with the coming changes to the Search results experience.

Web publishers should honestly just block googlebot at this point. Why should they provide credibility to whatever Google's stochastic parrot hallucinates if Google won't even give them any kickback?

So what do we search with instead of Google? There isn’t a lot of choice. There’s various flavours of Google or Bing.

Microsoft deprecated their Bing API back in August, instead telling people to use some Azure AI thing. DDG and the like weren't affected because they have contracts, but I can't imagine they'll be renewed.

  • source
  • hideshow 8 child comments
  • [–] 6 points 3 months ago (2 children)

    If sites start blocking googlebot en masse, then googlebot will just start ignoring robots.txt

  • source
  • parent
  • hideshow 4 child comments
  • [–] 4 points 3 months ago (1 child)

    Then you can just block the user agent in nginx or whatever you use, like all the other AI scrapers who ignore robots.txt (*cough* Amazon)

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 3 months ago (1 child)

    Then the user agent string will just quietly become randomised so you can't match it reliably because it turns out that honouring robots.txt was always little more than a "gentleman's handshake".

  • source
  • parent
  • hideshow 2 child comments
  • [–] [S] 3 points 3 months ago (1 child)

    this is a problem we have had for a while now, i assure you

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 3 months ago

    Yeah an adversary like Google isn't something you can easily block without really annoying legitimate users unfortunately. Nothing is stopping them from turning every chrome instance into a botnet node except for the angry article that would run in Ars Technica.

  • source
  • parent
  • [–] 3 points 3 months ago (1 child)

    I think it's possible to allow google search bots but not Gemini bots?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 13 points 3 months ago (1 child)

    The point is that there's going to be no difference, soon Google search will be just another chatbot interface.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 3 months ago (1 child)

    Guess I'll just have to use ddgo, then ¯⁠\⁠_⁠(⁠ツ⁠)⁠_⁠/⁠¯

  • source
  • parent
  • hideshow 2 child comments
  • [–] 5 points 3 months ago (2 children)

    Unfortunately DuckDuckGo sucks ass at search, even compared to how much the Google search results have degraded over time. I use the no-AI version as my primary search engine, but I have to resort to using Google to find the thing I'm looking for about 1 in 5 times.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 8 points 3 months ago

    DDG has been my main search engine for over a decade but it has degraded as it became basically a reseller of Bing results after Russia started the current phase of the Ukraine war and they stopped partnering with Yandex.

  • source
  • parent