585
Based on this graph, and this graph alone, guess at what time I completely blocked OpenAI crawlers (thelemmy.club)
submitted 6 months ago by to c/selfhosted@lemmy.world
79 comments fedilink hide all child comments
 

I really hope they die soon, this is unbearable…

A graph of MySQL queries showing a violent drop in requests after blocking OpenAI Crawlers
you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 6 months ago (6 children)

Y'all need to learn to cache things, shiit

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 5 points 6 months ago (3 children)

    This is not how things work on the modern web. Did you just wake up from a 20 year coma?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 6 months ago* (last edited 6 months ago) (2 children)

    Whenever you’re browsing even a semi popular website these days there’s probably a 98% chance you’re hitting a cloudflare cached version of it. Have you been asleep the last 10 years?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 6 months ago (1 child)

    For static sites, yes. To actually protect dynamic sites against AI crawlers, Cloudflare has to do much more than just caching.

    And besides that, Cloudflare is a huge single point of failure and highly privacy invasive.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 6 months ago

    Dynamic sites still get cached.

    Cloudflare definitely is a huge single point of failure, and it is a huge problem imo - but what can we do? Their product is so widely used because of how comprehensive, good, and necessary it is.

  • permalink
  • fedilink
  • source
  • parent