585
Based on this graph, and this graph alone, guess at what time I completely blocked OpenAI crawlers (thelemmy.club)
submitted 5 months ago by hylobates@jlai.lu to c/selfhosted@lemmy.world
80 comments fedilink hide all child comments

I really hope they die soon, this is unbearable…

you are viewing a single comment's thread
view the rest of the comments
[-] quick_snail@feddit.nl 4 points 5 months ago

Y'all need to learn to cache things, shiit

[-] poVoq@slrpnk.net 5 points 5 months ago

This is not how things work on the modern web. Did you just wake up from a 20 year coma?

[-] FreedomAdvocate@lemmy.net.au 1 points 5 months ago* (last edited 4 months ago)

Whenever you’re browsing even a semi popular website these days there’s probably a 98% chance you’re hitting a cloudflare cached version of it. Have you been asleep the last 10 years?

[-] poVoq@slrpnk.net 1 points 4 months ago

For static sites, yes. To actually protect dynamic sites against AI crawlers, Cloudflare has to do much more than just caching.

And besides that, Cloudflare is a huge single point of failure and highly privacy invasive.

[-] FreedomAdvocate@lemmy.net.au 1 points 4 months ago

Dynamic sites still get cached.

Cloudflare definitely is a huge single point of failure, and it is a huge problem imo - but what can we do? Their product is so widely used because of how comprehensive, good, and necessary it is.

[-] hylobates@jlai.lu 1 points 5 months ago

There is cache but for some reason this does not affect every page I serve currently

[-] quick_snail@feddit.nl 3 points 5 months ago

Fix the cache.

this post was submitted on 13 Feb 2026
585 points (98.8% liked)

Selfhosted

60723 readers
171 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Detailed Rules Post

  1. Be civil.

  2. No spam.

  3. Posts are to be related to self-hosting.

  4. Don't duplicate the full text of your blog or readme if you're providing a link.

  5. Submission headline should match the article title.

  6. No trolling.

  7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

  8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
ayyy@sh.itjust.works
curbstickle@anarchist.nexus
curbstickle_lw@lemmy.world