585
Based on this graph, and this graph alone, guess at what time I completely blocked OpenAI crawlers (thelemmy.club)
submitted 6 months ago by to c/selfhosted@lemmy.world
79 comments fedilink hide all child comments
 

I really hope they die soon, this is unbearable…

A graph of MySQL queries showing a violent drop in requests after blocking OpenAI Crawlers
you are viewing a single comment's thread
view the rest of the comments
[–] 150 points 6 months ago (4 children)

Reference for lazy ones: https://zadzmo.org/code/nepenthes/

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 78 points 6 months ago

    Oh interesting! Ive done something similar but not didnt put as much effort.

    For me, I just made an unending webpage that would create a link to another page...that would say bullshit. Then it would have another link with more bullshit....etc...etc...And it gets slower as time goes on.

    Also made a fail2ban banning IPs that reached a certain number of links down. It worked really well, traffic is down 95% and it does not affect any real human users. Its great :)

    I have a robots.txt that should tell them not to look at the sites. But if they dont want to read it, I dont want to be nice.

  • permalink
  • fedilink
  • source
  • parent