162
Aggressive AI scrapers are making it kinda suck to run wikis
(weirdgloop.org)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 21 May 2026
162 points (99.4% liked)
Fuck AI
7133 readers
988 users here now
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
founded 2 years ago
MODERATORS
No. CF lost years ago, and the checks can be bypassed easily. It's just that it blacklists ips generating insane traffic but there is a lot of margin
I'll admit that I am a newbie, so I ask in ignorance: have you tried using Anubis + BadBotBlocker + Fail2Ban?
It genuinely worked wonders for my tiny site that was being bombarded.
Crawlers have never been a problem for me as my internet subscription is unlimited. My experience comes from being the one crawling sites and bypassing the CF challenge
I put my little blog behind Cloudflare because I was tired of it going down due to scrapers overwhelming my little VPS.