This is a little bit of a wall of text but I hope it will be a nice sample for such a setup.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 4 hours ago

For me, every single request is coming from a different random IP address (different ASN, different country, etc.) using a random plausible User-Agent. There's no way to discern them as bot requests. And since every request is coming from a random source, things like fail2ban or CrowdSec don't really work. You can block a few major ASNs but there will still be lots of requests making it through.

And these things work by going through a list of URLs previously collected. About a week ago I've configured a redirect for ALL requests to my personal Gitea instance to a zip bomb. It's been 7 days now where the website was inaccessible and I still see requests for deep links, e.g. specific commits or lists of pull requests. These things are relentless. That's also one of the arguments against Anubis - those scrapers just don't care.

I've found some comments that at least a few ISPs check AbuseIPDB.com and cancel their customer's contracts if their IP appears in that database. So, I've now resorted to occasionally collect those IPs hammering my poor Raspberry Pi and reporting them to the AbuseIPDB. If people are unknowingly running a "residential proxy" (why don't we call them "open proxies" anymore?) somebody needs to let them know, I guess.

  • source
  • parent