585
I really hope they die soon, this is unbearable…
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 13 Feb 2026
585 points (98.8% liked)
Selfhosted
60723 readers
172 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil.
-
No spam.
-
Posts are to be related to self-hosting.
-
Don't duplicate the full text of your blog or readme if you're providing a link.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
-
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 3 years ago
MODERATORS
It's best to use either Cloudflare (best IMO) or Anubis.
If you don't want any AI bots, then you can setup Anubis (open source; requires JavaScript to be enabled by the end user): https://github.com/TecharoHQ/anubis
Cloudflare automatically setups robots.txt file to block "AI crawlers" (but you can setup to allow "AI search" for better SEO). Eg: https://blog.cloudflare.com/control-content-use-for-ai-training/#putting-up-a-guardrail-with-cloudflares-managed-robots-txt
Cloudflare also has an option of "AI labyrinth" to serve maze of fake data to AI bots who don't respect robots.txt file.
If you're relying on Cloudflare are you even self-hosting?
If you build a house, but hire a guard for the front gate, do you even own the house?!
If you use DNS at all, do you even own your street address!?!?
Yes if it's tunneled to your self-hosting setup. With CGNAT you have to use similar services if you want to self-host.
Pretty sure I've repeatedly heard about the crawlers completely ignoring robots.txt, so does Cloudflare really do that much?
Like a lock on a door, it stops the vast majority but can't do shit about the actual professional bad guys
Cloudflare definitely can and does stop the vast majority of actual professional bad guys.
Yes, CloudFlare blocks agents completely if they ignore it's restrictions. The key is scale - CloudFlare has a birds eye view of traffic patterns across millions of sites and can do statistical analysis to determine who is a bot.
I hate the necessity but it works