this post was submitted on 15 Jul 2023
12 points (100.0% liked)
Reddit Migration
16 readers
2 users here now
### About Community Tracking and helping #redditmigration to Kbin and the Fediverse. Say hello to the decentralized and open future. To see latest reeddit blackout info, see here: https://reddark.untone.uk/
founded 1 year ago
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
On one hand, I support a strong robots.txt being in place. It keeps data from being used by honest engines (though what constitutes "honesty" varies). But at the same time, indexing and caching is how we can grow. If you want the site to grow, you want to get it to 1st position on first SERP.
It's a tricky balance.