I found that using a wildcard domain cert for HTTPS and an uncommon higher port number works just fine to avoid crawlers and scrapers. I also hide my ssh port behind WireGuard because I got annoyed with the bots filling up my logs with failed attempts. I've been doing that for nearly a year now and I have had no more issues with bots
I'm going off the assumption that the people who run crawlers and scrapers are going for known common ports for quick ease of access
I don't feel the need to run anything extra such as Anubis either. The inconvenience of using an uncommon higher port number has been minimal and I still get to enjoy my tiny corner of the internet