you are viewing a single comment's thread
view the rest of the comments
[–] 56 points 1 day ago (3 children)

If I didn't enjoy the cat & mouse of it, I'd definitely walk away from running a site. Just out of curiosity, I created an hourly graph today showing malicious bot, legitimate bot and human ratio graphs for one of my sites. The malicious hits make up around 80% hourly and human hits are usually well under 10%.

image

  • source
  • hideshow 3 child comments
  • [–] 1 point 8 hours ago (2 children)

    What sorts of measure you have taken to prevent bots from exhausting your compute and bandwidth

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 hours ago*

    Another example expanding on my previous comment.

    This IP from Hong Kong hit my site over 5,000 times in a couple minutes. The first 12 page loads were legitimate URLs and got served. On their first exploit attempt, they were blocked and the remaining 4,98x page loads were no-effort 403s. This type of thing has made it possible for me to have a cheap public-facing site as I wouldn't be able to afford an endless array of malicious IPs doing this day in and day out. Before I went this route, I would have to shut the site down during high impact periods of these visitors as they'd corrupt and crash the database.

    image

  • source
  • parent
  • [–] 2 points 3 hours ago* (last edited 3 hours ago)

    As soon as an IP is determined to be malicious, subsequent pageloads result in a single query to match the IP and they get dumped to a 403 static error.

    With the bots that hammer a site looking for the top 1,000 or so exploitable URLs or partake in speed scraping, this has worked wonders for my server load. I catch them in their 1st-5th page load and the rest just result in basically no effort on the part of the script.

    image

  • source
  • parent