I'm trying to understand the bot problem in the internet and finding more ways to defend myself. One thing that I can't seem to understand is why most bots, scrapers and crawlers seem to have residential IPs.

  • Is it that ISPs are being paid by tech-bros to assign them these IPs?
  • Is it that residential devices have been hacked /contain malware that does this?
  • Is it trivial for companies to assign themselves residential IPs?
  • Paid volunteers are doing this for AI companies?

Or is there is some other reason for this?

Obviously this is a problem because one can rotate / cycle through residential IPs and if I aggressively block each offender in my logs permanently, then the next person assigned this IP who may be a legitimate user will be unable to access my site.

you are viewing a single comment's thread
view the rest of the comments
[–] 8 points 2 months ago (1 child)

Not the ISPs problem at all.

Blame IoT manufacturers for providing shit devices that can easily be hacked.

Theere is no difference from a technical standpoint between a business and residential IP. It's allocation is tied to the network provider.

People install extensions onto browsers for any number of reasons without knowing what it does. It was a problem is the 90s that never went away.

  • source
  • hideshow 2 child comments