46
submitted 2 days ago by [email protected] to c/[email protected]

Almost every website and services are getting scraped at alarming rate, are Lemmy servers facing this issue?

Please share mitigations you've seen applied to this.

you are viewing a single comment's thread
view the rest of the comments
[-] [email protected] 29 points 2 days ago

I think lemmy content is scraped too, just how the whole web is beeing scraped. I do not have any proof for it though.

I have seen a user add a like anti-commercial AI license as a footer for every comment he writes lol

[-] [email protected] 13 points 2 days ago* (last edited 2 days ago)

Those are truly useless to go against bad actors and is instead only annoying for the humans that read. And good actors with proper licenses won't be scraping Lemmy, Reddit or Twitter.

You just cannot prevent it on Lemmy because if an instance places filters like Anubis, another will not. And it is not feasable to mandate every instance to do so. Also, this is an open platform by nature and there is no group or company that can mandate rules of access. As you are limiting non-humans, you might also be limiting real users with peculiar configurations or under heavy privacy middlewares.

[-] [email protected] 2 points 2 days ago

The point (as I see it) is not so much to stop scraping as it is to prevent bots from effectively DDOS-ing web services. As others have said ActivityPub content is public and there are ways to get it without slamming instances with scraper bots.

load more comments (5 replies)
this post was submitted on 10 Jul 2025
46 points (96.0% liked)

Asklemmy

49361 readers
642 users here now

A loosely moderated place to ask open-ended questions

Search asklemmy ๐Ÿ”

If your post meets the following criteria, it's welcome here!

  1. Open-ended question
  2. Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
  3. Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
  4. Not ad nauseam inducing: please make sure it is a question that would be new to most members
  5. An actual topic of discussion

Looking for support?

Looking for a community?

~Icon~ ~by~ ~@Double_[email protected]~

founded 6 years ago
MODERATORS