Another web scraping bot blocked.

Made some time at work, don't tell my boss...

Think of lemmy instances as countries: when you visit a country/instance you're expected to abide by their local rules, whatever they may be. And when users from other countries/instances interact with Australia/aussie.zone they're expected to behave in line with our rules.

Not agreeing with the population of a country/instance doesn't mean you can't visit it. If/When you visit, you can't expect them to adhere to your rules at home... or to apply their own rules consistently.. just like the real world.

Defederation of an instance will only be done when:

Legal- the instance is generating content that may raise legal concerns for Aussie Zone. For example porn.

Technical- the instance is generating content that may cause performance/security issues for Aussie Zone. For example large volumes of automated traffic or malicious traffic.

Trolls- an instance whose users predominantly interact in bad faith with communities outside of their instance.

I don't believe lemmy.ml meets any of these, their "quirky" politics are largely self contained on their own communities from what I've seen. If they're crashing aussie.zone communities and posting outside of our rules, please report them.

To address some comments in this thread (paraphrasing):

we should defederate instances with bad admins or admins heavily moderating alternative views to their own

So far as their actions do not trigger any of the defederation criteria noted above, they can do as they see fit (even if it does make hypocritical dickheads).

no incentive to create alternative communities
If there is no incentive, then it can't be a large enough issue to enough users.

pro-authoritarian bias
Given how simple it is for a user to block an instance for themself, I'd rather allow our users had the option to decide this for themselves. Some prefer to engage with people from an alternate perspective, for those that don't they can block the instance.

TLDR

Not defederating lemmy.ml at this point in time, if their user behaviour outside of their instance changes in future my stance may change too.

It was removed deliberately during the reddit exodus in order to direct new Lemmy users elsewhere. Rather than to overload lemmy.ml further.

pict-rs upgraded to 0.5... which is now performing its own backend update. No idea how long it will take... currently ~7%.

The way lemmy caches images isn't well documented. Some third party images are not cached, others are. I don't want to risk it.

Nuking it from orbit, only way to be sure.

Tldr of what I've posted elsewhere; I'm undecided at this stage but leaning towards defederating. When more information is available I'll make a post soliciting feedback, I don't want to make a change like this without user feedback.

When I looked there previously it looked like a bunch of mostly harmless cookers. Looking again now, I see hateful posts such as this:

I've now defederated them. The NSFW focused instances are defederated to minimise potential legal complications, ie to protect this instance.

Mushroom mushroom!

Damn I'm old. For those too young, check this out.

On the point of being unsustainable I disagree. Instances will need to find an equilibrium between cost/expense and retention of old content. The higher the revenue/cost tolerance, the older the content that can be retained. I expect most instances will end up purging non-local content after an amount of time, but retain local content as long as possible. Maybe I'm naive, but I have confidence that people smarter than me will come up with systems to do this. It may result in a usenet style setup where instances boast about their retention periods.

On your second point re: community contributions, I agree entirely. I've been very fortunate that there have been some generous donations from aussie.zone users, so I'm not worried about server costs at this point. Server costs will go up as data volumes increase, that is unavoidable. How the community decides to handle this in the future is the real question, based on what I've experienced so far I'm confident we'll be around for a long time to come.

Check 12ft.io

Many of the largest instances block lemmygrad. You can check by clicking the "instances link at the bottom of the page.