351
submitted 1 week ago* (last edited 5 days ago) by [email protected] to c/[email protected]

Edit: unexpectatly we are back online. Will share how we found a solution later. There might be still some federation issues though.

Alt account of [email protected] here.

Our instance is currently down and I can't get remote access to the servers. It appears that there might have been a hardware failure of the main firewall, which is the one thing I can't work around remotely.

I am still trying a few things, but I am not very optimistic that I can get access.

The really unfortunate part is that just now I am on one of my rare work deployments abroad, so I also can't access it physically during the next few weeks and my usual back up that could restart it is not available either.

As something like that never happened in 3 years operating the servers, I thought I can risk it, but murphy's law seems inescapable 😓

I will try to keep you posted here on any updates, but probably there will not be much I can do for a while. Really bad timing 😥

Edit: we might use this "opportunity" to migrate the instance to Piefed, which has been an idea for quite some time now. I will keep you posted on that.

(page 3) 36 comments
sorted by: hot top new old
[-] [email protected] 3 points 1 week ago

You guys have always been excellent. As others have stated, no need to worry.

I'll be giving piefed some love in the meantime 👍

[-] [email protected] 2 points 1 week ago

I also can’t access it physically during the next few weeks and my usual back up that could restart it is not available either.

Sounds like your 'bus factor' is too high and you need a few more backup admins.

load more comments (13 replies)
[-] [email protected] 1 points 1 week ago* (last edited 1 week ago)

I am a slrpnk user. Oh well, shit happens. What makes me a bit sad was that a few months (?) ago something similar happened. Slrpnk.net was down for aa day or two? Don't remember the exact amount of time. What I do remember is asking if there was intention of adding a second admin and making the point that it would be good to create that sort of redundancy. I was ignored.

Mid of july is basically 1 and a half months from now. Hopefully a new solarpunk instance emerges, one that is safer from such scenarios.

Enjoy your vacation and don't stress too much! Shit happens.

load more comments (6 replies)
[-] [email protected] 1 points 1 week ago

Opengear cellular backup appliance with serial/console connectivity to hardware.

load more comments
view more: ‹ prev next ›
this post was submitted on 04 Jun 2025
351 points (100.0% liked)

Fediverse

34512 readers
510 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to [email protected]!

Rules

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration)

founded 2 years ago
MODERATORS