Selfhosted

39250 readers

213 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago

MODERATORS

[email protected]

Have I been DoS'd? (lemmy.world)

submitted 1 year ago by [email protected] to c/[email protected]

31 comments fedilink hide all child comments

Hi,

I know this is quite impossible to diagnose from afar, but I came across the posting from lemmy.world admins talking about the attacks they are facing where the database will get overwhelmed and the server doesn't respond anymore. And something similar seemed to have happened to my own servers.

Now, I'm running my own self-hosted Lemmy and Mastodon instances (on 2 seperate VPS) and had them become completely unresponsive yesterday. Mastodon and Lemmy both showed the "there is an internal/database error" message and my other services (Nextcloud and Synapse) didn't load or respond.

Login into my VPS console showed me that both servers ran at 100% CPU load since a couple of hours. I can't currently SSH into these servers, as I'm away for a couple of days and forgot to bring my private SSH key on my Laptop. So, for now I just switched the servers off.

Anyway, the main question is: what should I look at in troubleshooting when I'm back home? I'm a beginner in selfhosting and I run these instances just for myself and don't mind if I'd have to roll them back a couple days (I have backups). But I would like to learn from this and get better at running my own services.

For reference: I run everything in docker containers behind Nginx Proxy Manager as my reverse proxy. I have only ports 80, 443 and 22 open to the outside. I have fail2ban set up. The Mastodon and Lemmy instances are not open for registration and just have 2 users each (admin + my account).

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (1 children)

Oh, ok, you whitelist IPs in your firewall. That certainly works, if a bit brutal. :) (then again, I blacklist everyone who is triggering a 404 on my webserver, maybe I'm not the one to speak about brutality :P ) You don't even need a VPN, then, unless you travel frequently (or your ISP provides dynamic IP, I guess).

[–] [email protected] 1 points 1 year ago (1 children)

Well that's a bit of both: I need to be able to get on my server from work (with my phone... Yeah not great but that works), because I often break stuff haha ; also a nice thing to have when I'm on the bus and want to add more music or movies to listen to during the travel!

Are there ISPs that don't provide dynamic IPs? I had to setup a script and get some API keys for different services to ensure the IP is properly updated on my DNS servers.

Speaking of brutality, I considered doing the same but then I would have banned myself from testing the APIs of my services 🤧

[–] [email protected] 2 points 1 year ago

Oh, I see. Totally makes sense. :)

I guess it depends on the country, but here in France, yes, most landline ISPs provide static IPs (maybe all? there are a couple I haven't try ; mobile IPs are always dynamic, though). It was not always the case, but I haven't had a dynamic IP since the 2000'. I feel you, dealing with pointing a domain to a dynamic IP is a PITA.

Ahah, yeah, I protected myself against accidentally banning my own IPs. First, my server is a Pi at home, so I can just plug a keyboard and a screen to it in case of problem. But more importantly, as I do that blacklisting through fail2ban, I just whitelisted my IPs and those of my relatives (it's the ignore_ip variable in /etc/fail2ban/jail.conf)., so we never get banned even if we trigger fail2ban rules (hopefully, grandma won't try to bruteforce my ssh!). It allowed me to do an other cool stuff : I made a script ran through cron that parses logs for 404 and checks if they were generated by one of the IPs in that list, mailing me if it's the case. That way, I'm made aware of legit 404 that I should fix in my applications.