858
I didn't know you were supposed to disable root user... (linux.community)
submitted 2 years ago by to c/linuxmemes@lemmy.world
170 comments fedilink hide all child comments
 

Background: 15 years of experience in software and apparently spoiled because it was already set up correctly.

Been practicing doing my own servers, published a test site and 24 hours later, root was compromised.

Rolled back to the backup before I made it public and now I have a security checklist.

you are viewing a single comment's thread
view the rest of the comments
[–] 52 points 2 years ago (1 child)

Interesting. Do you know how it got compromised?

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] [S] 73 points 2 years ago* (8 children)

    I published it to the internet and the next day, I couldn't ssh into the server anymore with my user account and something was off.

    Tried root + password, also failed.

    Immediately facepalmed because the password was the generic 8 characters and there was no fail2ban to stop guessing.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 16 child comments
    •
  • [+] 96 points 2 years ago* (last edited 1 year ago) (1 child)
    [deleted]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 53 points 2 years ago* (3 children)

    More importantly, don't open up SSH to public access. Use a VPN connection to the server. This is really easy to do with Netbird, Tailscale, etc. You should only ever be able to connect to SSH privately, never over the public net.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 32 points 2 years ago (1 child)

    It's perfectly safe to run SSH on port 22 towards the open Internet with public key authentication only.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 5 points 2 years ago (1 child)

    https://nvd.nist.gov/vuln/detail/cve-2024-6409 RCE as root without authentication via Open SSH. If they've got a connection, that's more than nothing and sometimes it's enough.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 23 points 2 years ago (2 children)

    That attack vector is exactly the same towards a VPN.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 2 years ago (1 child)

    A VPN like Wireguard can run over UDP on a random port which is nearly impossible to discover for an attacker. Unlike sshd, it won't even show up in a portscan.

    This was a specific design goal of Wireguard by the way (see "5.1 Silence is a virtue" here https://www.wireguard.com/papers/wireguard.pdf)

    It also acts as a catch-all for all your services, so instead of worrying about the security of all the different sshds or other services you may have exposed, you just have to keep your vpn up to date.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -3 points 2 years ago (1 child)

    Yeah I don't do security via obscurity :D I agree you need to keep your Internet facing services up to date.

    (No need to educate me on Wireguard, I use it. My day job is slightly relevant to the discussion)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 4 points 2 years ago (1 child)

    Yeah I don’t do security via obscurity

    Another one who misunderstands that phrase... Yes, obscurity shouldn't be your only line of defense, but limiting discoverability of your systems should be an integral part of your security strategy.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago

    There's no difference to the work I need to do to secure an open SSHd vs an open WireGuard server. None.

    Yes I harden, and penetrate, systems for a living. If your systems need remote access there is no standard (neither in fintech or military) that classifies SSHd as being "worse" than a VPN.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 2 years ago (1 child)

    Are you talking a VPN running on the same box as the service? UDP VPN would help as another mentioned, but doesn't really add isolation.

    If your vpn box is standalone, then getting root is bad but just step one. They have to own the VPN to be able to even do more recon then try SSH.

    Defense in depth. They didn't immediately get server root and application access in one step. Now they have to connect to a patched, cert only, etc SSH server. Just looking for it could trip into some honeypot. They had to find the VPN host as well which wasn't the same as the box they were targeting. That would shut down 99% of the automated/script kiddie shit finding the main service then scanning that IP.

    You can't argue that one step to own the system is more secure than two separate pieces of updated software on separate boxes.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 30 points 2 years ago (3 children)

    wow crazy that this was the default setup. It should really force you to either disable root or set a proper password (or warn you)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 12 points 2 years ago* (last edited 1 year ago) (1 child)

    Id consectetur dolore eiusmod culpa.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 9 points 2 years ago (3 children)

    Which ones? I'm asking because that isn't true for cent, rocky, arch.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 14 points 2 years ago (2 children)

    Mostly Ubuntu. And... I think it's just Ubuntu.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [+] 6 points 2 years ago* (last edited 1 year ago) (2 children)
    [deleted]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    Many cloud providers (the cheap ones in particular) will put patches on top of the base distro, so sometimes root always gets a password. Even for Ubuntu.

    There are ways around this, like proper cloud-init support, but not exactly beginner friendly.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago* (last edited 2 years ago) (1 child)

    Yeah I was confused about the comment chain. I was thinking terminal login vs ssh. You're right in my experience...root ssh requires user intervention for RHEL and friends and arch and debian.

    Side note: did you mean to say "shot themselves in the root"? I love it either way.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 8 points 2 years ago (1 child)

    Love Hetzner. You just give them your public key and they boot you into a rescue system from which you can install what you want how you want.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 10 points 2 years ago (1 child)

    I think their auction servers are a hidden gem. I mean the prices used to be better. Now they have some kind of systrem that resets them when they get too low. But the prices are still pretty good I think. But a year or two ago I got a pretty good deal on two decently spec'd servers.

    People are scared off by the fact you just get their rescue prompt on auctions boxes... Except their rescue prompt has a guided imaging setup tool to install pretty much every popular distro with configurable raid options etc.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 7 points 2 years ago (1 child)

    Yeah, I basically jump from auction system to auction system every other year or so and either get a cheaper or more powerful server or both.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 6 points 2 years ago

    I monitor for good deals. Because there's no contract it's easy to add one, move stuff over at your leisure and kill the old one off. It's the better way to do it for semi serious stuff.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 11 points 2 years ago* (1 child)

    because the password was the generic 8 characters and there was no fail2ban to stop guessing

    Oof yea that'll do it, your usually fine as long as you hardened enough to at least ward off the script kiddies. The people with actual real skill tend to go after...juicer targets lmao

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 6 points 2 years ago (2 children)

    I ran a standard raspian ssh server on my home network for several years, default user was removed and my own user was in it's place, root was configured as standard on a raspbian, my account had a complex but fairly short password, no specific keys set.

    I saw constant attacks but to my knowledge, it was never breached.

    I removed it when I realized that my ISP might take a dim view of running a server on their home client net that they didn't know about, especially since it showed up on Shodan...

    Don't do what I did, secure your systems properly!

    But it was kinda cool to be able to SSH from Thailand back home to Sweden and browse my NAS, it was super slow, but damn cool...

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 4 points 2 years ago (1 child)

    Why would a Swedish ISP care? I've run servers from home since I first connected up in ... 1996. I've had a lot of different ISPs during that time, although nowadays I always choose Bahnhof because of them fighting the good fights.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago*

    But it was kinda cool to be able to SSH from Thailand back home to Sweden and browse my NAS, it was super slow, but damn cool...

    That feels like sorcery, doesn't it? You can still do this WAY safer by using Wireguard or something a little easier like Tailscale. I use Tailscale myself to VPN to my NAS.

    I get a kick out of showing people my NextCloud Memories albums or Jellyfin videos from my phone and saying "This is talking to the box in my house right now! Isn't that cool!?" Hahaha.

    I'm almost glad I had to go that route. Most of our ISPs here in the U.S will block outgoing ports by default, so they can keep the network safe sell you a home business plan lol.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 5 points 2 years ago (1 child)

    Lol ssh has no reason to be port exposed in 99% of home server setups.

    VPNs are extremely easy, free, and wireguard is very performant with openvpn also fine for ssh. I have yet to see any usecase for simply port forwarding ssh in a home setup. Even a public git server can be tunneled through https.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago* (1 child)

    Yeah I'm honest with myself that I'm a security newb and don't know how to even know what I'm vulnerable to yet. So I didn't bother opening anything at all on my router. That sounded way too scary.

    Tailscale really is magic. I just use Cloudflare to forward a domain I own, and I can get to my services, my NextCloud, everything, from anywhere, and I'm reasonably confident I'm not exposing any doors to the innumerable botnet swarms.

    It might be a tiny bit inconvenient if I wanted to serve anything to anyone not in my Tailnet or already on my home LAN (like sending al someone a link to a NextCloud folder for instance.), but at this point, that's quite the edge case.

    I learned to set up NGINX proxy manager for a reverse proxy though, and that's pretty great! I still harden stuff where I can as I learn, even though I'm confident nobody's even seeing it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago* (1 child)

    Honestly, crowdsec with the nginx bouncer is all you need security-wise to start experimenting. It isn't perfect security, but it is way more comprehensive than fail2ban for just getting started and figuring more out later.

    Here is my traefik-based crowdsec docker composer:

    services:
  crowdsec:
    image: crowdsecurity/crowdsec:latest
    container_name: crowdsec
    environment:
      GID: $PGID
    volumes:
      - $USERDIR/dockerconfig/crowdsec/acquis.yaml:/etc/crowdsec/acquis.yaml
      - $USERDIR/data/Volumes/crowdsec:/var/lib/crowdsec/data/
      - $USERDIR/dockerconfig/crowdsec:/etc/crowdsec/
      - $DOCKERDIR/traefik2/traefik.log:/var/log/traefik/traefik.log:ro
    networks:
      - web
    restart: unless-stopped

  bouncer-traefik:
    image: docker.io/fbonalair/traefik-crowdsec-bouncer:latest
    container_name: bouncer-traefik
    environment:
      CROWDSEC_BOUNCER_API_KEY: $CROWDSEC_API
      CROWDSEC_AGENT_HOST: crowdsec:8080
    networks:
      - web # same network as traefik + crowdsec
    depends_on:
      - crowdsec
    restart: unless-stopped

networks:
  web:
    external: true

    https://github.com/imthenachoman/How-To-Secure-A-Linux-Server this is a more in-depth crash course for system-level security but hasn't been updated in a while.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •