67
From Docker with Ansible to k3s: I don't get it...
(sh.itjust.works)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 09 Jul 2025
67 points (98.6% liked)
Selfhosted
49326 readers
485 users here now
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
founded 2 years ago
MODERATORS
Hey there,
I made a similar journey a few years ago. But I only have one home server and do not run my services in high availability (HA). As @[email protected] mentioned, to run a service in HA, you need more than "just scaling up". You need to exactly know what talks when to whom. For example, database entries or file writes will be difficult when scaling up a service not ready for HA.
Here are my solutions for your challenges:
kubectl apply -ffor each file. I would strongly recommend helm. Then you just have to runhelm installper service. If you want to write each service by yourself, you will end up with multiple.yamlfiles. I do it this way. Normally, you create one repository per service, which holds all YAML files. Alternatively, you could use a predefined Helm Chart and just customize the settings. This is comparable to DockerHub..yamlconfiguration multiple replicas are defined, k8s will automatically balance these replicas on multiple servers and split the entire load on all servers in the same cluster. If you just look for configuration examples, look into Helm Charts. Often service provide examples only for Docker (and Docker Compose) and not for K8s.Changelog:
Never, ever install anything this way. The trend of "just run this shell script off the internet" is a menace. You don't know what that script does, what repositories it may add, what it may install, whether somebody is typo-squatting the URL and you're running something else, etc.
It's just a bad idea. If you disagree then I have one question - how would you uninstall k3s after you ran that blackbox?
Yes, just running a random script from the internet is a very bad idea. You should also not copy and paste the command from above, since I'm only a random lemmy user. Nevertheless, if you trust k3s, and they promote this command on the official website (make sure it's the official one) you can use it. As you want to install k3s, I'm going to assume you trust k3s.
If you want to review the script, go for it. And you should, I agree. I for myself reviewed (or at least looked over it) when I used the script for myself.
For the uninstallment: just follow the instructions on the official website and run
/usr/local/bin/k3s-uninstall.shsourceI really want to push back on the entire idea that it's okay to distribute software via a
curl | shcommand. It's a bad practice. I shouldn't be reading 100's of lines of shell script to see what sort of malarkey your installer is going to do to my system. This application creates an uninstall script. Neat. Many don't.Of the myriad ways to distribute Linux software (deb, rpm, snap, flatpak, AppImage) an unstructured shell script is by far the worst.
I think that distributing general software via
curl | shis pretty bad for all the reasons that curl sh is bad and frustrating.But I do make an exception for "platforms" and package managers. The question I ask myself is: "Does this software enable me to install more software from a variety of programming languages?"
If the answer to that question is yes, which is is for k3s, then I think it's an acceptable exception.
curl | shis okay for bootstrapping things like Nix on non Nix systems, because then you get a package manager to install various versions of tools that would normally try to get you to install themselves withcurl | bashbut then you can use Nix instead.K3s is pretty similar, because Kubernetes is a whole platform, with it's own package manager (helm), and applications you can install. It's especially difficult to get the latest versions of Kubernetes on stable release distros, as they don't package it at all, so getting it from the developers is kinda the only way to get it installed.
Relevant discussion on another thread: https://programming.dev/post/33626778/18025432
One of my frustrations that I express in the linked discussion is that it's "developers" who are making bash scripts to install. But k3s is not just developers, it's made by Suse who has their own distro, OpenSuse, using OpenSuse tooling. It's "packagers" making k3s and it's install script, and that's another reason why I find it more acceptable.
Microk8s manages to install with a snap. I know that snap is "of the devil" around these parts but it's still better than a custom bash script.
Custom bash scripts will always be worse than any alternative.
I've tried snap, juju, and Canonical's suite. They were uniquely frustrating and I'm not interested in interacting with them again.
The future of installing system components like k3s on generic distros is probably systemd sysexts, which are extension images that can be overlayed onto a base system. It's designed for immutable distros, but it can be used on any standard enough distro.
There is a k3s sysext, but it's still in the "bakery". Plus sysext isn't in stable release distros anyways.
Until it's out and stable, I'll stick to the one time bash script to install Suse k3s.
You're welcome to make whatever bad decisions you like. I can manage snaps with standard tooling. I can install, update, remove them with simple ansible scripts in a standard way.
Bash installers are bad. End of.
Canonical's snap use a proprietary backend, and comes at a risk of vendor lock in to their ecosystem.
The bash installer is fully open source.
You can make the bad decision of locking yourself into a closed ecosystem, but many sensible people recognize that snap is "of the devil" for a good reason.
You've made this about snap. Flatpak, rpm, deb, etc. all work too.
Except k3s does not provide a deb, a flatpak, or a rpm.
Ah - you have discovered my complaint.
That is precisely why I went with microk8s instead. I don't install software from people who can't be bothered to package their software using standard deployment tools which has been the correct way to distribute Linux software for decades.
So instead you decided to go with Canonical's snap and it's proprietary backend, a non standard deployment tool that was forced on the community.
Do you avoid all containers because they weren't the standard way of deploying software for "decades" as well? (I know people that actually do do that though). And many of my issues about developers and vendoring, which I have mentioned in the other thread I linked earlier, apply to containers as well.
In fact, they also apply to snap as well, or even custom packages distributed by the developer. Arch packages are little more than shell scripts, Deb packages have pre/post hooks which run arbitrary bash or python code, rpm is similar. These "hooks" are almost always used for things like installing. It's hypocritical to be against
curl | bashbut be for solutions like any form of packages distributed by the developers themselves, because all of the issues and problems withcurl | bashapply to any form of non-distro distributed packages — including snaps.You are are willing to criticize bash for not immediately knowing what it does to your machine, and I recognize those problems, but guess what snap is doing under the hood to install software: A bash script. Did you read that bash script before installing the microk8s snap? Did you read the 10s of others in the repo's used for doing tertiary tasks that the snap installer also calls?
The bash script used for installation doesn't seem to be sandboxed, either, and it runs as root. I struggle to see any difference between this and a generic bash script used to install software.
Although, almost all package managers have commonly used pre/during/post install hooks, except for Nix/Guix, so it's not really a valid criticism to put say, Deb on a pedestal, while dogging on other package managers for using arbitrary bash (also python gets used) hooks.
But back on topic, in addition to this, you can't even verify that the bash script in the repo is the one you're getting. Because the snap backend is proprietary. Snap is literally a bash installer, but worse in every way.
Dude - you gotta get off the snap hate train for a bit.
Do you not understand the difference between "hey, run this rando shell script on the internet" and "hey, use this standardized installer which may run some shell scripts"?
I don't give a shit about all the canonical hate. For me snap does what I want:
flatpak run something.something.somethingBS)It's not bash I'm criticizing. Do you understand that? Because stop reading if you don't and go back through my list. I'll wait.
So good - you get that bash isn't the problem. It's the bespoke unstructured installer/upgrader/unisntaller part that is bad. You could write your installer in C, Python, etc. and I'll levy the same complaints. You want me to install your python app? It should be available through pypi and pip. Not some rando bespoke installer.