So I have been running a fair amount of selfhosted services over the last decade or so. I have always been running this on a Ubuntu LTS distribution running on a intel NUC machine. Most, if not all of my services run in a docker container, and using a docker compose file that brings everything up. The server is headless. I connect over ssh into a tmux config so I am always ready to go.
Ubuntu has been my stable server choice over the years. I’ve made the upgrade from 16, 18, 20 and 22 LTS release and everything has kept working. I even upgraded the hardware (old NUC to a new NUC) and just imaged the disk from the old one onto the new machine, and the server kept chugging along quite nicely, after I configured the hardware (specifically the Intel QuickSync for hardware transcoding in the Plex container).
Since Ubuntu has been transitioning from a really open community driven effort into a commercial enterprise, I feel it may be time to look at other distributions. On the other hand, it will require a fair amount of work to make the switch. But if it needs to be done, than so be it. I guess I am looking for opinions on what Linux distribution would fit my particular use case, and am wondering what most of us here are running.
TLDR; What stable, long term supported Linux distributions do you recommend for a headless server running a stack of docker containers?
You must log in or register to comment.
Does someone use nixOS? How does that hold up for this usecase?
I wouldn’t specifically say nixOS is stable in the same sense debian is but yes, it can totally handle this use case. I mainly run k8s on it, but a few home machines run docker (or, rather, podman) containers.
A thing about nixOS is that quite often you won’t need containers at all and would be better off without them, managing your apps as part of the system state as a whole. I only do that because I can’t be bothered to properly switch to nixOS services for ELK (which is supported by nixOS).
It’s a very stable solution in general and usually ends with a configuration that either doesn’t apply at all or applies with no issues. Gitops included for pretty much free. It requires understanding nix, and it can be tricky, but not overly tricky.
All and all I haven’t had an Ubuntu in homelab for two years now and can’t be happier about that.
Works well for me. Quick to set up as well if you are already familiar for nixos
Still a few Ubuntu Server stragglers here and there, but it works quite well as long as you keep your base config fairly lean and push the complexity into the containers.
Documentation tends to be either good or nonexistent depending on what you’re doing, so for anything beyond standard configuration but it can usually be pieced together from ArchWiki and the systemd docs.
All in all, powerful and repeatable (and a lot less tedious than Ansible, etc), but perhaps not super beginner-friendly once you start getting into the weeds. Ubuntu Server is just better documented and supported if you need something super quick and easy.
Proxmox with Debian containers.
Dude the guy is running on system with restricted resources and you’re suggesting the most bloated and prone to fail thing ever.
The OP would be way better with Debian + Cockpit (also provider a webUI and virtual machine manager) or Debian + LXD/LXC (containers + VMs, optional webUI). Both of those solutions are way lighter and won’t mess your base system.
It’s a NUC so sufficiently poweful. Proxmox isn’t fat by any means. If you run your stuff in containers then Proxmox (I aways install it on top of Debian) is your hypervisor is your base system. You typically don’t install stuff on your hypervisor, though I do some very select things.
If you run your stuff in containers then Proxmox (I aways install it on top of Debian) is your hypervisor is your base system
I believe you’re missing my point. Cockpit also works as an hypervisor for most people, a very light and stable one, besides…
If you’re running containers on Proxmox then you’re running LXC containers… why not just use LXD/Incus (also another hypervisor) to manage those containers that is considerably lighter, comes by default in the Debian repository, was designed to manage LXC container (not hacked around like Proxmox was) and isn’t mostly made by a for profit entity that sometimes likes to hold important patches on their subscription-only repositories? Or constantly nag you to buy a subscription?
I hear you, but Proxmox does a great many more things than just run containers. Admittedly, many selfhosters won’t need these.
LXD/Incus also does clustering, storage management, has a WebUI etc.
I just spun up Proxmox - it’s using 1GB of ram, and trivial CPU.
For a host, nothing else looked as easy to install and get running. The hardest thing so far was learning how to use ZFS raid0 on 3 drives. Doing so requires going commando (command line), because it’s not a safe thing to do, and really should never be done.
Once you get the concepts in Proxmox, it’s really straightforward.
Also, it’s backed by a commercial business, so it’s likely to see continued support and dev. With the recent shitshow with VMware, I suspect it’ll be a choice for many businesses - which hopefully translates to a growth in user base (and financial support).
Also, it’s backed by a commercial business, so it’s likely to see continued support and dev. With the recent shitshow with VMware, I suspect it’ll be a choice for many businesses - which hopefully translates to a growth in user base (and financial support).
That’s kind of the problem with Proxmox, you never know when they’ll start requiring a license for everyone and what prices will look like. That’s one of the reasons why I’m telling people to look at LXD/Incus. It’s truly free/open and financed by the linux containers initiative that gets resources from multiple big vendors and providers.
LXD/Incus. It’s truly free/open
Please stop saying this about lxd. You know it isn’t true, ever since they started requiring a CLA.
LXD is literally less free than proxmox, looking at those terms, since Canonical isn’t required to open source any custom lxd versions they host.
Also, I’ve literally brought this up to you before, and you acknowledged it. But you continue to spread this despite the fact that you should know better.
Anyway, Incus currently isn’t packaged in debian bookworm, only trixie.
The version of lxd debian packages is before the license change so that’s still free. But for people on other distros, it’s better to clarify that incus is the truly FOSS option.
Any small Linux distro would do. Just install Docker and maybe Portainer (as container itself of course) if you want a web UI.
Debian
Personally I prefer Rocky linux. Default to using SElinux which I also like. not a big fan of debian though as things are quite old and sometimes outdated for my needs.
I’m in the Rocky camp too. Before the IBM buyout I would have said CentOS.
Since you’re already familiar with a debian based distro, switching to the OG debian would be an option.
I do Debian on my server. I use openSUSE tumbleweed on the desktop but I could see leap being good for a server. YAST is pretty great .
Since you run everything in docker, I guess you have experienced the benefits of containerization. So why not leverage that for your host too?
Fedora IoT is a container-based host that runs on your hardware, with a focus on edge device deployment.
https://fedoraproject.org/iot/ I have it running on two servers as well, and it works great. The only thing I changed is that I layered docker on it instead of using podman, because at the time I had trouble getting my reverse proxy working properly over ipv6
I use ubuntu because my provider has that by default. It’s not my favourite distro these days, but gets the job done.
I also moved to Debian after years of using Ubuntu Server LTS. No problem so far on those servers for container workloads, the small ones run docker compose, the medium ones run k3s, and a small cluster runs rke2.
Unraid is the GOAT when it comes to being headless for docker. They know how to walk the fine line of “this is too much. It’s bloat,” and “this is minimal and bare bones.” I seriously can’t hype them enough, and can’t do justice to describing their ease of use.
debian. stable, reliable, easy to learn
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
Fewer Letters More Letters Git Popular version control system, primarily for code IoT Internet of Things for device controllers LTS Long Term Support software version LXC Linux Containers NAS Network-Attached Storage NUC Next Unit of Computing brand of Intel small computers SSH Secure Shell for remote terminal access VPS Virtual Private Server (opposed to shared hosting) ZFS Solaris/Linux filesystem focusing on data integrity k8s Kubernetes container management package
[Thread #601 for this sub, first seen 14th Mar 2024, 10:35] [FAQ] [Full list] [Contact] [Source code]
deleted by creator
My 4 host machines run debian (proxmox). I have a lot of different guest flavors running though, debian, fedora, rocky, one old guest still running Ubuntu and even a mint sandbox machine.
I probably have a bit more complicated self host than others because I am using it both for my useful internal services (jellyfin, git, pihole, etc.) I also run a whole lot of services for learning, such as kubernetes and dns. Plus a whole lot of other mostly useless stuff that I only use to test different architectures or automations that come in handy as an SRE.
If you’re using containers for everything anyways, the distro you use doesn’t much matter.
If Ubuntu works for you and switching away would mean significant effort, I see no reason to switch outside of curiosity.
