I recently migrated my homelab server from a bootc (bootable container) setup on Fedora to NixOS. The trigger wasn’t philosophical — it was a real breakage caused by the iptables/nft transition and Fedora’s kernel policy.

The bootc setup#

bootc is a model where your entire OS is defined as an OCI container image. You write a Containerfile, build the image, push it to a registry, and the server pulls and boots from it. It’s immutable infrastructure taken to its logical extreme — your OS has a CI/CD pipeline.

I genuinely liked the concept:

  • OS defined as a Containerfile
  • Atomic updates via image pull
  • Rollback by booting the previous image
  • Everything versioned and reproducible

But then the kernel broke things.

The iptables / nft problem#

To understand what happened, you need some context on Linux networking.

The history#

Linux used to use iptables for packet filtering and NAT. The original iptables is now called iptables-legacy. Its successor is nftables (nft) — a more modern, flexible framework.

The problem: you can’t rewrite every application that uses iptables to use nft overnight. So a compatibility layer was created: iptables-nft. It provides the iptables frontend (same CLI, same API) but uses nft as the backend. Legacy applications keep calling the iptables API, but under the hood they’re modifying nft rules.

Why it breaks#

This compatibility layer is not a long-term solution. nft keeps evolving, and new features in nft have no iptables counterpart. For the compatibility layer, these are effectively breaking changes — not breaking for nft itself, but for iptables-nft.

A concrete example: in kernel 7.1+, nft introduced a bitmask in NAT rules that allows attaching metadata to packets flowing through chains. Tailscale uses this to tag its own packets — so it can distinguish traffic generated by Tailscale from external traffic for proper routing.

This bitmask has no iptables equivalent. So if you run iptables-save, it doesn’t know how to interpret the rule and it breaks. Worse: if an application that uses iptables-nft tries to modify rules in a chain that contains nft-native features, it can corrupt the entire nft ruleset.

The real-world impact#

The rule is simple: either all your applications speak native nft, or you risk that the ones using iptables-nft will break — and in doing so, break the nft rules set by other applications.

In my case:

  • Tailscale uses nft natively
  • kube-router and kube-proxy use iptables as their frontend

These two worlds collided. kube-router couldn’t modify certain rules because they contained nft-native constructs it didn’t understand. My Kubernetes networking broke.

Why the server world stays conservative#

This is why the general direction in server environments is to stick to well-tested, older LTS kernel versions. This gives application developers time to build workarounds or migrate to native nft. Projects like kube-router and kube-proxy have acknowledged that moving to native nft would be a massive effort — it’s not happening overnight.

The Fedora problem#

Here’s where bootc + Fedora became a problem. Fedora is a rolling release when it comes to the kernel. Both Fedora 43 and 44 ship with kernel 7.x (such as kernel 7.2), whereas CNI plugins for Kubernetes — and server software in general — are not tested against these newer netfilter behaviors.

My setup broke because of this: kube-router on a kernel that introduced nft features incompatible with the iptables frontend.

The solution: Switching to NixOS with an LTS Kernel#

Since I was already running NixOS on my desktop and loving it, moving the server to NixOS was the logical answer.

Instead of dealing with Fedora pushing kernel 7+ releases that broke Kubernetes CNI plugins, NixOS allows me to explicitly pin the server to a stable LTS kernel like 6.18 (pkgs.linuxPackages_lts) with a single line of configuration:

boot.kernelPackages = pkgs.linuxPackages_lts;

This gave me the best of both worlds: a stable, conservative kernel environment where kube-router and iptables-nft work without surprises, combined with all the power of NixOS.

My NixOS configuration is in a single repo — github.com/MishowHD/mishow-nix — covering both desktop and server.

I’m running k3s on NixOS with an LTS kernel, and so far it’s been rock solid. The benefits are clear:

  • LTS kernel stability — no unexpected kernel updates breaking network rules
  • Single source of truth — the server config is just more Nix modules in the same repo
  • No configuration drift — if it’s not in the config, it’s not on the server
  • Atomic rollbacks — if an update breaks something, I boot the previous generation
  • Reproducible — I can rebuild the exact same server from the config alone

What I lost#

  • bootc’s elegance — the OCI model is genuinely elegant. Building your OS as a container image and pulling updates is a clean workflow
  • Fedora’s ecosystem — Fedora has great tooling and a strong community. The kernel policy is the only reason I left

Conclusion#

The move was driven by a real, painful breakage — not ideology. The iptables/nft transition is a messy period for Linux networking, and running bleeding-edge kernels on servers is risky if your stack includes anything that still speaks iptables.

NixOS gave me what bootc promised — declarative, reproducible, rollbackable — while letting me pin a conservative LTS kernel. For my homelab server, it’s been the right choice.

Note: All technical content, experiments, and opinions are 100% my own. AI tools are used solely for proofreading, editing, and translation assistance.