I recently migrated my homelab server from a bootc (bootable container) setup on Fedora to NixOS. The trigger wasn’t philosophical — it was a real breakage caused by the iptables/nft transition and Fedora’s kernel policy.

The bootc setup#

bootc is a model where your entire OS is defined as an OCI container image. You write a Containerfile, build the image, push it to a registry, and the server pulls and boots from it. It’s immutable infrastructure taken to its logical extreme — your OS has a CI/CD pipeline.

I genuinely liked the concept:

  • OS defined as a Containerfile
  • Atomic updates via image pull
  • Rollback by booting the previous image
  • Everything versioned and reproducible

But then the kernel broke things.

The iptables / nft problem#

To understand what happened, you need some context on Linux networking.

The history#

Linux used to use iptables for packet filtering and NAT. The original iptables is now called iptables-legacy. Its successor is nftables (nft) — a more modern, flexible framework.

The problem: you can’t rewrite every application that uses iptables to use nft overnight. So a compatibility layer was created: iptables-nft. It provides the iptables frontend (same CLI, same API) but uses nft as the backend. Legacy applications keep calling the iptables API, but under the hood they’re modifying nft rules.

Why it breaks#

This compatibility layer is not a long-term solution. nft keeps evolving, and new features in nft have no iptables counterpart. For the compatibility layer, these are effectively breaking changes — not breaking for nft itself, but for iptables-nft.

A concrete example: when modern services like Tailscale manage network rules, they write directly to nftables using native features — such as complex sets, dynamic maps, or custom bitmasks on packet marks to isolate and route Tailscale traffic.

These native nft constructs have no direct equivalent in the legacy iptables data model. When a legacy tool relying on iptables-nft (or iptables-save) inspects or modifies network tables, it cannot parse the native nft expressions present in those chains. As a result, iptables-nft throws parsing errors or fails to apply updates. Worse: attempting to modify or flush a shared chain containing native nft rules can break network routing or corrupt rule synchronization.

The real-world impact#

The rule is simple: either all your applications speak native nft, or you risk that the ones using iptables-nft will break — and in doing so, break the nft rules set by other applications.

In my case:

  • Tailscale uses nft natively
  • kube-router and kube-proxy use iptables as their frontend

These two worlds collided. kube-router couldn’t modify certain rules because they contained nft-native constructs it didn’t understand. My Kubernetes networking broke.

Why the server world stays conservative#

This is why the general direction in server environments is to stick to well-tested, older LTS kernel versions. This gives application developers time to build workarounds or migrate to native nft. Projects like kube-router and kube-proxy have acknowledged that moving to native nft would be a massive effort — it’s not happening overnight.

The Fedora problem#

Here’s where bootc + Fedora became a problem. Fedora updates its kernel very aggressively. With rapid, bleeding-edge kernel updates, version mismatches between kernel netfilter enhancements and userspace iptables-nft tools surface quickly — long before Kubernetes CNI plugins (and server software in general) are updated or tested against them.

My setup broke because of this: kube-router relied on iptables-nft to manage cluster networking, but collided with the native nftables ruleset created by Tailscale and netfilter updates on bleeding-edge kernels, causing rule synchronization to fail.

The solution: Switching to NixOS with an LTS Kernel#

Since I was already running NixOS on my desktop and loving it, moving the server to NixOS was the logical answer.

Instead of dealing with Fedora pushing bleeding-edge kernel updates that risk breaking Kubernetes CNI plugins due to netfilter mismatches, NixOS allows me to explicitly pin the server to a stable LTS kernel like 6.18 (pkgs.linuxPackages_lts) with a single line of configuration:

boot.kernelPackages = pkgs.linuxPackages_lts;

This gave me the best of both worlds: a stable, conservative kernel environment where kube-router and iptables-nft work without surprises, combined with all the power of NixOS.

My NixOS configuration is in a single repo — github.com/MishowHD/mishow-nix — covering both desktop and server.

I’m running k3s on NixOS with an LTS kernel, and so far it’s been rock solid. The benefits are clear:

  • LTS kernel stability — no unexpected kernel updates breaking network rules
  • Single source of truth — the server config is just more Nix modules in the same repo
  • No configuration drift — if it’s not in the config, it’s not on the server
  • Atomic rollbacks — if an update breaks something, I boot the previous generation
  • Reproducible — I can rebuild the exact same server from the config alone

What I lost#

  • bootc’s elegance — the OCI model is genuinely elegant. Building your OS as a container image and pulling updates is a clean workflow
  • Fedora’s ecosystem — Fedora has great tooling and a strong community. The kernel policy is the only reason I left

Conclusion#

The move was driven by a real, painful breakage — not ideology. The iptables/nft transition is a messy period for Linux networking, and running bleeding-edge kernels on servers is risky if your stack includes anything that still speaks iptables.

NixOS gave me what bootc promised — declarative, reproducible, rollbackable — while letting me pin a conservative LTS kernel. For my homelab server, it’s been the right choice.

Note: All technical content, experiments, and opinions are 100% my own. AI tools are used solely for proofreading, editing, and translation assistance.