Are we ready for Embedded Kubernetes? — Round 2

In the first round of this research, I compared KubeSolo, K3s, and Talos on identical virtual hardware, stepping the available memory down until each installation eventually became unusable. KubeSolo came out lightest (382 MB), Talos heaviest but most predictable (726 MB), and K3s landed in between at 591 MB. In this second round, I extend that comparison to three candidates round 1 only mentioned in passing: k0s, MicroK8s, and Kairos (paired with K3s).

These three turned out to be less comparable in practice than expected, and one produced a failure pattern more severe than all three round-1 candidates combined. Yet those separate incidents have more in common than they first appear to: under the hood, the same architectural weakness shows up in four of the six configurations tested, spread across three different Kubernetes implementations.

Three new architectures

k0s follows the same philosophy as round 1's K3s: a compact, standalone Kubernetes distribution running on top of an existing operating system. It does split kubelet out as a separate process from the rest of the control plane, a middle ground between K3s's full bundling and Talos's full separation. MicroK8s takes the same general approach, but is tied inseparably to systemd through Canonical's snap packaging system, and therefore to an Ubuntu-like OS base. Kairos, finally, isn't a runtime-on-top-of-an-OS like the other two, but its own framework for immutable operating systems — conceptually closer to round 1's Talos. Later in this article, that comparison turns out to hold up only partially.

What does the bare operating system actually cost?

This round's setup deviates from round 1 in a couple of places. MicroK8s was deliberately tested on Ubuntu Server Minimal instead of Alpine: MicroK8s is distributed via snap, and snapd requires systemd, which Alpine doesn't have. Kairos also forced a change of plan — its official Alpine flavor turned out to no longer be actively built by the time I ran this research. Current releases only ship Hadron, Kairos's own Linux distribution.

Because the OS base differs between candidates, this round effectively compares complete combinations of operating system and Kubernetes runtime, not bare runtimes in isolation. That's why, for the first time, I also measured the bare operating systems themselves, with no Kubernetes components, three minutes after a clean reboot. That baseline was missing from round 1, but turns out to matter for interpreting the differences between candidates correctly. The Alpine, Ubuntu, and Hadron measurements below are therefore core to this round, not a side note.

Two things stand out. First, Ubuntu's "minimal" cloud image comes in at 266 MB, nearly three times as heavy as bare Alpine, before MicroK8s has even started. That matters for a project that calls itself "micro" but is required to run on this OS base. Second, Hadron, Kairos's own distribution built up from scratch, lands at a strikingly close 123 MB to Alpine, despite both Hadron and Ubuntu running a full systemd stack. The fact that systemd itself isn't what explains Ubuntu's weight comes back later in the article.

Memory usage at 2GB: the complete solution

With those OS baselines in mind, here's the total memory usage for each candidate: operating system, Kubernetes runtime, and default add-ons combined, measured with the exact same method as round 1 (three minutes after Ready, 2 GB of RAM allocated).

At 635 MB, k0s is the lightest of the three, a bit heavier than round 1's K3s (591 MB), which makes sense given that kubelet runs as a separate process here. MicroK8s is the heaviest at 828 MB, and most of that is explainable: 266 MB is already the Ubuntu baseline, and another 338 MB goes to Calico, the full BGP-based CNI that MicroK8s ships by default instead of the lighter kube-router or flannel used by the other candidates. Kairos+K3s on Hadron lands in between at 772 MB, heavier than that same K3s version would be on Alpine (more on that later).

What the documentation promises, and what maintainers should watch for

k0s officially lists 1 GB of RAM as the minimum for a combined controller+worker node, the lowest official figure of all six candidates so far. That number deserves more context than a simple comparison against my own measurements, because what's actually behind it is more nuanced than just "an optimistic estimate."

The official requirements table states that 1 GB with no architecture caveat at all: the same figure for x86_64, aarch64, armv7l, and riscv64. But the same page carries a footnote the table itself doesn't reflect: "The operating system and application requirements must be considered in addition to the k0s part." So that 1 GB is for k0s alone, not for the OS and k0s combined. The only concrete measurement the project itself publishes was taken on an AWS t3.xlarge with 16 GB of RAM and Ubuntu Server, where they note themselves that the OS used roughly 180 MB extra, on a machine where memory pressure never came into play.

I tested at 1 GB total, exactly the amount k0s's own documentation marks as "for k0s alone." Even adding my own, much lighter Alpine baseline (96 MB) to their own 510 MB figure still lands comfortably under the 1 GB mark where memory pressure would start to matter. That's not a knock against the k0s project (the footnote is right there, properly documented), but it is worth flagging: the headline table and that footnote contradict each other in practice, and the figure has never been validated under real memory pressure. For exactly the audience that relies on numbers like this, embedded/edge users with a fixed, tight memory budget, a clearer combined figure (OS + k0s, tested under pressure) would be worth a lot.

Four ways Kubernetes fails under memory pressure

And that last point isn't an exaggeration. At 1 GB, k0s showed the most severe failure pattern of all six candidates tested so far, and MicroK8s added a new, previously unseen failure pattern at the same level. Combined with Talos's kubelet memory-pressure eviction from round 1, I've now seen four fundamentally different ways a Kubernetes installation can go down under memory pressure.

To check whether the back-of-envelope math from the previous section holds up in practice (k0s's own 510 MB plus my Alpine baseline of 96 MB, comfortably under the 1 GB line), I tried one extra, non-standard level in between: 1.2 GB. That stayed fully stable for twenty minutes (load average between 0.00 and 0.24), versus a load average of almost 19 within four minutes at 1 GB. It's not a standardized rung on the ladder like the other levels in this research, but it is a useful spot check: somewhere between 1 GB and 1.2 GB sits a hard cliff, not a gradual transition. The footnote-based math turns out to roughly check out.

kine: the shared weak point across three Kubernetes implementations

What KubeSolo, K3s, and k0s have in common is kine: a shim that mimics etcd's own gRPC API, but actually stores the data in SQLite instead of a real Raft-based distributed store. Those are three different Kubernetes implementations. Because Kairos uses K3s as its runtime in this round, the kine cascade shows up in four tested configurations in total (KubeSolo, K3s on Alpine, k0s, and Kairos+K3s), but still only three underlying implementations. kine is therefore a common factor across those three. It helps explain why these configurations are built differently from Talos, but under heavy memory pressure it also turns out to be their shared weak point. As memory pressure rises, SQLite queries that normally take milliseconds stretch out to 7-33 seconds. That triggers a chain reaction of timeouts and retries throughout the control plane.

An important nuance: kine itself doesn't cost much memory. It isn't why the footprint is large, it's why the floor collapses once you go looking for it. The likely cause is architectural, not SQLite itself: in all three implementations, kine runs in the same process as apiserver and scheduler, with no resource isolation of its own. When that process has to lean harder on the garbage collector under memory pressure, kine takes the side effect. MicroK8s's k8s-dqlite (a genuinely distributed SQLite with Raft consensus, not a simple translation shim) therefore appears to fail in a different, somewhat more robust category: dqlite itself never crashes, only the apiserver/scheduler bundle around it (kubelite) does.

Kairos+K3s on Hadron confirms that this mechanism is independent of the OS base: at 1 GB, this combination followed a functionally identical trajectory to k0s's collapse — load climbing from 11 to 15 within eight minutes, eventually losing SSH entirely, memory staying within budget the whole time. Same kine layer, same cascade, regardless of whether the underlying OS is called Alpine or Hadron, and regardless of the Kairos layer in between.

What does Kairos's own layer cost, separate from Hadron and K3s?

Because Kairos's Alpine flavor fell away, I could no longer isolate Kairos from its OS base the way I'd planned. As an alternative, I measured the three layers separately: bare Hadron, Hadron with the Kairos agent added (no Kubernetes), and then with K3s on top.

The answer turned out simpler than expected: the Kairos agent costs next to nothing in steady-state use. systemctl list-units shows kairos-agent.service as inactive/dead. The agent applies the cloud-config and A/B partition verification once at boot, and doesn't stick around as a background service afterward. The full 123 MB of the bare "core" image is therefore almost pure Hadron itself. The immutable layer that conceptually sets Kairos apart from a regular Linux install is, in RAM terms, nearly free — the overhead sits in boot time, not in memory usage.

K3s itself did cost noticeably more on Hadron than that same version did on Alpine in round 1: 649 MB on top of the Hadron baseline, versus 495 MB on top of Alpine's baseline in round 1. That's a difference of roughly 150 MB that the OS baseline alone doesn't explain. Possible explanations include a different containerd configuration or systemd's cgroup driver instead of Alpine's cgroupfs; I didn't chase this further, given its limited relevance to this research's main question.

Kairos and Talos: rivals in name, not in architecture

Kairos and Talos are often mentioned in the same breath: both are immutable, both target Kubernetes, both avoid traditional package management. That resemblance doesn't hold up well under closer inspection. Talos is a single-purpose operating system, built specifically around Kubernetes itself: no general-purpose userland, no shell, no SSH, manageable only through its own API. It has no identity apart from Kubernetes. Irreverently put, it's Kubernetes that happened to need an operating system.

Kairos is the opposite: a generic framework for immutable operating systems (A/B upgrades, cloud-config bootstrapping, atomic updates) that has no opinion of its own about what runs on top. It can run K3s or k0s, or nothing at all (the "core" image used in this article), on top of, in principle, multiple underlying Linux distributions. That puts Kairos conceptually closer to Fedora CoreOS or Flatcar Container Linux with an optional Kubernetes layer than to Talos's single-purpose design. Both can end up serving the same use case, an immutable node for a Kubernetes cluster, but the path there, and how much choice remains along the way, differ fundamentally.

The final verdict comes in the next episode

This round confirms something round 1 already hinted at, but more sharply this time: memory usage alone doesn't tell the whole story. A low footprint says nothing about how a system fails once you push it to the memory limit, and that turns out to differ substantially by architecture, independent of the raw numbers. k0s's lowest official minimum requirement came paired with the most severe failure behavior of all six candidates; MicroK8s's heaviest footprint came paired with the most "civilized" failure pattern, precisely because of a more architecturally robust datastore choice. The key results, side by side:

This was also the final experimental test round in this series. With six candidates on the table, KubeSolo, K3s, and Talos from round 1, and k0s, MicroK8s, and Kairos (with K3s) here, there's enough material for a well-grounded conclusion. That kine/SQLite sets the practical floor for at least half the candidates is one of the most important findings to carry forward.

The next and final episode in this series will bring my verdict: concrete advice on which combination to prefer, and when, based on everything these two test rounds have produced.

All measurements were run directly on a Proxmox homelab cluster, October 2026. k0s and MicroK8s were re-run with a fixed serviceCIDR and, for MicroK8s, a corrected disk partition after a provisioning error. Methodology and full raw results are available on request.