Are we ready for Embedded Kubernetes? — Round 1

Kubernetes and embedded systems don't look like an obvious pairing at first glance. Kubernetes grew up in datacenters, where compute, memory, and storage are typically abundant. Embedded systems tend to sit at the opposite end of that spectrum: they have to get the job done with constrained hardware, a fixed power budget, and components that sometimes stay unchanged in production for years. Still, the landscape has shifted in recent years. With lighter-weight Kubernetes distributions and purpose-built operating systems now available, it's become increasingly worth investigating whether Kubernetes is practically viable on significantly smaller systems too.

From my experience in the Cloud Native ecosystem, I've seen Kubernetes's complexity become steadily more manageable. We've used Kubernetes for edge computing for years, but moving down to smaller embedded systems raises different questions. How much memory does a Kubernetes installation actually need? How much does the choice of distribution affect resource usage? And at what point does a system become so constrained that it stops functioning reliably? Documentation alone doesn't always answer those questions. Published minimum requirements are a useful starting point, but they don't necessarily tell you what happens when you actually push the available hardware to its limit.

That's why I compared three combinations of operating system and Kubernetes distribution: Alpine Linux with KubeSolo, Alpine Linux with K3s, and Talos Linux. I tested all three on identical virtual hardware, ran the same workload, and then stepped the available memory down incrementally. The goal wasn't to crown a winner upfront, but to understand the practical differences between these approaches and establish where each configuration's limits actually lie.

Three approaches to a small Kubernetes footprint

Although all three candidates can run Kubernetes, they differ fundamentally in design philosophy. Alpine Linux with KubeSolo represents an approach that pairs a minimal Linux base with a Kubernetes distribution built for single-node use. Alpine was chosen here as a relatively minimal, general-purpose Linux distribution, which lets me examine KubeSolo's characteristics without also introducing the complexity of a custom embedded Linux build.

K3s is the second candidate. This distribution was built to make Kubernetes simpler and lighter to deploy, without fully abandoning the capabilities of a full-fledged Kubernetes environment. Pairing it with Alpine Linux keeps the OS base similar to the first candidate, which makes it easier to isolate differences in resource usage within the bounds of this test setup.

Talos Linux takes a different approach. Rather than installing Kubernetes on top of a general-purpose Linux distribution, Talos is an operating system purpose-built for Kubernetes. It's immutable and managed entirely through an API, with no traditional shell or SSH access. That makes Talos's focus a minimal, tightly controlled operational surface, but that doesn't automatically translate into the smallest memory footprint. It's precisely that distinction, between a minimal OS and a minimal Kubernetes installation, that makes the comparison interesting.

Test setup and measurement method

To keep the results as comparable as possible, I ran all three configurations on the same Proxmox cluster. The virtual machines shared the same configuration: two vCPUs, identical virtual hardware settings, and the same storage backend. The workload was identical too: every installation ran traefik/whoami, using the exact same container image digest. That keeps the key variables fixed, so differences in the results mainly trace back to the chosen software configuration.

The first round of measurements used 2 GB of allocated memory. Three minutes after a system reached Ready, I recorded its memory usage. From there, I stepped the available RAM down incrementally, testing at 1 GB, 768 MB, and 512 MB, among others. At every step, the system was rebooted from a clean state to check whether the configuration still worked. That wasn't just about whether Linux booted, but whether Kubernetes stayed available and the test application actually ran.

An important caveat: these tests ran on virtual x86-64 hardware, not on physical embedded hardware such as an ARM-based board. The results offer a controlled comparison between the tested software configurations, but they're not a direct prediction of minimum hardware requirements for a real embedded device. The lower bounds found here apply strictly to the configurations and workload tested.

The first results: substantial differences in memory usage

The first measurement shows that the choice of Kubernetes distribution, and the system architecture behind it, makes a noticeable difference in memory usage. With 2 GB of RAM allocated to each VM, measured three minutes after reaching Ready, KubeSolo came in at 382 MB, K3s at 591 MB, and Talos at 726 MB. Within this test setup, K3s used roughly one-and-a-half times as much memory as KubeSolo, while Talos used nearly twice as much.

These numbers are interesting, but they don't tell the full story. A snapshot of memory usage says little about how much memory a system actually needs to run reliably under all conditions. An installation can use relatively little RAM when given plenty of headroom, yet still run into trouble once memory gets tight. That's why I didn't just look at usage at 2 GB, but also examined how each system behaved as available resources kept shrinking.

The result of that second test is at least as interesting as the difference in the first measurement. KubeSolo kept working at 768 MB of RAM in this configuration, but failed at 512 MB. K3s stayed stable at 1 GB, but became unusable at 768 MB. Talos worked at 2 GB, but was no longer usable at 1 GB in this test. The lowest working memory configuration tested, in other words, varied considerably between the three candidates.

It's important to interpret these results correctly. They show at which memory configurations the tested installations did or didn't work in this specific setup. They don't prove that 768 MB is KubeSolo's universal minimum requirement, or that K3s always needs at least 1 GB under every condition. Different workloads, software versions, configurations, and hardware can produce different results. What the measurements do offer is a concrete starting point for further research, and they show that the difference between these solutions can matter in practice.

Why the lightest option isn't automatically the best one

Laying the results side by side, it's tempting to conclude that KubeSolo is the most attractive candidate for severely memory-constrained systems. That conclusion is only partially right. KubeSolo came out lightest in this test, but how much memory a system uses is just one aspect of its suitability for an embedded use case. Available functionality, the operational model, security, and the ability to manage the system remotely all play a role too.

The difference between K3s and Talos illustrates this well. K3s focuses on simplifying and shrinking Kubernetes itself. It bundles several components into a more compact distribution and defaults to SQLite as the datastore for a single-server install, instead of the usual etcd setup. Talos takes a different approach: it minimizes and locks down the OS layer, while Kubernetes itself keeps an architecture closer to the upstream components. These different design choices affect the number of processes, how the system is managed, and how many resources it needs.

On the other hand, Talos offers a practical advantage that doesn't show up in memory usage: there's no separate operating system to maintain. Because Talos is immutable and fully API-managed, traditional OS-level package management, patching, and configuration simply go away. That can contribute to a more stable baseline, with less risk of configuration drift, and potentially less operational overhead over time. That trade-off can put Talos's higher memory footprint in a different light: spend a bit more on base hardware in exchange for potentially lower operational costs over the device's lifetime.

That makes the term lightweight less clear-cut than it first appears. An operating system can carry very few management components and still use relatively more memory because it runs several separate Kubernetes processes. A distribution that shrinks Kubernetes itself can need less memory without putting the same emphasis on an immutable OS. Which approach fits best depends on what the end product actually requires. A device running a single application locally may call for different trade-offs than an industrial gateway that's managed remotely and needs to run multiple applications.

Documented minimum requirements versus practice

The measurements are also worth comparing against the minimum requirements the projects document themselves. KubeSolo's original documentation, for the configuration examined here, lists 512 MB of RAM as a target for ARM hardware. In our x86-64 test, the installation was no longer usable at that same memory level. That gap isn't by itself proof the documentation is wrong: the platform differs, and the conditions under which a minimum requirement is specified aren't necessarily the same as our test setup.

For K3s, the picture looks different. Although the documentation we examined lists 2 GB as the minimum, the tested installation kept working with 1 GB of RAM. That suggests a lower memory configuration is possible under certain conditions than the documentation assumes. Caution is warranted here too: a system that stays stable under a limited test workload isn't automatically fit for production with a larger application, extensive monitoring, or additional network components.

Talos, in this test, lines up most closely with its stated 2 GiB floor. The configuration kept working at 2 GB of allocated memory but was unusable at 1 GB. That makes the documented requirement a more realistic starting point here than for the other candidates. That doesn't mean the three sets of documentation were produced the same way, or that the stated figures are directly comparable: minimum requirements can refer to different configurations and use cases.

The broader lesson is that documentation is a useful starting point, but no substitute for testing with the actual workload and target hardware platform. This matters especially for embedded systems, where the memory configuration is often fixed upfront and can't easily be expanded later, so it's worth leaving enough headroom for load variation and future maintenance.

Compatibility deserves attention

Beyond resource usage, I also looked at a number of functional characteristics across the three configurations. The solutions don't offer the same capabilities on every front. Shell and SSH access availability differs, the number of default system pods varies, and not every candidate is designed for the same kind of cluster management. KubeSolo targets single-node use, while K3s and Talos can also be deployed as multi-node clusters.

In practice, though, it's unlikely you'd actually build a multi-node cluster on an embedded device: most embedded deployments run standalone, without multiple devices forming a single cluster together. For most real-world scenarios, this difference matters less than it looks on paper. It only becomes relevant if a product later ends up in a scenario where devices genuinely are managed together. Something similar applies to shell and SSH access: embedded devices generally aren't managed interactively, so this difference also carries less practical weight than it suggests on paper.

The comparison shows that resource efficiency and functional fit are two separate questions. A solution that uses little memory can be an excellent fit for a specific use case, yet still carry limitations that make it unacceptable for production. Conversely, a solution that uses more memory can be attractive precisely because of how updates, management, and security are handled.

CPU and storage: less decisive in this test

Although RAM was the main differentiator between the three candidates, I also looked at CPU usage and storage footprint. In a five-second idle measurement with two vCPUs allocated, KubeSolo came in at roughly 2.5% CPU usage, K3s at 1.9%, and Talos at 4.9%. Talos showed the highest measured idle usage, while K3s showed the lowest in this snapshot. These are limited measurements, though: they say nothing about peak load, boot-time behavior, or performance once applications are actively running.

Storage usage differed too. Measured writable storage came to roughly 925 MB for KubeSolo, 1.2 GB for K3s, and 1.26 GB for Talos, with Talos additionally reporting a separate OS image of around 80 MB. For interpretation, it matters to stick to the same measurement definition: an image's size, the storage used after installation, and the total storage a product needs aren't necessarily the same thing. Container images, logs, updates, and temporary files can push the required capacity higher still.

Based on these limited measurements, RAM appears to have been the main limiting factor in the configurations tested. That's a conclusion about this test, not about embedded Kubernetes in general. On a device with a severely constrained CPU, slow flash storage, or specific hardware acceleration, other factors can become far more important. Establishing how representative these results are for real embedded products will require additional tests on physical hardware.

What do these results actually tell us?

The main takeaway from this research isn't that one Kubernetes distribution is the best choice under every condition. The results mainly show that a solution's architecture affects how much memory it needs, and how an installation behaves once available resources run out. Within our test setup, KubeSolo had the smallest measured memory footprint and kept working at the lowest RAM level tested. K3s landed in between, while Talos needed more memory in this configuration.

That difference has to be weighed against each solution's characteristics. For a single-node device with a very tight memory budget, KubeSolo is an interesting candidate worth investigating further. K3s combines a more compact Kubernetes distribution with the option to scale to multi-node, a property that will matter in only a limited set of embedded scenarios, but one worth exploring when that scaling is actually on the table. Talos offers a different combination of properties, with an immutable OS and API-driven management as key design choices, which can offset its higher memory requirement. The right choice ultimately follows from the product's requirements, not from a single benchmark number.

The test also shows why it's important to distinguish between a system that can boot, a system that can run a trivial workload, and a system that stays reliably functional under realistic conditions. Those three situations still get lumped together in discussions about minimum hardware requirements. For an embedded application, the last one matters most, since the device typically has to keep running unattended for a long time after deployment and can't simply be handed more memory or compute later.

What's next

In the next phase, I want to extend this comparison to other solutions, including k0s, Kairos, and MicroK8s. That will build a broader picture of the different ways Kubernetes can be deployed in a constrained environment. The goal isn't to list as many distributions as possible, but to better understand which design choices affect resource usage, manageability, and the functionality that remains available.

A particularly interesting follow-up is pairing this with Yocto Linux. Instead of starting from an existing general-purpose distribution, that would let us investigate what happens when the operating system is purpose-built for a specific embedded use case. That makes it possible to isolate the Linux base's contribution, and get a clearer picture of how much of the total resource usage comes from the operating system, the Kubernetes distribution, and the applications running on top.

The central question stays the same: how small can Kubernetes get without losing the properties that make it valuable for a given use case? The answer almost certainly won't come down to a single universal number of megabytes. It will depend on the combination of hardware, operating system, distribution, workload, and operational requirements. This first comparison gives us a starting point for investigating those factors systematically.

All measurements were run directly on a Proxmox homelab cluster, October 2026. The workload was identical (traefik/whoami, same image digest) across all three candidates. Methodology and full raw results are available on request.