Using Zephyr RTOS as a unikernel for improved application performance

Unikernels help increase the performance of applications by tightly coupling them with a heavily trimmed operating system, and compiling them into a single binary that runs directly on hardware or a hypervisor. Compared to traditional operating systems like Linux, unikernels don’t have a separate userspace and kernelspace. Instead, they share a single address space for both the system components and the user applications. Additionally, unikernels provide only the basic functionality required to run a given application, which means they can be smaller, require fewer resources (e.g. RAM or disk space) and offer better performance compared to traditional operating systems. For those reasons unikernels are employed in use cases requiring fast boot times, low resource consumption and high security, e.g. for deploying applications in the cloud, edge computing or IoT devices.

While the open source Zephyr RTOS is not typically used as a unikernel, it provides many features that facilitate using it as such, e.g. low resource usage, highly customizable builds, and POSIX standard support. As an active contributor to the project, Antmicro introduced several improvements that further increase Zephyr’s usability as a unikernel, including Virtio, virtiofs, and bootargs support.

In the following paragraphs we will describe the advantages of unikernels for application deployment and show how Zephyr can be utilized to streamline this process. We will also discuss the benchmarks of Zephyr as unikernel, focusing on the performance, and assess the security aspects.

Unikernels: overview

There are many different methods to deploy an operating system and your application(s) on your device. The diagram below demonstrates the three main ones, and their layering. Unikernels, as one such, contain fewer components; the kernel and the application are compiled to a single binary that is stripped of everything that may be deemed unnecessary.

Virtualization means that you deploy several isolated apps on a single host, so you need multiple separate Virtual Machines (VMs), each running its own guest OS. Each VM carries a full OS stack, leading to duplicate resource usage, longer boot times, and higher system overhead; VMs are full Operating Systems, like Linux, where the app is completely separated from the OS.

Containers, on the other hand, use no hypervisors and the apps are run as standalone (containerized), which can be compared to overlays, on top of the OS. They share the same kernel but maintain isolated user spaces. Compared to VMs, they are lighter, faster to start, and use less memory and disk. However, containers introduce additional attack surface and runtime dependencies.

Finally, unikernels are a single-purpose, single-address-space image. They include only the required parts of the OS and application logic - there are no shells, no package managers, no unused ports, and any other components which you may deem redundant for your use case. Compared to the previous two solutions, unikernels are lightweight, secure (smaller attack surface) and boot very fast.

Unikernels enable microVMs, which can be specialized and used to isolate critical system components (such as the steering controls in a car), for enhanced safety and security. This approach also makes it easy to isolate any unknown or untrusted components which you might wish to subject to thorough testing first. An existing market example of such a solution is Firecracker.

Zephyr as a unikernel: contributions

By design, Zephyr is dedicated for resource-constrained devices, and built with security in mind.

Antmicro’s previous contributions to Zephyr increased its usability as a unikernel by adding support for bootargs, Virtio and virtiofs.

• Virtio is a technology designed for efficient communication between the hypervisor and the VM, used to implement devices like network adapters, block devices or entropy sources.

• virtiofs enables access to specified directories in the host filesystem, allowing for the easy sharing of data with the unikernel and its persistence.

• bootargs is a functionality which allows passing arguments to the app running in Zephyr, just like passing arguments to an app on Linux (e.g. Linux’s linux_app arg1 arg2 becomes Zephyr’s qemu-system-x86_64 -kernel zephyr_app.elf -append "arg1 arg2").

There are more upcoming features which will further enhance Zephyr’s usability as unikernel, such as virtio-console, which provides a paravirtualized console device, reducing the overhead of virtualizing/simulating a real serial interface, and virtio-net, which, like virtio-console, uses the Virtio transport mechanism to implement a simpler paravirtualized NIC, or Network Interface Card.

Notably, Virtio is already in substantial use, with more community contributions being added, such as the support for the MMIO transport driver, and a Xen-MMIO backend. There is also virtio-entropy, which provides access to the entropy source, which, depending on the hypervisor configuration, might be used as a hardware random number generator.

Benchmarking Zephyr unikernel apps: performance and security

Using the latest Zephyr version, which included the (published) contributions mentioned above, we measured the performance of the Zephyr unikernel with a small C app, in terms of several aspects of its execution.

Zephyr used as a unikernel requires just a little bit more of disk space in comparison to the app alone, while providing ability to run in the VM. For example, the size of the C app built with Zephyr is bigger by only 50 KB than the native Linux app in comparison to at least a few MBs required by containers, and up to multiple GBs required by full Linux distributions.

Platform	Drive usage
Linux native app	16 KB
Linux VM (Debian)	1.6 GB
Docker (Debian)	117 MB
Docker (Alpine)	7.8 MB
Zephyr	70 KB
Zephyr (with userspace)	118 KB

In the case of RAM usage, the same C app requires just 0.3 MB of more RAM when it is running in a Zephyr unikernel.

Platform	RAM usage
Linux native app	1.28 MB
Linux VM (Debian)	78 MB
Docker (Debian)	5.6 MB
Docker (Alpine)	3 MB
Zephyr	1.59 MB
Zephyr (with userspace)	1.72 MB

While the startup time of the Linux native app is lower than the one using unikernel, the latter is noticeably faster than the full Linux VM or containerization, allowing to boot the VM with unikernel in a fraction of second, including hypervisor startup time.

Platform	Startup time
Linux native app (process)	0.0007 s
Linux VM (Debian)	10.12 s
Docker (Debian)	0.351 s
Docker (Alpine)	0.347 s
Zephyr	0.102 s
Zephyr (with userspace)	0.108 s

Because unikernels do not contain redundant processes or system services, more CPU time is available to the applications running in them. This is an advantage in CPU-intensive tasks. In a purely CPU-bound benchmark, unikernels were around 4% faster than Linux running in VM.

Platform	Execution time
Linux native app	0.862 s
Linux VM (Debian)	0.880 s
Docker (Debian)	0.862 s
Docker (Alpine)	0.861 s
Zephyr	0.850 s
Zephyr (with userspace)	0.850 s

As for the security features, Zephyr implements those known from full operating systems, including Write xor Execute (W^X) policy which prevents the execution of the sections that are designated as writable (or vice versa, prevents writing to the sections marked as executable), together with Hardware Stack Protection (enabled with CONFIG_HW_STACK_PROTECTION) and Stack Canaries (enabled with CONFIG_STACK_CANARIES) to prevent stack and buffer overflows when running in supervisor mode, as well as counter return address modifications - which is not always the case with other unikernels.

Additionally, Zephyr allows (optional) userspace and kernelspace isolation - a feature that is not typically sought after in unikernels, for performance reasons, but it allows further enhancing their security, by running the application core with lower permission level than kernel.

Using Zephyr as unikernel to isolate your essential apps

As a system engineer, you can use Zephyr-based unikernels for deploying applications in the cloud, as well as for testing applications for your edge computing applications, and for IoT devices - there is a broad scope of solutions which can be implemented this way. While you may already use VMs and containers in your systems, you may be looking for a lightweight, secure and fast-booting alternative, in which case a Zephyr-based unikernel is a potent option to consider.

If you would like to discuss using Zephyr as a unikernel for your use case, or testing your devices running Zephyr at scale, reach out to us at contact@antmicro.com - we will be happy to assist you.