Testing Zephyr software using new CMock/Unity module and Renode

Proper testing of embedded software is very difficult, but also crucial to successful product development. In Antmicro’s work, testing has always played a pivotal role - the open source Renode simulation framework that we maintain was initially created as an internal tool to improve testability and reliability of our work. With Renode, we promote thorough testing at every stage of a project, on all levels of complexity, including multi-node communication and simulation of external conditions.

While complex, end-to-end tests are always advisable and fit well with Renode’s capabilities, it’s also important to test software on a lower level with unit tests. Some of our customers use the combination of Unity and CMock for this. Unity is a library that allows you to write unit tests targeting embedded systems while CMock allows you to write mocks and stubs for parts of your code so that you can focus on what is important for a specific test.

As part of a customer project using Renode for end-to-end testing, Antmicro has recently developed generic, platform-agnostic integration between Zephyr, CMock and Unity for an even more streamlined testing workflow. In this note we will show you how to use the CMock/Unity integration in practice, and reproduce the tests in Renode.

Testing Zephyr applications with CMock and Unity

Zephyr includes a flexible dependency management and workflow automation meta-tool called west which allows you to easily integrate external modules. We used that capacity to develop the CMock/Unity integration.
Unity provides the ability to add assertions to your code as well as a set of macros and functions that simplify the process of writing unit tests. CMock extends those capabilities with complete mocking and stub generation support. CMock creates a mock version of each function to allow you to test the code in a controlled environment, track the exact code path, and compare it to the expected path to spot any anomalies.

CMock and Unity are often used alongside each other to skip the tedium typically associated with creating your own stubs to quickly achieve 100% branch and line testing for any C file in your project. Testing with CMock allows you to define expectations about the execution of your code, such as what arguments you expect to see and what you want them to return, which can be verified using Unity assertions.

Let’s imagine you want to test your GPIO-driven application and verify different code paths depending on the state of a button - quite a difficult scenario to achieve in an automated CI setup since you would need a machine physically pressing the button.

In this simple example, we set up a test to pretend that one GPIO input line is high and the other is low:

#include "drivers/mock_gpio.h"

void test_cmock(void) {
	const struct device *gpio_dev = device_get_binding("gpio0");

	/* Expect pin 0 to be high */
__wrap_gpio_pin_get_raw_ExpectAndReturn(gpio_dev, 0, 1);

	/* Expect pin 1 to be low */
__wrap_gpio_pin_get_raw_ExpectAndReturn(gpio_dev, 1, 0);

	/* Test if pin 0 is high */
	TEST_ASSERT_TRUE(gpio_pin_get_raw(gpio_dev, 0));

	/* Test if pin 1 is low */
	TEST_ASSERT_FALSE(gpio_pin_get_raw(gpio_dev, 1));
}

void main(void)
{
	(void)unity_main();
}

In the test above, we expect the device_get_binding("gpio0") function to return the value 0 when called with the argument 1 and return the value 1 when called with the argument 0. We can then test these expectations using Unity assertions like TEST_ASSERT_TRUE(gpio_pin_get_raw(gpio_dev, 0));.

After the tests are done, the tool will provide you with information about the results and the location of each executed test in your project, e.g.:

src/main.c:3:test_cmock:PASS

Platform-neutral testing tool

The CMock/Unity module is a generic tool not tied to any specific hardware and the west integration lets you comfortably work with your unit testing setup using the default Zephyr flow. Thanks to Renode’s west integration, you can also run whatever tests you create, even without actual hardware, and since Renode is just a piece of software, you can easily parallelize testing of your embedded software in the cloud on a large scale. Additionally, the deterministic nature of Renode can help you ensure consistency in test results, making it easier to identify and replicate issues in both single- and multi-node scenarios.

To test the integration of the module with Zephyr on a test project, simply clone it from the CMock/Unity module Github repository and follow the instructions from its README. Eventually, you will run the west build command and specify your target hardware:

west build -p -b hifive_unleashed tests/unity -t run
west build -p -b hifive_unleashed tests/cmock -t run

These commands will run sample tests for Unity and CMock in Renode on a virtual hifive-unleashed board, but you can use any of the Zephyr platforms supported in Renode, picking from hundreds of boards as showcased by the Renode Zephyr Dashboard.

After the tests are finished, you will see a report in your terminal summarizing the test results and a generated Zephyr ELF file, which you can load into the Renode simulation of your embedded target to run it again.

After you start the simulation, you will see the UART output from the attached binary as well as the time spent executing it:

[INFO] uart0: [host: 0.25s (+0.25s)|virt: 37µs (+37µs)] *** Booting Zephyr OS build zephyr-v3.2.0 ***
[INFO] uart0: [host: 0.25s (+6.44ms)|virt: 0.3ms (+0.26ms)] src/main.c:9:test_unity:[42mPASS[00m
[INFO] uart0: [host: 0.25s (+0.28ms)|virt: 	0.3ms (+0s)]
[INFO] uart0: [host: 0.25s (+0.38ms)|virt: 0.5ms (+0.2ms)] -----------------------
[INFO] uart0: [host: 0.25s (+0.41ms)|virt: 0.7ms (+0.2ms)] 1 Tests 0 Failures 0 Ignored
[INFO] uart0: [host: 0.25s (+0.18ms)|virt: 0.8ms (+0.1ms)] [42mOK[00m
[INFO] uart0: [host: 0.25s (+0.2ms)|virt: 	0.8ms (+0s)] PROJECT EXECUTION SUCCESSFUL

To extract more details from the run, e.g. a trace of executed functions, you can also run Renode manually from the module’s main directory and use one of the many tracing options described in Renode’s documentation.

To manually load a Zephyr binary to your simulation, specify it in the Renode Monitor before loading a .resc file of your project and start your simulation:

(monitor) $bin=@build/zephyr/zephyr.elf
(monitor) include @zephyr/boards/riscv/hifive_unleashed/support/hifive_unleashed.resc
(hifive-unleashed) start

Reproducible testing of embedded systems with Antmicro

Through integrations like the CMock/Unity module for Zephyr and the Renode simulation framework, Antmicro is continuously working towards providing a flexible open source environment for a variety of possible testing scenarios.

Antmicro offers comprehensive commercial support for your embedded and IoT projects at any stage of development, from pre-silicon simulation to in-field fleet management and automated configuration testing. If you are interested in using Zephyr and Renode for your next embedded project, reach out to Antmicro at contact@antmicro.com.