Testing complex, heterogeneous AMP systems with Renode using Zynq UltraScale+

Asymmetric multiprocessing (AMP) setups are very common in modern SoCs which mix various types of cores or even architectures to provide sufficient processing power when needed, while keeping the system energy efficient overall. The AMP architecture is especially useful in applications requiring both time-critical and user-facing, high-performance processing, such as automotive.

The popular Zynq UltraScale+ MPSoC is a good example of such an AMP system, offering two or four Cortex-A53 cores (here called APU) for running Linux with non-critical software, and two Cortex-R5 cores (here called RPU) for real-time processing e.g. with Zephyr RTOS, on top of an FPGA for custom user logic. Antmicro has been using the UltraScale+ platform in a multitude of projects across diverse verticals including computer vision, data center or mining, and our Renode simulation framework has supported it for quite a while now.

In this article, we will present how to test complex, heterogeneous AMP systems like this with Renode. We will use a Linux+Zephyr setup first introduced in an article some years ago, and reproduce it in simulation; the demo setup described here now ships with mainline Renode. We’ll also show how you can test two-way communication between the APU and RPU using the OpenAMP demo in which the APU sends data and receives a response from the RPU accordingly.

Heterogeneous setups for safety-critical applications

In use cases like automotive, a typical vehicle may run a version of Linux to realize all the user facing control, information and entertainment functions, and use an RTOS running independently of the main system to manage the safety-critical features, such as the braking system. Likewise, in a typical space application, Linux might be used to provide high level mission control and interface complex sensors like cameras while e.g. Zephyr can handle crucial flight controls and communications as is the case with our recently launched space AI compute platform.

Installing Renode and running a heterogeneous multi-core demo

In this guide, we illustrate how to install Renode locally and run a Zephyr RTOS example demo on a simulated Cortex-R5 RPU, triggered by a Linux system image running on the Cortex-A53 APU. Note that this demo is now included in mainline Renode for simplicity.

To begin, install Renode in one of the many possible methods. For example, you can use the portable package for Linux:

mkdir renode_portable
wget https://builds.renode.io/renode-latest.linux-portable.tar.gz
tar xf  renode-latest.linux-portable.tar.gz -C renode_portable --strip-components=1

Once Renode is installed, simply load the mainline .resc file containing the scenario for the Zynq UltraScale+ MPSoC with remoteproc simulation:

./renode scripts/single-node/zynqmp_remoteproc.resc

This will show the Renode Monitor, load the demo and open two UART windows in which the Linux and Zephyr output will be seen. In order to start the simulation, use the start command. When prompted, enter the root password root and wait for Linux to reach the command line.

To launch Zephyr and the included Dining Philosophers demo after loading Linux, just execute the following commands in order in the simulated Linux console:

modprobe zynqmp_r5_remoteproc
mkdir /lib/firmware
cp /elfs/philosophers.elf /lib/firmware
echo philosophers.elf > /sys/class/remoteproc/remoteproc0/firmware
echo start > /sys/class/remoteproc/remoteproc0/state

The output from the Zephyr RTOS running on the simulated Cortex-R5 RPU will be displayed in UART0.

To stop the Dining Philosophers demo, in the simulated Linux command line use:

echo stop > /sys/class/remoteproc/remoteproc0/state

Alternatively, you can run a Renode test which will automatically log in to Linux and run the Dining Philosophers demo with a single command:

./renode-test ./tests/platforms/zynqmp.robot -f "Should Start And Stop Remoteproc" --enable-xwt 

Running an AMP system in Renode with two-way communication

In the demo described above, communication flows in one direction only, i.e. from Linux to Zephyr. More advanced use cases require two-way communication between the APU and RPU, and to illustrate this, we have also included an OpenAMP Echo demo, which can be run from the Linux command line by using:

./renode ./scripts/single-node/zynqmp_openamp.resc

Followed by:

modprobe zynqmp_r5_remoteproc
echo rpmsg-echo.out > /sys/class/remoteproc/remoteproc0/firmware
echo start > /sys/class/remoteproc/remoteproc0/state
./echo_test

Or alternatively, you can launch the OpenAMP Echo script with:

./renode-test ./tests/platforms/zynqmp.robot -f "Should Run OpenAMP Echo Sample" --enable-xwt 

To stop and restart the script, use the following:

echo stop > /sys/class/remoteproc/remoteproc0/state
echo start > /sys/class/remoteproc/remoteproc0/state

The demo is similar to the Dining Philosophers demo in that a program is started on the Cortex-R5 using remoteproc, but in this example, both the APU and RPU are actively communicating with each other. Here, arrays of constants with value 0xA5 are sent by the APU, and the array size increases with each iteration up to 488 bytes. In response, the RPU sends the same data back to the APU, i.e. implements an “echo” functionality. The output of the demo is shown below:

In Renode, the peripherals or cores running inside the FPGA onboard the Zynq UltraScale+ can also be simulated. On top of using your RTL directly through a DPI-enabled simulator, you can use SystemC models, or write your functional models in C# and Python.

Advanced simulation testing with Renode

The ability to run heterogeneous multi-core setups - including co-simulated FPGA payloads - in a single simulation is very useful for the development and testing of complex platforms based on hardware like the Zynq UltraScale+ widely used in industrial, automotive and space devices. Moving beyond that to multi-node simulations, such as those including supplementary MCUs for controlling or providing data to the main Linux device, is also possible, and thus, as you develop and test your product, Renode can assist you throughout its entire lifecycle.

Contact us at contact@antmicro.com if you would like to explore how Renode can help improve collaborative development processes as well as test existing heterogeneous AMP systems featuring multiple cores from a range of architectures.