Renode GitHub Action for automated testing on simulated hardware

Testing in simulation is the go-to strategy to enable efficient vertical integration based on parallel development of software and hardware starting at the initial phases of a project, no matter the complexity of the system under development. Renode, Antmicro’s open source simulation framework, lets you capitalize on this capability by providing a deterministic development environment that offers a constantly expanding portfolio of integrations for designing advanced test suites, co-simulation capabilities, as well as availability of simulation models for thousands of hardware building blocks.

Automation of testing workflows in Continuous Integration systems is an integral part of all customer and internal projects Antmicro is involved in, as it plays a fundamental role in terms of reliability and project turnaround time. Examples of this CI-centric approach for code hosted on public repositories include an automated testing and verification setup which we introduced for the RISC-V VeeR core family as used within the Caliptra Root of Trust project (led by AMD, Google, Microsoft and NVIDIA) which relies on the Verible Linter GitHub Action we created. Antmicro has also developed a GitHub Action capable of running a full-fledged Linux distribution with a custom kernel for testing v4l2 setups simulated in Renode. Additionally we often develop infrastructural solutions like our custom GitHub runners for CI setups in large-scale projects that exceed the compute and storage capacity of standard runners.

In order to simplify the integration of the testing capabilities Renode offers for each iteration of code pushed to GitHub repositories, we have introduced a dedicated GitHub Action for Renode which lets you easily integrate the simulator into CI pipelines, specify tests and run them on simulated hardware. Below, we go over the features that renode-test-action provides, present a quick set-up guide based on an example workflow, and frame it in a wider context by showcasing a demo workflow utilizing the action for performing several tests in a project using the Zephyr RTOS.

Automated tests with logs and reports

renode-test-action is designed to automatically set up Renode and execute tests written in the Robot Framework using Renode’s predefined keyword library in GitHub Actions.
The predefined keywords for Robot Framework tests include:

• Start Emulation – starts the emulation
• Reset Emulation – clears the emulation
• Execute Command – executes a command in the Renode Monitor
• Allocate Temporary File – allocates a file in the Renode temporary folder
• Wait For Line On UART – waits for a line to be displayed on the emulated machine’s UART
• Write Line To UART – writes a line to the emulated machine’s UART
• Assert LED State – waits for an emulated LED to turn on or off
• PC Should Be Equal – asserts that the CPU’s Program Counter is equal to the specified value

For an extensive list of keywords, you can refer to Renode’s keyword library.

The action can be used on all operating systems supported by GitHub Actions: Linux, Windows and macOS. It can be used to run tests, or simply to download and build Renode. Executing the action for running tests generates artifacts in the form of logs and summaries of the performed tests that break down the tests per job and provide statistics. Below, you can see such sample reports from our CI pipeline running parts of the open sourced original Tesla Roadster we recreated in Renode.

Including renode-test-action in your CI pipeline

In order to add the action to your CI config, include the following:

steps:
- uses: antmicro/renode-test-action@v4
  with:
    renode-revision: 'master'
    tests-to-run: 'tests/**/*.robot'

You can see the full configuration of the action itself in its .yml file.

The action accepts the following parameters:

• tests-to-run – path to the Robot files you want to execute.
• renode-arguments – optional, additional arguments passed to Renode. See Renode README for details
• artifacts-path – optional, path where test artifacts should be stored. This includes Robot logs and HTML reports. Default: current directory
• renode-revision – indicates the Renode version to be built. Can be the name of a branch or tag in the repository or a commit hash. The default is Renode’s master branch
• renode-repository – indicates the repository containing the Renode source to build. The default is the official Renode repository
• gather-execution-metrics – gathers execution metrics for executed binaries. Sample metrics are visible below.

Further down you can see renode-test-action within a configuration for running tests with Zephyr RTOS in a simulated demo setup featured in Antmicro’s System Designer.

      - name: Setup Zephyr project
        uses: zephyrproject-rtos/action-zephyr-setup@v1
        with:
          app-path: source
          toolchains: arm-zephyr-eabi

      - name: Build application
        shell: bash
        run: |
          mkdir -p artifacts

          west build --pristine -b nrf52840dk_nrf52840 source/app
          cp build/zephyr/zephyr.elf artifacts/software.elf

      - name: Run test in Renode
        uses: antmicro/renode-test-action@v4
        with:
          tests-to-run: source/app/shell_module.robot
          artifacts-path: artifacts/results
          renode-arguments:
            --variable KEYWORDS:${{ github.workspace }}/zephyr/tests/robot/common.robot
            --variable RESC:@${{ github.workspace }}/simulation.resc
            --variable UART:sysbus.uart0
            --variable ELF:@${{ github.workspace }}/artifacts/software.elf

      - name: Store artifacts
        uses: actions/upload-artifact@v4.4.0
        with:
          name: build-artifacts
          path: |
            artifacts/software.elf
            artifacts/results/*.html

The action is also used in automation pipelines for other tools in Antmicro’s open source portfolio, including downloading and building the simulator for samples for Renode’s DPI co-simulation capability and SystemC integration.

Automate your testing and upgrade your CI infrastructure with Antmicro

renode-test-action lets you take advantage of the testing infrastructure built around Renode to introduce a system of checks and balances for software and hardware co-development in your GitHub-hosted projects with just several lines of configuration.

To discuss your automation needs beyond what renode-test-action offers, use its functionality in your existing CI setup, customize it or build an entire infrastructure setup from the ground up, based on Antmicro’s extensive open source workflow and tool portfolio, be sure to reach out to us at contact@antmicro.com.