eCos with SMP support for Zynq FPGA SoC

The combination of a dual-core ARM CPU and FPGA found in the Zynq SoC is perfect for applications which can benefit from parallel execution. Using the FPGA fabric of Zynq, critical functionalities of the system can be accelerated using customized IP.

Typically, Zynq users will run Linux on the ARM CPU, but in solutions with real-time constraints or where code size and more fine-grained control over the behaviour of the system are important, RTOS such as eCos are a good alternative.

Back in 2012, soon after first Zynq System on Modules (SoMs) like the Enclustra Mars ZX3 appeared on the market enabling a broad adoption of this FPGA SoC in industrial applications, we have ported eCos to Zynq and have been supporting it ever since.

Our original port of eCos utilized only a single core of the ARM CPU, as we typically run it (and expected others to do so too) it in AMP (Asynchronous Multicore Processing) mode, with the other core reserved for Linux, or in situations where the CPU side was mostly used for configuration and real-time control purposes.

If however you want to perform some more processing also on the ARM cores, it makes sense to use both of them in an SMP (Synchronous Multicore Processing) mode. Therefore we decided to add SMP support to our eCos port so that threads can be distributed on both ARM cores to optimize the runtime of multi-threaded applications.

Below you will find a short tutorial on how to download and run our eCos port with SMP (symmetric multiprocessing) support. As usual, we will be using our partner Enclustra’s Mars ZX3 module for that end.

Getting the sources and tools

Grab a copy of the eCos sources using Git:

git clone https://github.com/antmicro/ecos-mars-zx3.git ecos
cd ecos

Building the system

First of all, the ECOS_REPOSITORY environment variable needs to be set.
To do so, execute:

cd packages
export ECOS_REPOSITORY=`pwd`
cd ..

eCos is always built outside the source tree.
Create a new directory in which the system will be built:

mkdir build
cd build

Choose the configuration file to build upon using the ecosconfig tool:

ecosconfig --config=../mars_zx3_ecos_smp.ecc tree

Building the system is triggered using the Makefile system, simply run:

make -j9

Then run basic tests, including a test that verifies the actual multiprocessing:

make tests -j9

The output executable can be found in install/tests/kernel/current/tests/smp relative to your current directory.

Deploying the example application

The following instruction describes how to run the application on the Mars ZX3 module on Mars PM3 baseboard.

First, get U-Boot running on the board.
For instructions on how to get U-Boot running on the Mars ZX3 module please refer to the user documentation.

Put the executable file either on the SD card or inside a TFTP server directory, and load it using U-Boot.

For SD cards run:

fatload mmc 0 0x0 smp

For the TFTP server, make sure you have the ipaddr and serverip environment variables set correctly, and the physical Ethernet cable connection is working and run:

tftpboot 0x0 smp

Once the file is loaded, it can be run using the bootelf command.

bootelf 0x0

The application runs a few tests in a loop to check whether all CPUs are active and whether the scheduling and rescheduling of specific threads works as expected.

Part of the application output:

INFO:<Timeslice Test: done>
INFO:<CPU Test: Check CPUs functional>
INFO:<CPU Test: done>
INFO:<Timeslice Test: Check timeslicing works>
 Thread    CPU  0   CPU  1   Total
      0         0    36308   36308
      1     41705    36570   78275
      2     42273    36491   78764
      3     42200    36560   78760
      4     42238       53   42291
      5         0        0       0
 Total     168416   145982
Threads         4        5
INFO:<Timeslice Test: done>
INFO:<CPU Test: Check CPUs functional>
INFO:<CPU Test: done>
INFO:<Timeslice Test: Check timeslicing works>
 Thread    CPU  0   CPU  1   Total
      0         0    36288   36288
      1     41682    36568   78250
      2     42271    36492   78763
      3     42200    36551   78751
      4     42237       52   42289
      5         0        0       0
 Total     168390   145951
Threads         4        5
INFO:<Timeslice Test: done>
PASS:<SMP tests OK>
EXIT:<done>

As you can see, basic SMP works. All of the CPUs are used, new threads get scheduled to the least occupied CPU.

Good luck multithreading!