Adapting OpenTitan for open source FPGA prototyping and tooling development

Antmicro’s projects often involve helping companies address their specific processing, security and other needs through designing and prototyping ASIC solutions with open source tools and SoC components, including adapting the RTL to run on specific FPGA boards for emulation and testing purposes.

As most cutting-edge ASIC development projects use SystemVerilog, for a while now Antmicro has been working on improving support for it in open source synthesis tools, specifically by extending Yosys with the Surelog and UHDM SystemVerilog frontend in a collaboration with Google within the CHIPS Alliance. As a fairly involved SystemVerilog design used in some of the ASIC development projects Antmicro participates in, the OpenTitan silicon Root of Trust (RoT) project is a good target for testing and hardening the tools in a real-world setting.

In this note, we’ll describe how we extended OpenTitan’s scaling-down capabilities to generate designs that fit it into standard FPGA boards by introducing a custom-tiny OpenTitan variant and the efforts toward enabling this configuration in the fully open source Yosys/UHDM flow.

OpenTitan overview

OpenTitan’s development infrastructure and the Bazel build system used for building the project’s hardware and software are quite complex but flexible, with a lot of potential for customization. However, the main development workflow revolves around the standard OpenTitan configuration called Earl Grey, consisting of a large number of peripherals, including a RV32IMCB RISC-V Ibex core, 128kB main SRAM, fixed-frequency and AON timers, and interfaces such as GPIO, UART, I2C, and SPI.

The richness of the SoC building blocks provided by OpenTitan make it an interesting template and peripheral repository for other open source ASIC designs of that kind. For the same reason it is an excellent test subject for open source tooling development. In most such scenarios, both to test real-world usability and enable wider reproducibility of the work, it is useful to test on real, commodity FPGAs.

However, certain assumptions in the OpenTitan build system made it hard to freely generate needed configurations, and to avoid manually extracting specific peripheral blocks from OpenTitan to build OpenTitan subsets as required, we set out to generalize the build system to remove those limitations.

As an additional argument for making OpenTitan more freely customizable for smaller designs, many of the project’s peripherals are supported in Renode, Antmicro’s open source simulation framework. Thanks to that, a simulation of a specific design can be easily prepared by building a custom Renode Platform .repl file, and even partially automated by using OpenTitan’s Hjson configuration files.

Synthesizing the design with the open source Yosys/UHDM flow

Yosys, a widely used open source RTL synthesis tool, by default doesn’t fully support the newest SystemVerilog constructs, as visible in the SV compliance tests dashboard provided by CHIPS Alliance’s SystemVerilog Tests framework.

In the context of its ASIC customers, Antmicro has been working on making Yosys work with modern SystemVerilog for a while now and the custom-tiny design is an important stepping stone towards expanding SystemVerilog support in Yosys using our custom UHDM plugin, described in detail in a separate blog note.

The plugin is used for parsing SV code with Surelog, which produces an intermediate representation of the code in the UHDM format. Then, this UHDM output is further processed with Yosys, which allows us to add new functionality to Yosys without modifying the tool itself, making it easy to stay in sync with the mainline project.

The original OpenTitan design uses all the modern features of SystemVerilog, and together with all of its peripheral blocks is fairly complex to be processed in its entirety. The currently ongoing development efforts for the Yosys SV plugin can take advantage of the availability of a smaller target such as the custom-tiny configuration for a more traceable, milestone-based development that can be tested on real FPGAs.

Adapting the OpenTitan design

An OpenTitan-based design is driven by a configuration which tells the build system which hardware and software components to use and how. The config is used together with a template provided by the topgen.py utility to generate an RTL description and some of the software for the final SoC. Although in theory the template and the build system should not make any assumptions about the OpenTitan config, during our work on the custom design, we had to remove some hard-coded elements in the template itself (e.g. the USB core) or in the build system.

To enable separate build targets for multiple independent and differing configurations, we had to rework the Bazel build system. This included adding a new set of rule generators for C/C++ software components that inject the #define guards specific to an OpenTitan config. The new rule generators also allowed us to simply specify a list of OpenTitan configs for all targets, instead of manually copying their definitions. Another change included modifying the rule responsible for ROM and flash binary images scrambling so that separate binaries are written for each configuration scrambled with appropriate keys. Another new Bazel rule generator was added for simulation targets that use Verilator.

To enable more flexibility in OpenTitan’s use of the FuseSoC RTL build system, several issues had to be fixed. We renamed IP cores generated by topgen.py so that there are no name conflicts between different OpenTitan configurations and added flags controlling which IP cores should be used. The template itself also required some rework - all config-specific details had to be removed to make it generic. We also added support for setups missing some of the non-vital peripheral modules (e.g. USB, ADC, I2C, SPI).

As for the topgen utility itself, we modified it to allow for the absence of some system pieces in the design, such as system clocks or nonexistent core-to-core connections, which prevents topgen from crashing. We also added support for FPGA platform-specific signal connections and replaced some of the complex blocks necessary for standard OpenTitan operation (EDN and KMAC) with much simpler alternatives.

Because of the differences in RTL, we also had to alter some common parts of the software. Specifically, we renamed defines and constants from TOP_EARLGREY_* and kTopEarlgrey* to OPENTITAN_* and kOpentitan* respectively. We also added a proxy include file opentitan.h which includes appropriate headers for each OpenTitan config, and replaced the inclusion of top_earlgray.h with opentitan.h in every instance. This enables a common set of C/C++ source and header files for drivers and tests.

Below you can see a block diagram showing which parts of the original Earl Grey design were removed in custom-tiny:

We also updated the project to support the recent Verilator v5.0+, which includes Antmicro’s work on scaling Verilator for large designs as well as efforts toward enabling open source SystemVerilog/UVM support.

The changes introduced in Antmicro’s custom-tiny configuration enable adding more OpenTitan configurations in the future to support further FPGA prototyping and open source use.

Building the project on Artix A200T

The original Earl Grey SoC, due to its size, would not fit in an FPGA smaller than the Kintex 410T. However, Kintex 410T boards can be expensive and hard to come by - while Antmicro’s open hardware design portfolio includes a K410T-based board developed in the context of this project, the ability to use a wider array of more widely accessible hardware is always good for making the project easier to work with collaboratively. Antmicro’s custom-tiny configuration fits in a smaller, more affordable Artix A200T FPGA which is included in some popular boards including NexysVideo, which many users interested in working with OpenTitan might already have on their desks.

Thanks to Antmicro’s work on OpenTitan’s Bazel build system, building a bitstream for the NexysVideo board requires simply invoking a different target:

bazel build //hw/bitstream/vivado:fpga_nexysvideo

Loading the bitstream can be done e.g. with openFPGALoader:

openFpgaLoader -b nexysVideo <path to the .bit file>

Currently, the bitstream contains only the test_rom software and immediately upon each reset it should output a welcome message with the SoC version to the console. To view the output, connect the NexysVideo UART micro-USB to your PC and run a terminal, e.g. picocom:

picocom -b 115200 /dev/ttyUSB<n>

The custom-tiny configuration, as well as the original Earl Grey design, will also be tested on the new open hardware Kintex 410T board, which will be soon described in a separate blog note.

Open source ASIC development and prototyping tools

With our versatile experience in ASIC design and tooling development and automation, as well as a wide array of open source tools and SoC building blocks we’re working on, Antmicro is in a unique position to support silicon companies in expanding their FPGA/ASIC testing and prototyping capabilities and velocity, giving teams transparency and complete ownership over their development process.

If you need help with adapting an FPGA design to target a specific development platform or are looking to use open source tools in designing your next SystemVerilog ASIC, reach out to us at contact@antmicro.com.