Bazel-orfs: Combining Bazel with OpenROAD Flow Scripts for rapid ASIC design iteration with caching

The complexity of the multi-stage ASIC design process is reflected in the structure of the most popular open source project in this space called OpenROAD which offers a collection of ASIC design tools that can be put together into a complete ASIC flow in many ways. The default flow, OpenROAD-flow-scripts (ORFS), lets you go from RTL to GDSII in a number of steps (to learn more about this, you can read our previous note on optimizing OpenROAD) each of which can involve many calls to specific tools. This kind of flow naturally lends itself to speeding up with caching using the Bazel build system. The distributed caching offered by Bazel lets you reuse the build results for specific steps from previous runs or even from other developers, and can help your ASIC team move orders of magnitude faster.

While Bazel rules for OpenROAD have existed for a while, they followed a flow that is completely separate from the multi-stage ORFS flow that the project uses by default, which means that during support, debugging and optimizing OpenROAD as such, Bazel could not be used, and any problems or inefficiencies encountered in one flow were hard to reproduce in the other. The Bazel-orfs project which is a set of Bazel rules for wrapping Physical Design Flows provided by ORFS originally developed by Ascenium, a disruptive Norwegian computing platform company developing a General-Purpose Accelerator (GPA), aims to address that and combine the best of both worlds. In this article we will describe Antmicro’s work for Ascenium to optimize and improve the Bazel-orfs flow, and how it’s being employed in the MegaBoom use case to allow architectural exploration with the open source ASAP7 PDK.

We’ll start with describing how Bazel-orfs works and then walk you through the most important developments introduced by Antmicro and Ascenium to Bazel-orfs, including taking advantage of hierarchical designs built from reusable macros, multiple builds of individual macros running in parallel, rapid prototyping with abstract targets, reproducible builds, as well as enhanced collaboration possibilities thanks to the remote caching feature.

Enhancing OpenROAD Flow Scripts with Bazel

Bazel’s magic is achieved through providing a hermetic environment that enables reproducible builds. Since Bazel allows caching intermediate results, with Bazel-orfs you can divide a design into smaller parts (called targets in Bazel), which don’t have to be rebuilt each time a new component is added or a change irrelevant for previous stages is introduced. Moreover, the cache can be shared with the entire team using Bazel remote caching, or it can be made available publicly, greatly accelerating development.

Bazel-orfs is an example of separation between ORFS and different custom build flows. It follows the OpenROAD-flow-scripts convention and divides the whole build into stages, but apart from that, smaller parts of the design can be separated into distinct designs. This makes the build process more parallel and potentially faster. Bazel-orfs provides the user with a choice between two ways of running the physical design flow with ORFS, depending on the origin of the ORFS installation in the build environment. The flow can be executed through:

• Bazel Flow – the physical design flow that uses the official ORFS Docker image, but for technical reasons having to do with hermiticity does not run in Docker inside Bazel.
• Local Flow – an installation of ORFS that is optional on the machine that runs the flow, useful if you need to hack or create support requests for ORFS or OpenROAD.

Regardless of the chosen flow, Bazel-orfs provides targets consistent with the stages defined by ORFS in a given design:

• Dependencies target – generates scripts which build the specified stage of the design and prepares dependencies, such as results from the previous stage.
• ORFS stage target – runs the physical design flow up to the chosen stage.
• Abstract target – creates the mocked abstracts for macros.

Those targets (created with the following schemas) allow developers to easily prepare the environment for building and developing designs. Thanks to remote caching, they can quickly get started with the already built base - collect design files, investigate the results with GUI, debug the artifacts and continue development of specific parts.

Bazel-orfs: faster and reproducible builds with reusable macros and improved caching

Antmicro’s involvement in the Bazel-orfs project revolved around improving caching capabilities and optimizing the overall build flow. With the improvements introduced throughout the project, we managed to significantly speed up the iterative design process by taking advantage of reusable macros in the process of building hierarchical designs. Build artifacts cached locally and remotely can also be reused in subsequent builds. Together with Ascenium we also enabled running multiple builds of individual macros in parallel, as well as rapid prototyping with abstract targets). An abstract target has the same ports and size as its real equivalent with internal logic removed to speed up the placement and routing stages of the flow. Abstract targets are useful for estimating the sizes of macros with long build times as well as for checking if they will fit in modules which are higher in the design hierarchy, without having to run a time consuming place and route flow.

When it comes to Bazel’s caching capabilities, we improved the way the build artifacts are cached to minimize cache invalidation and allow collaborators to reliably reuse cache pushed to the server.

To make it easier to start using the project, we added encapsulating build stages based on the Docker image - this way developers do not have to prepare their workspace and build ORFS from scratch, but rather download a full-featured Docker image from the registry and jump right into development. Developers can easily run both the design’s building and a GUI preview for each stage from the container level. What is more, developers can seamlessly move between the Bazel and local environment, e.g. to only run stages involving tools that they don’t have locally, in the sandboxed environment, and use the modified ORFS, built locally, for development purposes.

To make the Docker image easily accessible, we optimized the build process significantly reducing its size and layers reusability through the cache. We also added tests for various usage scenarios of Bazel-orfs, involving all build stages for designs.

Additionally, we reworked the mechanism for generating and passing the flow configuration parameters to ORFS. Instead of passing all parameters through environment variables specified in the command line executed by Bazel, the Bazel-orfs generates a configuration file that becomes a dependency for Bazel targets. It contains the name of the design and the name of the PDK to use in the implementation process, as well as all the parameters that are specific to a given stage of the physical design flow. With that modification we improved the caching performance and provided a way of changing the flow parameters that works well with the script-based manual flow of the execution.

A real-life example, where running the physical design flow through Bazel-orfs improves the design exploration process of Berkeley’s Out-of-Order Machine (BOOM) open source RV64GC RISC-V core, can be found in the The-OpenROAD-Project/megaboom repository. In fact, some early work in the project involved untangling Bazel-orfs from MegaBoom, making Bazel-orfs a standalone project and a dependency of the MegaBoom design, as well as building a proper CI and benchmarking flow for the latter.

Streamlining large ASIC design with Antmicro

As a fast and scalable build tool, Bazel has been present in other projects that Antmicro contributes to, including Google’s XLS Mid-Level Synthesis toolchain and the OpenTitan Root of Trust. With our work on Bazel-orfs described in this article, we strive to bring the benefits of Bazel to any digital ASIC design.

Beyond the building stage, Antmicro’s services for ASIC design also include verification with Verilator, an open source tool we’ve been actively improving by introducing features such as assertion control, constrained randomization and UVM support. As mentioned earlier, we’ve also been working on speeding up OpenROAD by identifying bottlenecks in the flow and optimizing the code to yield faster turnaround times.

To find out more about Antmicro’s engineering services and how we can help you optimize and improve your ASIC design flow, don’t hesitate to contact us at contact@antmicro.com.