sv-bugpoint: pinpoint minimal bug-inducing SystemVerilog code subsets to improve debugging in Verilator and other SV tools

Debugging and extending ASIC and FPGA tools such as RTL simulators often means digging through vast codebases of not only the tooling itself but the designs they take as input. This can be cumbersome, especially in the context of very large, state-of-the-art, often proprietary RTL designs that we usually work with. Additionally, reporting and proposing fixes for bugs sometimes requires sharing code snippets to illustrate the problem – something that’s not always desirable or even possible. However, using dedicated code-reducing tools, you can iteratively remove portions of code irrelevant to the problem until you’re left with the smallest bug-inducing fragment, which can then be reasoned about and shared externally much more easily.

In this article we introduce sv-bugpoint, an open source tool for finding minimal subsets of SystemVerilog designs that pinpoint the real cause of bugs or gaps in SystemVerilog tools. We’ll also show examples of bugs identified and fixed thanks to sv-bugpoint in Verilator. However, sv-bugpoint is universal and can be used with any SystemVerilog tool, including projects Antmicro actively contributes to, such as OpenROAD, the Synlig synthesis tool, and more.

Minimizing SystemVerilog code with sv-bugpoint

The initial version of sv-bugpoint was created within the EU-funded TRISTAN project aiming at maturing and industrializing the RISC-V ecosystem, and has since been expanded and improved. The current implementation uses slang, a fast SystemVerilog parser with a flexible API and the best language coverage among open source SystemVerilog tools, as visualized in the SV test suite we had developed within the CHIPS Alliance a few years back. Slang parses SystemVerilog into a syntax tree that is then traversed and reduced part by part. Using slang allows sv-bugpoint to be smart about what to remove and in which order, in a way that keeps the syntax and dependencies correct.

To get started with sv-bugpoint, you need to prepare an input script that takes the path to a SystemVerilog file as the first argument and asserts that the same bug (or any other issue triggered by the code) is still occurring. We’ve prepared example scripts that can be used as reference. Next, the design is processed with slang.

Running sv-bugpoint involves multiple stages that are executed sequentially, as long as any of them manages to delete something. Removals are grouped and ordered in a way that maximizes the chance of:

• removing the “leaf part” (i.e. one that other parts don’t depend on)
• removing the part that introduces unnecessary dependencies (i.e. one that depends on as many parts as possible)
• removing the biggest parts as soon as possible (as it often results in faster runs of the check-script).

The most prominent example of this approach is removing bodies of subroutines and classes as one of the first stages. Then, you need to re-run the input script and see if you’re still getting the expected results. If yes (meaning, the issue can be reproduced), you can minimize the design further.

If you want to use sv-bugpoint to debug Verilator, you can try the sv-bugpoint-verilator-gen script to automatically generate an input test case and a check script template. For more information about using sv-bugpoint, refer to its README.

Identifying bugs in SystemVerilog tools with sv-bugpoint

We’ve already used sv-bugpoint to identify and fix bugs in several tools. Since Verilator is one of the most commonly used SystemVerilog tools in our customer projects, to which we contribute pretty heavily, the following examples will focus on this RTL simulator.

In an illustrative scenario, when running a UVM testbench, which together with the UVM library itself totals ~37kLoC, we noticed the verilation of the testbench was looping infinitely. Using sv-bugpoint, we managed to minimize it to ~200 lines, which was further minimized manually, resulting in just 11 lines. Thanks to that reduced test case, we were able to find and fix the problem, which was caused by indirect recursive function calls (i.e. a function calls itself behind dozens of other function calls).

To thoroughly test sv-bugpoint during the development phase, we also used it to reproduce different types of previously reported Verilator bugs. One such problem was identified in the Caliptra project. This test case is interesting because the invalid behavior was non-local – there was a problem with a force assign statement in a completely different place to where the bug had occurred. Automated checking for this bug also required analyzing the VCD generated by Verilator. In this case, sv-bugpoint was able to minimize the code from ~90k lines to 200. This example shows that sv-bugpoint can significantly reduce the testcase size even in such unusual scenarios.

Another example originating in the Caliptra project was a simple local case in which we managed to reduce the initial ~90k LoC to just 31.

While best illustrated on open source RTL designs, sv-bugpoint is especially useful for scenarios that we commonly work with when helping some of the world’s largest companies building next-gen data center and AI silicon, using our RTL development and DV services as well as productivity tooling.

Improving SystemVerilog tooling with Antmicro

Following the astsee suite we introduced earlier this year, sv-bugpoint is yet another addition to Antmicro’s SystemVerilog toolkit. Thanks to sv-bugpoint’s universal nature, the approach described above can be applied to any SystemVerilog tool.

Beyond fixing bugs and assisting in the adoption or version migrations, Antmicro offers a wide range of services around Verilator, with the most recent milestone including adding real-world UVM VIP support.

Get in touch at contact@antmicro.com to learn more about sv-bugpoint and commercial support for Verilator offered by Antmicro, as well as our open source ecosystem for ASIC/FPGA development.