Enhancing RTL coverage reporting in Verilator with new features and computation optimizations

Monitoring code coverage in digital design projects requires tracking both traditional software metrics such as executed code lines and visited branches, as well as more use-case specific concepts such bit field toggles of signals. Analyzing coverage information generated by HDL simulators can be done using Antmicro’s Coverview tool, and detailed, interactive visualization of the results of your tests helps ensure your design is thoroughly tested.

In our digital design projects, we often use (and improve) the open source Verilator RTL simulator, which includes support for different types of coverage required by RTL projects, and given its open source nature, we can improve and adapt it to the needs of our projects and tightly integrate it into our (and our customers’) workflows.

In this article we will summarize the current state of code coverage support in Verilator and describe the features and improvements we recently introduced to improve Verilator’s usefulness for this use case.

Coverage reporting in Verilator

Verilator handles coverage measurement by automatically inserting counters to SystemVerilog code, which are incremented each time a statement is executed or a bit changes its value. In most cases, users don’t need the exact values of counters, only the information whether they are 0 or not. If a counter is non-zero, it means that a statement/bit was covered by at least one of the tests at least once.

In the end, Verilator generates .info files that can be then used with Antmicro’s Coverview tool to generate interactive coverage dashboards, as described in more detail in a recent article.

There are four types of coverage that are currently supported in Verilator: branch, line, expression and toggle coverage.

Branch coverage shows how many times each of the branches was executed. For example, an if statement contains two branches: if (condition) { 1st branch } else { 2nd branch }. To measure this type of coverage, Verilator creates a counter for each branch and places its incrementation in that branch. This incrementation is executed the same way as any other statement.

Line coverage demonstrates how many times the code from each line was executed during the tests. Similarly to branch coverage, it is handled by counter insertion. With the exception of constructs that introduce branches, statements are executed one by one. Keeping a separate counter for each line would significantly reduce the performance of simulation, so instead, the counter information is stored for a given range (or ranges) of lines and only one incrementation is inserted. In the example below there is one counter for two ranges. The first range is from while to if (x == 10) , and the second from z++ to x = y + z;. The body of if isn’t evaluated in every iteration of while, so it has to have a separate counter. Statements after the if body are evaluated the same number of times as statements before if, so they have the same counter.

Expression coverage shows how many times sets of subexpressions of a Boolean expression cause its evaluation to a given value (true or false). It is handled by inserting if statements containing counters. Conditions of these if statements are conjunctions of subexpressions to which the counter relates. Each of these subexpression sets has its own counter.

Toggle coverage illustrates how many times each bit of each variable changed its value. Computing this type of coverage is more complicated than other types, as the counter is not incremented directly after the change of value. Previously, Verilator only had a single counter for changes both from 0 to 1 and from 1 to 0. As described in the section below, Antmicro recently introduced separate counters for each type of value change.

New features and improved coverage computation

While using Verilator to analyze code coverage for the digital design projects we are working on, we identified several ways to optimize the coverage computation process, but also found and fixed a number of bugs, some of them not strictly related to coverage. Coverage measurement in Verilator involves using constructs that are also used in other contexts, but some of them rarely. After we started using them more often for coverage computation, we could easier find bugs related to those constructs and later fix them, which led to improvements in Verilator beyond just coverage reporting. For example, coverage measurement utilizes an internal Verilator construct, which allows executing a list of statements within an expression. It is an equivalent to C++’s immediately invoked lambda function. In some cases, such an expression was cloned multiple times, which resulted in multiplication of side effects. We found and fixed an example in which the number of duplicated side effects grew exponentially.

In order to produce .info files, the raw Verilator coverage output (generated during simulations of the model created by Verilator) has to be processed with the verilator_coverage tool. Previously, if a user wanted to generate reports for both toggle and branch coverage, they had to run the whole process twice (i.e. verilation, compilation, simulation, processing with verilator_coverage). We added an option to select the coverage type in verilator_coverage, which means now only processing with verilator_coverage needs to be repeated, and since it’s the fastest step of that process, this change saves a lot of time.

We also introduced several improvements for specific coverage types, including adding ternary operators to branch coverage and variables defined in generate blocks to toggle coverage. Additionally, we separated toggle coverage counters for changes from 0 to 1 and from 1 to 0.

Some of Antmicro’s other recent contributions to Verilator related to coverage include:

• optimizing the way toggle coverage is computed,
• cleanup of the coverage-related code,
• fixing coverage of complex type variables.

Those improvements, as used in several projects we’re involved with including e.g. Caliptra, have allowed us to introduce more detailed and easily reproducible coverage reporting and measurement, making it easier to iterate and communicate around verification coverage in collaborative projects.

Open source RTL verification with Verilator

Verilator’s coverage reporting features, combined with its integration with tools such as cocotb and Renode, provide verification engineers with actionable insights into their designs. Antmicro offers engineering support in customizing and optimizing Verilator, even for large and complex designs. If you’re interested in integrating Verilator with your existing workflows or would like to learn more about Antmicro’s related tools for coverage analysis and test planning, reach out to us at contact@antmicro.com.

To learn more about improvements to coverage reporting and Antmicro’s other recent contributions to Verilator, join our talk at this year’s ORConf on 12-14 September in Valencia. We will highlight the most interesting results of our work, including UVM support, constrained randomization, power analysis using SAIF, and more.