Extending the open source Rowhammer testing framework to DDR5

DRAM is omnipresent in all kinds of electronic devices and each and every new DDR specification pushes the technological boundaries in order to deliver more storage and throughput combined with a lower power consumption and data latency. All these requirements need to be fulfilled while preserving data integrity, reliability and security of DRAM memories and systems which use them.

Our ongoing work with Google regarding the research of Rowhammer-class vulnerabilities in DRAM memories in both consumer electronics and data centers using custom FPGA platforms sparked great interest both in academia and the commercial world. As a result, we are observing a growing interest in developing custom DRAM controllers with built-in security mechanisms, where basing on existing open source solutions makes sense both from a transparency and vertical integration perspective.

At the same time, our Rowhammer research project goes on, following the developments in the dynamic memories market - and verifying the claims of increased security of new DDR generations in practice. Originally covering LPDDR4 and DDR4 RDIMM modules with two dedicated boards, the project is now expanding to cover a third, DDR5 capable platform. In this note we will describe the hardware and how it enables even more research opportunities into the Rowhammer vulnerability.

Developing a DRAM testing ecosystem with a modular approach

Antmicro often designs open source hardware platforms for FPGA development, including both general purpose, customizable baseboards for FPGA modules (which we then adapt into specific products for our customers) and specialized boards such as this one. The open source Rowhammer testing project started with an LPDDR4 test board which defined the general concept. Our DRAM testing hardware family is basically composed of a popular, reasonably-priced FPGA with typical IO peripherals and a slot for a memory module. The memory controller and PHY are implemented on the FPGA and are used for benchmarking the memory IC.

For the original board, we had to work around the fact that LPDDR4 memories are meant to be placed in close proximity to the processing SoC and typically they are implemented on the same PCB as the SoC itself. For a more modular approach, the LPPDR4 platform instead used an interchangeable LPDDR4 testbed which allows repeatable and easy testing of multiple versions of the LPDDR4 IC with the same base platform.
The Rowhammer tester framework created alongside that board allowed us and the Google team we’re working with to successfully perform and develop new Rowhammer attacks. The framework uses LiteX as the base for the FPGA system composition and includes the RISC-V VexRiscv core for debugging purposes.

The LPDDR4 Rowhammer testing setup soon sparked the interest of other groups, including those with a use case from almost the other side of the spectrum from consumer electronics (where LPDDR4 would be typically found) - namely data center security. For this application, where data reliability and security is definitely a top priority, the (DDR4) Data Center DRAM Tester hardware platform was created and integrated with the Rowhammer tester, enabling a new class of test cases involving DDR4, the RDIMM standard used in servers. That RDIMM memories are already typically sold and used in the form of modules was of course a convenient circumstance, since the new platform maintained the original concept.

With DDR5 being the most recent memory standard deployed in data centers, it was only a matter of time before we were asked to build yet another platform capable of handling it.

Staying on the very edge (and sometimes venturing beyond) the specified capabilities of the FPGA platforms we are using thanks to the full control over the parameters of the open source memory controller, the project is a great showcase how open source can push the boundaries of what is possible. Developing a platform for direct interfacing from FPGA (without a dedicated hard block) to memories using the high-speed and low power DDR5 standard is no trivial task: the FPGA used in the project (and, in fact, any of the FPGAs available on the market) were simply not designed to handle this specifically.

To make sure our assumptions were correct and we’re pushing in the right direction, we used the existing LPDDR4 platform (which conveniently uses a lower voltage than regular DDR4) together with an experimental testbed to prototype the functionality, in parallel with building the final hardware. This was yet another proof of how the original approach was robust enough to enable things that weren’t originally in scope or even planned.

With the proof of concept platform in hand, we could quickly move beyond the prototyping stage to the first version of the final hardware, and we can proudly announce that as of this writing, our DDR5 hardware testing platform has already been prototyped, verified and released on GitHub.

The current efforts are focused on enhancing the DRAM controller and PHY that is used in the Rowhammer tester platform to work with DDR5 modules which will unlock all the features that are currently available in the tester, such as row mapping or measuring cell retention for this type of DDR modules.

DDR5 Tester overview

The DDR5 Tester hardware platform follows the structure of its predecessors - specifically the data-center DDR4 tester. The board includes a Xilinx Kintex-7 series FPGA in a 676-BGA package providing 162,240 Logic cells (25,350 slices).

The board circuitry has been adjusted from the previous generation to meet the hardware requirements defined for DDR5 modules. The DDR5 memory modules utilize the same number of pins (288) as DDR4 but require a different mechanical key. However, contrary to the DDR4 solutions, the DDR5 modules include on-board power management circuitry so the baseboard (i.e. the hardware testing platform in our case) is no longer responsible for generating and sequencing the DRAM power supply rails.

DDR5 modules expand the functionality of the SPD (Serial Presence Detection) bus.
In DDR5 modules it is used not only for getting the basic information about the memory architecture, but also for monitoring the power supply states and monitoring the system health of the module. The overall power consumption of DDR5 modules has been reduced by decreasing the IO voltage. The nominal IO voltage for DDR5 modules is now 1.1V and the DDR5 Tester circuitry has been adjusted for that requirement. This is also the lowest IO voltage that can be handled by Xilinx Series 7 IO banks.

The DDR5 Tester features a single USB 2.0 interface which provides access to a 4-channel FTDI transceiver, which then allows us to reconfigure the FPGA and get access to the debug console. The on-board configuration flash can store the bitstream which will be loaded at startup. Additionally there is a 1Gb Ethernet interface so you can access the DDR5 Tester through the local networking infrastructure.

The DDR5 Tester includes an on-board 64Mb HyperRAM for storing critical runtime data.
There is also an HDMI interface connector which may come in handy while investigating video IP cores and buffering of video frames in a DRAM. There is also an SD card slot for optional removable storage.

The future of Rowhammer testing

Rowhammer attacks, vulnerabilities and countermeasures are an active research area and together with Google, Antmicro continues to adjust the Rowhammer test platform to most recent developments, opening the way for researchers and memory vendors to more sophisticated testing methods. Based on our experience with LPDDR4 and DDR4 we’re currently working on adding DDR5 support to the Rowhammer testing framework, which will enable testing of state-of-the-art memories used in data centers. This work bases on and complements other open source activities we lead as members of the RISC-V and CHIPS Alliance, aimed at making the hardware ecosystem more open, secure and collaborative. If you’re interested in open source solutions for DRAM security testing and memory controller development, or more broadly FPGA and ASIC design and verification, don’t hesitate to reach out to us at contact@antmicro.com.