10Gb Ethernet connectivity between Antmicro’s NVIDIA® Jetson Nano / Xavier NX Baseboards

At Antmicro, hardware and software engineering go hand in hand; being able to build sophisticated computing systems gives us the freedom to employ intricate engineering concepts to create a variety of state-of-the-art devices and applications.

New and exciting use cases in embedded systems and robotics have been driven by developments in advanced AI algorithms and the growingly better performance of modern and powerful devices such as NVIDIA Jetson Nano™ or Xavier™ NX.

Besides using the Jetson platform in a number of exciting client projects, we have been consistently broadening the ecosystem around it, delivering a range of open source extensions that add to the Jetson platform’s already impressive functionality. This time, our engineers have done a project providing fast network connectivity between two Antmicro open source Jetson Nano boards using off-the-shelf network adapters, to showcase how the PCI express exposed by the board can be used to achieve impressive results.

The growing need for speed

The Edge devices of the future will require increasingly faster Ethernet connections due to the growing amount, quality and complexity of the data they will have to transmit. The 1Gb that is so common in today’s embedded applications will not be enough to meet the constantly rising standards of various AI and vision systems. To be ready for that, we designed a setup showcasing a 10Gb Ethernet connection between a pair of NVIDIA Jetson Nano and Xavier NX SoMs.

The testing setup consisted of two NVIDIA Jetson Nano modules, Antmicro’s open source baseboards compatible with Jetson Nano / Xavier NX, recently upgraded with an M.2 connector, and Asus XG-C100C Network Cards. Due to different form factors on the network cards and the baseboards, we had to use our M.2 to PCIe adapter card, also released by Antmicro as open source hardware on GitHub, to connect the two components and add a 10Gb Ethernet interface to the baseboard. A similar setup where Jetson Nano was replaced with Xavier NX gave comparable results.

The XG-C100C is based on the Marvell AQC107 chip, for which drivers are already included in the kernel image in the official Jetson Nano BSP (L4T 32.3.1), meaning that no additional effort was necessary to get the network to adapters work with the Jetson Nano and bring-up the setup.

Establishing the high-speed connection

We started by configuring the two Antmicro baseboards to act as a DHCP server and as a client, respectively, using the iPerf tool to measure the connection throughput. The following command was used to start the server on one of the machines:

nvidia@jetson:~$ iperf3 -s

Next, the the connection was established on the second machine using:

nvidia@jetson:~$ iperf3 -c 10.42.0.1

The following output was received:

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  5.38 GBytes  4.62 Gbits/sec              

Although a markedly higher connection speed as compared to regular GbE was achieved in the first attempt, this was still far from the theoretical 10GbE maximum. Updating the drivers with their latest version from the network card manufacturer’s website improved the bandwidth slightly; however it was still far from the possible 10Gb:

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  6.00 GBytes  5.15 Gbits/sec

To remedy that we changed the network card’s default MTU (Maximum Transmission Unit) value of 1500 bytes to 9000 bytes. This resulted in a much lower number of interrupts that had to be handled by the CPU, which in turn increased the bandwidth to more than 8Gb/s.

- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.01  sec  9.70 GBytes  8.33 Gbits/sec      

Applications

High-speed Ethernet connections are crucial e.g. for time-sensitive applications. The lowest possible latency is sought-after in devices designed for performing remote medical procedures, human-operated machines working in hazardous conditions or sterevision vision systems that require precise timing for locating objects, tracking them or generating depth maps.

On top of that, the increased data throughput can allow next generation edge AI devices to transmit higher resolution or higher frame rate video data, or process more data streams using fewer cables than before.

Thanks to their robustness and high resistance to interference, high-speed optical cables like the ones used in this solution can also be applied in harsh industrial conditions, enabling new use cases where other high-speed links would not work.

Enabled by Antmicro’s open source components

Antmicro’s Jetson Nano / Xavier NX baseboard is a tried and tested platform which, especially when combined with the M.2-to-PCIe adapter, allows us to rapidly prototype advanced solutions, combining them with various off the shelf components like 10GbE network cards, GbE switches, external AI accelerators, FPGA processing cards or LTE modems.

Our rapid prototyping capabilities and software excellence let us to quickly de-risk your next generation advanced edge AI platform. Often, we work with our customers on Proof of Concept and Feasibility Study projects which use the wide ecosystem of open hardware we are developing, and once the path to the solution has been determined, we provide the engineering services to integrate those components more tightly into a final product.

The highly successful open Jetson SoM baseboard design is a great starting point for systems featuring custom boards that Antmicro can build for you, free from any royalties or licensing limitations whatsoever. We can then help you bring them to the market in a robust and sophisticated device over which you will have full control. On top of our hardware capabilities, we offer software services - BSPs, OTA update systems, applications - as well as AI algorithms and optimizations, complementing your team’s abilities where needed. If you are looking to develop your next edge AI product, reach out to us at contact@antmicro.com and let us know how we can help!