Allied Vision Alvium drivers released for Jetson TX2 (and Nano)

Throughout the last year, Antmicro has been heavily involved with Allied Vision in developing support for their innovative Alvium camera series. As a first step on a path towards a broader, cross-platform software support for the plug-and-play, ISP cameras, the community recently saw the open source release of the Linux drivers for Alvium cameras on the NVIDIA Jetson TX2 platform.

Based on popular demand, early Jetson Nano support is also available - opening up a way towards inexpensive, smart camera-enabled solutions that we’d been helping our customers build for many years now.

Alvium SoC

Alvium is a brand-new series of cameras by Allied Vision targeting embedded platforms. They are equipped with a custom-made Alvium System-on-Chip that controls the image sensor. The Alvium SoC provides a unified interface for controlling the camera, independent of what sensor it is equipped with. This creates an abstraction layer which takes away the differences in handling various image sensors from the software that controls it.

Thanks to this modularity aspect, it was possible to create a single, robust driver that supports all current (and future!) Alvium-based cameras and which is able to dynamically obtain a camera’s properties and capabilities, and present them to the user in a unified manner.

Open source drivers for NVIDIA Jetson TX2 (and Nano)

A great match for creating edge computing vision systems is the powerful NVIDIA Jetson TX2 platform - the widely used and less power-hungry/expensive predecessor of the current Jetson Xavier (now in both the original and Jetson Nano-compatible form factor). The Jetson TX2 is able to drive up to 6 CSI-2 cameras, and is equipped with a powerful GPU that can be used for image processing.

Antmicro had successfully helped its customers build a plethora of products using the TX2 - but varying degrees of support for different camera sensors, and their differing capabilities and interfaces have always been pain points. Working with Allied Vision to create a unified, open source solution has therefore been a very welcome project where we can fill an important gap.

The driver created by Antmicro for Allied Vision for the Jetson TX2 platform supports all of the cameras released so far in the Alvium series. The code, together with setup instructions, can be found on Allied Vision’s Github repository.

As released, the driver is configured to support Allied Vision’s adapter board for the TX2 devkit, but other adapters can be used, with adjustments to the configuration in the device tree files. Naturally, we have also been using AV’s cameras in our own projects and with our own hardware kits which feature our 2x4 lane MIPI CSI-2 connectors (as seen in the picture) for which we also have adapters.

A beta release is also available for early Jetson Nano support for those who cannot wait for the final version!

User interface

Diving in deeper into the drivers, they provide the user with a standard video4linux2 device. Available video formats are obtained dynamically from the camera, and the driver supports all of the possible formats (including Bayer, RGB and YUV). The CSI lane count can be configured to the desired value - from 1x up to 4x CSI lanes. The configurable camera parameters are available to the user as custom video4linux2 controls.

As the camera is available as a v4l2 interface, it can be easily used with any application that supports this standard. Here is an example of displaying the video stream using the qv4l2 GUI:

qv4l2 is also useful for discovering custom controls provided by the camera:

Frames from the camera can also be obtained using the v4l2-ctl CLI tool - below is an example command that will capture 10 frames from the camera and save them into a file called frames.raw:

v4l2-ctl --stream-mmap --stream-to=frames.raw --stream-count=10 --set-fmt-video=pixelformat=BX24

If there is a need to access camera controls remotely, or in a headless system, we have created a useful CLI utility for exactly this use case, called pyvidctrl.

Another useful tool for video processing is gstreamer - video can be streamed using gstreamer with the v4l2src element, and processed further using its powerful plugin system. Here is an example of redirecting the video to the xvimagesink element, which simply displays the stream in a window:

gst-launch-1.0 v4l2src ! video/x-raw,format=xRGB ! videoconvert ! xvimagesink sync=false

Develop drivers for cutting-edge platforms

We’re currently working with Allied Vision to produce Linux drivers supporting Alvium for a broad range of edge AI platforms - next up is the NXP i.MX 8. If you’re interested to get your cameras working in an elegant, scalable fashion for a new embedded device, Antmicro offers expertise in the NVIDIA Jetson Xavier, Xavier NX, Nano, the NXP i.MX series and many other platforms. Find out more at contact@antmicro.com.