Building video systems with NVIDIA Jetson using FRAMOS cameras

Antmicro’s AI projects typically involve different video input sources, including MIPI CSI-2 cameras from various camera vendors. Thanks to their broad offering of SONY sensors, the German imaging component supplier FRAMOS is a common choice in this ecosystem, and for a while now, Antmicro has been working with FRAMOS imagers to build real-world applications, such as smart city monitoring systems, industrial predictive maintenance systems, and advanced imaging solutions.

In our work, we develop and use a wide range of open source hardware video processing platforms, from boards targeting NVIDIA Jetson SoMs, including the most recent Orin family, through Snapdragon 845 and 625 baseboards, to FPGA-driven solutions such as the Kria K26 Devboard. These projects, in combination with a range of Antmicro’s camera accessories as well as software frameworks and video pipeline development services, serve as a starting point for Proof-of-Concept designs in edge AI scenarios. In that spirit, we recently released a FRAMOS CSI Adapter as well as Dual I-PEX CSI Interposer, enabling the integration of FRAMOS camera modules with a portfolio of platforms and technology demonstrators developed by Antmicro.

The FRAMOS ecosystem

FRAMOS camera sensors come in the form of compact, off-the-shelf video modules with interchangeable optics, so that multiple imagers can easily be tested with a single hardware setup, facilitating rapid prototyping. They also offer triggering signals for multiple synchronization scenarios, which is a common request among our customers. This modular system consists of the following elements:

• FSM - FRAMOS Sensor Modules holding the actual imagers with resolution ranging from 0.4 to 24 MP, facilitating connection via standard connectors, lens mounts and mechanical attachment points.
• FSA - FRAMOS Sensor Module Adapter that is stacked with the FRAMOS Sensor Module (FSM). The FSA provides sensor-specific power supplies, clocking and power sequencing. The supply voltages for the sensors are generated with low-noise circuitry for optimal performance and image quality.
• FPA - FRAMOS Processor Adapter for third-party processor development boards (including NVIDIA Jetson AGX Orin and AGX Xavier) connects up to four FSM+FSA combinations to the official NVIDIA Developer Kits. This board acts mostly as an interposer translating the unified pinout of the FSA into the one specific for a particular development board.

Our customer projects often have us develop and improve Linux (or Zephyr!) drivers for various peripheral devices, including especially - due to our focus on vision - camera sensors. FRAMOS cameras are no exception here, and thanks to their popularity we work with them on a regular basis. Covering the entire vertical stack from the low level tweaking of the camera capture to AI algorithm development, as well as building our own developer-oriented tools, helps us get the most out o for edge AI processing platforms, which we benchmark and optimize using our Kenning framework and test using our Renode simulation environment and CI.

Antmicro’s FRAMOS Camera Adapter and interposer

To make FRAMOS camera modules compatible with Antmicro’s open hardware video processing device portfolio (and their custom derivatives we build for our customers), we developed a set of camera adapters. The camera-facing side has been implemented with a FRAMOS CSI Adapter. This small and lightweight board connects to the FRAMOS Sensor Module Adapter (FSA) and exposes a 30-wire I-PEX micro coax connector on the other side.

The I-PEX connector can be optionally replaced with an FFC. Additionally, the FRAMOS CSI Adapter includes circuitry which makes it electrically compatible with the MIPI camera interfaces exposed on Antmicro’s platforms.

Thanks to its compact size, on-board clocking and power management, the adapter is suitable for space-constrained devices. Another important feature is the I-PEX connector for compact-sized MIPI CSI-2 connection towards the processing platform, which is useful in drone applications with limited space available and harsh electromagnetic environment, as compared to flat cables (FFCs).

The FRAMOS CSI Adapter board is the same size as the FSA and can be stacked with it, which simplifies mechanical integration of the camera holder. The FRAMOS CSI Adapter can be connected to the Dual I-PEX CSI Interposer, which in turn converts the 30-pin camera interface from the FRAMOS Camera Adapter to a 50-pin camera interface implemented in most of the video processing platforms developed by Antmicro. In such a scenario, a single 50-pin camera interface connector can handle up to two FRAMOS imagers connected with up to 4-lane MIPI CSI-2.

The interposer’s 50-pin FFC connector is compatible with a variety of video processing boards created by Antmicro, such as the most recent Jetson Orin Baseboard.

With the recent release of NVIDIA’s Orin developer kit and the new SoM family, Antmicro has also updated its Jetson AGX CSI adapter, enabling the use of a variety of video-related compatible accessories, including the FRAMOS Camera Adapter, as well as drivers we have been developing for various customer projects.

Open hardware for simplified FRAMOS cameras integration

The FRAMOS Camera Adapter is yet another addition to Antmicro’s vast portfolio of open source video processing accessories and platforms providing comprehensive and customizable solutions for customers across different industries and use cases. The open hardware camera adapters allow connecting FRAMOS camera modules of various kinds to the video processing platforms developed by Antmicro. This expands the ecosystem of compatible video accessories and simplifies the early sensor integration and evaluation process for a Proof-of-Concept phase.

Antmicro offers software and hardware development services including driver porting, integration and building technology demonstrators that combine advanced, AI-capable processing platforms with multiple imagers. If you use or want to use FRAMOS cameras and are looking for help with hardware integration and driver development, or are interested in building smart camera solutions for robots, drones or monitoring systems, reach out to us at contact@antmicro.com.