Multi camera systems with Epson S2D13P04 and Toradex Nvidia Tegra T30

Seeing the growing interest in our series of posts on vision systems spanning from low-cost analog video chips to high-end 4K filmmaking cameras that we have been working with, we decided to showcase yet another interesting use case and open source the related code.
This time we focus on the Epson S2D13P04 – an integrated chip designed for multiple camera systems.
As in many previous projects, we have created a driver that allows you to use it with an Nvidia Tegra embedded CPU, the popular T30 featuring a quad-core Cortex-A9 CPU.
To learn more about our partnership with Nvidia and services for Nvidia embedded solutions, especially in the cameras and vision field, see the relevant section on our website.

The S2D13P04 allows you to connect up to 4 analog PAL/NTSC cameras, simultaneously decoding the image and outputting the stream in various modes:

• all four input video streams on one output image (Merge Mode),
• single stream mode (Fixed Mode),
• automatically switching between input sources (Auto Scan Mode), and
• streaming sources in pairs and switching to the next pair (Compression Mode).

The chip can also convert the image from interlaced to progressive (480p) mode, so you do not need to struggle with the CPU-consuming conversion yourself.

The setup we were working with includes an internally designed adapter board for the Colibri and Apalis development/evaluation boards for T30 modules from our partner Toradex. As in many cases in our long-term cooperation with Toradex, we have integrated the driver with their official Linux git repository.

To get the S2D13P04 to work, follow the instructions below:

1. First, clone the latest kernel sources from the Toradex git repository:

   git clone git://git.toradex.com/linux-toradex.git -b tegra-next
   cd linux-toradex/
   

2. Configure the kernel so that you can build the S2D13P04 driver as a module:

   export ARCH=arm
   # modify this according to your host system setup:
   export CROSS_COMPILE=arm-linux-gnueabihf-
   make colibri_t30_defconfig
   make menuconfig
   

   You have to manually enable SoC camera support, Tegra soc_camera host driver and s2d13p04 support as modules. You can find them in Device Drivers -> Multimedia support -> Video Capture adapters

3. Now it’s time time to compile the kernel and modules:

   make uImage
   make modules
   

4. Let us assume you have a running Colibri T30 board, connected to your local network. Copy your newly built modules and the uImage to the board’s internal memory using scp:

   scp arch/arm/boot/uImage root@<board_IP_address_goes_here>:/media/boot/
   scp drivers/media/video/*.ko root@<board_IP_address_goes_here>:
   

5. Finally, restart the board, connect all the cables, and load the modules:

   insmod videobuf-core.ko
   insmod videobuf2-core.ko
   insmod videobuf2-memops.ko
   insmod videobuf2-dma-nvmap.ko
   insmod soc_mediabus.ko
   insmod soc_camera.ko
   insmod s2d13p04.ko
   insmod tegra_v4l2_camera.ko
   

   and check how it works:

   gst-launch v4l2src ! nvvidconv ! nvxvimagesink
   

You should now see the video stream divided into four sub-streams showing the input feed from the cameras (“Merge Mode” is currently the only mode we support). More detailed information about the S2D13P04 chip can be found at the Epson website.

If you would like to seek our help in developing your next multi-camera product, or have any inquiries about our embedded software development or PCB design services, do not hesitate to contact us at contact@antmicro.com.