YUV cameras on Jetson TK1

The multimedia capabilities of the Nvidia Tegra K1 CPU make it a powerful platform for image and video processing, especially as it is the first embedded platform including straightforward, integrated support for the CUDA architecture. Since vision systems and video are some of Antmicro’s key focus areas, we have been working with the K1 platform since it first appeared in the market, as a continuation of our involvement with the Nvidia Tegra CPU line which is very popular with our customers, especially those on the higher performance spectrum.

TK1’s computing power and vision processing abilities enable a wide range of previously less feasible applications, from navigating mobile robots, recognizing and understanding sign language to intelligent ovens.

Jetson TK1 is currently the most popular development board with a Tegra K1 CPU. Today we will look at how to interface cameras with Jetson, as a practical example of a platform that can be used for prototyping camera-related embedded projects for which you may want to use the K1 CPU.

Apart from a variety of USB connected cameras (typical for consumer applications), on an embedded platform you can use a MIPI CSI-2 interface which is preferred for transferring the image from sensor to processor. The Jetson TK1 board lets you connect up to two MIPI CSI-2 cameras (one 4-lane and one single-lane) using the GPIO expansion port, so we’ve made an adapter allowing you to connect cameras using dual RJ-45 connectors (like Omni Vision OV5640, which was described in one of our previous blog posts).

For our and our customers’ R&D purposes we focused on developing Linux drivers for the following YUV sensors/chips:

• Omni Vision OV5640, 5Mpx sensor, 1080p@30fps
• Analog Devices ADV7280M, analog camera decoder with deinterlacer, PAL/NTSC@60fps
• ON Semiconductor (formerly Aptina) AP1302 ISP + AR1820 image sensor, 13Mpx, 1080p@30fps
• Toshiba TC358743 HDMI to CSI-2 bridge, 1080p@60fps

The drivers listed above are still under development, hence their performance may not be perfect – e.g. the TC358743 works only in 640×480@30fps mode (see commit messages).

Below we present a brief tutorial on how to download and compile the up-to-date camera drivers we are currently working on.

We assume that you have a Jetson TK1 running the newest Linux For Tegra release from Nvidia.

Start by cloning our git repo:

git clone https://github.com/antmicro/linux-tk1.git -b YUV_cameras
cd linux-tk1/

Prepare the build environment:

export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabihf-
make tegra12_defconfig
make menuconfig

Find the camera drivers in Device Drivers –> Multimedia support –> Sensors used on soc_camera driver and mark them as modules. In case you are using OV5640, remember to disable the other drivers for the same chip (find it under: Device Drivers –> Multimedia support –> V4L platform devices).

Now you are ready to build the kernel image and the modules:

make -j$(nproc)
make -j$(nproc) modules

Meanwhile, power up the board and configure sshd to allow you to login remotely as root and set the root password.

Next, copy your newly built modules (let’s assume that the IP address of your Jetson TK1 is 192.168.1.24):

scp arch/arm/boot/zImage root@192.168.1.24:/boot/
scp $(find drivers/media/ -iname "*.ko") root@192.168.1.24:

Now switch to the Jetson, open a console with root privileges, and type:

rmmod nvhost_vi   # if it is already loaded
insmod videobuf2-dma-contig.ko
#choose ONE of the following:
#insmod ap1302.ko
#insmod ov5640.ko
#insmod tc358743.ko
insmod adv7280.ko
insmod tegra_camera.ko

Now, test the cameras (this command works with OV5640 – modify it accordingly for the other sensors):

gst-launch-0.10 -v v4l2src queue-size=1 ! 'video/x-raw-yuv,format=(fourcc)UYVY,width=1920,height=1080' ! xvimagesink

Enjoy – and let us know about your new products with the K1 CPU. Be sure to connect with us at contact@antmicro.com if you have any comments, ideas or questions!