Google’s recently released Gemma 4 family of models offers a powerful yet efficient solution for a variety of generation tasks (question answering, document analysis, summarization, reasoning, coding, sensor data analysis). These diverse models can operate on platforms ranging from data centers down to embedded devices. Gemma 4 has been released with a fully open source Apache 2.0 License, and came with Day 0 support across different runtimes (such as llama.cpp and vLLM) and platforms, including NVIDIA’s Jetson Orin and Jetson Thor Systems on Modules.
Some time ago, Antmicro released an open hardware baseboard for NVIDIA Jetson AGX Thor that has already been tested with multiple computer vision models running in parallel. Recently, we ran the Gemma 4 31B model in the largest available BF16 format, the balanced 26B A4B version, and the E4B Q4 variant on our baseboard with the Thor SoM, and measured their performance.
With this article we present an open source setup comprising the Antmicro baseboard for NVIDIA Jetson AGX Thor with the NVIDIA Jetson Thor SoM and the powerful Gemma 4 models, along with an evaluation report generated with our Kenning framework.
Open source setup with Antmicro Baseboard for NVIDIA Jetson AGX Thor and Gemma 4
The Gemma 4 models come in variants dedicated for different applications:
- Effective 2B (E2B) and 4B (E4B) suitable for mobile and edge devices,
- 12B, a recently released multimodal solution,
- 26B Mixture of Experts (MoE) offering advanced processing with high efficiency,
- 31B Dense, the most accurate, server-grade variant.
Antmicro’s open hardware baseboard for NVIDIA Jetson AGX Thor break-routes common I/O interfaces for standard desktop usage, and allows for potential customization for use in embedded products. It provides two 50-pin FFC connectors exposing four 4-lane CSI ports from the SoM with independent I2C configuration buses per each CSI port. With a PCB outline of just 100 x 180 mm (3.93 x 7.09 inch), it’s suitable for embedded and size-constrained devices. The baseboard is also compatible with various camera modules and video accessories released by Antmicro that can be used to develop complex devices, for example built around the NVIDIA Jetson AGX Thor Dual GMSL Setup or the Antmicro Baseboard for NVIDIA Jetson AGX Thor + SDI to MIPI Converter Setup. Since our baseboard supports hardware acceleration for devices using Vision Language Models (VLMs), it can be combined with CSI and GMSL accessories to build advanced vision systems with VLM AI services.
Besides various hardware platforms that serve as a starting point for building custom devices, Antmicro develops Kenning, an open source set of tools for optimizing and deploying AI models on embedded platforms. Kenning supports a wide range of platforms running Linux, Zephyr RTOS and bare metal applications, and is capable of generating detailed performance reports to help us find and tailor optimal AI solutions for our customers. We’ve already employed Kenning for benchmarking and optimizing AI models on NVIDIA Jetson AGX Thor, which we described in detail in a previous article.
Benchmarking Gemma 4 on Antmicro’s baseboard with NVIDIA Jetson AGX Thor SoM
The benchmark reports were generated with Kenning, using the llama.cpp for LLM runtime. We tested Gemma 4 31B as the model providing the best quality (while still fitting in the NCIDIA Jetson AGX Thor memory), Gemma 4 26B A4B which offers the best trade-off between quality and performance, and the fast Gemma E4B Q4. All Gemma 4 variants can be run on our open hardware baseboard with the NVIDIA Jetson AGX Thor SoM out of the box, without any modifications. Selected results are shown in the interactive diagrams below:
For an interactive version of the diagram, visit the desktop version of the website 
For an interactive version of the diagram, visit the desktop version of the website 
For an interactive version of the diagram, visit the desktop version of the website Complete product development with Gemma 4 and NVIDIA Jetson AGX Thor
The open source Gemma 4 models and Antmicro’s open hardware baseboard, combined with the powerful NVIDIA Jetson AGX Thor SoM, provide a flexible solution for developing advanced AI-enabled systems. With a broad portfolio of open hardware designs targeting the NVIDIA Jetson family and experience in developing complete AI solutions, Antmicro can help you harness the power of Google’s Gemma 4 models. Using our Kenning framework, we aid our customers in training, optimizing and deploying AI models for specific problems and target platforms.
If you would like to learn more about Antmicro’s engineering services and open source tools, don’t hesitate to contact us at contact@antmicro.com.
