Support for the CivetWeb HTTP server in Zephyr

HTTP support in Zephyr

Zephyr has always had a big advantage in the form of its custom-tailored networking stack. As the RTOS continued to grow, more and more networking applications were developed to be run on top of it. However, while the networking stack itself proved to be very useful for its original purpose - proving that Zephyr was a robust and stable choice for IoT devices - its custom nature was becoming a burden. As Zephyr finds its way into more and more use cases, not all of which are tiny and wireless, the decision to rally around existing standard networking APIs was becoming more obvious, and some time ago the decision was made to base upon the well-known BSD sockets API.

The biggest issue with switching to another networking API was the ensuing necessity to rewrite all the applications and libraries which had been using the previous API so that they do not break, as full backwards compatibility was not an option. To make the transition process manageable, the Zephyr networking team decided to temporarily drop support for multiple protocols, including HTTP.

Obviously, that was not an ideal situation, and as a Silver Member of the Zephyr project with a long history of contributions to it, Antmicro was approached by the Zephyr Project Networking community to bring the missing capabilities back fast, so that HTTP-based applications could continue to be built even in the transition period. There were severals ways to approach it, the most obvious ones being:

• doing what had already been done before, that is implementing our own HTTP support from scratch and tightly integrating it with Zephyr;
• implementing the HTTP support as an application/sample, allowing others to use our code as a starting point for their Zephyr applications that use HTTP;
• integrating an already existing third-party HTTP library with Zephyr.

Going the third-party route

As is our standard practice, we leaned towards reusing an existing library, and after a discussion with both our Client and in the broader forum we agreed that this route would be a valuable addition to the project. Zephyr is all about integrating with external libraries and frameworks, and one of the primary features of its helper meta-tool, West (yes, it’s a pun, in case you wondered), is multi-repo capability for pooling together code from various sources.

The third-party library route meant we could let the networking stack redesign and reimplementation proceed at its own pace, while we could fast-track to a fully-fledged implementation that had been proven to work before - and test how well Zephyr integrates with quite complex external libraries in the process. Another huge benefit of going that path was the possibility of testing the newly supported BSD sockets API - using a third-party library which had been working with that API for many years was a great way to verify the correctness and completeness of the Zephyr’s implementation.

An additional advantage here is that most HTTP libraries also rely on POSIX APIs, which Zephyr is working to be compliant with as well. The support for the POSIX APIs is still under development, but porting an external application which uses them can serve as a great starting point to improve Zephyr in that area.

CivetWeb turning out to be the best fit

After researching various open-source HTTP implementations, we decided that CivetWeb was the best candidate. Civetweb’s mission is to provide a permissively licensed, easy-to-use, powerful, C (C/C++) embeddable web server with optional CGI, SSL and Lua support. CivetWeb can be used as a library, adding web server functionality to an existing application, or it can work as a stand-alone web server running on Windows or Linux.

As it turned out, CivetWeb had everything we needed: it can work both as an HTTP client and HTTP server, it can be easily embedded into an already existing application, it can be used as a library and it is highly customizable so we could remove all the features we didn’t need, which made it easier to use it on resource-constrained devices that Zephyr is targetting. It uses both the BSD sockets API and the POSIX APIs, making it a great real-life test for Zephyr.

Making CivetWeb work with Zephyr

The project required our work on both ends. First we made it possible for CivetWeb to be compiled as a Zephyr library, by preparing a CMake configuration in CivetWeb so it could be included by the Zephyr buildsystem. We also enabled CivetWeb to work on OSes with no filesystem and added several Zephyr-specific modifications.
Then we added it to Zephyr as a West module. The final step was adding a simple sample application which could serve as a quick-start guide for other users.

We used the Microchip SAM E70 Xplained board for development and testing. Running the sample application on it results in the board serving an HTTP page at 10.0.0.111:8080 (or other, depending on the settings). It serves several URLs which are used to show various possibilities of the server (like serving static text, handling json requests or cookie usage). In addition to that, it can also be used to demonstrate handling of various HTTP errors (like 404 - not found).

It can be built like any other Zephyr sample, e.g. for the Atmel SAM E70 Xplained board, run:

west build -b sam_e70_xplained samples/net/sockets/civetweb

For more information about the sample refer to the README.

Tapping into Open Source

Zephyr is a popular, multi-purpose, security-focused and robust RTOS which owes its capabilities to active developers and code quality, as well as the open style of governance and flexibility. By turning to standard APIs used in the open-source world, Zephyr was able to harness the functionalities of numerous available software applications, making it even easier to build complex solutions that would not be feasible without the use of third-party libraries.
The ability to integrate with a very complex application like CivetWeb to provide HTTP implementation proves Zephyr’s modularity and versatility.

Antmicro has a long history of integrating great open source projects together - check out our recent work on combining TFLite with Zephyr.

If you have a project which could benefit from using Zephyr’s capabilities with third-party libraries, or are building a product which needs integrating many software components together, feel free to reach out to us at contact@antmicro.com.