MISC-TN-012: Secure over-the-air (OTA) software update of IoT gateway fleet

Info Box Applies to SBCSPG Applies to SBC Lynx Technical Notes Applies to IoT Applies to IoT Services Technical Notes

This technical note was validated against specific versions of hardware and software. It may not work with other versions.

History

Version	Date	Notes
1.0.0	June 2020	First public release

Introduction

In the last years, we have witnessed the dawning of the IoT age: billions of independent embedded computing devices capable of communicating over a network without requiring human-to-human or human-to-computer interaction have spread all over the world. The term "thing" designates a wide variety of systems, ranging from medical devices and autonomous vehicles to smart sensors and wearable technologies (just to give a few examples). In other words, both consumer and industrial electronics markets have been flooded/involved by this new technological trend. When it comes to the industrial context, a typical IoT ecosystem consists of sensors that gather data from the environment and actuators ready to intervene according to the acquired information. The collected data can be shared between devices inside the network through a so-called edge device such as an IoT gateway.

The old-fashioned industry is shifting towards IoT at a fast pace with the aim of monitoring and optimizing their production processes, thus the diffusion of edge devices is becoming increasingly widespread, thanks also to the recent introduction of the edge computing paradigm. Understandably, this scenario poses serious security issues concerning the deployed systems and the data they handle: the risk that confidential information can be stolen increases as the sharing becomes deeper, a bug or a vulnerability can be exploited to tamper with every device inside a network potentially causing significant economic losses or endangering people’s safety. Security has been considered a worthless design cost for a long time; therefore, negligible endeavors were put in this sense.

In recent times, however, hardware manufacturers and software companies dealt with this lack of sensibility against cyber threats: a novel awareness arose, exemplified by the principle of security by design. The demand for such secure systems is thus increasing, despite the greater complexity that characterizes their management. In general, the hardware and software mechanisms as well as the procedures that must be implemented to set up a secure IoT infrastructure depend on company's needs and assets: which kind of data do they want to protect? How valuable is the information they manage? Which types of devices are they using? Which are the most suitable security policies for their business? Nevertheless, lists of essential best practices to help secure IoT systems have been outlined by authorities such as the IoT Security Foundation. These guidelines, although primarily aimed at designers and developers, can be useful for end-users as well; for instance, they provide insights concerning physical security (protect device’s hardware resources from damage or tampering), credential management (handle and update passwords, encryption keys, and digital certificates in a proper way), encryption (guarantee data privacy, integrity, and authenticity). Many devices aren’t obviously able to satisfy every requirement due to their technical limitations: in such cases, the trade-offs between downgrades and resulting risks must be carefully considered. Among the cornerstones of IoT security, software update plays a critical role for various reasons:

• Unlike one-time configurations such as those concerning network connections, software’s update is a continuous process which may last for years; therefore, it requires constant effort, planning, and technical expertise.
• It can enhance the device’s original capabilities and allows the introduction of new functionalities.
• It’s essential to fix bugs and patch vulnerabilities.
• Every software layer is involved, from the kernel up to user applications.
• Software bugs and power cuts occurring during an update can brick the device, making it unusable.

Another major challenge to tackle is the updates’ distribution among all the deployed systems: the installation is often done manually by an operator, thus requiring physical access to the devices. This approach has three main downsides: first, there are cases where the systems’ working environment makes it difficult to reach in practice; secondly, it’s onerous both in time and economic terms (operator must be on-site and performs the updates one by one); last but not least, it lacks scalability (what if a company has thousands of devices?). Hence the need for an automated, over-the-air (OTA) update approach which allows handling the issue remotely in a parallel way, possibly by means of a client-server architecture, without sacrificing security and reliability.

This Technical Note (TN) shows how to implement a secure OTA software updating system for DAVE Embedded Systems' Industrial IoT gateway named SBCSPG satisfying the aforementioned requirements.

Overall system architecture

The following picture illustrates the overall system architecture of the solution.

Block diagram of the overall system architecture

Basically, it consists of two pieces of software:

• hawkBit running on the cloud side (version 0.3.0 was used)
• SWUpdate running on the edge device (version v2020.04.0 was used).

Gateway and server configurations

Regarding the gateway's software configuration, it resembles the one described here. In this case, however, a symmetric update scheme was adopted. As such, the resulting non-volatile memory partitioning is a little bit different. It is shown in the following image.

For the sake of simplicity, the first chosen update mechanism aims to update the Linux kernel image (along with the associated device tree blob) and the root file system as a whole. This approach is called monolithic update.

Symmetric update scheme

Symmetric updates rely on two mirror copies, each comprising Linux kernel and root filesystem in this specific case. They are labeled as A and B in the figure. If A is running, an incoming update overwrites B. Then the system reboots, switching to B. The two copies swap their roles after every successful update. Clearly, the system continues to run the same copy whenever a failure occurs: in any case, the device’s integrity is ensured. The most pleasing benefit of symmetric updates is that they allow the device to continue executing its usual tasks since the inactive copy is updated separately in the background. Furthermore, only the minimal downtime of a single reboot is required. There are two downsides as well: the storage space needed for a redundant copy is doubled compared to the asymmetric scheme and the large size of the update packages can put a strain on the network infrastructure. The latter can be mitigated by sending only the file-system blocks that have changed with respect to the previous version (binary delta update).

SWUpdate was built natively and it runs on the gateway as a daemon.

With regard to the hawkBit server, a virtual machine running the Lubuntu 18.04 distro was used.

Testing

The following image shows the messages printed by SWUpdate when started.

SWUpdate running on SBCSPG

The following picture illustrates a screenshot of the hawkBit's console. This interface allows to configure the update policies for the entire fleet of devices and to monitor the past, current, and scheduled roll-out operations. Several options are available, for example for defining different policies on a geographical basis.

Screenshot of the hawkBit's console

This specific screenshot shows several attempts to deploy the software to the target. Some of them were completed successfully, some were not. For instance, to verify the capability of the target to resume an interrupted updating, the gateway was turned off abruptly in the middle of downloading or flashing.

To test the overall remote update mechanism, appropriate images have to be prepared. Such images include a file listing platform-dependent operations required to apply the update on the device side. This file is processed by SWUpdate that carries out such operations.