ML-TN-001 - AI at the edge: comparison of different embedded platforms - Part 4

Info Box Applies to Machine Learning

Work in progress

History

Version	Date	Notes
1.0.0	September 2020	First public release

Introduction

This Technical Note (TN for short) belongs to the series introduced here. In particular, it illustrates the execution of different versions of an inference application (fruit classifier) that makes use of the model described in this section, when executed on the NXP i.MX8M Plus EVK. In addition, this document compares the results achieved to the ones produced by the platforms that were considered in the previous articles of this series.

Specifically, the following versions of the application were tested:

• Version 1: This version is the same described in this article. As such, inference in implemented in software and is applied to images retrieved from files.
• Version 2: This version is functionally equivalent to the version 1, but it leverages the Neural Processing Unit (NPU) to hardware accelerate the inference.
• Version 3: This is like version 3, but the inference is applied to the frames captured from an image sensor.

Test Bed

The kernel and the root file system of the tested platform were built with the L4.14.98_2.0.0 release of the Yocto Board Support Package for i.MX 8 family of devices. They were built with support for eIQ: "a collection of software and development tools for NXP microprocessors and microcontrollers to do inference of neural network models on embedded systems".

The following table details the relevant specs of the test bed.

NXP Linux BSP release	L4.14.98_2.0.0
Inference engine	TensorFlow Lite 1.12
Maximum ARM cores frequency [MHz]	1300
SDRAM memory frequency (LPDDR4) [MHz]	1600
Governor	ondemand

Building and running the applications

As stated in the first article of this series, one of the goals is to evaluate the performances of the inference applications. As known, before and after the execution of the inference, other operations, generally referred to as pre/post-processing, are performed. Technically speaking, these operations are not part of the actual inference and are measured separately.

In order to have reproducible and reliable results, some measures were taken:

• When possible, the inference was repeated several times and the average execution time was computed
• All the files required to run the test—the executable, the image files, etc.—are stored on a tmpfs RAM disk in order to make file system/storage medium overhead neglectable.