MISC-TN-011: Running an Azure-generated TensorFlow Lite model on Mito8M SoM using NXP eIQ
|This technical note was validated against specific versions of hardware and software. What is described here may not work with other versions.|
History[edit | edit source]
|1.0.0||March 2020||First public release|
|1.0.1||April 2020||Added TF model's graph|
|1.0.2||April 2020||Added TFL model's graph|
Introduction[edit | edit source]
This article combines the results shown in the TN's just mentioned. In other words, it describes how to run the same image classifier used in SBCX-TN-005 with the eIQ software stack. The outcome is an optimized C++ imaging classification application running on Mito8M SoM, which makes use of the eIQ software stack. In terms of hardware and software, the testbed used is the same described here.
Workflow and resulting block diagram[edit | edit source]
The following picture shows the block diagram of the resulting application and part of the workflow used to build it.
First of all, the TensorFlow (TF) model generated with Microsoft Azure Custom Vision was converted into the TensorFlow Lite (TFL) format.
Then, a new C++ application was written, using the examples provided by TFL as starting points. After debugging this application on a host PC, it was migrated to the edge device (a Mito8M-powered platform, in our case) where it was natively built. The root file system for eIQ, in fact, provides the native C++ compiler as well.
For the sake of completeness, the following images show the graphs of the original TF model and the converted TFL model (click to enlarge).
|Original TF model's graph||Converted TFL model's graph|
Running the application[edit | edit source]
The following block shows the execution of the classifier on the embedded platform:
root@mito8m:~/devel/image_classifier_eIQ# ./image_classifier_cv converted_model.tflite labels.txt testdata/red-apple1.jpg Original image size: 600x600x3 Cropped image size: 600x600x3 Resized image size: 224x224x3 Input tensor index: 0 Input tensor name: Placeholder Filling time: 25.3169 ms Inference time: 276.121 ms Total prediction time: 301.438 ms Output tensor index: 406 Output tensor name: model_outputs Top results: 0.997172 Red Apple 0.00214239 Green Apple
The prediction time is cut by about 88% compared to this implementation. Of course, this is due to several factors. The most relevant ones are:
- i.MX8M is faster than i.MX6Q
- The application is written in C++ and not in Python
- The TF model was replaced with a TFL model, which is inherently more suited for ARM-based devices
- The middleware provided by NXP eIQ is optimized for their SoC's.