dave_user, Administrators
5,166
edits
Changes
no edit summary
{{InfoBoxTop}}
{{AppliesToMachineLearning}}
{{AppliesTo Machine Learning TN}}
{{AppliesToMito8M}}
{{AppliesTo MITO 8M TN}}
{{InfoBoxBottom}}
==Introduction==
This Technical Note (TN for short) belongs to the series introduced [[ML-TN-001_-_AI_at_the_edge:_comparison_of_different_embedded_platforms_-_Part_1|here]].
Specifically, it illustrates the execution of an inference application (fruit classifier) that makes use of the model described in [[ML-TN-001_-_AI_at_the_edge:_comparison_of_different_embedded_platforms_-_Part_1#Reference_application_.231:_fruit_classifier|this sectioninference application (fruit classifier)]] when executed on the [[:Category:Mito8M|Mito8M SoM]], a system-on-module based on the NXP [https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-8-processors/i-mx-8m-family-armcortex-a53-cortex-m4-audio-voice-video:i.MX8M i.MX8M SoC].
=== Test bed ===
The kernel and the root file system of the tested platform were built with the L4.14.98_2.0.0 release of the Yocto Board Support Package for i.MX 8 family of devices. They were built with support for [https://www.nxp.com/design/software/development-software/eiq-ml-development-environment:EIQ eIQ]: "a collection of software and development tools for NXP microprocessors and microcontrollers to do inference of neural network models on embedded systems".
The following table details the relevant specs of the test bed. {| class="wikitable"|+!!!!style="margin: auto;"
|-
|'''NXP Linux BSP release'''
|L4.14.98_2.0.0
|-
|'''Inference engine'''
|TensorFlow Lite 1.12
|-
|'''Maximum ARM cores frequency''' '''[MHz]'''|1300|-|'''SDRAM memory frequency (LPDDR4)''''''[MHz]'''|1600|-|'''[https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt Governor]'''|ondemand
|}
==Model deploymentand inference application==
To run the model on the target, a new C++ application was written. After debugging this application on a host PC, it was migrated to the edge device where it was built natively. The root file system for eIQ, in fact, provides the native C++ compiler as well.
So, in the end, three converted models were obtained: a regular 32-bit floating-point one, an 8-bit half-quantized (only the weights, not the activations) one, and a fully-quantized one.
The following images show the graphs of the models before conversion (click to enlarge):
* All the files required to run the test—the executable, the image files, etc.—are stored on a tmpfs RAM disk in order to make file system/storage medium overhead neglectable.
The following blocks show sections detail the execution of the classifier on the embedded platform. With The [https://www.tensorflow.org/lite/performance/best_practices#tweak_the_number_of_threads number of threads] was also tweaked in order to test different configurations. During the floatingexecution, the well-point modelknow [https://en.wikipedia.org/wiki/Htop <code>htop</code>] utility was used to monitor the system. This tool is very convenient to get some useful information such as cores allocation, processor load, and number of running threads.
=== Floating-point model ===
<pre class="board-terminal">
root@imx8qmmek:~/devel/image_classifier_eIQ# ./image_classifier_cv 2 my_converted_model.tflite labels.txt testdata/red-apple1.jpg
</pre>
==== Tweaking the number of threads ====
The following screenshots show the system status while executing the application with different values of the thread parameter.
[[File:ML-TN-001 2 float default.png|thumb|center|600px|Thread parameter unspecified]]
[[File:ML-TN-001 2 float 1thread.png|thumb|center|600px|Thread parameter set to 1]]
[[File:ML-TN-001 2 float 2threads.png|thumb|center|600px|Thread parameter set to 2]]
=== Half-quantized model ===
<pre class="board-terminal">
root@imx8qmmek:~/devel/image_classifier_eIQ# ./image_classifier_cv 2 my_fruits_model_1.12_quant.tflite labels.txt testdata/red-apple1.jpg
[[File:ML-TN-001 2 weightsquant default.png|thumb|center|600px|Thread parameter unspecified]]
=== Fully-quantized model ===
<pre class="board-terminal">
root@imx8qmmek:~/devel/image_classifier_eIQ# ./image_classifier_cv 3 my_fruits_model_qatlegacy.tflite labels.txt testdata/red-apple1.jpg
1 Red Apple
</pre>
==== Tweaking the number of threads ====
The following screenshots show the system status while executing the application with different values of the thread parameter.
[[File:ML-TN-001 2 fullquant default.png|thumb|center|600px|Thread parameter unspecified]]
[[File:ML-TN-001 2 fullquant 4threads.png|thumb|center|600px|Thread parameter set to 4]]
== Results ==
