89
edits
Changes
m
TBD indicare versioni BSP, TF The kernel and the root file system of the tested platform were built with the L4.14.98_2.0.0 release of the Yocto Board Support Package for i.MX 8 family of devices. They were built with support for [https://www.nxp.com/design/software/development-software/eiq-ml-development-environment:EIQ eIQ]: "a collection of software and development tools for NXP microprocessors and microcontrollers to do inference of neural network models on embedded systems".
TBDTo run the model on the target, a new C++ application was written. After debugging this application on a host PC, it was migrated to the edge device where it was natively built. The root file system for eIQ, in fact, provides the native C++ compiler as well.
no edit summary
=== Environment ===
==Model deployment==
The application uses OpenCV 4.0.1 to pre-process the input image and TensorFlow Lite (TFL) 1.12 as inference engine. The model, originally created and trained with Keras of TensorFlow (TF) 1.15, was therefore converted into the TFL format. Then, the same model was recreated and retrained with Keras of TF 1.12. This allowed to convert it into TFL with post-training quantization of the weights without compatibility issues with the target inference engine. After that, it was also recreated and retrained with quantization-aware training of TF 1.15. In this way, a fully quantized model was obtained after conversion. So, at the end, three converted models were obtained: a regular 32 bit floating point one, an 8 bit half quantized (only the weights, not the activations) one, and a fully quantized one. The following images show the graphs of the models before and after conversion. TBD{ [ML - Keras1.15_fruitsmodel.png] [ML - TFL_float_fruitsmodel.png] [ML - Keras1.12_fruitsmodel.png] [ML - TFL_halfquant_fruitsmodel.png] [ML - TF1.15QAT_fruitsmodel.png] [ML - TFL_QAT_fruitsmodel.png]} ==Bulding and running Running the application=={In order to have reproducible and reliable results, some measures were taken:
* The inference was repeated several times and the average execution time was computed
* All the files required to run the test—the executable, the image files, etc.—are stored on a tmpfs RAM disk in order to make file system/storage medium overhead neglectable.}-------------------> ?The following blocks show the execution of the classifier on the embedded platform. With the floating point model: <pre class="board-terminal">root@imx8qmmek:~/devel/image_classifier_eIQ# ./image_classifier_cv 2 my_converted_model.tflite labels.txt testdata/red-apple1.jpg Number of threads: undefinedWarmup time: 233.403 msOriginal image size: 600x600x3Cropped image size: 600x600x3Resized image size: 224x224x3Input tensor index: 1Input tensor name: conv2d_8_inputSelected order of channels: RGBSelected pixel values range: 0-1Filling time: 1.06354 msInference time 1: 219.723 msInference time 2: 220.512 msInference time 3: 221.897 msAverage inference time: 220.711 msTotal prediction time: 221.774 msOutput tensor index: 0Output tensor name: IdentityTop results: 1 Red Apple 1.13485e-10 Orange 5.58774e-18 Avocado 7.49395e-20 Hand 1.40372e-22 Banana</pre>
== Results ==