View source for SBCX-TN-005: Using TensorFlow to implement a Deep Learning image classifier based on Azure Custom Vision-generated model

{{InfoBoxTop}}
{{AppliesToAxel}}
{{AppliesToAxelEsatta}}
{{AppliesToAxelLite}}
{{AppliesToAXEL Lite TN}}
{{AppliesToSMARX TN}}
{{AppliesToSBCX}}
{{AppliesToSBC Axel TN}}
{{AppliesToIoT}}
{{AppliesTo IoT TN}}
{{AppliesToMachineLearning}}
{{AppliesTo Machine Learning TN}}
{{InfoBoxBottom}}
{{WarningMessage|text=This technical note was validated against specific versions of hardware and software. It may not work with other versions.}}
== History ==
{| class="wikitable" border="1"
!Version
!Date
!Notes
|-
|1.0.0
|October 2019
|First public release
|}

==Introduction==
Nowadays, Machine Learning (ML) and Deep Learning (DL) technologies are getting popular in the embedded world as well. Several different approaches are available to deploy such technologies on embedded devices. This Technical Note (TN) describes such an approach, which makes use of a Tensor Flow model generated with [https://www.customvision.ai Microsoft Azure Custom Vision service].

==Testbed basic configuration==
The testbed consists of an [[:Category:SBC-AXEL|SBCX Single Board Computer]] equipped with an i.MX6Q-powered [[:Category:AxelLite|Axel Lite]] system-on-module (SoM).

Regarding the operating system, the board runs Armbian Buster GNU/Linux distribution, which is described in [[SBCX-TN-004:_Running_Armbian_Buster_(Debian_10)|this TN]].

==Test application==
The test application is a classical image classifier. The following classes are supported:
<pre>
Avocado
Banana
Green Apple
Hand
Orange
Red Apple
</pre>


The following image shows the application's architecture.


[[File:SBCX-image-classifier.png|thumb|center|600px|Test application's architecture]]


It mainly consists of the following blocks:
* The top-level application code (Python)
* The TensorFlow platform
* The TensorFlow model
* The OpenCV library.

As stated in the introduction, the classifier is based on a model that was generated with Azure Custom Vision. In particular, the model was retrieved from [https://dev.to/azure/creating-an-image-recognition-solution-with-azure-iot-edge-and-azure-cognitive-services-4n5i this project] by [https://dev.to/gloveboxes Dave Glover]. Glover's project is extremely useful to understand how Custom Vision—and, in general, Azure Cognitive Services—work. It is worth remembering that '''no particular Machine Learning-related skills are required to create such a model'''.

Glover's project follows the approach suggested by Azure, which makes use of containers. For the sake of simplicity, this Technical Note is based on a simpler strategy, which is closer to the usual approach used in the embedded world. As such, it doesn't make use of any container.

Once the TensorFlow model is deployed on the SBCX, the classifier can work without any Internet connection. In other words, the SBCX can perform the classification task autonomously.

TensorFlow release is 2.0.0. It was built with the proper configuration for the i.MX6 SoC. For more details, please refer to [https://github.com/lhelontra/tensorflow-on-arm this project] by [https://github.com/lhelontra Leonardo Lontra].

OpenCV library was installed by using the standard pre-built package provided by the distribution.

Last but not least, the application code is based on [https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/export-model-python this example].

==Performances==
The following box shows the output of the application while classifying an image that contains a red apple:
<pre class="board-terminal">
$ python3 image-classifier.py 
2019-10-25 11:17:15,288 - DEBUG - Starting ...
2019-10-25 11:17:15,289 - DEBUG - Importing the TF graph ...
Classified as: Red Apple
2019-10-25 11:17:21,591 - DEBUG - Prediction time = 2.567471504211426 s
Avocado 2.246000076411292e-05
Banana 3.769999921132694e-06
Green Apple 0.029635440558195114
Hand 4.4839998736279085e-05
Orange 0.0009084499906748533
Red Apple 0.9693851470947266
2019-10-25 11:17:21,594 - DEBUG - Exiting ...
</pre>
The image was classified as a "Red Apple" because the probability associated with this class was by far the highest (almost 97%).
[[File:Red-apple.jpg|thumb|center|300px|The image shown in the previous example]]



During the execution of the test application, the status of the processes and the ARM cores was observed with the help of the <code>htop</code> tool.


[[File:SBCX-image-classifier-1.png|thumb|center|600px|<code>htop</code> during the execution of the test application]]


By default, the scaling governor is set to interactive:
<pre class="board-terminal">
root@sbcx:~# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

interactive
</pre>
Therefore, during the execution of the program, the cores' frequency was scaled to 1 GHz as expected.


At first glance, it seemed that the TensorFlow platform does not exploit the available cores to implement significant parallel computing. To verify this guessing, the test was repeated by limiting the number of cores. The results are listed in the following table.

{| class="wikitable"
|+
!Platform
!# of cores
!Governor
!Prediction time
[s]
|-
|SBCX (i.MX6Q)
|1
|interactive
|3.0
|-
|SBCX (i.MX6Q)
|2
|interactive
|2.6
|-
|SBCX (i.MX6Q)
|4
|interactive
|2.5
|-
|SBCX (i.MX6Q)
|4
|userspace
(1 GHz)
|2.5
|}

As shown, the prediction time doesn't change significantly with increasing number of cores. This should confirm TensorFlow doesn't implement a parallel computing engine on this architecture.