89
edits
Changes
m
TBD=== <big>Version 1</big> ===
Inference Prediction times
!Notes
({| class="wikitable" style="margin: auto;"In addition, this document compares the results achieved to the ones produced by the platforms that were considered in the |+Prediction times!Model!Threads parameter!Prediction time[[MLms]|-| rowspan="3" |'''Floating-point'''|unspecified|90|-|1|160|-TN|2|130|-001 |'''Half- AI at the edge: comparison of different embedded platforms quantized'''|unspecified|180|- Part 1#Articles in this series|previous articles of this series]]rowspan="2")|'''Fully-quantized'''|unspecified|90|-|4|30|} == Results comparison ==TBD
no edit summary
== Results ==
The following table lists the prediction times for a single image depending on the model and the thread parameter.
{| class="wikitable" style="margin: auto;"
|+
!Model
!Threads parameter
!Inference Prediction time
[ms]
|-
| rowspan="3" |'''Floating-point'''
|unspecified
|220|90
|-
|1
|220|160
|-
|2
|390|130
|-
|'''Half-quantized'''
|unspecified
|330|180
|-
| rowspan="2" |'''Fully-quantized'''
|unspecified
|200|Four threads are created beside the main process (supposedly, this quantity is set accordingly to the number of physical cores available). Nevertheless, they seem to be constantly in sleep state.90
|-
|4
|80|Interestingly, 7 actual processes are created beside the main one. Four of them, however, seem to be constantly in sleep state.30
|}
The prediction time '''takes into account the inference time and the time needed to fill the input tensor with the image'''. Furthermore, it the inference time is averaged over several predictionsinferences.
The same tests were repeated using a network file system (NFS) over an Ethernet connection, too. No significant variations in the prediction times were observed.
In conclusion, to maximize the performance in terms of execution time, the model has to be fully-quantized and the number of threads has to be specified explicitly.
=== <big>Version 2</big> ===
In this case, only the fully-quantized model could be tested
