Changes

After performing the quantization with the vai_q_tensorflow tool and after the deployment on the target device, the model has an overall value of '''''accuracy of 88.87%''''' and an overall weighted average '''''F1-score of 88.91%''''' on the test subset of the dataset. The model is still performing well in correcly classify samples belonging to ''resistor'' class (97.65% F1-score). On the other hand , for the remaining classes, there is a substantial reduction in the value of this metric. The classes that exhibit the worst results are ''diode'' class (85.15% F1-score), ''IC'' (83.27% F1-score) , and, ''transistor'' class (81.97% F1-score). In general, the performance of the model is still good, but it is decidedly definitely lower than the one obtained with the ResNet models analyzed previously.

After performing the quantization with the ''vai_q_tensorflow tool'' and after the deployment on the target device, the model has an overall value of '''''accuracy of 93.34%''''' and an overall weighted average '''''F1-score of 93.34%''''' on the test subset of the dataset. The model is still performing very well in correcly classify samples belonging to ''resistor'' class (97.12% F1-score), ''inductor'' class (97.00% F1-score) , and, ''capacitor'' class (96.59% F1-score) by keeping a F1-score above 96.00%. However, for the remaining classes, the value of the metric is substantially reduced. The classes that exhibit the worst results are ''IC'' class(89.41% F1-score) because of a low value measured for precision metric (84.12% precision) , and, ''transistor'' class (87.75% F1-score) because of a very low value of the recall metric (82.80% recall). In general, the performance of the model is still good, similar to the one obtained with ResNet models.

The model, before performing the quantization with the ''vai_q_tensorflow'' tool, has an overall value of '''''accuracy of 97.53%''''' and an overall weighted average '''''F1-score of 97.53%''''' over the test subset of the dataset, showing a very high generalization capability on unseen samples. Five classes have a F1-score above 96.00%, actually very high for ''inductor'' class (98.66% F1-score) , and, ''resistor'' class (98.55% F1-score). The worst result is the one displayed by the ''transistor'' class by having a F1-score below 96.00% but, still very close (95.86% F1-score) mainly due to a low value of the precision metric (93.36% precision).

After performing the quantization with the ''vai_q_tensorflow'' tool and after the deployment on the target device, the model has an overall value of '''''accuracy of 93.34%''''' and an overall weighted average '''''F1-score of 93.34%''''' on the test subset of the dataset. The model is still performing very well in correcly classify samples belonging to ''resistor'' class (98.07% F1-score) and, ''capacitor'' class (96.23% F1-score) by keeping a F1-score above 96.00%. However, for the remaining classes, the value of the metric is reduced. In particular the worst results can be found in the ''IC'' class (90.80% F1-score) by having a low value of precision and recall metrics (91.73% precision, 89.90% recall) , and, ''transistor'' class by haveing having a low value of precision metric (87.88% precision).

After reviewing all the created models, showing their performances in terms of accuracy and other classification metrics such as precision, recall, and F1-score, and after evaluating DPU usage/latency for a single inference over the test samples, a comparison between them should can be made. The aim is to understand if there exists one model that can be considered the best to be used for solving the problem among between the proposed ones.

Since the original dataset was augmented to compensate for the lack of data, hence resulting in a balanced dataset with the same number of samples for each of the six classes, metrics such as precision, recall , and, F1-score can be omitted and only the accuracy can be taken into account. Note that, the accuracy of a model can actually be enhanced by furtherly further tweaking the training hyperparameters or simply by training the model for a higher number of epochs. Thus, the value of this metric can actually be higher (or even lower in case of overfitting) than the one obtained for this particular configuration (all the models were trained using the same configuration).

Therefore, to proceed with the evaluation, these the followgin features must be taken into account for a better understanding of the whole situation: * accuracy pre and post quantization, * DPU Kernel parameters size and total tensor count, * DPU cores latency, and finally * DPU throughput. 

*'''Parameters size''': indicates in the unit of MB, kB, or bytes, the amount of memory occupied by the DPU Kernel, including weight and bias. It is straightforward to check that the greater the number of parameters for an implemented model on the host, the greater the amount of memory occupied on the target device.

In the two figures below it is shown the DPU cores latency for 1, 2, and 4 threads; it . It is interesting to note that the core latency of Inception ResNet V1 is lower than the one of ResNet152, even though they have similar ''total tensor count'' and different values of DPU Kernel ''parameters size'' (actually greater for ResNet152). Vice versa, the ResNet101 and Inception V4 have a similar DPU Kernel ''parameters size'' and different values of ''total tensor count'', and , in this case, the core latency is higher for the latter. The same observation can be made for models ResNet50 and Inception ResNet V1 leading to the following statements:

Finally, it is possible to evaluate the DPU throughput in relation to the number of threads used by the benchmark application. In the figure below, it is really interesting to observe how all the models for , in the case of 1 thread, have similar values of FPS , but by increasing the level of concurrency the difference is more and more evidentwhen increasing the level of concurrency.

In conclusion, by summing up all the considerations that have been made, it is clearly evident that the solution with the best compromise between accuracy and inference latency is the ResNet50 model, followed by the ResNet101 , and Inception ResNet V1 models.