After calibration, the quantized model is transformed into a DPU deployable model (named as deploy_model.pb for vai_q_tensorflow) which follows the data format of a DPU. This model can be compiled by the Vitis AI compiler and deployed to the DPU. This quantized model cannot be used by the standard TensorFlow framework to evaluate the loss of accuracy; hence in order to do so, a second file is produced (named as quantize_eval_model.pb for vai_q_tensorflow).
For the current application, 100 images are sampled from the train dataset and augmented, resulting in a total number of 1000 images used for calibration. Furthermore, the graph is calibrated providing a batch of 10 images for 100 iterations. Following , the log of vai_q_tensorflow shows the result of the whole quantization process:
<pre>
===Compiling the model===
Compiling The Vitis AI compiler operates in a multi-stage process: # The compiler parses the topology of the optimized and quantized input model and produces a new computation graph consisting of a data flow and a control flow.# It will then optimize the data and control flow through processes such as fusing the batch normalization layers into the presiding convolution layers, efficient instruction scheduling by exploit inherent parallelism and exploiting data reuse.# Finally, it generates the code to be run. It must be noted that due to the limited number of operations supported by the DPU, the Vitis AI compiler automatically partitions the input network model into several kernels when there are operations not supported by DPU. For this particular case, two kernels are produced, due to the fact that the softmax activation layer is not currently supported by the DPU. Following, the log of vai_c_tensorflow shows the result of the compilation for the '''Baseline model''':
<pre>
</pre>
Compiling Following, the log of vai_c_tensorflow shows the result of the compilation for the '''Pruned model''':
<pre>