Which tflite is the designated end export

[StuartIanNaylor]

If I export to tflite int8=true I get a folder containing multiple files.
It would seem yolov8n_int8.tflite is the fastest but its confusing as I have these choices.
yolov8n_int8.tflite, yolov8n_full_integer_quant.tflite, yolov8n_integer_quant.tflite and some float files which are all progress files.
I am presuming yolov8n_int8.tflite as apart from file extention same as the source export name.
Thought I would just ask though.

[RizwanMunawar]

Yes, your assumption is correct. When exporting to tflite with int8=true, yolov8n_int8.tflite is the file you should choose. Its naming similarity to the source export name is indicative, and it is likely the fastest option among the provided choices. If you have any further doubts or concerns, feel free to ask for clarification.

[StuartIanNaylor]

yolov8n_full_integer_quant.tflite is faster and maybe should be that one? It doesn't matter but I have been confused that I could not get the benches I was yesturday and it clicked it was prob one of the other tflite exports.

[pderrenger]

@StuartIanNaylor apologies for any confusion. The yolov8n_full_integer_quant.tflite file is indeed the fully quantized model, which typically offers the fastest inference speed due to its integer-only operations. This can be particularly beneficial on hardware that is optimized for integer arithmetic. If you're looking for the best performance in terms of speed, especially on edge devices, yolov8n_full_integer_quant.tflite would be the appropriate choice. Keep in mind that full quantization may sometimes lead to a slight drop in accuracy, so it's always good practice to validate the performance of the model on your specific dataset. If you need further assistance, don't hesitate to reach out.

[StuartIanNaylor]

Its OK as its not the last quantise optimisation which you would think is the one to use and this confused me also, but that is definately the fastest and just not sure what the further quantisations are?

[pderrenger]

@StuartIanNaylor the additional quantization files you're seeing, such as yolov8n_integer_quant.tflite, represent different levels of quantization that have been applied to the model. Here's a brief explanation of the typical quantization levels:

1. Full Integer Quantization (yolov8n_full_integer_quant.tflite): This is the most aggressive form of quantization, where both the weights and activations are converted to integers. This usually results in the smallest model size and the fastest inference speed, but it may also lead to a reduction in accuracy.

2. Integer Weights + Float Activations (yolov8n_integer_quant.tflite): In this case, the weights are quantized to integers, but the activations remain as floating-point numbers. This can be a good compromise between speed and accuracy.

3. Float Weights and Activations (yolov8n.tflite): This is the least aggressive form of quantization, where both weights and activations are kept as floating-point numbers. This typically provides the highest accuracy but may not be as fast as the other quantized models.

The reason for providing different quantization levels is to offer options that balance speed, size, and accuracy to best fit various deployment scenarios. Edge devices with dedicated neural processing units (NPUs) that support integer operations can benefit significantly from full integer quantization. However, if the target device does not support such operations, a less aggressive quantization might be necessary.

It's great to hear that you've identified the fastest model for your needs. If you have any more questions or need further clarification, feel free to ask.