Skip to content

ModelZoo Status (tag=v1.9.0)

Jump to bottom
chudegao edited this page Aug 24, 2021 · 1 revision

Report generated at 2021-08-24T03:35:56Z via GitHub Actions (1160817698).

Environment

Package Version
Platform Linux-5.8.0-1039-azure-x86_64-with-glibc2.2.5
Python 3.8.11 (default, Jul 5 2021, 19:22:52) [GCC 9.3.0]
onnx 1.9.0
onnx-tf 1.9.0 (72c8144)
tensorflow 2.6.0

Summary

Value Count
Models 38
Total 126
✔️ Passed 105
⚠️ Limitation 14
❌ Failed 7
➖ Skipped 0

Details

1. bert-squad

text/machine_comprehension/bert-squad

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
❌ 1. bertsquad-10 416M 5 10 🆗 🆗 ['Items are not equal:', " ACTUAL: dtype('float32')", " DESIRED: dtype('int64')"]
❌ 2. bertsquad-8 416M 5 8 🆗 🆗 ValueError: Tried to convert 'input' to a tensor and failed. Error: None values not supported.

2. bidirectional_attention_flow

text/machine_comprehension/bidirectional_attention_flow

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
⚠️ 1. bidaf-9 42M 4 9 🆗 BackendIsNotSupposedToImplementIt: CategoryMapper is not implemented. ➖

3. gpt-2

text/machine_comprehension/gpt-2

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. gpt2-10 523M 6 10 🆗 🆗 🆗
✔️ 2. gpt2-lm-head-10 634M 6 10 🆗 🆗 🆗

4. roberta

text/machine_comprehension/roberta

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
❌ 1. roberta-base-11 476M 6 11 🆗 🆗 TypeError: Input 'y' of 'Equal' Op has type int64 that does not match type float64 of argument 'x'.
✔️ 2. roberta-sequence-classification-9 476M 6 9 🆗 🆗 🆗

5. t5

text/machine_comprehension/t5

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
⚠️ 1. t5-decoder-with-lm-head-12 620M 6 12 🆗 🆗 No test data provided in model zoo
⚠️ 2. t5-encoder-12 620M 6 12 🆗 🆗 No test data provided in model zoo

6. age_gender

vision/body_analysis/age_gender

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
⚠️ 1. age_googlenet 23M 6 11 🆗 🆗 No test data provided in model zoo
⚠️ 2. gender_googlenet 23M 6 11 🆗 🆗 No test data provided in model zoo
⚠️ 3. vgg_ilsvrc_16_age_chalearn_iccv2015 514M 6 11 🆗 🆗 No test data provided in model zoo
⚠️ 4. vgg_ilsvrc_16_age_imdb_wiki 514M 6 11 🆗 🆗 No test data provided in model zoo
⚠️ 5. vgg_ilsvrc_16_gender_imdb_wiki 512M 6 11 🆗 🆗 No test data provided in model zoo

7. arcface

vision/body_analysis/arcface

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. arcfaceresnet100-8 249M 3 8 🆗 🆗 🆗

8. emotion_ferplus

vision/body_analysis/emotion_ferplus

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. emotion-ferplus-2 33M 3 2 🆗 🆗 🆗
✔️ 2. emotion-ferplus-7 33M 3 7 🆗 🆗 🆗
✔️ 3. emotion-ferplus-8 33M 3 8 🆗 🆗 🆗

9. ultraface

vision/body_analysis/ultraface

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
⚠️ 1. version-RFB-320 1M 4 9 🆗 🆗 No test data provided in model zoo
⚠️ 2. version-RFB-640 2M 4 9 🆗 🆗 No test data provided in model zoo

10. alexnet

vision/classification/alexnet

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. bvlcalexnet-3 233M 3 3 🆗 🆗 🆗
✔️ 2. bvlcalexnet-6 233M 3 6 🆗 🆗 🆗
✔️ 3. bvlcalexnet-7 233M 3 7 🆗 🆗 🆗
✔️ 4. bvlcalexnet-8 233M 3 8 🆗 🆗 🆗
✔️ 5. bvlcalexnet-9 233M 3 9 🆗 🆗 🆗

11. caffenet

vision/classification/caffenet

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. caffenet-3 233M 3 3 🆗 🆗 🆗
✔️ 2. caffenet-6 233M 3 6 🆗 🆗 🆗
✔️ 3. caffenet-7 233M 3 7 🆗 🆗 🆗
✔️ 4. caffenet-8 233M 3 8 🆗 🆗 🆗
✔️ 5. caffenet-9 233M 3 9 🆗 🆗 🆗

12. densenet-121

vision/classification/densenet-121

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. densenet-3 31M 3 3 🆗 🆗 🆗
✔️ 2. densenet-6 31M 3 9 🆗 🆗 🆗
✔️ 3. densenet-7 31M 3 7 🆗 🆗 🆗
✔️ 4. densenet-8 31M 3 8 🆗 🆗 🆗
✔️ 5. densenet-9 31M 3 9 🆗 🆗 🆗

13. efficientnet-lite4

vision/classification/efficientnet-lite4

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. efficientnet-lite4-11 50M 6 11 🆗 🆗 🆗

14. googlenet

vision/classification/inception_and_googlenet/googlenet

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. googlenet-3 27M 3 7 🆗 🆗 🆗
✔️ 2. googlenet-6 27M 3 9 🆗 🆗 🆗
✔️ 3. googlenet-7 27M 3 7 🆗 🆗 🆗
✔️ 4. googlenet-8 27M 3 8 🆗 🆗 🆗
✔️ 5. googlenet-9 27M 3 9 🆗 🆗 🆗

15. inception_v1

vision/classification/inception_and_googlenet/inception_v1

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. inception-v1-3 27M 3 3 🆗 🆗 🆗
✔️ 2. inception-v1-6 27M 3 6 🆗 🆗 🆗
✔️ 3. inception-v1-7 27M 3 7 🆗 🆗 🆗
✔️ 4. inception-v1-8 27M 3 8 🆗 🆗 🆗
✔️ 5. inception-v1-9 27M 3 9 🆗 🆗 🆗

16. inception_v2

vision/classification/inception_and_googlenet/inception_v2

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. inception-v2-3 43M 3 3 🆗 🆗 🆗
✔️ 2. inception-v2-6 43M 3 6 🆗 🆗 🆗
✔️ 3. inception-v2-7 43M 3 7 🆗 🆗 🆗
✔️ 4. inception-v2-8 43M 3 8 🆗 🆗 🆗
✔️ 5. inception-v2-9 43M 3 9 🆗 🆗 🆗

17. mnist

vision/classification/mnist

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. mnist-1 27K 3 1 🆗 🆗 🆗
✔️ 2. mnist-7 26K 3 7 🆗 🆗 🆗
✔️ 3. mnist-8 26K 3 8 🆗 🆗 🆗

18. mobilenet

vision/classification/mobilenet

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. mobilenetv2-7 13M 6 10 🆗 🆗 🆗

19. rcnn_ilsvrc13

vision/classification/rcnn_ilsvrc13

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. rcnn-ilsvrc13-3 220M 3 3 🆗 🆗 🆗
✔️ 2. rcnn-ilsvrc13-6 220M 3 6 🆗 🆗 🆗
✔️ 3. rcnn-ilsvrc13-7 220M 3 7 🆗 🆗 🆗
✔️ 4. rcnn-ilsvrc13-8 220M 3 8 🆗 🆗 🆗
✔️ 5. rcnn-ilsvrc13-9 220M 3 9 🆗 🆗 🆗

20. resnet

vision/classification/resnet

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. resnet101-v1-7 171M 3 8 🆗 🆗 🆗
✔️ 2. resnet101-v2-7 170M 3 7 🆗 🆗 🆗
✔️ 3. resnet152-v1-7 231M 3 8 🆗 🆗 🆗
✔️ 4. resnet152-v2-7 230M 3 7 🆗 🆗 🆗
✔️ 5. resnet18-v1-7 45M 3 8 🆗 🆗 🆗
✔️ 6. resnet18-v2-7 45M 3 7 🆗 🆗 🆗
✔️ 7. resnet34-v1-7 83M 3 8 🆗 🆗 🆗
✔️ 8. resnet34-v2-7 83M 3 7 🆗 🆗 🆗
✔️ 9. resnet50-caffe2-v1-3 98M 3 3 🆗 🆗 🆗
✔️ 10. resnet50-caffe2-v1-6 98M 3 6 🆗 🆗 🆗
✔️ 11. resnet50-caffe2-v1-7 98M 3 7 🆗 🆗 🆗
✔️ 12. resnet50-caffe2-v1-8 98M 3 8 🆗 🆗 🆗
✔️ 13. resnet50-caffe2-v1-9 98M 3 9 🆗 🆗 🆗
✔️ 14. resnet50-v1-7 98M 3 8 🆗 🆗 🆗
✔️ 15. resnet50-v2-7 98M 3 7 🆗 🆗 🆗

21. shufflenet

vision/classification/shufflenet

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. shufflenet-3 5M 3 3 🆗 🆗 🆗
✔️ 2. shufflenet-6 5M 3 8 🆗 🆗 🆗
✔️ 3. shufflenet-7 5M 3 7 🆗 🆗 🆗
✔️ 4. shufflenet-8 5M 3 6 🆗 🆗 🆗
✔️ 5. shufflenet-9 5M 3 9 🆗 🆗 🆗
✔️ 6. shufflenet-v2-10 9M 6 10 🆗 🆗 🆗

22. squeezenet

vision/classification/squeezenet

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. squeezenet1 5M 3 3 🆗 🆗 🆗
✔️ 2. squeezenet1 5M 3 6 🆗 🆗 🆗
✔️ 3. squeezenet1 5M 3 7 🆗 🆗 🆗
✔️ 4. squeezenet1 5M 3 8 🆗 🆗 🆗
✔️ 5. squeezenet1 5M 3 9 🆗 🆗 🆗
✔️ 6. squeezenet1 5M 3 7 🆗 🆗 🆗

23. vgg

vision/classification/vgg

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. vgg16-7 528M 3 8 🆗 🆗 🆗
✔️ 2. vgg16-bn-7 528M 3 8 🆗 🆗 🆗
✔️ 3. vgg19-7 548M 3 8 🆗 🆗 🆗
✔️ 4. vgg19-bn-7 548M 3 8 🆗 🆗 🆗
✔️ 5. vgg19-caffe2-3 548M 3 3 🆗 🆗 🆗
✔️ 6. vgg19-caffe2-6 548M 3 6 🆗 🆗 🆗
✔️ 7. vgg19-caffe2-7 548M 3 7 🆗 🆗 🆗
✔️ 8. vgg19-caffe2-8 548M 3 8 🆗 🆗 🆗
✔️ 9. vgg19-caffe2-9 548M 3 9 🆗 🆗 🆗

24. zfnet-512

vision/classification/zfnet-512

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. zfnet512-3 333M 3 3 🆗 🆗 🆗
✔️ 2. zfnet512-6 333M 3 6 🆗 🆗 🆗
✔️ 3. zfnet512-7 333M 3 7 🆗 🆗 🆗
✔️ 4. zfnet512-8 333M 3 8 🆗 🆗 🆗
✔️ 5. zfnet512-9 333M 3 9 🆗 🆗 🆗

25. duc

vision/object_detection_segmentation/duc

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
❌ 1. ResNet101-DUC-7 249M 3 7 🆗 🆗 ['Not equal to tolerance rtol=0.001, atol=0.001', '', 'Mismatched elements: 529669 / 3040000 (17.4%)', 'Max absolute difference: 0.99922323', 'Max relative difference: 1.']

26. faster-rcnn

vision/object_detection_segmentation/faster-rcnn

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
❌ 1. FasterRCNN-10 160M 4 10 🆗 🆗 ['Not equal to tolerance rtol=0.001, atol=0.001', '', '(shapes (80, 4), (77, 4) mismatch)', ' x: array([[ 588.1974 , 206.06781 , 743.93445 , 259.08737 ],', ' [ 273.3301 , 432.5252 , 327.75058 , 490.80524 ],']

27. fcn

vision/object_detection_segmentation/fcn

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
⚠️ 1. fcn-resnet101-11 207M 6 11 🆗 OpUnsupportedException: Resize coordinate_transformation_mode=pytorch_half_pixel is not supported in Tensorflow. ➖
⚠️ 2. fcn-resnet50-11 135M 6 11 🆗 OpUnsupportedException: Resize coordinate_transformation_mode=pytorch_half_pixel is not supported in Tensorflow. ➖

28. mask-rcnn

vision/object_detection_segmentation/mask-rcnn

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
❌ 1. MaskRCNN-10 170M 4 10 🆗 🆗 ['Not equal to tolerance rtol=0.001, atol=0.001', '', '(shapes (63, 4), (62, 4) mismatch)', ' x: array([[ 475.39844 , 286.16537 , 803.5265 , 553.59546 ],', ' [ 873.8628 , 402.8742 , 909.85596 , 428.6966 ],']

29. retinanet

vision/object_detection_segmentation/retinanet

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. retinanet-9 218M 6 9 🆗 🆗 🆗

30. ssd

vision/object_detection_segmentation/ssd

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. ssd-10 77M 4 10 🆗 🆗 🆗

31. ssd-mobilenetv1

vision/object_detection_segmentation/ssd-mobilenetv1

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
❌ 1. ssd_mobilenet_v1_10 28M 5 10 🆗 🆗 FileNotFoundError: [Errno 2] No such file or directory: 'wiki/test_model_and_data/ssd_mobilenet_v1/test_data_set_1/input_0.pb'

32. tiny-yolov2

vision/object_detection_segmentation/tiny-yolov2

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. tinyyolov2-7 61M 5 7 🆗 🆗 🆗
✔️ 2. tinyyolov2-8 61M 5 8 🆗 🆗 🆗

33. tiny-yolov3

vision/object_detection_segmentation/tiny-yolov3

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. tiny-yolov3-11 34M 6 11 🆗 🆗 🆗

34. yolov2-coco

vision/object_detection_segmentation/yolov2-coco

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
⚠️ 1. yolov2-coco-9 195M 4 9 🆗 🆗 No test data provided in model zoo

35. yolov3

vision/object_detection_segmentation/yolov3

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. yolov3-10 236M 5 10 🆗 🆗 🆗

36. yolov4

vision/object_detection_segmentation/yolov4

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
⚠️ 1. yolov4 246M 6 11 🆗 OpUnsupportedException: Resize coordinate_transformation_mode=half_pixel and mode=nearest is not supported in Tensorflow. ➖

37. fast_neural_style

vision/style_transfer/fast_neural_style

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. candy-8 6M 4 8 🆗 🆗 🆗
✔️ 2. candy-9 6M 4 9 🆗 🆗 🆗
✔️ 3. mosaic-8 6M 4 8 🆗 🆗 🆗
✔️ 4. mosaic-9 6M 4 9 🆗 🆗 🆗
✔️ 5. pointilism-8 6M 4 8 🆗 🆗 🆗
✔️ 6. pointilism-9 6M 4 9 🆗 🆗 🆗
✔️ 7. rain-princess-8 6M 4 8 🆗 🆗 🆗
✔️ 8. rain-princess-9 6M 4 9 🆗 🆗 🆗
✔️ 9. udnie-8 6M 4 8 🆗 🆗 🆗
✔️ 10. udnie-9 6M 4 9 🆗 🆗 🆗

38. sub_pixel_cnn_2016

vision/super_resolution/sub_pixel_cnn_2016

Status Model Size IR Opset ONNX Checker ONNX-TF Converted ONNX-TF Ran
✔️ 1. super-resolution-10 234K 4 10 🆗 🆗 🆗
Clone this wiki locally