Skip to content

ResNeXt

Farley Lai edited this page Apr 30, 2021 · 1 revision

Validation of backbone deployment with TensorRT Validation.

ResNeXt101_WSL Validation of batch size 256 and crops of 224x224

PyTorch FP32 for 12GB

Test: [  0/196] Time  9.232 ( 9.232)    Loss 4.4442e-01 (4.4442e-01)    Acc@1  89.84 ( 89.84)   Acc@5  98.83 ( 98.83)
Test: [ 10/196] Time  1.395 ( 2.111)    Loss 8.7335e-01 (6.0189e-01)    Acc@1  81.25 ( 86.19)   Acc@5  95.70 ( 97.80)
Test: [ 20/196] Time  1.403 ( 1.776)    Loss 5.9951e-01 (6.1734e-01)    Acc@1  91.80 ( 86.01)   Acc@5  97.66 ( 97.73)
Test: [ 30/196] Time  1.405 ( 1.658)    Loss 7.0111e-01 (5.8449e-01)    Acc@1  86.33 ( 87.24)   Acc@5  96.48 ( 97.90)
Test: [ 40/196] Time  1.410 ( 1.598)    Loss 6.3284e-01 (6.3505e-01)    Acc@1  84.77 ( 86.41)   Acc@5  97.27 ( 97.72)
Test: [ 50/196] Time  1.436 ( 1.563)    Loss 2.9359e-01 (6.3548e-01)    Acc@1  91.80 ( 86.39)   Acc@5  98.44 ( 97.81)
Test: [ 60/196] Time  1.415 ( 1.539)    Loss 7.9863e-01 (6.6132e-01)    Acc@1  83.20 ( 85.89)   Acc@5  98.44 ( 97.81)
Test: [ 70/196] Time  1.415 ( 1.523)    Loss 5.2815e-01 (6.4469e-01)    Acc@1  84.38 ( 86.20)   Acc@5  98.83 ( 97.97)
Test: [ 80/196] Time  1.436 ( 1.510)    Loss 1.5031e+00 (6.7234e-01)    Acc@1  70.31 ( 85.81)   Acc@5  95.31 ( 97.79)
Test: [ 90/196] Time  1.437 ( 1.500)    Loss 1.9102e+00 (7.1119e-01)    Acc@1  63.28 ( 85.15)   Acc@5  92.19 ( 97.57)
Test: [100/196] Time  1.417 ( 1.492)    Loss 7.8826e-01 (7.4896e-01)    Acc@1  81.64 ( 84.46)   Acc@5  98.44 ( 97.34)
Test: [110/196] Time  1.437 ( 1.486)    Loss 6.5294e-01 (7.6261e-01)    Acc@1  88.67 ( 84.22)   Acc@5  98.05 ( 97.21)
Test: [120/196] Time  1.417 ( 1.481)    Loss 1.3365e+00 (7.7342e-01)    Acc@1  79.30 ( 84.19)   Acc@5  92.19 ( 97.10)
Test: [130/196] Time  1.418 ( 1.477)    Loss 2.7800e-01 (7.9937e-01)    Acc@1  92.19 ( 83.66)   Acc@5  99.22 ( 96.93)
Test: [140/196] Time  1.421 ( 1.473)    Loss 8.9998e-01 (8.1730e-01)    Acc@1  81.64 ( 83.38)   Acc@5  97.66 ( 96.88)
Test: [150/196] Time  1.440 ( 1.470)    Loss 8.8850e-01 (8.3202e-01)    Acc@1  85.55 ( 83.09)   Acc@5  94.92 ( 96.74)
Test: [160/196] Time  1.419 ( 1.467)    Loss 3.4630e-01 (8.3839e-01)    Acc@1  92.97 ( 82.96)   Acc@5  98.44 ( 96.68)
Test: [170/196] Time  1.441 ( 1.465)    Loss 3.5175e-01 (8.5224e-01)    Acc@1  89.06 ( 82.66)   Acc@5  98.83 ( 96.60)
Test: [180/196] Time  1.421 ( 1.463)    Loss 1.3991e+00 (8.6809e-01)    Acc@1  73.05 ( 82.38)   Acc@5  94.53 ( 96.51)
Test: [190/196] Time  1.419 ( 1.461)    Loss 1.3609e+00 (8.8242e-01)    Acc@1  67.97 ( 82.33)   Acc@5  97.66 ( 96.53)
 * Acc@1 82.436 Acc@5 96.552

TensorRT FP32 for 6.7GB

Test: [  0/196] Time  3.477 ( 3.477)    Loss 4.7527e-01 (4.7527e-01)    Acc@1  89.45 ( 89.45)   Acc@5  98.44 ( 98.44)
Test: [ 10/196] Time  0.682 ( 0.956)    Loss 8.9261e-01 (6.3254e-01)    Acc@1  80.86 ( 86.04)   Acc@5  95.70 ( 97.23)
Test: [ 20/196] Time  0.684 ( 0.826)    Loss 6.7233e-01 (6.5074e-01)    Acc@1  90.23 ( 85.49)   Acc@5  97.66 ( 97.21)
Test: [ 30/196] Time  0.686 ( 0.781)    Loss 8.1874e-01 (6.2646e-01)    Acc@1  85.55 ( 86.61)   Acc@5  95.70 ( 97.33)
Test: [ 40/196] Time  0.689 ( 0.758)    Loss 7.6912e-01 (6.8436e-01)    Acc@1  81.64 ( 85.57)   Acc@5  95.70 ( 97.14)
Test: [ 50/196] Time  0.691 ( 0.745)    Loss 3.4114e-01 (6.8707e-01)    Acc@1  91.41 ( 85.45)   Acc@5  98.05 ( 97.27)
Test: [ 60/196] Time  0.693 ( 0.736)    Loss 9.4137e-01 (7.1020e-01)    Acc@1  81.25 ( 84.91)   Acc@5  97.27 ( 97.29)
Test: [ 70/196] Time  0.694 ( 0.730)    Loss 5.4116e-01 (6.8780e-01)    Acc@1  83.59 ( 85.30)   Acc@5  99.22 ( 97.51)
Test: [ 80/196] Time  0.697 ( 0.726)    Loss 1.6142e+00 (7.1319e-01)    Acc@1  72.27 ( 84.97)   Acc@5  95.31 ( 97.32)
Test: [ 90/196] Time  0.696 ( 0.723)    Loss 1.9795e+00 (7.5153e-01)    Acc@1  62.50 ( 84.30)   Acc@5  92.97 ( 97.13)
Test: [100/196] Time  0.699 ( 0.720)    Loss 8.1100e-01 (7.9187e-01)    Acc@1  80.86 ( 83.53)   Acc@5  96.88 ( 96.87)
Test: [110/196] Time  0.700 ( 0.718)    Loss 6.6544e-01 (8.0488e-01)    Acc@1  87.50 ( 83.33)   Acc@5  97.27 ( 96.78)
Test: [120/196] Time  0.702 ( 0.717)    Loss 1.3923e+00 (8.1680e-01)    Acc@1  76.95 ( 83.23)   Acc@5  91.41 ( 96.66)
Test: [130/196] Time  0.703 ( 0.716)    Loss 2.8728e-01 (8.4375e-01)    Acc@1  91.02 ( 82.70)   Acc@5  99.61 ( 96.49)
Test: [140/196] Time  0.702 ( 0.715)    Loss 9.3971e-01 (8.6305e-01)    Acc@1  81.64 ( 82.41)   Acc@5  97.27 ( 96.42)
Test: [150/196] Time  0.704 ( 0.714)    Loss 8.9077e-01 (8.7604e-01)    Acc@1  85.16 ( 82.15)   Acc@5  94.14 ( 96.29)
Test: [160/196] Time  0.704 ( 0.713)    Loss 3.3352e-01 (8.8050e-01)    Acc@1  91.41 ( 82.04)   Acc@5  98.44 ( 96.25)
Test: [170/196] Time  0.705 ( 0.713)    Loss 4.2732e-01 (8.9533e-01)    Acc@1  89.06 ( 81.75)   Acc@5  98.44 ( 96.17)
Test: [180/196] Time  0.705 ( 0.712)    Loss 1.2044e+00 (9.1001e-01)    Acc@1  75.39 ( 81.47)   Acc@5  96.09 ( 96.09)
Test: [190/196] Time  0.705 ( 0.712)    Loss 1.4610e+00 (9.2504e-01)    Acc@1  69.92 ( 81.40)   Acc@5  96.48 ( 96.08)
 * Acc@1 81.520 Acc@5 96.112

TensorRT FP16 for 4.2GB

Test: [  0/196] Time  2.972 ( 2.972)    Loss 4.5834e-01 (4.5834e-01)    Acc@1  91.41 ( 91.41)   Acc@5  98.44 ( 98.44)
Test: [ 10/196] Time  0.194 ( 0.427)    Loss 8.8206e-01 (6.2900e-01)    Acc@1  80.08 ( 85.90)   Acc@5  95.70 ( 97.30)
Test: [ 20/196] Time  0.291 ( 0.304)    Loss 6.7802e-01 (6.4754e-01)    Acc@1  90.23 ( 85.29)   Acc@5  97.66 ( 97.21)
Test: [ 30/196] Time  0.144 ( 0.263)    Loss 7.9438e-01 (6.2259e-01)    Acc@1  85.94 ( 86.37)   Acc@5  96.09 ( 97.38)
Test: [ 40/196] Time  0.263 ( 0.245)    Loss 7.5554e-01 (6.8037e-01)    Acc@1  82.42 ( 85.38)   Acc@5  95.70 ( 97.17)
Test: [ 50/196] Time  0.145 ( 0.232)    Loss 3.3836e-01 (6.8349e-01)    Acc@1  91.41 ( 85.31)   Acc@5  98.05 ( 97.30)
Test: [ 60/196] Time  0.258 ( 0.224)    Loss 9.2939e-01 (7.0596e-01)    Acc@1  81.25 ( 84.82)   Acc@5  97.27 ( 97.32)
Test: [ 70/196] Time  0.147 ( 0.218)    Loss 5.3358e-01 (6.8394e-01)    Acc@1  83.59 ( 85.20)   Acc@5  99.22 ( 97.55)
Test: [ 80/196] Time  0.300 ( 0.215)    Loss 1.5985e+00 (7.0912e-01)    Acc@1  71.48 ( 84.89)   Acc@5  94.92 ( 97.36)
Test: [ 90/196] Time  0.165 ( 0.211)    Loss 1.9600e+00 (7.4698e-01)    Acc@1  62.89 ( 84.22)   Acc@5  92.19 ( 97.16)
Test: [100/196] Time  0.258 ( 0.207)    Loss 7.9871e-01 (7.8669e-01)    Acc@1  80.86 ( 83.47)   Acc@5  96.48 ( 96.87)
Test: [110/196] Time  0.163 ( 0.205)    Loss 6.5883e-01 (8.0050e-01)    Acc@1  87.50 ( 83.26)   Acc@5  97.27 ( 96.78)
Test: [120/196] Time  0.203 ( 0.202)    Loss 1.3898e+00 (8.1222e-01)    Acc@1  75.78 ( 83.15)   Acc@5  91.41 ( 96.65)
Test: [130/196] Time  0.146 ( 0.200)    Loss 2.8764e-01 (8.3857e-01)    Acc@1  91.02 ( 82.62)   Acc@5  99.61 ( 96.49)
Test: [140/196] Time  0.190 ( 0.198)    Loss 9.1512e-01 (8.5743e-01)    Acc@1  81.64 ( 82.34)   Acc@5  97.66 ( 96.43)
Test: [150/196] Time  0.189 ( 0.196)    Loss 8.7663e-01 (8.6981e-01)    Acc@1  85.55 ( 82.07)   Acc@5  94.53 ( 96.31)
Test: [160/196] Time  0.192 ( 0.195)    Loss 3.4019e-01 (8.7450e-01)    Acc@1  91.41 ( 81.95)   Acc@5  98.05 ( 96.27)
Test: [170/196] Time  0.145 ( 0.194)    Loss 4.2087e-01 (8.8943e-01)    Acc@1  89.06 ( 81.67)   Acc@5  98.44 ( 96.17)
Test: [180/196] Time  0.146 ( 0.193)    Loss 1.2045e+00 (9.0383e-01)    Acc@1  74.61 ( 81.38)   Acc@5  96.09 ( 96.10)
Test: [190/196] Time  0.146 ( 0.193)    Loss 1.4455e+00 (9.1888e-01)    Acc@1  69.53 ( 81.32)   Acc@5  96.48 ( 96.09)
 * Acc@1 81.440 Acc@5 96.122

TensorRT FP16/Int8 with calibration of 5000 images for 3GB

Test: [  0/196] Time  1.438 ( 1.438)    Loss 3.9788e-01 (3.9788e-01)    Acc@1  92.19 ( 92.19)   Acc@5  98.83 ( 98.83)
Test: [ 10/196] Time  0.088 ( 0.262)    Loss 8.6129e-01 (5.9844e-01)    Acc@1  81.25 ( 85.97)   Acc@5  94.92 ( 97.34)
Test: [ 20/196] Time  0.394 ( 0.234)    Loss 5.8548e-01 (6.1092e-01)    Acc@1  90.62 ( 85.42)   Acc@5  97.66 ( 97.34)
Test: [ 30/196] Time  0.088 ( 0.212)    Loss 6.8727e-01 (5.7613e-01)    Acc@1  85.55 ( 86.61)   Acc@5  96.09 ( 97.51)
Test: [ 40/196] Time  0.486 ( 0.210)    Loss 6.5767e-01 (6.1504e-01)    Acc@1  83.98 ( 85.58)   Acc@5  96.09 ( 97.39)
Test: [ 50/196] Time  0.091 ( 0.201)    Loss 3.5150e-01 (6.0748e-01)    Acc@1  91.41 ( 85.67)   Acc@5  98.05 ( 97.48)
Test: [ 60/196] Time  0.385 ( 0.199)    Loss 7.6129e-01 (6.3103e-01)    Acc@1  81.25 ( 85.07)   Acc@5  96.88 ( 97.49)
Test: [ 70/196] Time  0.089 ( 0.195)    Loss 5.5690e-01 (6.1278e-01)    Acc@1  83.98 ( 85.45)   Acc@5  99.22 ( 97.67)
Test: [ 80/196] Time  0.464 ( 0.195)    Loss 1.2981e+00 (6.3479e-01)    Acc@1  71.88 ( 85.16)   Acc@5  94.14 ( 97.48)
Test: [ 90/196] Time  0.089 ( 0.192)    Loss 1.6582e+00 (6.6830e-01)    Acc@1  62.89 ( 84.46)   Acc@5  93.75 ( 97.25)
Test: [100/196] Time  0.429 ( 0.192)    Loss 6.8045e-01 (7.0026e-01)    Acc@1  81.25 ( 83.71)   Acc@5  96.88 ( 96.99)
Test: [110/196] Time  0.089 ( 0.190)    Loss 5.5815e-01 (7.1266e-01)    Acc@1  89.06 ( 83.53)   Acc@5  96.88 ( 96.86)
Test: [120/196] Time  0.341 ( 0.190)    Loss 1.2023e+00 (7.2295e-01)    Acc@1  75.78 ( 83.45)   Acc@5  92.19 ( 96.75)
Test: [130/196] Time  0.110 ( 0.189)    Loss 2.7903e-01 (7.4513e-01)    Acc@1  89.84 ( 82.93)   Acc@5  99.22 ( 96.60)
Test: [140/196] Time  0.183 ( 0.188)    Loss 8.0833e-01 (7.5996e-01)    Acc@1  80.47 ( 82.70)   Acc@5  97.27 ( 96.53)
Test: [150/196] Time  0.089 ( 0.187)    Loss 8.3221e-01 (7.7488e-01)    Acc@1  83.98 ( 82.39)   Acc@5  94.14 ( 96.35)
Test: [160/196] Time  0.379 ( 0.187)    Loss 3.9124e-01 (7.8135e-01)    Acc@1  90.62 ( 82.26)   Acc@5  97.27 ( 96.30)
Test: [170/196] Time  0.103 ( 0.187)    Loss 4.3080e-01 (7.9498e-01)    Acc@1  89.06 ( 81.98)   Acc@5  98.44 ( 96.21)
Test: [180/196] Time  0.135 ( 0.186)    Loss 9.4216e-01 (8.0894e-01)    Acc@1  76.95 ( 81.67)   Acc@5  96.88 ( 96.14)
Test: [190/196] Time  0.088 ( 0.186)    Loss 1.2157e+00 (8.1549e-01)    Acc@1  69.53 ( 81.61)   Acc@5  97.27 ( 96.13)
 * Acc@1 81.702 Acc@5 96.162

TensorRT FP16/Int8 strict with calibration of 5000 images for 3GB

Test: [  0/196] Time  1.415 ( 1.415)    Loss 3.8323e-01 (3.8323e-01)    Acc@1  90.23 ( 90.23)   Acc@5  99.22 ( 99.22)
Test: [ 10/196] Time  0.089 ( 0.264)    Loss 8.6448e-01 (5.9739e-01)    Acc@1  81.25 ( 85.55)   Acc@5  95.31 ( 97.30)
Test: [ 20/196] Time  0.416 ( 0.234)    Loss 5.8879e-01 (6.1027e-01)    Acc@1  90.23 ( 85.29)   Acc@5  97.66 ( 97.43)
Test: [ 30/196] Time  0.088 ( 0.212)    Loss 6.9124e-01 (5.7561e-01)    Acc@1  86.72 ( 86.59)   Acc@5  96.48 ( 97.51)
Test: [ 40/196] Time  0.449 ( 0.209)    Loss 6.8183e-01 (6.1648e-01)    Acc@1  83.59 ( 85.64)   Acc@5  96.88 ( 97.38)
Test: [ 50/196] Time  0.088 ( 0.200)    Loss 3.3351e-01 (6.0927e-01)    Acc@1  92.19 ( 85.73)   Acc@5  98.05 ( 97.46)
Test: [ 60/196] Time  0.409 ( 0.198)    Loss 7.8129e-01 (6.3201e-01)    Acc@1  81.25 ( 85.09)   Acc@5  97.27 ( 97.47)
Test: [ 70/196] Time  0.089 ( 0.194)    Loss 5.5974e-01 (6.1395e-01)    Acc@1  84.38 ( 85.43)   Acc@5  98.44 ( 97.63)
Test: [ 80/196] Time  0.469 ( 0.194)    Loss 1.3128e+00 (6.3612e-01)    Acc@1  73.05 ( 85.19)   Acc@5  94.14 ( 97.44)
Test: [ 90/196] Time  0.088 ( 0.190)    Loss 1.6791e+00 (6.7023e-01)    Acc@1  64.45 ( 84.50)   Acc@5  93.75 ( 97.24)
Test: [100/196] Time  0.294 ( 0.189)    Loss 6.9019e-01 (7.0364e-01)    Acc@1  80.08 ( 83.70)   Acc@5  97.27 ( 96.94)
Test: [110/196] Time  0.090 ( 0.188)    Loss 5.6252e-01 (7.1695e-01)    Acc@1  86.72 ( 83.52)   Acc@5  96.88 ( 96.80)
Test: [120/196] Time  0.203 ( 0.188)    Loss 1.2245e+00 (7.2648e-01)    Acc@1  75.78 ( 83.46)   Acc@5  92.58 ( 96.70)
Test: [130/196] Time  0.108 ( 0.187)    Loss 2.5387e-01 (7.4811e-01)    Acc@1  91.41 ( 82.97)   Acc@5  99.61 ( 96.56)
Test: [140/196] Time  0.139 ( 0.186)    Loss 7.8910e-01 (7.6274e-01)    Acc@1  81.25 ( 82.71)   Acc@5  97.27 ( 96.52)
Test: [150/196] Time  0.088 ( 0.185)    Loss 8.2756e-01 (7.7762e-01)    Acc@1  82.81 ( 82.41)   Acc@5  94.53 ( 96.37)
Test: [160/196] Time  0.291 ( 0.184)    Loss 3.6995e-01 (7.8373e-01)    Acc@1  90.23 ( 82.28)   Acc@5  97.27 ( 96.31)
Test: [170/196] Time  0.089 ( 0.184)    Loss 4.0839e-01 (7.9688e-01)    Acc@1  89.45 ( 82.01)   Acc@5  98.44 ( 96.22)
Test: [180/196] Time  0.220 ( 0.183)    Loss 9.3844e-01 (8.1054e-01)    Acc@1  76.17 ( 81.72)   Acc@5  96.88 ( 96.14)
Test: [190/196] Time  0.089 ( 0.183)    Loss 1.2494e+00 (8.1682e-01)    Acc@1  69.14 ( 81.65)   Acc@5  96.88 ( 96.13)
 * Acc@1 81.748 Acc@5 96.166

TensorRT Int8 strict with calibration of 5000 images for 3GB

Test: [  0/196] Time  1.330 ( 1.330)    Loss 3.9969e-01 (3.9969e-01)    Acc@1  91.02 ( 91.02)   Acc@5  99.22 ( 99.22)
Test: [ 10/196] Time  0.090 ( 0.269)    Loss 8.5212e-01 (5.9335e-01)    Acc@1  81.25 ( 86.01)   Acc@5  95.31 ( 97.37)
Test: [ 20/196] Time  0.406 ( 0.239)    Loss 5.6507e-01 (6.0948e-01)    Acc@1  90.62 ( 85.40)   Acc@5  97.27 ( 97.36)
Test: [ 30/196] Time  0.089 ( 0.214)    Loss 6.9770e-01 (5.7714e-01)    Acc@1  85.16 ( 86.53)   Acc@5  96.09 ( 97.45)
Test: [ 40/196] Time  0.446 ( 0.211)    Loss 6.9912e-01 (6.1686e-01)    Acc@1  83.20 ( 85.59)   Acc@5  96.48 ( 97.31)
Test: [ 50/196] Time  0.109 ( 0.203)    Loss 3.3754e-01 (6.0866e-01)    Acc@1  92.19 ( 85.71)   Acc@5  98.05 ( 97.42)
Test: [ 60/196] Time  0.381 ( 0.200)    Loss 7.5044e-01 (6.3092e-01)    Acc@1  81.64 ( 85.12)   Acc@5  97.66 ( 97.46)
Test: [ 70/196] Time  0.091 ( 0.196)    Loss 5.6149e-01 (6.1246e-01)    Acc@1  83.20 ( 85.42)   Acc@5  98.83 ( 97.66)
Test: [ 80/196] Time  0.413 ( 0.196)    Loss 1.3271e+00 (6.3436e-01)    Acc@1  71.09 ( 85.10)   Acc@5  94.92 ( 97.50)
Test: [ 90/196] Time  0.110 ( 0.192)    Loss 1.6551e+00 (6.6904e-01)    Acc@1  64.45 ( 84.43)   Acc@5  94.53 ( 97.29)
Test: [100/196] Time  0.407 ( 0.191)    Loss 6.9790e-01 (7.0235e-01)    Acc@1  80.08 ( 83.61)   Acc@5  97.27 ( 97.03)
Test: [110/196] Time  0.090 ( 0.189)    Loss 5.5995e-01 (7.1473e-01)    Acc@1  86.72 ( 83.44)   Acc@5  97.27 ( 96.90)
Test: [120/196] Time  0.333 ( 0.188)    Loss 1.2059e+00 (7.2535e-01)    Acc@1  76.17 ( 83.35)   Acc@5  91.80 ( 96.77)
Test: [130/196] Time  0.089 ( 0.186)    Loss 2.8905e-01 (7.4758e-01)    Acc@1  89.45 ( 82.80)   Acc@5  99.22 ( 96.60)
Test: [140/196] Time  0.362 ( 0.186)    Loss 8.0112e-01 (7.6202e-01)    Acc@1  80.47 ( 82.57)   Acc@5  97.27 ( 96.53)
Test: [150/196] Time  0.090 ( 0.185)    Loss 8.3108e-01 (7.7691e-01)    Acc@1  83.98 ( 82.28)   Acc@5  94.53 ( 96.38)
Test: [160/196] Time  0.397 ( 0.185)    Loss 3.7156e-01 (7.8349e-01)    Acc@1  89.84 ( 82.12)   Acc@5  97.27 ( 96.32)
Test: [170/196] Time  0.089 ( 0.184)    Loss 4.0384e-01 (7.9690e-01)    Acc@1  89.45 ( 81.86)   Acc@5  98.44 ( 96.22)
Test: [180/196] Time  0.307 ( 0.184)    Loss 9.6889e-01 (8.1081e-01)    Acc@1  77.73 ( 81.58)   Acc@5  95.70 ( 96.14)
Test: [190/196] Time  0.089 ( 0.184)    Loss 1.2080e+00 (8.1757e-01)    Acc@1  70.31 ( 81.52)   Acc@5  96.48 ( 96.13)
 * Acc@1 81.610 Acc@5 96.160