SynthText在MMOCR0.x和MMOCR1.x版本中IC15测试精度差异大

[jorie-peng]

branch: dev1.x
train dataset: SynthText + Syn90k
test dataset: IC15、IC13、IIIT5k
model: SATRN
config file: mmocr/configs/textrecog/satrn/satrn_shallow_5e_st_mj.py

version	Train	instances	Test
			IC15	IC13	CUTE80	SVT
MMOCR0.x	SynthText	7266686	84.1	95.7	90.3	93.5
	Syn90k	8919273
MMOCR1.x	SynthText	7266656	81.4	95.4	90.3	92
	Syn90k	8919273

测试结果如上表，在IC15上，相差2.7点之多，请问这个浮动是否正常？MMOCR是否有基于1.x的SATRN的复现结果？
additional information：

1. IC15和Syn90k都有单独测试过精度，数据集没有问题，但是加入synthtext之后精度就相差过大
2. SynthText在复现时，使用mmocr/tools/dataset_converters/textrecog/synthtext_converter.py，添加生成MMOCR1.x的json文件，比MMOCR0.x中缺少30个instance，已核查，polygon为负值，为无效图片。其他instance已和MMOCR0.x中label.txt核对一致。
3. 运行了3次不同种子的结果，均在81.4左右，和官网的84.1结果相差2.7个点。

附件
训练log：20221217_145047.log
SynthText 数据集和Syn90k数据集的json太大无法上传。

[gaotongxiao]

We have released all the results & checkpoints trained on 1.x, on 1.x's documentation:
https://mmocr.readthedocs.io/en/dev-1.x/textrecog_models.html#id41

The performance difference is due to the missing test-time augmentation in MMOCR 1.x:

mmocr/configs/_base_/recog_pipelines/satrn_pipeline.py

Line 24 in a4fe6bb

type='MultiRotateAugOCR',

It has been implemented in #1401, and will be brought back to MMOCR in the official 1.0 release.

[jorie-peng]

ok, Thanks!