Bottleneck Transformers for Visual Recognition Update 2021/03/14 support Multi-head Attention Experiments Model heads Params (M) Acc (%) ResNet50 baseline (ref) 23.5M 93.62 BoTNet-50 1 18.8M 95.11% BoTNet-50 4 18.8M 95.78% BoTNet-S1-50 1 18.8M 95.67% BoTNet-S1-59 1 27.5M 95.98% BoTNet-S1-77 1 44.9M wip Summary Usage (example) Model from model import Model model = ResNet50(num_classes=1000, resolution=(224, 224)) x = torch.randn([2, 3, 224, 224]) print(model(x).size()) Module from model import MHSA resolution = 14 mhsa = MHSA(planes, width=resolution, height=resolution) Reference Paper link Author: Aravind Srinivas, Tsung-Yi Lin, Niki Parmar, Jonathon Shlens, Pieter Abbeel, Ashish Vaswani Organization: UC Berkeley, Google Research
