[Performance] Reduce key accessing in transforms #7688

Job	Run time
build-wheel-linux (3.8, cp38-cp38, --extra-index-url https://download.pytorch.org/whl/cpu, "['cpu...	2m 40s
build-wheel-mac (3.8, 3.8)	1m 42s
build-wheel-linux (3.9, cp39-cp39, --extra-index-url https://download.pytorch.org/whl/cpu, "['cpu...	2m 6s
build-wheel-mac (3.9, 3.9)	1m 46s
build-wheel-linux (3.10, cp310-cp310, --extra-index-url https://download.pytorch.org/whl/cpu, "['...	2m 40s
build-wheel-windows (3.8, 3.8)	3m 27s
build-wheel-mac (3.10, 3.10.3)	2m 0s
build-wheel-windows (3.9, 3.9)	2m 39s
build-wheel-linux (3.11, cp311-cp311, --extra-index-url https://download.pytorch.org/whl/cpu, "['...	2m 16s
build-wheel-mac (3.11, 3.11)	1m 57s
build-wheel-windows (3.10, 3.10.3)	4m 40s
build-wheel-windows (3.11, 3.11)	3m 17s
test-wheel (linux, ubuntu-20.04, 3.8)	3m 6s
test-wheel (linux, ubuntu-20.04, 3.9)	3m 18s
test-wheel (linux, ubuntu-20.04, 3.10)	3m 43s
test-wheel (linux, ubuntu-20.04, 3.11)	3m 4s
test-wheel (mac, macos-latest, 3.8)	4m 44s
test-wheel (mac, macos-latest, 3.9)	3m 16s
test-wheel (mac, macos-latest, 3.10)	3m 12s
test-wheel (mac, macos-latest, 3.11)	2m 55s
test-wheel-windows (3.8)	7m 29s
test-wheel-windows (3.9)	4m 51s
test-wheel-windows (3.10)	5m 48s
test-wheel-windows (3.11)	6m 7s
	1h 22m 43s

Provide feedback