[BugFix] Minor fixes PPO / A2C examples #7706

Job	Run time
build-wheel-mac (3.8, 3.8)	1m 46s
build-wheel-linux (3.8, cp38-cp38, --extra-index-url https://download.pytorch.org/whl/cpu, "['cpu...	1m 38s
build-wheel-linux (3.9, cp39-cp39, --extra-index-url https://download.pytorch.org/whl/cpu, "['cpu...	1m 38s
build-wheel-windows (3.8, 3.8)	1m 51s
build-wheel-mac (3.9, 3.9)	1m 44s
build-wheel-linux (3.10, cp310-cp310, --extra-index-url https://download.pytorch.org/whl/cpu, "['...	1m 35s
build-wheel-windows (3.9, 3.9)	1m 42s
build-wheel-mac (3.10, 3.10.3)	1m 47s
build-wheel-linux (3.11, cp311-cp311, --extra-index-url https://download.pytorch.org/whl/cpu, "['...	1m 38s
build-wheel-mac (3.11, 3.11)	1m 34s
build-wheel-windows (3.10, 3.10.3)	1m 34s
build-wheel-windows (3.11, 3.11)	1m 46s
test-wheel-windows	1s
test-wheel	1s
	20m 15s

Provide feedback