[Feature] CQL compatibility with compile

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] GAIL compatibility with compile #2573
• [Feature] DQN compatibility with compile #2571
• [Feature] Discrete SAC compatibility with compile #2569
• [Feature] DT compatibility with compile #2556
• [Feature] DDPG compatibility with compile #2555
• [Feature] CROSSQ compatibility with compile #2554
• -> [Feature] CQL compatibility with compile #2553

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2553

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 8 Unrelated Failures

As of commit bbaa142 with merge base b4b5944 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  FAILED ../../../../../../tmp/test_objectives_benchmarks.py::test_iql_speed[True-None] - torch._dynamo.exc.Unsupported: Graph break under GenericContextWrappingVariable
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  FAILED ../../../../tmp/test_objectives_benchmarks.py::test_iql_speed[reduce-overhead-backward] - torch._dynamo.exc.Unsupported: Graph break due to unsupported builtin _C.expand. This function is either a Python builtin (e.g. _warnings.warn) or a third-party C/C++ Python extension (perhaps created with pybind). If it is a Python builtin, please file an issue on GitHub so the PyTorch team can add support for it and see the next case for a workaround. If it is a third-party C/C++ Python extension, please either wrap it into a PyTorch-understood custom operator (see https://pytorch.org/tutorials/advanced/custom_ops_landing_page.html for more details) or, if it is traceable, use torch.compiler.allow_in_graph.
• SOTA Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t cb4fb1c2c5767f60bdff15a4289186da8ca20b70154797e56a213213c5153144 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  test/test_transforms.py::TestTimeMaxPool::test_parallel_trans_env_check

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Linux Wheels / pytorch/rl (pytorch/rl, test/smoke_test.py, torchrl) / upload / manywheel-py3_9-rocm6_1 (gh) (trunk failure)
  ##[error]Unable to find an artifact with the name: pytorch_rl__3.9_rocm6.1_x86_64
• Build Linux Wheels / pytorch/rl (pytorch/rl, test/smoke_test.py, torchrl) / upload / manywheel-py3_9-rocm6_2 (gh) (trunk failure)
  ##[error]Unable to find an artifact with the name: pytorch_rl__3.9_rocm6.2_x86_64
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cpu (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda11_8 (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_4 (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Generate documentation / build-docs (3.10, 12.1) / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh) (trunk failure)
  AttributeError: _ARRAY_API not found
• Unit-tests on Windows / unittests-cpu / windows-job (gh) (trunk failure)
  ##[error]Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.