[QUESTION] Is CQL(rho) implemented?

[raunakkbanerjee]

Hello @takuseno,
For adroit tasks, the original paper mentions using CQL(rho). This can be used by setting min_q_version=2 in the code provided by the authors. I am curious if this is implemented in d3rlpy. If so, how can I use this version of CQL?

Many thanks for your time.

[takuseno]

@raunakkbanerjee Hi, thanks for the issue. Currently, it's not supported yet because it's usually performing worse and I don't see people using it in the recent publication. But, since it's actually easy to support in the upcoming release, I'll consider it in the next update. Or, I'm always welcoming contributions from you 😄

Just to be sure, in the rho version, the regularizer will be this simple term:
https://github.com/aviralkumar2907/CQL/blob/d67dbe9cf5d2b96e3b462b6146f249b3d6569796/d4rl/rlkit/torch/sac/cql.py#L245