random_act

[patricknaughton01]

When trying to use the PPO_RNN agent, I set random_act to some positive number to collect some initial samples. However, this triggers an error when calling record_transition since self._current_log_prob isn't set if random_act is called.

[Toni-SM]

Hi @patricknaughton01

As mentioned in #138 (comment), random samples are not implemented in on-policy algorithms (like PPO) yet.

The main reason, unlike the off-policy algorithms, is what would be the appropriate log_prob values for the random actions generated, since the log_prob will be used to compare the distributions (KL) during optimization... and I haven't really thought about it in depth.