Misc.

• Average-DQN (Anschel et al., 2016) proposes to increase the stability and performance of DQN by replacing the single target network (a copy of the trained network) by an average of the last parameter values, in other words an average of many past target networks.

• He et al. (2016) proposed fast reward propagation through optimality tightening to speedup learning: when rewards are sparse, they require a lot of episodes to propagate these rare rewards to all actions leading to it. Their method combines immediate rewards (single steps) with actual returns (as in Monte Carlo) via a constrained optimization approach.

• Never Give Up: Learning Directed Exploration Strategies (Badia et al., 2020b)

• Agent57: Outperforming the Atari Human Benchmark (Badia et al., 2020a)

• Human-level Atari 200x faster (Kapturowski et al., 2022)

Anschel, O., Baram, N., and Shimkin, N. (2016). Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learning. Available at: http://arxiv.org/abs/1611.01929.

Badia, A. P., Piot, B., Kapturowski, S., Sprechmann, P., Vitvitskyi, A., Guo, D., et al. (2020a). Agent57: Outperforming the Atari Human Benchmark. Available at: http://arxiv.org/abs/2003.13350 [Accessed January 17, 2022].

Badia, A. P., Sprechmann, P., Vitvitskyi, A., Guo, D., Piot, B., Kapturowski, S., et al. (2020b). Never Give Up: Learning Directed Exploration Strategies. Available at: http://arxiv.org/abs/2002.06038 [Accessed January 17, 2022].

He, F. S., Liu, Y., Schwing, A. G., and Peng, J. (2016). Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening. Available at: http://arxiv.org/abs/1611.01606.

Kapturowski, S., Campos, V., Jiang, R., Rakićević, N., van Hasselt, H., Blundell, C., et al. (2022). Human-level Atari 200x faster. doi:10.48550/arXiv.2209.07550.