References

Abbeel, P., and Ng, A. Y. (2004). Apprenticeship learning via inverse reinforcement learning. in Proceedings of the twenty-first international conference on Machine learning ICML ’04. (New York, NY, USA: Association for Computing Machinery), 1. doi:10.1145/1015330.1015430.

Amari, S.-I. (1998). Natural gradient works efficiently in learning. Neural Computation 10, 251–276.

Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., et al. (2017). Hindsight Experience Replay. arXiv:1707.01495 [cs]. Available at: https://arxiv.org/abs/1707.01495.

Arjovsky, M., Chintala, S., and Bottou, L. (2017). Wasserstein GAN. arXiv:1701.07875 [cs, stat]. Available at: https://arxiv.org/abs/1701.07875.

Arora, S., and Doshi, P. (2019). A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress. arXiv:1806.06877 [cs, stat]. Available at: https://arxiv.org/abs/1806.06877 [Accessed January 31, 2020].

Aubret, A., Matignon, L., and Hassas, S. (2019). A survey on intrinsic motivation in reinforcement learning. arXiv:1908.06976 [cs]. Available at: https://arxiv.org/abs/1908.06976 [Accessed January 17, 2021].

Bacon, P.-L., Harb, J., and Precup, D. (2016). The Option-Critic Architecture. arXiv:1609.05140 [cs]. Available at: https://arxiv.org/abs/1609.05140 [Accessed January 31, 2020].

Badia, A. P., Piot, B., Kapturowski, S., Sprechmann, P., Vitvitskyi, A., Guo, D., et al. (2020a). Agent57: Outperforming the Atari Human Benchmark. arXiv:2003.13350 [cs, stat]. Available at: https://arxiv.org/abs/2003.13350 [Accessed January 17, 2022].

Badia, A. P., Sprechmann, P., Vitvitskyi, A., Guo, D., Piot, B., Kapturowski, S., et al. (2020b). Never Give Up: Learning Directed Exploration Strategies. arXiv:2002.06038 [cs, stat]. Available at: https://arxiv.org/abs/2002.06038 [Accessed January 17, 2022].

Badrinarayanan, V., Kendall, A., and Cipolla, R. (2016). SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. arXiv:1511.00561 [cs]. Available at: https://arxiv.org/abs/1511.00561 [Accessed November 29, 2020].

Baird, L. C. (1993). Advantage updating. Wright-Patterson Air Force Base.

Bakker, B. (2001). Reinforcement Learning with Long Short-Term Memory. in Advances in Neural Information Processing Systems 14 (NIPS 2001), 1475–1482.

Barth-Maron, G., Hoffman, M. W., Budden, D., Dabney, W., Horgan, D., TB, D., et al. (2018). Distributed Distributional Deterministic Policy Gradients. arXiv:1804.08617 [cs]. Available at: https://arxiv.org/abs/1804.08617.

Barto, A. G. (2013). “Intrinsic Motivation and Reinforcement Learning,” in Intrinsically Motivated Learning in Natural and Artificial Systems, eds. G. Baldassarre and M. Mirolli (Berlin, Heidelberg: Springer), 17–47. doi:10.1007/978-3-642-32375-1_2.

Beck, J., Vuorio, R., Liu, E. Z., Xiong, Z., Zintgraf, L., Finn, C., et al. (2023). A Survey of Meta-Reinforcement Learning. doi:10.48550/arXiv.2301.08028.

Belkhale, S., Li, R., Kahn, G., McAllister, R., Calandra, R., and Levine, S. (2021). Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads. IEEE Robotics and Automation Letters, 1–1. doi:10.1109/LRA.2021.3057046.

Bellemare, M. G., Dabney, W., and Munos, R. (2017). A Distributional Perspective on Reinforcement Learning. arXiv:1707.06887 [cs]. Available at: https://arxiv.org/abs/1707.06887.

Bellemare, M. G., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., and Munos, R. (2016). Unifying Count-Based Exploration and Intrinsic Motivation. doi:10.48550/arXiv.1606.01868.

Bishop, C. M. (1994). Mixture Density Networks. Birmingham, UK: Neural Computing Research Group, Aston University.

Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., et al. (2016). End to End Learning for Self-Driving Cars. arXiv:1604.07316 [cs]. Available at: https://arxiv.org/abs/1604.07316 [Accessed September 17, 2020].

Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., and Hassabis, D. (2019). Reinforcement Learning, Fast and Slow. Trends in Cognitive Sciences 23, 408–422. doi:10.1016/j.tics.2019.02.006.

Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., and Efros, A. A. (2018a). Large-Scale Study of Curiosity-Driven Learning. arXiv:1808.04355 [cs, stat]. Available at: https://arxiv.org/abs/1808.04355 [Accessed February 6, 2021].

Burda, Y., Edwards, H., Storkey, A., and Klimov, O. (2018b). Exploration by Random Network Distillation. doi:10.48550/arXiv.1810.12894.

Chen, L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., et al. (2021). Decision Transformer: Reinforcement Learning via Sequence Modeling. arXiv:2106.01345 [cs]. Available at: https://arxiv.org/abs/2106.01345 [Accessed December 2, 2021].

Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., et al. (2023). Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. arXiv.org.

Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., et al. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation.

Chou, P.-W., Maturana, D., and Scherer, S. (2017). Improving Stochastic Policy Gradients in Continuous Control with Deep Reinforcement Learning using the Beta Distribution. in International Conference on Machine Learning.

Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. (2017). Deep Reinforcement Learning from Human Preferences. in Advances in Neural Information Processing Systems (Curran Associates, Inc.).

Co-Reyes, J. D., Liu, Y., Gupta, A., Eysenbach, B., Abbeel, P., and Levine, S. (2018). Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings.

Dabney, W., Rowland, M., Bellemare, M. G., and Munos, R. (2017). Distributional Reinforcement Learning with Quantile Regression. arXiv:1710.10044 [cs, stat]. Available at: https://arxiv.org/abs/1710.10044 [Accessed June 28, 2019].

Dayan, P., and Hinton, G. E. (1992). Feudal Reinforcement Learning. in Advances in Neural Information Processing Systems (Morgan-Kaufmann).

Dayan, P., and Niv, Y. (2008). Reinforcement learning: The Good, The Bad and The Ugly. Current Opinion in Neurobiology 18, 185–196. doi:10.1016/j.conb.2008.08.003.

Degrave, J., Felici, F., Buchli, J., Neunert, M., Tracey, B., Carpanese, F., et al. (2022). Magnetic control of tokamak plasmas through deep reinforcement learning. Nature 602, 414–419. doi:10.1038/s41586-021-04301-9.

Degris, T., White, M., and Sutton, R. S. (2012). Linear Off-Policy Actor-Critic. in Proceedings of the 2012 International Conference on Machine Learning.

Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., and Abbeel, P. (2016). RL$^2$: Fast Reinforcement Learning via Slow Reinforcement Learning. arXiv:1611.02779 [cs, stat]. Available at: https://arxiv.org/abs/1611.02779 [Accessed February 5, 2021].

Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., and Clune, J. (2021). First return, then explore. Nature 590, 580–586. doi:10.1038/s41586-020-03157-9.

Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., et al. (2018). IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures. doi:10.48550/arXiv.1802.01561.

Finn, C., Abbeel, P., and Levine, S. (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arXiv:1703.03400 [cs]. Available at: https://arxiv.org/abs/1703.03400 [Accessed February 6, 2021].

Fortunato, M., Azar, M. G., Piot, B., Menick, J., Osband, I., Graves, A., et al. (2017). Noisy Networks for Exploration. arXiv:1706.10295 [cs, stat]. Available at: https://arxiv.org/abs/1706.10295 [Accessed March 2, 2020].

Frans, K., Ho, J., Chen, X., Abbeel, P., and Schulman, J. (2017). Meta Learning Shared Hierarchies. arXiv:1710.09767 [cs]. Available at: https://arxiv.org/abs/1710.09767 [Accessed January 31, 2020].

Fu, J., Luo, K., and Levine, S. (2018). Learning Robust Rewards with Adversarial Inverse Reinforcement Learning. doi:10.48550/arXiv.1710.11248.

Fujimoto, S., Meger, D., and Precup, D. (2019). Off-Policy Deep Reinforcement Learning without Exploration. in Proceedings of the 36th International Conference on Machine Learning (PMLR), 2052–2062.

Fujimoto, S., van Hoof, H., and Meger, D. (2018). Addressing Function Approximation Error in Actor-Critic Methods. arXiv:1802.09477 [cs, stat]. Available at: https://arxiv.org/abs/1802.09477 [Accessed March 1, 2020].

Gers, F. (2001). Long Short-Term Memory in Recurrent Neural Networks.

Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al. (2014). Generative Adversarial Networks. arXiv:1406.2661 [cs]. Available at: https://arxiv.org/abs/1406.2661.

Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning. MIT Press.

Gruslys, A., Dabney, W., Azar, M. G., Piot, B., Bellemare, M., and Munos, R. (2017). The Reactor: A fast and sample-efficient Actor-Critic agent for Reinforcement Learning. arXiv:1704.04651 [cs]. Available at: https://arxiv.org/abs/1704.04651.

Gu, S., Holly, E., Lillicrap, T., and Levine, S. (2017). Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates. in Proc. ICRA.

Ha, D., and Eck, D. (2017). A Neural Representation of Sketch Drawings. arXiv:1704.03477 [cs, stat]. Available at: https://arxiv.org/abs/1704.03477 [Accessed January 17, 2021].

Ha, D., and Schmidhuber, J. (2018). World Models. arXiv:1803.10122 [cs]. doi:10.5281/zenodo.1207631.

Haarnoja, T., Hartikainen, K., Abbeel, P., and Levine, S. (2018a). Latent Space Policies for Hierarchical Reinforcement Learning.

Haarnoja, T., Tang, H., Abbeel, P., and Levine, S. (2017). Reinforcement Learning with Deep Energy-Based Policies. arXiv:1702.08165 [cs]. Available at: https://arxiv.org/abs/1702.08165 [Accessed February 13, 2019].

Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018b). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. arXiv:1801.01290 [cs]. Available at: https://arxiv.org/abs/1801.01290.

Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., et al. (2018c). Soft Actor-Critic Algorithms and Applications. arXiv:1812.05905 [cs, stat]. Available at: https://arxiv.org/abs/1812.05905 [Accessed February 5, 2019].

Hafner, D., Lillicrap, T., Ba, J., and Norouzi, M. (2020). Dream to Control: Learning Behaviors by Latent Imagination. arXiv:1912.01603 [cs]. Available at: https://arxiv.org/abs/1912.01603 [Accessed March 24, 2020].

Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., et al. (2019). Learning Latent Dynamics for Planning from Pixels. arXiv:1811.04551 [cs, stat]. Available at: https://arxiv.org/abs/1811.04551 [Accessed January 24, 2020].

Hafner, D., Lillicrap, T., Norouzi, M., and Ba, J. (2022). Mastering Atari with Discrete World Models. Available at: https://arxiv.org/abs/2010.02193 [Accessed July 28, 2022].

Hafner, D., Pasukonis, J., Ba, J., and Lillicrap, T. (2024). Mastering Diverse Domains through World Models. doi:10.48550/arXiv.2301.04104.

Hafner, R., and Riedmiller, M. (2011). Reinforcement learning in feedback control. Machine Learning 84, 137–169. doi:10.1007/s10994-011-5235-x.

Hansen, N., and Ostermeier, A. (2001). Completely Derandomized Self-Adaptation in Evolution Strategies. Evolutionary Computation 9, 159–195. doi:10.1162/106365601750190398.

Harutyunyan, A., Bellemare, M. G., Stepleton, T., and Munos, R. (2016). Q(λ) with off-policy corrections.

Hausknecht, M., and Stone, P. (2015). Deep Recurrent Q-Learning for Partially Observable MDPs. arXiv:1507.06527 [cs]. Available at: https://arxiv.org/abs/1507.06527.

He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv:1512.03385 [cs]. Available at: https://arxiv.org/abs/1512.03385.

Heess, N., Wayne, G., Silver, D., Lillicrap, T., Tassa, Y., and Erez, T. (2015). Learning continuous control policies by stochastic value gradients. Proc. International Conference on Neural Information Processing Systems, 2944–2952.

Hessel, M., Modayil, J., van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., et al. (2017). Rainbow: Combining Improvements in Deep Reinforcement Learning. arXiv:1707.06887 [cs]. Available at: https://arxiv.org/abs/1707.06887.

Ho, J., and Ermon, S. (2016). Generative Adversarial Imitation Learning. doi:10.48550/arXiv.1606.03476.

Hochreiter, S. (1991). Untersuchungen zu dynamischen neuronalen Netzen.

Hochreiter, S., and Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation 9, 1735–1780. doi:10.1162/neco.1997.9.8.1735.

Horgan, D., Quan, J., Budden, D., Barth-Maron, G., Hessel, M., van Hasselt, H., et al. (2018). Distributed Prioritized Experience Replay. arXiv:1803.00933 [cs]. Available at: https://arxiv.org/abs/1803.00933 [Accessed December 14, 2019].

Ibarz, B., Leike, J., Pohlen, T., Irving, G., Legg, S., and Amodei, D. (2018). Reward learning from human preferences and demonstrations in Atari. arXiv:1811.06521 [cs, stat]. Available at: https://arxiv.org/abs/1811.06521 [Accessed May 16, 2020].

Ioffe, S., and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv:1502.03167 [cs.LG]. Available at: https://arxiv.org/abs/1502.03167.

Kakade, S. (2001). A Natural Policy Gradient. in Advances in Neural Information Processing Systems 14.

Kakade, S., and Langford, J. (2002). Approximately Optimal Approximate Reinforcement Learning. Proc. 19th International Conference on Machine Learning, 267–274.

Kapturowski, S., Ostrovski, G., Quan, J., Munos, R., and Dabney, W. (2019). Recurrent experience replay in distributed reinforcement learning. in ICLR, 19.

Kaufmann, E., Bauersfeld, L., Loquercio, A., Müller, M., Koltun, V., and Scaramuzza, D. (2023). Champion-level drone racing using deep reinforcement learning. Nature 620, 982–987. doi:10.1038/s41586-023-06419-4.

Kendall, A., Hawke, J., Janz, D., Mazur, P., Reda, D., Allen, J.-M., et al. (2018). Learning to Drive in a Day. arXiv:1807.00412 [cs, stat]. Available at: https://arxiv.org/abs/1807.00412 [Accessed December 19, 2018].

Kidambi, R., Rajeswaran, A., Netrapalli, P., and Joachims, T. (2021). MOReL : Model-Based Offline Reinforcement Learning. arXiv:2005.05951 [cs, stat]. Available at: https://arxiv.org/abs/2005.05951 [Accessed October 5, 2021].

Kingma, D. P., and Welling, M. (2013). Auto-Encoding Variational Bayes. arXiv:1312.6114 [cs]. Available at: https://arxiv.org/abs/1312.6114.

Knight, E., and Lerner, O. (2018). Natural Gradient Deep Q-learning.

Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. in Advances in Neural Information Processing Systems (NIPS).

Kumar, A., Zhou, A., Tucker, G., and Levine, S. (2020). Conservative Q-Learning for Offline Reinforcement Learning. arXiv:2006.04779 [cs, stat]. Available at: https://arxiv.org/abs/2006.04779 [Accessed June 13, 2020].

Levine, S. (2018). Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review. arXiv:1805.00909 [cs, stat]. Available at: https://arxiv.org/abs/1805.00909 [Accessed December 18, 2020].

Levine, S., and Koltun, V. (2013). Guided Policy Search. in Proceedings of Machine Learning Research, 1–9.

Levine, S., Kumar, A., Tucker, G., and Fu, J. (2020). Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv:2005.01643 [cs, stat]. Available at: https://arxiv.org/abs/2005.01643 [Accessed May 10, 2020].

Levy, A., Konidaris, G., Platt, R., and Saenko, K. (2019). Learning Multi-Level Hierarchies with Hindsight. arXiv:1712.00948 [cs]. Available at: https://arxiv.org/abs/1712.00948 [Accessed January 31, 2020].

Li, W., Zhu, Y., and Zhao, D. (2022). Missile guidance with assisted deep reinforcement learning for head-on interception of maneuvering target. Complex & Intelligent Systems 8, 1205–1216. doi:10.1007/s40747-021-00577-6.

Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., et al. (2015). Continuous control with deep reinforcement learning. CoRR.

Lötzsch, W., Vitay, J., and Hamker, F. H. (2017). Training a deep policy gradient-based neural network with asynchronous learners on a simulated robotic problem. in INFORMATIK 2017. Gesellschaft für Informatik, eds. M. Eibl and M. Gaedke (Gesellschaft für Informatik, Bonn), 2143–2154.

Luo, J., Paduraru, C., Voicu, O., Chervonyi, Y., Munns, S., Li, J., et al. (2022). Controlling Commercial Cooling Systems Using Reinforcement Learning. doi:10.48550/arXiv.2211.07357.

Machado, M. C., Bellemare, M. G., and Bowling, M. (2018). Count-Based Exploration with the Successor Representation. arXiv:1807.11622 [cs, stat]. Available at: https://arxiv.org/abs/1807.11622 [Accessed February 23, 2019].

Madeka, D., Torkkola, K., Eisenach, C., Luo, A., Foster, D. P., and Kakade, S. M. (2022). Deep Inventory Management. doi:10.48550/arXiv.2210.03137.

Malibari, N., Katib, I., and Mehmood, R. (2023). Systematic Review on Reinforcement Learning in the Field of Fintech. doi:10.48550/arXiv.2305.07466.

Meuleau, N., Peshkin, L., Kaelbling, L. P., and Kim, K. (2000). Off-Policy Policy Search.

Micheli, V., Alonso, E., and Fleuret, F. (2022). Transformers are Sample Efficient World Models. doi:10.48550/arXiv.2209.00588.

Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., et al. (2016). Asynchronous Methods for Deep Reinforcement Learning. in Proc. ICML.

Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., et al. (2013). Playing Atari with Deep Reinforcement Learning. arXiv:1312.5602 [cs]. Available at: https://arxiv.org/abs/1312.5602.

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., et al. (2015). Human-level control through deep reinforcement learning. Nature 518, 529–533. doi:10.1038/nature14236.

Munos, R., Stepleton, T., Harutyunyan, A., and Bellemare, M. G. (2016). Safe and Efficient Off-Policy Reinforcement Learning.

Nachum, O., Gu, S., Lee, H., and Levine, S. (2018). Data-Efficient Hierarchical Reinforcement Learning.

Nachum, O., Norouzi, M., Xu, K., and Schuurmans, D. (2017). Bridging the Gap Between Value and Policy Based Reinforcement Learning. arXiv:1702.08892 [cs, stat]. Available at: https://arxiv.org/abs/1702.08892 [Accessed June 12, 2019].

Nagabandi, A., Kahn, G., Fearing, R. S., and Levine, S. (2017). Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning. arXiv:1708.02596 [cs]. Available at: https://arxiv.org/abs/1708.02596 [Accessed March 3, 2019].

Nair, A., Srinivasan, P., Blackwell, S., Alcicek, C., Fearon, R., De Maria, A., et al. (2015). Massively Parallel Methods for Deep Reinforcement Learning. arXiv:1507.04296 [cs]. Available at: https://arxiv.org/abs/1507.04296.

Ng, A. Y., Harada, D., and Russell, S. J. (1999). Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping. in Proceedings of the Sixteenth International Conference on Machine Learning ICML ’99. (San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.), 278–287.

Ng, A. Y., and Russell, S. J. (2000). Algorithms for Inverse Reinforcement Learning. in Proceedings of the Seventeenth International Conference on Machine Learning ICML ’00. (San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.), 663–670.

Nielsen, M. A. (2015). Neural Networks and Deep Learning. Determination Press.

Niu, F., Recht, B., Re, C., and Wright, S. J. (2011). HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent. in Proc. Advances in Neural Information Processing Systems, 21–21.

O’Donoghue, B., Munos, R., Kavukcuoglu, K., and Mnih, V. (2016). Combining policy gradient and Q-learning. arXiv:1611.01626 [cs, math, stat]. Available at: https://arxiv.org/abs/1611.01626 [Accessed February 13, 2019].

Oh, J., Guo, Y., Singh, S., and Lee, H. (2018). Self-Imitation Learning.

OpenAI, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., et al. (2019). Dota 2 with Large Scale Deep Reinforcement Learning. doi:10.48550/arXiv.1912.06680.

Ostrovski, G., Bellemare, M. G., Oord, A. van den, and Munos, R. (2017). Count-Based Exploration with Neural Density Models. doi:10.48550/arXiv.1703.01310.

Oudeyer, P.-Y., Gottlieb, J., and Lopes, M. (2016). “Chapter 11 - Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies,” in Progress in Brain Research Motivation., eds. B. Studer and S. Knecht (Elsevier), 257–284. doi:10.1016/bs.pbr.2016.05.005.

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., et al. (2022). Training language models to follow instructions with human feedback. doi:10.48550/arXiv.2203.02155.

Pateria, S., Subagdja, B., Tan, A., and Quek, C. (2021). Hierarchical Reinforcement Learning: A Comprehensive Survey. ACM Computing Surveys 54, 109:1–109:35. doi:10.1145/3453160.

Pathak, D., Agrawal, P., Efros, A. A., and Darrell, T. (2017). Curiosity-driven Exploration by Self-supervised Prediction. arXiv:1705.05363 [cs, stat]. Available at: https://arxiv.org/abs/1705.05363 [Accessed February 6, 2021].

Peshkin, L., and Shelton, C. R. (2002). Learning from Scarce Experience.

Peters, J., and Schaal, S. (2008). Reinforcement learning of motor skills with policy gradients. Neural Networks 21, 682–697. doi:10.1016/j.neunet.2008.02.003.

Pong, V., Gu, S., Dalal, M., and Levine, S. (2018). Temporal Difference Models: Model-Free Deep RL for Model-Based Control. arXiv:1802.09081 [cs]. Available at: https://arxiv.org/abs/1802.09081.

Popov, I., Heess, N., Lillicrap, T., Hafner, R., Barth-Maron, G., Vecerik, M., et al. (2017). Data-efficient Deep Reinforcement Learning for Dexterous Manipulation. arXiv:1704.03073 [cs]. Available at: https://arxiv.org/abs/1704.03073.

Precup, D., Sutton, R. S., and Singh, S. (2000). Eligibility traces for off-policy policy evaluation. in Proceedings of the Seventeenth International Conference on Machine Learning.

Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. in Advances in Neural Information Processing Systems (Curran Associates, Inc.), 53728–53741. doi:10.52202/075280-2338.

Rakelly, K., Zhou, A., Quillen, D., Finn, C., and Levine, S. (2019). Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables. arXiv:1903.08254 [cs, stat]. Available at: https://arxiv.org/abs/1903.08254 [Accessed February 5, 2021].

Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597 [cs]. Available at: https://arxiv.org/abs/1505.04597 [Accessed November 29, 2020].

Roy, R., Raiman, J., Kant, N., Elkin, I., Kirby, R., Siu, M., et al. (2022). PrefixRL: Optimization of Parallel Prefix Circuits using Deep Reinforcement Learning. doi:10.1109/DAC18074.2021.9586094.

Ruder, S. (2016). An overview of gradient descent optimization algorithms.

Rusu, A. A., Colmenarejo, S. G., Gulcehre, C., Desjardins, G., Kirkpatrick, J., Pascanu, R., et al. (2016). Policy Distillation. arXiv:1511.06295 [cs]. Available at: https://arxiv.org/abs/1511.06295 [Accessed January 26, 2020].

Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. (2017). Evolution Strategies as a Scalable Alternative to Reinforcement Learning.

Savinov, N., Raichuk, A., Marinier, R., Vincent, D., Pollefeys, M., Lillicrap, T., et al. (2019). Episodic Curiosity through Reachability. doi:10.48550/arXiv.1810.02274.

Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized Experience Replay. arXiv:1511.05952 [cs]. Available at: https://arxiv.org/abs/1511.05952.

Schmidhuber, J. (2010). Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010). IEEE Transactions on Autonomous Mental Development 2, 230–247. doi:10.1109/TAMD.2010.2056368.

Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., et al. (2019). Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. arXiv:1911.08265 [cs, stat]. Available at: https://arxiv.org/abs/1911.08265 [Accessed November 24, 2019].

Schulman, J., Chen, X., and Abbeel, P. (2017a). Equivalence Between Policy Gradients and Soft Q-Learning. arXiv:1704.06440 [cs]. Available at: https://arxiv.org/abs/1704.06440 [Accessed June 12, 2019].

Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015a). Trust Region Policy Optimization. in Proceedings of the 31 st International Conference on Machine Learning, 1889–1897.

Schulman, J., Moritz, P., Levine, S., Jordan, M., and Abbeel, P. (2015b). High-Dimensional Continuous Control Using Generalized Advantage Estimation. arXiv:1506.02438 [cs]. Available at: https://arxiv.org/abs/1506.02438.

Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017b). Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs]. Available at: https://arxiv.org/abs/1707.06347.

Sekar, R., Rybkin, O., Daniilidis, K., Abbeel, P., Hafner, D., and Pathak, D. (2020). Planning to Explore via Self-Supervised World Models. Available at: https://arxiv.org/abs/2005.05960 [Accessed January 29, 2024].

Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489. doi:10.1038/nature16961.

Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., et al. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 1140–1144. doi:10.1126/science.aar6404.

Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014). Deterministic Policy Gradient Algorithms. in Proc. ICML Proceedings of Machine Learning Research., eds. E. P. Xing and T. Jebara (PMLR), 387–395.

Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. International Conference on Learning Representations (ICRL), 1–14. doi:10.1016/j.infsof.2008.09.005.

Sutton, R. S. (1990). Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming. Machine Learning Proceedings 1990, 216–224. doi:10.1016/B978-1-55860-141-3.50030-4.

Sutton, R. S., and Barto, A. G. (1998). Reinforcement Learning: An introduction. Cambridge, MA: MIT press.

Sutton, R. S., and Barto, A. G. (2017). Reinforcement Learning: An Introduction. 2nd ed. Cambridge, MA: MIT Press.

Sutton, R. S., McAllester, D., Singh, S., and Mansour, Y. (1999a). Policy gradient methods for reinforcement learning with function approximation. in Proceedings of the 12th International Conference on Neural Information Processing Systems (MIT Press), 1057–1063.

Sutton, R. S., Precup, D., and Singh, S. (1999b). Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112, 181–211. doi:10.1016/S0004-3702(99)00052-1.

Szita, I., and Lörincz, A. (2006). Learning Tetris Using the Noisy Cross-Entropy Method. Neural Computation 18, 2936–2941. doi:10.1162/neco.2006.18.12.2936.

Tang, J., and Abbeel, P. (2010). On a Connection between Importance Sampling and the Likelihood Ratio Policy Gradient. in Adv. Neural inf. Process. Syst.

Teh, Y. W., Bapst, V., Czarnecki, W. M., Quan, J., Kirkpatrick, J., Hadsell, R., et al. (2017). Distral: Robust Multitask Reinforcement Learning. arXiv:1707.04175 [cs, stat]. Available at: https://arxiv.org/abs/1707.04175 [Accessed January 26, 2020].

Todorov, E. (2008). General duality between optimal control and estimation. in 2008 47th IEEE Conference on Decision and Control, 4286–4292. doi:10.1109/CDC.2008.4739438.

Toussaint, M. (2009). Robot Trajectory Optimization Using Approximate Inference. in Proceedings of the 26th Annual International Conference on Machine Learning ICML ’09. (Montreal, Quebec, Canada: ACM), 1049–1056. doi:10.1145/1553374.1553508.

Uhlenbeck, G. E., and Ornstein, L. S. (1930). On the Theory of the Brownian Motion. Physical Review 36. doi:10.1103/PhysRev.36.823.

van Hasselt, H. (2010). Double Q-learning. in Proceedings of the 23rd International Conference on Neural Information Processing Systems - Volume 2 (Curran Associates Inc.), 2613–2621.

van Hasselt, H., Guez, A., and Silver, D. (2015). Deep Reinforcement Learning with Double Q-learning. arXiv:1509.06461 [cs]. Available at: https://arxiv.org/abs/1509.06461.

Vezhnevets, A. S., Osindero, S., Schaul, T., Heess, N., Jaderberg, M., Silver, D., et al. (2017). FeUdal Networks for Hierarchical Reinforcement Learning. arXiv:1703.01161 [cs]. Available at: https://arxiv.org/abs/1703.01161 [Accessed January 31, 2020].

Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., et al. (2017). Learning to reinforcement learn. arXiv:1611.05763 [cs, stat]. Available at: https://arxiv.org/abs/1611.05763 [Accessed February 5, 2021].

Wang, Z., Li, Z., Mandlekar, A., Xu, Z., Fan, J., Narang, Y., et al. (2024). One-Step Diffusion Policy: Fast Visuomotor Policies via Diffusion Distillation. doi:10.48550/arXiv.2410.21257.

Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., and de Freitas, N. (2016). Dueling Network Architectures for Deep Reinforcement Learning. arXiv:1511.06581 [cs]. Available at: https://arxiv.org/abs/1511.06581 [Accessed November 21, 2019].

Watkins, C. J. (1989). Learning from delayed rewards.

Weber, T., Racanière, S., Reichert, D. P., Buesing, L., Guez, A., Rezende, D. J., et al. (2017). Imagination-Augmented Agents for Deep Reinforcement Learning. arXiv:1707.06203 [cs]. Available at: https://arxiv.org/abs/1707.06203.

Wierstra, D., Foerster, A., Peters, J., and Schmidhuber, J. (2007). “Solving Deep Memory POMDPs with Recurrent Policy Gradients,” in (Springer, Berlin, Heidelberg), 697–706. doi:10.1007/978-3-540-74690-4_71.

Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8, 229–256.

Williams, R. J., and Peng, J. (1991). Function optimization using connectionist reinforcement learning algorithms. Connection Science 3, 241–268.

Wu, P., Escontrela, A., Hafner, D., Goldberg, K., and Abbeel, P. (2022). DayDreamer: World Models for Physical Robot Learning. doi:10.48550/arXiv.2206.14176.

Ye, W., Liu, S., Kurutach, T., Abbeel, P., and Gao, Y. (2021). Mastering Atari Games with Limited Data. doi:10.48550/arXiv.2111.00210.

Yu, C., Liu, J., and Nemati, S. (2020). Reinforcement Learning in Healthcare: A Survey. doi:10.48550/arXiv.1908.08796.

Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., et al. (2019). Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning. arXiv:1910.10897 [cs, stat]. Available at: https://arxiv.org/abs/1910.10897 [Accessed October 30, 2019].

Ziebart, B. D., Maas, A., Bagnell, J. A., and Dey, A. K. (2008). Maximum Entropy Inverse Reinforcement Learning. in AAAI Conference on Artificial Intelligence, 6.