[∼Re] Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress - Action, Cognition, Interaction et Décisions Encorporées
Article Dans Une Revue The ReScience journal Année : 2024

[∼Re] Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress

Résumé

The goal of a model-based reinforcement learning agent is to maximize external returns by understanding the structure of the environment it explores. However, simple reinforcement learning agents may struggle with tasks where external rewards are scarce. Taking inspiration from developmental sciences which show that infants spontaneously explore their environment in the absence of extrinsic rewards, intrinsically motivated reinforcement learning investigates how agents can explore their environments with little extrinsic motivation. In 2012, Lopes and colleagues proposed two new intrinsically motivated models in model-based reinforcement learning [1]. Building upon two optimistic-in-the-face-of-uncertainty agents, they demonstrate theoretical convergence properties for one of their agents, and provide three experiments to show that their models are more versatile than state-of-the-art models. However, due to missing information in their protocol, we only managed to partially reproduce the results of the original article. In our reproduction article, we show the results obtained for several alternative ways to interpret the text of the original article, and discuss what this implies in terms of performance of the different agents. For each tested variant, we performed parameter optimization in order to give the best chances to replicate the figures of the original article.
Fichier principal
Vignette du fichier
Chartouny2024-MBDE.pdf (1.09 Mo) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04683838 , version 1 (02-09-2024)

Identifiants

Citer

Augustin Chartouny, Jean-Pierre Nadal, Mehdi Khamassi. [∼Re] Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress. The ReScience journal, 2024, 73, ⟨10.5281/zenodo.13627804⟩. ⟨hal-04683838⟩
16 Consultations
3 Téléchargements

Altmetric

Partager

More