Truly Batch Model-Free Inverse Reinforcement Learning about Multiple Intentions

Giorgia Ramponi, Amarildo Likmeta, Alberto Maria Metelli, Andrea Tirinzoni, and Marcello Restelli

International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.

CORE 2020: A   GGS 2018: A+

Abstract
We consider Inverse Reinforcement Learning (IRL) about multiple intentions, \ie the problem of estimating the unknown reward functions optimized by a group of experts that demonstrate optimal behaviors. Most of the existing algorithms either require access to a model of the environment or need to repeatedly compute the optimal policies for the hypothesized rewards. However, these requirements are rarely met in real-world applications, in which interacting with the environment can be expensive or even dangerous. In this paper, we address the IRL about multiple intentions in a fully model-free and batch setting. We first cast the single IRL problem as a constrained likelihood maximization and then we use this formulation to cluster agents based on the likelihood of the assignment. In this way, we can efficiently solve, without interactions with the environment, both the IRL and the clustering problem. Finally, we evaluate the proposed methodology on simulated domains and on a real-world social-network application.

[Link] [Code] [Talk] [BibTeX]

 @incollection{ramponi2020truly,
    author = "Ramponi, Giorgia and Likmeta, Amarildo and Metelli, Alberto Maria and Tirinzoni, Andrea and Restelli, Marcello",
    title = "Truly Batch Model-Free Inverse Reinforcement Learning about Multiple Intentions",
    booktitle = "International Conference on Artificial Intelligence and Statistics ({AISTATS})",
    pages = "2359--2369",
    year = "2020",
    volume = "108",
    publisher = "PMLR",
    url = "http://proceedings.mlr.press/v108/ramponi20a.html"
}