Truly Batch Model-Free Inverse Reinforcement Learning about Multiple Intentions

Giorgia Ramponi, Amarildo Likmeta, Alberto Maria Metelli, Andrea Tirinzoni, and Marcello Restelli

Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, 2020.

Abstract
We consider Inverse Reinforcement Learning (IRL) about multiple intentions, \ie the problem of estimating the unknown reward functions optimized by a group of experts that demonstrate optimal behaviors. Most of the existing algorithms either require access to a model of the environment or need to repeatedly compute the optimal policies for the hypothesized rewards. However, these requirements are rarely met in real-world applications, in which interacting with the environment can be expensive or even dangerous. In this paper, we address the IRL about multiple intentions in a fully model-free and batch setting. We first cast the single IRL problem as a constrained likelihood maximization and then we use this formulation to cluster agents based on the likelihood of the assignment. In this way, we can efficiently solve, without interactions with the environment, both the IRL and the clustering problem. Finally, we evaluate the proposed methodology on simulated domains and on a real-world social-network application.

[Paper] [BibTeX]

 @InProceedings{ramponi2020truly,
    author = "Ramponi, Giorgia and Likmeta, Amarildo and Metelli, Alberto Maria and Tirinzoni, Andrea and Restelli, Marcello",
    editor = "Chiappa, Silvia and Calandra, Roberto",
    title = "Truly Batch Model-Free Inverse Reinforcement Learning about Multiple Intentions",
    booktitle = "Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics",
    pages = "2359--2369",
    year = "2020",
    volume = "108",
    series = "Proceedings of Machine Learning Research",
    address = "Online",
    publisher = "PMLR",
    pdf = "http://proceedings.mlr.press/v108/ramponi20a/ramponi20a.pdf",
    url = "http://proceedings.mlr.press/v108/ramponi20a.html"
}