Dealing with multiple experts and non-stationarity in inverse reinforcement learning: an application to real-life problems

Abstract
In real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can be employed to retrieve the reward function implicitly optimized by expert agents acting in real applications. Scaling IRL to real-world cases has proved challenging as typically only a fixed dataset of demonstrations is available and further interactions with the environment are not allowed. For this reason, we resort to a class of truly batch model-free IRL algorithms and we present three application scenarios: (1) the high-level decision-making problem in the highway driving scenario, and (2) inferring the user preferences in a social network (Twitter), and (3) the management of the water release in the Como Lake. For each of these scenarios, we provide formalization, experiments and a discussion to interpret the obtained results.

[Link] [BibTeX]

 @Article{likmeta2021dealing,
    author = "Likmeta, Amarildo and Metelli, Alberto Maria and Ramponi, Giorgia and Tirinzoni, Andrea and Giuliani, Matteo and Restelli, Marcello",
    title = "Dealing with multiple experts and non-stationarity in inverse reinforcement learning: an application to real-life problems",
    journal = "Machine Learning",
    volume = "110",
    number = "9",
    pages = "2541--2576",
    year = "2021",
    url = "https://doi.org/10.1007/s10994-020-05939-8",
    doi = "https://doi.org/10.1007/s10994-020-05939-8"
}