Configurable Markov Decision Processes

Alberto Maria Metelli*, Mirco Mutti*, and Marcello Restelli

European Workshop on Reinforcement Learning 14 (EWRL 14), 2018.

Abstract
In many real-world problems, there is the possibility to configure, to a limited extent, some environmental parameters to improve the performance of a learning agent. In this paper, we propose a novel framework, Configurable Markov Decision Processes (Conf-MDPs), to model this new type of interaction with the environment. Furthermore, we provide a new learning algorithm, Safe Policy-Model Iteration (SPMI), to jointly and adaptively optimize the policy and the environment configuration. After having introduced our approach, we present the experimental evaluation in two explicative problems to show the benefits of the environment configurability on the performance of the learned policy.

[Link] [BibTeX]

 @article{metelli2018configurableWorkshop,
    author = "Metelli*, Alberto Maria and Mutti*, Mirco and Restelli, Marcello",
    title = "Configurable Markov Decision Processes",
    journal = "European Workshop on Reinforcement Learning 14 (EWRL 14)",
    year = "2018",
    url = "https://ewrl.files.wordpress.com/2018/09/ewrl\_14\_2018\_paper\_5.pdf"
}