Approximating Euclidean by Imprecise Markov Decision Processes

Publikation: Bidrag til bog/antologi/rapport/konference proceedingKonferenceartikel i proceedingForskningpeer review

1 Citationer (Scopus)

Abstrakt

Euclidean Markov decision processes are a powerful tool for modeling control problems under uncertainty over continuous domains. Finite state imprecise, Markov decision processes can be used to approximate the behavior of these infinite models. In this paper we address two questions: first, we investigate what kind of approximation guarantees are obtained when the Euclidean process is approximated by finite state approximations induced by increasingly fine partitions of the continuous state space. We show that for cost functions over finite time horizons the approximations become arbitrarily precise. Second, we use imprecise Markov decision process approximations as a tool to analyse and validate cost functions and strategies obtained by reinforcement learning. We find that, on the one hand, our new theoretical results validate basic design choices of a previously proposed reinforcement learning approach. On the other hand, the imprecise Markov decision process approximations reveal some inaccuracies in the learned cost functions.
OriginalsprogEngelsk
TitelInternational Symposium on Leveraging Applications of Formal Methods (ISoLA 2020)
RedaktørerTiziana Margaria, Bernhard Steffen
Antal sider15
Vol/bind12476
UdgivelsesstedLNCS
ForlagSpringer
Publikationsdato2020
Sider275-289
ISBN (Trykt)978-3-030-61361-7
ISBN (Elektronisk)978-3-030-61362-4
DOI
StatusUdgivet - 2020
BegivenhedInternational Symposium on Leveraging Applications of Formal Methods 2020: ISoLA 2020 - Rhodos, Grækenland
Varighed: 20 okt. 202030 okt. 2020

Konference

KonferenceInternational Symposium on Leveraging Applications of Formal Methods 2020
LandGrækenland
ByRhodos
Periode20/10/202030/10/2020
NavnLecture Notes in Computer Science
ISSN0302-9743

Fingeraftryk Dyk ned i forskningsemnerne om 'Approximating Euclidean by Imprecise Markov Decision Processes'. Sammen danner de et unikt fingeraftryk.

Citationsformater