Prioritizing point-based POMDP solvers

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

12 Scopus citations

Abstract

Recent scaling up of POMDP solvers towards realistic applications is largely due to point-based methods such as PBVI, Perseus, and HSVI, which quickly converge to an approximate solution for medium-sized problems. These algorithms improve a value function by using backup operations over a single belief point. In the simpler domain of MDP solvers, prioritizing the order of equivalent backup operations on states is well known to speed up convergence. We generalize the notion of prioritized backups to the POMDP framework, and show that the ordering of backup operations on belief points is important. We also present a new algorithm, Prioritized Value Iteration (PVI), and show empirically that it outperforms current point-based algorithms. Finally, a new empirical evaluation measure, based on the number of backups and the number of belief points, is proposed, in order to provide more accurate benchmark comparisons.

Original languageEnglish
Title of host publicationMachine Learning
Subtitle of host publicationECML 2006 - 17th European Conference on Machine Learning, Proceedings
PublisherSpringer Verlag
Pages389-400
Number of pages12
ISBN (Print)354045375X, 9783540453758
DOIs
StatePublished - 1 Jan 2006
Event17th European Conference on Machine Learning, ECML 2006 - Berlin, Germany
Duration: 18 Sep 200622 Sep 2006

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4212 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference17th European Conference on Machine Learning, ECML 2006
Country/TerritoryGermany
CityBerlin
Period18/09/0622/09/06

Fingerprint

Dive into the research topics of 'Prioritizing point-based POMDP solvers'. Together they form a unique fingerprint.

Cite this