Scaling Up: Solving POMDPs Through Value Based Clustering

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

10 Scopus citations

Abstract

Partially Observable Markov Decision Processes (POMDPs) provide an appropriately rich model for agents operating under partial knowledge of the environment. Since finding an optimal POMDP policy is intractable, approximation techniques have been a main focus of research, among them point-based algorithms, which scale up relatively well - up to thousands of states. An important decision in a point-based algorithm is the order of backup operations over belief states. Prioritization techniques for ordering the sequence of backup operations reduce the number of needed backups considerably, but involve significant overhead. This paper suggests a new way to order backups, based on a soft clustering of the belief space. Our novel soft clustering method relies on the solution of the underlying MDP. Empirical evaluation verifies that our method rapidly computes a good order of backups, showing orders of magnitude improvement in runtime over a number of benchmarks.

Original languageEnglish
Title of host publicationAAAI-07/IAAI-07 Proceedings
Subtitle of host publication22nd AAAI Conference on Artificial Intelligence and the 19th Innovative Applications of Artificial Intelligence Conference
Pages1290-1295
Number of pages6
StatePublished - 28 Nov 2007
EventAAAI-07/IAAI-07 Proceedings: 22nd AAAI Conference on Artificial Intelligence and the 19th Innovative Applications of Artificial Intelligence Conference - Vancouver, BC, Canada
Duration: 22 Jul 200726 Jul 2007

Publication series

NameProceedings of the National Conference on Artificial Intelligence
Volume2

Conference

ConferenceAAAI-07/IAAI-07 Proceedings: 22nd AAAI Conference on Artificial Intelligence and the 19th Innovative Applications of Artificial Intelligence Conference
Country/TerritoryCanada
CityVancouver, BC
Period22/07/0726/07/07

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Scaling Up: Solving POMDPs Through Value Based Clustering'. Together they form a unique fingerprint.

Cite this