Stochastic over-subscription planning using hierarchies of MDPs

Nicolas Meuleau, Ronen Brafman, Emmanuel Benazera

Research output: Contribution to conferencePaperpeer-review

11 Scopus citations


In over-subscription planning (OSP), the set of goals is not achievable jointly, and the task is to find a plan that attains the best feasible subset of goals given resource constraints. Recent classical OSP algorithms ignore the uncertainly inherent in many natural application domains where OSPs arise. And while modeling stochastic OSP problems as MDPs is easy, the resulting models are too large for standard solution approaches. Fortunately OSP problems have a natural twotiered hierarchy, and in this paper we adapt and extend tools developed in the hierarchical reinforcement learning community in order to effectively exploit this hierarchy and obtain compact, factored policies. Typically, such policies are suboptimal, but under certain assumptions that hold in our planetary exploration domain, our factored solution is, in fact, optimal. Our algorithms work by repeatedly solving a number of smaller MDPs, while propagating information between them. We evaluate a number of variants of this approach on a set of stochastic instances of a planetary rover domain, showing substantial performance gains.

Original languageEnglish
Number of pages10
StatePublished - 24 Jul 2006
Externally publishedYes
EventICAPS 2006 - 16th International Conference on Automated Planning and Scheduling - Cumbria, United Kingdom
Duration: 6 Jun 200610 Jun 2006


ConferenceICAPS 2006 - 16th International Conference on Automated Planning and Scheduling
Country/TerritoryUnited Kingdom

ASJC Scopus subject areas

  • Engineering (all)


Dive into the research topics of 'Stochastic over-subscription planning using hierarchies of MDPs'. Together they form a unique fingerprint.

Cite this