Collaborative distributed sensor management for multitarget tracking using hierarchical Markov decision processes

In this paper, we consider the problem of collaborative sensor management with particular application to using unmanned aerial vehicles (UAVs) for multitarget tracking. The problem of decentralized cooperative control considered in this paper is an optimization of the information obtained by a number of unmanned aerial vehicles (UAVs) equipped with Ground Moving Target Indicator (GMTI) radars, carrying out surveillance over a region which includes a number of confirmed and suspected moving targets. The goal is to track confirmed targets and detect new targets in the area. Each UAV has to decide on the most optimal path with the objective to track as many targets as possible maximizing the information obtained during its operation with the maximum possible accuracy at the lowest possible cost. Limited communication between UAVs and uncertainty in the information obtained by each UAV regarding the location of the ground targets are addressed in the problem formulation. In order to handle these issues, the problem is presented as a decentralized operation of a group of decision-makers lacking full observability of the global state of the system. Markov Decision Processes (MDPs) are incorporated into the solution. Given the MDP model, a local policy of actions for a single agent (UAV) is given by a mapping from a current partial view of a global state observed by an agent to actions. The available probability model regarding possible and confirmed locations of the targets is considered in the computations of the UAVs' policies. The authors present multi-level hierarchy of MDPs controlling each of the UAVs. Each level in the hierarchy solves a problem at a different level of abstraction. Simulation results are presented on a representative multisensor-multitarget tracking problem.