Reasoning about Human Behavior in Ad Hoc Teamwork

Ad hoc teamwork is a decentralized multi-agent problem in which agents must collaborate online without pre-coordination. An interesting challenge in ad hoc teammate design is working efficiently with human agents, which may require a model of how these agents behave in a team. In this paper, we investigate a scenario in which one of the teammates is a human, as part of a work in progress to construct an ad hoc teammate that can collaborate in mixed human-agent environments. This paper presents an experiment that evaluates human behavior in ad hoc teamwork under three different conditions: A control group which is given a basic set of instructions and two treatment groups which are given varying levels of additional information about the collaborative nature of the task. We measure the users' performance in terms of optimality and legibility. We show that these values are significantly different between the conditions, thus highlighting the importance of acquiring a model that encompasses different human behaviors.