Facilitating the evaluation of automated negotiators using Peer Designed Agents

Raz Lin, Sarit Kraus, Yinon Oshrat, Ya'akov Gal

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

24 Scopus citations

Abstract

Computer agents are increasingly deployed in settings in which they make decisions with people, such as electronic commerce, collaborative interfaces, and cognitive assistants. However, the scientific evaluation of computational strategies for human-computer decision-making is a costly process, involving time, effort and personnel. This paper investigates the use of Peer Designed Agents (PDA)-computer agents developed by human subjects-as a tool for facilitating the evaluation process of automatic negotiators that were developed by researchers. It compares the performance between automatic negotiators that interacted with PDAs to automatic negotiators that interacted with actual people in different domains. The experiments included more than 300 human subjects and 50 PDAs developed by students. Results showed that the automatic negotiators outperformed PDAs in the same situations in which they outperformed people, and that on average, they exhibited the same measure of generosity towards their negotiation partners. These patterns occurred for all types of domains, and for all types of automated negotiators, despite the fact that there were individual differences between the behavior of PDAs and people. The study thus provides an empirical proof that PDAs can alleviate the evaluation process of automatic negotiators, and facilitate their design.

Original languageEnglish
Title of host publicationAAAI-10 / IAAI-10 - Proceedings of the 24th AAAI Conference on Artificial Intelligence and the 22nd Innovative Applications of Artificial Intelligence Conference
PublisherAI Access Foundation
Pages817-822
Number of pages6
ISBN (Print)9781577354659
StatePublished - 1 Jan 2010
Event24th AAAI Conference on Artificial Intelligence and the 22nd Innovative Applications of Artificial Intelligence Conference, AAAI-10 / IAAI-10 - Atlanta, GA, United States
Duration: 11 Jul 201015 Jul 2010

Publication series

NameProceedings of the National Conference on Artificial Intelligence
Volume2

Conference

Conference24th AAAI Conference on Artificial Intelligence and the 22nd Innovative Applications of Artificial Intelligence Conference, AAAI-10 / IAAI-10
Country/TerritoryUnited States
CityAtlanta, GA
Period11/07/1015/07/10

Fingerprint

Dive into the research topics of 'Facilitating the evaluation of automated negotiators using Peer Designed Agents'. Together they form a unique fingerprint.

Cite this