Behavior-Based Quality Assurance in Crowdsourcing Markets

Michael Feldman, Abraham Bernstein

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Quality assurance in crowdsourcing markets has appeared to be an acute problem over the last years. We propose a quality control method inspired by Statistical Process Control (SPC), commonly used to control output quality in production processes and characterized by relying on time-series data. Behavioral traces of users may play a key role in evaluating the performance of work done on crowdsourcing platforms. Therefore, in our experiment we explore fifteen behavioral traces for their ability to recognize the drop in work quality. Preliminary results indicate that our method has a high potential for real-time detection and signaling a drop in work quality.

Original languageEnglish
Title of host publicationProceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014
EditorsJeffrey P. Bigham, David Parkes
PublisherAAAI press
Pages14-15
Number of pages2
ISBN (Electronic)9781577356820
DOIs
StatePublished - 5 Nov 2014
Externally publishedYes
Event2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014 - Pittsburgh, United States
Duration: 2 Nov 20144 Nov 2014

Publication series

NameProceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014

Conference

Conference2nd AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2014
Country/TerritoryUnited States
CityPittsburgh
Period2/11/144/11/14

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Human-Computer Interaction

Fingerprint

Dive into the research topics of 'Behavior-Based Quality Assurance in Crowdsourcing Markets'. Together they form a unique fingerprint.

Cite this