Inducing Honest Behaviour on Amazon's Mechanical Turk


Arthur Carvalho
Arthur Carvalho
  • Speaker
Rotterdam School of Management (RSM), Erasmus University Rotterdam

Event Information

Type
Research Seminar
Programme
Information Management
Date
Tue. 15 Apr. 2014
Contact
Wolfgang Ketter
Time
12:00-13:30 hours
Location
Mandeville Building T09-67


Abstract

Recent years have seen an increased interest in crowdsourcing as a way of obtaining information from a large group of workers at a reduced cost. It has been widely reported, however, that crowd workers are not necessarily honest when performing a task in the absence of a well-chosen incentive structure. A crucial question that then arises is how to incentivize self-interested workers to behave honestly in crowdsourcing settings.

In this talk, I will describe how economic incentives might induce honest behaviour on Amazon’s Mechanical Turk, currently one of the most popular crowdsourcing platforms. In particular, I will show how in theory a simple payment function that works by rewarding agreements between workers’ reports is enough to induce honest behaviour on Amazon’s Mechanical Turk.

From an empirical perspective, I will describe the results of a content-analysis experiment on Amazon’s Mechanical Turk that indicate that the presence of economic incentives based on pairwise comparisons results in reports that are more accurate than when such incentives are not present.

Biography:
Arthur Carvalho is a PhD candidate in the David R. Cheriton School of Computer Science at the University of Waterloo. His research interests are in the design of economic mechanisms to incentivize self-interested agents to behave in desirable ways, and in the design of methods to elicit and aggregate information from agents in order to build collective intelligence.

Wolfgang Ketter
Professor of Next Generation Information Systems
  • Coordinator
Ting Li
Endowed Professor of Digital Business
  • Coordinator