TurKit

Little, Greg; Chilton, Lydia B.; Goldman, Max; Miller, Robert C.

doi:10.1145/1866029.1866040

Cited by 232 publications

(43 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Microsoft Word plug-in is written using Microsoft Visual Studio Tools for Office (VSTO) and the Windows Presentation Foundation (WPF). Backend scripts use the TurKit Mechanical Turk toolkit [16].…”

Section: Methodsmentioning

confidence: 99%

“…Heer and Bostock explored Mechanical Turk as a testbed for graphical perception experiments, finding reliable results when they implemented basic measures like qualification tests [10]. Little et al advocate the use of human computation algorithms on Mechanical Turk [16]. Find-Fix-Verify may be viewed as a new design pattern for human computation algorithms.…”

Section: Crowdsourcingmentioning

confidence: 99%

See 1 more Smart Citation

Soylent

Bernstein

Little

Miller

et al. 2010

Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology

Self Cite

509

View full text Add to dashboard Cite

This paper introduces architectural and interaction patterns for integrating crowdsourced human contributions directly into user interfaces. We focus on writing and editing, complex endeavors that span many levels of conceptual and pragmatic activity. Authoring tools offer help with pragmatics, but for higher-level help, writers commonly turn to other people. We thus present Soylent, a word processing interface that enables writers to call on Mechanical Turk workers to shorten, proofread, and otherwise edit parts of their documents on demand. To improve worker quality, we introduce the Find-Fix-Verify crowd programming pattern, which splits tasks into a series of generation and review stages. Evaluation studies demonstrate the feasibility of crowdsourced editing and investigate questions of reliability, cost, wait time, and work time for edits.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Crowdsourcingmentioning

confidence: 99%

Soylent

Bernstein

Little

Miller

et al. 2010

Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology

Self Cite

509

View full text Add to dashboard Cite

show abstract

“…It is widely suspected that Amazon outsources some questions to Mechanical Turk. The TurKit library on which our quikTurkit is built lets programmers easily employ multiple turkers using common programming paradigms [19]. To our knowledge, quikTurkit is the first attempt to get work done by web-based workers in nearly real-time.…”

Section: Human-powered Servicesmentioning

confidence: 99%

“…quikTurkit is an abstraction layer on top of the TurKit Mechanical Turk API [19]. quikTurkit primarily achieves low latency by queuing workers before they are needed, but also encapsulates heuristics to help its Human Intelligance Tasks (HITs) be quickly picked up by workers.…”

Section: Vizwizmentioning

confidence: 99%

VizWiz

Bigham

Jayant

et al. 2010

Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)

Self Cite

112

View full text Add to dashboard Cite

The lack of access to visual information like text labels, icons, and colors can cause frustration and decrease independence for blind people. Current access technology uses automatic approaches to address some problems in this space, but the technology is error-prone, limited in scope, and quite expensive. In this paper, we introduce VizWiz, a talking application for mobile phones that offers a new alternative to answering visual questions in nearly real-time-asking multiple people on the web. To support answering questions quickly, we introduce a general approach for intelligently recruiting human workers in advance called quikTurkit so that workers are available when new questions arrive. A field deployment with 11 blind participants illustrates that blind people can effectively use VizWiz to cheaply answer questions in their everyday lives, highlighting issues that automatic approaches will need to address to be useful. Finally, we illustrate the potential of using VizWiz as part of the participatory design of advanced tools by using it to build and evaluate VizWiz::LocateIt, an interactive mobile tool that helps blind people solve general visual search problems.

show abstract

“…collective intelligence [10] or human computation [8]), are currently implemented using either ad hoc methods or programming frameworks [1,9] that make no use of agent technology. Within the agents community, on the other hand, agent communication languages and interaction protocols [3] have been widely used to design and deploy a wide range of agent coordination mechanisms, many of which bear close similarity to those needed in social computation systems.…”

Section: Introductionmentioning

confidence: 99%

Agent Protocols for Social Computation

Rovatsos

Diochnos

Craciun

2015

Communications in Computer and Information Science

View full text Add to dashboard Cite

Abstract. Despite the fact that social computation systems involve interaction mechanisms that closely resemble well-known models of agent coordination, current applications in this area make little or no use of the techniques the agent-based systems literature has to offer. In order to bridge this gap, this paper proposes a data-driven method for defining and deploying agent interaction protocols that is entirely based on using the standard architecture of the World Wide Web. This obviates the need of bespoke message passing mechanisms and agent platforms, thereby facilitating the use of agent coordination principles in standard Web-based applications. We describe a prototypical implementation of the architecture and experimental results that prove it can deliver the scalability and robustness required of modern social computation applications while maintaining the expressiveness and versatility of agent interaction protocols.

show abstract

TurKit

Cited by 232 publications

References 16 publications

Soylent

Soylent

VizWiz

Agent Protocols for Social Computation

Contact Info

Product

Resources

About