Architecting Real-Time Crowd-Powered Systems

Lasecki, Walter S.; Homan, Christopher M.; Bigham, Jeffrey P.

doi:10.15346/hc.v1i1.5

Cited by 17 publications

(9 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, it shows that tools with very low latency constraints that need to be operated continuously can be powered by the crowd. A number of tools have been developed with this approach, both to support accessibility and for general use [Lasecki et al, 2014b[Lasecki et al, , 2011[Lasecki et al, , 2014a[Lasecki et al, , 2015. Second, it demonstrates the ability of multiple nonexperts working in conjunction to perform the task of an expert.…”

Section: Discussionmentioning

confidence: 99%

“…These crowd-powered systems, as they have come to be called [Bernstein, 2012, Lasecki et al, 2014b, address difficult problems, many of which are of particular and practical interest to people with disabilities, by combining human intelligence and computation in new ways. People Figure 1.1: A diagram of the Faxcess system from the early 1990s for helping blind people interpret visual information.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Crowdsourcing Accessibility: Human-Powered Access Technologies

Brady

Bigham

2015

Self Cite

View full text Add to dashboard Cite

People with disabilities have always engaged the people around them in order to circumvent inaccessible situations, allowing them to live more independently and get things done in their everyday lives. Increasing connectivity is allowing this approach to be extended to wherever and whenever it is needed. Technology can leverage this human work force to accomplish tasks beyond the capabilities of computers, increasing how accessible the world is for people with disabilities. This article outlines the growth of online human support, outlines a number of projects in this space, and presents a set of challenges and opportunities for this work going forward.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Crowdsourcing Accessibility: Human-Powered Access Technologies

Brady

Bigham

2015

Self Cite

View full text Add to dashboard Cite

show abstract

“…In recent years, researchers have developed methods to recruit workers on demand from crowdsourcing platforms and created techniques to reduce latency. Such methods enable real-time crowdsourcing (RTC) [39] and the resulting systems that build on top of the RTC concept are known as real-time crowd-powered systems. VizWiz [40] is one of the earliest real-time crowd-powered system which helps blind users in answering visual questions about their surroundings by sending object's photos and audio questions from their phones to crowd workers.…”

Section: Real-time Crowd-powered Systemsmentioning

confidence: 99%

Crowd of Oz: A Crowd-Powered Social Robotics System for Stress Management

Abbas

Khan

Gadiraju

et al. 2020

Sensors

View full text Add to dashboard Cite

Coping with stress is crucial for a healthy lifestyle. In the past, a great deal of research has been conducted to use socially assistive robots as a therapy to alleviate stress and anxiety related problems. However, building a fully autonomous social robot which can deliver psycho-therapeutic solutions is a very challenging endeavor due to limitations in artificial intelligence (AI). To overcome AI’s limitations, researchers have previously introduced crowdsourcing-based teleoperation methods, which summon the crowd’s input to control a robot’s functions. However, in the context of robotics, such methods have only been used to support the object manipulation, navigational, and training tasks. It is not yet known how to leverage real-time crowdsourcing (RTC) to process complex therapeutic conversational tasks for social robotics. To fill this gap, we developed Crowd of Oz (CoZ), an open-source system that allows Softbank’s Pepper robot to support such conversational tasks. To demonstrate the potential implications of this crowd-powered approach, we investigated how effectively, crowd workers recruited in real-time can teleoperate the robot’s speech, in situations when the robot needs to act as a life coach. We systematically varied the number of workers who simultaneously handle the speech of the robot (N = 1, 2, 4, 8) and investigated the concomitant effects for enabling RTC for social robotics. Additionally, we present Pavilion, a novel and open-source algorithm for managing the workers’ queue so that a required number of workers are engaged or waiting. Based on our findings, we discuss salient parameters that such crowd-powered systems must adhere to, so as to enhance their performance in response latency and dialogue quality.

show abstract

“…For example, Adrenaline [4] employed a retainer model, while in Viz-Wiz [7] workers signed up in advance and were kept engaged, at a cost, until the work arrived. Other approaches relied on very large crowds [42], using queuing theory or predictive recruitment to model workers' arrival and task assignment [5], or repeatedly posting the tasks to make them more visible to workers [7]. In our experiments, we incorporated a combination of these ideas to generate our on-demand crowd: We repeatedly posted tasks, recruited in advance, requested for a larger crowd than required, and used an audio alert to prompt workers when the contest started.…”

Section: Timely Workermentioning

confidence: 99%

Beyond Monetary Incentives

Feyisetan

Simperl

2019

Trans. Soc. Comput.

View full text Add to dashboard Cite

More than a decade after the term was introduced by Jeff Howe in 2006 [33], crowdsourcing seems to be considered by many as a, if not "the," universal means to solve virtually any kind of problem, online and offline, that requires sustained human involvement. We see it used to motivate employees to engage with less rewarding work routines, attract the best possible ideas to boost innovation, enhance artificial intelligence algorithms, and support ambitious social and entrepreneurial initiatives. Achieving a goal by collecting contributions from many individuals has a long tradition, far beyond the most recent developments in the digital world. In fact, some of the most successful exemplars of the "wisdom of the crowds" in modern times, for instance Wikipedia, predate Howe's article. And yet, with the rise of social networks, smart mobile devices and online platforms, the phenomenon has found a new dimension in terms of scale and achievements-it routinely manages to mobilise very large groups of people in a relatively short period of time, helping organisations from tech and government to the military and marketing improve the ways they operate, decide, and engage with the world. The crowdsourcing landscape is as diverse as its applications. It includes paid microtask platforms such as Amazon's Mechanical Turk 1 and CrowdFlower (now Figure Eight), 2 alongside online labour marketplaces such as TaskRabbit 3 and UpWork, 4 and open innovation contests in the style of Kaggle 5 and Innocentive. 6 It features volunteer citizen science systems such as Galaxy Zoo 7 and SciStarter, 8 and games with a purpose (GWAPs) such as PhraseDetective 9 and EyeWire. 10 Putting aside the principled differences between these forms of crowdsourcing, which make them from the offset amenable only to specific types of problems, the success of any crowdsourcing endeavour will depend on the ability of the "requester"-the person or institution reaching out to the crowdto benefit from crowd outputs and attract a critical mass of contributors. The frame conditions can be challenging as well: To be effective, crowdsourcing needs to provide a real alternative to existing solutions of the problem the requester is trying to solve, in terms of quality, timeliness, and costs. Getting this mixture right is not trivial, as it requires insight into a wide array of subjects from artificial intelligence and data analysis to user experience design and behavioural sciences [21, 22]. Aligning the motivations of the crowd with the goals of the requester and finding the right mix of rewards to drive purposeful crowd engagement are particularly critical. Existing crowdsourcing platforms already provide some level of support-whether paid microtasks, challenges, or citizen science projects, each of these forms of crowdsourcing makes some basic assumptions about what drives people to contribute to their "open call" [33], and offers sometimes bespoke incentives to encourage a certain style of behaviour. For example, people register on Mechanical Turk primarily for f...

show abstract

Architecting Real-Time Crowd-Powered Systems

Cited by 17 publications

References 69 publications

Crowdsourcing Accessibility: Human-Powered Access Technologies

Crowdsourcing Accessibility: Human-Powered Access Technologies

Crowd of Oz: A Crowd-Powered Social Robotics System for Stress Management

Beyond Monetary Incentives

Contact Info

Product

Resources

About