Expert crowdsourcing with flash teams

Abstract

We introduce flash teams, a framework for dynamically assembling and managing paid experts from the crowd. Flash teams advance a vision of expert crowd work that accomplishes complex, interdependent goals such as engineering and design. These teams consist of sequences of linked modular tasks and handoffs that can be computationally managed. Interactive systems reason about and manipulate these teams' structures: for example, flash teams can be recombined to form larger organizations and authored automatically in response to a user's request. Flash teams can also hire more people elastically in reaction to task needs, and pipeline intermediate output to accelerate completion times. To enable flash teams, we present Foundry, an end-user authoring platform and runtime manager. Foundry allows users to author modular tasks, then manages teams through handoffs of intermediate work. We demonstrate that Foundry and flash teams enable crowdsourcing of a broad class of goals including design prototyping, course development, and film animation, in half the work time of traditional self-managed teams.

Coordinating Advanced Crowd Work: Extending Citizen Science

“…For the later, researchers have proposed ways to automatically generate workflows [e.g., 2] or to use the crowd to create them [e.g., 17]. Others have examined ways to chain modular tasks [e.g., 29].…”

The probabilistic innovation theoretical framework

“…The ultimate goal of this process is to have the crowd also provide direction and interactive systems to manage R&D work as it unfolds in real time. The work of Retelny et al (2014) is now given special attention in order to highlight these concepts, which are considered especially important to the emerging PI literature.…”

“…However, this becomes more difficult when it comes to more complex real-world tasks requiring deep domain knowledge that is less easy to decompose into independent microtasks which any member of the crowd can complete; with regard to their suggested system of structured collaborations between experts from the crowd as a solution to this problem, Retelny et al (2014) offer the following vision to: enable anybody with a napkin sketch of a design idea to ask the crowd to follow the user-centred design process and create a user-tested, high-fidelity prototype of that idea within twentyfour hours? (p. 75) The importance of this body of work is in its focus on design; to have the crowd design its own solutions to design, system and quality assurance problems would be an important dimension of the PI literature.…”

“…(p. 75) The importance of this body of work is in its focus on design; to have the crowd design its own solutions to design, system and quality assurance problems would be an important dimension of the PI literature. Retelny et al (2014) suggest that the use of expert crowd work can be designed around the concept of flash teams, organised around sequences of linked tasks, which have the same coordinating strength as more lightweight team structures while at the same time being used to leverage and support collaboration, automatically create teams, manage the size of these teams and instantaneously combine teams into larger organisations.…”

Crowd-based ontology engineering with the uComp Protégé plugin

“…Increasing task complexity. Recent advances in crowdsourcing include moving away from solving very simple tasks towards enabling experts drawn from the crowd to solve more complex tasks by creating micro-organizations and managing these organizations themselves [21]. While encouraging results have been reported on solving creative tasks such as animation creation or course curricula design, the applicability of this new approach for knowledge creation tasks is still unclear.…”