Toward Complexity Measures for Systems Involving Human Computation

Crouser, R. Jordan; Hescott, Benjamin; Chang, Remco

doi:10.15346/hc.v1i1.4

Cited by 19 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AI has made constant efforts to model human intelligence; on the other hand, HC pursue solving tasks that are computationally challenging. These two research areas "hinge on the long-held belief that there exist problems that require human-level intelligence and reasoning to solve" [27]. When machine interpretations are not feasible or dependent on human interpretations then KM becomes the only source of knowledge creation.…”

Section: Human Computations In Citizen Crowds: a Knowledge Management...mentioning

confidence: 99%

Human Computations in Citizen Crowds: A Knowledge Management Solution Framework

Khan

Shaikh²,

Shaikh³

2018

Mehran Univ. res. j. eng. technol.

View full text Add to dashboard Cite

In this paper, we present an attempt to create a framework by implementing a simple solution, called ExamCheck, to focus on the generation of knowledge, feedback on that knowledge and recording the results of that knowledge in academic settings. Our solution, based on HC, shows that a structured KM framework can address a complex problem in a context that is important for participants themselves. KM in citizen crowds makes use of inbuilt human habits of participatory problem-solving to externalize human cognition and memory, including machine-centric argumentation through information management and strategy encapsulated in process workflows [7]. In general, communities seek innovative outcomes when a large number of their members face similar problems, which we call PCI (Problems of Common Interest). These problems mainly relate to people such that they face and solve them using collaboration and collective intelligence. In terms of crowdsourced HC, both requestors and workers share problem and solution domains. We believe that the core underlying problem is in selecting one out of many solutions, thus the need to expressed the problem as a KM process. This paper is organized in seven sections: this introduction, background and related work, the scanrio-

show abstract

Section: Human Computations In Citizen Crowds: a Knowledge Management...mentioning

confidence: 99%

Human Computations in Citizen Crowds: A Knowledge Management Solution Framework

Khan

Shaikh²,

Shaikh³

2018

Mehran Univ. res. j. eng. technol.

View full text Add to dashboard Cite

show abstract

“…Recently Sacha et al reviewed the role of visual analytics in machine learning, and outlined several open challenges [44]. Crouser et al suggested that the complexity of human computation can be measured [15]. Visual analytics can provide model developers with an effective means to explore data points in a high-dimensional space (e.g., [13,30]).…”

Section: Human-centric Classificationmentioning

confidence: 99%

An Analysis of Machine- and Human-Analytics in Classification

Tam

Kothari

Chen

2017

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

In this work, we present a study that traces the technical and cognitive processes in two visual analytics applications to a common theoretic model of soft knowledge that may be added into a visual analytics process for constructing a decision-tree model. Both case studies involved the development of classification models based on the "bag of features" approach. Both compared a visual analytics approach using parallel coordinates with a machine-learning approach using information theory. Both found that the visual analytics approach had some advantages over the machine learning approach, especially when sparse datasets were used as the ground truth. We examine various possible factors that may have contributed to such advantages, and collect empirical evidence for supporting the observation and reasoning of these factors. We propose an information-theoretic model as a common theoretic basis to explain the phenomena exhibited in these two case studies. Together we provide interconnected empirical and theoretical evidence to support the usefulness of visual analytics.

show abstract

Toward Theoretical Techniques for Measuring the Use of Human Effort in Visual Analytic Systems

Crouser

Franklin

Endert

et al. 2017

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Visual analytic systems have long relied on user studies and standard datasets to demonstrate advances to the state of the art, as well as to illustrate the efficiency of solutions to domain-specific challenges. This approach has enabled some important comparisons between systems, but unfortunately the narrow scope required to facilitate these comparisons has prevented many of these lessons from being generalized to new areas. At the same time, advanced visual analytic systems have made increasing use of human-machine collaboration to solve problems not tractable by machine computation alone. To continue to make progress in modeling user tasks in these hybrid visual analytic systems, we must strive to gain insight into what makes certain tasks more complex than others. This will require the development of mechanisms for describing the balance to be struck between machine and human strengths with respect to analytical tasks and workload. In this paper, we argue for the necessity of theoretical tools for reasoning about such balance in visual analytic systems and demonstrate the utility of the Human Oracle Model for this purpose in the context of sensemaking in visual analytics. Additionally, we make use of the Human Oracle Model to guide the development of a new system through a case study in the domain of cybersecurity.

show abstract

Toward Complexity Measures for Systems Involving Human Computation

Cited by 19 publications

References 11 publications

Human Computations in Citizen Crowds: A Knowledge Management Solution Framework

Human Computations in Citizen Crowds: A Knowledge Management Solution Framework

An Analysis of Machine- and Human-Analytics in Classification

Toward Theoretical Techniques for Measuring the Use of Human Effort in Visual Analytic Systems

Contact Info

Product

Resources

About