HyEnA: A Hybrid Method for Extracting Arguments from Opinions

Abstract

The key arguments underlying a large and noisy set of opinions help understand the opinions quickly and accurately. Fully automated methods can extract arguments but (1) require large labeled datasets and (2) work well for known viewpoints, but not for novel points of view. We propose HyEnA, a hybrid (human + AI) method for extracting arguments from opinionated texts, combining the speed of automated processing with the understanding and reasoning capabilities of humans. We evaluate HyEnA on three feedback corpora. We find that, on the one hand, HyEnA achieves higher coverage and precision than a state-of-the-art automated method, when compared on a common set of diverse opinions, justifying the need for human insight. On the other hand, HyEnA requires less human effort and does not compromise quality compared to (fully manual) expert analysis, demonstrating the benefit of combining human and machine intelligence.

Use of Public Opinion Data to Inform COVID-19 Policymaking

“…This review identified a range of methods and approaches to semantic analysis of large datasets and demonstrated their ability to analyse millions of citizen-generated comments. This area is continuing to evolve and more sophisticated techniques are therefore emerging (Belkahla Driss et al 2019, van der Meer 2022. In determining approaches to POD data collection and analysis, practitioners should determine if and how AI could be deployed to optimise POD initiatives.…”

Inferring Values via Hybrid Intelligence

“…Then, value identification is the process of identifying the set of values relevant to a decision context. Inspired by Value Sensitive Design [11], we advocate for methods that take into account the decision context [22,24] and involve stakeholders, e.g., via data-driven methods [8,40,41]. Subsequently, value estimation is the process of determining an individual's value preferences over the identified values.…”

Do Differences in Values Influence Disagreements in Online Discussions?

“…Further, questions arise on how to control for disagreement to enhance reciprocity (Esau and Friess, 2022), and how too much exposure to opposing views drives polarization (Bail et al, 2018). Analysis methods for online discussions currently cannot accurately represent such diverse perspectives (Cabitza et al, 2023;van der Meer et al, 2022a), and measuring deliberative quality is an open challenge (Vecchi et al, 2021;Shortall et al, 2022).…”