Modelling Contexts for Interactions in Dynamic Open-World Scenarios

Böck, Ronald; Wrede, Britta

doi:10.1109/smc.2019.8914656

Cited by 6 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The combination of understanding the ongoing communication and a link to contextual ("holistic") information enables not only the evaluation of the current performance but is also related to aspects like familiarisation. The authors in [9] see familiarisation as an essential part of communication (amongst humans only or in human-machine interaction), especially in "open-world" scenarios (cf. naturalistic interaction in [38]), helping to improve task solving in further interactions.…”

Section: Discussionmentioning

confidence: 99%

“…Moreover, the goal as well as the gatherings context (cf. e.g., [9]) influence the way of communicating in the group (e.g., [6,7,10]). As already stated in [11]: "An informal family gathering is mainly related to fun aspects and the (social) feeling of closeness.…”

Section: Introductionmentioning

confidence: 99%

“…Keeping this in mind, [13] further argues that such meeting are often used for familiarisation in the group (cf. also [9]), especially when the group members do not know each other, yet. Therefore, even a longer-lasting meeting can be perceived as effective and considered as a kind of "bet" in a perhaps more efficient interaction later on [13].…”

Section: Introductionmentioning

confidence: 99%

“…Regarding whose acoustic samples (leader or remaining group) might be used for prediction tasks, currently a (slight) trend towards the remaining group's samples is given. • Contextual information (as in the remaining group) is beneficial for an improvement in the performance predictions (related to (theoretical) discussions in [9,11,13]).…”

mentioning

confidence: 98%

See 3 more Smart Citations

Group Leader vs. Remaining Group—Whose Data Should Be Used for Prediction of Team Performance?

Böck

2023

MTI

View full text Add to dashboard Cite

Humans are considered to be communicative, usually interacting in dyads or groups. In this paper, we investigate group interactions regarding performance in a rather formal gathering. In particular, a collection of ten performance indicators used in social group sciences is used to assess the outcomes of the meetings in this manuscript, in an automatic, machine learning-based way. For this, the Parking Lot Corpus, comprising 70 meetings in total, is analysed. At first, we obtain baseline results for the automatic prediction of performance results on the corpus. This is the first time the Parking Lot Corpus is tapped in this sense. Additionally, we compare baseline values to those obtained, utilising bidirectional long-short term memories. For multiple performance indicators, improvements in the baseline results are able to be achieved. Furthermore, the experiments showed a trend that the acoustic material of the remaining group should use for the prediction of team performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 98%

See 2 more Smart Citations

Group Leader vs. Remaining Group—Whose Data Should Be Used for Prediction of Team Performance?

Böck

2023

MTI

View full text Add to dashboard Cite

show abstract

“…These usability heuristics serve as guidelines for development and are used in heuristic evaluation by usability experts to identify problems. While elements of these assessment approaches can be borrowed and combined, additional criteria are needed for collaborative machine partners’ more unique aspects, such as for greater use of various kinds of context ( Brézillon 1996 ; Abowd et al, 1999 ; Oh, Schmidt, and Woo 2007 ; Jain et al, 2018 ; Böck and Wrede 2019 ), multiple communication modalities ( Malchanau et al, 2019 ), and the generation of end-products that are often creative, complex, and lack a priori correct answers or even objective measures of completion ( Hashimoto et al, 2019 ).…”

Section: Introductionmentioning

confidence: 99%

Assessing Open-Ended Human-Computer Collaboration Systems: Applying a Hallmarks Approach

Kozierok

Aberdeen

Clark

et al. 2021

Front. Artif. Intell.

View full text Add to dashboard Cite

There is a growing desire to create computer systems that can collaborate with humans on complex, open-ended activities. These activities typically have no set completion criteria and frequently involve multimodal communication, extensive world knowledge, creativity, and building structures or compositions through multiple steps. Because these systems differ from question and answer (Q&A) systems, chatbots, and simple task-oriented assistants, new methods for evaluating such collaborative computer systems are needed. Here, we present a set of criteria for evaluating these systems, called Hallmarks of Human-Machine Collaboration. The Hallmarks build on the success of heuristic evaluation used by the user interface community and past evaluation techniques used in the spoken language and chatbot communities. They consist of observable characteristics indicative of successful collaborative communication, grouped into eight high-level properties: robustness; habitability; mutual contribution of meaningful content; context-awareness; consistent human engagement; provision of rationale; use of elementary concepts to teach and learn new concepts; and successful collaboration. We present examples of how we used these Hallmarks in the DARPA Communicating with Computers (CwC) program to evaluate diverse activities, including story and music generation, interactive building with blocks, and exploration of molecular mechanisms in cancer. We used the Hallmarks as guides for developers and as diagnostics, assessing systems with the Hallmarks to identify strengths and opportunities for improvement using logs from user studies, surveying the human partner, third-party review of creative products, and direct tests. Informal feedback from CwC technology developers indicates that the use of the Hallmarks for program evaluation helped guide development. The Hallmarks also made it possible to identify areas of progress and major gaps in developing systems where the machine is an equal, creative partner.

show abstract