Competent but Rigid: Identifying the Gap in Empowering AI to Participate Equally in Group Decision-Making

Zheng, Chengbo; Wu, Yirong; Shi, Chuhan; Ma, Shuai; Luo, Jiehui; Ma, Xiaojuan

doi:10.1145/3544548.3581131

Cited by 11 publications

(2 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this approach, LLMs are used to streamline and facilitate the team's interaction process, particularly during the facilitating phase. This is typically achieved by integrating an agent within the team that aids in communication [16], decision-making [83], information sharing, and coordination. By smoothing out the interaction process, the LLM ensures that the team's workflow continues seamlessly and effectively, enhancing collaboration and productivity.…”

Section: Mode 3: Context-basedmentioning

confidence: 99%

A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration

Gao,

Gebreegziabher,

Choo

et al. 2024

Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

With ChatGPT's release, conversational prompting has become the most popular form of human-LLM interaction. However, its effectiveness is limited for more complex tasks involving reasoning, creativity, and iteration. Through a systematic analysis of HCI papers published since 2021, we identified four key phases in the human-LLM interaction flow-planning, facilitating, iterating, and testing-to precisely understand the dynamics of this process. Additionally, we have developed a taxonomy of four primary interaction modes: Mode 1: Standard Prompting, Mode 2: User Interface, Mode 3: Context-based, and Mode 4: Agent Facilitator. This taxonomy was further enriched using the "5W1H" guideline method, which involved a detailed examination of definitions, participant roles (Who), the phases that happened (When), human objectives and LLM abilities (What), and the mechanics of each interaction mode (How). We anticipate this taxonomy will contribute to the future design and evaluation of human-LLM interaction. CCS CONCEPTS• Human-centered computing → Natural language interfaces.

show abstract

Section: Mode 3: Context-basedmentioning

confidence: 99%

A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration

Gao,

Gebreegziabher,

Choo

et al. 2024

Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…AI technology is increasingly crucial in supporting human decisionmaking across various domains [2,7,47,84,90,92,93]. In AIassisted decision-making, AI provides recommendations while leaving the final decision to humans [90].…”

Section: Introductionmentioning

confidence: 99%

Will You Accept the AI Recommendation? Predicting Human Behavior in AI-Assisted Decision Making

Wang

Yin

2022

Proceedings of the ACM Web Conference 2022

View full text Add to dashboard Cite

In AI-assisted decision-making, it is crucial but challenging for humans to achieve appropriate reliance on AI. This paper approaches this problem from a human-centered perspective, "human selfconfidence calibration". We begin by proposing an analytical framework to highlight the importance of calibrated human self-confidence. In our first study, we explore the relationship between human selfconfidence appropriateness and reliance appropriateness. Then in our second study, We propose three calibration mechanisms and compare their effects on humans' self-confidence and user experience. Subsequently, our third study investigates the effects of self-confidence calibration on AI-assisted decision-making. Results show that calibrating human self-confidence enhances human-AI team performance and encourages more rational reliance on AI (in some aspects) compared to uncalibrated baselines. Finally, we discuss our main findings and provide implications for designing future AI-assisted decision-making interfaces. CCS CONCEPTS• Human-centered computing → Empirical studies in HCI.

show abstract