A Meta-Analysis of the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making

Schemmer, Max; Hemmer, Patrick; Nitsche, Maximilian; Kühl, Niklas; Vössing, Michael

doi:10.1145/3514094.3534128

Cited by 46 publications

(43 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From another perspective, the results show that descriptive studies abound (Figure 13 and Figure 14), but those that provide decision-making inputs (predictive and diagnostic) are almost not carried out. In contrast, the research in AI-assisted decision-making is experiencing tremendous growth (Schemmer et al, 2022) because it provides insights and methods that enable decision-making (Yong et al, 2022). Consider CL as a practical resource for developing competencies with AI, for example, defining the most effective way to distribute learners in groups for all participants and assessing the quality of interaction in learning environments, among others.…”

Section: Discussionmentioning

confidence: 99%

Collaborative learning and skill development for educational growth of artificial intelligence: A systematic review

et al. 2023

View full text Add to dashboard Cite

The diversity of topics in education makes it difficult for artificial intelligence (AI) to address them all in depth. Therefore, guiding to focus efforts on specific issues is essential. The analysis of competency development by fostering collaboration should be one of them because competencies are the way to validate that the educational exercise has been successful and because collaboration has proven to be one of the most effective strategies to improve performance outcomes. This systematic review analyzes the relationship between AI, competency development, and collaborative learning (CL). PRISMA methodology is used with data from the SCOPUS database. A total of 1,233 articles were found, and 30 passed the inclusion and exclusion criteria. The analysis of the selected articles identified three categories that deserve attention: the objects of study, the way of analyzing the results, and the types of AI that could be used. In this way, it has been possible to determine the relationship offered by the studies between skill development and CL and ideas about AI’s contributions to this field. Overall, however, the data from this systematic review suggest that, although AI has great potential to improve education, it should be approached with caution. More research is needed to fully understand its impact and how best to apply this technology in the classroom, minimizing its drawbacks, which may be relevant, and making truly effective and productive use of it.

show abstract

Section: Discussionmentioning

confidence: 99%

Collaborative learning and skill development for educational growth of artificial intelligence: A systematic review

et al. 2023

View full text Add to dashboard Cite

show abstract

“…One of the most common collaboration forms between humans and AI is AI-assisted decision-making-a setting in which an AI model provides recommendations to support the human. The human is in the role of making the final decision and can, therefore, either accept or override the recommendation [61,71]. Establishing an appropriate level of reliance on the AI model becomes one of the central challenges [60].…”

Section: Related Workmentioning

confidence: 99%

“…On the one hand, this may be due to limited model capacity, limited training data, or outliers unknown to the AI model. On the other hand, humans might have access to side information that is not readily available to the AI model, enabling them to make more accurate decisions for particular cases [32]. These potentially complementary capabilities motivated researchers to investigate how the abilities of humans and AI models can be combined to further improve overall decision-making performance [6,33].…”

Section: Introductionmentioning

confidence: 99%

Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction

Hemmer

Westphal

Schemmer

et al. 2023

Proceedings of the 28th International Conference on Intelligent User Interfaces

Self Cite

View full text Add to dashboard Cite

Recent work has proposed artificial intelligence (AI) models that can learn to decide whether to make a prediction for an instance of a task or to delegate it to a human by considering both parties' capabilities. In simulations with synthetically generated or contextindependent human predictions, delegation can help improve the performance of human-AI teams-compared to humans or the AI model completing the task alone. However, so far, it remains unclear how humans perform and how they perceive the task when they are aware that an AI model delegated task instances to them. In an experimental study with 196 participants, we show that task performance and task satisfaction improve through AI delegation, regardless of whether humans are aware of the delegation. Additionally, we identify humans' increased levels of self-efficacy as the underlying mechanism for these improvements in performance and satisfaction. Our findings provide initial evidence that allowing AI models to take over more management responsibilities can be an effective form of human-AI collaboration in workplaces. CCS CONCEPTS• Human-centered computing → Empirical studies in HCI; • Computing methodologies → Artificial intelligence.

show abstract

“…e open-source system SENNA [5]. According to the idea of transfer learning and the use of Transformer structure, researchers proposed a large-scale pre training model represented by Schemmer et al [6]. e pre training model generally adopts a multi-layer Transformer structure to perform unsupervised pre training on a large corpus.…”

Section: Related Workmentioning

confidence: 99%

Artificial Intelligence-Assisted Decision-Making Method for Legal Judgment Based on Deep Neural Network

2022

Mobile Information Systems

View full text Add to dashboard Cite

With the arrival of the third revolution of artificial intelligence, the applications of artificial intelligence in the fields of automatic driving, image recognition, smart home, machine translation, medical services, e-sports, and so on can be seen everywhere, and topics about artificial intelligence are constantly emerging. Since 2017, the discussion on artificial intelligence in the field of law has become more and more active. In this context, the application of artificial intelligence in the field of legal judgment and the hypothetical system based on this technology in court judgment has also become the object of discussion from time to time. In this paper, based on the artificial intelligence decision-making method of the deep neural network, aiming at the three subtasks of legal judgment prediction, namely, crime prediction, law recommendation, and sentence prediction, a multi-task judgment prediction model BERT12multi and a sentence interval prediction model BERT-Text CNN are proposed, which improve the prediction accuracy and adopt the knowledge distillation strategy to compress the model parameters and improve the reasoning speed of the judgment model. Experiments on the CAIL2018 data set show that the performance of the deep neural network model in crime prediction and law recommendation tasks can be significantly improved by adopting the pre training model adaptive training, grouping focus loss, and gradient confrontation training strategies. Using a step-by-step sentence prediction strategy can realize the weight sharing of pre training model and make use of the prediction results of charges and laws in sentence prediction. The recall training-prediction strategy can avoid error accumulation and improve the accuracy of sentence prediction. By integrating the artificial intelligence decision-making method, the case reasoning speed can be greatly improved, the highest compressible model volume can be about 11% of the original one, and the reasoning speed can be increased by about 8 times. At the same time, performance close to that of the deep neural model can be obtained, which is superior to other legal decision prediction models based on word embedding.

show abstract

A Meta-Analysis of the Utility of Explainable Artificial Intelligence in Human-AI Decision-Making

Cited by 46 publications

References 80 publications

Collaborative learning and skill development for educational growth of artificial intelligence: A systematic review

Collaborative learning and skill development for educational growth of artificial intelligence: A systematic review

Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction

Artificial Intelligence-Assisted Decision-Making Method for Legal Judgment Based on Deep Neural Network

Contact Info

Product

Resources

About