Trust in AI-Enabled Decision Support Systems: Preliminary Validation of MAST Criteria

Chiou, Erin K.; Salehi, Pouria; Blasch, Erik; Sung, James C.; Cohen, Myke C.; Pan, Anna; Mancenido, Michelle V.; Mosallanezhad, Ahmadreza; Ba, Yang; Bhatti, Shawaiz

doi:10.1109/ichms56717.2022.9980623

Cited by 9 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This same additional context could help improve performance in security operations. DHS is currently funding research through its Center for Accelerating Operational Efficiency (CAOE) that is focused on evaluating the Multisource AI Scorecard Table for System Evaluation (MAST) (Blasch et al, 2021) criteria as a checklist for assessing trust in AI-enabled decision support systems across users with different skills and various types of work environments (Chiou et al, n.d.). It would be interesting to know if automated face matching systems that provide this type of feedback to professional screeners result in higher on MAST criteria (i.e., are rated more understandable and trustworthy) and are associated with increased performance (shorter reaction time, greater sensitivity and accuracy).…”

Section: Discussionmentioning

confidence: 99%

Face Matching as a Function of Prior Identity Information in Professional Screeners

Korbelak,

Zish,

Endres

2023

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

Using a computer-based face matching task we objectively measured face matching performance (reaction time, sensitivity, accuracy) as a function of prior identity source type (Artificial Intelligence (AI), human, none), prior information accuracy (accurate, inaccurate) and task difficulty (high, low) in professional screeners. Participants were required to judge how similar they thought a pair of faces were, to decide whether the faces in each pair were the same person, and then to judge the difficulty of that decision. Professional screeners were more accurate, faster, and, more sensitive when normative task difficulty was low. Professional screeners were also more accurate, faster, and more sensitive when prior identity source information was accurate. There was no main effect of prior identity source type on performance (there was a trend-level effect). Face matching accuracy positively correlated with normative data from non-professional screeners. Professional screeners were more accurate 80.6% of the time, compared to non-professional screeners.

show abstract

Section: Discussionmentioning

confidence: 99%

Face Matching as a Function of Prior Identity Information in Professional Screeners

Korbelak,

Zish,

Endres

2023

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

show abstract

“…Information relevance is critical for user decision making. The evaluation of relevance can be viewed from different evaluation and reporting requirements such as with the Multi-source AI Scorecard Table (MAST) [33,34]. Among the nine MAST criteria used for determining the value of an information fusion system, "relevance" elements include:…”

Section: Information Relevancementioning

confidence: 99%

Value-based sensor and information fusion

Blasch

2024

Signal Processing, Sensor/Information Fusion, and Target Recognition XXXIII

View full text Add to dashboard Cite

There are many examples, methods, and processes showing the importance of sensor, data, and information fusion. However, there is a need to determine the value added of information fusion in the context of data (e.g., multimodal, amount, and resolution), software (e.g., artificial intelligence/machine learning), hardware (e.g., size, weight, and performance), as well as architectures (e.g., cloud, fog, and edge computing). This paper utilizes the analytical hierarchy process (AHP) to determine the pairwise performance needs among different human-machine information fusion system tradeoffs to show the value added from sensor fusion. The paper examines the potential value added by coordinating deep, active, and reinforcement learning for information fusion systems. Among the information metrics, the combined methods of artificial intelligence learning methods highlight user requirements for accuracy, confidence, and timeliness.

show abstract

“…In the case of generative approaches, data augmentation through a digital twin can be viewed as a "what-if" scenario to check on the integrity of the processing and reduce uncertainty. The DT can provide Analysis of Alternatives to support user decision making [76,77]…”

Section: Aerospace Systemsmentioning

confidence: 99%

Digital twin meets information fusion: panel summary

Blasch,

Chen,

Chen

et al. 2024

Signal Processing, Sensor/Information Fusion, and Target Recognition XXXIII

View full text Add to dashboard Cite

Within the recent years, the concept of Digital Twins (DT) emerged to support digital engineering physical-systems design in the processing and analysis of device components, data flows, and networked systems. As an element of systems engineering, the DT serves as a method for life cycle management for operations and maintenance through monitoring, diagnostics, and prognostics. Key to DT methods is the use of distributed sensors to monitor the system and determine the functioning with the use of physical design information, such as a series of distributed edge sensors and the layout of a physical electrical grid. While industries like industrial manufacturing, electrical power, space systems, and healthcare maintenance have embraced DT; other groups are utilizing the concept for analysis. Given that a large number of sensors are used to gather data on the health of system, it is natural that data fusion, estimation theory, and signals processing support digital twin fusion (DTwF); but there are a variety of challenges such as big data, distribution fusion, and edge analytics. The panel will discuss areas in which data and information fusion techniques enhance DT applications.

show abstract

Trust in AI-Enabled Decision Support Systems: Preliminary Validation of MAST Criteria

Cited by 9 publications

References 0 publications

Face Matching as a Function of Prior Identity Information in Professional Screeners

Face Matching as a Function of Prior Identity Information in Professional Screeners

Value-based sensor and information fusion

Digital twin meets information fusion: panel summary

Contact Info

Product

Resources

About