The Next Frontier: AI We Can Really Trust

Holzinger, Andreas

doi:10.1007/978-3-030-93736-2_33

Cited by 74 publications

(25 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bias is harmful when (a) it contributes to unfair discrimination, (b) which is systemic and (c) leads to unfair outcomes. Bias not only results in discrimination, but also impacts the trust of the users of such system Fairness is one of the components needed for trustworthy AI [6].…”

Section: Bias As the Potential For Unfairnessmentioning

confidence: 99%

Fairness–accuracy tradeoff: activation function choice in a neural network

McCarthy

Narayanan

2023

AI Ethics

View full text Add to dashboard Cite

Models can have different outcomes based on the different types of inputs; the training data used to build the model will change its output (a data-centric outcome), while the hyperparameter selection can also affect the model's output and performance (a model-centric outcome). When building classification models, data scientists generally focus on model performance as the key metric, often focusing on accuracy. However, other metrics should be considered during model development that assesses performance in other ways, like fairness. Assessing the fairness during the model development process is often overlooked but should be considered as part of a model's full assessment before deployment. This research investigates the fairness-accuracy tradeoff that occurred by changing only one hyperparameter in a neural network for binary classification to assess how this single change in the hidden layer can alter the algorithm's accuracy and fairness. Neural networks were used for this assessment because of their wide usage across domains, limitations to current bias measurement methods, and the underlying challenges in their interpretability. Findings suggest that assessing accuracy and fairness during model development provides value while mitigating potential negative effects for users while reducing organizational risk. No particular activation function was found to be fairer than another. Notable differences in the fairness and accuracy measures could help developers deploy a model with high accuracy and robust fairness. Algorithm development should include a grid search for hyperparameter optimization that includes fairness along with performance measures, like accuracy. While the actual choices for hyperparameters may depend on the business context and dataset considered, an optimal development process should use both fairness and model performance metrics.

show abstract

Section: Bias As the Potential For Unfairnessmentioning

confidence: 99%

Fairness–accuracy tradeoff: activation function choice in a neural network

McCarthy

Narayanan

2023

AI Ethics

View full text Add to dashboard Cite

show abstract

“…In Ref. [16], the author describes human-in-the-loop techniques for building trustworthy artificial intelligence. These methods are potentially capable to describe causal relationships that cannot be achieved with just supervised learning.…”

Section: Related Workmentioning

confidence: 99%

Evaluating deep learning predictions for COVID-19 from X-ray images using leave-one-out predictive densities

Hernández

López-Cortés

2023

Neural Comput & Applic

View full text Add to dashboard Cite

Early detection of the COVID-19 virus is an important task for controlling the spread of the pandemic. Imaging techniques such as chest X-ray are relatively inexpensive and accessible, but its interpretation requires expert knowledge to evaluate the disease severity. Several approaches for automatic COVID-19 detection using deep learning techniques have been proposed. While most approaches show high accuracy on the COVID-19 detection task, there is not enough evidence on external evaluation for this technique. Furthermore, data scarcity and sampling biases make difficult to properly evaluate model predictions. In this paper, we propose stochastic gradient Langevin dynamics (SGLD) to take into account the model uncertainty. Four different deep learning architectures are trained using SGLD and compared to their baselines using stochastic gradient descent. The model uncertainties are also evaluated according to their convergence properties and the leave-one-out predictive densities. The proposed approach is able to reduce overconfidence of the baseline estimators while also retaining predictive accuracy for the best-performing cases.

show abstract

“…For trustworthy AI, it is imperative to include ethical and legal aspects, which is a crossdisciplinary goal, because all trusted AI solutions must be not only ethically responsible but also legally compliant [12]. Dimensions of trustworthiness for AI include: security, safety, fairness, accountability (traceability, replicability), auditability (verifiability, checkability), and most importantly, robustness and explainability; see [13].…”

Section: Artificial Intelligencementioning

confidence: 99%

Digital Transformation in Smart Farm and Forest Operations Needs Human-Centered AI: Challenges and Future Directions

Holzinger

Saranti

Angerschmid

et al. 2022

Sensors

Self Cite

View full text Add to dashboard Cite

The main impetus for the global efforts toward the current digital transformation in almost all areas of our daily lives is due to the great successes of artificial intelligence (AI), and in particular, the workhorse of AI, statistical machine learning (ML). The intelligent analysis, modeling, and management of agricultural and forest ecosystems, and of the use and protection of soils, already play important roles in securing our planet for future generations and will become irreplaceable in the future. Technical solutions must encompass the entire agricultural and forestry value chain. The process of digital transformation is supported by cyber-physical systems enabled by advances in ML, the availability of big data and increasing computing power. For certain tasks, algorithms today achieve performances that exceed human levels. The challenge is to use multimodal information fusion, i.e., to integrate data from different sources (sensor data, images, *omics), and explain to an expert why a certain result was achieved. However, ML models often react to even small changes, and disturbances can have dramatic effects on their results. Therefore, the use of AI in areas that matter to human life (agriculture, forestry, climate, health, etc.) has led to an increased need for trustworthy AI with two main components: explainability and robustness. One step toward making AI more robust is to leverage expert knowledge. For example, a farmer/forester in the loop can often bring in experience and conceptual understanding to the AI pipeline—no AI can do this. Consequently, human-centered AI (HCAI) is a combination of “artificial intelligence” and “natural intelligence” to empower, amplify, and augment human performance, rather than replace people. To achieve practical success of HCAI in agriculture and forestry, this article identifies three important frontier research areas: (1) intelligent information fusion; (2) robotics and embodied intelligence; and (3) augmentation, explanation, and verification for trusted decision support. This goal will also require an agile, human-centered design approach for three generations (G). G1: Enabling easily realizable applications through immediate deployment of existing technology. G2: Medium-term modification of existing technology. G3: Advanced adaptation and evolution beyond state-of-the-art.

show abstract

The Next Frontier: AI We Can Really Trust

Cited by 74 publications

References 39 publications

Fairness–accuracy tradeoff: activation function choice in a neural network

Fairness–accuracy tradeoff: activation function choice in a neural network

Evaluating deep learning predictions for COVID-19 from X-ray images using leave-one-out predictive densities

Digital Transformation in Smart Farm and Forest Operations Needs Human-Centered AI: Challenges and Future Directions

Contact Info

Product

Resources

About