Benchmarking and survey of explanation methods for black box models

Bodria, Francesco; Giannotti, Fosca; Guidotti, Riccardo; Naretto, Francesca; Pedreschi, Dino; Rinzivillo, Salvatore

doi:10.1007/s10618-023-00933-9

Cited by 56 publications

(10 citation statements)

References 140 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Computational advances facilitate more powerful but also more complex models such as black box models [20][21][22][23]. Feed forward artificial neural networks (ANNs) [24], e.g., for image classification, possess about 5 to 155 × 10 6 trainable parameters while performing up to 8 × 10 10 computational operations for a single prediction [25].…”

Section: Black Box Modelsmentioning

confidence: 99%

A Meta Algorithm for Interpretable Ensemble Learning: The League of Experts

Vogel,

Schlosser,

Manthey

et al. 2024

MAKE

View full text Add to dashboard Cite

Background. The importance of explainable artificial intelligence and machine learning (XAI/XML) is increasingly being recognized, aiming to understand how information contributes to decisions, the method’s bias, or sensitivity to data pathologies. Efforts are often directed to post hoc explanations of black box models. These approaches add additional sources for errors without resolving their shortcomings. Less effort is directed into the design of intrinsically interpretable approaches. Methods. We introduce an intrinsically interpretable methodology motivated by ensemble learning: the League of Experts (LoE) model. We establish the theoretical framework first and then deduce a modular meta algorithm. In our description, we focus primarily on classification problems. However, LoE applies equally to regression problems. Specific to classification problems, we employ classical decision trees as classifier ensembles as a particular instance. This choice facilitates the derivation of human-understandable decision rules for the underlying classification problem, which results in a derived rule learning system denoted as RuleLoE. Results. In addition to 12 KEEL classification datasets, we employ two standard datasets from particularly relevant domains—medicine and finance—to illustrate the LoE algorithm. The performance of LoE with respect to its accuracy and rule coverage is comparable to common state-of-the-art classification methods. Moreover, LoE delivers a clearly understandable set of decision rules with adjustable complexity, describing the classification problem. Conclusions. LoE is a reliable method for classification and regression problems with an accuracy that seems to be appropriate for situations in which underlying causalities are in the center of interest rather than just accurate predictions or classifications.

show abstract

Section: Black Box Modelsmentioning

confidence: 99%

A Meta Algorithm for Interpretable Ensemble Learning: The League of Experts

Vogel,

Schlosser,

Manthey

et al. 2024

MAKE

View full text Add to dashboard Cite

show abstract

“…However, the mechanisms of most current emerging ML models are unclear, especially deep learning models, which exhibit "black box" properties for the predictions of target materials. [68] Such phenomena originated from the data-driven nature of the ML algorithms, which makes it more difficult for researchers to understand and explain the in-depth mechanisms or correlations behind the outputs proposed by the ML models. In this scenario, the efforts for investigating explainable ML models are valuable since it allows the researchers to inspect the underlying mechanisms of ML to achieve accurate predictions, which are particularly important for rational design and optimizations of perovskite materials.…”

Section: Establishment Of Explainable Models For MLmentioning

confidence: 99%

Machine Learning‐Accelerated Development of Perovskite Optoelectronics Toward Efficient Energy Harvesting and Conversion

Chen,

Huang

2023

Adv Energy and Sustain Res

View full text Add to dashboard Cite

For next‐generation optoelectronic devices with efficient energy harvesting and conversion, designing advanced perovskite materials with exceptional optoelectrical properties is highly critical. However, the conventional trial‐and‐error approaches usually lead to long research periods, high costs, and low efficiency, which hinder the efficient development of optoelectronic devices for broad applications. The machine learning (ML) technique emerges as a powerful tool for materials designs, which supplies promising solutions to break the current bottlenecks in the developments of perovskite optoelectronics. Herein, the fundamental workflow of ML to interpret the working mechanisms step by step from a general perspective is first demonstrated. Then, the significant contributions of ML in designs and explorations of perovskite optoelectronics regarding novel materials discovery, the underlying mechanisms interpretation, and large‐scale information process strategy are illustrated. Based on current research progress, the potential of ML techniques in cross‐disciplinary directions to achieve the boost of material designs and optimizations toward perovskite materials is pointed out. In the end, the current advances of ML in perovskite optoelectronics are summarized and the future development directions are shown. This perspective supplies important insights into the developments of perovskite materials for the next generation of efficient and stable optoelectronic devices.

show abstract

“…Machine learning (ML) has emerged as a crucial domain in science and technology, exerting a substantial socioeconomic-environmental influence on various aspects of human and natural systems [1,2]. ML allows us to learn from vast amounts of data and improve the predictive performance of models.…”

Section: Introductionmentioning

confidence: 99%

“…Considerable concern has been expressed about relying on opaque models that may result in decisions that are not fully comprehended or, even worse, violate ethical principles regarding business and the environment or legal norms [1,8]. These risks are particularly relevant for decision-making in real-life scenarios and for access to public benefits [9], for example, digitalization in agriculture [10] and terrestrial conservation [11].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Tree-Based Modeling for Large-Scale Management in Agriculture: Explaining Organic Matter Content in Soil

Lee,

Lee

2024

Applied Sciences

View full text Add to dashboard Cite

Machine learning (ML) has become more prevalent as a tool used for biogeochemical analysis in agricultural management. However, a common drawback of ML models is the lack of interpretability, as they are black boxes that provide little insight into agricultural management. To overcome this limitation, we compared three tree-based models (decision tree, random forest, and gradient boosting) to explain soil organic matter content through Shapley additive explanations (SHAP). Here, we used nationwide data on field crops, soil, terrain, and climate across South Korea (n = 9584). Using the SHAP method, we identified common primary controls of the models, for example, regions with precipitation levels above 1400 mm and exchangeable potassium levels exceeding 1 cmol+ kg−1, which favor enhanced organic matter in the soil. Different models identified different impacts of macronutrients on the organic matter content in the soil. The SHAP method is practical for assessing whether different ML models yield consistent findings in addressing these inquiries. Increasing the explainability of these models means determining essential variables related to soil organic matter management and understanding their associations for specific instances.

show abstract

Benchmarking and survey of explanation methods for black box models

Cited by 56 publications

References 140 publications

A Meta Algorithm for Interpretable Ensemble Learning: The League of Experts

A Meta Algorithm for Interpretable Ensemble Learning: The League of Experts

Machine Learning‐Accelerated Development of Perovskite Optoelectronics Toward Efficient Energy Harvesting and Conversion

Tree-Based Modeling for Large-Scale Management in Agriculture: Explaining Organic Matter Content in Soil

Contact Info

Product

Resources

About