A new approximation method for the Shapley value applied to the WTC 9/11 terrorist attack

Campen, Tjeerd; Hamers, Herbert; Husslage, B.G.M.; Lindelauf, Roy

doi:10.1007/s13278-017-0480-z

Cited by 54 publications

(21 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Having an analytical solution for Shapley values of large systems is an NP-complete problem [58]. Therefore, heuristics [59], predictors [30], and estimators [28,60] are used and are under development to solve this issue. Interestingly, Shapley value has found a unique spot in the field of explainable machine learning [61] and is used to understand deeper and more complicated neural network architectures [61], prune the unnecessary elements [62], and even correct biased networks [60].…”

Section: Discussionmentioning

confidence: 99%

Systematic Perturbation of an Artificial Neural Network:A Step Towards Quantifying Causal Contributions in The Brain

Fakhar

Hilgetag

2021

Preprint

View full text Add to dashboard Cite

Lesion inference analysis is a fundamental approach for characterizing the causal contributions of neural elements to brain function. Historically, it has helped to localize specialized functions in the brain after brain damage, and it has gained new prominence through the arrival of modern optogenetic perturbation techniques that allow probing the functional contributions of neural circuit elements at unprecedented levels of detail.While inferences drawn from brain lesions are conceptually powerful, they face methodological difficulties due to the brain’s complexity. Particularly, they are challenged to disentangle the functional contributions of individual neural elements because many elements may contribute to a particular function, and these elements may be interacting anatomically as well as functionally. Therefore, studies of real-world data, as in clinical lesion studies, are not suitable for establishing the reliability of lesion approaches due to an unknown, potentially complex ground truth. Instead, ground truth studies of well-characterized artificial systems are required.Here, we systematically and exhaustively lesioned a small Artificial Neural Network (ANN) playing a classic arcade game. We determined the functional contributions of all nodes and links, contrasting results from single-element perturbations and perturbing multiple elements simultaneously. Moreover, we computed pairwise causal functional interactions between the network elements, and looked deeper into the system’s inner workings, proposing a mechanistic explanation for the effects of lesions.We found that not every perturbation necessarily reveals causation, as lesioning elements, one at a time, produced biased results. By contrast, multi-site lesion analysis captured crucial details that were missed by single-site lesions. We conclude that even small and seemingly simple ANNs show surprising complexity that needs to be understood for deriving a causal picture of the system. In the context of rapidly evolving multivariate brain-mapping approaches and inference methods, we advocate using in-silico experiments and ground-truth models to verify fundamental assumptions, technical limitations, and the scope of possible interpretations of these methods.Author summaryThe motto “No causation without manipulation” is canonical to scientific endeavors. In particular, neuroscience seeks to find which brain elements are causally involved in cognition and behavior of interest by perturbing them. However, due to complex interactions among those elements, this goal has remained challenging.In this paper, we used an Artificial Neural Network as a ground-truth model to compare the inferential capacities of lesioning the system one element at a time against sampling from the set of all possible combinations of lesions.We argue for employing more exhaustive perturbation regimes since, as we show, lesioning one element at a time provides misleading results. We further advocate using simulated experiments and ground-truth models to verify the assumptions and limitations of brain-mapping methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Systematic Perturbation of an Artificial Neural Network:A Step Towards Quantifying Causal Contributions in The Brain

Fakhar

Hilgetag

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In [2], random sampling Monte-Carlo (MC) estimation method is proposed to approximate SV with fewer utility evaluations. In [23], the authors proposed a structured random sampling MC estimation method to approximate SV eiciently. Since the utility values of diferent coalitions vary without a clear pattern, the proposed method swaps players to diferent positions in the coalition orders.…”

Section: Fl Participant Contribution Evaluation Via Shapley Valuementioning

confidence: 99%

GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning

Liu

Chen

et al. 2022

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

Federated Learning (FL) bridges the gap between collaborative machine learning and preserving data privacy. To sustain the long-term operation of an FL ecosystem, it is important to attract high-quality data owners with appropriate incentive schemes. As an important building block of such incentive schemes, it is essential to fairly evaluate participants’ contribution to the performance of the final FL model without exposing their private data. Shapley Value (SV)-based techniques have been widely adopted to provide a fair evaluation of FL participant contributions. However, existing approaches incur significant computation costs, making them difficult to apply in practice. In this paper, we propose the Guided Truncation Gradient Shapley (GTG-Shapley) approach to address this challenge. It reconstructs FL models from gradient updates for SV calculation instead of repeatedly training with different combinations of FL participants. In addition, we design a guided Monte Carlo sampling approach combined with within-round and between-round truncation to further reduce the number of model reconstructions and evaluations required, through extensive experiments under diverse realistic data distribution settings. The results demonstrate that GTG-Shapley can closely approximate actual Shapley values, while significantly increasing computational efficiency compared to the state of the art, especially under non-i.i.d. settings.

show abstract

“…In [2], random sampling Monte-Carlo (MC) estimation method is proposed to approximate SV with fewer utility evaluations. In [23], the authors proposed a structured random sampling MC estimation method to approximate SV efficiently. Since the utility values of different coalitions vary without a clear pattern, the proposed method swaps players to different positions in the coalition ordering.…”

Section: Fl Participant Contribution Evaluation Via Shapley Valuementioning

confidence: 99%

GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning

Liu¹,

Chen²,

Han³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) bridges the gap between collaborative machine learning and preserving data privacy. To sustain the long-term operation of an FL ecosystem, it is important to attract high quality data owners with appropriate incentive schemes. As an important building block of such incentive schemes, it is essential to fairly evaluate participants' contribution to the performance of the final FL model without exposing their private data. Shapley Value (SV)-based techniques have been widely adopted to provide fair evaluation of FL participant contributions. However, existing approaches incur significant computation costs, making them difficult to apply in practice. In this paper, we propose the Guided Truncation Gradient Shapley (GTG-Shapley) approach to address this challenge. It reconstructs FL models from gradient updates for SV calculation instead of repeatedly training with different combinations of FL participants. In addition, we design a guided Monte Carlo sampling approach combined with within-round and between-round truncation to further reduce the number of model reconstructions and evaluations required, through extensive experiments under diverse realistic data distribution settings. The results demonstrate that GTG-Shapley can closely approximate actual Shapley values, while significantly increasing computational efficiency compared to the state of the art, especially under non-i.i.d. settings.

show abstract

A new approximation method for the Shapley value applied to the WTC 9/11 terrorist attack

Cited by 54 publications

References 20 publications

Systematic Perturbation of an Artificial Neural Network:A Step Towards Quantifying Causal Contributions in The Brain

Systematic Perturbation of an Artificial Neural Network:A Step Towards Quantifying Causal Contributions in The Brain

GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning

GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning

Contact Info

Product

Resources

About