Generalized Additive Models from a Neural Network Perspective

Waal, de; Toit, J.V. Du

doi:10.1109/icdmw.2007.127

Cited by 4 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We will show how a shallow network, the Multi-Layer Perceptron (MLP) can be fully explained by formulating it as a General Additive Neural Network (GANN). This methodology has a long history [4]. However, to our knowledge there is no method to derive the GANN from data, rather a model structure needs to be assumed or hypothesized from experimental data analysis.…”

Section: Introductionmentioning

confidence: 99%

Explaining the Neural Network: A Case Study to Model the Incidence of Cervical Cancer

Lisboa

Ortega‐Martorell

Olier

2020

Information Processing and Management of Uncertainty in Knowledge-Based Systems

View full text Add to dashboard Cite

Neural networks are frequently applied to medical data. We describe how complex and imbalanced data can be modelled with simple but accurate neural networks that are transparent to the user. In the case of a data set on cervical cancer with 753 observations excluding, missing values, and 32 covariates, with a prevalence of 73 cases (9.69%), we explain how model selection can be applied to the Multi-Layer Perceptron (MLP) by deriving a representation using a General Additive Neural Network.The model achieves an AUROC of 0.621 CI [0.519,0.721] for predicting positive diagnosis with Schiller's test. This is comparable with the performance obtained by a deep learning network with an AUROC of 0.667 [1]. Instead of using all covariates, the Partial Response Network (PRN) involves just 2 variables, namely the number of years on Hormonal Contraceptives and the number of years using IUD, in a fully explained model. This is consistent with an additive non-linear statistical approach, the Sparse Additive Model [2] which estimates non-linear components in a logistic regression classifier using the backfitting algorithm applied to an ANOVA functional expansion.This paper shows how the PRN, applied to a challenging classification task, can provide insights into the influential variables, in this case correlated with incidence of cervical cancer, so reducing the number of unnecessary variables to be collected for screening. It does so by exploiting the efficiency of sparse statistical models to select features from an ANOVA decomposition of the MLP, in the process deriving a fully interpretable model.

show abstract

Section: Introductionmentioning

confidence: 99%

Explaining the Neural Network: A Case Study to Model the Incidence of Cervical Cancer

Lisboa

Ortega‐Martorell

Olier

2020

Information Processing and Management of Uncertainty in Knowledge-Based Systems

View full text Add to dashboard Cite

show abstract

“…The idea of generating feature-level interpretability in deep neural networks by translating GAMs into a neural framework was already introduced by Potts (1999) and expanded by de Waal and du Toit (2007). While the framework was remarkably parameter-sparse, it did not use backpropagation and hence did not achieve as good predictive results as GAMs, while remaining less interpretable.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Chang et al (2021) introduced NODE-GAM, a differentiable model based on forgetful decision trees developed for high-risk domains. All these models follow the additive framework from GAMs and learn the nonlinear additive features with separate networks, one for each feature or feature interaction, either leveraging MLPs (Potts, 1999;de Waal and du Toit, 2007;Agarwal et al, 2021;Yang et al, 2021;Radenovic et al, 2022), using decision trees (Chang et al, 2021) or using Splines (Rügamer et al, 2020;Seifert et al, 2022;Luber et al, 2023).…”

Section: Literature Reviewmentioning

confidence: 99%

Estimating and evaluating mixed and semiparametric models with statistical and deep learning methods

Kruse

View full text Add to dashboard Cite

Semiparametric models are well-established, versatile and effective statistical models for analysing complex data by combining both parametric and non-parametric components and are used in a wide range of applications and fields. Meanwhile, deep learning models are a type of machine learning model designed to identify and represent intricate patterns and relationships in data, which are gaining popularity due to their impressive performance in various applications, including analysing structured tabular data and, in contrast to statistical models, unstructured data such as images, text, and sound recordings. However, the complexity of these models can make them challenging to understand and interpret, limiting their transparency and interpretability.The purpose of this thesis is twofold. Firstly, develop more efficient model selection and evaluation methods that can handle model uncertainty for semiparametric and deep learning models.Secondly, to merge traditional statistical methods with machine and deep learning concepts, combining their strengths to mitigate their weaknesses.Based on Stein's unbiased risk estimate, part one of this thesis introduces a criterion for determining squared loss optimal weights for model averaging of (conditional) linear mixed models. Furthermore, the complicated underlying optimisation of the presented criterion is discussed, and a possible solution via a specifically customised algorithm based on the augmented Lagrangian is introduced.An essential part of model evaluation and selection in statistics is model complexity, often measured in degrees of freedom. In the second part of this thesis, three different methods for measuring model complexity based on the concept of covariance penalties associated with degrees of freedom are presented and ultimately compared and analysed using different simulations.The third and final section of the thesis introduces a new type of neural additive models for location, scale, and shape by combining the GAMLSS distribution regression framework with neural network techniques and principles. This approach differs from previous deep learning methods as it can model the entire response distribution rather than just the mean response. The effectiveness of this method is evaluated using simulated and real data and compared against established statistical and deep learning methodologies. iii Zusammenfassung Semiparametrische Modelle sind bewährte, vielseitige und effiziente statistische Modelle für die Analyse komplexer Daten, die parametrische und nichtparametrische Komponenten kombinieren und in einer Vielzahl von Anwendungen und Bereichen eingesetzt werden. Deep Learning Modelle sind eine Art Machine Learning-Modelle, die darauf ausgelegt sind, komplexe Muster und Beziehungen in Daten zu erkennen und darzustellen. Sie erfreuen sich aufgrund ihrer beeindruckenden Leistungsfähigkeit in verschiedenen Anwendungen immer größerer Beliebtheit, z.B. bei der Analyse von strukturierten Tabellendaten oder, im Gegensatz zu statistischen Modellen, von unstrukturie...

show abstract

Generalized additive neural network with flexible parametric link function: model estimation using simulated and real clinical data

Brás-Geraldes

Papoila

Xufre³

2017

Neural Comput & Applic

View full text Add to dashboard Cite

Generalized Additive Models from a Neural Network Perspective

Cited by 4 publications

References 12 publications

Explaining the Neural Network: A Case Study to Model the Incidence of Cervical Cancer

Explaining the Neural Network: A Case Study to Model the Incidence of Cervical Cancer

Estimating and evaluating mixed and semiparametric models with statistical and deep learning methods

Generalized additive neural network with flexible parametric link function: model estimation using simulated and real clinical data

Contact Info

Product

Resources

About