Estimating and evaluating mixed and semiparametric models with statistical and deep learning methods

Abstract

Semiparametric models are well-established, versatile and effective statistical models for analysing complex data by combining both parametric and non-parametric components and are used in a wide range of applications and fields. Meanwhile, deep learning models are a type of machine learning model designed to identify and represent intricate patterns and relationships in data, which are gaining popularity due to their impressive performance in various applications, including analysing structured tabular data and, in contrast to statistical models, unstructured data such as images, text, and sound recordings. However, the complexity of these models can make them challenging to understand and interpret, limiting their transparency and interpretability.The purpose of this thesis is twofold. Firstly, develop more efficient model selection and evaluation methods that can handle model uncertainty for semiparametric and deep learning models.Secondly, to merge traditional statistical methods with machine and deep learning concepts, combining their strengths to mitigate their weaknesses.Based on Stein's unbiased risk estimate, part one of this thesis introduces a criterion for determining squared loss optimal weights for model averaging of (conditional) linear mixed models. Furthermore, the complicated underlying optimisation of the presented criterion is discussed, and a possible solution via a specifically customised algorithm based on the augmented Lagrangian is introduced.An essential part of model evaluation and selection in statistics is model complexity, often measured in degrees of freedom. In the second part of this thesis, three different methods for measuring model complexity based on the concept of covariance penalties associated with degrees of freedom are presented and ultimately compared and analysed using different simulations.The third and final section of the thesis introduces a new type of neural additive models for location, scale, and shape by combining the GAMLSS distribution regression framework with neural network techniques and principles. This approach differs from previous deep learning methods as it can model the entire response distribution rather than just the mean response. The effectiveness of this method is evaluated using simulated and real data and compared against established statistical and deep learning methodologies. iii Zusammenfassung Semiparametrische Modelle sind bewährte, vielseitige und effiziente statistische Modelle für die Analyse komplexer Daten, die parametrische und nichtparametrische Komponenten kombinieren und in einer Vielzahl von Anwendungen und Bereichen eingesetzt werden. Deep Learning Modelle sind eine Art Machine Learning-Modelle, die darauf ausgelegt sind, komplexe Muster und Beziehungen in Daten zu erkennen und darzustellen. Sie erfreuen sich aufgrund ihrer beeindruckenden Leistungsfähigkeit in verschiedenen Anwendungen immer größerer Beliebtheit, z.B. bei der Analyse von strukturierten Tabellendaten oder, im Gegensatz zu statistischen Modellen, von unstrukturie...