Variational Bayes survival analysis for unemployment modelling

Abstract

Mathematical modelling of unemployment dynamics attempts to predict the probability of a job seeker finding a job as a function of time. This is typically achieved by using information in unemployment records. These records are right censored, making survival analysis a suitable approach for parameter estimation. The proposed model uses a deep artificial neural network (ANN) as a non-linear hazard function. Through embedding, high-cardinality categorical features are analysed efficiently. The posterior distribution of the ANN parameters are estimated using a variational Bayes method. The model is evaluated on a time-to-employment data set spanning from 2011 to 2020 provided by the Slovenian public employment service. It is used to determine the employment probability over time for each individual on the record. Similar models could be applied to other questions with multi-dimensional, high-cardinality categorical data including censored records. Such data is often encountered in personal records, for example in medical records.

Predictive algorithms in the delivery of public employment services

“…France is reported to work on a system using artificial neural networks (reported in Griffin et al, 2020), a system based on a neural network approach for survival analysis is developed for Slovenia (Boškoski et al, 2021), and Kern et al (2021) compare (penalized) logistic regressions, random forest and gradient boosting approaches for Germany. Black et al (2007) find that profiling (the WPRS program) reduced the duration of jobseekers in the unemployment insurance system.…”