Stable clinical risk prediction against distribution shift in electronic health records

Deep continual multitask out-of-hospital incident severity assessment from changing clinical features

“…Finally, while there are studies addressing the training and deployment of Machine Learning models in the context of medical data with temporal shifts [12,13,[43][44][45], it is challenging to find similar studies for out-of-hospital emergencies. Furthermore, although the predefined feature domain strategy shares some similarities with the domain invariant feature approach proposed by [12] and the foundational model strategy described in [44], our approach in this work differs from previous solutions.…”

“…Furthermore, although the predefined feature domain strategy shares some similarities with the domain invariant feature approach proposed by [12] and the foundational model strategy described in [44], our approach in this work differs from previous solutions. We do not rely on raw feature aggregation [43], pre-shift patient weighting [13], or parsimonious models [45]. We instead integrate various strategies to address both the variability in the feature domain and the dynamic nature of parameter updating.…”

“…As discussed in previous paragraphs, when developing a Machine Learning-based system for providing triage decision support, it is crucial to consider the dataset shift phenomenon that naturally appears over time [12, 13]. In the Valencian Region, the absence of measures or strategies to mitigate the negative effects of distributional drifts can lead to a gradual decline in system performance, as illustrated in Figure 1.…”

“…However, due to data availability reasons, the data considered to train and evaluate these multiple networks covered the period from 2009 to 2012. As previously discussed, when developing a Machine Learning-based system for providing triage decision support, it is crucial to consider the dataset shift phenomenon that naturally occurs over time [12, 13]. In the context of the Valencian region, shifts associated with the appearance and disappearance of clinical features over time pose a tough challenge that, if unaddressed, harms model performance through time, as exemplified in Figure 1.…”

“…After their design, we have implemented and applied them to multiple temporal sets of out-of-hospital emergency medical data from the Valencian region, to assess their effectiveness. Despite the existence of previous works addressing the training and deployment of Machine Learning models in the context of medical data with temporal dataset shifts [12, 13, 16–18], to our knowledge, this is the first study facing evolving clinical features within the domain of out-of-hospital emergencies.…”