Summary We discuss the validation of machine learning models, which is standard practice in determining model efficacy and generalizability. We argue that internal validation approaches, such as cross-validation and bootstrap, cannot guarantee the quality of a machine learning model due to potentially biased training data and the complexity of the validation procedure itself. For better evaluating the generalization ability of a learned model, we suggest leveraging on external data sources from elsewhere as validation datasets, namely external validation. Due to the lack of research attractions on external validation, especially a well-structured and comprehensive study, we discuss the necessity for external validation and propose two extensions of the external validation approach that may help reveal the true domain-relevant model from a candidate set. Moreover, we also suggest a procedure to check whether a set of validation datasets is valid and introduce statistical reference points for detecting external data problems.

Class-prediction accuracy provides a quick but superficial way of determining classifier performance. It does not inform on the reproducibility of the findings or whether the selected or constructed features used are meaningful and specific. Furthermore, the class-prediction accuracy oversummarizes and does not inform on how training and learning have been accomplished: two classifiers providing the same performance in one validation can disagree on many future validations. It does not provide explainability in its decision-making process and is not objective, as its value is also affected by class proportions in the validation set. Despite these issues, this does not mean we should omit the class-prediction accuracy. Instead, it needs to be enriched with accompanying evidence and tests that supplement and contextualize the reported accuracy. This additional evidence serves as augmentations and can help us perform machine learning better while avoiding naive reliance on oversimplified metrics.

No abstract

Dendritic cells residing in the skin represent a large family of antigen presenting cells, ranging from long-lived Langerhans cells (LC) in the epidermis to various distinct classical dendritic cell subsets in the dermis. Through genetic fate mapping analysis and single cell RNA sequencing we have identified a novel separate population of LC-independent CD207+CD326+ LClike cells in the dermis that homed at a slow rate to the LNs. These LClike cells were long-lived and radioresistant but, unlike LCs, they were gradually replenished by bone-marrow-derived precursors under steady state. LClike cells together with cDC1s were the main migratory CD207+CD326+ cell fractions present in the LN and not, as currently assumed, LCs, which were barely detectable, if at all. These findings bring new insights into the dynamism of cutaneous dendritic cells and opens novel avenues in the development of treatments to cure inflammatory skin disorders.