SummaryWearable electroencephalography devices emerge as a cost‐effective and ergonomic alternative to gold‐standard polysomnography, paving the way for better health monitoring and sleep disorder screening. Machine learning allows to automate sleep stage classification, but trust and reliability issues have hampered its adoption in clinical applications. Estimating uncertainty is a crucial factor in enhancing reliability by identifying regions of heightened and diminished confidence. In this study, we used an uncertainty‐centred machine learning pipeline, U‐PASS, to automate sleep staging in a challenging real‐world dataset of single‐channel electroencephalography and accelerometry collected with a wearable device from an elderly population. We were able to effectively limit the uncertainty of our machine learning model and to reliably inform clinical experts of which predictions were uncertain to improve the machine learning model's reliability. This increased the five‐stage sleep‐scoring accuracy of a state‐of‐the‐art machine learning model from 63.9% to 71.2% on our dataset. Remarkably, the machine learning approach outperformed the human expert in interpreting these wearable data. Manual review by sleep specialists, without specific training for sleep staging on wearable electroencephalography, proved ineffective. The clinical utility of this automated remote monitoring system was also demonstrated, establishing a strong correlation between the predicted sleep parameters and the reference polysomnography parameters, and reproducing known correlations with the apnea–hypopnea index. In essence, this work presents a promising avenue to revolutionize remote patient care through the power of machine learning by the use of an automated data‐processing pipeline enhanced with uncertainty estimation.
