Background: There is a need to better understand the association between sleep and chronic diseases. In this study we developed a natural language processing (NLP) algorithm to mine polysomnography (PSG) free-text notes from electronic medical records (EMR) and evaluated the performance. Methods: Using the Veterans Health Administration EMR, we identified 46,093 PSG studies using CPT code 95,810 from 1 October 2000–30 September 2019. We randomly selected 200 notes to compare the accuracy of the NLP algorithm in mining sleep parameters including total sleep time (TST), sleep efficiency (SE) and sleep onset latency (SOL), wake after sleep onset (WASO), and apnea-hypopnea index (AHI) compared to visual inspection by raters masked to the NLP output. Results: The NLP performance on the training phase was >0.90 for precision, recall, and F-1 score for TST, SOL, SE, WASO, and AHI. The NLP performance on the test phase was >0.90 for precision, recall, and F-1 score for TST, SOL, SE, WASO, and AHI. Conclusions: This study showed that NLP is an accurate technique to extract sleep parameters from PSG reports in the EMR. Thus, NLP can serve as an effective tool in large health care systems to evaluate and improve patient care.

Introduction Obstructive sleep apnea (OSA) is a major sleep disorder that presents with excessive daytime sleepiness (EDS). Epworth Sleepiness Scale (ESS) as a measure of EDS is used to monitor population sleep health. We aim to assess the association between ESS and apnea hypopnea index (AHI) extracted from polysomnography (PSG) reports using natural language processing (NLP) algorithms. Methods We curated 90,483 PSG notes from the nationwide Corporate Data Warehouse (CDW) of Veteran Affair (VA) from 10/1999 to 10/2020. We used rule-based nearest neighbor and forward-backward NLP techniques to extract ESS and AHI from PSG reports. We reported the performance of NLP algorithm compared to chart review. We used AHI>5 as the cut-point to identify OSA and incremented the threshold for abnormal ESS from 5 to 15 to find the best cut-point for stratification of EDS. We used logistic regression to report the performance of the ESS cut-point using the area under the curve (AUC). The model also adjusted for age, sex, BMI, race, ethnicity, and Charlson comorbidity index. Results 39,318 patient clinical notes’ (age 50±15 years; BMI 30±5) notes documented both AHI and ESS. (50±15 year; 30±5 BMI). NLP algorithms accuracy was ≥ 90%. We observed the same level of sensitivity (57%) or specificity (42%) for ESS of 5 and 10, but the ESS of 5 resulted in the best negative (27%) and positive (72%) predictive value for AHI prediction. The area under curve for prediction OSA based on ESS cut-off of 5 was 0.50 (95%CI, 0.49, 0.51) and the AUC improved to 0.65 (95%CI, 0.64, 0.65) after adjustment. Conclusion Our results suggest that ESS is not an effective tool for predicting AHI. Reducing the ESS cut-off to 5 may enhance its predictive value, but it does not have clinical implication in identifying patient with OSA. Sleep medicine clinicians may use other sleep questionnaires, such as the STOP-Bang questionnaire, which is reported to be a reliable, concise, and easy-to-use screening tool. Support (if any) This work is supported by the VA Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413); National Institute of Health (NHI) Grant # 1K25HL152006-01 (PI: Razjouyan) and Grant # R01NR018342 (PI: Nowakowski).