Identification of thresholds for accuracy comparisons of heart rate and respiratory rate in neonates

Ginsburg, Amy Sarah; Coleman, Jesse; Ochieng, Roseline; Macharia, William; Dunsmuir, Dustin; Zhou, Guohai; Ansermino, J. Mark; Karlen, Walter

doi:10.12688/gatesopenres.13237.2

Cited by 4 publications

(9 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In setting the 30% threshold for the spread between 95% LOA, we considered the large withinneonate RR variability present in the verification phase data and avoided setting the target agreement threshold to be more restrictive than the within-neonate variation. 11 We chose to normalise the spread between 95% LOA and used a percentage error threshold to allow for comparison of performance over a wide range of RR values. Use of a ±30% error threshold for LOA to determine whether to accept a new method in cardiac output method comparison studies was proposed in 1999 and has since been used extensively in the field.…”

Section: Discussionmentioning

confidence: 99%

“…During Masimo Rad-97 reference device verification, we tested functionality and estimated within-neonate and betweenneonate variability to pre-specify thresholds for agreement comparisons. 11 We completed an initial round of open-label data collection from both devices in which complete reference device data were shared with EarlySense prior to analysis. Three rounds of closed-label analysis followed, in which the first 15-30 min of reference device data from each neonate were provided to Early-Sense to confirm time synchronicity across data sources (online supplemental appendices B, C).…”

Section: Study Design and Proceduresmentioning

confidence: 99%

“…between the 95% upper and lower LOAs of 30% was selected for both RR and HR. 11 We calculated the root-mean-square deviation (RMSD) for each comparison. Clarke error grids were constructed with zones of 20% discrepancy to improve clinical interpretability of results.…”

Section: Global Child Healthmentioning

confidence: 99%

See 2 more Smart Citations

Evaluation of a contactless neonatal physiological monitor in Nairobi, Kenya

Wang¹,

Macharia

Ochieng

et al. 2021

Arch Dis Child

Self Cite

View full text Add to dashboard Cite

BackgroundGlobally, 2.5 million neonates died in 2018, accounting for 46% of under-5 deaths. Multiparameter continuous physiological monitoring (MCPM) of neonates allows for early detection and treatment of life-threatening health problems. However, neonatal monitoring technology is largely unavailable in low-resource settings.MethodsIn four evaluation rounds, we prospectively compared the accuracy of the EarlySense under-mattress device to the Masimo Rad-97 pulse CO-oximeter with capnography reference device for heart rate (HR) and respiratory rate (RR) measurements in neonates in Kenya. EarlySense algorithm optimisations were made between evaluation rounds. In each evaluation round, we compared 200 randomly selected epochs of data using Bland-Altman plots and generated Clarke error grids with zones of 20% to aid in clinical interpretation.ResultsBetween 9 July 2019 and 8 January 2020, we collected 280 hours of MCPM data from 76 enrolled neonates. At the final evaluation round, the EarlySense MCPM device demonstrated a bias of −0.8 beats/minute for HR and 1.6 breaths/minute for RR, and normalised spread between the 95% upper and lower limits of agreement of 6.2% for HR and 27.3% for RR. Agreement between the two MCPM devices met the a priori–defined threshold of 30%. The Clarke error grids showed that all observations for HR and 197/200 for RR were within a 20% difference.ConclusionOur research indicates that there is acceptable agreement between the EarlySense and Masimo MCPM devices in the context of large within-subject variability; however, further studies establishing cost-effectiveness and clinical effectiveness are needed before large-scale implementation of the EarlySense MCPM device in neonates.Trial registration numberNCT03920761.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Study Design and Proceduresmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of a contactless neonatal physiological monitor in Nairobi, Kenya

Wang¹,

Macharia

Ochieng

et al. 2021

Arch Dis Child

Self Cite

View full text Add to dashboard Cite

show abstract

“…In order to identify agreement thresholds for comparison with Sibel’s ANNE technology, we assed functionality and estimated within- and between-neonate variability while verifying Masimo’s Rad-97 technology [ 8 ]. We ran an initial round of open-label data collection from both Sibel’s ANNE technology and the reference technologies to test the accuracy testing methods.…”

Section: Methodsmentioning

confidence: 99%

“…ETNA endeavours to understand real-world clinical feasibility, performance, and accuracy of novel technologies. The current study analyzes the clinical accuracy of an investigational MCPM technology compared to verified reference technologies [ 8 ].…”

Section: Introductionmentioning

confidence: 99%

Evaluation of Sibel’s Advanced Neonatal Epidermal (ANNE) wireless continuous physiological monitor in Nairobi, Kenya

Coleman¹,

Ginsburg

Macharia

et al. 2022

PLoS ONE

Self Cite

View full text Add to dashboard Cite

Background Neonatal multiparameter continuous physiological monitoring (MCPM) technologies assist with early detection of preventable and treatable causes of neonatal mortality. Evaluating accuracy of novel MCPM technologies is critical for their appropriate use and adoption. Methods We prospectively compared the accuracy of Sibel’s Advanced Neonatal Epidermal (ANNE) technology with Masimo’s Rad-97 pulse CO-oximeter with capnography and Spengler’s Tempo Easy reference technologies during four evaluation rounds. We compared accuracy of heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2), and skin temperature using Bland-Altman plots and root-mean-square deviation analyses (RMSD). Sibel’s ANNE algorithms were optimized between each round. We created Clarke error grids with zones of 20% to aid with clinical interpretation of HR and RR results. Results Between November 2019 and August 2020 we collected 320 hours of data from 84 neonates. In the final round, Sibel’s ANNE technology demonstrated a normalized bias of 0% for HR and 3.1% for RR, and a non-normalized bias of -0.3% for SpO2 and 0.2°C for temperature. The normalized spread between 95% upper and lower limits-of-agreement (LOA) was 4.7% for HR and 29.3% for RR. RMSD for SpO2 was 1.9% and 1.5°C for temperature. Agreement between Sibel’s ANNE technology and the reference technologies met the a priori-defined thresholds for 95% spread of LOA and RMSD. Clarke error grids showed that all HR and RR observations were within a 20% difference. Conclusion Our findings suggest acceptable agreement between Sibel’s ANNE and reference technologies. Clinical effectiveness, feasibility, usability, acceptability, and cost-effectiveness investigations are necessary for large-scale implementation.

show abstract

Clinical feasibility of a contactless multiparameter continuous monitoring technology for neonates in a large public maternity hospital in Nairobi, Kenya

Ginsburg

Nia

Chomba

et al. 2022

Sci Rep

View full text Add to dashboard Cite

Multiparameter continuous physiological monitoring (MCPM) technologies are critical in the clinical management of high-risk neonates; yet, these technologies are frequently unavailable in many African healthcare facilities. We conducted a prospective clinical feasibility study of EarlySense’s novel under-mattress MCPM technology in neonates at Pumwani Maternity Hospital in Nairobi, Kenya. To assess feasibility, we compared the performance of EarlySense’s technology to Masimo’s Rad-97 pulse CO-oximeter with capnography technology for heart rate (HR) and respiratory rate (RR) measurements using up-time, clinical event detection performance, and accuracy. Between September 15 and December 15, 2020, we collected and analyzed 470 hours of EarlySense data from 109 enrolled neonates. EarlySense’s technology’s up-time per neonate was 2.9 (range 0.8, 5.3) hours for HR and 2.1 (range 0.9, 4.0) hours for RR. The difference compared to the reference was a median of 0.6 (range 0.1, 3.1) hours for HR and 0.8 (range 0.1, 2.9) hours for RR. EarlySense’s technology identified high HR and RR events with high sensitivity (HR 81%; RR 83%) and specificity (HR 99%; RR 83%), but was less sensitive for low HR and RR (HR 0%; RR 14%) although maintained specificity (HR 100%; RR 95%). There was a greater number of false negative and false positive RR events than false negative and false positive HR events. The normalized spread of limits of agreement was 9.6% for HR and 28.6% for RR, which met the a priori-identified limit of 30%. EarlySense’s MCPM technology was clinically feasible as demonstrated by high percentage of up-time, strong clinical event detection performance, and agreement of HR and RR measurements compared to the reference technology. Studies in critically ill neonates, assessing barriers and facilitators to adoption, and costing analyses will be key to the technology’s development and potential uptake and scale-up.

show abstract

Identification of thresholds for accuracy comparisons of heart rate and respiratory rate in neonates

Cited by 4 publications

References 22 publications

Evaluation of a contactless neonatal physiological monitor in Nairobi, Kenya

Evaluation of a contactless neonatal physiological monitor in Nairobi, Kenya

Evaluation of Sibel’s Advanced Neonatal Epidermal (ANNE) wireless continuous physiological monitor in Nairobi, Kenya

Clinical feasibility of a contactless multiparameter continuous monitoring technology for neonates in a large public maternity hospital in Nairobi, Kenya

Contact Info

Product

Resources

About