Fundamental Importance of Reference Glucose Analyzer Accuracy for Evaluating the Performance of Blood Glucose Monitoring Systems (BGMSs)

Bailey, Timothy S.; Klaff, Leslie J.; Wallace, Jane F.; Greene, Carmine; Pardo, Scott; Harrison, Bern; Simmons, David A.

doi:10.1177/1932296816634356

Cited by 16 publications

(22 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Is it at least partly caused by the use of different YSI instruments in different centers? The lead author coauthored an article titled "Fundamental Importance of Reference Glucose Analyzer Accuracy for Evaluating the Performance of Blood Glucose Monitoring Systems (BGMSs)" in this journal earlier this year, 5 indicating that this a known issue. Because Figure 1 shows professional measurements, which are expected to provide accurate results, the negative bias might be attributed to differences between the study site's and the device manufacturer's YSIs used for calibration of the BGMS.…”

Section: Original Articlementioning

confidence: 99%

Analysis of “Accuracy and User Performance Evaluation of a New, Wireless-Enabled Blood Glucose Monitoring System That Links to a Smart Mobile Device”

Rutschmann

2017

J Diabetes Sci Technol

View full text Add to dashboard Cite

Blood glucose monitoring systems (BGMS) are widely used by people with diabetes to manage their disease and to improve control of blood glucose levels. Therefore many studies are performed to examine the accuracy of BGMS currently on the market or becoming available soon according to the applicable ISO standard 15197. 1,2 Studies that examine the accuracy of BGMS are quite difficult and often prone to significant errors. Error sources include the acquisition of meters and test strips, the performance of the comparison method and the correct interpretation and application of ISO 15197:2013.Bailey and colleagues 1 state that the presented evaluation was an accuracy and user performance evaluation of a new BGMS, performed according to ISO 15197:2013 prerequisites. One evaluation was performed according to clause 6.3 (ie, system accuracy evaluation by trained personnel) and in the other evaluation clause 8 (ie, user performance evaluation) of the technical standard were applied.In these studies, the BGMS showed accurate results in the hands of laboratory personnel as well as in the hands of untrained users, fulfilling the accuracy criteria of ISO 15197:2013. The studies were well-conducted, taking into account the requirements of ISO 15197:2013. This standard addresses manufacturers of BGMS, but may also be used by researchers investigating available BGMS.Although these studies were, in general, well designed, they face some limitations. The first limitation is that study funding was provided by the manufacturer, which is common in such studies and does not imply that the results are not to be trusted. In addition, the test strips and meters were provided by the device manufacturer. Ideally, such trials should be performed by an independent organization using testing strips purchased from the market place to exclude a preselection by the manufacturer.3 In addition, funding of such studies should not depend on the manufacturer of the tested BMGS but rather be financed by funding pools.Another limitation, that interestingly is becoming obvious due to the presented excellent performance of the BGMS, relates to the used comparison method. Every accuracy evaluation is based on the comparison of a BGMS to a comparison method; therefore one limiting factor in such analyses is the quality of this comparison method, which is frequently called "reference method." The authors of the study use YSI analyzers, as recommended by the manufacturer, to "obtain reference values for comparison." Ideally, a "gold standard" analytical method such as isotope dilution mass spectrometry (ID-GC/ MS) should be used, but this method is not suitable for routine laboratory measurements. When an instrument like the YSI is AbstractIn the study published in Journal of Diabetes, Science and Technology, Timothy S. Bailey and coauthors evaluated system accuracy and user performance of a new, wireless-enabled blood glucose monitoring system, the Contour Plus ONE blood glucose monitoring system. The authors declare that the study protocol was performed...

show abstract

Section: Original Articlementioning

confidence: 99%

Analysis of “Accuracy and User Performance Evaluation of a New, Wireless-Enabled Blood Glucose Monitoring System That Links to a Smart Mobile Device”

Rutschmann

2017

J Diabetes Sci Technol

View full text Add to dashboard Cite

show abstract

“…However, differences in accuracy data depending on the applied comparison method/system were found which is consistent with results of two previous ISO 15197-based studies performed at our institute [ 9 , 32 ]. This influence might be reduced if manufacturers would adjust calibration of their reference methods [ 33 ].…”

Section: Discussionmentioning

confidence: 99%

User Performance Evaluation of Four Blood Glucose Monitoring Systems Applying ISO 15197:2013 Accuracy Criteria and Calculation of Insulin Dosing Errors

et al. 2018

View full text Add to dashboard Cite

IntroductionThe international standard ISO 15197:2013 requires a user performance evaluation to assess if intended users are able to obtain accurate blood glucose measurement results with a self-monitoring of blood glucose (SMBG) system. In this study, user performance was evaluated for four SMBG systems on the basis of ISO 15197:2013, and possibly related insulin dosing errors were calculated. Additionally, accuracy was assessed in the hands of study personnel.MethodsAccu-Chek® Performa Connect (A), Contour® plus ONE (B), FreeStyle Optium Neo (C), and OneTouch Select® Plus (D) were evaluated with one test strip lot. After familiarization with the systems, subjects collected a capillary blood sample and performed an SMBG measurement. Study personnel observed the subjects’ measurement technique. Then, study personnel performed SMBG measurements and comparison measurements. Number and percentage of SMBG measurements within ± 15 mg/dl and ± 15% of the comparison measurements at glucose concentrations < 100 and ≥ 100 mg/dl, respectively, were calculated. In addition, insulin dosing errors were modelled.ResultsIn the hands of lay-users three systems fulfilled ISO 15197:2013 accuracy criteria with the investigated test strip lot showing 96% (A), 100% (B), and 98% (C) of results within the defined limits. All systems fulfilled minimum accuracy criteria in the hands of study personnel [99% (A), 100% (B), 99.5% (C), 96% (D)]. Measurements with all four systems were within zones of the consensus error grid and surveillance error grid associated with no or minimal risk. Regarding calculated insulin dosing errors, all 99% ranges were between dosing errors of − 2.7 and + 1.4 units for measurements in the hands of lay-users and between − 2.5 and + 1.4 units for study personnel. Frequent lay-user errors were not checking the test strips’ expiry date and applying blood incorrectly.ConclusionsData obtained in this study show that not all available SMBG systems complied with ISO 15197:2013 accuracy criteria when measurements were performed by lay-users.Trial RegistrationThe study was registered at ClinicalTrials.gov (NCT02916576).FundingAscensia Diabetes Care Deutschland GmbH.Electronic supplementary materialThe online version of this article (10.1007/s13300-018-0392-6) contains supplementary material, which is available to authorized users.

show abstract

“…This is particularly relevant to the YSI comparison method, which although in widespread use, is complex and variable in terms of reported maintenance 21 calibration and checks, 22,23 traceability, 24 target and acceptance values, imprecision, and the exact procedure used for sample analysis. 15,21,25…”

Section: Reference/comparison Methodsmentioning

confidence: 99%

“…The use of an appropriate well-managed reference/comparison methods remains of paramount importance in performing accuracy evaluations correctly. 15 Its role becomes even more important with increasingly strict performance requirements for BGMSs. Reference/comparison analyzer errors and systematic measurement differences (biases) between methods that previously may have been considered acceptable may now be consequential and particularly important for comparison against analytical accuracy criteria which are more stringent than clinical accuracy requirements.…”

Section: Reference/comparison Methodsmentioning

confidence: 99%

Analysis of “Capillary and Venous Blood Glucose Accuracy in Blood Glucose Meters Versus Reference Standards: The Impact of Study Design on Accuracy Evaluations”

Thorpe¹

2018

J Diabetes Sci Technol

View full text Add to dashboard Cite

The article by Macleod and coauthors outlines an accuracy study of two blood glucose monitoring systems (BGMSs) against glucose-oxidase- and hexokinase-based assays showing how the reference/comparison method and inappropriate sample comparisons can affect accuracy conclusions. The dangers of independent institutions producing inappropriate conclusions when the methodology used for product regulatory or registration accuracy requirements is not according to best practice are stressed. The authors highlight several important aspects of a multistep accuracy evaluation protocol. However, it is essential that anyone undertaking or reporting BGMS accuracy studies should have a clear understanding of each and every protocol point, best practice, and how each can influence accuracy conclusions. Claims against regulatory accuracy criteria should be made only if the detailed specified protocol design and analysis is followed.

show abstract

Fundamental Importance of Reference Glucose Analyzer Accuracy for Evaluating the Performance of Blood Glucose Monitoring Systems (BGMSs)

Abstract: These findings suggest that the ability to demonstrate that a BGMS meets accuracy guidelines is influenced by reference instrument accuracy.

Cited by 16 publications

References 4 publications

Analysis of “Accuracy and User Performance Evaluation of a New, Wireless-Enabled Blood Glucose Monitoring System That Links to a Smart Mobile Device”

Analysis of “Accuracy and User Performance Evaluation of a New, Wireless-Enabled Blood Glucose Monitoring System That Links to a Smart Mobile Device”

User Performance Evaluation of Four Blood Glucose Monitoring Systems Applying ISO 15197:2013 Accuracy Criteria and Calculation of Insulin Dosing Errors

Analysis of “Capillary and Venous Blood Glucose Accuracy in Blood Glucose Meters Versus Reference Standards: The Impact of Study Design on Accuracy Evaluations”

Contact Info

Product

Resources

About