An Evaluation of Methods for Monitoring Annual Quality Measures by Month to Predict Year-End Values

Abstract

ObjectiveOur objective was to evaluate four different methods for monitoring annually defined quality measures on a monthly basis, so as to best predict year-end performance throughout the reporting year.MethodsWe developed four methods for monitoring annual measures by month: 1) Monthly Proportion: The proportion of patients with one or more encounters in the month who still needed to meet the measure at their first encounter of the month and met the measure by the end of the month; (2) Monthly Lookback Proportion: The proportion of patients seen in the month who met the measure by the end of the month, regardless of whether it was met in that month or previously in the reporting year; (3) Rolling 12 Month: The annual measure reported as if each month was the twelfth month of a twelve-month reporting period; and (4) YTD (Year-to-Date) Cumulative: The proportion of patients with one or more visits from the start of the reporting year through the month who satisfy the measure. We applied each method to two annual dental quality measures using data from two reporting years, and four different dental sites. We used mean squared error (MSE) to evaluate year-end predictive performance.ResultsMethod 3 (Rolling 12 Month) had the lowest MSE in 11 out of 16 cases (2 measures X 2 years X 4 sites) and lowest total MSE across all 16 cases. In 5 of the 16 cases, YTD Cumulative had the lowest MSE.ConclusionOur results demonstrate that the Rolling 12 Month method was best for predicting the year-end value across both measures and all four sites.