Quantifying the Robustness of a Broth-Based Escherichia coli O157:H7 Growth Model in Ground Beef

Abstract

The robustness of a microbial growth model must be assessed before the model can be applied to new food matrices; therefore, a methodology for quantifying robustness was developed. A robustness index (RI) was computed as the ratio of the standard error of prediction to the standard error of calibration for a given model, where the standard error of calibration was defined as the root mean square error of the growth model against the data (log CFU per gram versus time) used to parameterize the model and the standard error of prediction was defined as the root mean square error of the model against an independent data set. This technique was used to evaluate the robustness of a broth-based model for aerobic growth of Escherichia coli 0157:H7 (in the U.S Department of Agriculture Agricultural Research Service Pathogen Modeling Program) in predicting growth in ground beef under different conditions. Comparison against previously published data (132 data sets with 1,178 total data points) from experiments in ground beef at various experimental conditions (4.8 to 45 degrees C and pH 5.5 to 5.9) yielded RI values ranging from 0.11 to 2.99. The estimated overall RI was 1.13. At temperatures between 15 and 40 degrees C, the RI was close to and smaller than 1, indicating that the growth model is relatively robust in that temperature range. However, the RI also was related (P < 0.05) to temperature. By quantifying the predictive accuracy relative to the expected accuracy, the RI could be a useful tool for comparing various models under different conditions.

Quantifying the Robustness of a Broth-Based Model for Predicting Listeria monocytogenes Growth in Meat and Poultry Products

“…To avoid dangerous errors when using growth models for risk assessment (or other application), predictive models should be validated against independent data relevant to the application. In prior studies, the broth-based PMP growth model for Escherichia coli O157:H7 underpredicted (i.e., fail-dangerous) microbial counts when compared to data in ground beef (6,22,23). Similar results were reported in the PMP for the Clostridium perfringens growth model against data from broth (21).…”

“…An accurate prediction may require a consideration of whether a model is easy to use (the simplest one for a given purpose and data quality), whether it is robust and accurate (it must reflect reality), and whether it is validated against independent data sets (19). The validation or performance evaluation of a model can also be referred to as the robustness of the model (6). The robustness indicates how well a model predicts future independent results across a wide domain of conditions.…”

“…The true measure of product safety is actual microbial counts, not model parameters. Campos et al (6) introduced a new methodology to evaluate the robustness of a microbial growth model in terms of microbial counts. The robustness index (RI) was defined as the ratio of the standard error of prediction to the standard error of calibration.…”

“…A robust model will have an RI value near to or less than 1, meaning that the overall performance of a microbial model tested against an independent data set is within the expected error (standard error of calibration) of the model. Campos et al (6) also stated that the RI value alone does not tell whether the observed values are above or below the predicted values; therefore, the mean relative error (RE) is used with the RI to provide this information.…”

Status of Microbial Modeling in Food Process Models

“…Although the original, broth‐based data sets were large (hundreds of curves), so that 30 parameters could be realistically estimated from the data, there should be some concern of over‐fitting when a model includes so many parameters. Nevertheless, several studies have tested the validity of the PMP growth models against independent data in food products and have shown reasonably good predictions (Campos and others 2005; Martino and others 2005).…”