Good modeling practice for PAT applications: Propagation of input uncertainty and sensitivity analysis

Abstract

The uncertainty and sensitivity analysis are evaluated for their usefulness as part of the model-building within Process Analytical Technology applications. A mechanistic model describing a batch cultivation of Streptomyces coelicolor for antibiotic production was used as case study. The input uncertainty resulting from assumptions of the model was propagated using the Monte Carlo procedure to estimate the output uncertainty. The results showed that significant uncertainty exists in the model outputs. Moreover the uncertainty in the biomass, glucose, ammonium and base-consumption were found low compared to the large uncertainty observed in the antibiotic and off-gas CO(2) predictions. The output uncertainty was observed to be lower during the exponential growth phase, while higher in the stationary and death phases - meaning the model describes some periods better than others. To understand which input parameters are responsible for the output uncertainty, three sensitivity methods (Standardized Regression Coefficients, Morris and differential analysis) were evaluated and compared. The results from these methods were mostly in agreement with each other and revealed that only few parameters (about 10) out of a total 56 were mainly responsible for the output uncertainty. Among these significant parameters, one finds parameters related to fermentation characteristics such as biomass metabolism, chemical equilibria and mass-transfer. Overall the uncertainty and sensitivity analysis are found promising for helping to build reliable mechanistic models and to interpret the model outputs properly. These tools make part of good modeling practice, which can contribute to successful PAT applications for increased process understanding, operation and control purposes.

Application of a mechanistic model as a tool for on‐line monitoring of pilot scale filamentous fungal fermentation processes—The importance of evaporation effects

“…With this understanding of the expected variation in inlet conditions, it is possible to simulate the range of expected evaporation rates over a year period. Latin Hypercube Sampling (Helton and Davis, 2003) is used to simulate 250 data points which represent this year of data, taking into consideration the correlation in input variables using the method of Iman and Conover (Iman and Conover, 1982;Sin et al, 2009). These 250 sample points are then simulated by Monte Carlo simulations in order to show the variation in evaporation rate, for a fixed set of processing conditions, as shown in Figure 9.…”

Application of modeling and simulation tools for the evaluation of biocatalytic processes: A future perspective

“…[33][34][35] Different sensitivity analysis methods are available, each with their own strengths and weaknesses. 36,37 For example, local methods such as differential analysis provide in-depth insight into the sensitivity function of a parameter. However, the results are conditional to the point in parameter-space where the analysis is performed, and therefore, the results cannot be extrapolated.…”

“…For further discussion, the reader is referred to specific literature in the field. 33,36,37 That said, one observes that the models developed so far within biocatalysis have only partially been exposed to formal statistical analysis. For example, among the models listed in Table 1, rather few studies have performed a check on the resulting quality of parameter estimates.…”

“…Needless to say this requires a formal treatment using a statistical method such as linear error propagation or a Monte-Carlo procedure. 36,37,39 This uncertainty issue becomes particularly important when the model is to be used to support engineering decisions, e.g., extrapolating models outside the conditions of the experimental data used for parameter identification. Process design is an example to this end (see below for an illustration of identifiability vs. uncertainty).…”

“…In fact, a closer look at the confidence intervals shows, however, that the true values (see above) are in fact included within the reported confidence interval, which is rather large (around 100% of the estimated value for K m ). Using the linear error method for uncertainty analysis 37 (the method used for this exercise is provided as Supporting Information), the parameter estimation errors were propagated to quantify the error in the model predictions-shown as dashed lines on Figure 4-right. These confidence intervals indicate the degree of uncertainty surrounding each prediction.…”