Automated Real-Time Process Data Analysis Using Online Wavelet Transforms: Fault Detection, Data Cleansing and Compression

Abstract

Improvements in sampling methods and sensor technology have resulted in the creation of vast amounts of process data that can be analyzed to improve the operation of chemical facilities. Automated real-time data analysis methods sort through incoming data immediately, aiding in real-time decision making and preparing the data for further analysis. In this study, the application of online wavelet transforms (OWT) to several aspects of real-time data analysis was considered, namely, fault detection, data cleansing and data compression. In the first application, the OWT was used to build a model and detect faults in an uncorrelated tag (variable) which cannot use traditional data driven modelling that relies on underlying relationships between variables. The OWT was successfully able to detect and identify various common sensor faults including missing and frozen values, linear drift and spikes. Recursive least squares (RLS) was then used to predict replacement values, and was a significant improvement on current standards, particularly with highly variable data. The new algorithm (OWT and RLS together) also updates statistics with each new value, so decision making is accurate but does not require repeated calculations on the whole data set. This application directly affects real-time decision making as detected faults can be addressed immediately, and any predictive models built from the data will be representative of real operation. The OWT is also used to create a novel real-time data compression algorithm to economically store large quantities of data. The industry standard swinging door algorithm requires extensive manual input, in addition to other drawbacks. The new method is an online filter that is completely automated, with calculations unique to each tag. This new algorithm showed much higher compression ratios with less loss of information, and was clearly superior to the old method.