A Pipeline Blockage Identification Model Learning from Unbalanced Datasets Based on Random Forest

Mingyue, Fang; Feng, Zao; Xiao-dong, Wang; Ma, Jun

doi:10.1109/ccdc52312.2021.9602663

Cited by 5 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Random Forestmentioning

confidence: 99%

Optimization of land subsidence prediction features based on machine learning and SHAP value with Sentinel-1 InSAR Data

Su,

Xu,

Xion

et al. 2024

Preprint

View full text Add to dashboard Cite

Land subsidence has always been a concern of geoscience, and exploring the factors affecting land subsidence to predict future land subsidence is essential research. However, current research rarely has a scientific and unified feature screening process for land subsidence features. This study applies neural networks and SHAP values to land subsidence prediction. We used SHAP values instead of the traditional random forest (RF) to quantify land subsidence features and neural networks to predict the areas where land subsidence is likely to occur in the cities of Chongqing and Chengdu, encompassing the majority of the possible land subsidence scenarios in the future. The results show that the prediction of land subsidence using neural networks improves the model accuracy by 16% compared to the traditional method. After input features optimization, the performance improves by nearly 22%. We found that the feature optimization method based on SHAP values proposed in this study is more helpful for land subsidence prediction, and the factors affecting land subsidence derived from data analysis with complex terrain are also consistent with the results of previous studies. This feature optimization method can contribute to the input variable selection process for the land subsidence prediction model, improve accuracy, and provide solid theoretical support for preventing urban land subsidence.

show abstract

Section: Random Forestmentioning

confidence: 99%

Optimization of land subsidence prediction features based on machine learning and SHAP value with Sentinel-1 InSAR Data

Su,

Xu,

Xion

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The GAN-based model for generating samples is used to expand the data of the two minority categories of line hanging error and power meter multiplier error, so that the number of samples among each category is balanced. It will solve the problem of inaccurate classification of imbalanced datasets on traditional classifier models [31][32][33].…”

Section: Generating Samples Of Abnormal Line-transformer Relationship...mentioning

confidence: 99%

Intelligent Identification of the Line-Transformer Relationship in Distribution Networks Based on GAN Processing Unbalanced Data

et al. 2022

View full text Add to dashboard Cite

The wrong line-transformer relationship is one of the main reasons that leads to the failure of the line loss assessment of the distribution network with voltage levels of 10 kV and below. The traditional manual method to verify the line-transformer relationship is time-consuming, labor-intensive and inefficient. At the same time, due to the small sample size of the data with abnormal line-transformer relationship, the unbalanced sample data reduces the accuracy of the artificial intelligence algorithm. To this end, this paper proposes an intelligent identification method for distribution network line-transformer relationship based on Generative Adversarial Networks (GAN) processing unbalanced data. Firstly, perform data preprocessing and feature extraction based on the input power of the distribution line and the power consumption of each distribution transformer; then, build a GAN-based model for expanding the data of only a small number of abnormal line-transformer relationship samples, so as to solve the problem of unbalanced sample data distribution; and finally, establish a support vector machine (SVM) to realize the classification of the line-transformer relationship. The results of the example simulation show that, compared with the traditional Synthetic Minority Oversampling Technique (SMOTE) for processing unbalanced data, the classification effect of the proposed GAN-based data augmentation method has been significantly improved. In addition, the recall rate of the three types of the line-transformer relationship (line hanging error, magnification error and normal) under the line-transformer relationship identification method proposed in this paper is more than 92%, which proves the effectiveness and feasibility of the method.

show abstract

“…Its core idea is to construct multiple decision trees by randomly selecting samples and feature subsets and combining their predictions to reduce the variance of the model and improve the generalization ability. The model can handle unbalanced data(Mingyue et al 2021), balance the error by itself, and is insensitive to missing values and outliers, but overfitting can occur when the data is noisy.3.1.2 BP Neural NetworkBPNN (Backpropagation Neural Network) is a common artificial neural network model for solving classification and regression problem(Ang et al, 2022). It consists of an input layer, a hidden layer and an output layer, and is trained and weighted by a backpropagation algorithm.…”

mentioning

confidence: 99%

Enhancement of land subsidence prediction capabilities using machine learning and SHAP value analysis with Sentinel-1 InSAR Data

Su,

Xu,

Xion

et al. 2024

Preprint

View full text Add to dashboard Cite

Land subsidence has been a significant focus of geoscience studies, and researching the factors contributing to it and predicting future incidents is essential. However, current research requires a systematic and coherent strategy to identify symptoms of land sinking with scientific rigor. This study employs neural networks and SHAP values to forecast land subsidence. We utilized SHAP values as a substitute for the usual random forest (RF) technique to evaluate the attributes of land subsidence. In addition, we employed neural networks to predict the prospective areas where land subsidence may occur in Chongqing and Chengdu in the future, considering a diverse set of conceivable situations. The results indicate that using neural networks for land subsidence prediction improves the model's accuracy by 16% compared to the traditional approach. The performance is enhanced by approximately 22% after optimizing the input characteristics. The feature optimization strategy proposed in this study, which relies on SHAP values, is particularly advantageous for predicting land subsidence. Moreover, the determinants of land sinking, ascertained by data analysis considering intricate topography, correspond with the conclusions of previous research. Utilizing this feature optimization method can improve the selection of input variables for the land subsidence prediction model, increasing accuracy and providing a solid theoretical foundation for preventing urban land subsidence.

show abstract

A Pipeline Blockage Identification Model Learning from Unbalanced Datasets Based on Random Forest

Cited by 5 publications

References 4 publications

Optimization of land subsidence prediction features based on machine learning and SHAP value with Sentinel-1 InSAR Data

Optimization of land subsidence prediction features based on machine learning and SHAP value with Sentinel-1 InSAR Data

Intelligent Identification of the Line-Transformer Relationship in Distribution Networks Based on GAN Processing Unbalanced Data

Enhancement of land subsidence prediction capabilities using machine learning and SHAP value analysis with Sentinel-1 InSAR Data

Contact Info

Product

Resources

About