A comparative study on the most effective machine learning model for blast loading prediction: From GBDT to Transformer

Li, Qilin; Wang, Yang; Shao, Yanda; Li, Ling; Hao, Hong

doi:10.1016/j.engstruct.2022.115310

Cited by 25 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LightGBM [29]: LightGBM is the second mainstream Boosting algorithm implemented under the GBDT [30] framework since XGBoost [31]. LightGBM adopts a decision tree algorithm based on histograms, using GOSS single edge gradient sampling algorithm and mutually exclusive feature bundling algorithm (EOF) to improve model running speed, while supporting parallel training of class features and data, providing better parallel performance.…”

Section: B Comparison Methodsmentioning

confidence: 99%

A Predictive Adaptive Learning Method for Multivariable Time Series With Mooney Viscosity Prediction as an Application Case

Zhu,

Ying,

Zhou

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Time series prediction involves static and dynamic features. Extraneous input information hampers model performance, and the statistical attributes of time series change over time, affecting distribution. Targeted processing of input data for feature and distribution dynamics is vital. This article introduces AdaDynaTrans, an attention-based optimization model. It dynamically combines features from static covariate extraction and temporal variable evolution modules, learning time relationships at various scales. Using self-designed dynamic residual and feature selection units, it suppresses irrelevant information.AdaDynaTrans tackles time covariate shift through temporal segmentation and segmented training mechanisms. Additionally, a relative position-optimized transformer is employed to capture local dependencies within temporal data, thereby achieving exceptional performance within real-world scenarios. Through comprehensive evaluations on both public datasets and industrial scenarios, the considerable efficacy of AdaDynaTrans is demonstrated. Ablation experiments are conducted to analyze the effectiveness of each constituent element, and the interpretability of the model is showcased using a case study involving Mooney viscosity prediction in rubber production. This research contributes substantively to the field by presenting a model that not only achieves high performance but also offers insights into the temporal dynamics of complex industrial processes.

show abstract

Section: B Comparison Methodsmentioning

confidence: 99%

A Predictive Adaptive Learning Method for Multivariable Time Series With Mooney Viscosity Prediction as an Application Case

Zhu,

Ying,

Zhou

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The transformer model's deep learning architecture uses attention processes to discover connections between input features. It comprises numerous feedforward and self-attention neural network layers, allowing the model to recognize intricate correlations between variables and produce precise predictions [59]. FTT has several advantages over more conventional methods.…”

Section: Classifiersmentioning

confidence: 99%

Performance Comparison of Deep Learning (DL)-Based Tabular Models for Building Mapping Using High-Resolution Red, Green, and Blue Imagery and the Geographic Object-Based Image Analysis Framework

Hossain,

Chen

2024

Remote Sensing

View full text Add to dashboard Cite

Identifying urban buildings in high-resolution RGB images presents challenges, mainly due to the absence of near-infrared bands in UAVs and Google Earth imagery and the diversity in building attributes. Deep learning (DL) methods, especially Convolutional Neural Networks (CNNs), are widely used for building extraction but are primarily pixel-based. Geographic Object-Based Image Analysis (GEOBIA) has emerged as an essential approach for high-resolution imagery. However, integrating GEOBIA with DL models presents challenges, including adapting DL models for irregular-shaped segments and effectively merging DL outputs with object-based features. Recent developments include tabular DL models that align well with GEOBIA. GEOBIA stores various features for image segments in a tabular format, yet the effectiveness of these tabular DL models for building extraction still needs to be explored. It also needs to clarify which features are crucial for distinguishing buildings from other land-cover types. Typically, GEOBIA employs shallow learning (SL) classifiers. Thus, this study evaluates SL and tabular DL classifiers for their ability to differentiate buildings from non-building features. Furthermore, these classifiers are assessed for their capacity to handle roof heterogeneity caused by sun exposure and roof materials. This study concludes that some SL classifiers perform similarly to their DL counterparts, and it identifies critical features for building extraction.

show abstract

Sparse attention is all you need for pre-training on tabular data

Isomura,

Shimizu,

Goto

2024

Neural Comput & Applic

View full text Add to dashboard Cite

In the world of data-driven decision-making, tabular data reigns supreme as the most prevalent and crucial format, especially in business contexts. However, data scarcity remains a recurring challenge. In this context, transfer learning has emerged as a potent solution. This study explores the untapped potential of transfer learning in the realm of tabular data analysis, with a focus on leveraging deep learning models—especially the Transformer model—that have garnered significant recognition. Our research investigates the intricacies of tabular data and illuminates the shortcomings of conventional attention mechanisms in the Transformer model when applied to such structured datasets. This highlights the pressing requirement need for specialized solutions tailored to tabular data. We introduce an innovative transfer learning method based on series of thoroughly designed experiments across diverse business domains. This approach harnesses Transformer-based models enhanced with optimized sparse attention mechanisms, offering a groundbreaking solution for tabular data analysis. Our findings reveal the remarkable effectiveness of enhancing the attention mechanism within the Transformer in transfer learning. Specifically, pre-training with sparse attention proves increasingly powerful as data volumes increase, resulting in superior performance on large datasets. Conversely, fine-tuning with full attention becomes more impactful when data availability decreases in downstream tasks, ensuring adaptability in situations with limited data. The empirical results presented in this study provide compelling evidence of the revolutionary potential of our approach. Our optimized sparse attention model emerges as a powerful tool for researchers and practitioners seeking highly effective solutions for tabular data tasks. As tabular data remain the backbone of business operations, our study promises to revolutionize data analysis in critical domains. This work bridges the gap between limited data availability and the requirement for effective analysis in business settings, marking a significant step forward in the field of tabular data analysis.

show abstract

A comparative study on the most effective machine learning model for blast loading prediction: From GBDT to Transformer

Cited by 25 publications

References 22 publications

A Predictive Adaptive Learning Method for Multivariable Time Series With Mooney Viscosity Prediction as an Application Case

A Predictive Adaptive Learning Method for Multivariable Time Series With Mooney Viscosity Prediction as an Application Case

Performance Comparison of Deep Learning (DL)-Based Tabular Models for Building Mapping Using High-Resolution Red, Green, and Blue Imagery and the Geographic Object-Based Image Analysis Framework

Sparse attention is all you need for pre-training on tabular data

Contact Info

Product

Resources

About