Reinforcement Learning for the Face Support Pressure of Tunnel Boring Machines

Soranzo, Enrico; Guardiani, Carlotta; Wu, Wei

doi:10.3390/geosciences13030082

Geosciences

2023

DOI: 10.3390/geosciences13030082

|View full text |Cite

Reinforcement Learning for the Face Support Pressure of Tunnel Boring Machines

Enrico Soranzo

Carlotta Guardiani

Wei Wu

Abstract: In tunnel excavation with boring machines, the tunnel face is supported to avoid collapse and minimise settlement. This article proposes the use of reinforcement learning, specifically the deep Q-network algorithm, to predict the face support pressure. The algorithm uses a neural network to make decisions based on the expected rewards of each action. The approach is tested both analytically and numerically. By using the soil properties ahead of the tunnel face and the overburden depth as the input, the algorit… Show more

Help me understand this report

View preprint versions

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

2024

Publication Types

Select...

Article5

Relationship

Self Cite0

Independent5

Authors

Journals

Cited by 8 publications

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

A systematic review and meta-analysis of artificial neural network, machine learning, deep learning, and ensemble learning approaches in field of geotechnical engineering

Yaghoubi,

Khamees

et al. 2024

Neural Comput & Applic

View full text Add to dashboard Cite

Artificial neural networks (ANN), machine learning (ML), deep learning (DL), and ensemble learning (EL) are four outstanding approaches that enable algorithms to extract information from data and make predictions or decisions autonomously without the need for direct instructions. ANN, ML, DL, and EL models have found extensive application in predicting geotechnical and geoenvironmental parameters. This research aims to provide a comprehensive assessment of the applications of ANN, ML, DL, and EL in addressing forecasting within the field related to geotechnical engineering, including soil mechanics, foundation engineering, rock mechanics, environmental geotechnics, and transportation geotechnics. Previous studies have not collectively examined all four algorithms—ANN, ML, DL, and EL—and have not explored their advantages and disadvantages in the field of geotechnical engineering. This research aims to categorize and address this gap in the existing literature systematically. An extensive dataset of relevant research studies was gathered from the Web of Science and subjected to an analysis based on their approach, primary focus and objectives, year of publication, geographical distribution, and results. Additionally, this study included a co-occurrence keyword analysis that covered ANN, ML, DL, and EL techniques, systematic reviews, geotechnical engineering, and review articles that the data, sourced from the Scopus database through the Elsevier Journal, were then visualized using VOS Viewer for further examination. The results demonstrated that ANN is widely utilized despite the proven potential of ML, DL, and EL methods in geotechnical engineering due to the need for real-world laboratory data that civil and geotechnical engineers often encounter. However, when it comes to predicting behavior in geotechnical scenarios, EL techniques outperform all three other methods. Additionally, the techniques discussed here assist geotechnical engineering in understanding the benefits and disadvantages of ANN, ML, DL, and EL within the geo techniques area. This understanding enables geotechnical practitioners to select the most suitable techniques for creating a certainty and resilient ecosystem.

show abstract

A systematic review and meta-analysis of artificial neural network, machine learning, deep learning, and ensemble learning approaches in field of geotechnical engineering

Yaghoubi,

Khamees

et al. 2024

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

Geological adaptive intelligent control of earth pressure balance shield machine based on deep reinforcement learning

Liu,

Zhang,

Shao

et al. 2024

Artif Intell Rev

View full text Add to dashboard Cite

Scientific and precise control of tunnelling parameters is of utmost importance during the construction of shield machines. Given the complexity of the working environment, manual operation is highly prone to causing safety accidents. Therefore, achieving intelligent control of the shield machine is crucial. Based on this, this paper proposes a geological adaptive intelligent control method of earth pressure balance shield machine using the Deep Deterministic Policy Gradient (DDPG) algorithm as the framework, with Actor-Critic as the basis. Firstly, DDPG agent is constructed to replace the screw conveyor control system as the main body of strategy implementation. Secondly, an environmental model is established by utilizing the mechanism model between the sealed cabin pressure and the screw conveyor speed. The real-time sealed cabin pressure, target pressure, and pressure error serve as the state space, while the screw conveyor speed is used as the action space. A combined reward function is set based on safety and accuracy. Finally, the Actor network interacts with the environment under the supervision of the reward function and Critic network. Successful training is achieved when the cumulative reward value is maximized, resulting in the output of optimal control strategy. In this paper, the method dynamically regulates the screw conveyor speed by interacting with the geological environment, to realize the precise control of the sealed cabin pressure and ensure the dynamic balance between sealed cabin pressure and excavation face pressure. The test results show that this method has a good control effect on the sealed cabin pressure under various geological conditions, and can complete 72 kinds of soil transition tasks. It has strong soil adaptability and can respond well to the dynamic changes of soil conditions. This approach enhances the intelligence of the shield machine, mitigating inaccuracies attributed to human operation, which provides a guarantee of safe shield machine operation, whilst exhibiting valuable engineering applications.

show abstract

Reinforcement learning-based optimizer to improve the steering of shield tunneling machine

Elbaz,

Shen,

Zhou

et al. 2023

Acta Geotech.

View full text Add to dashboard Cite

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Reinforcement Learning for the Face Support Pressure of Tunnel Boring Machines

Cited by 8 publications

References 70 publications

A systematic review and meta-analysis of artificial neural network, machine learning, deep learning, and ensemble learning approaches in field of geotechnical engineering

A systematic review and meta-analysis of artificial neural network, machine learning, deep learning, and ensemble learning approaches in field of geotechnical engineering

Geological adaptive intelligent control of earth pressure balance shield machine based on deep reinforcement learning

Reinforcement learning-based optimizer to improve the steering of shield tunneling machine

Contact Info

Product

Resources

About