Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Li, Ning; Tang, Jichuan; Li, Zhongxian; Gao, Xiuyu

doi:10.1002/stc.3035

Cited by 13 publications

(8 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Actuators layout and sensors for shaking table 15 . (A) Horizontal actuators, (B) vertical acutators, (C) sensors on each actuator, and (D) accelerometers on table.…”

Section: Underwater Shaking Table Array Facilitymentioning

confidence: 99%

Digital twins for the underwater shake table array facility

2022

Earthquake Engineering and Resilience

Self Cite

View full text Add to dashboard Cite

In this study, the three‐dimensional six‐degree‐of‐freedom underwater shaking table array facility for seismic simulation, which is built in Tianjin University, China, is taken as the research object. A digital twin of the shaking table is constructed. First, the components of the underwater shaking table array are introduced. And the simplified mechanical property of different components is illustrated. By using the system identification algorithm based on the measured dynamic responses, the basic model parameters, including the dynamics of the servo‐valve, the horizontal and vertical effective mass, the effective stiffness and damping coefficient of the actuators, the accumulators, and the friction of each actuator in the system are identified. The suitability of various experimental data sets used to identify the digital twin's model parameters is described. The prediction of various shaking table testing instances, as well as the comparison of digital twin and real system performance, are then presented. The digital twins shaking table system's performance and accuracy are examined. The shaking table's performance for various reproduction signals is compared to the digital twins model, which validates the suggested model's feasibility. Finally, the digital twins model is adopted and tried to tune the facility offline meanwhile the shaking table facility is configured iteratively. These findings highlight the applicability of the digital twins technique, which is the next research direction.

show abstract

“…Actuators layout and sensors for shaking table 15 . (A) Horizontal actuators, (B) vertical acutators, (C) sensors on each actuator, and (D) accelerometers on table.…”

Section: Underwater Shaking Table Array Facilitymentioning

confidence: 99%

Digital twins for the underwater shake table array facility

2022

Earthquake Engineering and Resilience

Self Cite

View full text Add to dashboard Cite

show abstract

“…Li et al applied the deep deterministic policy gradient algorithm combined with feedforward compensation for RTHS. 34 Tang et al employed a neural network-based offline method for RTHS to eliminate necessity for real-time data interaction between the experimental and numerical substructures. 35 On the other hand, Tsokanas et al employed feedforward neural networks for model order reduction, replacing the full-order numerical substructure to ensure reliable RTHS without a notable loss of accuracy.…”

Section: Introductionmentioning

confidence: 99%

“…Li et al. applied the deep deterministic policy gradient algorithm combined with feedforward compensation for RTHS 34 . Tang et al.…”

Section: Introductionmentioning

confidence: 99%

Development and verification of real‐time hybrid simulation with deep learning‐based nonlinear numerical substructure

Chen,

Hsu,

2024

Earthq Engng Struct Dyn

View full text Add to dashboard Cite

Real‐time hybrid simulation (RTHS) provides an effective approach for assessing structural responses under dynamic excitation. However, performing RTHS with a complex nonlinear numerical substructure is challenging, as computations must be completed within predefined time steps. In this study, a RTHS framework which contains a rate‐dependent experimental substructure and a nonlinear numerical substructure has been proposed and verified. An OpenSees model was constructed to simulate a three‐story, one‐bay steel building with viscous dampers at each floor and was used to generate the training dataset through a large number of nonlinear time‐history analyses. A Recursive Long Short–Term Memory (LSTM) neural network model was trained to predict the nonlinear structural responses using ground acceleration and time‐delayed damper force located at the first story. Hence, the Recursive‐LSTM model served as a surrogate model for the numerical substructure, implicitly incorporating delay compensation for the experimental substructure. After the training was completed, offline testing was performed to realize the stability of the RTHS framework. Then, online RTHS with a virtual damper taken as the experimental substructure was conducted to further confirm the feasibility and accuracy. Afterwards, a nonlinear rotary fluid viscous damper (RFVD) was fabricated as the actual experimental substructure, whose dynamic response was not considered in training the Recursive‐LSTM model. Finally, RTHS with the RFVD was completed successfully and stably, demonstrating the capability of a well‐trained Recursive‐LSTM model to serve as a nonlinear numerical substructure incorporating constant delay compensation for RTHS. The potential of the proposed RTHS framework is worth further studies in the future.

show abstract

“…Zhou and Li (2021) proposed a two-stage feedforward method considering error of identified model. Furthermore, combining the deep deterministic policy gradient algorithm in reinforcement learning (RL) and feedforward (FF) compensation, Li et al (2022) introduced a novel control method, which emphasizes the implementation of shaking table control and substructure RTHS.…”

Section: Introductionmentioning

confidence: 99%

Quantitative analysis of time-varying delay for real-time hybrid simulation

Xue

Zhou

2023

Journal of Vibration and Control

Self Cite

View full text Add to dashboard Cite

Real-time hybrid simulation is an effective method to study the seismic performance of large or complex structures. Time delay of the hydraulic servo system is unavoidable, which affects test accuracy and may cause instability. Previous studies shown that the reality of time-varying delay has a substantial negative effect on system stability than the assumption of constant delay. However, there is currently a lack of research for the actuating system with time-varying delay in the reality time domain. Therefore, a quantitative analysis method of time-varying delay is proposed in this study. The advantage is that only the command and the measured displacement signals are needed to obtain the time-varying delay in the time domain without any prior knowledge of the system. To overcome the influence of the amplitude error on the solution results, a processing method for the amplitude modulation of the measured displacement is proposed. To accurately search for the command corresponding to the measured signal, it is proposed to solve the time-varying delay by taking segment operation on the signal according to the maximum and minimum points. The efficiency of the proposed method is verified by numerical simulation, actuator displacement tracking test, and RTHS.

show abstract

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Cited by 13 publications

References 39 publications

Digital twins for the underwater shake table array facility

Digital twins for the underwater shake table array facility

Development and verification of real‐time hybrid simulation with deep learning‐based nonlinear numerical substructure

Quantitative analysis of time-varying delay for real-time hybrid simulation

Contact Info

Product

Resources

About