Structural displacement monitoring using deep learning-based full field optical flow methods

Dong, Chuan‐Zhi; Celik, Ozan; Çatbaş, F. Necati; O’Brien, Eugene J.; Taylor, Su

doi:10.1080/15732479.2019.1650078

Cited by 129 publications

(67 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Combined with computer vision technology, the networks of deep learning represented by convolutional neural network (CNN) are developed to conduct the smart detection of the cracking, corrosion, and looseness for various structural components 42–47 . Moreover, scholars provide the comprehensive approach of deep learning to capture the nonlinear behavior of structure and evaluate the complex process of the evolution of structural performance 48–50 . Instead of the manual operation, the abnormal signal in the real‐time monitoring data can be intelligently detected and recovered by the network of deep learning 51,52 .…”

Section: Introductionmentioning

confidence: 99%

Digital modeling on the nonlinear mapping between multi‐source monitoring data of in‐service bridges

Zhao

Wei

et al. 2020

Struct Control Health Monit

View full text Add to dashboard Cite

Nonlinear mapping of the fuzzy relation exists between structural inputs and outputs, as well as between structural global and local response. It is difficult for the numerical simulation to introduce the nonlinear effects of the variability of loading effects and the uneven deterioration of structure. The big monitoring data make it feasible to mine these nonlinear effects, and the network of deep learning is a good tool to establish the nonlinear mapping model between the multi-source monitoring data. Based on the temperature, strain, and dynamic displacement data from the structural health monitoring (SHM) system of an in-service bridge, the deep learning regression network of long short-term memory (LSTM) is designed and trained, which is used for modeling the nonlinear mapping between the structural input-output and the structural global-local response. The digital model of the nonlinear mapping of the temperature to temperature-induced strain, dynamic displacement to vehicleinduced strain, and vehicle-induced strain to dynamic displacement is established by the deep learning of LSTM regression network. The established nonlinear mapping model can be used to recover the abnormal and missing data with multi-source data, and it digitally linked the structural input-output and the structural global-local response. Then, the utilization of SHM data analytics will be enhanced, and the digital modeling for the in-service state of the bridge structure will be fast implemented.

show abstract

Section: Introductionmentioning

confidence: 99%

Digital modeling on the nonlinear mapping between multi‐source monitoring data of in‐service bridges

Zhao

Wei

et al. 2020

Struct Control Health Monit

View full text Add to dashboard Cite

show abstract

“…Compared with the above-mentioned 6-DOF measurement instruments or methods, the vision method has the advantages of non-contact, high accuracy, and wide measurement range [ 23 , 24 , 25 , 26 ]. With the development of image processing and deep learning, visual measurement methods have strong environmental adaptability [ 27 , 28 ].…”

Section: Introductionmentioning

confidence: 99%

Multi-Camera-Based Universal Measurement Method for 6-DOF of Rigid Bodies in World Coordinate System

Zhao

Zhu

et al. 2020

Sensors

View full text Add to dashboard Cite

The measurement of six-degrees-of-freedom (6-DOF) of rigid bodies plays an important role in many industries, but it often requires the use of professional instruments and software, or has limitations on the shape of measured objects. In this paper, a 6-DOF measurement method based on multi-camera is proposed, which is accomplished using at least two ordinary cameras and is made available for most morphological rigid bodies. First, multi-camera calibration based on Zhang Zhengyou’s calibration method is introduced. In addition to the intrinsic and extrinsic parameters of cameras, the pose relationship between the camera coordinate system and the world coordinate system can also be obtained. Secondly, the 6-DOF calculation model of proposed method is gradually analyzed by the matrix analysis method. With the help of control points arranged on the rigid body, the 6-DOF of the rigid body can be calculated by the least square method. Finally, the Phantom 3D high-speed photogrammetry system (P3HPS) with an accuracy of 0.1 mm/m was used to evaluate this method. The experiment results show that the average error of the rotational degrees of freedom (DOF) measurement is less than 1.1 deg, and the average error of the movement DOF measurement is less than 0.007 m. In conclusion, the accuracy of the proposed method meets the requirements.

show abstract

“…Furthermore, the applied STC algorithm treats the whole region of the context equally, which weakens the effectiveness of the context information. As for the camera motion problem, which is getting increasing attention in vision measurement practices, a general solution is to compensate for camera motion-induced errors by subtracting the movement of reference points on the background [ 17 , 18 ]. The methods applied therein to identify the reference-point movement are critical to the measurement accuracy, especially when camera jitter, one particular kind of camera motion undergoing dramatic change and hence increasing challenges, is taken into account.…”

Section: Introductionmentioning

confidence: 99%

An Improved Vision Method for Robust Monitoring of Multi-Point Dynamic Displacements with Smartphones in an Interference Environment

Chen

Zhou

2020

Sensors

View full text Add to dashboard Cite

Current research on dynamic displacement measurement based on computer vision mostly requires professional high-speed cameras and an ideal shooting environment to ensure the performance and accuracy of the analysis. However, the high cost of the camera and strict requirements of sharp image contrast and stable environment during the shooting process limit the broad application of the technology. This paper proposes an improved vision method to implement multi-point dynamic displacement measurements with smartphones in an interference environment. A motion-enhanced spatio-temporal context (MSTC) algorithm is developed and applied together with the optical flow (OF) algorithm to realize a simultaneous tracking and dynamic displacement extraction of multiple points on a vibrating structure in the interference environment. Finally, a sine-sweep vibration experiment on a cantilever sphere model is presented to validate the feasibility of the proposed method in a wide-band frequency range. In the test, a smartphone was used to shoot the vibration process of the sine-sweep-excited sphere, and illumination change, fog interference, and camera jitter were artificially simulated to represent the interference environment. The results of the proposed method are compared to conventional displacement sensor data and current vision method results. It is demonstrated that, in an interference environment, (1) the OF method is prone to mismatch the feature points and leads to data deviated or lost; (2) the conventional STC method is sensitive to target selection and can effectively track those targets having a large proportion of pixels in the context with motion tendency similar to the target center; (3) the proposed MSTC method, however, can ease the sensitivity to target selection through in-depth processing of the information in the context and finally enhance the robustness of the target tracking. In addition, the MSTC method takes less than one second to track each target between adjacent frame images, implying a potential for online measurement.

show abstract

Structural displacement monitoring using deep learning-based full field optical flow methods

Cited by 129 publications

References 57 publications

Digital modeling on the nonlinear mapping between multi‐source monitoring data of in‐service bridges

Digital modeling on the nonlinear mapping between multi‐source monitoring data of in‐service bridges

Multi-Camera-Based Universal Measurement Method for 6-DOF of Rigid Bodies in World Coordinate System

An Improved Vision Method for Robust Monitoring of Multi-Point Dynamic Displacements with Smartphones in an Interference Environment

Contact Info

Product

Resources

About