Vision‐based displacement and joint rotation tracking of frame structure using feature mix with single consumer‐grade camera

Abstract

A monocular vision-based response tracking method is proposed for simultaneously identifying translational floor displacement and rotational joint deformation of frame structures at multiple locations. The framework incorporates subpixel-level feature matching and tracking and is designed by introducing the feature-mix concept and feature quality evaluation for the initialization and postprocessing stage, respectively. The necessity of monitoring frame joint rotation is presented with the damage observations. A large-scale reinforced concrete frame structure is employed to investigate the proposed response estimation method through comprehensive shaking table tests by using a consumer-grade camera. The initial feature population, time-history agreement, overall estimation accuracy, survival feature ratio, and measurement uncertainty are further analyzed. In addition, the potential values and advantages of the proposed response tracking method for dynamic measurement and seismic evaluation are discussed illustratively.

Vision‐based displacement measurement enhanced by super‐resolution using generative adversarial networks

“…The Shi‐Tomasi corner detector 54 is used to identify the corners in a given ROI and then the pixel distances between the corresponding corners are calculated. The Shi‐Tomasi corner detector determines whether a pixel is a corner based on the local intensity gradient of the image and is widely used in the determination of geometric key points for vision‐based structural displacement measurement tasks 15,18,19 . After the integer corner coordinates are obtained using the Shi‐Tomasi corner detector, the commonly used sub‐pixel algorithm 55 can be used to obtain the corner coordinates with sub‐pixel precision.…”

“…The SIFT algorithm is adopted to detect the feature points in the ROIs before and after the displacement, and the feature points are matched according to the Euclidean distance between descriptors, and the target pixel distance is then calculated 7 . To increase the robustness of feature point matching, the RANSAC algorithm is used to filter the outliers 16,19 . The SIFT algorithm involves key point matching based on the feature descriptor and is thus used in the experiment to investigate the effect of texture enrichment via SR on displacement measurement.…”

“…The Shi-Tomasi corner detector determines whether a pixel is a corner based on the local intensity gradient of the image and is widely used in the determination of geometric key points for vision-based structural displacement measurement tasks. 15,18,19 After the integer corner coordinates are obtained using the Shi-Tomasi corner detector, the commonly used sub-pixel algorithm 55 can be used to obtain the corner coordinates with sub-pixel precision. This method is used in the experiment to validate the robustness of the proposed method to the local intensity gradient.…”

“…7 To increase the robustness of feature point matching, the RANSAC algorithm is used to filter the outliers. 16,19 The SIFT algorithm involves key point matching based on the feature descriptor and is thus used in the experiment to investigate the effect of texture enrichment via SR on displacement measurement.…”

“…Meanwhile, modern cities are equipped with a large number of surveillance video cameras that can record images and videos of urban buildings and infrastructures without additional installation efforts, which has the potential to support SHM applications. 5 In terms of the structural displacement measurement problem, there are three main types of commonly-used methods: (1) template matching: this method searches for the most similar patch in the target with a predefined template based on certain correlation metrics and then calculates the structural displacement [6][7][8][9] ; (2) key point matching: this method detects sparse points with significant and stable features in the images and computes the feature description operators to match the set of points before and after structural displacement or directly defines key points based on the geometric features of the targets for specific problems and tracks the position change of the points [10][11][12][13][14][15][16][17][18][19] ; (3) optical flow tracking: optical flow is a method used to describe the motion of pixels in sequential images, where the motion of certain pixels or the full scene is calculated based on differential or phase equations. [20][21][22][23][24] Other research has developed full-field measurement techniques for specific problems, utilizing the advantage of the vision-based system.…”