Two-step long short-term memory method for identifying construction activities through positional and attentional cues

Cai, Jiannan; Zhang, Yuxi; Cai, Hubo

doi:10.1016/j.autcon.2019.102886

Cited by 43 publications

(24 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been found that human attention is cognitively driven to differentially allocate between visual areas and shift over time. The lack of such a mechanism means that computers cannot learn and utilize human attentional cues, which is one of the important factors limiting the development of computer vision technology for CHR [31]. On the other hand, computer vision for CHR that lays the foundation of the above cognitive models usually judges only based on the existing observable scene components, and cannot combine the association and inference of possible future states to further infer whether the existing scene has the possibility of hazard occurrence.…”

Section: Literature Review 21 the Development Of Computer Vision In Chr: Grounded In Human Cognitive Mechanismsmentioning

confidence: 99%

Temporal Visual Patterns of Construction Hazard Recognition Strategies

Rui

Wang

2021

IJERPH

View full text Add to dashboard Cite

Visual cognitive strategies in construction hazard recognition (CHR) signifies prominent value for the development of CHR computer vision techniques and safety training. Nonetheless, most studies are based on either sparse fixations or cross-sectional (accumulative) statistics, which lack consideration of temporality and yielding limited visual pattern information. This research aims to investigate the temporal visual search patterns for CHR and the cognitive strategies they imply. An experimental study was designed to simulate CHR and document participants’ visual behavior. Temporal qualitative comparative analysis (TQCA) was applied to analyze the CHR visual sequences. The results were triangulated based on post-event interviews and show that: (1) In the potential electrical contact hazards, the intersection of the energy-releasing source and wire that reflected their interaction is the cognitively driven visual area that participants tend to prioritize; (2) in the PPE-related hazards, two different visual strategies, i.e., “scene-related” and “norm-guided”, can usually be generalized according to the participants’ visual cognitive logic, corresponding to the bottom-up (experience oriented) and top-down (safety knowledge oriented) cognitive models. This paper extended recognition-by-components (RBC) model and gestalt model as well as providing feasible practical guide for safety trainings and theoretical foundations of computer vision techniques for CHR.

show abstract

Section: Literature Review 21 the Development Of Computer Vision In Chr: Grounded In Human Cognitive Mechanismsmentioning

confidence: 99%

Temporal Visual Patterns of Construction Hazard Recognition Strategies

Rui

Wang

2021

IJERPH

View full text Add to dashboard Cite

show abstract

“…This method was further enhanced to identify working groups, i.e., which objects are working together for a certain operation in a later study (Luo et al, 2020c). Cai et al (2019) also designed a two-step LSTM model to achieve working group identification and activity classification. Interaction analysis has been also studied for safety monitoring purposes.…”

Section: Object-to-object Interaction Analysismentioning

confidence: 99%

“…Fortunately, recent studies shown that the integration of various features can represent the object-to-object interactions more effectively than using only spatial features. Cai et al (2019) exploited both spatial (e.g., locations, directions) and attentional cues (e.g., worker's head pose) when identifying interacting groups and classifying their activity types. Luo et al (2020c) also improved the performance of group identification with the additional considerations of deep visual features (extracted by CNN).…”

Section: Object-to-object Interaction Analysismentioning

confidence: 99%

“…Luo et al (2020c) also improved the performance of group identification with the additional considerations of deep visual features (extracted by CNN). Further, as indicated by Cai et al (2019), temporal features can also play a key role in detecting visual relationships of construction objects; this is because the longer observations are available, the better the dynamic interactions among objects can be understood. Moreover, it would be also effective to incorporate operational contexts, which means that a particular type of construction object tends to have a certain type of relationship with a specific type of object.…”

Section: Object-to-object Interaction Analysismentioning

confidence: 99%

See 1 more Smart Citation

Visual Analytics for Operation-Level Construction Monitoring and Documentation: State-of-the-Art Technologies, Research Challenges, and Future Directions

Kim

2020

Front. Built Environ.

View full text Add to dashboard Cite

Operation-level vision-based monitoring and documentation has drawn significant attention from construction practitioners and researchers. To automate the operation-level monitoring of construction and built environments, there have been much effort to develop computer vision technologies. Despite their encouraging findings, it remains a major challenge to exploit technologies in real construction projects, implying that there are knowledge gaps in practice and theory. To fill such knowledge gaps, this study thoroughly reviews 119 papers on operation-level vision-based construction monitoring, published in mainstream construction informatics journals. Existing research papers can be categorized into three sequential technologies: (1) camera placement for operation-level construction monitoring, (2) single-camera-based construction monitoring and documentation, and (3) multi-camera-based onsite information integration and construction monitoring. For each technology, state-of-the-art algorithms, open challenges, and future directions are discussed.

show abstract

“…The first is construction individual-related issues, which receive more attention by recognizing workers’ activities and usage of personal protective equipment (PPE). In activities recognition: Luo and Li et al [ 40 , 41 , 42 ] used various computer vision algorithms for construction worker activity recognition; Cai et al [ 43 ] and Liu et al [ 44 ] also carried out computer vision-based approaches for construction activities’ recognition. Cai used a two-step LSTM (long short-term memory network), while Liu combined computer vision and natural language processing methods; Han et al [ 45 ], Yu et al [ 46 ] and Yang et al [ 47 ] extracted workers’ joint coordinate to recognize the activity and judge the safety status.…”

Section: Introductionmentioning

confidence: 99%

Evaluating the Work Productivity of Assembling Reinforcement through the Objects Detected by Deep Learning

Zhao

Zhou

et al. 2021

Sensors

View full text Add to dashboard Cite

With the rapid development of deep learning, computer vision has assisted in solving a variety of problems in engineering construction. However, very few computer vision-based approaches have been proposed on work productivity’s evaluation. Therefore, taking a super high-rise project as a research case, using the detected object information obtained by a deep learning algorithm, a computer vision-based method for evaluating the productivity of assembling reinforcement is proposed. Firstly, a detector that can accurately distinguish various entities related to assembling reinforcement based on CenterNet is established. DLA34 is selected as the backbone. The mAP reaches 0.9682, and the speed of detecting a single image can be as low as 0.076 s. Secondly, the trained detector is used to detect the video frames, and images with detected boxes and documents with coordinates can be obtained. The position relationship between the detected work objects and detected workers is used to determine how many workers (N) have participated in the task. The time (T) to perform the process can be obtained from the change of coordinates of the work object. Finally, the productivity is evaluated according to N and T. The authors use four actual construction videos for validation, and the results show that the productivity evaluation is generally consistent with the actual conditions. The contribution of this research to construction management is twofold: On the one hand, without affecting the normal behavior of workers, a connection between construction individuals and work object is established, and the work productivity evaluation is realized. On the other hand, the proposed method has a positive effect on improving the efficiency of construction management.

show abstract

Two-step long short-term memory method for identifying construction activities through positional and attentional cues

Cited by 43 publications

References 37 publications

Temporal Visual Patterns of Construction Hazard Recognition Strategies

Temporal Visual Patterns of Construction Hazard Recognition Strategies

Visual Analytics for Operation-Level Construction Monitoring and Documentation: State-of-the-Art Technologies, Research Challenges, and Future Directions

Evaluating the Work Productivity of Assembling Reinforcement through the Objects Detected by Deep Learning

Contact Info

Product

Resources

About