Sound-based multiple-equipment activity recognition using convolutional neural networks

Sherafat, Behnam; Rashidi, Abbas; Asgari, Sadegh

doi:10.1016/j.autcon.2021.104104

Cited by 31 publications

(16 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our study successfully achieved accuracies of up to 97% in the construction equipment and activity classifications. The CNN (Sherafat et al 2022) excelled in equipment classification but was less adept at handling activities, raising questions about its adaptability to complex sound data. Our analysis yielded two essential observations.…”

Section: Epochs Vs Activity Accuracymentioning

confidence: 99%

Acoustic-based Multitask Construction Equipment and Activity Recognition Using Customized ResNet-18

Ahammed,

Kim,

Song

et al. 2024

Preprint

View full text Add to dashboard Cite

This study presents an acoustic-based multitask technique for distinguishing construction equipment types and their associated activities using a customized ResNet-18 deep learning (DL) model. Construction equipment activities on sites require efficient recognition during real-time operations to ensure maximum productivity and safety. While other studies examine typical recognition, particularly visual or sensor-based methods, this study investigates the potential of equipment-generated audio signals as a feasible option for activity classification. Leveraging the acoustic signal and advanced DL techniques, we collected and preprocessed the audio signals from construction equipment and trained them on the customized pretrained ResNet-18 model to accurately recognize equipment and its corresponding activities. The proposed method shows superior accuracy and good performance in equipment and activity identification. Our findings demonstrate the capability and efficiency of the developed approach in accurately classifying equipment and activity types, achieving classification accuracies of up to 97%. Compared to other models, our method is more computationally efficient. The results suggest that the proposed method holds promise for real-time recognition of heavy equipment and associated activities. Its capability can reduce construction risks, enhance productivity, and improve decision-making at construction sites.

show abstract

Section: Epochs Vs Activity Accuracymentioning

confidence: 99%

Acoustic-based Multitask Construction Equipment and Activity Recognition Using Customized ResNet-18

Ahammed,

Kim,

Song

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Scarpiniti et al demonstrated up to 98% accuracy in categorizing more than 10 different types of construction equipment and device using a DBN [22]. Sherafat et al developed CNN models for recognizing multiple-equipment activities [23,40]. They used a two-level multi-label sound classification scheme that enables concurrent detection of the device kind and their associated activities.…”

Section: Review and Knowledge Gapmentioning

confidence: 99%

“…The great majority of the research performed has focused on the identification of distinct single construction activities, based on the premise that only one sort of sound is active at any given moment. Recent study has investigated situations with mixed construction sounds, but this work is still susceptible to certain limitations, such as the assumption that two kinds of construction noises always occur concurrently [23]. Construction sites are polyphonic settings with noise disruptions, since many construction workers or construction machines often operate concurrently [24].…”

Section: Introductionmentioning

confidence: 99%

Sound-Based Construction Activity Monitoring with Deep Learning

Xiong

Xu²,

Chen³

et al. 2022

Buildings

View full text Add to dashboard Cite

Automated construction monitoring assists site managers in managing safety, schedule, and productivity effectively. Existing research focuses on identifying construction sounds to determine the type of construction activity. However, there are two major limitations: the inability to handle a mixed sound environment in which multiple construction activity sounds occur simultaneously, and the inability to precisely locate the start and end times of each individual construction activity. This research aims to fill this gap through developing an innovative deep learning-based method. The proposed model combines the benefits of Convolutional Neural Network (CNN) for extracting features and Recurrent Neural Network (RNN) for leveraging contextual information to handle construction environments with polyphony and noise. In addition, the dual threshold output permits exact identification of the start and finish timings of individual construction activities. Before training and testing with construction sounds collected from a modular construction factory, the model has been pre-trained with publicly available general sound event data. All of the innovative designs have been confirmed by an ablation study, and two extended experiments were also performed to verify the versatility of the present model in additional construction environments or activities. This model has great potential to be used for autonomous monitoring of construction activities.

show abstract

“…Since construction equipment produces a distinct sound pattern, the auditory signal recorded on construction sites can be used to provide additional information on construction equipment actions. Several studies have already shown the effectiveness of utilizing auditory signals in classifying construction equipment activities by proposing computer audition models that extract semantic features from raw auditory signals (Akbal et al., 2022; Cheng et al., 2017; Scarpiniti et al., 2021; Sherafat et al., 2022). However, for action detection of construction equipment, additional improvements are needed to use auditory signals since computer audition models could not estimate the location of construction equipment.…”

Section: Introductionmentioning

confidence: 99%

Visual–auditory learning network for construction equipment action detection

Jung

Jeoung

Lee

et al. 2023

Computer aided Civil Eng

View full text Add to dashboard Cite

Action detection of construction equipment is critical for tracking project performance, facilitating construction automation, and fostering construction efficiency in terms of construction site monitoring. Particularly, the auditory signal can provide additional information on computer vision‐based action detection of various types of construction equipment. Therefore, this study aims to develop a visual–auditory learning network model for the action detection of construction equipment based on two modalities (i.e., vision and audition). To this end, both visual and auditory features are extracted from the multi‐modal feature extractor. In addition, the multi‐head attention and detection module is designed to conduct the localization and classification tasks in separate heads in which different attention mechanisms for each task are applied. Particularly, the content‐based attention mechanism and the dot‐product attention mechanism are, respectively, adopted for spatial attention in the localization head and channel attention in the classification head. The evaluation results show that the precision and recall of the proposed model can reach 86.92% and 84.00% with the adoption of the multi‐head attention and detection module, which has proven to improve overall detection performance by utilizing different correlations of visual and auditory features for localization and classification, respectively.

show abstract

Sound-based multiple-equipment activity recognition using convolutional neural networks

Cited by 31 publications

References 35 publications

Acoustic-based Multitask Construction Equipment and Activity Recognition Using Customized ResNet-18

Acoustic-based Multitask Construction Equipment and Activity Recognition Using Customized ResNet-18

Sound-Based Construction Activity Monitoring with Deep Learning

Visual–auditory learning network for construction equipment action detection

Contact Info

Product

Resources

About