Inter-rater reliability of cyclic and non-cyclic task assessment using the hand activity level in appliance manufacturing

Abstract

This study evaluated the inter-rater reliability of the American Conference of Governmental Industrial Hygienists (ACGIH®) hand activity level (HAL), an observational ergonomic assessment method used to estimate physical exposure to repetitive exertions during task performance. Video recordings of 858 cyclic and non-cyclic appliance manufacturing tasks were assessed by sixteen pairs of raters using the HAL visual-analog scale. A weighted Pearson Product Moment-Correlation Coefficient was used to evaluate the agreement between the HAL scores recorded by each rater pair, and the mean weighted correlation coefficients for cyclic and non-cyclic tasks were calculated. Results indicated that the HAL is a reliable exposure assessment method for cyclic (r̄-barw = 0.69) and non-cyclic work tasks (r̄-barw = 0.68). When the two reliability scores were compared using a two-sample Student's t-test, no significant difference in reliability (p = 0.63) between these work task categories was found. This study demonstrated that the HAL may be a useful measure of exposure to repetitive exertions during cyclic and non-cyclic tasks. Relevance to industry Exposure to hazardous levels of repetitive action during non-cyclic task completion has traditionally been difficult to assess using simple observational techniques. The present study suggests that ergonomists could use the HAL to reliably and easily evaluate exposures associated with some non-cyclic work tasks.

Reliability Analysis of Observation-Based Exposure Assessment Tools for the Upper Extremities: A Systematic Review

“…The selection of raters was also a source of variation within the articles reviewed. Many studies included students [ 16 , 26 , 29 , 36 , 37 , 38 , 39 , 40 ] and university faculty [ 16 , 36 , 40 ] as their raters. Most raters had prior knowledge and experience in physiotherapy and/or ergonomics.…”

“…Video recordings provide a consistent view of the process and ensure that all raters have the same information. As shown in the study by Paulsen et al [ 40 ], a downside is that sometimes the body part to be observed is not clearly visible. Trask et al [ 63 ] stated, “Observers rank partly visible postures on video frames differently [between raters] than fully visible postures…partly visible data, especially when observers disagree as to the level of visibility, introduces more between observer variability when compared to fully visible data”.…”

“…The challenge of ensuring that observations remain consistent between all raters arises with any observational exposure assessment tool [ 40 ] and in test–retest studies when the studied items change over time [ 24 , 61 , 62 ]. Video recordings provide a consistent view of the process and ensure that all raters have the same information.…”

Development of an educational protocol for ergonomic risk assessment of working postures to enhance the competence of occupational health nurses

“…Secondly, we reviewed the strategies for collecting data using a video recorder to improve the quality of the data. We reviewed the technical procedures for (1) using two side‐angled cameras, including the duration of the recording (Bao et al, 2009); (2) using a tripod to capture the whole body (Paulsen et al, 2014) and specific body parts (National Institute of Occupational Health and Safety, 2014); and (3) documenting the postures, activities, tools, and handling (Hajaghazadeh et al, 2019). We found these to be effective data collection strategies.…”

A frequency–duty cycle equation for the ACGIH hand activity level

“…Ebersole and Armstrong (2006) reported that inter-rater reliability for repetition was high with an interclass correlation coefficient value of 0.71 prior to discussion and 0.87 after discussion. Paulsen et al (2014) recently reported that HAL inter-rater reliability was a reliable exposure assessment method for 858 cyclic ( r̄ -bar w = 0.69) and non-cyclic work tasks ( r̄ -bar w = 0.68).…”