Validity and Reliability of Firefighting Simulation Test Performance

Abstract

Objective To assess the validity and reliability of a firefighting simulation test (FFST).Methods Sixty-nine operational firefighters completed a best-effort FFST on one occasion and twenty-two participants completed a further FFST. All participants completed a maximal treadmill test to determine cardiorespiratory fitness (VO 2 max). Results Time to complete the FFST demonstrated a strong inverse relationship with VO 2 max (r = -0.73), although the prediction error was high. Reliability of the FFST was high (r = 0.84, p = 0.01), demonstrating a coefficient of variation of 4.5%. Conclusions The FFST demonstrated reasonable validity as a surrogate assessment of cardiorespiratory fitness for firefighting. The FFST also demonstrated good reliability. Given the apparent magnitude of the prediction error, the FFST would be best used as a training tool, rather than as a primary means of assessing cardiorespiratory fitness for firefighting.

Physical and Physiological Performance Determinants of a Firefighting Simulation Test

“…The FFST was previously validated for assessing occupational performance in UK firefighters and conforms to best practice guidance and safety regulations of the UK Fire and Rescue Service 21 . The FFST in this study was a continuous circuit of three tasks completed on a 25 m shuttle course as described previously 21,23 . Before beginning the circuit, a full verbal brief of the test was given and throughout the test a project researcher followed the participant and gave verbal instructions.…”

Implementation of Physical Employment Standards for Physically Demanding Occupations

“…The published version is available in the Journal of Environmental and Occupational Medicine. Manuscript #JOEM-20-8072R1 DOI: 10.1097/JOM.0000000000001921 https://journals.lww.com/joem/Abstract/9000/Implementation_of_Physical_Employment_Standards.98177.aspx 15 achieved by (1) defining performance standards , (2) identifying cut-scores associated with each performance standard using physical demands analysis data [19,20] and (3) agreeing a process with industry stakeholders for triaging and managing test performances [18]. This was represented using a modified traffic-light system as 'red' (physically incapable); 'amber' (unclear) and 'green' (physically capable) to ensure the PES were easy to understand to all stakeholders in the organisation.…”

“…Other practical factors may include finances for testing equipment, the time taken to administer each battery of tests as well as the knock-on effect to operational availability, which will all influence the decisions around how PES are implemented. The physical capability tests used in conjunction with PES are often debated and whilst both PST and DTS are regularly used for the assessment of both applicants and incumbents [1,2,19,20,[46][47][48][49], limitations have been identified with both approaches when used for the assessment of individual's appropriateness for work [1,18,30,32,[50][51][52]. This is the author-accepted manuscript not the final published manuscript.…”

“…Manuscript #JOEM-20-8072R1 DOI: 10.1097/JOM.0000000000001921 https://journals.lww.com/joem/Abstract/9000/Implementation_of_Physical_Employment_Standards.98177.aspx 13 intensity physical activity screen), to ultimately ascertain their physical capability (i.e. capable or incapable) to undertake safe and effective work [18].…”

“…In recognition that physical fitness can impact the health, safety and operational performance of workers in physically arduous jobs, the implementation of robust, evidencebased physical employment standards (PES) to ensure both the initial and on-going physical competencies of workers have become increasingly important in recent years [12,[15][16][17][18][19][20][21][22].…”