RoB-SPEO: A tool for assessing risk of bias in studies estimating the prevalence of exposure to occupational risk factors from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury

Abstract

Background The World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates). For this, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors will be conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methods is to assess risk of bias (RoB) of individual studies. In this article, we present and describe the development of such a tool, called the R isk o f B ias in S tudies estimating P revalence of E xposure to O ccupational risk factors (RoB-SPEO) tool; report results from RoB-SPEO's pilot testing; note RoB-SPEO's limitations; and suggest how the tool might be tested and developed further. Methods Selected existing RoB tools used in environmental and occupational health systematic reviews were reviewed and analysed. From existing tools, we identified domains for the new tool and, if necessary, added new domains. For each domain, we then identified and integrated components from the existing tools (i.e. instructions, domains, guiding questions, considerations, ratings and rating criteria), and, if necessary, we developed new components. Finally, we elicited feedback from other systematic review methodologists and exposure scientists and agreed upon RoB-SPEO. Nine experts pilot tested RoB-SPEO, and we calculated a raw measure of inter-rater agreement ( P i ) for each of its domain, rating P i < 0.4 as poor, 0.4 ≤ P i ≥ 0.8 as substantial and P i > 0.80 as almost perfect agreement. Results Our review found no standard tool for assessing RoB in prevalence studies of exposure to occupational risk factors. We identified six existing tools for environmental and occupational health systematic reviews and found that their components for assessing RoB differ considerably. With the new RoB-SPEO tool, assessors judge RoB for each of eight domains: (1) bias in selection of participants into the study; (2) bias due to a lack of blinding of study personnel; (3) bias due to exposure misclassification; (4) bias due to incomplete exposure data; (5) bias due to conflict of interest; (6) bias due to selective reporting of exposures; (7) bias due to difference in numerator and denominator; and (8) other bias. The RoB-SPEO's ratings are low, probably low, probably high, high or no information. Pilot testing of the RoB-SPEO tool found substantial inter-rater agreement for six domains (range of P i for these d...

The effect of exposure to long working hours on ischaemic heart disease: A systematic review and meta-analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury

“…WHO and ILO, supported by a large network of experts, are in parallel also producing a systematic review of studies estimating the prevalence of exposure to long working hours ( Li et al, 2018 ). The review of prevalence of exposure is applying novel systematic review methods (e.g., the RoB-SPEO risk of bias tool ( Pega et al, 2020 ). The organizations are also in parallel conducting several other systematic reviews and meta-analyses on other additional risk factor-outcome pairs ( Descatha et al, 2018 , Godderis et al, 2018 , Hulshof et al, 2019 , Li et al, 2018 , Mandrioli et al, 2018 , Paulo et al, 2019 , Rugulies et al, 2019 , Teixeira et al, 2019 , Tenkate et al, 2019 ).…”

The effect of exposure to long working hours on alcohol consumption, risky drinking and alcohol use disorder: A systematic review and meta-analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury

“…We present our findings in the current paper. WHO and ILO are in parallel also producing a systematic review of studies estimating the prevalence of exposure to long working hours (forthcoming), applying their novel systematic review methods ( Pega et al, 2019 ). The organizations are also conducting or have completed several other systematic reviews and meta-analyses on other additional risk factor-outcome pairs ( Hulshof et al, 2019 , Mandrioli et al, 2018 , Paulo et al, 2019 , Teixeira et al, 2019 , Tenkate et al, 2019 , Li et al, 2018 , Li et al, in press , Rugulies et al, 2019 , Descatha et al, 2018 , Descatha et al, in press ).…”

The effect of occupational exposure to welding fumes on trachea, bronchus and lung cancer: A protocol for a systematic review and meta-analysis from the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury

“…Standard risk of bias tools do not exist for systematic reviews of effects of exposure to occupational risk factors on health outcomes in occupational and environmental health ( Pega et al, 2019 ). The five methods specifically developed for occupational and environmental health are for either or both hazard identification and risk assessment and they differ substantially in the types of studies (randomized, observational and/or simulation studies) and data (e.g.…”