Does Training Improve Sanitary Inspection Answer Agreement between Inspectors? Quantitative Evidence from the Mukono District, Uganda

Abstract

Sanitary inspections (SIs) are checklists of questions used for achieving/maintaining the safety of drinking-water supplies by identifying observable actual and potential sources and pathways of contamination. Despite the widespread use of SIs, the effects of training on SI response are understudied. Thirty-six spring supplies were inspected on two occasions, pre- and post-training, by an instructor from the research team and four local inspectors in the Mukono District of Uganda. SI score agreement between the instructor and each inspector was calculated using Lin’s concordance correlation coefficient. Average SI score agreement between the instructor and all inspectors increased post-training for the Yes/No answer type (0.262 to 0.490). For the risk level answer type (e.g., No, Low, Medium, High), average SI score agreement between the instructor and all inspectors increased post-training (0.301 to 0.380). Variability of SI scores between the four inspectors was calculated using coefficient of variation analysis. Average SI score variability between inspectors reduced post-training for both answer types, Yes/No (21.25 to 16.16) and risk level (24.12 to 19.62). Consistency of answer agreement between the four inspectors for each individual SI question was calculated using index of dispersion analysis. Average answer dispersion between inspectors reduced post-training for both answer types, Yes/No (0.41 to 0.27) and risk level (0.55 to 0.41). The findings indicate that training has a positive effect on improving answer agreement between inspectors. However, advanced training or tailoring of SI questions to the local context may be required where inconsistency of responses between inspectors persists, especially for the risk level answer type that requires increased use of inspector risk perception. Organisations should be aware of the potential inconsistency of results between inspectors so that this may be rectified with appropriate training and, where necessary, better SI design and customisation.

The Potential for Citizen Science to Improve the Reach of Sanitary Inspections

“…In agreement with previous studies [8,12], our results indicate the need to ensure that questions are unambiguous and that there is sufficient guidance, such as the provision of training, within the SI package.…”

“…Such personnel shortages may be attributed to the resource challenges frequently associated with small drinking-water supply management [11], or to surveillance agencies being overwhelmed by large numbers of small supplies which tend to be spread sparsely over vast, often geographically remote, areas. There has been limited research into SI result accuracy and consistency between inspectors with varying levels of experience (from novice to experienced professional) and training [12][13][14] with no studies identified into the use of citizens for SI. In comparison with more definite indicators that water quality results provide, the measure of consistency of SI risk scores is noted as one of the challenges with regards to the use of the form [12].…”

“…There has been limited research into SI result accuracy and consistency between inspectors with varying levels of experience (from novice to experienced professional) and training [12][13][14] with no studies identified into the use of citizens for SI. In comparison with more definite indicators that water quality results provide, the measure of consistency of SI risk scores is noted as one of the challenges with regards to the use of the form [12].…”