Assessing community readiness online: a concurrent validation study

Abstract

BackgroundCommunity readiness for facilitation and uptake of interventions can impact the success of community-based prevention efforts. As currently practiced, measuring community readiness can be a resource intensive process, compromising its use in evaluating multisite community-based prevention efforts. The purpose of this study was to develop, test and validate a more efficient online version of an existing community readiness tool and identify potential problems in completing assessments. This study was conducted in the context of a complex community-based childhood obesity prevention program in South Australia.MethodsFollowing pre-testing, an online version of the community readiness tool was created, wherein respondents, with detailed knowledge of their community and prevention efforts, rated their communities on five anchored rating scales (Knowledge of Efforts, Leadership, Knowledge of the Issue, Community Climate, and Resources). Respondents completed the standard, over-the-phone community readiness interview (“gold standard”) and the new online survey. Paired t-test, St. Laurent’s correlation coefficient and intra-class correlation (ICC) were used to determine the validity of the online tool. Contact summary forms were completed after each interview to capture interview quality.ResultsTwenty-five respondents completed both assessments. There was a statistically significant difference in the overall community readiness scores between the two methods (paired t-test p = 0.03); online scores were consistently higher than interview scores. St. Laurent’s correlation of 0.58 (95 % CI 0.42–0.73) was moderate; the ICC of 0.65 (95 % CI 0.35–0.83) was good. Only for the leadership and resources dimensions was there no statistically significant difference between the scores from the two methods (p = 0.61, p = 0.08 respectively). St Laurent’s correlation (r = 0.83, 95 % CI 0.71–0.92) and the ICC (0.78, 95 % CI 0.57–0.90) were excellent for leadership. Qualitative results from the standard interview method suggest that some respondents felt reluctant to answer questions on behalf of other community members. This may have impacted their self-selected ratings and/or responses to questions during the interview.ConclusionsConcurrent validity for the online method was supported for the Leadership dimension only. However, the online method holds promise as it reduces time and resource burden, allowing for a quicker return of results to the community to inform program planning, implementation and evaluations to improve community health.

Assessing change in perceived community leadership readiness in the Obesity Prevention and Lifestyle program

“…The leadership readiness dimension from the survey used in the concurrent validation study 10 was used for the present study.…”

“…Assessing all five dimensions of the CRT would have provided a more comprehensive assessment of community readiness, however only the leadership dimension has been validated for online assessment. 10 Finally, given the staggered nature of OPAL implementation in the communities, different phases were assessed at different stages of the intervention. Phase 1 and 2 communities were rated post intervention, whereas Phases 3 and 4 were rated after 4 and 3 years of intervention, respectively.…”

“…A concurrent validation study supported the utility and feasibility of assessing perceived community leadership readiness (PCLR) in large scale intervention programs. 10 The present study assessed PCLR in a state-wide, community-based childhood obesity prevention program implemented in 168 suburbs across 20 communities in South Australia (SA). The objectives of the present study were:…”

How ready are communities to implement actions to improve diets of adolescent girls and women in urban Ghana?

“…The CRM has most frequently been used within the U.S.A, for alcohol and drug use [14]. However, the tool has also been used to assess readiness for overweight and obesity prevention in high-income countries (HICs) including the U.S.A [15, 16], Australia [17, 18] and the UK [19] and applied to ethnic minority [20] or disadvantaged communities [21]. Limited research has been conducted in low- and middle-income countries (LMICs) using the CRM.…”

Protocol for the measurement of changes in knowledge and engagement in the stepped wedge cluster randomised trial for childhood obesity prevention in Australia: (Reflexive Evidence and Systems interventions to Prevent Obesity and Non-communicable Disease (RESPOND))

“…The common elements within these tools emphasise the importance of knowledge, resources and leadership. These constructs, although shown to relate to successful obesity prevention [ 26 ], are difficult constructs to measure [ 27 ] and were not specifically designed to measure changes in underlying determinants of obesity over time.…”