Concept mapping: an approach for developing an evaluation framework and realistic outcome indicators for community-based nutrition interventions

Abstract

Complex interventions, particularly in the case of policy and community interventions, tend to generate multiple outcomes that can be difficult to combine when evaluating the intervention. The aim of the present project was to develop a realistic timeline for evaluation of nutrition-related community interventions. The model used in the present study was based on the anticipated impact of the Schools (Health Promotion and Nutrition) (Scotland) Act 2007 (1) in primary schools. The Act was implemented in primary schools in August 2008 and aimed to improve the nutritional standards of food and drinks provided in schools and increase health promotion in schools.Concept mapping is a method that has been used for developing evaluation frameworks for interventions and has been described as a technique for 'simplifying complexity' (2) . It involves consultation with stakeholders and comprises three stages: brainstorming the anticipated outcomes; sorting outcome statements into higher-order themes; rating the outcome statements. A list of outcome statements was generated during a workshop held with academic researchers and policy makers. Participants were asked to complete the statement 'I think the introduction of the Schools Act (2007) in primary schools will . . .'. Additional statements were generated from interviews carried out with key stakeholders from local authorities and the National Health Service (NHS) and focus groups run with parents of children in primary schools. A total of 177 statements were generated, which after reviewing and removing duplications left eighty-five statements for the sorting and rating. The statements were rated, using Likert scales, on their importance (from 1, not at all to 5, extremely important), the likelihood of them occurring (from 1, not at all to 5, extremely likely) and the earliest time to measurable impact (from 0, immediate to 6, ‡6 years). Participants from academic research, national government, government agency, local authority, NHS and community organisations completed sorting and rating the statements on an online website (Concept Systems software; Concept Systems Inc., Ithaca, NY, USA).The statements were sorted into higher-order themes by 111 participants and 102 participants completed all three sets of ratings. Multidimensional scaling and hierarchical cluster analysis were used to combine the individually-sorted statements into meaningful concepts. Initially, fifteen clusters were identified with a stress value of 0.25 and this low stress value indicates a good fit of the model. The clusters comprised statements with similar themes (e.g. health issues, diet of children, attitudes, knowledge and food preferences, financial implications). The rating scores highlighted differences between the importance and the likelihood of the outcomes as well as the anticipated earliest time to measurable impact. For example, the 'health' cluster, which included issues of obesity, diabetes, dental health and blood pressure was rated highly for importance (4.29) but lower on likel...

Concept mapping: an approach for developing an evaluation framework and realistic outcome indicators for community-based nutrition interventions

“…Concept mapping is a method that has been used for developing evaluation frameworks for interventions and has been described as a technique for 'simplifying complexity' (2) . It involves consultation with stakeholders and comprises three stages: brainstorming the anticipated outcomes; sorting outcome statements into higher-order themes; rating the outcome statements.…”

Using Concept Mapping to Identify Action Steps for Physical Activity Promotion in Cancer Treatment

“…Statements were generated based on the above mentioned prompt either through interviews, a workshop at a conference, or focus-groups (Macdiarmid et al, 2010). A total of 40 individuals participated.…”

“…Thirteen clusters were created based on the work of these stakeholders (Macdiarmid et al, 2010). Examples of the clusters include: "Promote Well-Being", "Increase Food Literacy", and "Encourage Healthy Diet Choices".…”