Potential adverse effects of an educational intervention: Development of a framework

Oxman, Matt; Chesire, Faith; Mugisha, Michael; Ssenyonga, Ronald; Ngatia, Benson; Nsangi, Allen; Lewin, Simon; Moberg, Jenny; Sewankambo, Nelson K.; Kaseje, Margaret; Melby‐Lervåg, Monica; Fretheim, Atle; Oxman, Andrew D; Rosenbaum, Sarah

doi:10.1101/2022.07.27.22278097

Cited by 3 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We did not ask them to monitor specific events. We also will explore potential adverse effects in the process evaluations conducted together with each trial, 20–22 and a qualitative evidence synthesis 30 . Potential adverse effects identified through the process evaluations will be measured quantitatively in the 1‐year follow‐up study.…”

Section: Methodsmentioning

confidence: 99%

Effects of the Informed Health Choices secondary school intervention: A prospective meta‐analysis

Chesire,

Mugisha,

Ssenyonga

et al. 2023

J Evidence Based Medicine

Self Cite

View full text Add to dashboard Cite

AimThe aim of this prospective meta‐analysis was to synthesize the results of three cluster‐randomized trials of an intervention designed to teach lower‐secondary school students (age 14–16) to think critically about health choices.MethodsWe conducted the trials in Kenya, Rwanda, and Uganda. The intervention included a 2‐ to 3‐day teacher training workshop, digital resources, and ten 40‐min lessons. The lessons focused on nine key concepts. We did not intervene in control schools. The primary outcome was a passing score on a test (≥9 of 18 multiple‐choice questions answered correctly). We performed random effects meta‐analyses to estimate the overall adjusted odds ratios. Secondary outcomes included effects of the intervention on teachers.ResultsAltogether, 244 schools (11,344 students) took part in the three trials. The overall adjusted odds ratio was 5.5 (95% CI: 3.0–10.2; p < 0.0001) in favor of the intervention (high certainty evidence). This corresponds to 33% (95% CI: 25–40%) more students in the intervention schools passing the test. Overall, 3397 (58%) of 5846 students in intervention schools had a passing score. The overall adjusted odds ratio for teachers was 13.7(95% CI: 4.6–40.4; p < 0.0001), corresponding to 32% (95% CI: 6%–57%) more teachers in the intervention schools passing the test (moderate certainty evidence). Overall, 118 (97%) of 122 teachers in intervention schools had a passing score.ConclusionsThe intervention led to a large improvement in the ability of students and teachers to think critically about health choices, but 42% of students in the intervention schools did not achieve a passing score.

show abstract

Section: Methodsmentioning

confidence: 99%

Effects of the Informed Health Choices secondary school intervention: A prospective meta‐analysis

Chesire,

Mugisha,

Ssenyonga

et al. 2023

J Evidence Based Medicine

Self Cite

View full text Add to dashboard Cite

show abstract

“…We are exploring experiences and views of potential adverse effects in separate process evaluations 30,31,55 and in a qualitative evidence synthesis. 56 • We struggled finding appropriate examples of reliable and unreliable claims, as students often thought the lesson was about the example rather than the underlying Key Concept. Our solutions included finding a balance between real and fictional, well-labeled examples, guidance and prompts for teachers (including suggestions to find their own examples), and developing an example collection for teachers to find alternatives.…”

Section: Negative Findings and How We Addressed Themmentioning

confidence: 99%

“…We are exploring potential adverse effects of the final resources in process evaluations 30,31,55 and a qualitative evidence synthesis. 56…”

Section: Strengths and Limitationsmentioning

confidence: 99%

Teaching critical thinking about health information and choices in secondary schools: human-centred design of digital resources

Rosenbaum,

Moberg,

Chesire

et al. 2024

F1000Res

View full text Add to dashboard Cite

Background Learning to thinking critically about health information and choices can protect people from unnecessary suffering, harm, and resource waste. Earlier work revealed that children can learn these skills, but printing costs and curricula compatibility remain important barriers to school implementation. We aimed to develop a set of digital learning resources for students to think critically about health that were suitable for use in Kenyan, Rwandan, and Ugandan secondary schools. Methods We conducted work in two phases collaborating with teachers, students, schools, and national curriculum development offices using a human-centred design approach. First, we conducted context analyses and an overview of teaching strategies, prioritised content and collected examples. Next, we developed lessons and guidance iteratively, informed by data from user-testing, individual and group interviews, and school pilots. Results Final resources include online lesson plans, teachers’ guide, and extra resources, with lesson plans in two modes, for use in a classroom equipped with a blackboard/ﬂip-chart and a projector. The resources are accessible offline for use when electricity or Internet is lacking. Teachers preferred the projector mode, as it provided structure and a focal point for class attention. Feedback was largely positive, with teachers and students appreciating the learning and experiencing it as relevant. Four main challenges included time to teach lessons; incorrect comprehension; identifying suitable examples; and technical, logistical, and behavioural challenges with a student-computer mode that we piloted. We resolved challenges by simplifying and combining lessons; increasing opportunities for review and assessment; developing teacher training materials, creating a searchable set of examples; and deactivating the student-computer mode. Conclusion Using a human-centred design approach, we created digital resources for teaching secondary school students to think critically about health actions and for training teachers. Be smart about your health resources are open access and can be translated or adapted to other settings.

show abstract

The effects of teaching strategies on learning to think critically in primary and secondary schools: an overview of systematic reviews

Oxman,

Nsangi,

Martínez García

et al. 2024

F1000Res

View full text Add to dashboard Cite

Background We conducted an overview of systematic reviews about the effects of teaching strategies that can be used to teach primary and secondary school students to think critically. Our objective was to inform decisions about what teaching strategies to use in resources that we developed to teach critical thinking about health in secondary schools. Methods We mapped characteristics of systematic reviews of teaching strategies and summarised findings from the most relevant reviews to teaching students to think critically about health. We included reviews that assessed the effects of teaching strategies that could potentially be used in primary or secondary schools to teach students to think critically, had a Methods section with explicit selection criteria, reported at least one outcome measure of the ability to undertake one of four basic types of cognitive tasks (memory, procedural, comprehension, or opinion), and were published after 1999. Results We included 326 systematic reviews. The reviews evaluated a wide range of teaching strategies for a variety of purposes. Important limitations of the reviews included not considering adverse effects (99% of the reviews), not assessing the risk of bias for included studies (93% of the reviews), and not assessing the credibility of subgroup effects (100% of the reviews). We summarised the findings for 37 teaching strategies that we considered most relevant. The certainty of the evidence of the effects varied from very low to moderate. We used 12 of the strategies in resources that we developed to teach secondary students to think critically about health. Conclusions A tremendous amount of work has gone into evaluating the effects of a wide range of teaching strategies. The results of this research can inform decisions about how to teach critical thinking and future research. However, well-designed, up-to-date systematic reviews are still needed for many teaching strategies.

show abstract

Potential adverse effects of an educational intervention: Development of a framework

Cited by 3 publications

References 42 publications

Effects of the Informed Health Choices secondary school intervention: A prospective meta‐analysis

Effects of the Informed Health Choices secondary school intervention: A prospective meta‐analysis

Teaching critical thinking about health information and choices in secondary schools: human-centred design of digital resources

The effects of teaching strategies on learning to think critically in primary and secondary schools: an overview of systematic reviews

Contact Info

Product

Resources

About