Investigating the impact on learning outcomes through the use of EdTech during COVID-19: Evidence from an RCT in the Punjab province of Pakistan

Abstract

The objective of the current study is to test the impact of low-tech solutions during COVID-19 school closures on maths, Urdu, and English scores for students in the underdeveloped district of Bahawalnagar, which is situated in the southern part of Punjab in Pakistan. The target population for this study is Grade 8 students attending private schools. Simple random sampling technique is adopted for the selection of participants in the study. We have tested the impact of three interventions, namely: Teaching at the right level (TaRL), Fortnightly assessments (FAS), and Digital teacher training sessions (DTS). Our findings show a significant and positive ‘Intention to Treat’ (ITT) impact on Urdu and English scores of the students in the TaRL treatment group. The students increased their English and Urdu scores by 0.56 SD. However, we found no significant impact of the intervention on maths scores in the TaRL treatment group. Fortnightly assessments and digital teacher training sessions were also found to contribute to higher English scores of the students. However, we found no ITT impact on the maths and Urdu scores for these treatment groups. The Local Average Treatment Effect (LATE) analysis revealed positive and significant improvement in Urdu and English scores of the students in the TaRL treatment group. Key stakeholders whom we interviewed suggested that redesigning the curriculum and incorporating TaRL within this approach could facilitate enhancement in learning outcomes in students in deprived areas. Our findings are important to help inform policymakers on the importance of designing and implementing cost-effective, low-tech solutions to help reduce learning gaps.