2021 Seed Grant Award Problem Statements 

Education

Problem #20: A solution to identify variables influencing student learning outcomes using Machine Learning (ML) and Predictive Analysis and deepen the understanding of student learning and to tailor large scale programs for diverse needs and accelerating learning gains

Country/Region of execution:  India
Collaborating Organization: Transform Schools, People For Action
People For Action (PFA) is a registered society under the Indian Societies Registration Act (1860) and works with the government school system to support better teaching and management to improve learning outcomes in India through its Transform Schools (TS) programme. We do this by providing rigorously tested tools and training to teachers, students, governing bodies and parents. Our competency based targeted teaching learning approach - Transform Learning (TL) is delivered across four States by our government partners to improve learning outcomes for 4.9M students. TL’s; ability to promote student performance has been proven through an RCT run by J-PAL. Results; showed that TL adds up to over half a year of additional learning via 50 hours of targeted instruction.  (Beg et al; JPAL2020, https://drive.google.com/drive/folders/1lEmy2WwyKSqq2IyfjjNtQ-et0S_MGQYj?usp=sharing ).  TS team consists of social entrepreneurs, education, equity, management, analytics and evidence experts. Given our existing reach and efficacy, we see tremendous potential to leverage data to further improve outcomes.
Problem Statement Description: Rapid digitization has exposed stark digital divide and inherent inequities. Poor access to online learning has further exacerbated the impact of school closure on students from marginalized communities and girls. The solution will leverage data to identify learning gaps, poorly performing subsets, anticipate dropouts and reduce vulnerabilities directly contributing to SDGs 4 and 5. RCT evidence shows that our TL program improves teaching quality and student outcomes. We believe that our current gains are good but can be improved significantly. Currently Google forms are used for data collection and analyses. Cloud-based technology enabled with ML will generate newer insights to benefit particularly vulnerable students and promote education equity. Bottleneck: In response to school closure, our partner States employed blended delivery mechanisms to tide disruptive effects of COVID19 on participation, learning and continuity. However, database and analysis constraints limit us to predict students at risk of dropout owing to digital divide, and test innovative approaches. While our existing process allows for analyses/visualization, it is not sufficiently sophisticated to allow deeper analysis in shorter loops, for us to rapidly target our offerings and improve outcomes for 8.4M students across ten States by 2023. Key Constraints: TS uses a participatory monitoring and evaluation (M&E) system to measure program impact at scale. Transitioning from Google forms and excel to a cloud-based tool will help automate analyses and dashboards for nuanced understanding of program uptake and impact, predict performance and recommend program adjustments in real time. Following are some of the steps that were taken towards tackling the challenge: Research on ML based solutions and Predictive Analysis framework developed; Hypothesis establishing relation between student participation and performance tested; Error Analysis framework developed for identifying common gaps in student competencies. Funding and technical support are needed to develop the machine learning capabilities to conduct predictive analysis, assess user behavior, draw insights and generate actionable suggestions for better learning outcomes. Additional insights enabled by Machine Learning and Predictive Analysis / Benefits: Predict which students are at risk of falling behind / Inform alternative solutions (differentiated learning pathways) and curtail student dropout; Run adaptive experiments to identify learning gaps / Help in testing innovative targeting and learning approaches and improve learning outcomes; Explore heterogeneous treatment effects to understand which groups of students are benefitting (high potential performers) or are not benefitting from this program / Create new modes of support and allocate resources to students who need them most; Identify how variables such as access to digital resources influence participation and student learning, and predict differences in outcomes / Make changes in our targeting approach to improve both participation and performance and help reduce digital divide; Predict performance of teachers and ensure technology adoption and engagement using behavioral science methodology / Allocate professional development resources to teachers who need them most to enhance quality of teaching practices; Cloud-based technology platform with dashboards (real time and predictions) to derive block, district and State level performance / Allow aggregate performance visualizations based on predictions and improvement suggestions for rapid large-scale program improvements.