Avanti Fellows

Our educational solution is the NEEV Live Class program, an at-home online Maths remediation program consisting of online classes and digital homework delivered through open-source software and content. Till date, the program has helped more than 3,000 9th grade students from government schools in Haryana catch up to grade level learning. 35% of all students have been meaningfully active (attended more than 50% of all classes). In this population more than 50% of the below-grade students have achieved grade level competence. While these findings are encouraging, we believe that starting earlier (in Grade 6) would dramatically increase the number of students graduating into high school at grade level. We propose piloting the program (which starts with Maths competencies from Grade 3) with ~250 12-year-old 6th grade students in 4 schools. 

Our program design is user-centric in nature to ensure our solutions are closely aligned with grassroot realities and requirements. Hence, our team works closely with the affected community and all related stakeholders. These stakeholders include: 

a) students

b) parents

c) school teachers

d) school heads  

With the support of all these stakeholders, we conduct need analysis and other qualitative/quantitative studies to define the scope of our product as well as seek input on product and program design and usage. Some ways in which this is done is:

1. Conducting detailed student surveys and interviews to understand digital device accessibility and acceptability patterns in rural and urban areas. Our program schedule is designed around these findings, and has led to more than 70% product adoption by students. The link to one such survey’s result can be found here. 

1. Using teacher/students inputs and student-level assessments to customise the content and structure of our live classes. This has generated an early evidence of a >20% lift in scores

1. Conducting regular telephonic counselling calls to parents and monthly school visits to seek feedback on the program as well as identify and resolve barriers to participation in the program. This has enabled us to identify various points of exit for students in our program, as shown below, and iteratively incorporate remedial measures in the program to maximize program participation.

Fig 1: Program Funnel for Small Schools (schools with =<40 students)

Fig 2: Program Funnel for Large Schools (schools with >40 students)

Our theory of change is that remedial (Catch-up) instruction in Mathematics through live classes and digital homework would lead to more students graduating class 10th with mastery over critical competencies in Math. This would lead to more students choosing to pursue and qualifying for quality undergraduate professional courses that lead to meaningful and financially-sustainable careers. 

India has around 80M low-income students studying in its government schools. More than 70% of them are behind grade level by the time they are in 6th grade, a gap which widens drastically as they struggle to complete middle and high school. Given instructional time constraints, school teachers also have limited capacity to bridge this 4-grade gap by the time they reach grade 10th, leading to massive school dropout thereafter. 

An opportunity exists to bridge this gap through at-home learning, given medium to high digital penetration in various parts of India. Even within our districts of intervention in Haryana, as per a recent JPAL survey, roughly ~90% students have access to smartphones at home. 

Leveraging this, our solution is to conduct online classes for students so that they get 3 hours remedial math instruction per week. Through the coverage of ~250 important remedial learning outcomes over a period of two years, we seek to bring students to an at grade level, or at least a functional level of mathematics by the time they graduate grade 10th. In doing so they can access quality professional courses when they graduate school, or at the bare minimum, clear basic government exams to lead financially sustainable lives.

After seeing the initial success of our program implementation in 9th grade, our next hypothesis is that the impact on learning and sustainable livelihood would be higher if this program starts in lower grades. For this, we want to pilot the program with ~200 12-year-old 6th grade students in 4 schools. 

We have a dedicated Research and Innovation team which works closely with 15 “Sandbox” schools to pilot all new programs and interventions. Every intervention and change is implemented only after piloting and small-scale impact evaluation (both qualitative and quantitative evidence generation). The Research and Innovation team also works on larger program-level data and assessments to analyse wider adoption trends and overall learning impact.

Some of our major learnings from these exercises include:

1. Through a series of three internal assessments of more than 1000 students, we have learnt that students need to attend more than 50% classes of a learning unit to show any statistically significant meaningful learning gain. 

Fig 3: Regression analysis showing that effect of live classes becomes statistically significant at attendance levels above 50% of classes

1. On an average, attending live classes has benefited all groups of students, regardless of gender. However, the gain also depends on the student’s baseline score and the number of classes attended.

1. Based on insights from qualitative studies, we ensured parents play a pivotal role in the program. We involve them in the program through orientation and counselling calls. This has proven to be a very beneficial exercise. In a dataset of 5273 students, onboarding is higher by 81% in the first two weeks of the intervention if parent orientation is done.   

Fig 4: Effect of parent orientation on student onboarding

In addition to internal research, we also have external evaluators J-PAL and GMI who conduct annual assessments to analyse the impact of the program. 

NEEV Live Class program impact is being tracked both internally and externally by:

1. Our in-house Research and Innovation team through internal assessments and stakeholder surveys

2. The Abdul Latif Jameel Poverty Action Lab (J-PAL) through a 2-year RCT study

3. ConveGenius Insights through an additional bi-annual assessment exercise to assess the impact of our programs in treatment and control schools.

For internal evaluation, we measure increase in learning levels pre and post treatment, through internal baseline and endline assessments. We also study inter-quintile movement to assess the variable effect of treatment on students of different academic levels. Our sampling technique for this exercise is cluster sampling technique. The learning impact report from last year can be found here.

Additionally, a control:treatment school comparison of government-led SAT exam results will be conducted to test the effect of remedial learning on at-grade math learning levels. These quantitative assessments will also be backed with in-house qualitative surveys conducted by the research team for all stakeholders including students, parents and teachers. 

JPAL will be conducting an RCT with 80 treatment and 80 control schools and conducting independent baseline, midline and endline assessments over a course of two years (2022-2024) to test for learning gains at both below grade and at grade levels.

At-home online classes and digital homework to help government school students catch up to grade level in Maths.

• What research question would LEAP Fellows help you to answer?

  • Can we help a majority of government school students graduate Grade 10 at grade level through a consistent at-home online program starting in Grade 6. 

• What potential deliverables would be useful to your organization as outputs of the LEAP sprint? Keep in mind that LEAP projects take place over 12 weeks with approximately 2-4 LEAP Fellows each working 6-16 hours per week.

  • Measuring impact on learning of NEEV Live Class Program on grade 6th students

  • Analysing program adoption patterns in grade 6th and comparison of the same with grade 9th adoption

  • Creating a longitudinal projection of program impact with grade 6th as grade of initial intervention  

• What would the successful outcome of this project allow you to achieve? How would hosting a LEAP project enhance your organisation’s 5-year plan?

  • Successful outcomes would make a strong case to institutionalise the program across the entire state of Haryana.