Online, At-home Learning for Mathematics Remediation

Our solution solves for the strong mismatch between the level of classroom instruction and student learning levels in India (Glewwe and Muralidharan, 2016). As students move through the Indian education system, many learners consistently fall behind grade-level expectations, learn in classrooms along peers with highly heterogeneous learning levels, and are taught a curriculum far above their current level of understanding (Pritchett and Beaty, 2015; Muralidharan et al., 2019). The two-year COVID-19 lockdown has further worsened this situation. This means that over 24M of India’s 30M high-school students will either drop out or graduate significantly behind grade level. As a consequence, they are systematically shut-out of higher education and meaningful careers.

In order to meaningfully address this gap, students need to spend significant time learning subject matter that is at the right level for them. This is only possible at scale, through supplementary at-home programs. In order to work, any high scale solution would need to have three elements - high quality content, technology tools that allow both synchronous (online classes) and asynchronous (quizzes and recorded videos) instruction and personalization to ensure that students are taught at the right level. Few such solutions exist as open-source software and even fewer focus on local-language instruction. In addition, there is very little evidence and research to help design effective programs.

As a consequence most public-school systems continue to implement ineffective digital at-home learning programs with low engagement. 

Our solution allows teachers to conduct synchronous (live) online classes, send students interactive video lessons as homework and conduct quizzes all through open source, free-to-use web applications. Our applications integrate with Google Meet and Zoom to deliver live online classes and can use WhatsApp or SMS to interact with students. All student interactions are logged - allowing teachers access to rich student participation as well as academic achievement data. Currently we achieve personalization by organizing students at similar academic levels into batches. As we gather more student-level data we aim to automate both the sorting of students into batches as well as the specific homework assignments and quizzes recommended to students.

This solution is supported by an open source library of teaching aids, lesson plans, interactive video lessons and quizzes organized by learning level that have been compiled by Avanti. We currently cover over 1,500 competencies in Mathematics (https://sankalp.avantifellows.org/).

Our software also gives our program teams access to detailed, student level data on participation (attendance and homework completion) as well as student performance. We use automated WhatsApp nudges to reactivate students who are at risk of dropping out and counsel students and parents through targeted phone calls.

Specifically, our solution combines three tools. The first is a flexible authentication layer (portal). Teachers can take any resource (a web link, google form, google meet link, zoom meeting link etc) and convert it into a new shortened URL. They can also choose specific batches to make this link available to and choose the form of authentication (most of these are pre-set for each teacher by our operations team). When a student clicks on this new URL - they are taken to an authentication page where we match them to our student database and log their interactions thereafter. 

The second is Plio (www.plio.in) a tool that allows the creation of interactive video lessons. Users can take any YouTube video and add questions at specific time stamps. Students watch the video and attempt these questions at these timestamps and receive a scorecard at the end. This simple act of adding interactivity to these videos leads to a >5-fold increase in video completion rates. 

The third is a web application that allows Teachers to schedule live classes or find and send interactive lessons, worksheets or assessments directly to WhatsApp groups (https://haryana-teachers-af.web.app/). Most of our 3,000 students are currently organized on WhatsApp groups, where they receive a link that allows them to complete these tasks. Daily reports are compiled and sent to the teachers as automated WhatsApp messages every day. In the future, teachers will also be able to view detailed student-level data on their web-app.

Our solution is designed to serve the over 80M low-income students studying in India’s government schools. We are developing the only high-scale way in which these students can receive personalized, high effectiveness instruction that can help them catch up to grade level and access meaningful college and professional courses when they graduate school.

The majority of these students are rural and study in the local language of the state. Most have only one mobile device at their homes and need to work either in the fields or at home after school. The majority of these students (>70%) are behind grade level at Mathematics in grade 6, a gap that is not bridged throughout middle and high school. As a consequence most reach grade 10 more than 4 grades behind. It is incredibly difficult to address this problem in schools, since teachers are pressured to teach to the prescribed curriculum (at grade level) and have very limited time in school.

Our solution allows these students to dramatically improve the effectiveness of at-home learning. We conduct online classes in the late evenings to ensure that the students’ parents are back home and can share their devices with them. Our solution requires no additional software to be installed on the mobile device (we use web applications). Our software generates detailed student level data that enables both automated nudges and phone-based counseling for parents and students. 

In this way, we are able to get a majority of students (over 60%) to attend more than 2 hours a week of online classes consistently. Our data over the past two years shows that participation rates are more than three-fold that of conventional asynchronous whatsapp based learning models and we have early evidence of a >20% lift in scores which we will be valuditating through a two-year RCT starting in April 2022. 

Avanti currently works with over 50,000 students across five government school systems - Haryana, Himachal Pradesh, Tamil Nadu, Delhi and the Jawahar Navodaya Vidyalaya Schools (residential magnet schools). Over the past decade we have developed a nuanced understanding of our students and the administrators, teachers and principals in charge of providing them a productive and meaningful learning environment. We have been working on this specific problem (Math remediation for high-school students in Haryana) since 2014 and have conducted one Randomized Controlled Trial with JPAL in the same setting. In this trial we tested the effectiveness of in-school, in-class interventions to improve learning levels in Mathematics in 160 schools. This intervention involved providing specially designed workbooks and video lessons to teachers to use in their classrooms. It is the findings from this research that informed our move to developing solutions for supplementary, at-home learning. Avanti also ran the largest middle and high-school intervention to help students learn during the COVID-19 lockdowns in India. Over two million students learnt online from our open source content and technology. We were able to interact extensively with students and teachers during this period. These learnings have led to an increased emphasis on integrating online (synchronous) classes as part of our intervention.

Our current solution is being rigorously tested in a smaller cohort of 26 schools in haryana where we work every day with the students and teachers to understand the impact of and improve the effectiveness of our solution. 

Our team is composed of technologists, development sector experts and researchers who have worked on this specific problem for over five years.  

Core Team

Akshay Saxena (IIT Bombay, HBS), Co-Founder, Avanti Fellows leads this project.

Vandana Goyal (Claremont McKenna College), Executive Trustee at Avanti Fellows leads all operations and strategy for this project.

Panchali Dutta (Miranda House, Delhi University) leads the in-field operations.

Parul Jauhari (Amity University) leads the sandbox experimentation and program evaluation of student learning outcomes for this project.

Pritam Sukumar (IIT Bombay, IIT Bombay) is the technology lead, managing the tech infrastructure and roadmap.

Saksham Srivastava (IIT Kanpur, IIM Kozhikode) leads curriculum research and open content development.

Aman Dalmia (IIT Guwahati) leads tech strategy and implementation.

Deepansh Mathur (IIT Guwahati) leads the design and development.

Advisory Team 

Owen Henkel (University of Michigan, University of Oxford) serves as an advisor and external researcher for this project.

Dr. Andreas de Barros (Harvard PhD, MIT) serves as the advisor on the measurement of student learning outcomes for this project. 

We are applying to Solve to be able to build a better, more effective product for our students. We are excited to be part of the Solver Community for two primary reasons. The first is exposure to other like-minded organizations solving similar problems. The second is the exposure Solve would provide to donors, technical collaborators and researchers - all of whom will help us build a better product and get it in the hands of more students. 

Our solution focuses on equipping teachers in low resource settings to deliver both synchronous (online live classes) and asynchronous learning (interactive video lessons, digital homework) through a single platform. While there are some large commercial solutions that allow this (google classroom, tophat, canvas), there are no free-to-use open-source solutions that do this in a meaningful way. Additionally, our solution does not require the installation of any mobile application and works entirely through simple web applications. Ours is the only solution we know of that has been designed to be deployed at-scale in large-scale governments systems. We use existing student databases to authenticate students and gather student learning data and do not require any additional sign up from the students.

Pedagogically, we have a strong emphasis on supplementary at-home synchronous (live) classes. This is unlike most other interventions. We have strong early data that this in-person, live interactivity is the key to driving meaningful engagement and that asynchronous solutions that only rely on students completing tasks at home have dramatically lower efficacy. 

Lastly, we integrate fully with WhatsApp. WhatsApp groups continue to be the most engaging medium for sharing interactive lessons, worksheets, live class invites and engaging students (for e.g. through social-emotional learning activities). According to a JPAL survey, 57% of all students in our partner state of Haryana learned every day on these WhatsApp groups, in contrast with only 7% students, who accessed other EdTech apps. 

Over the next 2 years, we aim to demonstrate that we can get 50% of all enrolled students in 80 Government schools in Haryana to complete at least 80 hours of synchronous learning and 80 hours of interactive digital homework in Mathematics in a single year that should lead to a statistically significant improvement of at least 20% in the Math learning levels. We will measure this through a 2-year RCT being conducted by JPAL.   

In parallel we will enable 50,000 students in our partner states to learn Mathematics and Science as well as participate in online test preparation programs through our software. 

In the next five years, we aim to leverage the findings from our research to scale the Mathematics remediation program to over 5M students in our partner states. The relationships, agreement and funding needed for this is already available within our state government partners. We are consciously holding back on scaling up until we can have high quality research demonstrating the efficacy of the program and technology we are developing.

We measure long term progress for each student through their mastery of one of ~1,500 key competencies in Mathematics. Through each academic year, each unit is preceded by a baseline test and completed with an endline assessment (each unit lasts 4-6 weeks). We measure time on learning as our key formative metric which includes time spent in online live classes as well as time completing digital homework. Based on prior research our targets are to have 50% of all of our students spent at least 2 hours a week actively learning and to have student scores improve by an average of 20% in each unit.

We aim to provide each student with 80 hours a year of Mathematics instruction at the right level for them. This should lead to them learning prior-grade level competencies that they would not otherwise learn in school and catch up to grade level in a 2-3 year horizon. By doing so, we aim to increase high-school graduation rates, college readiness and participation in STEM careers. 

A recent metaanalysis (Martin et. al., 2021) reported a 0.15SD improvement in cognitive outcomes when synchronous online classes replace asynchronous online learning. This is in line with our emphasis on synchronous learning at the core of our intervention. 

Our approach builds on the idea of using instruction that teaches at the right level by improving the match between instruction and student learning levels. Existing research has used ‘Teaching at the Right Level’ (TARL) type of interventions where students are re-grouped as per learning needs and provided remedial instruction (Banerjee et al., 2016). A more sophisticated technological version of the TARL intervention uses personalization and dynamically adapts instruction to student learning levels. Muralidharan et al. (2019) find a 0.37 standard deviation improvement in math test scores after 4.5 months of exposure to the Mindspark software. Our approach of using a simple web-based tool that can run on feature phones has been substantiated repeatedly by prior research (Kumar et al. 2012; Valderrama Baham´ondez et al. 2014; Porter et al. 2016; Rush et al. 2016). 

However, MOOCs and other forms of semi or fully independent online learning have notoriously high attrition rates, and low engagement scores often attributed to the passivity of consuming recorded video content (Khalil, H.; Ebner, M. 2014). We solve for this by introducing interactivity into previously passive free online learning content. A growing body of research finds that adding interactivity to video lessons is a crucial element for improving the quality of online learning (Guri-Rosenblit, 2009; Siemens et al., 2015). Empirical studies also demonstrate the effectiveness of interactivity in extending the attention span of learners and enhancing their achievements (Cherrett, Wills, Price, Maynard, & Dror, 2009; Dror, Schmidt, & O'connor, 2011). A recent study (Geri et. al., 2017) reported a 20% improvement in retention through the introduction of interactivity to online videos. 

Our solution includes a modular technology stack based on microservices. We host and deploy these services using self-built Continuous Integration and Deployment pipelines on Amazon Web Services. 

Individual services talk to each other solely through APIs. This means that they can be deployed in various combinations to provide a custom solution for a particular program. We build solutions that are laser-focused on the end-user experience. For example, instead of building out a complex user interface for teachers to navigate, we developed a web app that allows teachers to send lesson plans to their students on Whatsapp with a single click. Our authentication layer solves unique challenges such as multiple students using the same device, and students not being able to remember their usernames and passwords. Our interactive video tool – simply by layering questions on top of videos – increases engagement on videos by upto 5x. 

A big part of our operations team, and the users we work with are familiar with Excel and the Google Suite of products. Because of this, we even allow data integration with Google Sheets, Google Docs and so on as a backend in addition to our own custom backend. 

We believe that our focus on building modular pieces of software as our technology solution rather than a single product allows us to be flexible and scalable while satisfying the differing needs of our stakeholders.

Avanti Fellows strives to be an inclusive organization with a team of close to 150 professionals (40% of whom are women) led by Akshay Saxena and Vandana Goyal. The leadership team has a 60:40 (female:male) gender split with professionals from diverse educational and professional backgrounds. The team closely works with each other on execution and iteration of programs, while offering autonomy in decision making to help fuel each teammate's growth. We are aggressively and consciously moving towards creating a very gender-diverse and gender-equitable organization.

The government spends over $5,000 per student in India over the course of their school education. Unfortunately only 10% of these students graduate at grade level making the effective cost per student graduating at grade level $50,000. 

We aim to more than double the number of students graduating at grade level through a 3 year intervention that costs a total of $200 (including instructor time). In simple terms that is a return of over 10-fold just at the point of graduation. The difference in lifetime earnings of students who go on to professional courses is in excess of $200,000 per student that we help pass this threshold. 

Currently, our program costs are funded by philanthropic donations by the Michael and Susan Dell Foundation, Sofina, The Morgridge Family Foundation and the Douglas B Marshall Foundation. Once proven, we will charge our government partners a service fee to maintain the software and systems and assist them in creating commercial tenders to hire the teachers and program staff needed to run our interventions.  

Avanti Fellows is an 11-year-old non-profit organization primarily funded by international grants, CSR donations and individual giving. The primary revenue model for this solution is international grants. This project is currently supported by the Michael and Susan Dell Foundation (5 years), Sofina (3 years), Morgridge Family Foundation (2 years), Douglas B Marshall Foundation (2 years). Historically, we have been supported by USAID, MacArthur Foundation, J. P. Morgan, Tata Group, Sequoia Capital. We aim to continue in our efforts to leverage additional funding from such partners of change.

There are three major categories of costs. The first is the development and maintenance of our software. This is currently managed by a team of 6 engineers with an annual budget of $400,000. The second is the curation of content and the creation of interactive lessons to create the initial library of content. This was a largely one-time cost which has already been covered by past grants. The third is the program management and operational effort required to train teachers, students and parents and conduct live classes. 

We are currently playing an active role with an expenditure of $400,000 per year but expect this cost to drop significantly as we transition a larger portion of the program management to the state governments. We have received unrestricted funding to support organizational development (including Board members) to enable us to fundraise further with the help of our Board members.

This particular project has been funded through $2M USD in grants from the Michael and Susan Dell Foundation and Sofina. 

We have received close to $7M USD in funding since inception in 2010 till date. Our major donors include Michael and Susan Dell Foundation, J.P. Morgan, Sofina, Marshall Foundation and domestic organizations like Edelgive Foundation and the renowned TATA group including TATA Motors and TATA AIG that support via corporate social responsibility initiatives. Historically, we have also been supported via HNI giving as well.