Click Learning
Executive Summary
Project Host:
Fellows:
Anthony Senanayake, Social Entrepreneur Fellow
Barbara Trudell, Research Fellow
Lissett Babaian, Social Entrepreneur Fellow
Teomara Rutherford, Research Fellow
Introduction
Although research shows that early literacy and numeracy are foundational to later achievement and employment, in South Africa, 82% of children in Grade 4 cannot read for meaning (Spaull, 2023) and 63% of learners in Grade 5 have no basic mathematical knowledge (HSRC, 2020). These educational deficits compound as learners move through the education system and contribute to South Africa’s high youth unemployment rates (Click Learning, 2024; O'Neill, 2024).
Given the complexity and magnitude of the learning crisis, traditional change efforts have often fallen short. Educational technology (edtech) has emerged as a promising solution to enhance learning outcomes and promote equity at scale. However, there is limited evidence regarding which programs are truly effective, for whom, and under what conditions. Although some intriguing trends highlight emerging best practices—like supplemental or after-school interventions and adaptive or personalized technologies—there are also common pitfalls. These include a mismatch between products and learners' needs, particularly concerning language, as well as various contextual factors that can impede the effectiveness and sustainability of edtech interventions.
These considerations are particularly vital in South Africa, where schools serve multilingual learners and grapple with infrastructure and human resource constraints. In this complex landscape, organizations like Click Learning, a South African NGO, play a crucial role in delivering locally led and contextually relevant solutions.
Click Learning's mission is to equip learners with the foundational literacy, numeracy, and digital skills necessary to build sustainable livelihoods in the future. Employing a holistic approach, Click Learning offers underserved schools access to top-notch digital learning programs, along with vital infrastructure (such as hardware, connectivity, and backup power) and essential human resources (including lab facilitators). By facilitating the integration of edtech in low-resourced schools, Click Learning is paving the way for thousands of children to succeed in school and beyond.
In the last ten years, Click Learning has collaborated with more than 340 schools, benefiting 230,000 literacy learners and 50,000 numeracy learners. Inspired by findings from a recent study indicating a significant link between enhanced learning outcomes and learners' cumulative time spent on Click Learning programs (Firdale, 2022, p.13), Click Learning partnered with a team of LEAP Fellows to investigate the most effective ways to structure learners' usage for optimal gains. This investigation aimed to determine the minimal or optimal time learners need to meet global proficiency standards in literacy and numeracy by Grade 3. Such insights would enable Click Learning to estimate the ideal balance between the number of sessions and the time spent per session and ensure that the time allocated for Click Learning results in high-quality learning.
During a three-month sprint, the team conducted a literature review to understand the evidence base related to edtech dosage and learning outcomes. After reviewing several studies, including multiple meta-analyses, the evidence remains limited, particularly within the South African context, regarding the impact of dosage, intensity, or duration of use on learning outcomes. This underscores the importance of studies like the present one.
The team also explored the data available for two of Click Learning’s programs, Reading Eggs and Reading Eggspress, examining associations between students’ use of the platform and their gains in reading comprehension from one year to the next. Below is a summary of our findings:
The study uncovered significant variability in weekly logins and platform usage among individual students, between students, and across schools, highlighting inconsistent patterns likely influenced by both internal (e.g., student preferences) and external (e.g., school policies) factors. Such variation can be leveraged in analyses of the links between time spent and performance gains, but variation driven by students indicates that confounding variables related to both time spent and performance may bias such analyses.
In non-regression-adjusted results, platform usage variables from Reading Eggs were negatively associated with reading comprehension scores from Click Learnings’s equiz, whereas those from Reading Eggspress showed positive associations. These perplexing results may stem from outliers or students accessing the platform beyond their grade level. For instance, our analysis focused on grades three through five, although Reading Eggspress targets grades six and seven. This suggests that students using Reading Eggspress in lower grades may be unusually proficient, or Reading Eggs might be too elementary for them. Further investigation is necessary for clarification.
There was substantial agreement between the data mined Reading Eggs time-on-platform variable and Click’s previously calculated platform time variable. However, this agreement was not found for the time-on-platform variable in Reading Eggspress. This suggests that an aggregated variable, such as the previously calculated one, may not capture the nuances in the time spent on specific programs within Click Learning.
In the regression-adjusted results, time on either Reading Eggs or Reading Eggspress was associated positively with score on the external reading comprehension test. For both programs, when students spent a standard deviation more time on the program each week (approximately 10 minutes per week), they gained one-fifth of a standard deviation (0.20) in reading comprehension score. This is a meaningful effect size in the context of reading instruction within South Africa.
From these analyses, it appears that for this sample, longer sessions are associated with greater gains in reading comprehension. For both programs, the time per login variable was a positive statistically significant predictor of reading comprehension score, with a beta of 0.07 for Reading Eggs and 0.11 for Reading Eggspress. Examining this balance of number of logins by time spent through an interaction term, the interaction of time by logins was only statistically significant for the Reading Eggspress model. In these data, it appears that the association between time spent on Reading Eggspress and reading comprehension score is stronger for those students who have fewer logins. This indicates that students should be given more time per session to engage with Reading Eggspress. These results should only be interpreted as preliminary given that there were a number of extremely short logins (less than two minutes) that may be skewing results.
Based on insights gained during this sprint, and informed by the literature reviews, the report offers customized recommendations related to:
the data collection and analysis of Reading Eggs and Reading Eggspress;
further exploration of the connection between dosage and learning outcomes;
the broader language and learning context of the Click Learning program; and
enhanced program implementation and evaluation.