Essential Indicators

A mixed expert/crowd knowledge-base system that helps educators identify and support children at-risk of educational, socioemotional and safety issues

Educators in low resource schools lack simple and user-friendly means to identify children at risk of educational, mental health and personal safety issues. At the same time, they also lack access to best-practice knowledge on how to approach the problem at hand. To address these issues, we are developing a low-tech software solution to support educators in assessing the drop-out risk of their students. We therefore want to use drop-out as a proof-of-concept and afterwards extend our solution to a much broader range of problems.

The tool will accept quantitative and narrative descriptions of the student's essential indicators necessary for providing suggestions for an intervention. Based on the risk assessment, the tool will recommend evidence-based action points based on a pre-trained expert knowledge database. Additionally, it will collect and distribute crowd-sourced ideas from local users, thus empowering them to identify their own intervention approaches. By eliciting the success of the intervention afterwards, the systems will self-learn over time what works in any given context.

The collection of successful interventions can be leveraged by stakeholders at all levels, school leaders at the school level and policy experts to design interventions at the country level to reduce drop-outs, improve learning outcomes subsequently.

Our tool is based on a state-of-the art natural language processing (NLP) artificial intelligence diagnostic system combined with knowledge provided by (a) world-class domain experts  and (b) local end-users themselves. The tool is based on a pre-trained model linking narrative descriptions of a problem situation with evidence-based interventions tailored to the problem at hand. Similar systems exist and have been successfully implemented, for instance, in supporting experts in medical diagnostics (e.g., Qui et al., 2020) and treatment prediction (e.g., Chien, 2018). Most importantly, first evidence suggests that such systems can work well in multi-language, noisy environments (a self-learning robot recognizing noisy speech in Indonesian classrooms; Andreas et al., 2019), and readily linked with speech-recognition systems. This will create a low-tech user-interface (mobile phone) for low-cost settings. Suitable NLP models will be used to classify problems based on scientific evidence, and the system will elicit missing information necessary to make a good diagnostic and prognostic decision. Initially, domain experts will provide evidence-based suggestions for possible interventions based on best-practice in the field. Later on, the system will self-learn about the most successful interventions. 

Drop-out has been extensively studied across many countries and by multiple agents (e.g., universities, NGOs, governments, Ripamonti, 2018). Most research centers on aggregated causes; data on individual risk and context-specific ways to mitigate drop-out are missing. Our tool will pioneer in collecting such data and build models identifying optimal, context-specific solutions for problem behaviors.

Evidence-base, on drop-out specifically educational interventions generally, will combine insights from machine learning, child development, school performance, neuroscience and behavioral sciences, and resilience literature. Literature on these aspects exists but has never been synthesized, to improve practice. Also, most evidence, e.g. on the development of resilience, focuses on developed countries. Machine learning will ensure generalizability of existing solutions to low-income, context-specific (language, culture) areas.

The core innovation of our solution is the ability to provide access to best-practice, evidence-based expert knowledge to low-resource low-tech settings that would not have had such access without our tool. By using a natural language user interface (voice recognition on the phone, chatbot in a mobile phone messenger etc.) the system allows to keep tech requirements and education requirements on the side of the user very low but still connects the user to a most sophisticated, state-of-the art expert system. Furthermore, it connects global and local knowledge by collecting and empirically validating solutions suggested by local users, which makes it a evidence-based research tool itself rather than a sophisticated way of only accessing information. Finally, the NLP/big data approach makes it easily scalable to multiple settings and various problem behaviors.

Not enough information on causes of drop-out at disaggregated (learner level) exists at present. This needs more data that may contribute to a better understanding.

If more learners remain in school, supported by empowered teachers and addressing learning (academic) and socio-emotional needs, it can significantly enhance the life chances of those learners currently most at risk of not successfully completing schooling. It can also assist them to develop the academic and personal resilience to become active, productive and responsible citizens.

The tool will also form the starting platform to enable similar insights and interventions around behavioral challenges and development of higher order (21st Century) skills.  

Children drop-out of school due to push and pull factors because they are unable to mitigate the risks. It is possible to identify and assist learners through data driven heuristic assessments by teachers (observations), using expert knowledge from analysis of existing data and research findings, enriched through crowd-driven data collection (mostly teachers). Where possible, applying machine learning to refine and develop contextual solutions. Teachers are the first line of support but currently are not qualified or empowered to successfully intervene. 

Assumption: Informed and supported teachers with agency can assist at-risk children in multiple ways to develop their academic and personal resilience, to stay in school, on track and achieve good educational outcomes. Furthermore, educators need simple technology to collect learner data that informs an early warning system. It enables them to assess the risk levels of students on various indicators. They can then identify and plan various interventions to support at risk children.  

Existing data: Datasets of existing causal factors to inform the heuristic tool for teachers. Data include administrative, educational performance, psychometric, disciplinary and developmental data. Much of this is in EMIS systems, even in developing countries.   

New data: The application enhances existing data with data on aspects of risk, in particular socio-emotional progress. This enables development of contextual heuristic knowledge, tools and interventions for teacher intervention. Data to be collected and analysed include absenteeism, behaviour changes, academic performance, disability, socio-emotional progress. Most of this is not collected regularly and comprehensively.  

Outputs: (1) Interactive information sharing with teachers in real-time; (2) Data reports on collected data informing ongoing R&D;  (3) Insights reports on drop-out trends and other developmental indicators, e.g. learning; (4) Possible teacher training - blended/ in person; (5) Possible hard copy heuristic templates . 

Outcomes: Reduced drop-out rates, improved educational outcomes (academic and personal)

Over the next year, we aim to develop the software, pre-train the NLP system with domain experts based on about 5,000 vignette scenarios and pilot the system in a small sample of about 20 low resource schools in India. We would want to use this period to identify the support mechanisms for the effective use of the product by the educators and also develop monitoring and evaluating systems to track the impact of the product we developed. Main goals for the accompanying research during this period would be to test the predictive accuracy of the NLP system and how well it can identify and elicit from the user information that is missing in the diagnostic/prognostic process. 

Based on the learnings from our work in India, in year 2 and 3, we aim to pilot in Kenya and South Africa where our team members have a presence. Overall we aim to use the first three years in developing a robust product that is adaptable in different contexts.

In years 4 and 5, we aim to reach about 1,000 schools in low resource settings across the world through partnerships with Global School Leaders, Global Schools Forum and other international organizations working in low and middle income countries.

A huge advantage of the proposed system is that it is easy to scale up for a large number of schools and a variety of problem behaviors. 

Operational: We will have to identify how to equip low resource schools to have the capacity to interface with the software. We will have to deal with educators' motivation to engage in the process considering that most school leaders and teachers are not equipped in dealing with at risk children.

Scale: We will have to develop a model that is easily adaptable in different contexts and therefore be able to build a flexible IT product that can suit different contexts.

Sustainability: A key challenge to address to make our solution sustainable is figuring out how to integrate our solution in the everyday practice of the educators so that they do not see this as an additional burden.

Partnerships for scale: We need access to interested stakeholders willing to adopt the proposed solution and to support the development process.

Operational: Low-tech solution allows access to contexts most in need for support; systems will be set up in a way that is meant to support educators and not adding burden to their everyday work. 

Scale: We will put in resources to ensure that the expertise and knowledge is generalizable across  different contexts- potentially pairing up with experts and think tanks to conduct separate pilots in different countries

Sustainability: We will use existing knowledge and strategies to establish a way to engage the education authorities to get their buy in

Partnerships for scale: Ministries of Education, UN Agencies, Universities, NGOs, software developers, citizens’ groups, etc. would all be interested in this once proof of concept has been achieved.