DEEP - Developmental Assessment on an E-Platform

Approximately 250 million children below 5 years of age in low and middle income countries are at risk of not attaining their developmental potential. Currently, child neurodevelopmental assessments are expensive, administered by skilled specialists or dependent on parent reports that differ based on parents’ knowledge of child development, thereby limiting their scalability and resulting in a huge ‘detection gap’. This leads to lost opportunities for early interventions known to improve outcomes. Our innovative solution titled “Developmental assessment on an E-Platform (DEEP)” is a gamified cognitive assessment tool for preschool children. It comprises direct assessment of children through gamified neuropsychological tests validated against a gold standard, with potential to be administered by non-specialist workers (NSWs) and, in the long term, parents themselves on mobile devices. When implemented globally, DEEP has the potential to drastically reduce the detection gap by aiding timely referral of children with faltering development to effective interventions.

There is increasing global emphasis in making sure that children thrive and reach their full developmental potential. To thrive, children need a nurturing environment but unfortunately, millions of children under 5 years of age are instead exposed to early life adversities that significantly impact their development. Unsurprisingly, LMICs bear the bulk of the global burden of children at risk for developing sub-optimally (250 million) with India alone contributing 65 million to this metric. This disadvantage manifests as a lack of readiness for school. The Annual Status of Education Report published by a leading Indian education non-profit has shown in 2014 that while most children enrol in primary school, a quarter of them have educational outcomes way below their grade level. Most of these children don’t get identified as faltering in their development because currently child developmental assessments can only be done by specialists who often reside in urban areas and use expensive and lengthy observational tools or is based on parent-report which differs based on the parent’s own knowledge of child development. This results in a huge ‘detection gap’, and a tragic missed opportunity to intervene when children are young and their brains are most plastic and amenable to change.

The direct benefit of DEEP is to families from low income settings with no access to neurodevelopmental assessments of their children. Our team has developed DEEP through extensive formative work done in a rural north Indian community called Rewari (see Bhavnani et al 2019 Global Health Action). Its development was informed by feedback from (1) NSWs who administered DEEP on children from this community and (2) from parents of these children. We used children’s performance on the game to develop its design and content so as to keep it locally relevant, while using universal images to enable global scalability in the future. Children identified to be faltering in their development would gain access to facilities that deliver interventions to marginalized populations (in India through the district early intervention centres of the Rashtriya Bal Swashthya Karyakram scheme of the Government). DEEP can also help identify communities at-risk of suboptimal neurodevelopment to institute community-based solutions (e.g. interventions based on the WHO Nurturing Care Framework targeting the first 1000 days of life). In the longer term, children identified through DEEP and receiving early interventions would benefit in academic and professional performance and help break the vicious cycle of intergenerational transmission of poverty. 

Our solution, DEEP – Developmental assessment on an e-platform, has been designed to assess cognitive abilities in preschool children through tablet-based gamified neuropsychological tasks. It has been validated against a gold standard assessment on a sample of 3-year old children in rural India and efforts are underway to expand its target age range and enable drawing of longitudinal developmental trajectories similar to growth charts. We are also validating it in diverse settings, both in India and Africa.

DEEP comprises discrete games with increasing levels of difficulty, woven together through a first person narrative to optimise child engagement and thus performance. While its content and design have been locally informed through formative work in a rural Indian community, it has intentionally been designed to be universal through use of popular imagery like monsters, moon and planets. It has been designed to be modular such that games can be added or removed based on the age of the child. It is a mobile application can be administered offline, on low-cost Android tablets in the comfort of a child’s home. DEEP is free of written instructions and instead contains a demo mode in which game instructions are conveyed to the child while demonstrating how to play, and the play mode in which child performance is recorded. There are rules to start and stop each game which are either built into the game itself or incorporated into its administration manual to minimise need for the person administering the tool to exercise judgement.

Data derived from a child’s interaction with DEEP is stored locally on the device and uploaded to a cloud server when connected to the internet. This data is processed using advanced analytics (machine learning) to predict child performance on a gold standard developmental assessment. Currently, in LMICs, neurodevelopmental assessments of children in the crucial preschool years are dependent on parents noticing a clinical need in their children and either reporting it to health workers through answers on a questionnaire or taking them to facilities for detailed evaluations. This often results in only severe cases being identified and mild/moderate ones being missed. Harnessing mHealth technology allows DEEP overcome these limitations in child developmental assessments by a) empowering NSWs within public health systems with an easily administered tool and b) in the long term, empowering parents themselves to use an evidence-based engaging game to track their child’s developmental trajectory.

Our key innovation lies in the use of mobile technology to assess child development in low-resource settings which allows for the following improvements –

• Direct assessment of child performance instead of relying on parent-report or observations by trained NSWs as is the case with existing tools
• Use of gamified neuropsychological tests which makes the tool highly acceptable and engaging to the child, improving data quality over traditional paper-pencil tests
• Ability to administer a child developmental assessment in the comfort of a child’s home instead of a clinic
• Ability to conduct child developmental assessments at scale in low resource settings by empowering non-specialist workers and parents with an evidence-based tool that is easy to administer
• Digital storage of backend game data and analysis using machine learning algorithms. This is ideal for big data analysis, iterative improvements in prediction accuracy, and improving generalizability as more data from diverse settings become available
• Predictions of a gold standard measure of child development as an outcome measure is also novel, as other similar tools quantify simple measures such as accuracy and reaction time to judge child abilities, without reference to any gold standards
• The modular nature of the tool allows flexibility to add new assessments based on findings of the research from diverse settings, locations and populations.

Overall, DEEP represents a scientifically validated gamified assessment of cognitive development that is conducive to task sharing, a concept promoted by the World Health Organization for last mile reach. 

Our solution utilises 2 core technologies –

• Mobile application: DEEP is a gamified assessment of cognitive abilities designed as a mobile application be delivered on Android tablet-computers. Tapping into this technology for child developmental assessments is novel and integral to conducting them at scale. It is also essential to be able to collect digital data reflective of child performance on developmental assessments tasks which makes it amenable for use in advanced analytics as described in point 2.
• Artificial intelligence: digital data recorded during a child’s interaction with DEEP is processed using a machine learning approach in which a combination of features are used to predict child performance on a gold standard developmental assessment.

Both these technologies are essential to achieving our long term aim of administering such an assessment at scale on multiple devices at multiple locations thereby empowering non-specialist workers in public health systems of LMICs and parents themselves to test cognitive abilities of children. Using cloud computing and big data analytics will allow for continuous updating and improvement of global developmental norms, leading ultimately to norms analogous to those used for anthropometric outcomes. The development of such global norms for neurodevelopment is essential to the aim of identification and interventions for children faltering in their developmental trajectories. 

DEEP’s direct impact will be to identify children faltering in their developmental trajectories and refer them to timely effective interventions. Families availing these services will benefit in the long-term through improved developmental outcomes of their children in preschool years, better educational achievements in school and higher earning potential in adulthood. Critical to this impact is evidence of DEEP’s a) acceptability as a child developmental assessment tool to children and their families, b) feasibility for administration by NSWs and parents and c) ability to accurately identify children faltering in their development. We have pilot tested DEEP on 3-year old children in rural Indian settings. In-depth interviews with mothers of these children has shown it to be a highly acceptable assessment. We have also demonstrated feasibility of trained NSWs (similar in qualifications to front-line workers in the Health and Women & Child welfare sectors of the Indian public health system) delivering the assessment in rural households (see Bhavnani et al 2019, Global Health Action). Finally, our pilot shows that DEEP is able to predict child performance on the cognitive domain test of Bayley’s Infant and Toddler Development (BSID-III), a gold standard developmental assessment, with a high accuracy (manuscript in preparation). Our current ML algorithm allows us to accurately identify poor BSID performers with a sensitivity of 77% and specificity of 67%. Given these pilot results, we are confident that DEEP will be instrumental in significantly reducing the detection gap and thereby helping children attain their full developmental potential.

DEEP has till date served 1369 families of 3 years old children residing in Rewari, Haryana in north India on whom it has been administered between January 2018-March 2019. It has been validated against a gold standard developmental assessment in 3 year old children and we have received funding to expand its scope by a) validating it in older children and following up the same population in Rewari over the next 3 years and b) validating it in urban settings in India and Malawi across the preschool years.

Thus, through these two overlapping projects, within the next one year i.e. by May 2020, we expect to serve the same 1369 families as their children turn 4.5 years old.

Within 5 years, we expect DEEP to have served approximately 7000 families by assessing their children on the key neurodevelopmental domain of cognition.

If we receive funding for an additional proposal which is being considered by a funding agency, the number of children served using DEEP would increase to 12000 in the next five years.

Having demonstrated validity of DEEP to predict child performance on a gold standard developmental assessment at 3 years of age and identify poor performers relatively accurately, we have the following goals for the next year –

• to adapt and validate DEEP for use in a wider age range encompassing the preschool age (2-6 years)
• to purposively sample 3-year old children likely to belong to the extremes of the distribution of BSID performance to improve our predictions errors

Over the next 5 years, we intend to achieve the following goals –

• demonstrate evidence of DEEP’s ability to assess neurodevelopment in a wide range of preschool children and accurately identify children faltering in their development
• demonstrate evidence of DEEP’s sensitivity to changing developmental abilities as children grow in the preschool years i.e. its ability to draw longitudinal trajectories of neurodevelopment
• demonstrate global scalability of DEEP by its successful use in a) Indian and African settings, b) rural and urban households and c) community centres such as preschools and clinics
• demonstrate evidence of DEEP’s ability to integrate with existing Indian governmental services through the successful completion of a scaling pilot
• Once validated in diverse age groups and geographies, make DEEP available for downloads from the Android Playstore, to assess child development from the wider community by empowering parents to administer it on their own children

Below are some key barriers to our long-term aim of impacting the lives of millions of children from low resource settings who might be at-risk of not attaining their full developmental potential –

Financial: Demonstrating DEEP’s ability to integrate with existing Indian governmental services is contingent on funding of a scale-up study to achieve our fourth five-year goal  

Technical:

a) Expanding the generalisability of DEEP which has currently only been piloted and validated on a narrow age-range of typically developing children within a single north Indian rural setting.

b) Access to cutting-edge data analytics and cloud computing pipelines that will be required once DEEP is taken to scale

c) Networking with government, other civil society and international bodies that will be integral to scaling DEEP globally

Infrastructural: A lack of effective interventions in most low income settings to which children can be referred at scale with an assurance of improvements in their developmental trajectories and the sudden increase of burden that this would cause on existing public health systems in these settings poses a major infrastructural barrier to achieving our ultimate aim of improving child outcomes. 

Financial: we have applied for funding for a scaling pilot, result is awaited.

Technical:

a) Generalisability of DEEP – we are expanding the age range and diversity of the population on which DEEP is validated through administration in Malawi in an ongoing study. In the long term, we expect both our expanded network (see point b below) and parents themselves to be able to contribute data to improve DEEP’s generalisability.

b) We intend to expand our national and international network to include 1) big data experts who will enable creation of an automated cloud data computing pipeline, 2) child development clinics that will provide regular access to typical and atypical children on whom DEEP can be further validated and 3) other researchers in LMICs interested in enhancing early child development that will demonstrate the global utility of DEEP.

Infrastructural: we will refer children identified to be faltering through our projects to interventions administered through community based district early intervention centres and top paediatric units in tertiary care hospitals with whom we already collaborate. Families of these children will receive training on an adapted module of the WHO Caregiver Skills Training. In the long term we will partner with international organizations that deliver ECD interventions to enable building referral pathways across the globe for children who are identified by DEEP. In parallel, our group at Sangath is working to develop and test the efficacy and cost-effectiveness of NSW delivered universal and targeted interventions to enhance early child development.

Our solution team is based at Sangath. Sangath is a non-governmental, non-profit organization committed to improving health across the lifespan and develops models of care that can be scaled up through government machinery. Its pioneering strategy has been to use task-sharing which utilises relatively low-cost human resources, by empowering ordinary people and community health workers, to deliver mental healthcare with appropriate training and supervision from experts. Its primary focus areas include child development, adolescent and youth health, and adult health and chronic disease. Sangath was awarded the prestigious MacArthur Foundation Prize in 2008 and the WHO Public Health Champions Award for India in 2016. In partnership with neuroscientists, clinicians and data experts at Harvard Medical School, Public Health Foundation of India and Sapien Labs, researchers at Sangath have been the key scientific personnel who have led the creation and validation of DEEP. Sangath has also led the data collection in the field and helped keep the solution locally rooted since it has been developed and tested in a community in which Sangath has had a presence since 2014.

Our team, spread across partnering organisations Sangath, Harvard Medical School, Public Health Foundation of India and Sapien Labs, comprises the following staff –

1. Senior researchers (part time) – 3

2. Mid-career researchers – part-time 4, full-time 1

3. Administrative and other support staff (full time) – 3

4. Field staff – full-time during data collection periods – varies from 8-16

Our team is part of an international and interdisciplinary Translational Neuroscience Platform, a research consortium involving academics from premiere institutions of public health, clinical research, neuroscience, and computer science based in India, USA, UK and Malawi with the common goal of improving early child development and is thus ideally placed to solve this problem. Vikram Patel, the lead PI, is the Pershing Square Professor in Global Health in Harvard Medical School. His research covers three major themes: (1) generating policy relevant evidence on the burden and impact of mental disorders; (2) developing and evaluating mental health assessments and interventions for delivery by NSWs; and (3) communicating research to diverse audiences. His expertise in global mental health ensures our solutions have the vision of global scalability. Gauri Divan, Director, Sangath Child Development Group, is a developmental pediatrician and has served on multiple national and international expert and technical resource groups related to developmental disabilities in children. She is also a member of the recent Lancet Commission on the Future of Care and Research in Autism and brings her clinical expertise in early child development to the team. Tara Thiagarajan, Sapien Labs, is a scientist and entrepreneur with an interest in understanding and enabling the productive evolution of the human mind and human systems. She brings her expertise in handling big datasets and complex data analytics to this work. Mid-career researchers on our team include neuroscientists, psychologists and data analysts.

We are partner with three organisations within the broader Translational Neuroscience Platform:

1. Harvard Medical School – Being Prof Patel’s host institution, it provides us access to a world-class academic researchers with a particular focus on research on child health through its Centre for the Developing Child. Students at Harvard have assisted us in analysing our pilot data using a machine learning approach.

2. Public Health Foundation of India – PHFI is a non-profit organisation that works closely with central and state governments to enhance the capacity of the existing public health workforce, evaluate governmental projects and schemes, and gather evidence to inform policy and we expect to tap into this network to take DEEP to scale.

3. Sapien Labs – Sapien labs is a non-profit research organization focused on understanding the diversity of human brain dynamics across the planet. Scientists at Sapien Labs bring their extensive expertise in managing big datasets and advanced data analytics to this partnership. Their approach to big data analysis, initially created for EEG data analysis, has been instrumental in shaping our thinking of how to analyse game data that is derived from child performance on DEEP.

The key beneficiaries of DEEP are low income families who will gain access to a developmental assessment of their children during the formative preschool years. Our measure of impact on these families will be the timely and accurate identification of children faltering in their development in the short-term and their improved educational outcomes through utilisation of effective interventions in the long-term. An additional impact of our solution will also be to enable identification of communities at-risk of poor developmental attainment and thereby allow for efficient allocation of limited governmental resources to institute community-based solutions (e.g. interventions based on the WHO Nurturing Care Framework targeting the first 1000 days of life) and monitor the implementation and efficacy of these interventions.

Given our solution will ultimately be made available open source, we expect partnerships with other national and international non-profit organisations, child development clinics and state and central government schemes aimed at improving early child development to be the channels through which we achieve scalability. 

Since DEEP is intended to be an open access tool, we plan to sustain it through grants and CSR funding in the initial stages. We have demonstrated the ability to garner such funding through a) continued relationships with our CSR funder and b) through the award of an internationally competitive grant from the Medical Research Council, Government of UK. We intend to utilise such sources of funding to first accumulate evidence of its validity and utility across geographies and age groups, and then conducting scaling pilots in which DEEP is embedded within the health system and administered by existing front-line workers. This will help us overcome implementation challenges which we can expect to encounter when scaling up. In the long term, we expect to achieve implementation at scale through partnerships with the Indian and international governments. This will be facilitated by the membership of our lead researchers in policy-making bodies nationally (e.g. National Health Mission, Rashtriya Bal Swasthya Karyakram), and internationally (WHO High-Level Independent Commission on NCD). We hope to incorporate DEEP into national policies (as a standard for identification of children with neurodevelopmental delays). DEEP is already aligned with the goal of these policies and will serve as a facilitator to overcome the challenges that hinder their effective implementation. Further, empowering parents to download DEEP from their mobile phone App Store and monitor their own children’s development will also allow us to ensure long-term sustainability.

We are applying to Solve to specifically address our technical and infrastructural barriers of a) increasing DEEP’s generalisability by generating global evidence of its ability to assess cognitive abilities in diverse preschool children from diverse settings, b) gaining access to leading big data experts and c) networking with international organisations to sustainably scale DEEP. We believe Solve will be instrumental in helping us create the partnerships that are essential to overcoming these barriers. We expect to gain exposure through presenting at the Solve Finals and at the workshop at MIT. We also expect Solve’s social media reach to benefit us greatly. We expect being a part of Solve’s network to also give us access to information about other international social impact conferences and events for networking. In addition to networking and creating partnerships, we expect interactions with other groups working in the space of enhancing early child development to help us learn from their experiences and exchange knowledge on best practices. Further, mentorship from leaders in the field of early child development will help ensure that our aim and approach is aligned with that of the best academics in the world.

We are looking to partner with the following organisations in low and middle income countries, or low resource settings within high income countries –

a) child development clinics. We would like them to administer DEEP longitudinally through repeated assessments on typical and clinical populations of children in their clinics and thereby generate global norms for development in preschool children.

b) those that are implementing effective interventions to improve early child development. We would like to partner with them to i) administer DEEP as an outcome measure to test its sensitivity to change brought about by effective interventions and ii) enable building referral pathways across the globe for children who are identified by DEEP as needing follow-up assessments and interventions.

In addition, a crucial partnership we seek is with big data experts that have successfully created and implemented pipelines for collection, cloud storage, analytics and report generation of global datasets. We would like to partner with them to a) develop a pipeline that would enable us to refine algorithms and improve DEEP’s prediction and precision as big data becomes available, b) investigate novel and cutting-edge methods to analyse our larger complex dataset which includes multiple technologies including gamified tests, EEG and eye tracking and c) generate growth charts containing trajectories for neurocognitive development across preschool years.

The use of artificial intelligence is integral to the analysis of data collected by DEEP and allow for utilisation of complex data science to improve its precision and predictions. If awarded this prize our team would utilise the money to improve the scalability of our solution. Specifically, we would a) enable it to push data automatically to a cloud server (currently the data is stored locally on the device and manually transferred to a server), b) tag each dataset with the identity of the tablet/device on which it has been collected, c) tag each dataset with the GPS coordinates at which the data has been uploaded, d) create an automated data analysis pipeline which would generate reports upon data being uploaded, e) create a version for use by parents that can be downloaded from any Android Play Store. These enhancements are critical to DEEP being scaled globally and data quality being closely monitored in a timely manner.

This would fit neatly within our larger program of work which, by the end of the next 5 years, will deliver 1) a scalable platform to assess key neurodevelopmental domains using mobile technology and machine learning, 2) normative data on these domains across multiple low-resource settings, 3) evidence of its clinical utility in assessment of neurodevelopmental domains at a population level in community settings, 4) evidence of feasibility of use by non-specialist workers and parents and 5) evidence of its potential to impact the lives of millions of young children globally.