CEPE Di Tella

Based at the School of Government at Universidad Torcuato Di Tella, the Center for Evaluation of Policies and Evidence-Based Policy-Making (CEPE Di Tella) specializes in conducting applied research that seeks to improve the quality of public policies through the provision and promotion of evidence-based initiatives and rigorous evaluation methods. Since it’s been founded in 2017, CEPE Di Tella and its affiliated researchers have been assigned to conduct several research projects funded by international organizations (World Bank, Inter-American Development Bank, UNICEF). Furthermore, it has been awarded several grants from philanthropic and private foundations (BBVA Foundation) and worked in partnership with international research institutions (Center for International Development at Harvard University). 

CEPE Di Tella’s Education and Behavioral Sciences Program focuses on the analysis of educational policies using social, political, pedagogical, economic, contextual, behavioral, and neuroscientific determinants of learning using applied research quantitative and qualitative methods. 

Our program focuses on researching learning and teaching capabilities across the lifespan, with a particular emphasis on populations that may lack equal resources and abilities. Our primary objective is to foster equity and provide equal opportunities for all children.

Since 2017, we are working along with schools from Argentina oriented towards enhancing the education of Deaf and Hard of Hearing (DHoH) children in order to develop the Learning to Learn Program (LeLePro), an evidence-based intervention program to improve these children’s linguistic abilities and provide them better educational opportunities. Our studies  target  children between the ages 3 to 12.

LeLePro involves the evaluation of different cognitive abilities with a software suite. DECILE screens oral language comprehension and production, while MetaTom assesses metacognition and Theory of Mind (ToM). The software also collects demographic, environmental and aetiological variables that may affect linguistic attainment. After initial screening of children, we seek to implement an intervention program based on neuroscientific evidence to improve the development of linguistic abilities, which are crucial for mental health, and the development of other cognitive skills.

Carolina Gattei, Team Lead of DECILE, is currently working as a Associate Professor in the Education Area at the School of Government, Universidad Torcuato Di Tella. Her research program, titled "Language and Policy: Tools for assessing and enhancing populations with language difficulties", focuses on creating evidence-based linguistic assessment tools for underserved groups, particularly young children.

At present, she holds the position of associate researcher in the Education and Behavioral Science Program at CEPE Di Tella, where she serves as a language expert consultant. As part of the Learning to Learn Program, she is leading the design and implementation of the Digital Device for the Evaluation of Communication during Childhood in Spanish (DECILE). She also collaborates in coordinating other research projects aimed at the Deaf community in Argentina, and leads a project that assesses narrative production in children from economically disadvantaged families in Mendoza, Argentina.

In her role, Carolina is responsible for translating the education sector's needs into practical and cost-effective research projects. She engages with public officials from the education sector and seeks funding sources to achieve the proposed objectives of these programs.

LeLePro is a collaborative research program that brings together expertise from various fields including experimental psychology, linguistics, education, and more recently, artificial intelligence. The program's primary objective aligns with the research interests of its three main leaders: Dr. Cecilia Calero, Dr. Carolina Gattei, and Dr. Agustin Gravano.

Dr. Cecilia Calero serves as Associate Professor at Torcuato Di Tella. Her primary focus lies in comprehending the underlying mechanisms involved in teaching and an efficient transmission of knowledge among people, in scenarios such as peer tutoring. As the principal investigator of LeLePro, she heads the development of MetaTom, a software designed to assess metacognitive and Theory of Mind skills in children facing language difficulties.

Dr. Carolina Gattei takes charge of the development of DECILE, a software aimed at evaluating language skills in Spanish-speaking children with communication difficulties. She also plays a crucial role as a language specialist and serves as a liaison between the academic sector, and schools catering to Deaf children and children with communication challenges. Additionally, she actively contributes to the development of MetaTom. 

A dedicated team of PhD and M.Sc. students is actively engaged in conducting field studies to develop LeLePro. Anali Taboh, a PhD student, plays a vital role in coordinating in-schools data collection for the DECILE aspect of the project, as well as overseeing the work of research assistants.

Heading the B.Sc. in Digital Technologies and the Artificial Intelligence Lab at Universidad Torcuato Di Tella, Dr. Agustin Gravano brings his expertise in machine learning to investigate how humans engage in oral conversations and enhance computer-based dialogue systems. Currently, he leads a research project that explores the performance of Spoken Language Processing tools  when exposed to oral input from children equipped with cochlear implants. Within the LeLePro framework, his aim is to model individual variations in children's language abilities and advance the development of AI-based tools that can provide valuable support and recommendations for educators and healthcare practitioners.

With a deep-rooted passion for scientific inquiry, these  researchers are driven by the power that evidence-based actions may have on the educational field. They are determined to make a substantial contribution that will empower children to flourish and reach their full potential. Together, they aspire to forge a new path that combines interdisciplinary knowledge and cutting-edge technologies to create innovative solutions that will revolutionize the way we support children facing linguistic and communication difficulties.

A language assessment app for Spanish-speaking children facing linguistic challenges.

According to the World Health Organization, approximately 430 million people worldwide suffer from incapacitating hearing loss. In Argentina, nearly a million individuals have permanent hearing limitations, including 64,000 children aged 0 to 14.

Hearing aids and cochlear implants (CI) have significantly improved the hearing abilities of deaf individuals, enabling many children to acquire spoken language skills with appropriate stimulation. However, hearing through a CI differs from typical auditory development, requiring greater effort to comprehend linguistic sounds. 

In Argentina, CIs are usually implanted after the age of one, or even later, due to a lack of information among parents and unstable healthcare policies influenced by budget constraints. As a result, Deaf and Hard of Hearing (DHoH) children whose families do not use sign language (representing about 95% of them), are not exposed to oral language until a later stage in childhood. This delay negatively impacts their linguistic, educational, emotional, and social development.

From a linguistic point of view, DHoH children who exclusively access oral language often face difficulties in morphology and syntax, showing delays in utterance length development, omissions or incorrect usage of determiners, prepositions, and inflectional morphology when compared to typically developing children. They also struggle with understanding and producing syntactically complex sentences.

The proposed solution takes into account two significant gaps in the existing literature:

Firstly, although there are numerous studies available on other languages, evidence regarding linguistic attainment of Spanish-speaking deaf children who use CIs and hearing aids is limited, and most of it comes from data obtained in high-income countries. 

Secondly, within the DHoH children’s group, a great heterogeneity in performance is usually reported, along with differences in children’s health history, socio-economic and linguistic background. Yet, the connection between these sources of heterogeneity and variations in performance remains poorly understood, and educational intervention programs are typically developed based on protocols that assume homogeneity within the group. Consequently, healthcare and education professionals rely on published results from these programs to make recommendations, leading to an unfortunate chain of misinformed decisions for these children.

DECILE is part of a software suite targeted at children with language difficulties. The app measures children’s performance on several linguistic metrics (receptive vocabulary, accuracy and response time in sentence production and comprehension, accuracy in sentence repetition and intelligibility) together with information about external factors that may affect language development.  

Targeted at the educative community, our team has focused on establishing the key characteristics of our educational software, with a particular emphasis on:

1. User-friendly and intuitive interface

2. Engaging and interactive content

3. Flexibility and scalability

4. Accessibility and inclusivity

5. Data privacy and security

Working together with schools that target neurodiverse children, the app has adapted school-based experimental tasks to a software experience that is engaging for children and  fast and easy to use for language therapists, teachers and parents. Evaluation tasks are game-based activities that take between 15 to 20 minutes, with a main character that guides children along the different tasks. Educators and practitioners can use this app to gain insights on particular difficulties the child might have, compare performance across sessions, and guide their interventions accordingly.

Teachers and practitioners using the app first create a profile for the child, entering demographic and etiological data if available. They then select the task and can adjust parameters such as response time and the number of response opportunities provided. The child begins the task with the assistance of Popi, a child-guide who provides examples and feedback to ensure understanding before the screening begins.

DECILE is grounded on scientific principles yet offers flexibility. The tasks are designed based on psycholinguistic metrics of Spanish language development but can be adapted to users' needs by modifying input sound and image files. This feature makes it suitable for assessment in other Spanish-speaking countries interested in customizing aspects such as Popi's pronunciation. A demo of one of the comprehension tasks can be found here.

Following each session, all data is securely recorded and stored in a central database, including accuracy rates, response times, and the child's responses in production tasks. During the pilot phase, the collected data will be utilized to train machine-learning models to identify language learning profiles among children, enabling practitioners and teachers to implement tailored interventions effectively.

DECILE began to take shape after numerous meetings with school principals who serve DHoH children—an area of keen interest for our team’s research. They were concerned that existing language assessment tools were not specific enough and took too long. They also worried that these tests seemed too clinical, which could make them less appealing to children and affect the results. Additionally, they sought to understand if their intervention and education programs effectively addressed the issues they observed in their students but struggled to measure accurately. Since 2017, our team has been dedicated to collaborating with these schools, aiming to provide practical, research-backed learning solutions. Specifically, three schools focusing on oral education for DHoH students requested the development of a diagnostic tool for assessing their Spanish language abilities. Since then, we have conducted various research studies within schools, using methods from cognitive neuroscience to address questions such as:

• What communication strategies do DHoH children equipped with cochlear implants employ during pedagogical events and how do they differentiate from those used by children with typical hearing development? What comes first, the ostensive channel or the content of knowledge? 

• What has been described in the literature as typical comprehension patterns in orally educated children with prelingual hearing loss? 

• How do Spanish-speaking deaf children perform in sentence comprehension? And how do they perform in oral production?

• When assessing receptive vocabulary in DHoH children with cochlear implants who speak Spanish, how much weight should be given to their hearing age? 

• How effective are current Spoken Language Processing (e.g., Automatic Speech Recognition) tools when used by DHoH children, and how can they be improved to enhance accessibility? 

• What is the impact of bimodal bilingualism in adults’ performance in tasks related to Executive Functions? To further explore this area, we have engaged three bilingual schools (teaching Argentine Sign Language and Spanish) to participate in additional experimental studies taking place in 2024. Our objective is to understand the consequences of bilingualism on linguistic attainment in these children, considering the significance of Executive Functions for learning.

In 2022, our team received a grant from the Inter-American Development Bank to launch a project using Behavioral Science methods to enhance Policy outcomes. This project focuses on exploring how parents of Deaf and Hard of Hearing (DHoH) children in Argentina make decisions about their children's linguistic education. Our goal is to promote bimodal bilingual (Argentine Sign Language - Spanish) education for these children, encouraging sign language learning among parents, caregivers, and close relatives. The project is a collaboration with the Ministries of Education in Buenos Aires and Mendoza, and is currently being implemented in all public schools serving DHoH children in these cities.

The ongoing research we've conducted has led to the creation of DECILE. Originally developed for Deaf and Hard of Hearing (DHoH) children, our aim is for this tool to eventually benefit other communities dealing with linguistic challenges as well.

Our team's foundational studies focused on shaping DECILE with input from the Deaf and Hard of Hearing community. We aim to craft a tool that truly meets their needs, using evidence from experimental psychology and neuroscience research.

Our work revealed that DHoH children with cochlear implants rely more on gestures for communication when oral skills are lacking, compared to typical peers, even when sign language is not part of their linguistic repertoire.

We also found certain sentence structures pose challenges for Spanish-speaking DHoH children, causing a delay in learning compared to their peers. However, they eventually learn the most complex ones.

Moreover, our studies revealed that assessing DHoH children's receptive vocabulary using their hearing age yields higher performance rates than using their chronological age, and that all of these children’s linguistic capabilities should be assessed accordingly.

In examining bimodal bilingual speakers, we noticed differences in executive function performance, particularly showing better results in tasks requiring inhibitory control, when compared to unimodal (Spanish-English) bilinguals and monolinguals.

Finally, our investigation into Automatic Speech Recognition Tools for DHoH speech showed lower performance rates compared to standard levels, suggesting a real need for better tools for this underserved population.

Our primary challenge lies in catering to a diverse community that seeks advice from an assessment tool suitable for all. Although our current school-based studies yield insights into general language development outcomes within the DHoH community, they lack the statistical power required to provide fine-tuned recommendations for enhancing learning in this population.

 Strengthening the evidence base demands scaling our solution and identifying performance patterns influenced by multiple factors. However, scaling experimental studies is a complex task, and it becomes even more challenging when dealing with a minority sample.

Considering these circumstances, we believe it is the right time to engage on a LEAP Project. Partnering with LEAP Fellows during our initial research stages can yield significant benefits because:

• Seeking expert guidance and strategies to scale our solution before implementation mitigates the risk of having to rerun the experimental trial.

• Implementing intermediate impact evaluation milestones enables us to assess the effectiveness of our solution at the early stages, allowing us to make informed decisions and take appropriate action. Consequently, this approach will help our team accelerate the impact of our solution.

Our solution aims at gathering the necessary data that may provide strong evidence regarding the impact of sociological and health-related variables for language learning among DHoH children. We expect that LEAP Fellows embarking on this project will help us understand:

1. How can we test the software usability and maximize user-satisfaction to enhance data collection?

2. How can we effectively scale our software across the country and other Spanish-speaking countries? What strategies should we employ to assess its impact during the intermediate stages of implementation?

3. How has machine-learning been used to assess educational outcomes in the past in very heterogeneous populations?

Expanding our solution across Argentina and, further on, the territory of Latin America and the Caribbean presents several challenges. We believe that the outcomes generated by LEAP Fellows will greatly contribute to achieving this goal.

Recognizing that our software caters to a minority, we anticipate that LEAP Fellows can offer expert guidance on conducting usability studies and implementing product adoption strategies. This invaluable input will help us reach as many potential users as possible.

Additionally, we expect Fellows to develop a roadmap outlining the necessary steps to transform our existing small-scale assessment software into a comprehensive web-based tool. This tool should serve:

• non-expert users

• Spanish-speaking populations from diverse backgrounds

• Spanish-speaking populations from other cities in Argentina

• Spanish-speaking populations from other Spanish-speaking countries

Considering the long-term goal of our solution, we understand that the implementation of machine learning methods strongly depends on data quantity and quality. By partnering with LEAP Fellows, our team would benefit from reports on how machine-learning approaches have been used in the past for predicting cognitive and educational outcomes in heterogeneous children’s populations. This knowledge will prove invaluable to our team's efforts.

The outputs generated by the Fellows will be promptly implemented. As we are currently in the early stages of product development and scaling, conducting usability tests and assessing potential adoption of the software will greatly enhance our future scaling efforts. In turn, the scaling roadmap will be applied by taking into consideration the recommendations and guidance provided.

Finally, reviews and reports on how machine-learning has been used to predict educational outcomes will be discussed before large-scale data collection to estimate the optimal sample size for improved model fitting. These steps are crucial as we move forward in planning and executing subsequent phases of scaling.

By hosting a LEAP Project, our team will immediately gain expertise in conducting qualitatively and quantitatively more reliable practices towards the design and implementation of evidence-based interventions targeted at DHoH children and children with communication disorders in general.  

After the 12-week LEAP Project sprint we expect the following outcomes:

• Short term: 

  • using the roadmap and guidance provided by Fellows to test usability and user satisfaction of our software in different school-based experimental studies. We plan to implement this within the community of schools that have required the design of our assessment tool.

  • taking into consideration reports of how machine-learning has been previously used for educational purposes in heterogeneous populations to design and improve data collection (by estimating potentially necessary sample size, taking into account possible drawbacks and planning scaling accordingly)

• Mid-term:

  • scaling the use of our software to further schools in Argentina and the Southern Cone.

• Long-term: 

  • scaling the use of our software to further countries in Latin America and the Caribbean.

  • using the data collected to build machine learning models to estimate possible profiles of DHoH children. We expect that this valuable information aids health practitioners, educators, and policymakers in offering timely and appropriate recommendations to parents whose children receive a diagnosis of deafness.