CEPE Di Tella

Universidad Torcuato Di Tella (UTDT) is a non-profit institution with the purpose of making a substantial contribution to academic life and education in Argentina. Today, the University is engaged in basic and applied research, undergraduate and graduate teaching in the arts and sciences.

Based at the School of Government at UTDT, the Center for Evaluation of Policies and Evidence-Based Policy-Making (CEPE Di Tella) specializes in conducting applied research that seeks to improve the quality of policy through the provision and promotion of evidence-based initiatives and rigorous evaluation methods. Since it’s been founded in 2017, CEPE Di Tella and its affiliated researchers have been assigned to conduct several research projects funded by international organizations (World Bank, Inter-American Development Bank, UNICEF). Furthermore, it has been awarded several grants from philanthropic and private foundations (BBVA Foundation) and worked in partnership with international research institutions (Center for International Development at Harvard University). 

CEPE Di Tella’s Education and Behavioral Sciences Program focuses on the analysis of educational policies using social, political, pedagogical, economic, contextual, behavioral, and neuroscientific determinants of learning using applied research quantitative and qualitative methods. Our program focuses on researching learning and teaching capabilities across the lifespan, with a particular emphasis on populations that may lack equal resources and abilities. Our primary objective is to foster equity and provide equal opportunities for all children. Since 2017, we are working along with schools from Argentina oriented towards enhancing the education of Deaf and Hard of Hearing (DHoH) children in order to develop the Learning to Learn Program (LeLePro), an evidence-based intervention program to improve these children’s linguistic abilities and provide them better educational opportunities. Our studies  target  children between the ages 3 to 12.

LeLePro involves the evaluation of different cognitive abilities with a software suite. DECILE screens oral language comprehension and production, while MetaTom assesses metacognition and Theory of Mind (ToM). The software also collects demographic, environmental and aetiological variables that may affect linguistic attainment. After initial screening of children, we seek to implement an intervention program based on neuroscientific evidence to improve the development of linguistic abilities, which are crucial for mental health, and the development of other cognitive skills.

Carolina Gattei is currently working as a postdoctoral research fellow and professor in the Education Area at the School of Government, Universidad Torcuato Di Tella. Her research program, titled "Language and Policy: Tools for assessing and enhancing populations with language difficulties" focuses on creating evidence-based linguistic assessment tools for underserved groups, particularly young children.

At present, she holds the position of associate researcher in the Education and Behavioral Science Program at CEPE Di Tella, where she serves as a language expert consultant. As part of the Learning to Learn Program, she is leading the design and implementation of the Digital Device for the Evaluation of Communication during Childhood in Spanish (DECILE). She also collaborates in coordinating other research projects aimed at the Deaf community in Argentina, and leads a project that assesses narrative production in children from economically disadvantaged families in Mendoza, Argentina.

In her role, Carolina is responsible for translating the education sector's needs into practical and cost-effective research projects. She engages with public officials from the education sector and seeks funding sources to achieve the proposed objectives of these programs.

LeLePro is a collaborative research program that brings together expertise from various fields including experimental psychology, linguistics, education, and more recently, artificial intelligence. The program's primary objective aligns with the research interests of its three main leaders: Dr. Cecilia Calero, Dr. Carolina Gattei, and Dr. Agustin Gravano.

Dr Cecilia Calero serves as the Head of the Education Area at Universidad Torcuato Di Tella. Her primary focus lies in comprehending the psychological mechanisms involved in effective teaching and the optimal transmission of knowledge. As the principal investigator of LeLePro, she heads the development of MetaTom, a software designed to assess metacognitive and Theory of Mind skills in children facing language difficulties.

Dr Carolina Gattei takes charge of the development of DECILE, a software aimed at evaluating language skills in Spanish-speaking children with communication difficulties. She also plays a crucial role as a language specialist and serves as a liaison between the academic sector and schools catering to Deaf children and children with communication challenges. Additionally, she actively contributes to the development of MetaTom.

Heading the B.Sc. in Digital Technologies and the Artificial Intelligence Lab at Universidad Torcuato Di Tella, Dr Agustín Gravano brings his expertise in machine learning to investigate how humans engage in oral conversations and enhance computer-based dialogue systems. Currently, he leads a research project that explores the performance of Spoken Language Processing tools  when exposed to oral input from children equipped with cochlear implants. Within the LeLePro framework, his aim is to model individual variations in children's language abilities and advance the development of AI-based tools that can provide valuable support and recommendations for educators and healthcare practitioners.

A dedicated team of PhD and M.Sc. students is actively engaged in conducting field studies to develop LeLePro. Analí Taboh, a PhD student, plays a vital role in coordinating in-schools data collection for the DECILE aspect of the project, as well as overseeing the work of research assistants.

With a deep-rooted passion for scientific inquiry, these accomplished researchers are driven by the power that evidence-based actions may have on the educational field. They are determined to make a substantial contribution that will empower children to flourish and reach their full potential. Together, they aspire to forge a new path that combines interdisciplinary knowledge and cutting-edge technologies to create innovative solutions that will revolutionize the way we support children facing linguistic and communication difficulties.

DECILE is a Spanish language assessment tool that accounts for individual differences, offering personalized recommendations for improved language development.

According to the World Health Organization, approximately 466 million people worldwide suffer from incapacitating hearing loss. In Argentina, nearly a million individuals have permanent hearing limitations, including 64,000 children aged 0 to 14.

Hearing aids and cochlear implants (CI) have significantly improved the hearing abilities of deaf individuals, enabling many children to acquire spoken language skills with appropriate stimulation. However, hearing through a CI differs from typical auditory development, requiring greater effort to comprehend linguistic sounds. 

In Argentina, CIs are usually implanted after the age of one, or even later, due to a lack of information among parents and unstable healthcare policies influenced by budget constraints. As a result, Deaf and Hard of Hearing (DHoH) children whose families do not use sign language (representing about 95% of them), are exposed to oral language at a later stage in childhood. This delay negatively impacts their linguistic, educational, emotional, and social development.

From a linguistic point of view, DHoH children who exclusively access oral language often face difficulties in morphology and syntax, showing delays in utterance length development, omissions or incorrect usage of determiners, prepositions, and inflectional morphology when compared to typically developing children. They also struggle with understanding and producing syntactically complex sentences.

The proposed solution takes into account two significant gaps in the existing literature:

Firstly, although there are numerous studies available on other languages, evidence regarding linguistic attainment of Spanish-speaking deaf children who use CIs and hearing aids is limited, and most of it comes from data obtained in high-income countries. 

Secondly, within the DHoH children’s group, a great heterogeneity in performance is usually reported, along with differences in children’s health history, socio-economic and linguistic background. Yet, the connection between these sources of heterogeneity and variations in performance remains poorly understood, and intervention programs are typically developed based on protocols that assume homogeneity within the group. Consequently, healthcare and education professionals rely on published results from these programs to make recommendations, leading to an unfortunate chain of misinformed decisions for these children.

DECILE is part of a software suite targeted at children with language difficulties. The prototype is currently at work as part of controlled experimental studies. We aim to take this software to the next step with the following aims in mind: 

• to quantitatively characterize language learning variability among Spanish speaking DHoH children from Argentina by running a large-scale linguistic assessment within this population and typically developing children of the same age.

• to build machine learning models that identify differential profiles of linguistic performance, to provide additional information for health and education professionals who work with this population to generate better-targeted intervention programs.

On this end, DECILE measures children’s performance on several linguistic metrics (receptive vocabulary, accuracy in sentence production and comprehension, accuracy in sentence repetition and intelligibility) together with information about external factors that may affect language development. These are: 

• Children’s health history and etiology of deafness

From a medical standpoint, the period from birth to CI fitting varies among children, impacting their exposure to oral language at different ages and potentially affecting language acquisition opportunities. Differences in performance may also arise based on the type of cochlear implant used or whether CI fitting is needed for one or both ears. Etiology could also introduce language learning variability. For instance, while deafness runs genetically in some families, it could be the consequence of an episodic disease, or the result of premature birth, leading parents to take action at different stages during language development.

• Children’s linguistic history

Linguistic education trajectories among DHoH children present great variability due to parental decision-making when facing their child’s deafness diagnosis. It is crucial to take into account that close to 95% deaf children come from hearing families. Consequently, congenitally deaf children whose parents use sign language show native domain of sign language and usually learn Spanish as a second language. Yet, most families have to choose if they want to learn sign language along with their child while awaiting CI fitting, or if they will choose exclusive oral linguistic education. Even if they choose this path, CI might not provide the expected outcome for families, and some children start learning sign language afterwards. Given this wide range of options, we cannot expect similar outcomes during the evaluation of Spanish skills.

• Demographic differences 

Demographic variations also influence the linguistic education and subsequent performance of DHoH children, who may belong to families with varying socio-economic backgrounds or live in countries with differing access to universal hearing screening. These factors frequently correlate with parents' access to information, which, in turn, leads to (mis)informed decisions and exacerbates inequalities among deaf children. Consequently, when assessing differences in language achievement to guide policy development, the data collected from DHoH children are biased due to this disparity.

Since 2017, our team has committed to work with schools from Buenos Aires, Argentina oriented towards the education of Deaf and Hard of Hearing (DHoH) children in order to provide evidence-based learning solutions for them. Mainly, heads of three schools imparting exclusively oral education for these children required the creation of a diagnostic tool assessing their Spanish linguistic abilities. Since then, we have conducted different school-based research studies using methods from cognitive neuroscience in order to respond to questions like:  

• What communication strategies do DHoH children equipped with cochlear implants employ during pedagogical events and how do they differentiate from those used by children who typically develop hearing? What comes first, the ostensive channel or the content of knowledge? 

• What has been described in the literature as typical comprehension patterns in orally educated children with prelingual hearing loss? 

• How do Spanish-speaking deaf children perform in sentence comprehension? And how do they perform in oral production?

• When assessing receptive vocabulary in DHoH children with cochlear implants who speak Spanish, how much weight should be given to their hearing age? 

• How effective are current Spoken Language Processing (e.g., Automatic Speech Recognition) tools when used by DHoH children, and how can they be improved to enhance accessibility?

We are currently expanding our research focus to include the bimodal bilingual community and have conducted an experimental study to evaluate the impact of bimodal bilingualism in adults’ performance in tasks related to Executive Functions. To further explore this area, we have engaged three bilingual schools (teaching Argentine Sign Language and Spanish) to participate in additional experimental studies. Our objective is to understand the consequences of bilingualism on linguistic attainment in these children, considering the significance of Executive Functions for learning.

In 2022, the Inter American Development Bank awarded our team a grant to develop a project that seeks to apply techniques from Behavioral Science towards the improvement of Policy. Our project aims to understand how parents of DHoH children from Argentina choose these children’s linguistic education and to encourage bimodal bilingual education for these children and sign language learning in parents, take carers and close relatives. The project involves the collaboration of the Ministry of Education of the cities of Buenos Aires and Mendoza, and is currently being implemented in all public schools that receive DHoH from those cities.

Foundational research carried out by our team has revealed the existence of linguistic variability among Deaf and Hard-of-Hearing (DHoH) children with early-onset hearing loss across multiple languages. Our initial experimental studies conducted in school settings further supported these findings. Similarly, our examination of standardized tests highlighted the limitations when attempting to define the strengths and weaknesses of children experiencing language development challenges, particularly in comparison to typically developing children.

Furthermore, a recent study focusing on the impact of bimodal bilingualism on executive functions demonstrated that hearing adults proficient in sign language as part of their daily work exhibit advantages in executive functions, such as inhibitory control and visuospatial working memory. However, due to the limited sample sizes of these studies, ranging from 50 to 100 participants, it becomes challenging to control for or account for the wide range of variability observed. Consequently, scaling becomes necessary to identify robust correlations among various factors that influence language development.

In light of these considerations, we have developed DECILE, a user-friendly software designed to assess different linguistic traits. This innovative tool aims to address the need for comprehensive evaluation while accommodating the complexity and individual differences that may affect language development.

On the initial phase, our team focused on achieving the following objectives:

1. Defining the specific measures to be collected using the software.

2. Identifying potential users and their needs.

3. Establishing the key characteristics of our educational software, with a particular emphasis on:

   1. User-friendly and intuitive interface

   2. Engaging and interactive content

   3. Flexibility and scalability

   4. Accessibility and inclusivity

   5. Data privacy and security

4. Adapting school-based experimental tasks to a software experience that is engaging for children and  fast and easy to use for language therapists, teachers and parents. 

Currently, we are actively preparing for the launch of DECILE in schools while planning a comprehensive experimental study with a large sample size. Our vision is to expand the reach of DECILE to as many schools as possible, enabling us to gather compelling evidence on the influence of different variables on language learning variability.

In the third phase of our project we are using machine learning techniques to build models capable of estimating the performance of DHoH children in different linguistic tasks, including oral vocabulary comprehension and morphosyntactic comprehension and production. The quality of these models will be assessed via standard evaluation metrics for regression. Once these models have been trained and evaluated, we will be able to analyze the relative impact of the different sociological and health-related variables on linguistic performance. These findings have the potential to provide valuable insights for the design of targeted intervention programs. Our goal is to supply health practitioners, educators, and policymakers with additional information, enabling them to offer appropriate and timely recommendations to parents whose children have received a diagnosis of deafness.

Our primary challenge lies in catering to a diverse community that seeks advice from an assessment tool suitable for all. Although our current school-based studies yield insights into general language development outcomes within the DHoH community, they lack the statistical power required to provide fine-tuned recommendations for enhancing learning in this population.

 Strengthening the evidence base demands scaling our solution and identifying performance patterns influenced by multiple factors. However, scaling experimental studies is a complex task, and it becomes even more challenging when dealing with a minority sample.

Considering these circumstances, we believe it is the right time to engage on a LEAP Project. Partnering with LEAP Fellows during our initial research stages can yield significant benefits because:

• Seeking expert guidance and strategies to scale our solution before implementation mitigates the risk of having to rerun the experimental trial.

• Implementing intermediate impact evaluation milestones enables us to assess the effectiveness of our solution at the early stages, allowing us to make informed decisions and take appropriate action. Consequently, this approach will help our team accelerate the impact of our solution.

Our solution aims at gathering the necessary data that may provide strong evidence regarding the impact of sociological and health-related variables for language learning among DHoH children. We expect that LEAP Fellows embarking on this project will help us understand:

1. How can we test the software usability and maximize user-satisfaction to enhance data collection?

2. How can we effectively scale our software across the country and other Spanish-speaking countries? What strategies should we employ to assess its impact during the intermediate stages of implementation?

3. How has machine-learning been used to assess educational outcomes in the past in very heterogeneous populations?

Expanding our solution across Argentina and, further on, the territory of Latin America and the Caribbean presents several challenges. We believe that the outcomes generated by LEAP Fellows will greatly contribute to achieving this goal.

Recognizing that our software caters to a minority, we anticipate that LEAP Fellows can offer expert guidance on conducting usability studies and implementing product adoption strategies. This invaluable input will help us reach as many potential users as possible.

Additionally, we expect Fellows to develop a roadmap outlining the necessary steps to transform our existing small-scale assessment software into a comprehensive web-based tool. This tool should serve:

• non-expert users

• Spanish-speaking populations from diverse backgrounds

• Spanish-speaking populations from other cities in Argentina

• Spanish-speaking populations from other Spanish-speaking countries

Considering the long-term goal of our solution, we understand that the implementation of machine learning methods strongly depends on data quantity and quality. By partnering with LEAP Fellows, our team would benefit from reports on how machine-learning approaches have been used in the past for predicting cognitive and educational outcomes in heterogeneous children’s populations. This knowledge will prove invaluable to our team's efforts.

The outputs generated by the Fellows will be promptly implemented. As we are currently in the early stages of product development and scaling, conducting usability tests and assessing potential adoption of the software will greatly enhance our future scaling efforts. In turn, the scaling roadmap will be applied by taking into consideration the recommendations and guidance provided.

Finally, reviews and reports on how machine-learning has been used to predict educational outcomes will be discussed before large-scale data collection to estimate the optimal sample size for improved model fitting. These steps are crucial as we move forward in planning and executing subsequent phases of scaling.

By hosting a LEAP Project, our team will immediately gain expertise in conducting qualitatively and quantitatively more reliable practices towards the design and implementation of evidence-based interventions targeted at DHoH children and children with communication disorders in general.  

After the 12-week LEAP Project sprint we expect the following outcomes:

• Short term: 

  • using the roadmap and guidance provided by Fellows to test usability and user satisfaction of our software in different school-based experimental studies. We plan to implement this within the community of schools that have required the design of our assessment tool.

  • taking into consideration reports of how machine-learning has been previously used for educational purposes in heterogeneous populations to design and improve data collection (by estimating potentially necessary sample size, taking into account possible drawbacks and planning scaling accordingly)

• Mid-term:

  • scaling the use of our software to further schools in Argentina and the Southern Cone.

• Long-term: 

  • scaling the use of our software to further countries in Latin America and the Caribbean.

    using the data collected to build machine learning models to estimate possible profiles of DHoH children. We expect that this valuable information aids health practitioners, educators, and policymakers in offering timely and appropriate recommendations to parents whose children receive a diagnosis of deafness.