Carvajal Foundation

"Aula Global" is a program to strengthen basic skills in language and mathematics for elementary school students, aimed at addressing the lag, dropout and repetition of children from vulnerable sectors of the city of Cali, Colombia. This program is constituted in a system of support to the educational process, directly influencing the reduction of the school dropout rate and academic repetition in the medium term and becoming a social intervention strategy to improve the educational quality of the participating educational institutions. 

By October 17, 2019, the World Bank set an educational goal, which is to "reduce the global rate of learning poverty by at least half by 2030. Learning poverty is defined as the percentage of 10-year-olds who cannot read and understand a simple story." (World Bank. New target: Reduce "learning poverty" by at least half by 2030. October 2019. Available at: https://www.bancomundial.org/es/news/press-release/2019/10/17/new-target-cut-learning-poverty-by-at-least-half-by-2030 [accessed 27 June 2022]).

This problem generates few possibilities for children in the future and limits the bases of fundamental knowledge, including mathematics and science. It is necessary to help children promote their development and prosperity by ensuring quality education. In this way, and taking into account that in Colombia 49% of children in the last grades of primary school in Colombia are not competent in reading, the Aula Global program seeks to provide opportunities for the development and strengthening of basic competencies in language and mathematics, and socio-emotional skills to prevent lag and school dropout in elementary school students (grades from second to fifth grade).

The World Bank made an analysis of the impacts of the Covid-19 crisis on education in Colombia, where "the estimated loss of learning is significantly greater for the vulnerable population. Due to differences in household resources for distance education (connectivity, digital devices, books and television) and parents' abilities to help students in their learning process (as measured by their educational level), it is estimated that learning loss for students in the poorest quintile will double that of students in the richest quintile (49% and 25%, respectively)" (World Bank. Impacts of the Covid 19 crisis on education and policy responses in Colombia. July. 2020. Available in: http://documents1.worldbank.org/curated/en/461641598291954248/Impactos-de-la-Crisis-del-Covid-19-en-la-Educacion-y-Respuestas-de-Politica-en-Colombia.pdf [accessed 27 June 2022])

 The problem that our initiative seeks to address: backwardness, dropout and repetition of school in primary school in official institutions of vulnerable sectors, is present at the national level in Colombia. The program is implemented for the first time in 2017, in partnership with the Ministry of Education of Cali and Harvard University and becomes a support system for the educational process, directly influencing the decrease in the rate of school dropout and academic repetition in the medium term and becoming a social intervention strategy to improve the educational quality of the participating institutions. Students, teachers and managers have been defined as beneficiaries, involving the entire educational community, to guarantee effectiveness in the strategies implemented and facilitate their sustainability in the participating institutions. Additionally, for each type of participant, a differentiated methodological strategy has been devised, which responds to their particular characteristics and takes into account their contextual realities.

 Although socioeconomic conditions can affect access, permanence and school performance, certain measures can be adopted in the educational institution that help break the cycle of lack of commitment and low student performance and, therefore, contribute to educational quality. Early loss of interest in school caused by unsatisfactory grades or academic failure can lead to dropping out of school. This is particularly worrying, as dropping out of school is later associated with risks considered poverty traps, such as child labour, teenage pregnancy, the use of psychoactive substances and the inability to access technical or higher education. The Organisation for Economic Co-operation and Development, OECD, in its study "Low-achieving students: Why they are left behind and how to help them succeed" recommends a series of measures to reduce low academic performance, among which are:

• Identify underperforming students and create targeted interventions

• Encourage students to make the most of available educational opportunities

• Create a demanding environment in schools where students are supported

Therefore, a strategy that includes targeted support to priority groups, provides supplementary support to students in difficulty and promotes the training of qualified teachers is of particular relevance. According to the OECD report, students with learning difficulties benefit from teachers who are interested in each other's learning, help them when they need it, work with them until they understand the course contents and give them the opportunity to express their opinions. This focused accompaniment is especially important for students with low performance, since they consider that their investment in learning is relatively inefficient.

 Traditional approaches to dealing with school backwardness, dropout and repetition in Colombia focus on working with the teacher and not on the student, understanding that the teacher replicates the strategies he appropriates with his students. Additionally, the programs focus on the end, conceiving it as that all children achieve a certain curriculum. On the contrary, our program prioritizes both teachers and students, including activities for both, and part of the identification of the strengths and opportunities for improvement of each of the children, valuing their knowledge and emphasizing the process of building on the skills they already have. 

The program is mainly developed in three phases: characterization, intervention and evaluation.

1. The characterization phase is related to the collection of information, the targeting of students for tutorials, application of tests and analysis of information. On the one hand, students are characterized through the application of the diagnostic tests EGRA (Early Grade Reading Assessment) and EGMA (Early Grade Math Assessment), internationally validated to obtain information related to the performance of students in basic skills in the areas of language and mathematics. On the other hand, the characterization of teaching practices is carried out through the application of a survey-type instrument, which provides information on the knowledge that teachers have about the program, the topics they consider important in language and mathematics, methodological strategies used for the teaching of these two areas of knowledge, their expectations of participation in the program,  the use of Information and Communication Technologies -ICT for language and mathematics classes and the use of the SYNTHETIC Index of Educational Quality ISCE in the development of its pedagogical practice. 

After the choice of the participating Educational Institution, a review of the databases of the students from second to fifth grade who would be the possible direct beneficiaries of the program is carried out, later a call is made to the families with the aim of arranging schedules and availability for the exercise and, finally, there is a second contact with the families by the applicators,  for the realization of the tests to the children, which take an estimated time of 20 to 25 minutes. It is important to mention that in the cases of families that do not have data services or wireless internet network, the program performs a data recharge so that children can participate and perform the test.

Once the tests have been carried out, the children who have some difficulties are chosen to participate in the program, thus initiating the intervention phase.

2. The intervention phase is carried out especially in three ways: tutorials with students, group meetings with teachers and meetings with families. As a result of the Covid-19 pandemic, the development of the tutorials is carried out virtually and consists of the sending by the teacher of the Global Classroom Guide that contains activities, videos and games, later, by telephone, accompaniment is made by the tutor and metacognition questions are asked; two days a week, two hours each day, autonomous work is carried out, where the student must send evidence of learning via WhatsApp.

The accompaniment to the teachers is led by the pedagogical advisors, who share the work material to the teachers, with which the development and strengthening of the appropriation of different strategies for the teaching and learning of language and mathematics is sought. This advice to teachers is expected to positively impact all students, including those who were not direct participants in the program, but who will be positively impacted by the improvement of teacherpractices .

 Finally, the work with families is carried out through socio-emotional workshops, tools for good parenting and home study strategies, which results in a comprehensive accompaniment that benefits children.

 3. Evaluation: After the completion of the intervention process, the EGRA and EGMA tests are carried out again, with the purpose of knowing what was the result of the implementation of the project in children, applying to all children, whether or not they were direct participants.

"Global Classroom – Basic Competences" responds to the social, educational and cultural needs of the current environment of the participating institutions, trying to link the postulates of the UNESCO Regional Bureau for Education for Latin America and the Caribbean, which has established that education as a right must include five dimensions: relevance, relevance, equity, effectiveness and efficiency,  and are closely related, to the point that the absence of any of them would determine a misconception of what could be understood by quality education (OREALC/UNESCO Santiago and LLECE, 2008 citing UNESCO/OREALC 2007).

The Carvajal Foundation has identified the need to provide opportunities for the improvement of the quality of primary and secondary education, as well as for the use of children's and adolescents' free time that contributes to the reduction of school dropout rates of this vulnerable population. Based on these considerations, the Foundation has structured its educational programs to contribute to the improvement of the quality, relevance, coverage and equity of the educational service with innovative proposals and supporting the inclusion of people with disabilities.

The actions that the Foundation has carried out have been aimed at achieving the improvement of educational quality, have focused on the implementation of interventions with greater emphasis on the school climate and in the classroom, configuring learning environments to promote the sense of belonging of students and contribute to the reduction of absenteeism and school dropout. In this sense, the Foundation accompanies the Educational Establishments (EE) in the organization and execution of the teaching and learning processes in the classroom with the support of the quality referents of the Ministry of National Education (MEN).

 The program seeks to implement a strategy of reception, well-being and educational permanence through the strengthening of territorial and institutional capacities from intervention schemes with direct tutoring (child to child) and through the teacher for the strengthening of the integrality and educational quality that results in the reduction of desertion in the population in a situation of vulnerability of the Certified Territorial Entities. Likewise, it seeks to implement a strategy of educational quality to promote the reception, well-being, and school permanence that from the territorial experience of the Carvajal Foundation, an accompaniment located in the management of the classroom, observation, monitoring of learning, use and strengthening of didactic resources, identification of students with low performance and learning difficulties, is carried out,  development of pedagogical strategies relevant to the management of diversity in the classroom and educational inclusion, in order to leave institutional capacity installed to facilitate the subsequent implementation, monitoring and evaluation of the educational quality strategy.

Additionally, the program investigates the socio-educational variables that generate lag and school performance, taking into account the high degree of importance presented by factors such as: the individual characteristics of the students, the climate of the classrooms, the educational materials, the experience and training of teachers, pedagogical practices, the management of managers and the factors of the sociocultural context of families and the school; that would explain between 49% and 60% of school performance. In this sense, it is important to note that the main selection criteria of the IEO participants of the program are chosen by the high levels of school dropout and repetition.

The work with the participating community is fundamental in the execution of the program, to the extent that the identification of children with school lag and who require the strengthening of basic skills is carried out, being an integral part of the process.  Since 2017 we have had impact evaluations and we have also had at the close of each process, annual, feedback processes for the identification of lessons learned, which have allowed us to adjust the content of the program. 

If the program is not designed to directly benefit all children in the classroom, it does benefit them indirectly, to the extent that the skills of teachers are strengthened.

Within the program, activities are developed such as workshops and meetings of teacher qualification in didactic strategies of language and mathematics; classroom visits and accompaniment to tutors for the incorporation of pedagogical practices; and meetings with rectors and teaching managers; development of tutorials for the reinforcement of basic competences in children with school challenges and ex ante and ex post diagnosis of basic skills (Application of EGMA AND EGRA tests), which, through the methodological guides, the educational material, the team of facilitators and tutors, and the training spaces, allow: the increase of the basic competences in mathematics, the increase of the basic competences in language and the appropriation of socio-emotional skills of the children  for the relationship with their peers and with the environment, achieving the improvement of the quality of educational institutions and the survival of children in the educational system.

In this sense, the Aula Global program aims to reduce learning poverty, this being the greatest expected impact.  Learning poverty, which means being unable to read and understand a short, age-appropriate text by age 10.  In Colombia, 49% of children in the last grades of primary school present learning poverty, which leads to grade repetition, school dropouts, over-age and frustrated students with low self-esteem. This is a problem exacerbated during the Covid 19 pandemic. The program prioritizes low performing students in language and mathematics and who, as a consequence, experience rejection, stigmatization and demotivation for learning.

Aula Global is an evidence-based program. Itis based on an impact  evaluation carried out in 2017 by the Harvard University  that showed that  semi-personalized tutorials as proposed, has an impact of 0.38 standard  deviation compared to the control group. From this moment, an evaluation of the project of the program is made, measuring the basic competences of the children  at the beginningand at the end of the program with the EGMA AND EGRA tests, through these results and the feedback exercise at the end of each year of implementation, n  decisions on the contents and relevance of some modules of the program, always seeking to generate better results for the participants. 

As for the Nesta Standards, we are at level 4, since we have an independent evaluation that validates the impact, which was carried out by Harvard University. Additionally, it is important to note that the impact of the program is achieved with an average child/year investment of USD $60.  Accompanying impact assessment document.

https://1drv.ms/b/s!AkmCZO9fBw...

The Aula Global program is a program that has three main characteristics of impact and innovation: 

-       Evidence-based program: Thesuccess of the program is measured through an experimental evaluation conducted by Harvard University. Dr. Felipe Barrera accompanied the Carvajal Foundation in the design of the program and its impact evaluation.

-       Differentiated pedagogy: for each level of teaching skill/practices adapted to the needs of each student.

-       Adaptation of contents: Since COVID 19, the methodological contents of the program have been adapted in a non-face-to-face manner, which allows maintaining the standards and reaching areas of difficult access. In the same way, the program can work in a dual way.

The main evaluation instruments are the EGMA and EGRA tests, which allow measuring the changes of children before and after the program is implemented, having identified the control group and the treatment group.

In 2017, Harvard University, carried out, headed by Felipe Barrera and Francisco Lagos, the impact evaluation of the first implementation of the program carried out in 2017 and in its report they assure that the results of the tests at the end of the academic year show that, while on average students in all schools and in all grades improve their performance compared to their reference results,  students who participated in the tutoring intervention outperform their peers in the control group. On average, students in the tutoring intervention perform 0.13 SD higher than comparable students in control schools. This positive and statistically significant effect increases as students participate in more tutoring sessions. There is great variability in the number of tutorials attended by the treated students, resulting in differences in test scores at the end of the year: for a student who attended 22 sessions, the effect is 0.26 SD; for a student participating in 32 sessions, the effect of the tutoring program is 0.38 SD. These effects are positive even considering that the number of observations is relatively low, and that this was the first time this intervention was implemented.

As a result of the implementation in 2020, in the EGMA tests, the comparison of the results between baseline and starting line show that the concentration of students at the Higher Level increased by 5 percentage points (pp). It is also important to mention that the concentration of students in the Appropriate Level presented an increase of 13 percentage points (pp), going from 41.3% to 55.1%. In contrast, when analyzing the Low Level, which concentrates the lowest scores, it is observed that in the starting line the number of students at that level decreases by more than 3 pp. The same happens in the Basic Level, where the most pronounced decrease of more than 15 pp is seen, falling from 49.3% to 33.7% of students evaluated.

With regard to the EGRA tests, the comparison of the results between baseline and starting line, show that the concentration of students in the Higher Level also increased, in this case the change was more than 7 pp. It is also important to mention that the concentration of students in the Appropriate Level presented an increase of approximately 11 pp, the largest increase of all levels, going from 24.2% to 35.8%. In contrast, when analyzing the Low Level, which concentrates the lowest scores, it is observed that in the starting line the number of students concentrated in that level decreases by approximately 4 pp. The same happens in the Basic Level, where the steepest decrease of more than 14 pp is seen, going from 49.7% to 35.1%.

Finally, the main indicators designed for the program have:

-       Percentage increase of students who are located at the appropriate and higher levels EGMA tests, to know the increment of the basic competences in mathematics.

-       Percentage increase of students who are placed at the appropriate and higher levels EGRA tests, to know the increment of basic competences in language

-       Percentage of students who are at a satisfactory and advanced level of performance in the Saber 5° Tests - Language and Mathematics in ei who participate in the program, to measure the quality of educational institutions

-       Percentage of students who remain in the education system one year after completing the process/leave the education system before the end of the school year - Intra-annual retention/dropout rate, to measure the permanence of boys and girls in the education system.

"Aula Global" is a program to strengthen competences and skills in language and mathematics for elementary school students, as a school lag; to reduce learning poverty and prevent desertion and repetition.

The Aula Global program incorporates the topic of evidence as an integral part of it. However, we managed to identify two challenges that are essential to address: sustainability and the incorporation of technology. We talk about sustainability to the extent that the program works if there are tutors, otherwise not; so the economic contribution of a cooperator is required to pay for those tutors and if they do not have them, the program is unviable.  On the other hand, in terms of the incorporation of technology, it is necessary that the program generates a greater impact at lower costs, so we identified the possibility of technology as a strategy for this. During the years 2020 and 2021  remote tutorials were done  due to the pandemic, but that did not necessarily reduce costs. At that time, taking into account the context and the need to continue the implementation of the program, there was no expert advice on issues of virtual education or similar, but it was a very intuitive process, so it would be fundamental for the Carvajal Foundation, but especially for the Global Classroom Program to have advice in this regard, to strengthen technology in a more specialized way, as an integral  part of this program, so that more tutorials can be done, depending less on the tutors, with a view also to facilitating the sustainability and cost-effectiveness of the  program.