Motriz

In a scenario where public services are not equitably delivered to the population and often lack the quality and reach needed to address Brazil’s complex issues, Motriz emerges to strengthen governments through the potential of people. We implement models capable of structuring real changes that can be replicated everywhere to generate significant transformations. The mission of Motriz is to empower governments through people, focusing on public education and the development of public leadership. To achieve this, we work with local governments to enhance their institutional capacity to deliver effective public policies that develop the full potential of students and attract and develop public leaders committed to addressing the country’s most significant challenges. In education, we collaborate with school districts to support the development and implementation of educational public policies, with a cross-cutting focus on racial equity, gender, and socio-environmental sustainability, as well as the professional development of public agents. With this dual strategy, we aim to build capacity within the public sector so that public policies can be well-implemented and promote sustainable impacts on children’s learning. Participating school districts undergo a four-year program, receiving tailored support to address their most pressing challenges. In the Amazon region, our educational program is adapted to local needs and incorporates a perspective of education for sustainability. Since 2016, we have partnered with over 40 school districts, reaching more than 2 million students in grades 1-9. On the leadership front, we support local governments in policies for attracting, selecting, developing, and engaging public leaders. Already, more than 679 public leaders have been reached. Our goal is to collaborate in building diverse teams and retaining competent, engaged professionals in strategic positions within governments, thus strengthening the Brazilian executive power to deliver better services to the population.

The technical leader plays a central role in ensuring our programs are coherent with evidence of impact and leads continuous improvement processes based on our impact assessments and updated field evidence. This professional heads the team responsible for monitoring these programs and collecting and analyzing data on the implementation of educational policies. Additionally, the leader oversees impact assessments and indicator management, ensuring that our programs adhere to standards of quality and sustainability. As such, he plays an essential role in guiding the Monitoring, Evaluation and Learning team towards our desired impact: increasing student learning and reducing educational inequalities.

The leaders and team members of the Monitoring, Evaluation and Learning team at Motriz are experts in their fields, bringing a wealth of experience from similar projects. Within Motriz, this area plays a fundamental role, and its strategy is central to our operations. The conduct of impact evaluations and analyses of our operations drives our organizational strategy through a well-established monitoring system and comprehensive assessments. This work front is already a priority in our organization, and the support of researchers from LEAP could be very valuable and enriching, as it would represent a natural continuation of our commitment to impact evaluation.

Our team consists of professionals with strong academic backgrounds and extensive experience in monitoring, impact evaluation and learning. Each member significantly contributes to our institutional legacy, with a team composed of economists, a social scientist, a public manager, and an expert in the implementation of educational policies. The projects in this area involve managing and creating indicators deeply linked to theoretical foundations and informed by the practical implementation of educational policies. The diagnostic system, composed of these indicators, provides fundamental practical directions for the organization’s operations in partner territories. Additionally, the team is committed to identifying the organization’s contribution to learning gains, whether quantitatively or qualitatively, in a way that enriches the inferences of impact research with the methodological repertoire accumulated through their qualifications and experiences.

Researchers will have the opportunity to engage with a diverse team that possesses complementary technical and managerial skills, fostering an environment conducive to collaboration and collective construction. We are fully prepared to integrate the LEAP Project into our global strategy, thereby strengthening our commitment to innovation and excellence. This unique opportunity allows us to maximize our impact further. Our team is dedicated to contributing to the success of the LEAP Project, drawing on our extensive experience and proven skills in Monitoring, Evaluation and Learning.

The Formar Program supports public education networks by promoting pedagogical coherence and efficient management to enhance student learning and retention.

Brazil has faced chronic educational challenges for a long time. Even before the pandemic, most students were not achieving satisfactory levels in reading and math. With almost two years of school closures, the situation has worsened considerably. It is estimated that in a ninth-grade class, around 24% of students are at least two years behind their appropriate grade. Even among those who complete elementary school, approximately 85% do not acquire basic math skills.

Additionally, there are troubling disparities in academic performance between Black and White students. Approximately 73% of Black students leave elementary school without fully mastering reading, writing, and reading comprehension skills, compared to 54% of White students. These educational issues have a profound impact on the academic and future development of students, especially those in vulnerable situations. It is essential to tackle these challenges comprehensively and effectively to ensure a quality and equitable education for all Brazilian students.

The Formar Program aims to transform education systems from the inside out. It begins with detailed diagnoses of school systems using a set of indicators called Predictors of Learning, which are methods based on evidence, studies, and good practices designed to measure the implementation and quality of educational policies related to learning. These diagnoses are essential for identifying specific challenges and opportunities within each educational context. Based on these analyses, the Program develops customized action plans for local governments. These plans are tailored to address the identified educational challenges and enhance the overall quality of education. Additionally, Formar offers ongoing support to implement these plans, providing training, knowledge sharing, and results monitoring. The goal is to empower local leaders and managers to sustain good results over time, promoting lasting transformation in education systems.

The Formar program team focuses its efforts on supporting network initiatives aimed at various objectives:

Aligning curricula, teaching materials, training, and assessments to ensure pedagogical coherence.

Monitoring enrollment and the school census to ensure accurate and up-to-date data on students.

Promoting consistent student attendance and active participation in school activities.

Implementing effective strategies to reduce school dropout rates.

Carrying out racial equity diagnostics to identify and address inequalities.

Strategically using assessment results to guide educational decision-making.

Efficiently structuring the administrative and financial operations of the Education Departments to ensure the proper use of available resources.

Another important aspect of the Formar program’s approach is cultivating relationships and results among the professionals and pedagogical coordinators working to improve public education. This involves creating a community of practice consisting of policymakers, civil servants, school principals, teachers, and pedagogical coordinators. These professionals receive ongoing training and support to strengthen their capacities and implement best educational practices in their institutions. The collaboration and exchange of experiences within this community help drive significant improvements in the quality of education offered.

In summary, the Formar program takes a comprehensive and holistic approach to improving public education in Brazil, working closely with local educational networks to promote systemic and sustainable change. By focusing on implementing evidence-based actions and strengthening the capacities of education professionals, Formar seeks to create an environment conducive to learning and development for all children, ensuring a brighter future for the country.

Since the beginning of our program, we have been committed to annual impact assessments, refining our approach each year. Our dedication to evidence-based practice extends beyond the application of Predictors of Learning; we continuously verify whether our initiatives are grounded in evidence that reflects our performance, carefully examining how our implementation strategies drive student progress.

To assess the impact of our program, we adopt two forms of impact evaluation:
Quasi-Experimental Impact Evaluations:
Using Propensity Score Matching (PSM), we create synthetic control groups to simulate a randomized impact evaluation. We then apply the Difference-in-Differences (DID) approach to analyze pre- and post-program results. Over three evaluation cycles (2015-2017, 2015-2019, and 2015-2021), comparing the results of the National Learning Assessments (SAEB), we observed a significant positive impact of the Formar Program. For example, we saw an average increase of 30.7 points in Portuguese and 34.7 points in Mathematics for grades 1 to 5. These results represent a substantial advancement over the annual learning expectations, indicating a positive impact of the program on student proficiency.

Dose-Response Impact Assessment:
Using generalized linear dose-response models (GLM DR), we analyzed the sustained impact of the Formar Program over time. Based on the 2015-2017 evaluation, which measured learning gains after one year of participation, and the subsequent 2015-2019 and 2015-2021 analyses, we identified consistent growth in results. For example, for grades 6-9, we recorded an average increase of 15.8 points in Portuguese and 3.3 points in Mathematics over four years. These figures reflect a positive impact of the program in these school groups, contributing to students’ academic progress over time.

The diagnostic system created for the program, called Predictors of Learning, serves as our primary source of information on the effect of our work with school networks. Through our survey, we engage school managers by inviting them to share their perspectives on educational policies. Predictors are collected annually and at the beginning of each network’s engagement with our programs. By monitoring the collection of this diagnostic system, we have observed the positive effects of specific indicators within the Predictors of Learning:

• Schools with a comprehensive, recognized, and utilized curriculum demonstrated a greater ability to increase proficiency, especially in the later grades (6th to 9th). An average increase of 12 points in Portuguese scores and 15 points in Math scores was observed in schools with well-implemented curricula.

• The implementation of structured assessments at the network level, especially when accompanied by pedagogical feedback, has shown the strongest correlation between learning and improved school performance. Schools that adopted this model saw an average increase of 20 points in Portuguese scores and 18 points in Math scores.

• Continuous professional development opportunities for teachers showed a positive relationship with growth in student proficiency. Schools that invested in this aspect experienced an average increase of 16 points in Portuguese scores and 14 points in Math scores.

• Schools that adopted a culture of planning and targets demonstrated good proficiency results. More extensive networks with centralized management saw an average increase of 18 points in Portuguese scores and 20 points in Mathematics scores, while networks with decentralized management recorded an average increase of 15 points in both subjects.

In addition to these specific results, we learned that the Synthetic Index of Predictors of Learning (IDEF), made up of these indicators, has a positive and significant relationship with student learning. Each point added to the IDEF increases, on average, 2 percentage points of students with adequate levels of learning in the educational network.

These data and results guide our future actions, allowing us to identify best practices and implement proven strategies to continuously improve educational performance. They also indicate that the diagnostic system we have formulated is deeply related to the main objective of the program: to create optimal conditions for student learning, promoting improvements in school performance sustainably and equitably in all the schools in the network.

Since the inception of the Formar Program, the evaluation, monitoring and learning process has played a strategic role, as it directs us toward our desired impact in a cost-effective manner. Our studies and research over the years have already demonstrated a significant impact on student learning and identified which educational policies are most effective while also highlighting areas where challenges to progress remain. Despite achieving substantial results in learning, we aspire to and believe that we can have an even more significant impact. We also recognize that our impact is dynamic and susceptible to both external factors and internal methodological decisions. Therefore, it is crucial to understand the impacts of the pandemic on our partner networks, whether we have managed to mitigate them, which interventions have generated the most impact, and what the emerging needs of students are now. Thus, our impact assessments must occur as frequently as national data is available so that we can maintain solid evidence bases, guiding both our decision-making and that of governments.

The year 2024 is particularly crucial for us. This is when the data from the SAEB test (Basic Education Assessment System conducted by the federal government every two years) will become available, allowing us to update our impact studies. The results of this impact assessment will be significant as they will guide the process of continuous improvement of our interventions and aid in programmatic and strategic investment decision-making. Additionally, these results are part of our transparency process with our local government partners and investors about the impact of the partnership. Looking ahead, we plan to significantly expand our support to more education networks in the coming years. In this process, deepening our understanding of our impact levers and identifying where the biggest challenges lie will be crucial for making this expansion as cost-effective as possible.

1) Does our program follow solid and transparent evidence of its impact on student learning?
Highlights the importance of using rigorous, transparent impact evaluations to demonstrate our program's effectiveness and build trust among stakeholders.

2) Which educational interventions demonstrate the greatest impact on student learning?
Seeks to identify the most effective educational practices and programs based on the implementation levels analyzed from our diagnostic system.

3) How can we adapt our interventions to meet students' emerging needs?
Explores how to adjust educational strategies to address specific and emerging needs, guided by trends in SAEB data and other sources.

1) Research Report:

A detailed report that documents the findings of the research, including an in-depth analysis of the educational interventions investigated, the methods used for impact assessment, and the results obtained. This report will serve as an essential tool for informing future educational and policy decisions and as a source for the organization’s publications, whether academic or otherwise.

2) Recommendations for Effective Interventions:

A series of clear and actionable recommendations based on the research findings, including advice on how to adapt and implement these interventions in various educational contexts.

3) Database:

A robust and well-structured database that contains detailed information used in the research, impact metrics, statistical analyses, and the results obtained. This database will facilitate further analysis and future comparisons, ensuring the reproducibility and transparency of the results.

To put these results into action, our organization will implement a comprehensive strategy focused on dissemination, engagement, and implementation within the education sector:

• Engaging Local Decision-Makers:

We will engage policymakers at local levels (municipalities and states) to promote evidence-based decision-making. This will involve presenting our research report and recommendations through technical notes, presentations, and targeted meetings with key stakeholders. Additionally, we will publish academic papers, technical notes, and articles in relevant journals and publications to contribute to the debate on evidence-based education.

• Scaling Effective Interventions:

We will identify and replicate the most effective interventions derived from our research in various contexts and regions. We aim to develop scalable and adaptable models that can be implemented on a large scale to benefit a greater number of schools, educators, and students. This may include forming strategic partnerships with other organizations and developing standardized toolkits to facilitate replication.

• Financial Sustainability Strategy:

Leverage research findings to bolster the program’s financial sustainability strategy. Demonstrate the measurable impact of evidence-based interventions on educational outcomes and return on investment. Seek partnerships with donors, government agencies, and the private sector that align with the organization’s mission to ensure long-term sustainable funding.

• Continuous Learning and Adaptation:

Utilize the insights gained from the research to enhance the organization’s internal capacity. This includes developing employees’ technical and managerial skills and fostering a culture of evidence-based learning and innovation.

The expected long-term results are twofold: i) to increase our impact on student learning by reducing educational inequalities between Black and White students, and ii) to institutionally strengthen our organization’s sustainability strategy. The LEAP project will be essential in achieving these objectives, as it will enable us to evaluate our interventions to ensure their impact and recommend adaptations for more cost-effective operations.

We are dedicated to transforming public education and ensuring that educational policies effectively develop students’ full potential. As part of our long-term strategy, we plan to expand our support for local governments, increase our impact, and influence local governments to adopt proven educational policies. The project will aid in developing and strengthening scalable, replicable models of action based on solid evidence. Furthermore, it will contribute to our organizational institutional strengthening and sustainability by enhancing our internal capacities, developing robust monitoring and evaluation systems, and facilitating strategic partnerships that support the growth and continuity of educational initiatives. Additionally, by publishing our evaluations, we aim to create a lasting legacy of improvement and innovation. These efforts underscore our commitment to promoting quality and accessible education for all, actively influencing local educational policies and practices based on evidence.