A digital patient feedback platform for Tanzania

Improvements in health outcomes in the Global South, including preventable deaths, are constrained by the low-quality delivery of public health services. This is in part attributable to lack of resources. However, a key challenge in many contexts is also limited information and weak accountability relationships within the health system. In Tanzania the outcome is that there are 11,000 maternal and 100,000 under-5 deaths per year. Moreover, the rate of maternal mortality is 50% higher than in neighboring Kenya.

Within the healthcare system in Tanzania there are inadequate citizen feedback channels. The typical facility receives just two pieces of citizen feedback per month - typically via paper and pen through suggestion boxes. 60% of citizens say that existing feedback mechanisms are inadequate. As a result, citizens' voices are insufficiently heard within the system. 

Consequently there is a lack of accurate, timely, and low-cost information about the needs of citizens and the performance of health service providers. This means even when the system has the resources to address its problem citizens, healthcare workers and government officials struggle to advocate for resources and implement high quality healthcare services. These issues can be broken down for three different groups of stakeholders within the healthcare system:

Citizens and communities:

• Bottom-up accountability: Citizens and communities can not systematically hold service providers to account

• Trust: Citizens and communities underutilize key healthcare services due to a lack of trust in the quality of services

Healthcare workers:

• Challenge and solution identification: Healthcare workers have limited ability to understand the real-time challenges of citizens and so prioritize and develop suitable solutions 

• Motivation: Healthcare workers have limited motivation to address citizen challenges as they have inadequate information to develop solutions

• Resource allocation: Healthcare workers can not effectively advocate for the resources they need to implement solutions

Government officials (both local to national government):

• Challenge and solution identification: Government officials have limited ability to understand citizens’ challenges and to prioritize and develop suitable solutions or policies for the services they support and the health system overall

• Motivation: Government officials have limited motivation to address citizen challenges as they have inadequate information to develop solutions

• Resource allocation: Government officials have limited information to ensure they allocate resources in line with these solutions and policies

• Top-down accountability: Government officials have limited ability to systematically monitor through readily available information the performance of the healthcare system (at different levels of aggregation)

Our innovation is a mobile citizen feedback service. After receiving healthcare services, citizens provide feedback, using free SMS and USSD surveys through their own phones (accessible to 85% of the population), on the quality of services they received. These surveys are triggered by citizens themselves and the platform is promoted within communities via printed, radio and community healthcare worker led campaigns. This data is analyzed and provided to healthcare workers and government officials in the form of easy to use alerts, dashboards and quality reports. The platform provides healthcare workers and officials with (1) large volumes of real-time, facility specific insights on the quality of services provided, (2) tools to support the selection of suitable solutions and (3) accountability prompts to encourage the use of the data and track the implementation of solutions. Together these components drive responsive and accountable decision making. Lastly, feedback summaries and information updates are provided back to citizens to close the feedback loop, to ensure they are well informed and to build trust in the system.

Compared to the dominant, existing feedback mechanisms (such as suggestion boxes) a mobile platform addresses barriers for citizens as they can provide feedback anonymously and reduces the literacy barrier to providing feedback relative to paper and pen options. For the government, the intervention increases the volume of feedback collected and enables it to be analyzed in real-time in a systematic way to enable evidence-based decision-making at all levels of the healthcare system.

Feedback data is collected across several domains of the IOM’s framework of care quality, including; (i) patient-centered care, (ii) effectiveness, (iii) timeliness, (iv) efficiency, (v) safety, and (vi) equity. After the data is collected it is analyzed to identify priority problems and integrated into existing government health information systems. The use of data is promoted through existing quality improvement and performance management processes at three levels of the healthcare system: community level (through social dialogues, community notice boards, community radio broadcasts), health facility levels (through quality improvement meetings and health facility governing committee meetings) and local government level (through supportive supervisions and quarterly budgeting and resourcing meetings). 

The program drives top-down accountability by ensuring strong buy-in and ownership of the program among local government officials who oversee healthcare workers. Simultaneously, the program triggers bottom-up accountability through the community, thus enabling community healthcare workers and citizens to better advocate for the services they need. 

This solution builds on the benefits of traditional social accountability interventions (such as social dialogues and community score cards) by utilizing low-cost, digital technologies which are scalable. Unlike recent work (Raffler 2020, Arkedis 2021) which finds limited impact from just bottom-up community feedback mechanisms, our approach builds on other recent work (Kosack 2021) and blends bottom-up feedback with top-down accountability by influencing government decision making.

Our innovation is designed to improve access to quality health care for all. To the extent that lower socio-economic populations and women bear the majority of the burden of caring for ill children and relatives, we expect they will disproportionately benefit from this innovation. In our pilot data, 100% of citizens who used the platform were female. The time and financial savings that would result from improved quality health care will contribute to reducing health inequity. 

These are some of the program design features we have considered which ensure the relevance for all populations. For example, our platform: 

• Is available within all types of healthcare facilities, including public primary care facilities which are disproportionately used by vulnerable populations.

• Is available on non-smart phones which are readily available to 85% of the population across Tanzania. 

• Also reduces the literacy barriers found in existing feedback channels through the use of single character responses.

The lead implementing organisation is Afya Pamoja, registered as a non-profit in Tanzania and as a 501c in the U.S. Afya Pamoja was established with the principal purpose of building digital citizen feedback platforms embedded into government performance management systems overseeing delivery of essential public services. 

Afya Pamoja has been working with Tanzania’s Ministry of Health and Ministry of Local Government since 2021 to co-design and test citizen feedback platforms and has established strong partnerships with these institutions which are formalised through a signed 5-year MOU, a research protocol approved by the National Institute of Medical Research and a government project document outlining the program’s operations and pathway to national scale over the next 5 years. Through this process Afya Pamoja and the Government have established a shared vision for how to design, test and scale a citizen feedback platform in the healthcare sector. 

In June 2022, Afya Pamoja launched a digital client feedback platform in 135 facilities in Dodoma Region of central Tanzania. Over the last 9 months the team have tested a variety of program innovations in order to deeply understand how feedback services can drive improvements in quality of service delivery. This has enabled Afya Pamoja’s program team to spend extensive time with users of the service, including clients who provide feedback as well as healthcare workers and local government officials who use the data for decision-making. Afya Pamoja have a unique insight into how to design this service for each of the key stakeholder groups in order to maximise its impact potential. 

Through the partnerships established with the Government, the experience of launching and implementing a client feedback service, and with a skilled team dedicated to this mission, Afya Pamoja is uniquely well-positioned to implement these activities. 

The management team includes:

Dr. Helga Mutasingwa, Director of Partnerships is a Tanzanian doctor and public healthcare professional with 10 years of experience. Prior to co-founding Afya Pamoja, she managed ICAP’s HIV-patient retention program in the Dar es Salaam Region, visiting public healthcare facilities, patients, facility staff and local government health monitoring teams on a daily basis. She has a deep, personal understanding of the challenges faced by the Tanzanian healthcare system due to lack of feedback, and has a strong network throughout government. She was awarded an Echoing Green fellowship for her work with Afya Pamoja. 

Marcos Mzeru, the CTO, is a Tanzanian healthcare technology professional. He has 20+ years of experience building mHealth solutions in Tanzania and previously worked in the ICT department at Tanzania’s Ministry of Health for 15 years, after which he worked with FHI360. He has led the designing, testing and scaling of multiple national-scale technology solutions in Tanzania’s healthcare sector. He has a strong network throughout the Government of Tanzania.

Simon DeBere, CEO, has 7 years’ experience in developing partnership and growth strategies for social enterprises and civic tech companies in East Africa. He also has experience in evaluative methods and policy analysis. 

50,000

There are three primary areas where we believe the Solve network could help us to advance our work: 

• Technology - We expect to build out our Technology team over the next 12 months, increasing from one contractor to three FTEs, and we would benefit from expert support to understand the skills that we need in the team, as well as on some of the key technology design decisions that need to be made.
• M&E - As we prepare to launch an RCT in 2024, we have a range of technical M&E challenges to take on related to data collection and analysis. We would also like to make improvements to our M&E reporting to ensure we present our impact and learnings in a powerful way. Connecting with other organisation or individuals who have expertise in this area would be helpful. 
• Public Relations - Currently our external communications is very limited and we are only putting 5% of one staffers' time into this. We want to put more investment into this and would want strategic advisory support for this. 

Our innovation is a mobile patient feedback service. Currently the voices of the vast majority of Tanzanians cannot be heard, and their experience of healthcare services is hard to understand. Afya Pamoja partners with the Government of Tanzania by putting citizens' voices at the centre of government decision-making. 

Through our platform, using free SMS surveys patients’ provide feedback, through their own phones, on the services they received at a healthcare facility. This data is analyzed and provided to healthcare workers and government officials in the form of easy to use alerts, dashboards and quality reports. The service provides healthcare staff with large volumes of real-time, facility specific and actionable insights on the quality of services provided to drive responsive and accountable decision making. Lastly, updates are provided back to patients to close the feedback loop and build trust. 

Insights are provided on the availability of services (such as those required during antenatal care visits) and the quality of services (such as family planning guidance). Healthcare workers can then identify and address the needs of patients in a systematic manner through supportive performance management tools. Government health officials can also identify service challenges (such as healthcare worker attendance) and gaps and so divert support (such as audits) and resources to these issues. 

Compared to existing feedback mechanisms (such as suggestion boxes) a mobile service addresses barriers for patients and government. For patients it resolves the key fear around anonymity and reduces the literacy barrier to providing feedback relative to paper and pen options. For the government, the innovation increases the volume of feedback collected and enables it to be analysed in real-time in a systematic way and directed to multiple levels of the healthcare system. The solution includes a level of flexibility such that healthcare managers can collect data on specific issues as and when they need, receiving specific information on their highest priority issues. Lastly, to ensure adoption and sustainability, data is integrated into existing government health information systems and insights directed through existing performance management processes (such as monthly district oversight meetings).

This solution builds on the benefits of traditional social accountability interventions (such as social dialogues and community score cards) by utilizing low-cost, digital technologies which are scalable and can ensure the impact of these interventions can reach many millions of people. Unlike recent work (Raffler 2020, Arkedis 2021) which finds limited impact from just bottom-up community feedback mechanisms, our approach blends bottom-up feedback with top-down accountability by influencing government decision making.

Over the next three years we will work with the Government of Tanzania to scale the client feedback platform nationally, covering all 10,000 facilities and the entire population of the country, 64 million people.

In Q1 2023 we completed a 9-month pilot focused on clients receiving care in the RCH space, which has provided evidence for the impact of our program. Over the next 12 months we are working with the Government of Tanzania on designing a new program that will enable all clients to provide feedback regardless of the type of care they have received. 

Currently we operate in 150 healthcare facilities in two districts in Dodoma Region. These 150 facilities serve a catchment population of approximately 1 million people. By the end of 2023 we have aligned with the Government that we will be operating in 500 facilities across 10 districts, which in turn provide healthcare services to 3.2 million people. By the end of 2024 we expect to be operating in 1,000 healthcare facilities which provide healthcare services to 6.4 million people.

In order to achieve our goal of scaling the program nationally we will follow a phased approach supported by a coalition of funders focused on scaling evidenced-based interventions. The RCT that we will launch in 2024 will be a key step by (1) providing evidence for the government about why this intervention is valuable, (2) providing evidence to support future fundraising activities, (3) how to scale the platform cost effectively across multiple regions of the country. 

Christensen 2021 finds that community feedback can result in 38% reductions in infant under-5 mortality. By replicating the impact of this study nationwide our program would reduce deaths by approximately 40,000 per annum, not accounting for other sources of deaths such as maternal mortality. This is equivalent to 4 lives saved per facility (10,000 across the whole country) at an average cost-per-life saved of 750 EUR, making it among the most cost-efficient life-saving interventions. The charity research organization Givewell estimates top-performing charities can save a life for between $3,000-$5,000. We will initially focus on public sector facilities which are disproportionately used by those of lower socioeconomic status and by women, so we expect the greatest impact to be realised among these segments of the population. 

To achieve this scale our major partners are Tanzania’s Ministry of Health and Ministry of Local Government with whom we have a strong relationship and vision alignment for this service. We have signed a 5-year MOU with the government to design and scale digital client feedback services in partnership with both ministries. We also have been formally approved to conduct Implementation Research on client feedback systems in Tanzania through the National Institute of Medical Research and are currently finalising a formal project document with both ministries which outlines the operational plan for how we will design, test and scale this service nationally over the next 5 years. We work closely with 5 director-level champions across both ministries. 

Our intervention is a health systems strengthening program which focuses on improving quality of care. We expect to see changes in the behaviour and knowledge of clients and healthcare workers, changes in quality of care indicators and changes in ultimate healthcare outcomes. 

There are two phases of our program measurement, internal and external.

Internal: In order to monitor our program performance, our internal M&E team will follow a routine monthly data collection process. The team collects a combination of quantitative and qualitative data based on an M&E framework which identifies indicators based on our Theory of Change. This enables us to track changes in indicators over time and compare baseline to endline readings. The indicators (not exhaustive) are listed in the table below.

Quantitative data comes from a range of sources including the digital platform itself and digital surveys issued to healthcare workers on a monthly basis. This data is collected from all facilities where the program is live. Qualitative data comes from monthly interviews with all program stakeholders including users of the platform, healthcare workers and local government officials. These are randomly sampled each month.

External: We will launch the RCT in Q1 2024 and run for 12 months. The study will involve 1,000 health facilities from 50 districts and their catchment’s areas. We are designing this RCT in collaboration with world-leading academics from Tanzania and the US. 

Indicators: monitored include the following:

Platform adoption 

• Number of clients registered to the platform
• Number of feedback data points provided by clients

Client knowledge and behaviour

• Client-reported trust in healthcare services
• Client knowledge of their rights according to the Client Service Charter

Healthcare worker and government knowledge and behaviour

• Number of facilities conducting monthly Quality Improvement meetings
• Number of facilities conducting monthly Quality Improvement meetings where client feedback data is used
• Number of facilities conducting quarterly Facility Governing Committee meetings
• Number of facilities conducting quarterly Facility Governing Committee meetings where client feedback data is used
• Healthcare workers’ knowledge of clients’ priority reported issues
• Number of local government Supportive Supervision activities conducted in facilities

Utilisation of care

• Number of OPD visits to the health facility per person
• Number of pregnant women attending 4+ ANC clinics
• Number of institutional deliveries
• Number of infants administered with complete vaccine schedule

Availability of care (selected indicators)

• Number of stockouts of essential tracer drugs package
• Healthcare worker attendance during working hours
• Average client waiting time

Client-reported quality of care (selected indicators)

• Client satisfaction with customer service / HCW interaction
• Client satisfaction with facility cleanliness
• Client satisfaction with waiting times

Mortality indicators

• Neonatal mortality rate
• Infant mortality rate
• Maternal mortality rate

Our theory of change centres on implementing a set of activities that trigger three pathways; (i) citizens and communities, (ii) healthcare workers, and (iii) government officials, which contribute to changes in outcomes of behaviour and knowledge, improved quality of care and mortality. 

Activities:

Through our program we collect large volumes of citizen feedback through our SMS technology platform. During our pilot clients perceived variable service quality by facility and by domain of care quality, indicating that feedback data is a valuable signal. Feedback data is shared at all levels of the healthcare system, and our pilot demonstrated that this is feasible.

Pathways:

#1 - Citizens and communities:

Citizen feedback interventions build stronger relationships and trust between citizens and healthcare workers and government officials (Danhoundo 2018, Mselle 2019). This leads to increased utilisation of services. 

After 6 months of our pilot, 71% of interviewed mothers who had used the feedback platform reported increased trust in the healthcare system and there was a 5.2 percentage point increase in institutional births relative to control districts (differences in differences analysis). Other research shows a 20% increase in utilisation of ANC visits (Gullo 2016).

#2 - Healthcare workers:

Citizen feedback data improves facility healthcare workers’ understanding of citizens’ needs and increases the motivation of healthcare workers to improve quality. During our pilot we observed that the % of total facilities conducting QI meetings increased by 32 percentage points from program launch to six months later. During interviews, healthcare workers described using feedback data to write QI action plans.

#3 - Government officials:

Sharing data with local government officials (i) improves understanding of performance issues, (ii) increases their ability to hold facilities to account and (iii) improves the efficiency of resource allocation among facilities. 

Studies (Callen 2020) find that providing data to public healthcare managers increases facility audit visits by 104% and doctor attendance by 75%. This improves resource allocations (e.g. medicines) and staff behaviours. Evidence suggests it can increase availability of essential medicines by 41% (Blake 2016).

Outcomes: 

From these pathways we expect to observe changes in stakeholder knowledge and behaviour and improved quality of care and healthcare outcomes. Key indicators include:

• Behaviour and knowledge:

  • Citizen trust in healthcare services and knowledge of healthcare rights

  • Citizens’ willingness to provide feedback 

  • Healthcare worker and local government officials’ correct identification of citizens’ reported priority challenges

  • Volume of healthcare worker quality improvement meetings, action plans created and issues resolved

  • Volume of local government oversight activities (e.g. supportive supervisions), action plans created and issues resolved

• Healthcare quality:

  • Facility infrastructure (e.g. cleanliness) 

  • Availability of inputs (e.g. HIV testing kits)

  • Healthcare worker attendance 

  • Waiting times

  • Customer service (e.g. disrespectful language)

  • Corruption

• Healthcare outcomes 

  • This will include near-term metrics including utilisation of care (e.g. ANC attendance) and ultimate outcomes including infant and maternal mortality rates and morbidity rates for malaria and HIV. Research shows that citizen engagement can reduce infant mortality by 38% (Bjorkmann & Svensson 2009, Christensen et al 2021)

The main part of our digital client feedback platform is based around SMS technology. We use an open-sourced SMS platform called RapidPro which enables us to design and manage the content of the system. The SMS or USSD prompts are sent to the clients who register to the platform and their responses are also aggregated within the RapidPro system. This aggregated data is then sent through an API to the government's healthcare information management system where it can be analysed and viewed by different users of that system, healthcare workers and government staff. 

We selected this design for a number of reasons:

• Accessibility - 85% of Tanzanians have access to a simple phone which has USSD and SMS technology and thus by choosing to design our system using this technology, it is widely-accessible to our target user population. 
• Cost - SMS and USSD technology is a relatively low-cost form of technology which means that the system is scalable and called feasibly become a nationally-scaled system. 
• Sustainability - Integrating our technology system into the government's healthcare information system helps to build the long-term sustainability and government buy-in for the service. 

Afya Pamoja is committed to building a team and organisational culture on the principles of diversity, equity and inclusion. 

Our organisation was co-founded by a Tanzanian woman, a Kenyan man and two British men, and the other member of our leadership team is a Tanzanian man. We are committed to hiring Tanzanians for all roles within the organisation, since this is our principal country of operation, and we are committed to ensuring that women account for 50% of all future hires in the organisation. Beyond race, gender and nationality, our team is diverse in other ways, by age and professional background. While our founders are all young, under 35 years of age, one member of the management team is over 50 years of age. Within our team we have expertise from clinical medicine, public health, digital technology, government, social enterprise, and business. We believe our range of different experiences is a major strength of the organisation and means that each of us can bring a unique, valuable perspective to any conversation.  

We have established organisational management and HR practices which seek to drive an equitable culture. This includes clear remuneration structures and performance assessment rubrics linked to salary raises which are applied equally across the entire team.   

The entire ethos and mission of our organisation is about inclusion and voice, and this extends to our internal team culture. We have a very flat organisational hierarchy and there are opportunities for all team members to contribute ideas and challenge the decisions of others. The management celebrate the idea of "radical candour" and 360 degree feedback so actively seek out the recommendations from the team and see this as an opportunity for growth and improvement. Every week the team conduct brainstorm activities in which any member of the team can put forward a topic and it is debated and discussed as a group.        

Afya Pamoja provides a digital feedback service for the Tanzanian healthcare system. There is high demand from the Government of Tanzania for this service given that it can support quality improvement initiatives at the community, facility and local government levels, and policy initiatives at the central government level. For citizens the service is free-of-charge, and provides them with a free-to-use, user-friendly platform for providing their feedback. 60% of citizens state that current feedback services are inadequate, yet 75% of citizens express a desire to provide feedback. 

The overall value proposition is based on the impact of the feedback service on healthcare outcomes. This is a healthcare systems strengthening intervention which can drive improved quality of care and experience of care for citizens. Afya Pamoja's service will be the first in Tanzania to provide large volumes of real-time, structured data. This data is used to drive decisions at multiple levels of the healthcare system, from the community level, to facility and local government level, leading to improved healthcare outcomes such as reduced maternal mortality.      

We are a grant funded non-profit. We receive grants from three types of funders which we use to support different parts of our work. (1) Innovation / research funders (e.g. Fund for Innovation in Development, J-PAL) to support innovation and evaluation work, (2) multilateral organizations (e.g. UNICEF) to support our implementation activities across new regions and (3) Foundations / Individuals (e.g. Echoing Green) who provide unrestricted funding to support organizational capacity development and overheads. In 2021 we raise $145k in grant funding and in 2022 $600k. Our funders require us to demonstrate our impact by running robust program M&E and sharing regular reports. Given the low-cost nature of our program, and the high impact potential, we believe this represents great value-for-money when compared against alternative public health investments. In 2024 we will conduct an RCT with world-leading academics from Tanzania and the US. The results of this RCT will support us to raise additional funding by providing a strong evidence base for our work.

At national-scale we project based on our unit costs that it will require 4 million USD per annum to run our program. Our goal is to secure this from three major sources: (i) 25% from government cost-sharing arrangements (for core implementation costs), (ii) 50% from long-term bi-/multilateral development partner commitments, and (iii) 25% from global health donors motivated by evidence based and cost effective interventions.

This excludes additional subsidized expenses to cover the on-going SMS costs  The Ministry of Health has already gained access to a fully-subsidized SMS communication line with zero costs for SMS and USSD. They have expressed interest in approving the feedback service to be implemented using this asset which would increase our level of government ownership.

Government cost-sharing: There is increasing willingness from the Government to provide increased budget allocations for digital health solutions, given the increase in access to technology among citizens. Once the evidence-base for the program is clearly established government will cover the core implementation costs such as healthcare worker and local government incentives. We have seen other digital health organisations who have been able to achieve this, such as M-Mama in Tanzania, and Jacaranda Health in Kenya.     

Bi/Multi-lateral development partners: 50% of healthcare spending in Tanzania comes from these partners. Typically these partners fund multi-year commitments or key parts of the nation's healthcare infrastructure. There is an established Working Group for these organisations which meets quarterly. Given our strong partnership with the Government, senior officials would advocate for these partners to fund the scaling of the client feedback program through Afya Pamoja.

Global health donors and foundations: This funding would come from a range of foundations, from large global health donors (e.g. Gates) which would support long-term program scaling and smaller family foundations, such as the Big Bang Philanthropy Group, who provide unrestricted, highly flexible funding which would support the core team.

To-date we have cumulatively raised $800,000 of funding to support our work. Our major funders include FID who awarded Afya Pamoja a Stage 1 grant in 2022 ($250k), a HNWI with excellent connections across the venture philanthropy sector ($200k), Echoing Green fellowship funding ($80k), UNICEF Tanzania to support our pilot ($90k), J-PAL ($75k). 

We are also currently working with a group of funders who are interested in supporting the scaling of this platform nationally. Between them we expect to secure $2M of additional funding by Q4 2023. These include Grand Challenges Canada, Draper Richard Kaplan Foundation, Horace Goldsmith Foundation, Patrick McGovern Foundation, and two HNWIs with interests in interventions which address maternal mortality.