The Client's Voice

Malawi has 20+ million inhabitants and remains one of the world's poorest countries, heavily reliant on donor funding for its health budget (58%), with a myriad of financing sources and temporary projects. Malawi's shortage of healthcare personnel is also the most severe in the region, with scarce health workers not evenly being distributed across the healthcare system. This has resulted in a fragmented healthcare system and poor quality health services. According to World Health Organization reports, Malawi health system ranks number 185 out of 190. Malawi has the lowest ranking health system among countries which are not affected by civil wars, however some of the delivered medical services rank even lower than that of some war torn countries.

Low quality health care services are still a major global issue - with more people now dying due to lack of quality than lack of access to care. One key aspect of the larger issue of poor quality care is that we don't sufficiently engage with clients to understand their experiences of care, meaning that we lose the opportunity to use their feedback to strengthen the system. Pervasive patient dissatisfaction and a sense of disenfranchisement are also drivers of low care-seeking behavior.

The specific challenge we want to address within this setting is the lack of mechanisms to systematically collect, analyze and utilize client feedback in health service design and delivery. This is a crucial component of quality healthcare services, client centered care and patient satisfaction. The current method of collecting patient feedback in Malawi is sporadic and inefficient, largely relying on outdated methods like suggestion boxes, which are seldom used due to issues as 29% of the population being illiterate, only 5% speaking foreign languages and fear of retribution. These methods do not provide continuous, actionable data contributing to poorly designed healthcare services that don’t have the client at the center.

Up to now, we have not seen solutions that truly give the community a voice, which is essential for establishing truly locally-led sustainable development. This leads to a lack of empowerment and a sense of disenfranchisement among clients as well as among the people working daily to try to enhance healthcare quality and pave the way for a more resilient and client -focused health system.

Our solution targets the inadequate capacity to routinely gather and utilize patient feedback for health service enhancement. We propose a novel approach, leveraging AI and avatar technologies to establish a robust, low-cost and user-friendly system for rapidly collecting and analyzing client feedback to ensure that healthcare services are continuously informed and improved based on client experiences.

The far majority of the population speaks Chichewa as either their first or second language. The core of our solution is the development of an AI-powered Chichewa voice model, disseminated using a digital avatar designed to interact with patients and collect their feedback in a respectful, intuitive, and engaging manner. This approach aims to overcome barriers related to literacy, language, and apprehension that patients might face in traditional feedback mechanisms. By using spoken language, we ensure inclusivity, enabling those who cannot read or write to share their insights and experiences.

Project partner Fortell will create a Large Language Model (LLM), using audio recordings and text to enable accurate processing of voice, transcription, translation, and interpretation. Jhpiego staff, with experience of having supported services to millions of clients in Malawi, will support the language model development and have selected a visual identity for the avatar to suit the cultural context.

The avatars are deployed directly to clients using basic tablets at healthcare facilities and can be used off-line in case of connectivity interruptions. Private stations have been established to invite patients to provide verbal feedback post-service. Unlike conventional methods, this approach is proactive, provider- and patient-friendly, encouraging higher participation rates. The avatars will guide clients through questions about their care experience, ensuring relevant feedback that covers various aspects of service delivery.

Feedback will be processed using AI algorithms that transcribe, translate, and analyze the data, identifying key themes, sentiments, and areas for improvement. This allows for the rapid aggregation and interpretation of feedback, making it actionable. Healthcare providers can easily access these insights through a dashboard, providing real-time data, trends and recommendations for service enhancement.

The AI model's scalability, adaptability and low-cost are crucial. The system is designed to incorporate additional rare languages if needed, addressing the diverse linguistic landscape of Malawi and beyond. This scalability ensures that the solution can be extended to different regions within the country and internationally, maximizing its impact.

By strengthening the feedback loops in the healthcare system, we not only address the challenge of collecting client input but also lay the groundwork for a more responsive and patient-centered healthcare ecosystem. Patients become active contributors to the quality of their services, and providers gain access to insights that can drive continuous improvement and innovation, even where resources are scarce.

This solution represents a transformative step toward enhancing healthcare quality and patient satisfaction in Malawi. By harnessing these technologies to bridge the feedback gap, we empower clients, enable healthcare providers, and contribute to the development of a health system that truly responds to the needs and preferences of its clientele.

The Chichewa LLM will be made public after its development.

Our solution serves the clients within Malawi's healthcare system, particularly those who have been traditionally marginalized in terms of being unheard in healthcare delivery. We will first serve a subset of our own clients during the development of the tool, but if it proves successful, we will rapidly scale to include individuals and services across Malawi. Our focus is on those living in rural and under-resourced areas who face challenges in communicating their healthcare experiences due to literacy barriers or lack of robust feedback mechanisms. By focusing on these groups, the solution aims to democratize the feedback process, ensuring that even the most underserved populations can provide input on their healthcare experiences and have an active role in shaping the services they receive.

These clients are underserved directly by poor health services, but also in their capacity to provide and see the impact of their feedback on service improvements. Traditional feedback mechanisms, like written surveys, are often inaccessible to them due to literacy issues or are not available in their primary languages. Moreover, their feedback rarely translates into noticeable changes in service delivery, leaving them disenfranchised and disconnected from the systems that should serve them. The lack of practical client-centric feedback loops results in decision-makers and service providers not having the information they need to tailor services for specific healthcare challenges.

Our solution will address the needs of these underserved populations by providing a user-friendly, accessible platform for them to voice their feedback in their native language. By leveraging voice technology, we remove literacy barriers, allowing patients to articulate their experiences, concerns, and suggestions orally. This not only facilitates a more inclusive feedback process but also ensures that the data collected is rich, nuanced, and representative of the patient's genuine experiences and needs. It will provide an easy access to relevant information to the understaffed health care systems. An additional benefit from our work is that we will make the Chichewa language model available over a creative commons license, free for impact-oriented usage anywhere, in any service.

The system analyzes the feedback to identify trends, issues, and areas for improvement. This will be a game-changer for healthcare providers who normally have little of the information needed for targeted interventions, service enhancements, and policy changes. This will directly benefit the clients by making healthcare more responsive, client-centered, and effective. Over time, clients will see a tangible impact from their contributions, fostering a sense of empowerment and engagement with their healthcare providers.

By giving clients a voice and ensuring their feedback is a cornerstone of healthcare improvement efforts, we create a more equitable healthcare system that values and responds to the experiences of all clients, particularly those who have been historically underserved. This approach not only improves healthcare outcomes but also enhances client satisfaction and trust in the healthcare system, leading to a more cohesive and client-oriented healthcare landscape in Malawi.

The team of Jhpiego and Fortell is uniquely positioned to deliver this solution due to our deep technical and programmatic experience both in Malawi as well as other complex environments in Africa, Asia, and Latin America. Jhpiego can leverage and build on our experience in the design and development of innovations and client-facing AI tools.

The proposed solution has been pre-tested on staff and expert clients in Malawi, and Jhpiego is poised to test it with a larger client group. In Malawi through the Bill & Melinda Gates Foundation-funded ANC/PNC Research Collective (ARC), Jhpiego is testing the use of new diagnostic technology and AI to meet the needs of pregnant women and midwives.  In India, Jhpiego is partnering with the government to embed an AI-driven digital virtual assistant that provides concierge client services into health facilities. 

Since 1999, Jhpiego has worked with at-risk and hard-to-reach client groups in Malawi and maintains proximity to service recipients through our physical offices throughout the country. Through the CDC-funded Gateway (2017-2023) project, Jhpiego has brought person-centered and client-tailored HIV services to adolescent girls and young women (AGYW), female sex workers (FSW), men who have sex with men (MSM), and other key and priority populations. Through the USAID-funded EMPOWER project (2020-2025) as well as CDC-funded Project Dolo (2020-2025) and DOD-funded HIV prevention awards, Jhpiego has reached millions of clients across Malawi – including in remote and rural settings - with demand generation and health services including Voluntary Medical Male Circumcision (VMMC) as a prevention method for HIV.

Jhpiego will ensure that the design and implementation of our solution is informed and meaningfully guided by client groups. Through the proposed intervention, we will involve communities by engaging 400 VMMC clients to test the AI platform. Jhpiego will compare 400 AI-generated and 400 human-generated transcripts to train the voice model in Chichewa and test its fidelity. Additionally, 20 clients will be randomly selected for key informant interviews to test acceptability and to ensure that we incorporate client feedback into the final product. Jhpiego has experience incorporating client feedback and has conducted human-centered design (HCD) in Ghana, Kenya, India, Nigeria, and Tanzania. In Malawi, we have engaged client champions to put clients at the helm of implementation.  

Technical partner Fortell is an impact intelligence company that focuses on building monitoring and evaluation technology for NGOs. They were established in 2023 to take advantage of breakthroughs in artificial intelligence. Fortell has already successfully delivered monitoring and evaluation projects across three continents, and their clients include Oxfam, the Inter-American Development Bank, and the Catholic Medical Mission Board. 

Jhpiego and Fortell are equipped with staff that have the technical and in-country expertise to effectively test this solution. Jhpiego’s Dr. Jeroen van ‘t Pad Bosch has 20 years experience in the field and is leading a team that is representative of the communities that we plan to reach with this intervention. Fortell was founded by seasoned entrepreneurs Rob Symes and Chris Bracegirdle who have been working in the field for thirteen years.

We have developed a realistic human looking digital avatar (we are able to generate any type of face) which has been selected and pretested by Jhpiego staff and expert clients / demand generation experts. The avatar is able to introduce itself, explain its purpose and ask a pre-tested client satisfaction survey, all in well-understandable Chichewa.

At this stage, we are looking into the tool's fidelity and acceptability, and are not yet analyzing clients' input on design and quality of services.
We have trained 4 data clerks (one per clinic; 4 clinics in total) in administrating the tool to 400 clients and we have started to administer the tool to clients.

We will compare 400 transcripts that are automatically generated through AI with 400 transcripts that will be human generated, based on the audio recordings of the 400 clients that will use the tool. This information will be used to improve the tool's ability to interpret and speak Chichewa and to measure the development of the tool's fidelity over time. It will provide valuable information on the amount of data needed to teach a LLM a new rare language.

We will compare 4 x 100 summaries that are automatically generated through AI with 4 x 100 summaries that will be human generated, based on the audio recordings of the 400 clients that will use the tool. The expectation is that the AI will be able and gradually improve in picking up patterns in the clients' feedback. It will provide valuable information on the amount of data needed for the AI algorithms to be able to provide such feedback.

We will randomly select 20 clients for a Key Informant Interview (KII) which will be qualitatively analyzed to measure acceptability of the tool. Pre-testing and the first client exposures are already indicating that clients highly appreciate being asked for feedback and that they do like the interaction with the avatar.

We have an approved Non-Human Subject Research Determination from Johns Hopkins University.

In case of a successful development, we would like to take this to scale - with the appropriate protocols and approvals - in a pilot in which we would start to use the tool to actually measure client satisfaction data. A next step could be a comparative analysis with other existing methods to collect and use client satisfaction data.

Our intention is to make the tool public for wider use and benefit as soon as it has been proven successful.

Our intention is to proof that with AI we can develop equitable, user-friendly, scalable and adaptable data collection tools at low cost, that will give clients a voice, in particular in underprivileged populations, to increase client satisfaction and contribute to stronger health systems.

We believe that Solve would allow us to increase our network and our ability to share and test ideas, as well as allow us to effectively share this tool across this network, to be freely adapted to local circumstances and languages to benefit a wide audience.

Our solution significantly enhances the traditional feedback mechanisms, which are often hindered by low literacy rates, language barriers, and cultural factors. As described above, the core of our solution is the development of an AI-powered Chichewa voice model, which is disseminated through a digital avatar designed to interact with clients in a respectful, intuitive, and engaging manner, encouraging them to voice their experiences and concerns post-service. This is a major shift from the conventional feedback methods that rely heavily on written surveys or suggestion boxes. 

First and foremost, our approach leverages spoken language to ensure inclusivity and accessibility. This is particularly crucial in Malawi, where a significant portion of the population is illiterate and may feel alienated by traditional written forms of feedback. The use of AI allows for the collection of feedback in real-time, directly at the point of care. This immediate data capture enables quicker response and adaptation by healthcare providers, enhancing the agility of the healthcare system to meet client needs and expectations. The system not only transcribes and translates the spoken feedback but also analyzes the data to identify key themes and sentiments. This capability enables healthcare administrators to access processed, actionable insights through a dashboard, promoting informed decision-making and continuous service improvement.

By addressing the linguistic and cultural nuances of the Malawian population, we will not only improve the quality of feedback collected but also ensure that this feedback is representative of the broader community's experiences, leading to more client-centered healthcare improvements. We hope that this approach could serve as a catalyst for broader positive impacts in the healthcare sector by setting a precedent for the integration of AI and avatar technology in client feedback systems. We can inspire other organizations to adopt similar technologies, thereby enhancing the inclusivity and effectiveness of healthcare services across different regions and languages. By demonstrating the successful implementation of a culturally sensitive and technologically advanced feedback mechanism, Jhpiego could encourage healthcare providers, policymakers, and technology developers to collaborate more closely. This should bring about a stronger focus on improvements in Malawi’s healthcare quality through more client-centered practices.

The ultimate aim is to transform the healthcare market in Malawi and across similar regions by establishing a new standard for client engagement and feedback utilization. If we can generate evidence as to the effectiveness of this technology in capturing and analyzing client feedback, this solution could encourage healthcare providers to shift from ineffective feedback methods to more efficient, dynamic, technology-driven approaches. This transition would lead to a more responsive healthcare system where client insights directly influence service improvements and eventually policy making. The success of this model could potentially contribute to broader adoption of AI in other sectors, promoting a more inclusive digital transformation that prioritizes user-friendly and accessible technologies for all.

Activities:

Development of an AI-powered Chichewa voice model: We will create a language model that understands and processes the local Chichewa language, enabling clients to provide feedback in their native language.

Upskilling of staff: We will train staff in our clinics on how to introduce the avatars and how the process is going to help them in their work.

Deployment of digital avatars in healthcare settings: These avatars will interact with clients to collect feedback immediately after healthcare services are rendered.

Real-time data processing: Feedback collected will be processed in real-time, using AI to transcribe, translate, and analyze the data to extract actionable insights.

Immediate Outputs:

Increased client orientation: We intend to conduct staff training sessions across four clinics, with support materials made available outside of the sessions.

Enhanced data collection: We will focus on generating 400 client responses and processing them, providing a more accurate representation of client experiences.

Actionable insights for healthcare providers: The system will generate real-time reports that highlight areas needing improvement, allowing healthcare providers to make immediate adjustments.

Longer-term Outcomes:

Improved healthcare quality: With continuous and reliable client feedback, healthcare providers can implement targeted improvements, leading to higher quality care.

Increased client satisfaction: As services are refined based on client feedback, overall satisfaction is expected to rise, fostering greater trust in the healthcare system.

Scalable and adaptable model: Initially tested in Malawi, this model can be adapted to other regions and languages, potentially transforming client feedback systems globally.

Supporting evidence:

Although still quite nascent, research on AI in healthcare has shown that AI can enhance the accuracy and efficiency of data collection and analysis in healthcare settings:

https://www.ncbi.nlm.nih.gov/p...

https://www.mdpi.com/2071-1050...

Empirical evidence (albeit unfortunately not in our target context) also suggests that effective client feedback systems contribute to improved healthcare quality and patient satisfaction:

https://journals.plos.org/plos...

https://evidence.nihr.ac.uk/co...

Pilot testing: Initial tests of our AI model and avatar system have shown promising results in engaging clients and collecting high-quality data.

Impact Goals:

Our primary impact goal is to enhance the quality of healthcare services in Malawi by making them more responsive and client-centered through effective use of client feedback.

Specifically, we aim to:

Increase the quantity, quality and accessibility of client feedback collected, ensuring it is representative of diverse client experiences, particularly those from underserved communities.

Improve healthcare service delivery by enabling healthcare providers to make informed decisions based on accurate, timely, and detailed client feedback.

Enhance client satisfaction and trust in the healthcare system, leading to increased care-seeking behavior and adherence to medical advice.

These goals are designed to contribute directly to SDG 3.

Impact indicators:

Feedback Volume and Diversity: We will track the number of feedback entries collected monthly and analyze demographic data to ensure broad representation across different client groups, including those from rural or marginalized communities.

Service Improvement Actions Taken: We will monitor the number of changes or improvements made in healthcare services based on the feedback collected. This will include adjustments in clinical practices, client communication strategies, and facility enhancements.

Client Satisfaction Rates: Through follow-up surveys and additional feedback mechanisms, we will measure changes in client satisfaction before and after implementing changes based on initial feedback.

Healthcare Access and Outcomes: We will assess improvements in healthcare outcomes, such as reduced wait times, better disease management, and lower readmission rates, which can be attributed to more responsive service delivery.

Engagement and Empowerment: We will evaluate client engagement through participation rates in the feedback process and through qualitative assessments of how empowered clients feel to voice their concerns and suggestions.

Data will be collected through the AI-driven feedback system, which automatically aggregates and analyzes input using advanced algorithms to highlight trends, sentiments, and actionable insights. 

To ensure the integrity and accuracy of our data, we will implement rigorous data validation processes, including cross-referencing AI-generated insights with manual assessments and external health outcome data. Regular audits will be conducted to ensure the client collection methods remain effective and unbiased.

Our approach thus includes continuous feedback loops where insights from the data are used to refine the AI model and improve the feedback collection process. This iterative process ensures that the system evolves in response to changing needs and conditions in the healthcare environment.

By systematically measuring these indicators, we will not only track progress towards achieving our impact goals but also provide valuable insights that can inform broader health system improvements in Malawi and potentially other regions. This data-informed approach ensures that our solution remains aligned with the needs of the communities we serve, where we hope to drive meaningful and sustainable improvements in healthcare quality.

Fortell presents its proprietary solution that leverages avatar technology and AI-powered analytics to conduct qualitative and quantitative interviews at scale in 200 languages.

Fortell's unique solution streamlines the client satisfaction process, enabling the collection of perception and sentiment data efficiently. Respondents interact with avatars, and their responses are transcribed and analyzed using advanced sentiment analysis algorithms.

The solution focuses primarily on qualitative aspects, providing rich, context-aware insights into the impact of projects. Fortell's innovative approach eliminates the need for expensive in-person surveys, making it practical for portfolio-level assessments.

Recorded interactions will be processed by Fortell’s technical team using technology derived from the Meta’s “Voicebox” (NB NO data will be uploaded to Meta platforms) and “No Language Left Behind” projects and Amazon Web Services (AWS) resources to build a voice model (a.k.a. digital avatar) in Chichewa that can synthesize audio from text (speech generation) transcribe audio recording outputs, and generate summary interpretations through Fortell’s platform.

Jhpiego has 11 staff working part-time on the solution team; Jhpiego hired Fortell as contractor who has 5 staff working part-time on the solution team. 

Jhpiego has been working on strengthening health systems and innovations for decennia.

Our team has been working on this particular solution for six months.

As one of the most gender-equal organizations in global health, Jhpiego promotes gender equity not only within its programs but also within its organization. Currently, 45% of Jhpiego’s global senior management is female. Moreover, from country to country Jhpiego's staff reflect the cultural and racial diversity of the populations we serve. Jhphiego believes that each country knows best how to help its people and achieve its goals. Jhpiego listens, and support country teams' efforts by sharing the most up-to-date knowledge and technology that helps them lead for generations to come. Jhpiego puts its clients in the driver’s seat. That’s how Jhpiego builds resilient communities.

Mission

Jhpiego creates and delivers transformative health care solutions that save lives. In partnership with national governments, health experts and local communities, Jhpiego builds health providers’ skills and develops systems that save lives now and guarantee healthier futures for women and their families.

Vision

Self-reliant countries, healthy families and resilient communities. All women and families, regardless of where they live, having equitable access to high-quality, lifesaving health care delivered by competent and caring providers.

Since its founding in 1973, Jhpiego has been innovating to save the lives of women and families worldwide. From the first day, Jhpiego has been asking the question: How can we make lifesaving services available and accessible to the people who need them—all over the world?

Early on, Jhpiego established itself as a leader in reproductive health training. Beginning in 1974, Jhpiego held training sessions on family planning/reproductive health for doctors and nurses in the USA In 1979, Jhpiego started its first in-country training programs in Tunisia, Brazil, Kenya, Nigeria, Thailand and the Philippines. From 1987 through 2004, Jhpiego conducted three global Training in Reproductive Health Projects, funded by USAID. Beginning in 1993, Jhpiego published learning materials on long-acting family planning methods.

Over the years—to respond more effectively to the needs of individual countries—Jhpiego became increasingly field-based and established its first field office in Kenya in 1993. Today, Jhpiego has field offices in more than 30 countries and works in 40+ worldwide. Similarly, Jhpiego’s programming areas have expanded to meet changing needs in the field. In addition to family planning and reproductive health, Jhpiego now has expertise in maternal and child health, infection prevention and control, HIV/AIDS and infectious diseases.

Jhpiego’s work has also expanded to address reproductive health policy and guidelines and to support health systems strengthening. For example, in 1996 in Brazil, Jhpiego launched a performance and quality improvement approach, now known as Standards-Based Management and Recognition (SBM-R), which has since been implemented in 30 countries. SBM-R empowers health workers and facilities to improve the performance and quality of their services by providing them with the tools and methods they need to make decisions, solve problems and innovate at the local level.

Jhpiego has been present since 1973 and has a solid funding base.

This current solution is being developed using Jhpiego's general / unrestricted funds as Jhpiego seeks to develop innovations that support the strengthening of health services globally.

Our pre-testing and the experiences with the first clients being exposed to the tool has shown an abundant appreciation by both clients and health care providers. Clients appreciate both the fact that they're being asked for feedback, as well as the tool itself. Healthcare providers appreciate the simplicity the tool offers to get valuable input from the clients on design and quality of received services.

Jhpiego and Fortell are jointly aiming at developing this tool at minimal cost to maximize scalability and sustainability and will make the tool publicly available once developed.