FLW Performance Monitoring and Feedback System (FPMS)

COVID-19 made us accept the harsh vulnerabilities and limitations of our already stressed healthcare systems. It accentuated the importance of well-trained, well-equipped, and highly motivated frontline health workers (FLWs) in place, despite healthcare spending making 10% of global GDP.  The World Health Organization estimates a global shortage of over 15 million health care workers by 2030. The unmet need for formal healthcare providers jeopardizes our capacity to achieve universal health coverage as part of Sustainable Development Goals (SDGs) by 2030, making it inevitable to prioritize supplementing the formal health workforce with FLWs. FLWs are even more pivotal to health strategies of the developing countries which are experiencing a growth rate of 6% in annual healthcare spending with prominent demand-supply gaps, which are economically unviable to meet. FLWs globally, while regularly overcoming deep-rooted cultural, socio-economical, and geographical barriers, have been at the forefront of serving the marginalized population, primarily rural. Studies show that FLWs could reduce maternal and under-5 child mortality rates by 42-78% and 13-60% respectively, while remaining a highly cost-effective medium to ensure Health for all, with possible economic returns of 10 USD for every 1 USD spent. FLWs focus on preventive care and disease management, which save costs for communities and country health systems. 

However, despite the proven effectiveness and imminent need of FLWs, health systems constantly face the dual challenge of varying work performance and motivation levels among FLWs, highlighted by high rates of attrition, of up to 44% in low and middle-income countries. The impact of reduced FLW performance directly implies clients missing life-saving vaccines and health check-ups. Hence, timely strategies to identify and address any performance bottlenecks and provide FLWs with constructive assistance, supportive supervision, training, and organizational and infrastructural support play a significant role in the success of the FLW programs. 

The recent advent of digital health and information systems has prompted the extensive application of digital interventions to facilitate and deliver these strategies efficiently. In 2018, the World Health Organization recommended the use of digital health tools as integral to the effective implementation of national FLW programs. Research has found digitization of the FLW program offers three key advantages:

• Improved  access and quality of services delivered by FLWs

• Efficient training and client management by FLWs

• Strategic use of data improving programmatic decision making, feedback, and facilitating related research and evaluation.

Therefore, health systems especially in developing countries need effective FLW performance monitoring systems in place for feedback and organizational decision-making. Such digital performance monitoring systems should not only assist health managers by providing continuous access to data, useful information about FLWs, and capabilities to measure the data seamlessly but should also improve the objectivity of feedback and performance appraisals. Furthermore, in addition to providing performance information, strategically designed performance monitoring and ranking systems, can also contribute to worker behavior changes.   Ultimately, such systems facilitate improvement in worker retention, motivation, and performance levels. 

However, recent research stresses the need to focus on better information support for FLW management in health. Data utilization and strong information support are key drivers to maximize ongoing digital advancements in health to improve the health system’s capacity to plan, monitor, implement, and evaluate, while focussing on performance monitoring, personalized feedback, and coaching for FLWs. Furthermore, the possible application of machine learning techniques,  combining prescriptive and predictive analytics can improve and proactive FLW feedback and decision-making substantially. But applications of machine learning techniques for FLW monitoring systems are rare,  especially in developing country health systems

Our Solution: We have developed an FLW performance monitoring and feedback system (FPMS), which simultaneously provides actionable information for policymakers, health managers, and frontline health workers. Using existing FLW databases (e.g. HMIS, DHIS2, Financial management system, etc.) and employing simple statistical analytics complemented by machine learning-based predictive analytics, we present personalized and usable data through an interactive dashboard, built using design principles anchored in the Goal Setting theory.

What does it do: Firstly, the FPMS allows policymakers to visualize predicted FLW performance at the beginning or earliest into the year, moreover, it categorizes the FLWs as high, medium, and low performing by geography and in each specific area of their work (e.g. maternal health, newborn health, immunization, family planning, etc.). This could allow policymakers, especially in resource-scarce, low and middle-income countries (LMICs) to plan, prioritize, and focus on specific areas of improvement and maximize the health outcomes per dollar spent. During the year, FPMS allows real-time actual performance data vis-à-vis the predictions.

Secondly, the health managers, who work directly with the FLWs on a daily basis can get benefited in a similar manner as above while planning training and field activities, FLW interactions, and feedback sessions. The FLW-wise granular data of actual and predicted performance allows for a personalized and tailored supportive supervision approach. 

Thirdly, the FPMS allows FLWs to identify specific areas of personal improvement and facilitates self-monitoring and personal goal setting and then monitoring their performance with respect to the set goals.

Finally, FPMS at the health system level will also provide an opportunity to investigate the circumstances under which some FLWs are doing good against others, and inform other health system components and actions needed such as shortage of supplies, human resources, and skills, among others. 

How is it developed - Design Science Approach: The system is conceptualized based on the task motivation theory of Goal Setting, facilitating the choice of setting one’s own individual goals as well as providing provisions for the management and supervisors to set goals for the FLWs based on the health system requirements. We have theoretically formulated and validated the “know-how” of designing such a system for facilitating re-use and replication by practitioners in other similar contexts. The design proposition also explores the application of machine learning techniques for information retrieval thus encouraging both proactive and reactive decision making facilitating appropriate and timely remedial strategies at the individual and policy levels alike. We have adopted the Design-Science based approach to achieve the system objectives. Design-Science is a popular methodology applied in information system development to contextualize the technical process and brings human-centric principles to it. Our design process was carried out through multiple iterations of the build-evaluate cycles attributing to the core principle of iterative development. During its first iteration, requirements and principles for the system were formulated in the form of prescriptive statements building on assertions from the established task motivation theory of goal setting. These statements were evaluated through interviews of experts in the domain using a re-usability framework. The prescriptions were then revised based on the responses from the stakeholders (at policymakers, health managers, and FLW level).  Further, in the second iteration, a prototype of the system was developed for FLWs workers (called ASHAs, a class of FLWs in India) in Uttar Pradesh, India using these prescriptions for field testing of the proposed system. The following prescriptions were eventually formulated for the design of FPMS:

• Principle of Goal Setting

• Principle of goal specificity 

• Principle of hard goals 

• Principle of time-bound goals 

• Principle of small wins

• Principle of modeling

• Principle of organizational support

How is it developed - Technology:

The FPMS dashboard is developed using R and Shiny - which is a visualization framework for R, and is hosted in a cloud server using dockerized environment. The server-side processing and database (MySQL/Postgre) wrangling is done using Python. The machine learning model and ensemble algorithms are developed using the CARET package in R.  While the current system uses R Shiny for visualization, a hybrid mobile application, using Flutter, is currently under development  to enable better FPMS access on Smartphones (both on Android and Mac). FPMS is data source agnostic and can leverage any data source (e.g. country HMIS, DHIS2, financial information systems, etc.).

Pilot Deployment of FPMS:

We are piloting FPMS with the support of the National Health Mission (NHM) of the state of Uttar Pradesh, India.  The state has a population of 230 million and over 300,000 FLWs. FLWs are called Accredited Social Health Activists (ASHAs) in India. The data source for FPMS (for NHM, Uttar Pradesh) is the existing digital financial records information system, which captures the monthly incentives claimed by ASHAs. We have transformed the well-audited financial data into a standardized and comparable metric indicating the proportion of beneficiaries covered by an ASHA under each health service category (called coverage index). The coverage index is then used to rank, aggregate, and analyze monthly program performance (individual ASHAs and overall). Moreover, FPMS uses a coverage prediction ML model, which generates predictions of the prospective coverage index category an ASHA will fall into by the end of the year.  FPMS can be leveraged to identify low-performing ASHAs at the beginning stages of the program (e.g. beginning of a financial year) and easily retrieve individualized feedback information for them so that the health system can focus on these ASHAs and initiate timely remedial measures thereby improving their task motivation and performance. 

FPMS attempts to address the gaps in accessing effective information for monitoring and providing contextualized feedback for FLW performance improvement and motivation. FPMS aims to equip all key stakeholders responsible for FLW performance and motivation, with timely and actionable information. Motivated FLWs would mean more people receiving timely healthcare services resulting in higher FLW income and low attrition rates leading to resilient and effective health systems. 

Hence, firstly FPMS directly enables FLWs to self-monitor their performance, set performance goals for themselves, and track the achievement of these goals. Specifically, for ASHAs (a cadre of FLWs) in India for which the system prototype is built, improving their coverage would not only mean more people getting health services and better community health outcomes, but also increased monthly earnings for FLWs as the program follows a performance-based payment system. Secondly, for health systems and policymakers, the proposed system provides easy access to performance information for appropriate and focused planning and organizational support. Lastly, for health program managers, the system provides the capability to granularly measure, analyze and compare the performance data in a timely manner with minimum effort and provides appropriate information for constructive and specific individualized feedback and motivation based on a strong theoretical foundation of task motivation and goal setting.

Tattva Foundation, since its inception in 2015  has been involved in multiple ICT projects in the frontline health worker space. Tattva’s FLW focussed innovative and pioneering interventions in the past have been highlighted by the WHO and National Health Mission, Government of India. We have been in constant and continuous interaction with FLWs as well as the health department in relation to multiple ongoing projects pertaining to their training and supervision, reporting, incentive payments, etc. As a result, our team is well-versed in the contextual requirements of such systems and has a clear idea of the expectations and capabilities of both FLWs and their managers. For this idea, in particular, a participatory design process had been followed with the last mile FLWs to have a more holistic system design for their motivation. Moreover, constituted of a multidisciplinary team of experts from fields including but not limited to computer science, economics, mathematics, and behavioral sciences with a background in public health, our team is well equipped to tackle human resource problems in health at large with strong theoretical foundation and practical relevance. In addition to all this, to further generalize and incorporate a global perspective to our ideas, our team has conducted multiple consultative sessions with global experts in the field.  

• Financial and operational partnerships for scale up and next phase development of the project. With such strategic partnerships, we intend to conduct the following activities:

1.    Conduct a Randomized Control Trial to measure efficacy of the model in improving performance of frontline workers so that Universal Health coverage of critical health intervention indicators can be achieved.

2.  Conduct a cost-benefit analysis for scalability of the solution in Low- and middle-income countries.

3.     Run a full-fledged model in the state of Uttar Pradesh for a period of one year to demonstrate the effectiveness of the solution and take appropriate course correction measures. 

We would also like to benefit from the MIT solve community’s learnings and guidance to:

• Improve and accelerate a perception change and capacity building towards the culture of data use among grassroot level communities and organizations such as FLWs.
• Attain a wider acceptance for the solution and trigger conversations regarding it among global stakeholders.
• Overcome the barriers of information asymmetry existing in such systems making performance information accessible to all stakeholders in the program alike for more harmonious performance improvement efforts.

The following innovative aspects of our solution (FPMS) make it novel:

• Theory-informed, replicable, and data-source agnostic nature of the solution design makes it easier to integrate with other performance-related data sources and its uptake by other national and global programs. 

• FPMS seeks to improve upon the existing practice and culture of reactive decision-making systems with a dynamic system having the power of both,  reactive and proactive decision-making, for FLW management. 

• The existing performance management systems are designed for summarized and generic analysis which hinders personalized feedback and performance improvement roadmap for FLWs. FPMS allows multi-dimensional granularity with a drill-down of the performance data to a specific health service for a specific FLW for her feedback and improvement. The summarization capability of FPMS allows to view the retrieved information at state and national levels for improved planning, efficient resource allocation, and focused feedback.

The novelty of FPMS, besides its capability to leverage any existing proxy data sources, is also that it does not increase the burden of data collection on FLWs or other stakeholders in the program. Moreover, FPMS  provides all key stakeholders with readily available, granular, and timely performance information which can lead to improved feedback, supervision, and management, ultimately creating a multiplier effect on improved population health outcomes.

We have planned our impact goals for the product in two phases: 

· Short term (12-16 months)

1.       Improve FPMS adoption among FLWs, Health managers, and policymakers. 

FPMS followed a rigorous design science approach for evidence based, participatory product and feature designs. In order to achieve FPMS goals its use among three key user groups will be important ( FLWs, Block program managers, health system decision makers). As we are not adding the burden of data collection by using an existing well-established data source, we have an opportunity to train the key user groups in strategic use of data using FPMS. We intend to e-train and monitor the usage of the system across 160,000 FLWs , related health managers, and state level policymakers in the state of Uttar Pradesh, India.  

2.       Improve the frequency of use of PFMS by different user groups for monitoring and feedback with respect to FLW performance. Ensuring that FPMS creates tangible value for all stakeholders.

The immediate objective of the system is to reduce the stress on the health system in retrieving feedback information from data sources and provide the users with specific, personalised, actionable information in a timely manner and with sufficient  granularity. This will improve the monitoring quality and provide constructive feedback for FLWs. In order to leverage the platform efficiently, we will handhold initial review meetings and demonstrate frameworks and pathways for retrieving and utilizing the right information from the platform. We will also conduct activities with FLWs to improve their capacity to self- monitor and optimize their performance. Besides we will create e-handholding platforms for continual support and capacity building.

· Long term (24-60 months)

1.       Improve the motivation levels and overall health service coverage of FLWs in Uttar Pradesh, India thus improving the SDG Goal 3 achievements of the state.

FPMS facilitate strategic use of data for FLW performance and feedback will result in high performing, and motivated FLWs, who will receive specific inputs to improve their performance and therefore number of beneficiaries served by them, which will ultimately improve community health outcomes for a greater number of people. 

2. Strong health system support and M&E framework for FLW programs

FPMS will allow the health system and health manager to judiciously plan and use the limited resources and focus on the specific needs and areas of FLW performance.  More low performing FLWs will shift towards average or better performing groups.

3. Evidence generation and scale-up (horizontal and vertical)

FPMS will be tested through a natural experiment and the evidence will be used to improve the health system inputs to create a high performance enabling environment for FLWs. We intend to publish our findings in peer reviewed journals and also establish partnerships with global academic institutions.

We expect the system to include more data-sources (vertical scale-up) in future to better capture and predict the performance of FLWs to identify specific areas of improvement which can be provided as feedback. Similarly, we expect FPMS to be used in other states of India and globally to create to create a global data-analytics platform for FLWs (horizontal scale-up)

We will be adopting the constructs of  theory of change as described in detail in the next response for identifying the indicators. The output and outcome indicators are listed below (not exhaustive). These indicators are developed based on the programmatic needs and to operationalise our theoretical constructs through the FLW performance management and feedback system (FPMS).

OUTPUT indicators

Customized and approved FPMS digital tool

# of FLWs registered in FPMS

# of health managers registered FPMS

# of health system policymakers registered in FPMS

SUD tool kit developed

# of modules developed and approved by health system 

Training conducted

# of FLWs trained in FPMS and SUD tool kit

# of health managers trained in FPMS and SUD tool kit

# of health system policymakers trained in FPMS and SUD tool kit

Use of FPMS digital tool 

# of FLWs regularly using  FPMS

# of health managers regularly using  FPMS

# of health system policymakers regularly using  FPMS

e-Helpline established and publicized

# of support requests received from health managers or FLWs using e-Helpline 

OUTCOME indicators

Health Managers Level

Improved performance monitoring and feedback by health managers in FLW meetings 

Proportion of FLW meetings where granular FPMS data was used for reactive monitoring.

Proportion of FLW meetings where predicted annual performance data was used for proactive monitoring and course-correction.

# of FLWs assigned a service category wise and, time-bound goal  by health managers for better feedback and motivation of FLWs

Frontline Health Worker Level

Improved self-monitoring, performance awareness, and motivation of FLWs.

Proportion of FLWs aware of their assigned goals. (goal specificity and timebound)

Proportion of FLWs monitoring their progress towards the assigned goals. (small wins)

Proportion of FLWs identifying role models for them to follow from their peers under each service category. (modelling)

Proportion of FLWs using  their predicted performance for better course correction in each service.

Proportion of FLWs aspiring to reach top 10 positions in performance for each service.

Proportion of of FLWs who achieved previously assigned goals in each meeting

Health System  Level

Improved program monitoring and planning

Proportion of of FLWs program planning activities which were informed by FPMS data

Proportion of predicted low performing FLWs which ended up as average or good performers.

*OUTPUT indicators (described in above question 3)

**OUTCOME indicators (described in above question 3)

***THEOETICAL foundation

Principle of Goal Setting :- The deployed Information system should allow health managers (users) to provide feedback with respect to set goals for FLWs (mechanism), because giving feedback with respect to the set performance goals motivates an individual in modifying his/her behaviour to reduce the discrepancy between the goals and their performance thus improving the performance levels (rationale).

1. Principle of goal specificity :- FPMS(enactor) will provide a metric to measure and monitor FLWs performance including but not limited to metrics pertaining to physical effort, quantity and quality of services, costs, profits and job behaviors (mechanism), because setting specific goals based on the quantifiable performance metric can enhance their awareness of what constitutes an effective performance and can facilitate proper measurement and feedback of progress towards the goal (rationale).

2. Principle of hard goals :-FPMS (enactor) will allow the health managers to set a minimum threshold goal (default value) for all the FLWs (mechanism), because challenging and difficult to achieve goals can nudge FLWs to commit to their goals better and achieve higher satisfaction levels on attaining those goals (rationale).

3. Principle of time bound goals :- FPMS (enactor) will provide a time limit for the FLWs to achieve those goals (mechanism), because specific hard goals with a boundedness on the time to achieve it would result in higher performance per unit time (rationale).

4. Principle of Outcome Expectancy:- FPMS (enactor) will  provide information regarding the expected performance that will be achieved by the end of time period leveraging on past performance data and prediction models and the desired performance trend to be followed to achieve the set performance goal (mechanism), because this information would make the FLWs aware of their outcome expectancies leading to sustained goal commitment and thereby improving their performance levels(rationale).

5. Principle of small wins:- FPMS (enactor) will set proximal (near) goals for FLWs based on the distal (faraway) goals assigned to them and provide feedback on their achievement (mechanism), because feedback on such small wins leads to better self-efficacy leading to improved performance levels (rationale).

6. Principle of modelling:- FPMS (enactor) will provide performance information about their peers with whom the FLWs identify the most (mechanism), because this information can instill a modelling effect leading to higher self-efficacy thereby improving their performance levels (rationale).

7. Principle of organizational support:- FPMS (enactor) will  should provide information for organizations to plan for and support FLWs to achieve their goals via prescriptive and predictive analytics(mechanism), because appropriate organizational support, timely planning and policy decisions will ensure the attainment of goals of FLWs thus further improving and sustaining their performance levels(rationale).

The core principle that guides our solution development (Design Sciences) offers a scientific and participatory design process. This approach entails continuous evaluation and iterative development which constantly incorporates the changes in the user needs and sentiments. 

The FPMS dashboard is developed using R and Shiny - which is a visualization framework for R, and is hosted in a cloud server using dockerized environment. The server-side processing and database (MySQL/Postgre) wrangling is done using Python. Core packages from R used for data processing, modelling and visualization includes tidyverse, dplyr, caret and highcharter.

While the current system uses R Shiny for visualization, a hybrid mobile application, using Flutter, is currently under development  to enable better FPMS access on Smartphones (both on Android and Mac). FPMS is data source agnostic and can leverage any data source (e.g. country HMIS, DHIS2, financial information systems, etc.).

Currently, we use FLW Performance payment MIS operated by the National Health Mission, Uttar Pradesh to generate the input data for our predictive models and the reactive reporting system. The data is collected on a monthly basis for each of the 160,000 + FLWs in the state of Uttar Pradesh, with a population of over 230 million.

The FLW performance payment system MIS has a registry of all FLWs in the state of Uttar Pradesh, to uniquely identify the FLW. The FLWs are mapped to villages, sub-health centers, blocks, and districts. 

Each FLW (called ASHA in India), every month submits the number of beneficiaries served by her in the village and type of service delivered to the beneficiary (pregnancy care, immunization, family planning etc.) . This data is digitized at the Block level and after approval from the medical officer, the FLW is paid her monthly incentive. The existing system also tracks any delays and defaults in the process of payment to FLWs. A central reporting system monitors the payment progress. The system is linked with banks and upon approval, an automatic payment is made. FLW is always kept informed of her payment status through automated SMS reminders.

The system was also developed by us (Tattva Foundation) in the year 2016 and has been in use since then. While it has reduced the delays in FLW payments, the system has also reduced paperwork for block managers, and time taking approval process for block medical officers as these processes are now digitized. Thus the system creates value for all key stakeholders. 

FPMS is interoperable with any kind of existing digital Health Management Information System (HMIS) such as DHIS2, or financial accounting system. The digital system can be tailored to meet the data needs of FPMS. 

We are a female led not-for-profit organization with team members who belong to four different religions, eight castes (social groups) or tribes, come from six different states of India and speak a dozen languages. 

We work at the intersection of technology, governance, and communities, with an aim to scale technology and innovation led solutions for last mile health workers and communities. We have in the past prioritized work in very challenging geographies from Afghanistan and Ethiopia, to insurgency prone states of India, namely Nagaland and Jharkhand. We have intentionally chosen to make our head office in the capital of the most populous state of India with a population of 230 million, primarily rural and with over 300,000 FLWs. Moreover, the core principles of equity and inclusivity is reflected in our extensive consumption and contribution towards the various open-source technology stacks.  

At Tattva, we thrive in a diverse environment as its most conducive to “inclusive innovation”, especially when the solutions have to transcend the boundaries of socio-economic status, digital divide, gender and social inequities, and geographical locations, to reach the last mile.

We propose to develop and test the solution further with the help of innovation/research grants. 

Later, we intend to get into service contracts with state governments/national governments to provide product, training, and model enhancement services on an ongoing basis. 

We intend to get into research tie-ups (MoUs) with leading, like minded academic institutions of public health and information systems to continue to evolve with advance data science techniques and enhance interoperability architecture with government national registries and digital health backbones (such as National Digital Health Mission, in India). 

We have received a service contract from UNICEF, Uttar Pradesh to customise the FPMS for maternal and newborn mortality related performance management of FLWs. The data source for this assignment was a labor room application, which we have developed and launched in over 4,000 public health delivery points in Uttar Pradesh. Approximately 1 million deliveries have already been captured by the system (called MaNTra). We plan to launch the FPMS FLW performance monitoring system by the end of next quarter after required customisation and integrations.