Strongr: AR Mental Health Caregiver Training

As is frequently emphasized in the news, the US is going through a mental health crisis. On one side there are increasing rates of mental illness, especially in the wake of COVID-19, and on the other are major gaps in access to care. According to a 2020 report, only 26% of the need of the US population is able to be met by the current supply of mental health professionals.

These problems are causing suffering across demographic groups, but they disproportionately affect people of color. In New York City, differing statistics across the boroughs and socioeconomic groups reveal stark inequities in access to care. In medically underserved neighborhoods such as Harlem (where Blacks and Latinxs make up 43% and 32% of the population, respectively), racial and ethnic minorities face multiple barriers to getting the mental health care they need.

In addition to having to contend with lower relative numbers of mental health professionals (about half of the state average), these marginalized groups have to grapple with the majority of NYC psychologists (84.2% White) lacking cultural and linguistic competency, greater risk of mental health challenges, and greater prevalence of stigma within their population, which can discourage individuals from seeking care. All of this adds up to racial and ethnic minorities being particularly disadvantaged with regards to getting support from critical mental health resources (only 30.3% of Black and 39.3% of Latinx New Yorkers with current depression received mental health care in the last 12 months, compared with 58.3% of White New Yorkers). 

Public health-focused organizations such as CUNY's Center for Innovation in Mental Health (CIMH) have been working to change these statistics and expand access to services for these communities with community-led R&D programs. CIMH's Harlem Strong Initiative seeks "to address the syndemic risks of mental health, social risks, institutional racism, and COVID-19 through a neighborhood based multisectoral coalition of ... organizations focused on mental health integration and coordination of care across the Harlem community."

One of CIMH's primary strategies for expanding mental health resources is capacity building through task-shifting, where certain tasks are given to individuals with less training than professionals. CIMH develops and offers trainings in evidence-based mental health care practices for community organizations with a focus on marginalized groups. Community leaders are thereby able to apply evidence-based practices (i.e. screening, psychoeducation, and referrals) in nontraditional healthcare settings such as schools, churches, and social service agencies, better engaging their community members where they are, increasing awareness of mental health issues, and reducing stigma.

Strongr helps expand the accessibility and effectiveness of CUNY's trainings in evidence-based mental health care practices by offering on-demand training simulation sessions in augmented reality (AR) with virtual AI patients in a variety of scenarios. Our solution helps close the gap between training and implementation by offering realistic practice of core mental health skills, greatly expanding on current practice offerings and increasing trainees' confidence in applying skills, emotional connection to content, and retention of skills. As we are in the prototyping and pilot testing phase, Strongr as a solution is in need of further development and testing.

So far, the Strongr product consists of one training scenario that we developed in collaboration with CUNY's training and community leader staff and user tested with a group of prospective trainees in a pilot study. We are currently in the phase of data analysis and paper writing for this study and seeking support for further developing and validating our unique training offering.

In our current setup, the trainee can access our training by reserving a time at a designated community training space, wearing AR glasses loaded with the training simulation, and interacting with a virtual patient sitting in a chair in front of them while practicing administering the PHQ-4 screening for anxiety and depression. The core technology consists of:

1. Magic Leap 2 AR glasses

2. Unity game engine

3. A customized Ready Player Me avatar

4. Mixamo animations

5. The Salsa LipSync real-time avatar animation plugin

Our solution's road map consists of:

1. Improving the fidelity and variability of character animations.

2. Increasing the number of avatars to represent a wider range of demographics and scenarios.

3. Building out additional curriculum and scripted scenarios.

4. Implementing conversational AI models, whether custom integrations of services like ChatGPT with a text-to-speech plugin such as Google Cloud TTS or a third-party integration such as Inworld.AI (for fully integrated AI 3D  characters in Unity). This will allow more personalized training experiences and dynamic realtime feedback based on trainees' dialogue, tone, and body language.

5. Developing in-app automated assessments of relevant training data such as measures of sentiment analysis, eye contact, quality of skills implementation, and duration/number of training rounds.

Currently our solution depends on trainees being onboarded with a learning module about a particular mental health care skill, such as screening for symptoms with the PHQ-4, which we may eventually explore building out AR versions of within our platform. For our first phase, we are focusing on supplementing CUNY's existing trainings with ongoing practice sessions for trainees to improve their ability to learn, retain, and apply mental health skills covered in the pre-existing online modules and workshops.

In a recent research report from professional services firm PwC, virtual reality training was shown to train VR learners 4x faster than in the classroom and make them 275% more confident to apply the skills learned after training, 3.75x more emotionally connected to content than classroom learners, and 4x more focused than their e-learning peers.

Link to product demo

Our solution directly serves the lives of organization members within a multisectoral coalition of community, faith-based, mental health, social service, health, and city organizations focused on mental health integration and coordination of care across the Harlem community (and potentially other communities as we continue to scale).

As mental health needs grow in NYC neighborhoods, the supply of mental health resources has not been able to keep up, let alone manage the current level of mental health needs, and organization members who currently serve diverse community members in a variety of capacities are not currently offered systematic training in the area of mental health skills and task-sharing in a scalable or sustainable way. Strongr AR Mental Health Caregiver Training will expand the scalability and effectiveness of existing training offerings that CUNY's Center for Innovation in Mental Health has been researching, developing, and implementing in small batch trials in the Harlem community. Our solution will address the trainees' needs for more wide-scale access effective, quality training in evidence-based techniques to better serve their community members' often unaddressed mental health needs, and in the process offer them opportunities to receive certifications and new job and economic opportunities based on these trainings.

Our solution indirectly serves the lives of diverse community members who are served by these organizations. By training more community leaders equipped with evidence-based mental health skills, our solution will help to promote mental health awareness and access to resources across the community, increase linkage to care through care navigation (referrals), build capacity in mental health task-sharing skills, support continuous quality improvement (through data-rich dashboards displaying aggregate data and analyses from our training simulations), and strengthen community resiliency.

In addition, as we grow as a team and expand our internal resources, we will seek to include community members increasingly in our organizational processes, going beyond participatory co-design strategies and actively involving community members in the research and AR and AI technology development and design processes. We will aim to provide opportunities for additional skill-building, career development, and technology innovation in the rapidly growing fields of immersive reality, artificial intelligence, workforce development/training, and digital mental health.

Overall, we have a diverse team composed of Black, Latinx, Asian, and White teammates in collaboration across leadership positions (from principal investigator to project manager and design lead). Our team members are not directly representative of the communities we are serving (i.e. neighborhoods in Harlem), and we have addressed biases or issues this may cause by incorporating participatory design strategies and regular input from direct community members into the process of our solution development. For example, we have conducted user testing sessions and solicited early feedback from CUNY's community outreach director and have conducted usability testing with other community leaders from Harlem community-based organizations to ensure that we are on the right track with the experimental technologies we are innovating with and doing the best we can to respond to their needs. We will be analyzing the data from the usability testing to inform our future design and development iterations once we progress to our next stage of funding.

In addition to soliciting community feedback on the form (i.e. usability) of our innovative training approach, we are adapting our training content from the curriculum that CUNY's Center for Innovation in Mental Health developed in collaboration with community members and continue to develop through current and future studies that put the communities' input and agendas front and center to their objective and methodology.

We have been designing and developing our latest AR glasses-based training prototype on the Magic Leap 2 device in the Unity game engine for several months now (building off of an initial prototype created during the XR Brain Jam last July). We have successfully developed it to the stage of early user testing as well as a small-scale usability study with community leader  participants to gather feedback and inform our next iteration.

We plan to continue to seek funding and build out iterations that have more features. Once we have a more robust prototype (including features described in our solution description response above), we will measure the effectiveness of our training and incorporate feedback from that study into future iterations until we have a proven product offering that effectively addresses the needs of current and future trainees.

At the moment we are in the research phase, so our solution only serves 5-8 community leaders in the sense of inspiration and excitement for continued co-development of our service/tool.

Financial: The minimum of $10,000 and longer-term exposure to other potential funding sources would greatly assist us in continuing to develop our training software and building out our team or transitioning current teammates into more committed roles (we are also preparing materials for other grant opportunities, such as an NIH grant).

Technical: We could use help from Solve's network of expertise to build out our design/development team, especially with the tuning and implementation of conversational AI / real-time large language models.

Cultural: We would love to be a part of a peer network with likeminded or similarly oriented organizations and share knowledge and support. In the longer term, we envision extending our partner network beyond CUNY and potentially beyond the US, and it would be great if Solve could help us get more exposure to digital and behavioral health and relevant cultural organizations.

Market barriers: We would appreciate Solve's strategic advice on how to scale nonprofit products and services and / or create for-profit offshoots of research projects.

We are developing a tech-driven solution to mental health education and training - by leveraging artificial intelligence and augmented reality technology, we’re able to personalize our training while increasing trainee engagement and retention through dynamic simulation exercises with virtual patients, in an accessible, on-demand, and scalable way. We also specifically build for and with frequent feedback from low-income and minority communities through our partnership with CUNY's Center for Innovation in Mental Health and their Harlem Strong Initiative.

Past training programs have been in-person workshops, which are time-consuming, expensive (on-site training staff and supervisors), and have uniform content with limited practice exercises, or online webinars, which struggle with engaging larger groups in addition to the aforementioned issues. As we continue to develop our training, we plan to expand to a variety of training scenarios and virtual patient demographics, offer more language options and automated translations, and implement conversational AI, which will allow each training session to be unique as the virtual patient dynamically responds to trainees' dialogue (and ideally even tone and body language), and allow the training to adapt based on trainees' key knowledge gaps and learning styles. While initially we aim to extend the resources and effectiveness of current training offerings, we envision our product growing its scope of content to eventually provide comprehensive training packages in fundamental mental health skills. 

As demand for mental health professionals grows, especially those with cultural sensitivity, our training offering, if successfully developed and launched, can help to efficiently and effectively meet that demand and rapidly expand the available mental health workforce for the communities with the greatest needs.

While the market for virtual soft skills trainings is growing with the increasing adoption and development of AR/VR technologies, it is still the early days, and core parallel technological developments in artificial intelligence that will drastically improve the fidelity and naturalness of virtual patient simulations are only just maturing to the real-time application-ready stage.

The next year:

1. Develop and validate a full-fledged prototype for 1-5 mental health skills modules and enroll 25-50 trainees into our AR training program.

The next 5 years:

1. Continue to develop and validate our AR training program as we build out more content to cover the full breadth of the CUNY Center for Innovation in Mental Health's Harlem Strong Mental Health Task-sharing curriculum while expanding the fidelity and demographic range of virtual patients within our practice scenarios; enroll 500-1,000 trainees in our AR training program.
2. Develop professionally recognized training certificates that can aid trainees who have completed our training program in expanding their current position's responsibilities / economic value or finding new employment opportunities in their specialty.
3. Launch a study measuring the impact of our program graduates' application of evidence-based mental health interventions on their communities members' mental health.

How we plan to achieve them:

1. Continue to build out our team and software with the help of additional grants, whether through programs like Solve or NIH grants.
2. Continue to build our partner network with educational institutions like CUNY's Center for Innovation in Mental Health to keep our training curricula up to date with the most effective evidence-based strategies, as well as with community-based organizations affiliated with CUNY or in other regions from which to recruit prospective trainees and gain regular co-designing / user testing feedback

While we are at the pilot stage of testing, we are focusing on assessing the usability and usefulness of our prototype with quantitative measures including the system usability scale (SUS) and analyses of qualitative data including interviews and cognitive walkthroughs. Once we have developed a viable training, we will measure more humanitarian impacts of our training on those we serve and assess the following indicators: -mental health outcomes in the communities we serve -quantity and quality ratings of mental health providers in the communities we serve -socioeconomic data of the communities we serve -employment rates and types of employment for graduates of our program

Inputs:

• Professional mental health trainers and researchers
• Evidence-based mental health skills curriculum and training materials for community leaders
• Pre-existing networks of trust with community organizations
• Training facility
• AR glasses

Activities:

• Identify and recruit potential mental health task-sharers
• Co-design with community leaders and prospective trainees
• Train the trainees in evidence-based mental health skills such as screening, psychoeducation, and referrals
• Collect feedback and data on the training's impact and continue to iterate with new and enhanced features

Outputs:

• Trained mental health task-sharers (community leaders) who have evidence-based skills they can use to provide greater psychological support to community members they interact with through their leadership roles
• An on-demand, AR mental health care training library at accessible community sites

Outcomes:

• Improved confidence for trainees who complete our program
• Improved socioeconomic outcomes for trainees who complete our program
• Increased access to affordable mental health care for underserved communities we serve
• Improved mental health, social, and economic outcomes for patients who interact with our trained task-sharers

Impact:

• Improved overall health and well-being of communities served by mental health task-sharers trained by Strongr
• Increased economic stability for individuals whose needs are better met through accessible and effective mental health services
• Improved mental health outcomes and reduced healthcare costs at a community level

Key research references:

• Behavioral health services significantly lower health care cost
• Effectiveness of mental health task-shifting training in NYC community-based organizations: large-scale training of mental health knowledge and skills led to CBOs successfully incorporating mental health practices into their work, new opportunities for New Yorkers to access mental health services, and improved social and economic outcomes 
• Benefits and effectiveness of using immersive technology (AR/VR) for soft skills training

Our core technology is a combination of augmented reality, real-time 3D avatar animation, and artificial intelligence. 

Augmented Reality

The medium of augmented reality allows us to place our virtual patient interaction in the real-world settings where trainees would be interacting with real patients once they have completed their training. While we have so far been prototyping with Magic Leap 2 augmented reality glasses, similar immersive devices such as mobile AR devices and VR headsets may be similarly effective in lending realism and embodiment to the simulated interaction with a virtual patient.

3D Avatar Animation

Our software is built with the Unity game engine, a real-time 3D platform. The Salsa LipSync Plugin for Unity allows us to dynamically animate our virtual patient character's mouth movements, eye contact, blinking, head movements, and even subtle emotional expressions, enhancing the realism and emotional connection for the trainees. We are currently working with a sequence of slightly modified Mixamo animations for animating our character's larger body movements throughout the training session.

Artificial Intelligence

We are starting to explore how conversational AI models (large language models and third-party plugins built on top of them such as Inworld.AI) can further lend realism to our virtual patient interactions while expanding the capabilities of our training to offer more dynamic (with real-time conversational feedback), non-linear, generative scenarios (and increasing the speed of our development process). Well-tuned AI models would help reduce the possibility that our scenarios could be "over-learned" (like more structured, linear simulations can be), rewarding trainees who participate in additional sessions with fresh practice contexts and continued development of skills and knowledge. Advances in conversational AI are the key to the potential of innovative soft skills simulation training.

Our current approach:

1. Organizing a diverse leadership team: Strongr's leadership team comes from a wide range of racial (Latinx, Asian, White, Black) and cultural backgrounds (citizens, immigrants), all aligned around a shared interest in co-creating more equitable and inclusive mental health services.
2. Designing with community members: Strongr's R&D process takes a participatory design approach to ensure maximum usability, relevance, and cultural sensitivity of training content.

Our goals:

1. Expanding the language options of our training program beyond English to other languages commonly spoken in the communities we serve, such as Spanish and Mandarin.
2. As our internal funding and resources increase, we will seek out more diverse teammates and more teammates/collaborators directly from the communities we serve.
3. We would like to explore ways of offering training on innovative technology / mental health innovation R&D, perhaps through an internship program, in addition to applied mental health skills training to expand the overall equitability of our impact.

In the initial stages, we are mainly focused on grant funding while we research and develop a viable training product.

We are still working out a longer term business model, but here are our initial thoughts: As we are focused on B2B services, we would plan to charge community-based organizations (many of which are funded by city programs such as NYC Care) a one-time fee (such as $100) or subscription fee ($15 / month) for each trainee who goes through the program (and brings added value to their organization with increased mental health task-sharing capacity).

While we are still in the R&D stage of our training program, we will be seeking funding from NIH grants, such as this one for Innovative Mental Health Services Research, or other grant programs, such as Inworld.AI's $50,000 Character Grants.

One plan is to develop an integrated social enterprise within our nonprofit, wherein our business activities overlap with our training programs, and the business acts as a funding mechanism to expand/enhance our training offerings.

B2B

• Initially starting with charging social service CBOs
• Down the line, we could expand our market to primary care centers and independent providers in the US behavioral health market (estimated at $132.4 billion by 2027) and potentially to other countries as well

So far we have received a $20,000 contract to build the initial prototype and conduct a pilot study, funded through CUNY's Center for Innovation in Mental Health's grants originally administered by the NIH and Robert Wood Johnson Foundation.

(NIH funding was provided by the NIH’s Transformative Research to Address Health Disparities and Advance Health Equity Initiative [Grant Number: 1U01OD033245-01])