LucidPath

People worldwide are living longer, driving up the need for scalable, high-quality memory-loss care solutions. By 2050, the global population of people aged 60 years and older will double from roughly one to two billion. Of particularly critical concern is the availability and cost of long-term care solutions for individuals with memory loss. Specifically, there are almost 7 million Americans with Alzheimer's disease (1 out of 9 people who are age 65 and above). The impacts of Alzheimer’s are extensive; it is responsible for more deaths than breast cancer and prostate cancer combined, and it takes a massive toll on quality of life for both people with the disease and their caregivers and loved ones. 

Despite considerable investment in developing effective treatments for Alzheimer's, there is still no cure in clear sight. Without a cure, the costs of treatment will likely rise dramatically in the coming years. By 2050, costs associated with memory loss are projected to rise three-fold to nearly $1 trillion in the US. This number doesn’t even consider unpaid caregivers’ contributions, which this year in the US accounted for 18 billion hours of care. These caregivers face a profound burden, with twice as many reporting significant emotional, financial, and physical challenges compared to those caring for individuals without dementia . The problem compounds when considered through the lens of health and economic equity. For instance, considering that older black and hispanic individuals are twice and one and a half times, respectively, more likely to develop the disease compared to older white individuals, families in these demographics are disproportionately economically affected by a lack of affordable (and quality) memory loss care options. Taken together, these statistics point to a need for innovation in this solution space. 

In both living facilities and at home, caregivers (often untrained family members) face complex communication challenges. Interaction failures occur within a caregiving team as well as between caregivers and the patient and their loved ones. 

To address these issues, LucidPath helps caregivers communicate about a care recipient’s needs and progress. For instance, families have limited visibility into the quality of care provided by facilities, resulting in low trust. One key reason is a lack of automated notetaking during therapy sessions. Similarly, families providing at-home care benefit from professional guidance during therapy sessions, but that is frequently unavailable. Our team gained an understanding of this problem first hand: through professional memory loss training, years of professionally providing therapy to patients in long-term facilities, extensive interviews with caregivers and families, and engagement with academic literature. Our team specializes in music and movement based memory loss care. This clinically validated method involves engaging with care recipients in listening, singing, and movement-based activities; leveraging these stimuli to bring about moments of lucidity and memory recollection. We adopted this method because, in either care context, it can help ameliorate burnout, improve clinical outcomes, and address quality of life needs. However, the effectiveness of music and movement based care is highly dependent on the aforementioned communication channels. 

LucidPath runs on an iPad and passively captures a patient’s experience during memory loss therapy through video, then uses privacy-sensitive algorithms and AI to identify clinically and personally significant moments, trends in data, and opportunities for personalized interventions. LucidPath extends a web-based platform we developed to aid in media curation for music and movement based memory loss therapy. This tool has been extensively tested by our collaborators at Memcara, specifically Memcara’s founder Christina Tadin, who is a music therapist and memory care expert. In this way, LucidPath augments the abilities of a caregiver, with the system becoming more tuned and tailored to a patient’s needs as additional data is collected over time. Currently, we have a computer vision pipeline that can generate per-second multi-label annotations according to 40 categories of interest, such as “recounting a memory,” “attention level: low/medium/high”, “emotion: …” or “medical symptom: agitation/repetitive movement/…”. These annotations also include patient and contextual metadata as well as on-screen events, such as what therapeutic media is playing. From these annotations, we can identify salient moments in sessions, such as  “moments of lucidity,” “moments when a memory is being recounted,” and other moments of clinical or personal significance. We then automatically extract and compile these video clips to form a “highlight reel” of a care session. 

Given the confidential nature of the data, we are not providing a link to it in our application provide. For more information about the highlights reel, please email dylan.e.moore.th@dartmouth.edu.

Our next steps are to: 1) leverage GPT-4 to summarize a patient’s care journey and memories, including links to video highlights and 2) close the loop of automated data capture and adaptive intervention by using insights from patient videos to augment a broad set of caregiver functions, such as content curation, clinical note taking, and caregiver coaching.

LucidPath is a tool that both professional and informal memory loss caregivers will be able to use to automate aspects of memory loss therapy. This is not about replacing, but rather augmenting, the human caregiver in areas where they underperform, such as note taking, intervention assistance, recording salient moments, or professional-level guidance. We believe LucidPath will enable millions of caregivers to provide better care, as well as capture beautiful moments that often emerge during therapy—e.g. So that a family can have a recording of their grandmother recalling a trip abroad during her college, captured during a therapy session at a long-term care facility. From our first hand experience, we know that caregivers are in urgent need of this type of assistance and that there are no tools available that address this particular need. Considering even our current multilabel cross validation accuracy level as a metric (>90%, to date) LucidPath has the potential to be a massively scalable, low cost, trustworthy, always-on caregiving assistive technology.

Our solution is designed for music and memory loss care in particular. However, our approach is broadly extensible. For instance, we could expand to cover other types of memory loss therapy or specifically engage with common comorbidities of memory loss, such as depression.

Our team is deeply knowledgeable about the problem we are solving and firmly rooted in the community of users that we intend to serve. All of the members of our team have a personal connection to memory loss. Most of us have or have had a loved one with the condition. We all share a commitment to leveraging AI to provide opportunities for better care. LucidPath consists of two groups that have come together, each side bringing different expertise. 

With an academic perspective, there are members of the Empower Lab at Dartmouth College. These individuals have significant expertise and professional connections related to AI, clinical study design, human-centered design, and software development. With a memory-loss care professional perspective, there are the members of Memcara, a four person therapist group based in upstate New York. This group brings a wealth of experience with music and movement based memory loss care, as well as a developed professional network of connections to memory loss facilities, other leading experts in memory loss care, and leaders of memory loss support groups. Our Dartmouth and Memcara teams united to jointly develop LucidPath after discovering our shared goal at the Dartmouth Digital Health Summit last year. Since then, we developed a prototype of our AI pipeline and have been running a user study (with a dozen participants) since early summer.

Music and movement based memory loss therapy is widely regarded as effective, however its adoption as a structured form of therapy is still limited. Memcara’s existing program takes the first step towards scalability, since the team has developed a custom web-based platform for media delivery. However, this scope is still limited: the Memcara team is small (four people) and the tool was not developed for external use. With LucidPath, we seek to embed the professional experience of the Memcara therapist team into every facet of our system, from “smart” sensing capabilities to an adaptive interface. Currently, we are building features that solve for the immediate needs of our team in their own work (automated note taking, as discussed earlier), while simultaneously accumulating a training set of videos that implicitly catalog Memcara’s method. No such database exists that we know of. As our system matures, we will be able to leverage this data, as well as other data sources we are investigating, to create a self-guided, publicly available therapy platform. We truly believe our approach can revolutionize memory loss care.

Our approach is aligned with target 3.8, specifically: “access to quality essential health-care services.”

Currently, our dataset consists of 250+ video sessions for a dozen patients. Each session involves one Memcara therapist and one care recepient. We collected this data in partnership with two long-term care facilities in upstate New York and are in the process of building new partner relationships. Data collection began at the start of summer remains ongoing, with new sessions automatically captured every day. In addition to video data, we have also been collecting other metrics that can be used as metadata or benchmarks, such as passive sensing data from a smartwatch, clinically conducted MOCA assessments of cognitive performance, and a considerable quantity of caregiver notes. We have not yet leveraged this data but plan to do so eventually. 

Each recording in our database has also been “annotated” by a Memcara therapist, using a custom tool developed by our team. As noted earlier, these annotations also include care recipient and contextual metadata (e.g. care recipient MOCA score, time of day, care recipient self reported happiness) as well as on-screen events, such as what media is playing. These annotations are per-second in fidelity and act as training labels for supervised learning. Once a video is captured, it is processed by our multistep pipeline. The first step in this is extracting “intermediate features” in parallel from the video based on facial expression, movement, speech, and text analysis—using word embeddings and large-language models (GPT-4). These intermediate features are then synthesized by a Long Short-Term Memory (LSTM) recurrent neural network into per-second multilabel annotations. We identify moments of salience based on annotations. We are also exploring other model designs, specifically using transfer learning to directly work with video input (instead of doing intermediate feature extraction). 

As we continue to capture sessions, we plan to capture feedback through other features, such as video highlight replays and shares. In the long term, we are specifically interested in approaches that will allow our model to continuously improve without the need for explicit labeling by trained professionals.

We take ethical and responsible use of AI and user data very seriously. To underscore the seriousness and complexity of the ethical considerations here, in many cases we are collecting sensitive data from individuals who are not able to themselves provide consent. Our methods have undergone rigorous independent reviews by the Dartmouth IRB and ethics boards at our two partner long-term care facilities. We strictly abide by the ethics regulations and standards of all of these institutions as well as our own understandings, as researchers and practitioners, of best practices. This includes, but is not limited to, receiving informed consent from not only participants and partner institutions but also families of loved ones. 

Notable risks associated with our solution include: providing poor therapy guidance, automated sharing or a leak of sensitive data (e.g. video content or analysis of video content), capturing bias in models from training data, and automated tooling disrupting a caregiver’s delivery of care. Beyond basic best practices, our approach to mitigating all of these risks involves  phased rollouts as well as maintaining a human in the loop when applicable. For instance, rather than supporting fully automated distribution of video highlights to family members, we make this data available to caregivers and allow them to filter and share what content they see fit. This provides a layer of human judgment.

As a team with years of first hand experience seeing the value of music and movement based therapy, we believe that the recent trend of interest in this approach will continue to accelerate. We are very excited about this and hope to ride this wave.

One year from now, we plan to have a working initial prototype of LucidPath. Specifically, this will include the two capabilities discussed in our earlier overview of the solution, namely, 1) the capacity to summarize a patient’s care journey and memories, and 2) the capacity to “close the loop” through data driven interventions. Initially, we anticipate focusing on content curation for this latter point.

Five years from now we intend for LucidPath to be the go-to solution for music and movement based memory loss therapy, in both professional and non professional contexts. By then, we see our product being used daily by millions of caregivers. Given the strengths and connections of our team, our goals are achievable with support from a program such as this.

Until now, LucidPath has been a collaboration between a lab at Dartmouth College (The Empower Lab) and Memcara, a four person music and memory loss therapy company the provides care a long term facilities in upstate New York.

For the next year, Dylan, Christina, Songyun, and Matthew would make up the full-time staff of the team, with other members of the Empower Lab and Memcara providing part time support. Beyond this, there are a number of folks who have expressed interest in joining our team and, if needed, we would be able to recruit additional graduate students and practicing therapists to join our team to fill in gaps in our existing expertise.

(see “what is your operational model and plan?” for more information on our team members' roles)

The Dartmouth and Memcara groups within our team came together one year ago. After finalizing an internal statement of work and getting IRB approval, we developed a working prototype of our intervention. We began testing with real users at the start of this past summer. Prior to this, the Memcara team joined together to form a music and movement based memory-loss therapy company in 2021.

Our team is deeply committed to diversity, equity, and inclusion, both collectively and independently. LucidPath brings together two groups, the Dartmouth Empower Lab and Memcara, both of which are small, diverse, female founded organizations focused on providing accessible, equitable, health solutions. We are passionate about a human-centered design approach to technology and have extensive experience in implementing solutions geared towards health equity and mitigating biases in healthcare artificial intelligence. We recognize that tackling challenges necessitates the involvement of individuals of various genders, backgrounds, and ethnicities. This diversity ensures a broad spectrum of perspectives, encouraging innovative thinking and the constructive questioning of traditional approaches. At Memcara, it is noteworthy that all founders are from different countries and diverse cultural backgrounds. All founders have global experience and have made incorporating diversity, equity and inclusivity in previous ventures a priority. The same global and inclusive mindset is being applied in Memcara today.

We have segmented our target customer base into three core groups: nursing homes, professional care agencies, and at-home caregivers. We will launch our product to a small number of customers in the first segment in January 2024.

Our team is currently set up as an industry-academia collaboration between Memcara and Dartmouth. This has worked incredibly well as the strengths and weaknesses of both sides are complementary. For instance, though small, Memcara is deeply embedded in the memory-loss care industry, is well equipped to run and scale a user study, and has bottom up and top down community support. On the other hand, the Dartmouth team is capable of implementing cutting edge AI solutions and scalable software systems. Dylan, who studies both AI for health and business, has been driving this collaboration for the past year and will continue to do so. This loose operational model has worked very well for us so far, given our problem space, our internal  division of expertise, and the strong alignment of our individual incentives. As we move towards bringing our solution to market we anticipate working with mentors at Solve to thoughtfully develop a scalable operational model. 

Some more specifics about our team: Dylan, a third year PhD and MEM student at Dartmouth and a recipient of the Dartmouth Innovation Fellowship, is leading our team. He has a professional and academic background in AI, design, and software engineering (prev. YouTube, Lark Health, Stanford) and will be focused on the technical and business aspects of the project. With him comes the support of the Empower Lab at Dartmouth, specifically Songyun Tao, a third year PhD student researching AI and music and movement based therapeutic interventions, his advisor Professor Elizabeth Murnane, and his mentor Professor Eric Fossum, who leads the innovation program. Matthew Seegmiller, a third year PhD student in Professor Sarah Preum’s lab, who studies natural language processing in computational health brings additional AI expertise to our team. On the clinical side, Christina, Markus, Brian, and Dio of the Memcara team bring extensive expertise and years of professional practice in music and movement based memory loss therapy.

We plan to initially monetize LucidPath as a support tool for professional caregivers working in long-term memory care facilities. In the longer term, we are interested in the direct to consumer market, i.e. focusing on informal caregivers supporting family a member with memory loss. 

We are currently bootstrapping this project through an academic fellowship, academic small grants, and (on Memcara’s side) angel funding (friends and family) through SAFE notes. For LucidPath, our goal so far has been to demonstrate a proof-of-concept version of our solution. We have been able to keep our operating costs low since our team has in-house software and AI expertise. Our current funding sources can be relied upon to cover living expenses for our team through the next year. Additional funding will be essential though, for instance for growing our team, continuing product development, and getting legal counsel.

We are asking for 50k, the minimum amount. We are enthusiastic about Solve’s ability to accelerate the development and growth of LucidPath and funding is definitely an essential part of that. In the immediate term (next year), this funding will help us cover server and data processing costs, which are growing at an unsustainable rate given our current funding resources. In the longer term, we are more interested in Solve because we hope to tap into a network of investors who see the value of our solution and can help us polish our offering and capture the market.

We are still at an early stage; we are confident that our solution is effective—and have demonstrated this, but do not yet have a finished product or a detailed growth strategy. We are confident that we can reach these goals but would greatly benefit from the mentorship that Solve can provide in these areas where we are weak. 

Beyond our known challenges, we expect that we have not anticipated all of the problems (e.g. legal and marketing) that we will face in taking our product to market. This is very exciting to our team. While these needs are somewhat abstract, the Cure XChange Challenge: Health AI For Good seems incredibly well aligned with them. We envision that being part of this community of innovators and participating in this challenge would offer us both support and guidance as we enter the next stage in our journey towards making professional quality music and movement based memory loss care widely accessible.