validly.ai local medical AI validation

AI is permeating all areas of our life, and that includes healthcare. And this is a great win overall, as doctors and nurses are overworked and do not have the time to constantly scan patient's charts for very signs of distress or issue. AI algorithms, on the other hand, can constantly monitor new test results, view them in the context of the patient's entire history, and catch warning signs as early as possible. AI algorithms can also serve as a second reviewer (second to an actual human radiologist) on X-Ray, CT, and MRI images, acting as a check which may find patterns that are too feint and early for a human to reliably detect. While this future of computer-supplemented doctors is a win-win for both clinical staff and patients, if the AI is not at least as reliable as a human (and ideally much more reliable), the system actually can degrade the standard of care below where it is currently. 

 The issue here is in how AI algorithms are trained. The gold standard in AI is to have as much data as possible, representing as many situations as possible. This goal is never met, and due to things like data cost, licensing, and acquisition time compromises are always made for what is "good enough". However, when the training data that an algorithm developer uses has demographic mismatches compared to the population a given hospitals serves (technically called class imbalance), errors come up because the algorithm's view of which cases are most common is no longer true in the micro-demographic of patients the hospital serves. This issue can be seen recently in the news, where around mid-year 2021 it was discovered that an AI algorithm developed by Epic health records (and used by customers of their electronic health record) to predict sepsis drastically underreported risk in African American populations. Whereas the algorithm was probably trained well and passed its tests in hospital systems whose demographics were not so diverse, when it was deployed to the real world hospitals which serve majority black populations used it. They then discovered that the algorithm's view of the world did not necessarily match with the reality of the patients the hospital served, and it lowered the standard of care.

Our solution is, quite simply, local validation of AI algorithms. There are two stages to this local validation: the initial testing and scoring to ensure the algorithm can meet its accuracy targets at a given hospital, and then continuous monitoring to ensure that drifts in patent demographics, social determinants of health, etc do not introduce new population characteristics which cause the algorithm to underperform.

In practice, this looks like a two stage process. First, when the hospital wants to buy an AI algorithm we (validly) are brought on as consultants and can back test the algorithm on the hospitals own past data. This process produces a score which is guaranteed relevant to the hospital and allows them to either integrate an algorithm or pass on it. If the hospital does choose to purchase or otherwise integrate an AI algorithm into their workflow, the continuous monitoring part allows quarterly (or more or less common depending on the hospital's needs) checks to make sure that the accuracy does not drift.

This solution is mainly comprised of initial consulting services, a proprietary scoring and result analysis algorithm, and automated monitoring and scoring which is used both in the initial assessment and as part of the continuous monitoring subscription. 

Though it may sound grandiose, our solution will (eventually) serve everyone on Earth. In the first stages, we are targeting health systems in the US, Canada, and Europe (specifically Germany and France to start). As we said in our problem statement, the fact is that AI is going to make its way into every aspect of life over then next few decades. In medicine, this means more efficient use of doctor's time in developed countries, as well as the expansion of expertise and diagnostic services to developing countries where the doctor shortage is much more drastic and where experts capable of certain diagnoses may be in especially short supply. 

More practically and directly, our solution benefits those who are part of minority or otherwise marginalized groups, as they are the ones most likely to be left our or underrepresented in the initial training of these AI models. These are the people who suffer when AI models fail, and their standard of care falls. This solution ensures that hospitals and clinics do not inadvertently implement AI software solutions which do not serve these populations. 

Our team is well positioned because of our past experiences with the health system and how it handles novel cases, as well as our personal experiences with AI. While AI is not pervasive enough for us to have experienced it directly in a medical setting, two of us (us being the three cofounders) have developed AI before and we both (separately, way before we had this idea) came to the conclusion that AI technology right now is not well regulated or treated with the caution and respect it deserves. Way too often we have seen or used AI products which promise amazing results, but when you get it you realize those results were only in very specific use cases and you cannot do anything to fix it as the AI is treated as a black box to just be used. This is ok for things like phone autocorrect (who hasn't had an embarrassing AI autocorrect correction?), but in healthcare that level of unaudited sales claim is unacceptable and can be dangerous to people's health.

Additionally, all three of us are in some way from minority communities or groups that have a history of not being well represented in healthcare (at least in the US). We have two POC male cofounders and one Eastern European female cofounder. Two of us have chronic/genetic health issues which place constraints on any additional treatment (even for common things). One cofounder is from (and currently lives in) Colombia, where may algorithms will eventually be used by which is not a large source of the training data which makes those algorithms.

Legally, the medical field has high barriers to entry. Especially given our goal of eventual international launch. In this regard we want experts who can advise us on best practices when handling patient data (our solution do not involve data leaving the clinic, but even though we will never take the data we want to handle it in a way which is trustworthy and legally compliant).  There is also a large barrier to integrating with healthcare IT systems, and we need technical specialists who not only know the common electronic health record systems but also how hospitals interact with them so that we can build our product in a way that integrates into existing workflows.

Being a team of technical people (and a designer/sales person), we also always need financial advice, from pricing our product to positioning ourselves in the market. This also ideally would include experts in partnering with hospitals and applying for NIH or PCORI government grants.

For almost the entirety of its existence, the core issue of AI has been going from the specific to the general. Taking hundreds, thousands, or millions of individual points and drawing conclusions and correlations from them that can be applied to unlimited amounts of new data. Our solution inverts this paradigm, and addresses how a general solution can then go back and maintain its accuracy and relevance when put into specific environments that differ from its training in very systemic ways (specifically, demographic variations). 

Addressing this question is something that is at the cutting edge of health tech and policy, and it is one to which we believe we have a marketable technical solution. We expect our product to not only allow hospitals to ensure quality and efficient care for all equally, but to also enable those who make health AI software to use this same system (in partnership with the hospitals we serve) to better stress test their products before they ever make it to market. The end positive impact is better quality care of patients, with added benefit of fewer cases of  clinician burnout and lower costs for governments and insurers. 

Healthcare is a conservative industry in terms of how quickly it changes, and we believe our product de-risks AI integration to the point that adoption can spread quickly and bring the promised benefits to all, with none of the current downsides.

Our impact goal for the next year is contributing to the clinical patient-centered outcomes research on how AI algorithms can further health outcome disparities. This involves using fist prototype scoring system with academic and clinical partners to run real-life case studies. We believe that in addition to sharing our solution, it is our job (both in terms of making a viable company and in terms of being ethical humans who share their knowledge for the greater good) to establish this problem as one of the biggest ones healthcare will face in its digital transformation.

Over 5 years our impact is clinic-centered. We intend four our non-sales success to be a measured by the inequality we help prevent. This means that we hope to play an active role (through the sale and use of our product) in clinics tailoring their software-assistance suites to their population. 

For our first goal, we measure our progress in terms of partnerships made and case studies completed, with the final measurement being the publication of a white paper or academic journal article (along with our partners) on the improvements that can be made.

For our long term goal, our measurement aligns with the UN SDG metrics broadly found within goal 3 (Ensure healthy lives and promote well-being for all at all ages). Such examples of specific targets we believe we can address are those which generally can be addressed with AI monitoring or diagnostic software, but for which demographic issues may hinder the AI accuracy, or those which require access to a specialist (where AI could form an intermediate guide). These include (among others) 3.2.1 (under-five mortality rate), 3.2.2 (neonatal mortality rate).We also address  3.8.2 (Proportion of population with large household expenditures on health as a share of total household expenditure or income) as our solution does aim to lower the cost of healthcare through efficiency gains. By alleviating the load on healthcare workers and reducing burnout, we also partially address 3.c.1 (Health worker density and distribution). Our continuous monitoring solution can provide fundamental data to address metric 3.d.1 (International Health Regulations (IHR) capacity and health emergency preparedness)

Our technology is powered by AI as well as the technology and techniques surrounding AI scoring and evaluation (a field which is currently on the rise as the risks of bad AI predictions become more known). 

The fact is that as an early project, we do not have a formal policy. However, our approach to DEI is to combine the pragmatic approach with recognition of the history that has brought us where we are. We recognize that our opportunity to make this company is rooted in a history of exclusion of minorities of all types (racial, disabled, etc) from core parts of society such as health research and policy making. We also recognize that in our efforts to address the inequalities which this history has left us with today, we need to seek opinions and experiences from everyone to build a solution that works for everyone. Pragmatically that means that the responsibility of inclusion in this endeavor falls onto us as founders, and as we seek advice and formulate our strategy we need to actively seek out those whose lives we wish to improve to understand how a solution can work for them. 

Our business model has both an initial consulting component, and then the sale of an automated continuous monitoring software solution subscription. 

In approaching hospitals who may want to use or already use AI in their workflow, we will provide initial consulting for a fee to evaluate the efficacy of the algorithm on their patient population. 

We then offer the hospitals a continuous monitoring solution via subscription so that hospitals can monitor the algorithms they have and know if there is a drift in accuracy (such a drift can be easily imagined during large events like COVID, where a tuberculosis diagnosing algorithm might lose accuracy due to the large demographic shift in people who have or have had another serious respiratory disease such as COVID).

Our revenue model (as mentioned above) has both a consultation fee sale component as well as service and software license contracts. We are also pursuing government research grants, and plan to be looking for venture capital in the medium term (~2 to 3 years). Within 5 years we hope to have the established customer base (and growth into new markets and customers) to be self-sustaining without any external capital. 

We are a relatively new solution, so we have not become financially self-sustainable. However we are actively engaging with partners to apply for government grants (notably from the US NIH and PCORI) to further our first impact goal.