AI needs to starts with quality data
AI regulation is taking shape with the first two assurance labs going live in the U.S. by the end of 2024.
This is a monthly newsletter of Faces of Digital Health - a podcast that explores the diversity of healthcare systems and healthcare innovation worldwide. Interviews with policymakers, entrepreneurs, and clinicians provide the listeners with insights into market specifics, go-to-market strategies, barriers to success, characteristics of different healthcare systems, challenges of healthcare systems, and access to healthcare. Find out more on the website, tune in on Spotify or iTunes.
If 2023 was full of excitement related to generative AI, 2024 is sliding towards maturity, partnerships and practical applications of AI. Rock Health tracked 3.3 billion USD in investments to healthcare AI from January to October 2024 in the U.S., representing 36% of all funding in digital health.
Incorporating AI into every solution has become a necessity to capture investor interest. Which means that now, solution will require innovative elements to truly stand out, and have AI just to appear up to date.
Expectations for AI's benefits are high, but we're still in the early stages of realizing its full potential. The 2024 AI Adoption in Healthcare Report, prepared by HIMSS and Medscape, reveals that while clinicians are increasingly leveraging AI to enhance efficiency, it has yet to reduce their workload or lower costs.
In old news: clinicians are still concerned about privacy and ethical dilemmas in using AI.
For patients and providers to feel safe about AI, it needs to be regulated and wear a stamp of approval. The question is, who can do the approval with credibility?
Several organisations popped up for AI regulation in the last few years:
CHAI - Coalition for health AI - is an industry membership organisation which is setting up assurance labs across the USA to validate AI solutions,
TRAIN - Trustworthy & Responsible AI Network is an intiative giving guidance on the practical steps to responsibly operationalize AI,
HealthAI: The Global Agency for Responsible AI in Health - is a Geneva-based non-profit organisation establishing global standardization to advance the development and adoption of Responsible AI solutions in health,
and many more.
In Europe, the EU AI Act defines 4 levels of risk for AI systems, and different requirements based on the level.
Assurance Labs for Healthcare AI in the U.S.
The further along with efforts to validate AI solution is CHAI - The Coalition for Health AI, which has over 1,300 member organizations, including tech giants like Microsoft, Google, and Amazon. CHAI is establishing a network of quality assurance labs to evaluate AI models in healthcare. According to the CEO Brian Andersson, two assurance labs will be open by the end of 2024.
These labs aim to provide independent, transparent assessments of AI models' performance across different populations. To get the details, I interview Andersson and asked who will pay, how will model cards work and more.
Model Cards and Evaluation
With the increasing complexity of new tech, we need approaches to simplify quality recognition. A very common idea is to have food nutrition type labels for technology. The European project Label2Enable promotes a quality and reliability label for health apps. This is a European project, and not a standard of practice in European countries just yet. ORCHA (Organisation for the Review of Care and Health Apps) is working globally to provide digital health apps assessments and offer clinicians clarity on what to recommend, and what not. It’s therefore no surprise that CHAI also wants to introduce labels for AI models.
AI model card presented by Brian Andersson at Digital Health and AI Innovation Summit in Boston 2024.
Model cards are filled out by vendors and need to describe their training data, methodology, indications, and limitations. Assurance labs provide independent evaluation reports.
Assurance labs aim to balance innovation with safety in AI development. They can help identify model performance issues across different populations and accelerate improvements.
Open questions with AI assurance labs
The introduction of AI assurance labs is very welcome and intended to build trust in AI among healthcare providers and patients. However, several question remain open: will healthcare providers trust the assessments, given that CHAI is industry led? Looking at the other side of the coin - the industry knows healthtech development best and is best positioned to assess technology, especially in the challenging workforce shortages in healthcare IT.
The second question is: how expensive will assessments be and will this cost be an additional barrier to success for early-stage innovators? According to Anderson, CHAI is creating a competitive marketplace of quality assurance labs to keep costs reasonable. “The cost of evaluations is expected to be in the range of thousands of dollars, not millions,” he said.
Future expectations
Evaluation reports will be published in a public registry for transparency.
While vendors won't need to disclose the metadata their models are built on, the added transparency regarding AI model quality is a significant step forward. It will uncover new challenges, providing developers with valuable insights and a clearer direction for future advancements.
CHAI is also exploring partnerships with health systems and NGOs to establish quality assurance labs in the EU. The initiative aims to be scalable and adaptable to different geographic regions and populations.
In other episodes …
This is what was covered on Faces of digital health in the last two months.
How Can Hospitals Advance Their Digital Transformation With HIMSS Maturity Models? (John Rayner, Senior Director Analytics - EMEA at HIMSS)
The future of data in in collaboration among standards bodies (Rachel Dunscombe)
How does remote patient monitoring look like in South Africa and Nigeria?
Zayna Khayat: Language, Mindset and the Future of Care
Don’t miss!
Holiday Edition: Future Trends in Digital Health & AI (Daniel Kraft, Lucien Engelen, Zayna Khayat)
In this special holiday edition of Faces of Digital Health, healthcare futurist and strategist Zayna Khayat, futurist and founder of NextMed Health Daniel Kraft and digital health expert, CEO of Transform.health Lucien Engelen discuss key trends in digital health and AI. The panel also touches on the role of new players in the health space like supermarkets and tech giants, the importance of prevention in healthcare, and the promising technologies poised to revolutionize the industry.