Transcription

• Dr. M. E. de Baca

  Welcome to CIPI Connections, the podcast of the College of American Pathologists' Council on Informatics and Pathology Innovation. Here, we connect you with the leaders and committees shaping the future of pathology. I'm Dr. M. E. DeBaca, Chair of the College of American Pathologists' Council on Informatics and Pathology Innovation, also known as CIPI. In today's episode of SIPI Connections, we will be talking about how pathology meets artificial intelligence. We're going to dive into how the pathology community validates and implements new technologies and how AI applies to those frameworks. Today, you'll be part of a conversation with three of my good friends and members of the CAP Artificial Intelligence Committee. Dr. Matthew Hanna is a surgical pathologist and the chair of the CAP AI Committee. Dr. Nick Spies is a clinical pathologist and a pioneer for introducing AI implementation specialist roles. And Dr. Larisa Furtado. a molecular pathologist who is currently leading the development of the CAP AI Implementation Guide. Together, they'll explore what works, what doesn't work, and what we've learned from real-world AI deployments in pathology. Take it away.

• Dr. Matthew Hanna

  Thank you, Doc. We really appreciate you and all your support and really excited about the conversation today. So let's start with the good news. In anatomic pathology, we've long relied on structured workflows, We all know the traditional specimen handling, slide preparation, diagnostic reporting, all governed by CLIA and CAP standards. So these are the frameworks that really emphasize accuracy, reproducibility, and above all, patient safety. So that mindset really translates beautifully to AI, which we can think of as just another test. So all of what we want to portray in this concept is Having AI be just another data point that is like any other test you would be reviewing and reporting. For example, validating a new immunochemical stain. It involves evaluating the performance criteria, the control tissues, documenting all of that. And that same rigor applies to AI model evaluation, doing the analytical validation, the precision testing, the clinical validation, and ongoing monitoring. Dr. Spies, what area of laboratory medicine have shown success in translating well to supporting AI workflows.

• Dr. Nick Spies

  Yeah, absolutely. So in clinical pathology, we certainly have that advantage in that a lot of what we do is focusing on numerical input data, which makes it a lot easier to start building out some of these proof of concept applications for areas like quality control, error detection, workflow optimization, and many others. We certainly have plenty of examples of this kind of high dimensionality data. that would be really well suited for your classical machine learning techniques. You know, things like panels, your basic metabolic panels, your complete blood counts and so on, or data sets like flow cytometry, where you're running a lot of immune markers, or things like mass spectrometry, where you have a full spectra of input data that you have to kind of deconvolute and interpret.

• Dr. Matthew Hanna

  Wonderful. And what about molecular pathology, Dr. Furtado?

• Dr. Larissa Furtado

  Molecular pathology we already use bioinformatics pipelines for NGS analysis and validating an AI model is to a certain extent similar because there is a need to train multiple models, fine-tune with separate data sets, lock configurations and validate performance and all of these steps have corresponding procedures when developing bioinformatics pipelines. So generally the CIP checklist requirements for molecular tests apply well to AI-based methods used in molecular oncology testing.

• Dr. Matthew Hanna

  So generally, the CAP checklist requirements for molecular tests really apply well to AI-based methods used in molecular oncology testing. Actually, Dr. Furtado, there was a recent article I read that looked at mapping some of those CAP checklist requirements and translating them to molecular workflows. Would you be able to comment on that?

• Dr. Larissa Furtado

  Yeah, we did a study in which we really looked at how Well, the existing CP checklist would translate into applications for AI-based molecular tests in molecular oncology. And what we noticed was that there are already several existing checklists that could be used for implementation of a molecular test that uses AI. Because, for example, we do need to validate a test before we implement it clinically. And every time we make changes, we need to revalidate the test. There are some... other pre-analytic considerations such as determining the minimum input for tumor testing that we do normally for sequencing. We also know that most of the AI-based tests that use molecular that are used for molecular, you need that as well. And there are existing requirements to determine cutoffs for qualitative tests to differentiate positive from negative. And for classifiers or those AI-based methods that are intended for classification, it's important also to define those thresholds so you can increase the reliability of the call, whether that is a positive or negative call, you know, it kind of gives you a baseline for that. So there is a lot that translates, which is not surprising because if you think about the AI models that we use, we're not implementing them as autonomous systems in the laboratory, they are always part of a test. So they're either built in as part of a pipeline that analyzes the data, such as, you know, the methylation classifiers using DNA array data. Or they can actually be adjunct to the existing molecular pipeline. You can actually build an extra model such as micro-satellite instability analysis to be able to provide that additional data. So there is a lot that translates already.

• Dr. Matthew Hanna

  That's great to hear. I mean, I wish everything would translate over as easy or we would have hopefully solved the issue of AI validation already. Looking at AI validation results, some of them might actually look similar to an assay trending out of range under Levy-Jennings plots. Unfortunately, with AI, it may not be as easy or as detectable. And we've heard a lot about generalizability concerns. And just because a model works well at one site because it was trained on their local data, it doesn't mean that a different lab with a different patient demographics or different specimen profiles will see the same level of performance. Do our existing validation frameworks in pathology, do those conventional frameworks still work for molecular testing?

• Dr. Larissa Furtado

  In general, yes, but there are some gaps. For instance, the current frameworks don't address all the important aspects of data preparation, data management, or the use of independent datasets for training and validating AI models. There aren't clear standards for data quality either. But we know that AI models are only as good as the data they are trained on. And if the training data isn't diverse or representative of the real-world entities and conditions that the model will encounter, the test might not perform well, especially for those underrepresented sample types. So it's really important for the training data set to be aligned with the intended use of the test. We also face challenges with explainability because unlike bioinformatics pipelines, which are rule-based systems, AI models, particularly neural networks, can be black boxes, which makes it hard to understand how they make decisions. But yet, the current regulations don't define minimum thresholds for explainability. And we also need strategies for updating models over time. For example, when is a model update just a patch, and when does it require full revalidation?

• Dr. Nick Spies

  Yeah, I think that's a really interesting question to kind of dive a little bit deeper on. At least in clinical pathology, we're really used to these kind of deterministic systems where we have, you know, hundreds or if not thousands of discrete rules built into our middleware or our lab information systems that actually take kind of direct actions based on whether a value is too high or too low or these really kind of discrete, perfectly explainable sets of kind of heuristics for these AI models. The complexity is both a blessing and a curse in that you can get a lot of increased performance out of whatever you're trying to do with some of these more complex models, but it often comes at the expense of that explainability piece. And so, as you mentioned, the data that we get out of these models is only as good as the data that we put in. And the data that we would kind of add to these models is going to be changing all day, every day as our systems change, our assays change, our analyzers change. So at what point? Do we really need to stop the works and retrain the models or revalidate that the model still works? Or how do we even build these real-time monitoring systems on top of the QC that we're already used to perform for our real-time assays is a really interesting question, but one that definitely needs to be fleshed out in more detail before we can really feel comfortable as laboratorians applying these models all across the laboratory.

• Dr. Matthew Hanna

  Those are all really great examples. Thank you for sharing. I mean, I think this all really just circles back to patient safety. We can even try to create messaging around AI is that it's not really the delta in performance, but it's really that patient safety net that sits for all the patients who are getting all this routine testing that it will act as a patient safety net to catch and act as a triaging workflow. for those who are rendering those interpretations and diagnoses. And we know that CLIA requires validation for any lab-developed test. Thankfully, we can still do those. And AI is no exception. I think most labs won't be building their own AI models. They're going to be licensing them or working with vendors to deploy them at their own hospitals. But the vendor claims don't replace that local validation that's needed or local verification that's needed. The labs and medical directors really still have to confirm that the AI works in their own lab, on their own cases, with their own equipment. So validation isn't a checkbox, it's a continuous process. We talked about ongoing monitoring, error tracking, quality management systems. These are all areas that are continuing to grow and are really essential in deploying any test that involves machine learning or AI as part of it. Dr. Spies, I've been hearing a lot of really exciting pioneering work on a new role that might be helping with our AI deployments and ongoing monitoring and called AI implementation specialists. Would love to hear more about that.

• Dr. Nick Spies

  Yeah, I think obviously it's a really exciting time to be in pathology with all of the kind of new technologies and new tools that we'll certainly be able to start playing with and a lot of laboratories will start implementing themselves. But with all of this new technology, Obviously, there comes a need for... medical directors, laboratory staff, for really all of us to get as familiar as possible with not only the nuts and bolts of how they work, but much more importantly, with how do we ensure that they are safely and effectively doing what we are asking them to do. And so this idea of the pathologist as the implementation specialist is, I think, a world that we certainly should be moving towards and getting more comfortable with. As medical directors, quality management is certainly part and parcel with what we do all day, every day. And a lot of that relies on a deep understanding of the technologies we're using, the assays and how they work, and most importantly, their failure modes and how we address them. All of that will be equally, if not more important as we move into the age of AI across our systems. And so having pathologists that understand enough about the technology and about these validation studies, these ongoing monitoring and deployment studies, and all of the added nuance that comes with these really powerful tools will just be another added kind of skill set that will be really important for us to both develop kind of organically and to explicitly start to teach in our residency programs and our fellowships and beyond.

• Dr. M. E. de Baca

  This has been a fascinating conversation. As you can see, pathology has strong framework. that can guide AI integration, but we need to adapt to meet the unique challenges. The emerging roles of AI implementation specialists and guides will be critical. If you're attending the CAP25, please check out the presentations at our Innovation Hub. Let me be the first here to thank Drs. Furtado, Spies, and Hanna for joining us today. And thank you, our listeners, for tuning in to CIPI Connections. Thanks for joining us for insights, updates, and the people behind the innovation. This has been CIPI Connections, where ideas meet action in pathology.