Highlighting AI Diagnostic and Prognostic Developments in Oncology Care

Imaging assisted by artificial intelligence (AI) may detect biomarkers and mutational statuses that may have been previously impossible to identify, according to Arturo Loaiza-Bonilla, MD, MSEd, FACP.

CancerNetwork® spoke with Loaiza-Bonilla, systemwide chief of Hematology and Oncology at Saint Luke’s University Health Network (SLUHN), about different digital pathology and multimodal approaches he believes may have the potential to transform clinical practice.

Loaiza-Bonilla suggested that AI is effective with diagnostics and pattern recognition, an idea that was first realized when AlexNet—a deep learning model known as a convolutional neural network (CNN)—solved the ImageNet challenge, effectively revealing that CNNs process images better than humans. He further expressed that the ability of AI tools to recognize and describe images helps enhance pathology, particularly as it relates to brain imaging.

Next, Loaiza-Bonilla expressed that an impact was observed for single-cell sequencing, which has aided clinicians in identifying changes in tumor microenvironments based on subtypes of lymphocytes as well as discovering new biomarkers. He indicated that AI has the capacity to identify aspects of a microenvironment that cannot be processed by humans alone.

Furthermore, Loaiza-Bonilla touched upon the significance of AI-enhanced hematoxylin and eosin (H&E) imaging, which can detect biomarker and mutational status in tissue samples and give prognostics. Additionally, he expressed that multimodal approaches in pathology enable a combination of imaging, genomics, and clinical data to create foundational models. According to Loaiza-Bonilla, these foundational models enable clustering of large numbers of patients and data into discernable pathologic categories.

Transcript:

Diagnostics and pattern recognition are things that AI is pretty [effective with]...and we knew that AI was something that was going to impact us all when the models were able to solve this challenge called the ImageNet challenge when it came up. AlexNet [then] came up and was able to [perform] better than humans. Now, we have the ability to post a picture and [have] any of these tools recognize and describe it in full detail. Pathology, being something that looks at patterns within our brains, can really help us. In using computer vision, doing segmentation of what we call kernels—which are like the pixels that comprise images on pathology when we do whole-slide uploading in these models—is very promising.

We can go from spatial transcriptomics to single-cell sequencing; we are now seeing a major impact on these. For example, [we were] curious about how patients may benefit from immunotherapy because we have changes in the microenvironment based on subtypes of lymphocytes. Now, we are using data pathology to give us those biomarkers that were sometimes unknown to us; they are giving us new insights on things that we were not aware of. We are making an impact on response to treatment. Our eyes are amazing tools that have evolved for all of [human] history, but we know now that when you put billions of data points in terms of visual pixels, we can start seeing things that even our eyes and brains typically do not process. They may process [them], but they do not surface because we do not know they exist.

Now, we can do a hematoxylin and eosin [H&E] imaging, and they can detect if that sample has positivity for not only a biomarker, such as HER2 or PD-L1, but even beyond, like mutational status. Right now, we do not know how, but the system is able to detect those things.

Things that we never even thought were possible before can now be found with these multimodal approaches in pathology. The idea is to combine pathology images with genomics and clinical data, so we can develop these multimodal predictive and prognostic approaches––called foundational models––which is having tons of patients put under this analysis and being able to start putting those clusters of patients and data together.