AI Assisted Radiologists See Improved Performance in Detection of Breast Cancer
An artificial intelligence algorithm demonstrated better diagnostic performance in breast cancer detection compared to radiologists, who also saw improved performance when aided by the technology.
Data published in The Lancet Digital Health indicated that an artificial intelligence (AI) algorithm developed with large-scale mammography data demonstrated better diagnostic performance in breast cancer detection compared to radiologists.1
When aided by AI, the significant improvement observed in radiologists’ performance supports the application of AI to mammograms as a diagnostic support tool.
“One of the biggest problems in detecting malignant lesions from mammography images is that to reduce false negatives – missed cases – radiologists tend to increase recalls, casting a wider safety net, which brings an increased number of unnecessary biopsies,” corresponding author Eun-Kyung Kim, MD, breast radiologist at Yonsei University Severance Hospital in South Korea, said in a press release.2 “It requires extensive experience to correctly interpret breast images, and our study showed that AI can help find more breast cancer with lesser recalls, also detecting cancers in its early stage of development.”
The AI algorithm, being marketed as Lunit INSIGHT MMG by Lunit, was developed and validated using 170,230 mammography examinations collected from 5 institutions in South Korea, the US, and the UK, consisting of Asian and Caucasian female breast images. Across the cohort, there were 36,468 cancer-positive cases confirmed by biopsy, 59,544 benign cases confirmed by biopsy or follow-up imaging, and 74,218 cases without issue.
For the multicenter, observer-blinded, reader study, 320 mammograms (160 cancer-positive, 64 benign, 96 normal) were independently obtained from 2 institutions. Moreover, 14 radiologists participated as readers and evaluated each mammogram in regard to likelihood of malignancy (LOM), location of malignancy, and necessity to recall the patient, first without and then with assistance of the AI algorithm. Performance was assessed in terms of LOM-based area under the receiver operating characteristic curve (AUROC) and recall-based sensitivity and specificity.
“It is an unprecedented quantity of data with accurate ground truth – especially the 36,000 cancer cases, which is 7 times larger than the usual number of datasets from resembling studies conducted previously,” first author Hyo-Eun Kim, chief product officer for Lunit. “The quality of data has also been assured, with ethnic diversity, covering various imaging and scanning conditions. The marriage between the diversity of the dataset and the uniqueness of our algorithm, designed in interaction with one another, has been key to years of development of Lunit INSIGHT MMG since early 2016.”
The standalone performance for the AI was AUROC 0.959 (95% CI, 0.952–0.966) overall, and 0.970 (95% CI, 0.963–0.978) in the South Korea dataset, 0.953 (95% CI, 0.938–0.968) in the USA dataset, and 0.938 (95% CI, 0.918–0.958) in the UK dataset. In the reader study, the performance level of AI was 0.940 (0.915–0.965), which was significantly higher than that of the radiologists without AI assistance (0.810; 95% CI 0.770–0.850; P < 0.0001). When AI assistance was added, the radiologists' performance saw an improvement to 0.881 (0.850–0.911; P < 0.0001).
Additionally, AI was more sensitive to detect cancers with mass (90% vs 78% of 59 cancers detected; P = 0.044) or distortion or asymmetry (90% vs 50% of 20 cancers detected; P = 0.023) than radiologists. AI was also better at detecting T1 cancers (91% vs 74% of 80; P = 0.0039) or node-negative cancers (87% vs 74% of 119; P = 0.0025) than radiologists.
“This result shows that AI can be used as an effective diagnostic support tool for breast cancer detection, which is worth evaluating in prospective trials,” the authors wrote.
The Korean Ministry of Food and Drug Safety has already approved Lunit INSIGHT MMG, and the AI algorithm is already commercially available and under clinical use. Further, the AI is pending approval by the European Commission within the first quarter and anticipating FDA clearance sometime later this year.
Refererences:
1. Kim H, Kim HH, Han B, et al. Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. The Lancet Digital Health. doi:10.1016/S2589-7500(20)30003-0.
2. AI-assisted Radiologists Can Detect More Breast Cancer with Reduced False-positive Recall [news release]. Seoul, South Korea. Published February 10, 2020. prnewswire.co.uk/news-releases/ai-assisted-radiologists-can-detect-more-breast-cancer-with-reduced-false-positive-recall-821740767.html. Accessed February 28, 2020.
