Intracranial hemorrhage (ICH) is a major cause of mortality and morbidity, resulting from variable underlying causes such as hypertension, trauma, ruptured aneurysms, malignancy and coagulation disorders.
"The accurate and timely detection of ICH are critical factors for patient survival and appropriate management," said Cody Savage, BS, a 4th year medical student at the University of Alabama at Birmingham (UAB) School of Medicine.
The current standard for detecting ICH is non-contrast head CT followed by interpretation by a radiologist – a process that can be supported by Artificial Intelligence. In fact, AI solutions for computer-aided detection and triage (CADt) of ICH identified on non-contrast head CT have U.S. Food and Drug Administration (FDA) clearance and are already being used at multiple medical centers.
But just how effective is this technology at improving detection and interpretation?
That is the question Savage decided to ask.
Human vs AI
According to Savage, data showing that AI supported CADt increases diagnostic accuracy and/or reduces report turn-around-time (TAT) have mostly been obtained from retrospective single institutional studies.
"The problem is that retrospective studies are prone to bias and assumptions that may not hold true in a real-world clinical practice environment," explained Savage.
To address these shortcomings, a recent UAB study evaluated a CADt system's actual effectiveness for detecting ICH in non-contrast head CT in a clinical practice setting. The large single-center prospective study evaluated accuracy, miss rate and report TAT for radiologists working with and without a CADt system in a real-world setting.
"What we found was that AI assistance did not improve the accuracy, miss rate or report TAT of radiologists," Savage said.
Savage went on to note that the ability of the CADt system to improve report TAT in the clinical practice setting likely depends on the deployment method.
Furthermore, while AI assistance was shown to decrease the miss rate in the outpatient and ED settings and mildly improve the accuracy in the outpatient setting, this was not the case for the inpatient setting where the prevalence of ICH, post-surgical cases and repeat imaging are higher.
"In my opinion, the benefits of AI for detection and triage of ICH are probably overstated," said Savage.
A High Degree of Skepticism Needed
Because these results conflict with those of prior retrospective studies, the results from those studies need to be interpreted with a high degree of skepticism.
"Radiology groups interested in using a CADt system should carefully consider their clinical practice setting, expectations for improvement and deployment method," concluded Savage. "AI vendors should also review the results of our study and work to improve the human-to-AI interface accordingly."
Access the presentation, "Prospective Real-World Comparison of Standard of Care vs. AI for Detection and Triage of Intracranial Hemorrhage on Non-contrast Head CT," (W6-SSNR12-2) on demand at Meeting.RSNA.org.