For the selection of appropriate imaging tests, ChatGPT can deliver recommendations that may be comparable with those of the European Society of Radiology (ESR) iGuide, according to findings presented in a scientific poster.
Rosen
In ChatGPT, blocks of text—or tokens—are provided to the model in a dataset of unstructured text from websites, books, articles and other publicly available sources, explained Shani Rosen, MSc, director of the DataMED laboratory at Tel Aviv University's Department of Nursing in Israel.
By assimilating these tokens, the model gains proficiency in grammar, syntax and semantics, and learns to predict the words or phrases that are likely to appear next.
Rosen and her coauthor, DataMED head Mor Saban, PhD, demonstrated that when ChatGPT is presented with clinical data about patient symptoms, it can generate recommendations that may help clinicians select appropriate imaging tests.
The authors collected data from 97 cases at a tertiary hospital, of patients who presented to the ED with abdominal symptoms and who underwent abdominal CT. Given the patients' information, ChatGPT produced recommendations for each case. The authors compared those recommendations with the ESR iGuide and analyzed cases of disagreement.
"We ruled out three cases in which the ESR iGuide could not provide recommendations due to missing indications in the ESR iGuide system," Rosen noted. In cases of disagreement, she said ChatGPT's language was sometimes vague. One example given was, "Additionally, an echocardiogram could be done to evaluate the heart structure and function." Another recommendation was, "Depending on the clinical presentation, radiography, ultrasound, a CT or MRI could be used."
Specific Recommendations for Imaging Surprisingly Accurate
Next, the team selected 66 cases in which ChatGPT recommended a chest/abdominal/pelvic CT, and they asked four specialists to grade the appropriateness of the referrals.
"I was surprised by how specific the tool can be in many cases and by the high accuracy level, with 87% of cases deemed accurate," Rosen said. "That is incredible for a language tool, one that isn't even designed specifically for medical tasks."
Clinical, ethical and regulatory implications need to be addressed before ChatGPT is considered for clinical use, the authors emphasized. But it holds potential to help clinicians select appropriate imaging procedures for their patients.
"I am looking forward with excitement to seeing what language tools will be able to do in the years ahead, once they have been trained for medical use," Rosen said, "and to seeing how they may empower clinicians in their medical decision-making."
Access the presentation, "Evaluating the Reliability of ChatGPT as a Tool for Imaging Test Referral: A Comparative Study with a Clinical Decision Support System," (R5B-SPIN-7) on demand at Meeting.RSNA.org.
