Using AI to Increase Residents’ Exposure to Important Pathologies

To address this issue, Dr. Cummings and colleagues created a precision education curriculum by supplementing curated teaching cases using a large language model (LLM), which categorized pathologies seen by each resident. 

“While the idea of a teaching library is not new, I think the fascinating innovation here is the use of artificial intelligence to improve the efficiency and precision of educational interventions,” Dr. Cummings said.  

“Especially in a busy reading room, it is critical to introduce teaching cases in the most efficient and non-intrusive way possible,” he added. “By targeting supplemental cases specifically to address educational gaps, I think we make the best use of our residents’ and teaching attendings’ time.”

Members of the institution’s musculoskeletal (MSK) division compiled a list of 140 important pathologies. They then assigned each pathology a priority level (high, intermediate or low) for the three required MSK rotations (PGY2, 3 and 4). Three MSK radiologists curated a teaching file that contained multiple examples of each pathology (2,885 total cases). 

The investigators then used a HIPAA-compliant GPT-4 model to analyze the amount of exposure to these pathologies in the three years prior to the creation of the precision education curriculum. During a four month period, residents who were on four-week MSK rotations each received three supplemental teaching file cases per day. The algorithm chose these cases based on the resident’s prior exposure and PGY level priority.

The investigators tallied the pathologies residents saw before and after the algorithm was implemented and compared the numbers to a group of 56 MSK rotations who did not receive supplemental cases.  

The researchers found that the average number of unique important pathologies seen by residents increased 114% for PGY-2, 65% for PGY-3, and 113% for PGY-4. When the investigators excluded low-priority pathologies, they found an even greater increase of 227% and 112% for PGYs 2-4, respectively.

Finding Value in Enhanced, Personalized Resident Education

“I was pleasantly surprised by the degree to which the precision education curriculum increased the diversity of pathologies seen by residents—more than tripling the diversity of high priority cases seen by our first-year trainees,” Dr. Cummings said. 

“Most residents received only three supplemental cases per day, and some even fewer, but this manageable number was more than sufficient to substantially augment the diversity of pathologies they saw during their rotations,” he added. 

Dr. Cummings hopes this study encourages others to enhance and personalize resident education. 

“Precision resident education promises to increase the efficiency and comprehensiveness of radiology resident training, which may lead to more accurate diagnoses and better treatment plans for future patients,” he said, noting that AI has broader relevance to radiology than image recognition alone. 

“LLMs like GPT-4 can powerfully manipulate written language, in this case enabling us to automatically identify educational gaps and fill them,” he said. “I am excited to see other ways LLMs are being put to use in radiology.”

Access the presentation, “AI-Driven Case Supplementation Augments and Diversifies Resident Training Exposure to Important Pathology: Initial Experience in Precision Education,” (S3B-SPIN-3) on demand at RSNA.org/MeetingCentral.