Building AI Systems to Close the Gap in Health Care Inequities

Artificial intelligence (AI) can help radiologists do their jobs more efficiently and effectively but in some ways may unintentionally exacerbate inequities that already exist in modern health care.

Understanding those biases and how radiologists can ensure AI does not make them worse, was the subject of a Tuesday Hot Topic Session, "Artificial Intelligence and Implications for Health Equity: Will AI Improve Equity or Increase Disparities?"

Moderator Judy Wawira Gichoya, MBChB, MS, assistant professor in the Department of Radiology at Emory University School of Medicine, Atlanta, said radiologists must understand that AI, and medicine generally, do not exist in a vacuum.

"The data we use is collected in a social system that already has cultural and institutional biases," Dr. Wawira Gichoya said. "If we just use this data without understanding the inequities then algorithms will end up habituating, if not magnifying, our existing disparities."

Training Smarter AI

AI is only as good as the data it is built on, which is why it is important to make sure the data being fed into the algorithm is diverse and representative of the larger population, said Regina Barzilay, PhD, professor of computer science and electrical engineering at the MIT Institute of Medical Engineering & Science.

"We need to make sure the algorithm is not learning from the wrong information," Dr. Barzilay said.

Dr. Barzilay spoke passionately about her collaboration with Constance Lehman, MD, PhD, professor of Radiology at Harvard Medical School, Director of Breast Imaging, and Co-Director of the Avon Comprehensive Breast Evaluation Center at Massachusetts General Hospital in Boston. The two presented their partnership using deep learning to build an algorithm to help identify women's risk for breast cancer based on their mammogram alone and reduce risk of bias based on race or other factors.

Dr. Barzilay herself was diagnosed with breast cancer in her 40s, which made her a minority. "It taught me how important it is to create systems that are fair for everyone."

"It is a myth that AI would introduce bias or worsen healthcare disparities, we saw ample evidence in our research that AI could reduce disparities," Dr. Lehman said. This year, thousands of mammograms were canceled at Massachusetts General alone due to COVID-19, millions nationwide. Dr. Lehman said her institutions has used risk-based screening to determine which women to bring back in for screening first, a process that has been aided by AI.

But, AI isn't meant to replace radiologists, so the humans who are responsible for clinical decisions should understand how the model works and be able to filter out any possible bias that the machine introduces, Dr. Barzilay said.

"The most important thing to know is that you can't just look at the AI model and see that it is biased," said Luke Oakden-Rayner, MD, PhD, director of medical imaging research at the Royal Adelaide Hospital in Australia.

Instead, he suggested conducting exploratory error analysis to look at every error case and find common threads.

"Look at the cases it got right and those it got wrong," Dr. Oakden-Rayner said. "All the cases AI got right will have something in common and so will the ones it got wrong, then you can find out what the system is biased toward."

The Pain Gap Phenomenon

Ziad Obermeyer, MD, associate professor of health policy and management at the Berkeley School of Public Health, CA, discussed what he calls the pain gap, a phenomenon where pain of white patients is treated or investigated until a cause is found, while in other races it may be ignored or overlooked.

"Pain is concentrated in society's most disadvantaged, non-white, low income, lower educated patients," Dr. Obermeyer said. "These patients are reporting severe pain much more often. An obvious explanation is that maybe they have a higher prevalence of painful conditions, but that doesn't seem to be the whole story."

He referenced studies showing that if two patients have similar knee X-rays and are complaining of pain, the same image may be read differently depending on race.

"The stakes are pretty high," he said. "If you have severe pain and the disease is judged to be in your knee then you are a candidate for joint replacement surgery." Dr. Obermeyer went on to describe a common experience for non-white patients, "But if you are told it's nothing and sent away, that can lead to over-prescription of pain medication, emotional distress, loss of work, and other negative consequences," he said.

Rethinking How to Use AI

He explained that even AI is not a perfect solution to this problem. The Kellgren Lawrence grading for osteoarthritis severity was developed in 1957 using a group of coal miners in England, likely not a diverse dataset. If an AI model was trained to read knee X-rays using Kellgren Lawrence alone, it would be replicating the inequities currently in place, not improving upon them, he pointed out.

"Instead of learning from the radiologist, why don't we try listening to the patient. Instead of predicting the grade that the radiologist assigns, let's predict the experience of pain in that patient's knee," Dr. Obermeyer said, challenging radiologist to rethink how they use AI.

While that data may be harder to find, Dr. Obermeyer referenced an NIH-sponsored dataset that helped him experiment with this new type of algorithm. He found more than double the number of Black patients with severe pain in their knees who would be eligible for surgery than before.

"There's a lot of reason for optimism," Dr. Obermeyer said. "Machine learning opens up new ways to fight bias if we have the right data."

For More Information:

View the RSNA 2020 session Hot Topic Session: Artificial Intelligence and Implications for Health Equity: Will AI Improve Equity or Increase Disparities? — SPSH40 at RSNA2020.RSNA.org.