A recent study published in The Lancet Digital Health has found that artificial intelligence has the ability to determine the race of a patient using medical imaging. In the study, a team of researchers from Harvard Medical School and MIT developed an AI capable of correctly identifying a patient’s self-reported race using only medical images like X-rays. The researchers found that an AI program that had been trained to analyze CT and X-ray images could accurately predict a person’s race 90% of the time. Even in instances where low-quality images are used, the deep learning model trained on high-pass filtered images could produce accurate results when human radiologists were unable to identify the image as an X-ray at all.
Using a collection of clinical nursing notes containing information about patients who self-reported their race as either black or white, the models identified terms related to skin disorders that are diagnosed more frequently in white patients and comorbidities that are more common in black patients. The researchers attempted to restrict their race-predicting algorithms by eliminating the words that were the most accurate predictors of either race. However, the algorithm performs just as well. The researchers concluded that developing a racially unbiased algorithm is very difficult, if not impossible.
The study raises the troubling possibility that AI-based diagnostic systems may accidentally produce racially biased outcomes at a time when AI software is being utilized more and more to assist doctors in making diagnoses. For example, if given access to X-rays, an AI may automatically suggest a set course of treatment for all black patients, regardless of whether it is suitable for that particular patient. The patient’s actual doctor wouldn’t be aware that the AI’s diagnosis was based on racial information. An industry-wide adoption of a model like this can be extremely detrimental if it is widely used in hospitals and other settings. Medical negligence as a result of medical racism has an incredibly grim history, one that the medical field is still struggling to overcome. This has had an irrevocable impact on how communities of color use (or don’t use) the healthcare system. The already strained relationship between people of color and the healthcare industry could worsen if an AI were to be developed that could identify a patient’s race based on an X-ray alone.
Interestingly, the researchers have been unable to figure out exactly how the program determines a patient’s race when limited to medical images. It is suspected that melanin may be the determining factor, as X-rays and CT scans may recognize the increased melanin concentration of darker skin and include this information in a way that humans have never seen before. The researchers deduced that healthcare professionals should avoid utilizing AI diagnostic tools that can automatically produce biased results. AI should only be implemented when it is known for certain that they are not making decisions based on biases.
