AI can speed up cancer diagnoses from medical images, but the models that have been developed can be hacked.
That’s according to a study by researchers at the University of Pittsburgh that has been published in Nature Communications.
In a statement, the researchers said the study brings attention to a potential safety issue for medical AI known as “adversarial attacks,” which seek to alter images or other inputs to make models arrive at incorrect conclusions.
"What we want to show with this study is that this type of attack is possible, and it could lead AI models to make the wrong diagnosis – which is a big patient safety issue," senior author Shandong Wu, associate professor of radiology, biomedical informatics and bioengineering at Pitt, said in a statement. "By understanding how AI models behave under adversarial attacks in medical contexts, we can start thinking about ways to make these models safer and more robust.”
Given the increasing reliance on deep learning models to augment human expertise in diagnostic roles, the researchers noted that "It is imperative to build trustworthy, reliable and safe AI systems for clinical deployment."
The main problem is that despite the advantages they introduce, AI technologies also are at risk from cyberthreats. Potential motivations for such attacks include insurance fraud from health care providers looking to boost revenue or companies trying to adjust clinical trial outcomes in their favor. So-called “adversarial attacks” on medical images range from tiny manipulations that change the AI’s decision, but are imperceptible to the human eye, to more sophisticated versions that target sensitive contents of the image, such as cancerous regions—making them more likely to fool a human.
For their study, the Pitt team used mammogram images to train an algorithm to distinguish breast cancer-positive cases from negative ones. They then developed generators to create intentionally misleading data by "inserting" cancerous regions into negative images or "removing" regions from positive images.
The trick worked: The model was fooled by 69.1% of the fake images.
The researchers then recruited five human radiologists to spot whether images were real or fake. Results varied: Depending on the person, the experts ranged from 29% to 71% accuracy on spotting the images' authenticity.
"Certain fake images that fool AI may be easily spotted by radiologists. However, many of the adversarial images in this study not only fooled the model, but they also fooled experienced human readers," said Wu. "Such attacks could potentially be very harmful to patients if they lead to an incorrect cancer diagnosis."
According to Wu, the next step is developing ways to make AI models more robust to adversarial attacks.
“One direction that we are exploring is ‘adversarial training’ for the AI model,” he explained. “This involves pre-generating adversarial images and teaching the model that these images are manipulated.”
Photo by scyther5/Getty Images