In recent years, the Radiology Society of North America (RSNA), has issued an AI Challenge in the name of encouraging the creation of AI tools for radiology.
For the 2019 challenge, researchers from two medical societies, RSNA and the American Society of Neuroradiology (ASNR), along with 60 volunteers, assembled a dataset of over 25,000 brain CTs contributed by four research institutions and worked with volunteers from the American Society of Neuroradiology to create the largest public brain hemorrhage image database
According to reports, it is hoped the creation of the database will speed up the development of machine learning algorithms that will aid with the detection and characterization of the life-threatening condition. Accuracy in diagnosing the presence and type of intracranial hemorrhage is a vital part of effective treatment, as even a small hemorrhage can lead to death if it is in a critical location.
For the database, the competition’s organizers compiled the images from Stanford University in Palo Alto, California, Universidade Federal de São Paulo in São Paulo, Brazil, and Thomas Jefferson University Hospital in Philadelphia, Pennsylvania.
“The value of this challenge is to create a dataset that might lead to a generalizable solution, and the best way to do that is to train a model from data originating from multiple institutions that use a variety of CT scanners from various manufacturers, scanning protocols and a heterogeneous patient population,” explained lead author, Dr Adam Flanders, neuroradiologist and professor at Thomas Jefferson University Hospital. “In this case, we had data from three institutions and international participation. The dataset is unique, not only in terms of the volume of abnormal images but also the heterogeneity of where they all came from.”
The 60 volunteers annotated 874,035 brain haemorrhage CT images in 25,312 unique exams, marked each image as normal or abnormal. For the abnormal images, they indicated the hemorrhage subtype.
“It was a nail-biter all the way along,” Dr Flanders said of the process. “We were building the airplane while it was in flight. When you consider the number of images that we had to de-identify locally, consume, curate, label, cross-check and then organize into just the right datasets to release to the contestants, there was a lot of work involved by the volunteer workforce, the RSNA Machine Learning Subcommittee, data scientists, contractors and RSNA staff. The 10 top solutions came from all over the world. Some of the winners had absolutely no background in medical imaging.”
The dataset was released under a non-commercial license, meaning it is freely available to the AI research community for non-commercial use and further enhancement.
Dr Flanders also noted the the objective of engaging with a subspecialty society to leverage their unique expertise in developing a high-quality dataset is an effective and useful pathway to follow for future collaborations.