• Posted April 16, 2024

AI Matches Experts in Spotting Radiology Errors

Artificial intelligence can serve as a helpful backup editor to radiologists, making sure their reports are accurate and reliable, a new study says.

The OpenAI program GPT-4 performed about as well as a group of radiologists in detecting errors that had been placed into reports, according to findings published April 16 in the journal Radiology.

"This efficiency in detecting errors may hint at a future where AI can help optimize the workflow within radiology departments, ensuring that reports are both accurate and promptly available, thus enhancing the radiology department's capacity to deliver timely and reliable diagnostics,"said lead researcher Dr. Roman Gertz, a radiology resident at University Hospital of Cologne in Germany.

This is the first study to compare GPT-4 directly against human performance in reviewing radiology reports for accuracy, speed and cost-effectiveness, Gertz said.

For the study, Gertz and his colleagues collected 200 radiology reports based on X-ray, CT scan and MRI imaging between June and December 2023 at a single hospital.

The researchers intentionally inserted 150 errors into 100 of the reports, including omissions, spelling errors, confusing passages and other mistakes.

They then asked both GPT-4 and a group of six radiologists -- two senior radiologists, two attending physicians and two residents -- to separately pore over the reports and find the mistakes.

Overall, GPT-4 had an error detection rate of 83% compared with 89% for senior radiologists and 80% for attending radiologists and radiology residents, researchers said.

There was no evidence of a significant difference in the average performance of GPT-4 compared to the radiologists.

The AI program also required less processing time per report than even the fastest human reader, and it had lower average correction cost per report than the most cost-efficient radiologist.

The study shows that AI can improve patient care by enhancing the accuracy of radiology reports and the speed with which the reports can be returned, Gertz said.

"The study addresses critical health care challenges such as the increasing demand for radiology services and the pressure to reduce operational costs,"Gertz said in a journal news release. 

"Ultimately, our research provides a concrete example of how AI, specifically through applications like GPT-4, can revolutionize health care by boosting efficiency, minimizing errors and ensuring broader access to reliable, affordable diagnostic services -- fundamental steps toward improving patient care outcomes,"Gertz added.

More information

The Brookings Institution has more on AI in health care.

SOURCE: Radiological Society of North America, news release, April 16, 2024