AI-generated fake X-rays are now so realistic that neither experienced radiologists nor leading AI models can reliably detect them, according to a study published March 24 in Radiology. Led by Mickael Tordjman, M.D., at the Icahn School of Medicine at Mount Sinai, the research tested 17 radiologists from 12 institutions across six countries on 264 X-ray images — half real, half generated by ChatGPT and RoentGen.
Without being told the dataset contained fakes, only 41% of radiologists noticed anything suspicious. Even after being informed, their mean detection accuracy reached just 75%, with individual scores ranging from 58% to 92%. Years of experience made no difference. Four multimodal AI models — GPT-4o, GPT-5, Gemini 2.5 Pro and Llama 4 Maverick — performed between 57% and 85% accuracy, barely better than the humans.
The researchers warn of serious real-world risks: fabricated fractures could be used in fraudulent litigation, and hackers gaining hospital network access could inject synthetic images to manipulate diagnoses. As generative AI improves, the medical imaging community faces an urgent need for detection tools and authentication standards.
