Home›Oncology› AI diagnostic tools show high accuracy but limited real-world validation in systematic review
AI diagnostic tools show high accuracy but limited real-world validation in systematic reviewArtificial intelligence shows high accuracy in detecting various medical conditions
Frontiers in MedicinePublished July 18, 2026Study authors: Nafiz Fahad, Ridwan Jamal Sadib, Rakib Hossain Sajib, Md Kishor Morol, Dip Nandi, Tze Hui LiewDOI ↗Editorial oversight: Dr. Julia Lee, PhD · Oncology, Genomics & Drug Development
AI-generated summary of the cited source, checked by automated accuracy review.
How we work
Share
Key Takeaway
Interpret AI diagnostic accuracy above 90% cautiously due to internal validation and curated datasets.
This systematic review synthesizes 24 studies evaluating AI or deep learning for disease detection or diagnostic support across X-ray, CT, MRI, mammography, ultrasound, dermoscopy, retinal fundus imaging, optical coherence tomography, and abdominal CT. Conditions covered include lung diseases, COVID-19, pneumonia, lung cancer, breast cancer, melanoma, dermatological disorders, brain tumors, diabetic retinopathy, and pancreatic ductal adenocarcinoma.
Several papers reported accuracy or sensitivity above 90%, but the authors caution that these values may be inflated by methodological factors such as internal validation, use of curated public datasets, class-balanced splits, augmentation, and limited demographic reporting. The review also discusses explainability methods (Grad-CAM, Grad-CAM++, LIME, SHAP, saliency maps, layer-wise relevance propagation), fairness, privacy-preserving learning, uncertainty estimation, and human-centered clinical trust.
Limitations include that many studies relied on internal validation and used curated public datasets with class-balanced splits and augmentation. Limited demographic reporting was noted. The authors emphasize that responsible medical-imaging AI should be evaluated through multidimensional evidence, including external and subgroup validation, calibration, privacy risk analysis, clinician-centered explanation assessment, workflow integration, regulatory readiness, and post-deployment monitoring.
How this fits prior evidence
This systematic review extends prior coverage by synthesizing AI diagnostic accuracy across multiple diseases and imaging modalities, whereas prior items focused on specific conditions or treatments. It confirms that high accuracy values (above 90%) are commonly reported but may be inflated by internal validation, echoing cautions from prior coverage about interpreting preclinical or early-phase results cautiously. The review addresses a gap by systematically cataloging methodological limitations across diverse AI applications.
Doctors and technicians often rely on imaging like X-rays, CT scans, and MRIs to spot serious illnesses. New research looked at how artificial intelligence (AI) helps identify these conditions. The review of 24 studies found that many AI systems reached accuracy or sensitivity levels above 90% when looking for issues like lung cancer, breast cancer, and skin disorders.
While these high numbers are impressive, the researchers suggest we look closer at how those results were achieved. Many of the studies used internal validation, which means they tested the AI on the same data it was trained on. They also used curated datasets and balanced samples to make the results look consistent. These factors can sometimes make a tool seem more ready for everyday use than it actually is.
To move forward safely, the experts say we need more rigorous testing. This includes checking how AI performs in diverse groups of people and ensuring the tools are easy for doctors to trust and use in a real clinic. For now, these high accuracy scores should be viewed with caution until more varied testing is done.
What this means for you:
AI shows high accuracy in medical imaging, but results may be inflated by specific study designs.
Common questions
How accurate is AI at finding diseases like cancer?
Several studies reported that AI systems achieved accuracy or sensitivity levels above 90% when identifying conditions such as lung cancer, breast cancer, and melanoma. However, these high numbers should be interpreted cautiously because some studies used internal validation or curated datasets which can make results appear higher than they might be in a real-world setting.
What types of medical images can AI help analyze?
AI and deep learning are being tested across many different imaging methods. This includes X-rays, CT scans, MRIs, mammography, ultrasound, dermoscopy, retinal fundus imaging, and optical coherence tomography to help detect various conditions like lung diseases and skin disorders.
Is AI ready to replace doctors in diagnosing patients?
The research suggests that for AI to be used safely in clinics, it must undergo more thorough testing. This includes checking how well it works with different groups of people, ensuring it is easy for clinicians to understand and trust, and monitoring its performance after it is deployed in a real medical setting.
Responsible artificial intelligence (AI) in medical imaging requires more than high diagnostic accuracy; it also requires transparent reasoning, equitable performance across patient subgroups, privacy protection, calibrated uncertainty, and clinical trustworthiness.
This PRISMA-informed systematic review synthesized 24 studies published between 2020 and 2025 that used AI or deep learning for disease detection or diagnostic support in X-ray, CT, MRI, mammography, ultrasound, dermoscopy, retinal fundus imaging, optical coherence tomography, and abdominal CT. PubMed, Scopus, Web of Science, IEEE Xplore, ScienceDirect, SpringerLink, and Google Scholar were searched, and extracted evidence was appraised qualitatively using adapted QUADAS-2 and PROBAST-AI domains.
The included studies covered lung diseases, COVID-19, pneumonia, lung cancer, breast cancer, melanoma and other dermatological disorders, brain tumors, diabetic retinopathy, chest abnormalities, and pancreatic ductal adenocarcinoma. Explainability methods such as Grad-CAM, Grad-CAM++, LIME, SHAP, saliency maps, and layer-wise relevance propagation dominated the evidence base, whereas fairness, privacy-preserving learning, uncertainty estimation, and human-centered clinical trust were represented by fewer studies. Several papers reported accuracy or sensitivity above 90%, but these values should be interpreted cautiously because many studies relied on internal validation, curated public datasets, class-balanced splits, augmentation, or limited demographic reporting.
Responsible medical-imaging AI should be evaluated through multidimensional evidence, including external and subgroup validation, calibration, privacy risk analysis, clinician-centered explanation assessment, workflow integration, regulatory readiness, and post-deployment monitoring.
