Study Finds GenAI Diagnostic Skills Comparable to Non-Specialist Physicians
Click to start listening
Synopsis
A new study indicates that the diagnostic capabilities of generative AI (GenAI) are similar to those of non-specialist doctors, highlighting the potential for AI to support healthcare in resource-limited areas.
Key Takeaways
- Generative AI shows diagnostic abilities similar to non-specialist doctors.
- Study conducted by Osaka Metropolitan University.
- Analysis of 83 research papers from 2018 to 2024.
- ChatGPT was the most studied model.
- Further research needed in complex clinical scenarios.
New Delhi, April 18 (NationPress) A recent study has revealed that the diagnostic abilities of generative AI (GenAI) are on par with those of non-specialist doctors. The application of generative AI in diagnostics has gained significant attention in the healthcare industry, leading to numerous research papers being published about it.
However, due to varying evaluation metrics across studies, a thorough analysis was essential to assess the potential use of AI in real-world medical environments and to identify its advantages over human doctors.
A research team spearheaded by Dr. Hirotaka Takita and Associate Professor Daiju Ueda at Osaka Metropolitan University’s Graduate School of Medicine conducted a meta-analysis examining the diagnostic capabilities of generative AI. This analysis included 83 research papers published between June 2018 and June 2024 that spanned various medical fields.
Among the large language models evaluated, ChatGPT emerged as the most frequently studied.
The comparative assessment indicated that medical specialists achieved a 15.8 percent greater diagnostic accuracy than generative AI, as detailed in the study published in the journal npj Digital Medicine.
On average, generative AI demonstrated a diagnostic accuracy of 52.1 percent, with the latest AI models occasionally matching the accuracy of non-specialist doctors.
Dr. Takita noted, “This research shows that generative AI’s diagnostic capabilities are comparable to non-specialist doctors. It could be used in medical education to support non-specialist doctors and assist in diagnostics in areas with limited medical resources.”
According to the researchers, “Further studies are necessary, including evaluations in more complex clinical scenarios, performance assessments using real medical records, enhancing AI decision-making transparency, and validation across diverse patient populations.”
Founded in Osaka, one of Japan's largest public universities, Osaka Metropolitan University is dedicated to shaping society's future through the “Convergence of Knowledge” and advancing world-class research.
