Communications of the ACM
Researchers Test AI-Powered Chatbot's Medical Diagnostic Ability
When the AI model provided the correct diagnosis in its differential, the mean rank of the diagnosis was 2.5 (SD, 2.5).
Credit: Getty Images
Physician-researchers at Beth Israel Deaconess Medical Center (BIDMC) tested ChatGPT-4's ability to make accurate diagnoses in challenging medical cases and found that the generative AI selected the correct diagnosis as its top diagnosis nearly 40 percent of the time, and provided the correct diagnosis in its list of potential diagnoses in two-thirds of challenging cases.
"We wanted to know if a generative model could 'think' like a doctor, so we asked one to solve standardized complex diagnostic cases used for educational purposes," says Dr. Adam Rodman, co-director of the iMED Initiative at BIDMC and an instructor at Harvard Medical School. "It did really, really well." The research is published in JAMA.
"While chatbots cannot replace the expertise and knowledge of a trained medical professional, generative AI is a promising potential adjunct to human cognition in diagnosis," says first author Dr. Zahir Kanjee at BIDMC and assistant professor at Harvard Medical School.
From Beth Israel Deaconess Medical Center
View Full Article
No entries found