Researchers have unveiled an innovative artificial intelligence system named Dr. CaBot, designed to articulate its diagnostic reasoning as it navigates complex medical cases. In a groundbreaking move, the New England Journal of Medicine (NEJM) has published a diagnosis generated by Dr. CaBot alongside that of a human clinician in its esteemed medical case study series. This development holds significant promise for the fields of medical education and research.
The introduction of Dr. CaBot represents a familiar scenario in medicine: an expert diagnostician presents a challenging case to a group of colleagues, meticulously walking them through the patient’s symptoms and initial test results. The expert explains their reasoning in detail, considering every possibility with the aid of a slide deck. At the end of the presentation, the diagnosis and recommended next steps are revealed. However, this time, the diagnostician is an AI system.
Dr. CaBot: A New Era in Medical Diagnostics
Developed by researchers at Harvard Medical School, Dr. CaBot is intended as a tool for medical education. It operates in both presentation and written formats, demonstrating its reasoning process through a case by offering a differential diagnosis—a comprehensive list of potential conditions—and narrowing these possibilities to reach a final diagnosis. This ability to articulate its “thought process” sets Dr. CaBot apart from other AI diagnostic tools, which often focus solely on accuracy.
“We wanted to create an AI system that could generate a differential diagnosis and explain its detailed, nuanced reasoning at the level of an expert diagnostician,” said Arjun (Raj) Manrai, assistant professor of biomedical informatics at the Blavatnik Institute at HMS. Manrai developed Dr. CaBot with Thomas Buckley, a doctoral student at Harvard’s Kenneth C. Griffin School of Arts and Sciences and a member of the Manrai lab.
Breaking New Ground in Medical Publications
Although not yet ready for clinical use, Dr. CaBot has been demonstrated at Boston-area hospitals. The AI system’s capabilities were showcased in the NEJM’s Case Records of the Massachusetts General Hospital, marking the first time the journal published an AI-generated diagnosis. This publication offers a glimpse into Dr. CaBot’s potential for medical educators and students, and hints at its future application in clinical settings.
The concept of clinicopathological conferences (CPCs) dates back to the late 1800s, when Massachusetts General Hospital physicians began using patient case studies for medical education. In 1900, pathologist Richard Cabot formalized these as part of the curriculum for HMS doctors-in-training. Since 1923, NEJM has continuously published these cases as CPCs, teaching physicians how to reason through complex cases.
“The cases are pretty legendary. They’re known to be extremely challenging, filled with distractions and red herrings,” Manrai said.
AI and Human Diagnosticians: A Comparative Analysis
The October 8 NEJM article features a typical case presentation alongside a differential diagnosis from expert diagnostician Gurpreet Dhaliwal of the San Francisco Veterans Affairs Medical Center and the University of California, San Francisco. Dr. CaBot’s differential diagnosis follows, and researchers were encouraged to see that although the AI reasoned through the case differently than Dhaliwal, it arrived at a comparable final diagnosis.
During his graduate studies, Manrai was intrigued by how CPCs demystify the diagnostic process, reminiscent of the mystery novels he enjoyed. His lab has since studied the accuracy of AI models in providing patient diagnoses, leading to the development of Dr. CaBot.
The Technological Backbone of Dr. CaBot
Dr. CaBot is powered by OpenAI’s o3 large language reasoning model. Buckley, a Dunleavy Fellow in HMS’ AI in Medicine track, enhanced o3 with new capabilities, such as efficiently searching millions of clinical abstracts from high-impact journals. This feature helps Dr. CaBot properly cite its work and avoid factual hallucinations. The AI can also search its “brain” of several thousand CPCs to emulate the style of an expert diagnostician in NEJM.
Dr. CaBot delivers two main products: a five-minute, narrated, slide-based video presentation and a detailed written version of its reasoning and diagnosis. The presentations are “surprisingly lifelike,” according to Buckley, complete with colloquial phrases and filler words that resonate with physicians.
“The realness of the narrated presentation seems to connect with physicians,” Manrai noted.
Looking Ahead: The Future of AI in Medicine
As Dr. CaBot continues to evolve, its creators hope it will serve as a model for other medical-AI teams worldwide. The system’s ability to articulate its diagnostic reasoning could transform medical education and potentially enhance clinical practice. By bridging the gap between AI and human expertise, Dr. CaBot represents a significant step forward in the integration of technology in healthcare.
As the medical community continues to explore the potential of AI systems like Dr. CaBot, the implications for patient care and medical training are profound. The journey from Dr. Cabot to Dr. CaBot highlights a century-long evolution in medical diagnostics, paving the way for future innovations.
