Microsoft claims AI diagnostic system beats physicians on complex cases

Microsoft has entered the high-stakes race to prove artificial intelligence can outperform human doctors in complex clinical scenarios.

The company released a study in Science on Thursday showing that its new diagnostic system, MAI-DxO, correctly identified difficult medical cases far more often than generalist physicians. In a simulation using complex cases from the New England Journal of Medicine, the AI system achieved an accuracy rate of 83 percent. By comparison, primary care doctors and general internists working individually with reference materials were correct only 36 percent of the time. Mustafa Suleyman, the CEO of Microsoft AI, described this result as a significant step toward what the company calls medical superintelligence.

The system operates by coordinating several AI agents that function like a panel of specialists. These agents question the patient, order necessary tests, and debate potential diagnoses before arriving at a conclusion. Microsoft noted that such complex cases are typically handled by specialists or teams in real-world settings, who were not included in the direct comparison. Dominic King, vice president of Health at Microsoft AI, emphasized the importance of the finding in addressing the growing gap between medical demand and the supply of expert knowledge.

Cost and model dependency were key factors in the analysis. Microsoft tested the system using models from various companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The best performance came when paired with OpenAI’s GPT-5 and GPT-5.5 models. While the orchestration system reduced costs by about $3,000 per case compared to using GPT-5 alone, it only improved accuracy by roughly four points on complex cases. For emergency department cases, the improvement in both accuracy and cost was minimal when using the same models.

The advantages of the multi-agent approach became more apparent with less powerful models. For instance, Gemini 2.5 Flash scored 46.4 percent on its own but rose to nearly 69 percent with the assistance of the diagnostic orchestration system. Researchers noted that as models approach the theoretical maximum of accuracy, the incremental gains from orchestration naturally diminish. The system proved most useful for optimizing both accuracy and cost in scenarios where the underlying model was not at its peak capability.

Microsoft is now moving toward real-world validation through a partnership with Beth Israel Deaconess Medical Center in Boston. The company will evaluate the clinical reasoning of the system across different hospital settings. Patients will receive standard care, including triage and specialist consultation, while those who consent will also interact with the AI tool. Researchers will then compare the AI’s recommendations against the decisions made by human doctors. King called this the largest study of its kind, aiming to gather compelling evidence for broader deployment.

Suleyman, who co-founded DeepMind before joining Microsoft in 2024, is pushing for a major breakthrough in medical record handling. He predicts that by 2027, the company will be on the verge of a coding style breakthrough that allows for streaming highly accurate predictions about how a medical record might unfold. He is also working with the Mayo Clinic to train a foundation model for healthcare, with a release planned for next year. Suleyman believes this infrastructure will enable enormous leaps in clinical decision support.

The company’s efforts coincide with the publication of a manifesto defining medical superintelligence. This is defined as the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care. Microsoft aims to move beyond simple benchmarking tasks and integrate deeply with clinicians and healthcare systems. Suleyman stated that this focus could serve as a differentiator for Microsoft compared to other frontier AI labs, as the company already sells dictation and radiology tools to clinicians.

The competitive landscape is heating up as Google and OpenAI also claim their AI systems can match or outperform doctors. Google recently published a clinical study alongside Beth Israel in The Lancet to evaluate its conversational diagnostic AI in urgent care visits. Microsoft’s leadership remains confident in the rapid pace of progress. Suleyman expects healthcare models to become an order of magnitude better in capability within the next 24 to 36 months. He cited the flywheel effect seen in coding and chatbots, where real-world usage helps identify gaps and drive rapid improvements. The company is committed to building the number one state-of-the-art medical clinical decision support product in the coming years.