Microsoft AI says its diagnostic system beat doctors on hard cases in Science study

Microsoft AI says its diagnostic system, MAI-DxO, outperformed generalist physicians on hard diagnostic cases in a study published in Science on 22 October 2026 [1][2]. On complex cases from The New England Journal of Medicine it reached 83.2% accuracy, against 36.1% for doctors using reference tools [2].

What did Microsoft AI test?

The study introduces SDBench, a benchmark that turns 304 NEJM case records and 325 emergency-department admissions from the MIMIC-IV-ED dataset into step-by-step cases [2]. A diagnostician, human or AI, starts with a short case summary, then asks questions, orders tests or commits to a diagnosis, and each test adds an estimated cost [2].

MAI-DxO, also called the Diagnostic Orchestrator, simulates a panel of physicians who propose diagnoses, choose questions or tests and challenge each other over several rounds [1]. In the paper it ran on OpenAI's GPT-5, and Microsoft says the gains held when it repeated the evaluation with GPT-5.5 [1][2].

How big was the gap to doctors?

Microsoft AI says its MAI-DxO system reached 83.2% accuracy on complex NEJM cases in a study published in Science. Generalist physicians in the same benchmark reached 36.1% with reference tools and 22.3% without them. On emergency-department cases the system reached 80.3% against 46.8% for physicians [2].

Cost is more nuanced than the headline. Without a budget limit, MAI-DxO's average estimated test cost on NEJM cases was $5,431, compared with $4,725 for physicians using references; a budget-limited setting reached 80.9% at $2,241 [2].

TechCrunch notes that much of the gain comes from stronger underlying models, and that with GPT-5.5 the orchestrator added little on emergency-department cases, where both setups scored about 85% [4].

What are the limits of the study?

The authors write that the comparisons do not establish superiority over specialists, multidisciplinary teams or real-world clinical practice [2]. The cases were text-only, excluded healthy patients and left out imaging and pathology slides, so the benchmark measures reasoning under idealized conditions rather than bedside performance [2].

Microsoft AI plans a prospective study with Beth Israel Deaconess Medical Center to test the system with real patients. In that study, patients will interact with the system directly, and its performance will be measured against physicians' independent assessments [1]. Microsoft has not said when or how the technology might become available, The Verge reports [3].

What does Microsoft mean by medical superintelligence?

Microsoft AI describes the work as part of its vision for medical superintelligence, outlined in a paper in BMJ Digital Health and AI [1]. The Verge describes the term as AI that can outperform the best human-led healthcare systems [3]. Suleyman told TechCrunch that Microsoft is working with Mayo Clinic on a healthcare foundation model planned for release next year [4].

Our read

Opinion: Publishing in Science and open-sourcing SDBench makes this far easier to scrutinize than a press release. But a benchmark built from teaching cases, scored against individual generalists, is not a hospital, and the Beth Israel study is the result that will actually matter.

What we could not verify

• How MAI-DxO would perform in real hospitals; all results come from past case records in a controlled, text-only benchmark.
• How the system compares with specialists or physician teams, who were not part of the comparison.
• When or how Microsoft might make the system available; the company has not said.
• Independent replication; all authors of the Science paper are Microsoft employees who hold equity in the company.