AI Panel Beats Doctors in Complex Care

For patients facing difficult medical diagnoses, the landscape of who makes the final call may soon include a digital panel of experts. A new study indicates that artificial intelligence systems are beginning to outperform human generalists in handling complex cases. This shift suggests that the support systems available to clinicians will evolve significantly, potentially altering how primary care providers approach ambiguous symptoms.

The system in question, named MAI-DxO, operates by coordinating multiple AI agents that mimic a team of doctors. Rather than relying on a single model, these agents question the patient, order necessary tests, and debate the findings before reaching a conclusion. In a simulation using complex cases from the New England Journal of Medicine, this collaborative approach was correct 83% of the time. By comparison, primary care doctors and general internists working individually with reference materials were correct only 36% of the time.

However, the comparison comes with important caveats. The study focused on generalist physicians, while in real-world settings, such complex cases are often handled by specialists or teams of doctors. Those specialists were not part of the head-to-head comparison. Dominic King, vice president of Health at Microsoft AI, noted that the core issue remains the growing mismatch between the demand for medical expertise and the available supply of professionals.

The results show that much of the performance improvement stems from the underlying power of the AI models themselves. Microsoft tested the system with models from various companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest accuracy scores were achieved using OpenAI’s GPT-5 and GPT-5.5. When GPT-5 was used alone with a basic prompt, it scored 78.9% on complex cases at a cost of approximately $8,400 per case. The orchestration system reduced that cost by about $3,000 but increased accuracy by only four points.

With the newer GPT-5.5 model, the gains were more pronounced for complex cases, pushing accuracy from 72% to nearly 87% at a slightly lower cost. Yet, the system provided little benefit for emergency department cases. In those scenarios, both the standalone model and the orchestrated system scored around 85% at similar costs. This suggests the multi-agent approach is most valuable for difficult, non-urgent diagnoses rather than rapid triage.

Mustafa Suleyman, CEO of Microsoft AI, described the work as a step toward medical superintelligence. He explained that the company is collaborating with the Mayo Clinic to train a foundation model specifically for healthcare, with a release expected next year. Suleyman believes this infrastructure will allow for enormous leaps in clinical decision support, particularly in the ability to predict how medical records might unfold over time.

To validate these findings in real-world settings, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. The evaluation will compare the AI’s recommendations against those of human doctors in both emergency and urgent care environments. Patients will receive standard care while consenting to have the AI run its diagnostic process in parallel. This study aims to provide compelling evidence on the scalability of these systems.

The timing of this release coincides with a broader industry push to define medical superintelligence. Microsoft views this as a move beyond simple benchmarking, aiming to integrate AI into the full arc of patient care. Suleyman expressed confidence that healthcare models will improve significantly within the next two to three years, driven by rapid iteration and real-world usage. He described the team as true believers in the potential of AI to solve persistent medical challenges.