How the study was run

200 real visits. One physician plan. 24 AI plans.

Researchers took 200 de-identified outpatient and emergency-department notes written by practising US clinicians, and cut each one at the moment before the physician's assessment.

The treating physician's documented plan was priced at 2026 Medicare rates: every test, every specialist referral. Then 24 AI systems received the same presentation, wrote their own plan, and were priced the same way.

The live dashboard has since added 10 systems released after the study, run on the identical pipeline: 34 in all.

200 real visits · NEJM AI, October 2026

What the physician ordered.

Outpatient and emergency visits. Each treating physician's documented plan, priced at 2026 Medicare rates: tests plus specialist referrals.

In most visits, nothing was ordered at all. Median: $0.

The same 200 patients

Read by 34 AI systems.

Each system saw the presentation and wrote its own plan. Every one of them ordered more than the physician.

Not some. All 34. From 2.1× to 6.1×.

The headline excess

Per visit, on average.

$314 against $97: 3.2× the physician, for the 20 general-purpose systems in the published panel.

The ten systems released after the panel closed: $360, 3.7×.

Across successive versions

Newer is not cheaper.

Follow each family from its oldest to its newest model. Some families came down. Most did not.

Claude +$209 · GPT +$90 · Qwen +$83 · Gemini −$51 · Grok −$192.

Every version update is a new, unmeasured system.

What the money is

Tests, and above all referrals.

Specialist referrals per visit: physician 0.15, AI 0.87. Diagnostic testing: $71 against $176.

An unnecessary blood panel costs dollars. An unnecessary referral costs weeks of someone's queue.

Exploratory · not a result of the paper

Does spending more buy agreement?

Agreement with the physician's diagnosis against cost, with each family followed from its oldest to its newest model.

Claude runs right: about $209 more a visit for nine points of agreement. Gemini and Grok ran the other way: cheaper and more concordant. GPT barely moved.

Across all 34 systems the trend is flat, r = 0.13. Accuracy and cost are separate axes.

Mitigation · 19 systems × 3 prompts

Can one sentence fix it?

A parsimonious-but-safe instruction moves every system left. Some land below the physician, ordering too little. A cost-aware one-liner made two medical models more expensive.

Prompting is a guardrail, not a cure. The cheapest operating point is not the safest.

Before you deploy

Ask for the bill.

Benchmark any model, on your own case mix, before purchase and after every version. Pinpoint where the excess comes from. Train and monitor the safe operating point.

MedBAR.ai · explore the measures

Clinical AI, benchmarked on the trade-offs that matter.

Switch the measure above the chart: total cost, diagnostic tests, specialist referrals, new medications. The 34 systems re-rank; the physician's line moves with them.

Cost, resources, risk, time, invasiveness. Measured on real cases, against real physicians, with every order priced.