Semiconductors / AI

🔥 Nvidia's B200 Costs 2.3x More Than AMD's MI355X to Serve the Same AI Model

A SemiAnalysis benchmark says serving Kimi K3 costs 2.3x more per token on Nvidia's B200 than on AMD's MI355X — with a big catch.

By Tech Arcade··5 min read

Nvidia's B200 Costs 2.3x More Than AMD's MI355X to Serve the Same AI Model — Tech Arcade
Photo: Florian Krumm / Unsplash

For two years, the only question in AI hardware was how fast Nvidia could ship. Here’s a newer one: what does it actually cost you to serve a token once the chip is bolted into a rack?

A benchmark published this week by the research firm SemiAnalysis puts a hard number on it. Running the Kimi K3 2.8T model at a brisk 100 tokens per second per user, Nvidia’s B200 costs $0.179 per million tokens. AMD’s MI355X does the same work for $0.077 (SemiAnalysis InferenceX, via FourWeekMBA). That’s the Nvidia part costing about 2.3 times as much to push out identical output.

The number lands in the middle of the most lopsided chip race in tech, where AMD has spent years playing a distant second. And it points at a shift in how buyers are starting to shop: not by raw speed, but by dollars per token served.

🧠 Why This Matters

The gap doesn’t come from where you’d expect. AMD’s chip isn’t dramatically cheaper to rent. On SemiAnalysis’s July 2026 pricing model, a B200 runs $1.73 per chip-hour and the MI355X runs $1.50 — Nvidia is only about 15% pricier by the hour (FourWeekMBA).

The real difference is throughput. At that 100-tokens-per-second setting, a single MI355X chews through 5,435.8 tokens per second. A B200 manages 2,677.5 — roughly half (FourWeekMBA). When one chip serves twice the tokens for nearly the same rent, the cost-per-token math tips hard.

For anyone running inference at scale — the part of AI that actually bills customers — a 2x swing in unit cost is the difference between a product that makes money and one that quietly bleeds.

📊 Deep Dive

The Kimi K3 test ran at FP4 precision inside SemiAnalysis’s “AgentX” scenario, which replays real coding-agent sessions rather than a synthetic prompt. Across three interactivity settings, the MI355X stayed ahead:

  • 57 tokens/sec per user: B200 $0.089 vs MI355X $0.046 per million tokens — about 1.9x
  • 100 tokens/sec per user: B200 $0.179 vs MI355X $0.077 — about 2.3x
  • 144 tokens/sec per user: B200 $0.275 vs MI355X $0.146 — about 1.9x

A second comparison, pitting the MI355X against Nvidia’s rack-scale GB200 NVL72, tells a gentler version of the same story: the AMD part comes in 46% cheaper per million tokens at the highest interactivity level, $0.15 against $0.22, narrowing to a 20% edge in the mid-range (AI Weekly).

And it isn’t just Kimi K3. On a separate run with the GLM-5 model using the SGLang framework at FP8, the MI355X came out up to 29.6% cheaper than the B200 (SemiAnalysis InferenceX). Two models, two frameworks, the same direction of travel.

Worth noting the models being served here — Kimi K3 from Moonshot AI and GLM-5 from Zhipu AI — are both built by Chinese labs, and both are open-weight, which is exactly why independent firms can benchmark them on competing hardware in the first place.

⚠️ The Catch

Before you rip out the Nvidia racks, read the fine print, because there’s a lot of it.

First, the win has a ceiling. On the GLM-5 run, SemiAnalysis is blunt about where the AMD advantage stops:

MI355X MTP remains cheaper up to the crossover near 89.13 tok/s/user, after which B200 is cheaper. — SemiAnalysis, InferenceX

Push past roughly 90 tokens per second per user and the lead flips back to Nvidia. The cheaper option depends entirely on how fast you need each user served.

Second, these are modeled costs, not receipts. The dollar figures lean on SemiAnalysis’s own pricing surveys and TCO assumptions, interpolated from benchmark runs — not prices any buyer actually paid. AI Weekly flagged that the methodology is “one vendor’s model,” and that the stated percentages don’t even line up perfectly with the listed prices (AI Weekly).

Third, there’s a density tax. At the highest interactivity setting, the MI355X served roughly 4 users per chip where the Nvidia rack served about 8 (AI Weekly). Cheaper per token, but you may need more chips to serve the same crowd.

And SemiAnalysis itself scopes the GLM-5 result tightly:

This result is single-node, aggregated, FP8 only. — SemiAnalysis, InferenceX

One model family. One precision. Inference only — none of this touches training, where Nvidia’s lead remains uncontested.

🎯 What Happens Next

The scoreboard that matters is procurement. Hyperscalers and AI labs sign chip orders quarters in advance, and the people writing those purchase orders now have a credible third-party number suggesting AMD can undercut Nvidia on serving cost for certain workloads. Expect AMD to wave this benchmark in every sales meeting between now and its next earnings call.

Nvidia’s counter is already shipping: the B300, priced at $2.26 per chip-hour in the same survey, plus a software stack that most inference teams already run in production. The question buyers will actually test is whether AMD’s ROCm software can match that reliability at scale — because a cost win on a benchmark means nothing if the chips are harder to deploy.

Watch the next few weeks for independent labs trying to reproduce these runs on hardware they control. That’s when “SemiAnalysis says” becomes “we measured.”

🧩 Bigger Picture

The AI hardware market has spent two years measuring everything in raw performance and GPU allocations. This benchmark is a marker that the conversation is moving to the boring, decisive metric: cost per unit of useful output. That’s the number that decides whether an AI feature survives a budget review.

AMD doesn’t have to beat Nvidia everywhere to change the market. It just has to be credibly cheaper somewhere that matters — and inference, the part that runs every time a user hits enter, matters enormously. A genuine second source puts pricing pressure on the whole stack, which is good news for anyone paying an AI bill and uncomfortable news for the company that’s enjoyed a near-monopoly.

The caveats are real and the benchmark is one firm’s model. But the direction is hard to ignore: for the first time in this cycle, the cheapest way to serve a token might not have a green logo on it.

In AI, the fastest chip wins the headline. The cheapest chip wins the invoice — and invoices compound.

Sources

❓ Quick answers

Is AMD's MI355X cheaper than Nvidia's B200 for AI inference?

On SemiAnalysis's InferenceX benchmark published around Oct 31-Nov 1, 2026, serving Kimi K3 2.8T at 100 tokens/sec per user cost $0.077 per million tokens on the MI355X vs $0.179 on the B200 — about 2.3x cheaper. It's inference-only and based on one firm's modeled costs.

Why is the MI355X cheaper per token if it rents for a similar price?

Throughput. At 100 tokens/sec per user the MI355X served about 5,436 tokens/sec per chip versus 2,678 for the B200, while costing only ~15% less per chip-hour ($1.50 vs $1.73), so the per-token math tips toward AMD.

Does AMD win at every speed?

No. On the GLM-5 test, SemiAnalysis found the MI355X stays cheaper only up to a crossover near 89 tokens/sec per user, after which the B200 becomes cheaper. The result is single-node, FP8-only, and inference-only, not training.