On 12 October 2026, Scott Guthrie (Executive Vice President, Cloud + AI at Microsoft) wrote in Microsoft’s Maia 300 announcement that Microsoft expects Maia 300 (its next custom chip for answering with AI, not for teaching the AI) to deliver 40% more tokens per megawatt than leading GPU-based systems. Read that sentence as an electricity sentence. It does not say the answers get 40% righter. A token (a small chunk of text, often a word or part of a word, that the model reads or writes one piece at a time) that is wrong costs the same watt (a rate of electricity use: a phone charger is on the order of 10 to 20 watts) to emit as a token that is right.
What is Maia 300? Training versus inference
A model (the finished program that continues text by guessing the next small piece) is not the chip. The chip is the kitchen. The model is the recipe. Training (the long, expensive job of adjusting the recipe until the guesses are useful) happens first, mostly once. Inference (using the finished recipe to produce an answer for a person) happens every time you press send. Maia is an inference chip. Guthrie calls the 300 an inference-first design. It is not a claim that Microsoft has a new way to teach models.
An accelerator (a chip built to do one heavy kind of math quickly) is the general word. A GPU (graphics processing unit: a chip first built to draw pictures, now also used for model math because it does a huge number of small multiplications at once) is the accelerator almost everyone already rents. Nvidia sells most of those. Maia is Microsoft’s attempt to stop renting the whole kitchen.
A watt is a speed of energy use, not a lump of energy. Leave a 1-watt bulb on for an hour and you have used 1 watt-hour. A kilowatt (kW) is 1,000 watts. A megawatt (MW) is 1,000 kilowatts, a million watts, roughly the draw of a few hundred homes at once. Tokens per megawatt and tokens per watt are the same ratio with a different ruler. Forty percent more of either means forty percent more pieces of answer per unit of electricity, if the claim holds. It does not mean forty percent less waiting, and it does not mean a better sentence.
Nadella’s post the same day drops a word Guthrie kept. Guthrie wrote “we expect” 40% more tokens per megawatt. Nadella wrote that Maia 300 “delivers” 40% more tokens per watt. Same ratio, harder tense. The blog post is the one with the hedge. Treat “expect” as the careful sentence and “delivers” as the CEO’s shorter one. Neither post names the GPU system being beaten, and neither posts a method you can rerun.
Why memory bandwidth matters more than raw math for Maia 300
When you send a long prompt (the text you typed, plus the instructions the app added), the chip first reads all of it. That read is called prefill (the gulp: take in the prompt before any new word comes out). Then the model writes the answer one token at a time. That writing is called decode (the drip: one new token, then another, each one depending on the last). Agents (a model put in a loop that is allowed to take several steps, not just answer once) live in the drip. A coding agent that thinks for a minute is decoding for a minute.
Each new token has to remember what came before. The usual trick is a KV cache (the model’s running notebook of this conversation, so it does not re-read the whole prompt from scratch for every new word). A long document or a long agent trace makes that notebook huge. The chip then spends its time moving the notebook in and out of memory, not multiplying. That is what “memory-bound” means: the bottleneck is the hallway between the math and the memory, not the speed of the math. Guthrie says Maia 300 is aimed at long-context (a long prompt or a long notebook), memory-intensive, decode-heavy work. The design bet follows from that hallway, not from a slogan.
HBM (high bandwidth memory: memory chips stacked close to the processor, so data does not have to cross a whole motherboard) is the hallway Microsoft is widening. The 12 October post says eight HBM4 stacks give 18.0 TB/s of peak memory bandwidth inside a 950-watt TDP. TB/s means terabytes per second (a terabyte is about a thousand gigabytes; per second is the speed, not the size of the tank). TDP (thermal design power: the heat and power budget the chip is built around) is not a measurement of one prompt. A 950-watt budget is a space heater sitting in a cabinet. Maia 200, described in Guthrie’s 26 January 2026 Maia 200 post, was a 750-watt budget with 216 GB of HBM3e at 7 TB/s. The new hallway is wider. The new heater is also hotter. Bandwidth per watt, using those two pairs of numbers, is about twice Maia 200’s. That ratio is simple division of Microsoft’s own numbers, not a sentence Microsoft printed.
Maia 300 chiplets, TSMC N2P, SRAM, and 4-bit math
Printing every job onto one giant chip is like carving a whole kitchen out of one slab. One crack ruins the slab. A chiplet (a smaller chip that does one job, then gets bonded into the same package as the other small chips) lets the stove, the fridge, and the door be made separately. Guthrie says Maia 300 separates three jobs: compute (the math), the memory controller (the doorman that talks to the HBM), and I/O (input/output: the doors that talk to the rest of the rack). The compute chiplets use TSMC’s N2P process. TSMC (Taiwan Semiconductor Manufacturing Company) is the factory. N2P is its name for a 2-nanometer-class generation. A nanometer here is a generation name, not a ruler you can lay on a transistor and read off. The October post does not say which process the memory and I/O chiplets use.
Maia 200, from the January post, was one design on TSMC’s 3nm process, with 272 MB of on-chip SRAM (a small, very fast scratchpad sitting on the chip itself, much closer than HBM, and much smaller). The October post says Maia 300 also uses on-die SRAM as a software-managed scratchpad, and does not say how many megabytes. Do not carry 272 MB forward. It was last generation’s number.
Low precision is the other memory trick, and it is easy to hear as “dumber math.” A weight (one stored number inside the model) can be written with many bits (a bit is a single 0 or 1) or with few. FP8 and FP4 mean roughly 8-bit and 4-bit numbers. Fewer bits means a smaller notebook and a narrower trip down the hallway, at some cost in accuracy that the model makers try to absorb. MXFP4 is Microsoft’s name, in this post, for a 4-bit format they say shrinks both the math and the KV cache. The January Maia 200 post claimed over 10 petaFLOPS of FP4 and over 5 petaFLOPS of FP8. A petaFLOP is a thousand trillion simple arithmetic steps per second, a brag about the stove, not about the food. The 12 October post does not give a FLOPS number for Maia 300. Anyone who quotes one today got it from somewhere else.
The Maia 300 rack: 96 chips, 167 kW, and one gap
A rack (a tall cabinet in a data center) of Maia 300, Guthrie says, holds 96 accelerators inside a 167-kilowatt power envelope, with 27.6 TB of HBM capacity and 1.7 PB/s of aggregate HBM bandwidth. A petabyte is about a thousand terabytes. Check the bandwidth before you applaud it.
Ninety-six times 18.0 TB/s is 1,728 TB/s. If a petabyte is 1,000 terabytes, that is 1.73 PB/s, which is the “1.7 PB/s” in the post. The rack’s bandwidth is the chip’s bandwidth times 96. It is not a second miracle.
Ninety-six times 950 watts is 91,200 watts, which is 91.2 kilowatts. The rack envelope is 167 kilowatts. The difference is about 76 kilowatts. The post does not say what that 76 kilowatts is. Cooling, power conversion, and the network are the usual residents of that gap. The post does not assign them. The point is that 167 is not 96 times 950, so you cannot treat the rack number as “the chips, and nothing else.”
Twenty-seven point six terabytes divided by 96 is 0.2875 terabytes, about 288 GB, if the memory is split evenly and no capacity is lost to spare parts. Microsoft states the rack total, not a per-chip capacity. Maia 200’s stated per-chip memory was 216 GB. If the even split is fair, Maia 300 holds about a third more memory per chip, not several times more. The thing that more than doubled is the speed of the hallway (7 TB/s to 18 TB/s), not the size of the tank.
A bigger cabinet is not a more efficient cabinet. Efficiency, in the sentence Microsoft wants quoted, is tokens out per watt in. A 167 kW rack can be the better deal and still be the hungrier object in the room. “More tokens per megawatt” can be true while the rack draws more power than the last rack. Those are different measurements. Headlines that turn the first into “uses less power” are changing the subject.
Maia 200 today versus Maia 300 in 2027
Maia 200 is the chip you can already be talking to without knowing it. Guthrie says it is in production, serving Microsoft workloads including Microsoft Copilot, and supporting the MAI team (Microsoft’s model group) while they build next models. It runs OpenAI models, Microsoft’s own MAI models, and what he calls leading open-source families. GitHub Copilot and Microsoft Foundry (Microsoft’s service for running models) “follow by end of year.” That is not the same as “they are on Maia today.” Some writeups flatten those into one list. The blog does not.
The same post says Maia 200 is deploying in the United States, Europe, and Australia, and is delivering up to 1.7 times more tokens per watt for built-in AI in Teams, Copilot, and Cowork, compared with leading GPU infrastructure. “Up to” is the hedge. The comparison GPU is not named here either. This is still the nearer claim, because this chip is in production. Maia 300 is not.
| Claim | Whose words | When it applies |
|---|---|---|
| Maia 200 serves production Copilot | Guthrie, 12 October, stated as fact | Now |
| Up to 1.7x tokens per watt in Teams, Copilot, Cowork | Guthrie, “up to”, GPU not named | Maia 200, now, on those experiences |
| GitHub Copilot and Foundry on Maia | Guthrie, “to follow by end of year” | Not yet, on this wording |
| 40% more tokens per megawatt than leading GPUs | Guthrie, “we expect”. Nadella says “delivers” | Maia 300, and no method is published in the post |
| Up to 7x the rack-scale token throughput of Maia 200 | Guthrie, a full 96-accelerator configuration | Throughput, not quality. Production in 2027 |
| Primary inference system for Microsoft’s own products | Guthrie and Nadella | Multi-gigawatt deployments starting in 2027 |
Seven times is the number that will get stripped of its noun. The noun is rack-scale token throughput (how many tokens a cabinet emits per second) of Maia 200, in a full 96-accelerator Maia 300 configuration. It is not seven times smarter, and it is not seven times Maia 200’s tokens per watt. A cabinet can emit more tokens because it holds more chips, because each chip is faster, or because both. The post does not split the 7x. Maia 200’s rack size is not restated here, so you cannot reverse the 7x into a per-chip speedup with the numbers on this page.
“Multi-gigawatt” is a size of power plant, not a count of chips. One gigawatt is a thousand megawatts. A plan to scale to multi-gigawatt deployments starting in 2027 is a plan about electricity and buildings. Reuters, the same day, reported that Microsoft plans to deploy potentially more than a million Maia 300 chips next year. That count is not in Guthrie’s post. “Potentially” is Reuters’ word. It is not a specification.
The same wire story compared a Maia rack’s 167 kilowatts and 96 chips with an Nvidia GB300 rack it described as 142 kilowatts and 72 GPUs. If those two pairs are right, the Maia cabinet draws more power, not less, and it packs only modestly more accelerators per kilowatt (96/167 against 72/142). That is not where a 40% token claim would come from. The 40% has to be tokens out of each accelerator, and Microsoft has not shown the worksheet.
What Maia 300 does not change for your laptop or Nvidia
Maia 300 does not ship in a PC. There is no driver you install. Production deployments begin in 2027, inside Microsoft’s cloud. The January chip, Maia 200, is the one already under some Copilot replies. You will not see a setting called Maia.
Guthrie’s close is easy to skip and it matters. He says the platform is open, heterogeneous (more than one kind of chip), and model-diverse, and that customers should be able to count on it. “Primary inference system for Microsoft products and services” is about Microsoft’s own products. It is not a sentence that Azure GPU virtual machines (a rented slice of a computer in the cloud) disappear, or that Nvidia is out of the building. Nadella’s post says the goal includes “continuing to bring the latest innovations from across the ecosystem into our fleet.” A fleet with a favorite chip is still a fleet.
The Verge reported Mustafa Suleyman (CEO of Microsoft AI) saying they have to work “all the way down to the silicon” and that running Microsoft AI models on Maia is a critical focus. That is a stack argument (owning the layers from the model down to the chip so they can be designed together). It is not a benchmark.
A cage around an agent and a chip under an agent solve different problems. The cage is which files and network the agent may touch. That is the subject of Microsoft Execution Containers is not Docker. Maia does not lock those doors. It tries to make the drip of tokens cheaper in electricity. Buying one does not give you the other.
What to watch with Maia 300
If you pay per token, the number that reaches you is the price on the bill and the seconds you wait, not the name on the lid. A 40% gain in tokens per watt can be kept by Microsoft, handed to you as a lower price, or spent on longer agent traces that use the savings up. The post does not say which. “Economics of expanding AI services” is a company sentence.
If you run long-context agents, the relevant design claim is the hallway: HBM bandwidth, a scratchpad, and a smaller KV cache through 4-bit numbers. That claim is coherent. It is still a vendor’s description of a chip that starts production in 2027. The number worth putting in a spreadsheet today is Maia 200’s “up to 1.7x tokens per watt” on Teams, Copilot, and Cowork, with “up to” and “GPU not named” written in the cell next to it.
Do not repeat “7x” without “rack-scale token throughput.” Do not repeat “40%” without “tokens per watt” and “we expect.” Do not tell anyone the answers got better. The post never said they did.
Common questions about Maia 300
Is Maia 300 a GPU?
No. Maia 300 is Microsoft’s custom accelerator for inference, meaning it runs finished models to produce answers rather than training them. Guthrie calls it an inference-first design.
What does 40% more tokens per megawatt mean for Maia 300?
It is an electricity claim. Microsoft expects Maia 300 to produce 40% more pieces of answer per unit of power than leading GPU-based systems. It does not mean answers are more accurate or arrive faster, and the post names no GPU system and publishes no method.
What are the Maia 300 memory specs?
Eight HBM4 stacks give 18.0 TB/s of peak memory bandwidth inside a 950-watt TDP. A full rack holds 96 accelerators in a 167-kilowatt envelope with 27.6 TB of HBM and 1.7 PB/s of aggregate bandwidth.
When will Maia 300 be available?
Production deployments begin in 2027 inside Microsoft’s cloud, along with the plan for multi-gigawatt deployments. Maia 200 is the chip already serving Microsoft Copilot today.
Can you buy Maia 300 for a PC?
No. Maia 300 does not ship in a PC and there is no driver to install. It runs inside Microsoft’s data centers.