Custom AI Chips vs GPUs: The Economics

Broadcom reported $10.8 billion of AI semiconductor revenue in its fiscal second quarter of 2026, up 143% from a year earlier, driven by custom AI accelerators and AI networking. Custom chips are no longer an experiment. They are a large commercial market.

Watch it play outCustom silicon is already a large commercial market

Broadcom sells custom AI accelerators and AI networking. A year earlier, that revenue filled this disc.

In fiscal Q2 2026 it reached $10.8 billion, up 143% from a year earlier.

Custom chips are no longer an experiment. They are a large commercial market.

Replay

That doesn't make this a winner-takes-all fight. GPUs stay the flexible default, while custom chips win specific, enormous, repeatable workloads where a platform can design the whole system around them. The real question is which system produces the result at the lowest total cost, with acceptable risk. Below, we work out what a "30% better" claim is worth in dollars, and how many chips it takes before a custom design pays for itself.

Watch it play out30% better performance per dollar cuts spend by about 23%

Hypothetical: a platform spends $10 billion on hardware for a fixed amount of work. Each dot is a slice of that spend.

A chip with 30% better performance per dollar does 1.3 times the work for each dollar.

So the same work costs $10bn ÷ 1.3, about $7.7 billion. The last 23 dots fade out.

The saving is about $2.3 billion: about 23% less spend, not 30%.

Replay

The short version

• Microsoft says its Maia 200 inference chip gives 30% better performance per dollar than the latest hardware in its existing fleet. It carries 216 GB of HBM3e at 7 TB per second.
• AWS says almost one million Trainium2 chips are involved in training and serving Claude through Anthropic's Project Rainier.
• Custom silicon makes the most sense for stable, high-volume inference. GPUs keep the edge where workloads change fast, such as frontier research.
• The likely future is a mixed fleet: merchant GPUs, hyperscaler chips, CPUs and several memory tiers run as one system.

Watch it play outEach chip wins its own work, then they run as one fleet

Custom chips win stable, high-volume inference, where a platform can design the whole system around the job.

GPUs keep the edge where workloads change fast, such as frontier research.

The likely future is a mixed fleet: merchant GPUs, hyperscaler chips, CPUs and several memory tiers.

All of it runs as one system, so the question is which mix delivers the result at the lowest total cost.

Replay

Check the claimThree claims to test as you read

ClaimCustom chips will replace GPUs.Tap to checkMythExpect a mixed fleet. Microsoft deploys Maia alongside NVIDIA and AMD, and GPUs remain the flexible default.Flip backClaim30% better performance per dollar means a 30% saving.Tap to checkMythOn a fixed amount of work it's about 23% less spend, as the worked example shows.Flip backClaimCustom chips are only for inference.Tap to checkPartly trueStable inference is their strongest case, but custom chips can train models too: Trainium was built for training.Flip back

What's the difference between a GPU and a custom AI chip?

An AI accelerator is any processor built to speed up machine-learning work. A modern GPU already is one, but it's sold as a broad, programmable platform. A custom accelerator (often called an ASIC, a chip built for one purpose) narrows the design around selected workloads. It trades flexibility for better performance per dollar, power efficiency or control over supply.

Modern GPUs are themselves highly specialized. "Custom chip" usually means one designed around the workloads and infrastructure of a single platform or a small group of customers. A hyperscaler is one of those giant cloud platforms.

Why GPUs are hard to displace

• Software is part of the product. Developers need compilers, kernels, libraries, debuggers, orchestration and documentation. A chip with great specs underperforms if software can't keep it busy.
• Research keeps changing. Model architectures, data types, context lengths and training methods move fast. Flexible hardware is worth more when tomorrow's workload is unknown at design time.
• Merchant scale spreads the cost. A GPU vendor sells one architecture to clouds, AI labs, enterprises, researchers and governments. That scale pays for a fast roadmap and heavy software investment.
• Customers want portability. Cloud customers may prefer infrastructure that works like what they can rent elsewhere. A proprietary chip can have attractive economics but a smaller pool of compatible software and expertise.

Merchant scale funds the software that keeps GPUs busy

1. Merchant scale (Clouds, labs, enterprises) to Fast roadmap (Heavy software investment): Pays for
2. Fast roadmap (Heavy software investment) to Portable software (Keeps chips busy)
3. Portable software (Keeps chips busy) to Merchant scale (Clouds, labs, enterprises): More buyers

Why hyperscalers build their own anyway

At hyperscaler scale, even a modest gain in performance per dollar or per watt can be worth billions.

• Inference repeats itself. Inference, running a trained model to answer requests, does similar operations at enormous volume. Once a platform knows the model and the serving pattern, it can tune silicon, memory, networking and software together.
• Supply control has strategic value. Custom silicon adds another source of compute and more say over capacity. It still depends on foundries, memory, packaging and networking, but it reduces reliance on one merchant roadmap.
• The whole system can be co-designed. The edge may come from the full stack rather than the chip: model design, compiler, data types, memory hierarchy, interconnect, rack design, cooling and scheduler.

The edge may come from the full stack, not the chip

1. Model design and data types
2. Compiler and scheduler
3. Custom siliconOne layer of the co-design
4. Memory hierarchy and interconnect
5. Rack design and cooling

Tuned together

What the big platforms have shown

Sources: Microsoft's Maia 200 announcement, Google's TPU documentation, AWS's Trainium customer page and Broadcom's fiscal Q2 2026 results.

Training and inference want different chips

Training a new frontier model rewards flexibility, high precision, massive clusters and a mature ecosystem. The workload can change mid-project.

Inference rewards low cost, low latency, energy efficiency, memory capacity and predictable operations. A stable, high-volume service can justify deeper specialization.

Training and inference reward different designs

TrainingFrontier models

• Flexibility
• High precision
• Massive clusters
• Mature ecosystem
• Workload can change mid-project

Rewards flexibility

vs

InferenceStable, high-volume services

• Low cost
• Low latency
• Energy efficiency
• Memory capacity
• Predictable operations

Can justify specialization

The line isn't fixed. Custom chips can train models, and GPUs can serve inference efficiently. What shifts is how much each design choice is worth for a given job.

Total cost beats chip price

A fair comparison counts:

• Chip purchase or rental price
• High-bandwidth memory
• Networking and optical interconnect
• Power and cooling
• Software engineering and migration
• Utilization
• Time to deploy
• Reliability and support
• Model quality at the chosen precision
• The risk that the workload changes

A cheaper chip can cost more if porting software takes months or it sits idle. An expensive one can pay off if it reaches revenue sooner and stays busy.

Idle time can outweigh a cheaper chip

1. Chip purchase or rental price
2. Memory, networking and optics
3. Power and cooling
4. Software, migration and supportPorting can take months
5. Utilization and time to deployIdle chips erase the saving
6. Workload riskThe job may change

Total cost

Member research: The price of a chip reaches earnings slowly, as depreciation spread over its useful life. Issue 03 finds four hyperscalers put almost $4 into Q1 infrastructure for every $1 of depreciation, and asks where the margin test lands next.

Worked example: what "30% better" is worth, and when it pays back

Part 1: the saving. Hypothetical example: a platform spends $10 billion on hardware to handle a fixed amount of work. A chip with 30% better performance per dollar does 1.3 times the work for each dollar.

Make a guess

A chip does 1.3 times the work for each dollar. On a fixed amount of work, how much less do you spend?

Tap your guess

0–15%Your guess15–25%Your guess25–30%Your guess30–50%Your guess

01234567890123456789%Actual

Spot on.Close, a little low.Close, a little high.Way off, too low.Way off, too high. It was 23%.

$10bn ÷ 1.3 ≈ $7.7bn, a saving of about $2.3bn. Headline percentages overstate savings.

Actual23%

Headline performance-per-dollar gain30%

Part 2: the break-even. That saving has to repay the upfront bill for designing the chip and porting software. Hypothetical example: $1 billion of upfront cost, and each custom chip saves $5,000 a year against the GPU alternative over a four-year useful life.

Utilization and useful life decide the break-even

Pick the yearly saving per chip and its useful life

Saving per chip per year

$5,000$2,500

Useful life

4 years3 years

Chips needed to break even50,00066,667100,000133,333

50,000133,333

Base caseLower utilization

Three lessons:

• Headline percentages overstate savings. "30% better performance per dollar" means about 23% less spend on the same work.
• Custom silicon is a volume game. The fixed cost only fades when spread across a very large fleet. For a sense of hyperscale, AWS says almost one million Trainium2 chips are involved in training and serving Claude through Project Rainier.
• Utilization and useful life decide the answer. Halving the saving doubles the break-even volume, which is why idle chips are so costly.

Try it yourself

A 10-minute exercise with the free tools:

1. Open the AI silicon break-even monitor and start with the base preset. Note the break-even units and payback.
2. Cut utilization and useful life, then raise the failure probability. Watch how far break-even moves.
3. Switch to the bear preset (slow ramp) and compare.
4. Open the inference cost per million tokens calculator. Enter a cost per accelerator-hour, then cut it by 23%, the saving from Part 1, and see what happens to cost per million tokens.
5. Reread the Maia 200 announcement and note exactly which hardware Microsoft compares against.

You should now be able to answer: at what volume and utilization does a custom chip beat the GPU alternative, and how much does that answer depend on useful life?

Our read

Custom accelerators don't need to kill GPUs to become a huge market. They only need to win selected, enormous workloads where co-design and repetition produce better economics, and the Broadcom, Microsoft and AWS figures say that's already happening. GPUs remain the flexible default for many customers and research workloads.

What we'd watch:

• Performance per dollar on real workloads. Vendor peak specs aren't enough. Look for throughput, latency, model quality, utilization and total system cost.
• Outside customers. Internal use proves a hyperscaler can run the chip. External adoption tests whether other buyers like the software and economics.
• Merchant GPU demand. Custom silicon can grow fast while GPU demand also grows, if total compute expands faster than custom chips take share.
• Design concentration. Custom-chip suppliers can depend on a few giant programs. One slip, customer change or failed generation can dent revenue.
• Memory and networking attach. Custom-chip growth can lift demand for HBM, networking, packaging and power even as it reshuffles processor vendors.

The custom-silicon case would weaken if:

• Porting costs stay too high for outside users
• Model designs change faster than chip-development cycles
• Low utilization erases the cost advantage
• Merchant GPUs improve performance per dollar faster than custom designs
• Foundry, packaging or memory limits block scale
• A custom generation misses its schedule or required model quality

Track the fleet mix inside the 2026 hyperscaler AI capex tracker.

Questions readers ask

Will custom chips hurt GPU makers?

Not automatically. Custom silicon can grow fast while GPU demand also grows, as long as total compute expands faster than custom chips take share. Microsoft, for example, deploys Maia alongside NVIDIA and AMD hardware rather than replacing every GPU.

Why doesn't every company design its own AI chip?

The upfront design and software bill is large, and it only pays back across a very large fleet running stable workloads. The break-even example shows how quickly the required volume rises when utilization or useful life disappoints.

Is "30% better performance per dollar" a 30% saving?

No. On a fixed amount of work it's about 23% less spend, as Part 1 shows. It is also Microsoft's own comparison under selected internal conditions, not an independent benchmark.

Who makes money when hyperscalers build custom chips?

Design partners such as Broadcom, whose AI semiconductor revenue is driven by custom accelerators and AI networking. Custom chips also still need HBM, networking, packaging and power, so those suppliers can benefit whichever processor wins.

Go deeper with member research

The chip economics decide who wins share. These member reports follow the money to the suppliers:

• AMD’s server franchise can finance a second AI platform: What AMD’s CPU and accelerator evidence showed on February 4, 2026, kept separate from the partnerships that came later.
• Issue 05: NVIDIA Is Building the Credit Market for AI: How NVIDIA turns its roadmap into cheaper capital for customers, and why AI infrastructure could move into private credit, insurance balance sheets and securitized debt.

Limits of this data

• Vendor performance claims are workload-specific. Test them against independent results and each deployment's needs.
• The worked-example costs are hypothetical round numbers. They are not estimates for any real chip.