AI Model-to-HBM Demand Calculator
What it calculates
Model weights are only one part of inference memory. This calculator adds KV cache, context, concurrency, and runtime overhead, tests the result against manufacturer-published accelerator capacity and bandwidth, and translates a deployment scenario into total HBM and stack demand. It is a physical exposure model, not a supplier revenue or shipment forecast.
How this ai model-to-hbm demand calculator works
The calculator combines model weights, KV cache, concurrent sequences, and runtime overhead, then compares the requirement with a selected accelerator's manufacturer-rated HBM capacity. It separates a capacity fit calculation from the theoretical bandwidth specification and scales the result across a deployment.
Formula
Required HBM = model weights + KV cache + runtime overhead; minimum accelerators = ceiling(required HBM ÷ usable HBM per accelerator)
This is a physical-capacity scenario, not an HBM shipment, supplier-share, pricing, or revenue forecast. Model implementations can use architectures and memory-management techniques that differ from the simplified inputs.
Before you use the result
Assumptions
- • Weight precision and KV-cache bytes are applied uniformly across the selected model scenario.
- • The selected runtime reserve approximates activations, fragmentation, communication buffers, and framework overhead.
- • Published peak bandwidth is theoretical; application throughput depends on kernels, interconnect, batching, and utilization.
Quick start
- 1. Enter the model architecture, precision, context, and concurrency assumptions.
- 2. Choose the accelerator or enter a custom HBM capacity and bandwidth.
- 3. Scale the deployment and test how quantization or context changes physical HBM demand.
Inputs and units
| Input | Unit | Default |
|---|---|---|
| Model parameters | billions | 70B |
| Bytes per parameter | bytes (2 = FP16/BF16) | 2 |
| Context length | tokens per sequence | 32,768 |
| Concurrent sequences | count | 16 |
| Layers, KV heads, head dimension | model architecture | 80 / 8 / 128 |
| Runtime overhead | percent of weights + KV cache | 15% |
| Accelerator HBM capacity | gigabytes | 141 GB |
| Accelerator bandwidth | terabytes per second | 4.8 TB/s |
Worked example (hypothetical)
AI Model-to-HBM Demand Calculator: worked example
Hypothetical 70B-parameter model served at 16-bit precision with the calculator's default architecture inputs.
| Parameters × bytes | 70B × 2 |
|---|---|
| Context × concurrency | 32,768 tokens × 16 |
| HBM per accelerator | 141 GB |
| Model weights | 140 GB |
|---|---|
| KV cache | 171.8 GB |
| Working set incl. overhead | 358.6 GB |
| Minimum accelerators | 3 |
At long context the KV cache (172 GB) exceeds the weights (140 GB), so memory capacity, not parameter count alone, sets the accelerator count.
How to interpret the result
The capacity result tells you how many accelerators are needed to hold the working set. The bandwidth estimate is separate: a deployment can fit in memory and still be limited by how fast weights and cache can be read.
Frequently asked questions
What does the model-to-HBM calculator measure?
It estimates model weights, KV cache, and runtime overhead, then calculates the minimum accelerator count and total HBM capacity required for the selected deployment scenario.
Why does context length increase HBM demand?
During inference, the KV cache stores attention state for tokens in each active sequence. Longer context and higher concurrency can make that cache large enough to exceed the model weights themselves.
Does fitting in HBM guarantee target inference speed?
No. Capacity fit and throughput are different constraints. Published bandwidth is theoretical, while realized performance depends on kernels, batching, interconnect, quantization, model architecture, and utilization.
How are HBM stacks estimated?
The tool divides required HBM capacity by an editable capacity-per-stack assumption and rounds up to whole stacks. Actual package layouts, stack densities, yields, and supplier allocation vary by accelerator.
Does deployment HBM equal annual supplier shipments or revenue?
No. Deployment capacity is a physical scenario. Shipment timing, inventory, replacement cycles, yields, pricing, stack configuration, qualification, packaging, and supplier share require separate evidence.
More calculators
The math is easy. The macro is hard.
AI Model-to-HBM Demand Calculator shows the arithmetic. Members get the monthly call that decides which inputs matter, plus written breakpoints when the facts move. $39/month or $249/year.
Members-only research
$39/mo or $249/yr · cancel future renewals anytime · sources included