Plan models how your workload would run on each GPU and cloud instance you're considering, using your model, your latency target and the settings Optimize would use. You get throughput, latency and cost for each option, so the hardware decision is based on numbers.
You describe the workload once: the model, prompt and output lengths, arrival rate, expected growth and latency target. Plan simulates the serving engine on each GPU and cloud instance and shows throughput, tail latency, power and cost in one table.
The same first-order model Plan starts from: decode limited by memory bandwidth, prefill by compute, tensor-parallel traffic over the real link. Plan itself runs a full simulation of the traffic.
Covers H100 to B200, MI300X, L40S, consumer cards and edge modules, including the details that affect results, such as NVLink, memory size, FP8 and FP4 support and interconnect bandwidth.
Add seasonality, launches and growth, and Plan shows when you would run out of capacity.
Forecasts are compared with Optimize runs on real hardware, and the error is included in the report.
NVIDIA H100, H200, B200, GB200, L40S, L4 · AMD MI300X, MI325X · AWS, Google Cloud, Azure, CoreWeave, Lambda and on-prem price books · vLLM, SGLang and TensorRT-LLM performance models.
No. It simulates the serving engine, scheduler and KV cache against your traffic, using performance models calibrated on measured runs. You can export the results to a spreadsheet.
Yes. Each option is simulated with the settings Optimize would use on that hardware, so you aren't sizing for an untuned setup.
Yes. Edge uses the same performance models, with memory, bandwidth and power per device, to find the largest model that meets your accuracy threshold.
Send us the model, the hardware and your target, whether that's tokens per second, latency, memory or cost. In a 30-minute call we'll tell you whether we think it's doable and roughly what it would take.
