HBM and Memory Planning

conceptstub

Sizing models, KV cache, and activations against HBM.

Planned content

• Training memory: weights, grads, optimizer, activations
• Inference memory: weights + KV cache
• Worked sizing examples