MoE models changed what "fits on my GPU" means, and most hardware advice hasn't caught up

AI Summary

Mixture of Experts (MoE) models have fundamentally altered GPU memory requirements compared to traditional dense models, as they only activate a subset of parameters during inference. Despite this significant shift in how model size relates to hardware needs, most current GPU purchasing guides and hardware recommendations still base their advice on outdated assumptions about dense model architectures.