The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
arXiv:2609.18063v1 Announce Type: new Abstract: Mixture-of-experts (MoE) inference on consumer hardware is bounded by weight memory: a 35B-class model is 19.5GB at 4-bit, and sparsity shrinks the compute per token, not the bytes that must be held…
Read the full story at arXiv cs.AI ↗
ImpactNotable 31/100
Why it mattersRule-based estimate: event keywords (+4); trust 6/10.
RegionsGlobal
Published3 d ago (Thu, 17 Sep 2026 04:00:00 GMT)
RetrievedThu, 17 Sep 2026 15:30:40 GMT via rss
ClassifiedThu, 17 Sep 2026 15:30:51 GMT by heuristic
AuthorYu Lin, Yiming Wang, Runyuan Cai, Hanze Liu, Xiaodong Zeng