LLM Serving Optimization with Variable Prefill and Decode Lengths

arXiv:2508.06133v5 Announce Type: replace-cross Abstract: We study offline scheduling for large language model (LLM) serving under a fixed KV-cache memory budget, where requests have heterogeneous prompt (prefill) and response (decode) lengths…

Read the full story at arXiv cs.AI ↗