Data-Free Pruning of Self-Attention Layers in LLMs

arXiv:2512.20636v2 Announce Type: replace-cross Abstract: Many self-attention sublayers in large language models (LLMs) can be removed with little to no loss. We attribute this to the Attention Suppression Hypothesis: during pre-training, some deep…

Read the full story at arXiv cs.AI ↗