https://ahren09.github.io/tags/self-distillation/Self-DistillationHugo Blox Builder (https://hugoblox.com)en-usSat, 30 May 2026 00:00:00 +0000https://ahren09.github.io/media/icon_hu_eee6347cbdb2cc3f.pnghttps://ahren09.github.io/tags/self-distillation/https://ahren09.github.io/publication/neurips26_unisd/Sat, 30 May 2026 00:00:00 +0000https://ahren09.github.io/publication/neurips26_unisd/<h2 id="abstract">Abstract</h2> <p>Self-distillation has emerged as a powerful technique for improving large language models without external teacher signals, but existing approaches are fragmented across diverse objectives, training signals, and model components. We introduce UniSD, a unified self-distillation framework that consolidates these directions into a single, modular formulation. UniSD enables systematic comparison of self-distillation variants and supports new combinations across data, representation, and decoding levels.</p> <h2 id="links">Links</h2> <ul> <li> </li> </ul>