UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
UniSD unifies the fragmented landscape of self-distillation for large language models, providing a principled framework that supports systematic comparison and new combinations across data, representation, and decoding levels.
May 30, 2026
UniSD
A unified self-distillation framework for large language models that consolidates fragmented self-distillation directions into a single, modular formulation across data, representation, and decoding levels. Under review at NeurIPS 2026.
May 30, 2026