UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
A unified study of LLM self-distillation that combines teacher agreement, EMA stabilization, contrastive learning, feature matching, and divergence clipping to improve adaptation without stronger external teachers.
May 7, 2026
