基于知识蒸馏的轻量化睡眠分期模型研究

Research on a Lightweight Sleep Staging Model Based on Knowledge Distillation

摘要: 面向大规模睡眠模型难以在资源受限环境部署以及轻量模型特征表示能力不足的问题，本文提出一种基于知识蒸馏的轻量化睡眠分期方法，实现大规模睡眠模型向轻量网络的知识迁移。该方法以SleepGPT作为教师模型，将教师模型输出的软标签知识迁移至轻量学生网络SpectralContextSleepNet，通过教师软目标指导学生模型训练，在降低模型复杂度的同时提升轻量模型的睡眠阶段判别能力。本文在Sleep-EDF SC数据集上开展实验，从睡眠分期性能和模型复杂度两个方面对所提出方法进行评估。实验结果表明，基于知识蒸馏训练的轻量学生模型相比直接训练的轻量学生模型获得更优的睡眠分期性能，并在保持有效分类能力的基础上显著降低模型计算和存储开销。其中，学生模型包含1,048,725个参数，相比教师模型参数量降低约130倍，表明知识蒸馏在大规模睡眠模型轻量化迁移中具有应用潜力。本文方法通过利用大规模睡眠模型中的软标签知识增强轻量网络分类能力，在模型性能与计算效率之间取得较好的平衡，为构建高效、轻量化自动睡眠分期模型提供了一种可行方案。

Abstract: Large-scale sleep models face challenges in deployment under resource-constrained environments due to their high computational and storage costs, while lightweight models often suffer from limited feature representation capability. To address these issues, this paper proposes a knowledge distillation-based lightweight sleep staging method, which transfers knowledge from a large-scale sleep model to a lightweight network. Specifically, SleepGPT is employed as the teacher model, and the soft-label knowledge generated by the teacher model is transferred to the lightweight student network SpectralContextSleepNet. By optimizing the student model with teacher-guided soft targets, the proposed method improves the sleep-stage discrimination capability of the light-weight model while reducing model complexity. Experiments are conducted on the Sleep-EDF SC dataset to evaluate the proposed method from both sleep staging performance and model complexity perspectives. Experimental results demonstrate that the distilled lightweight student model achieves better sleep staging performance than the directly trained lightweight model while significantly reducing computational and storage costs. The student model contains 1,048,725 parameters, achieving approximately 130× parameter reduction compared with the teacher model, indicating the potential of knowledge distillation for lightweight migration of large-scale sleep models. The proposed method enhances the classification capability of lightweight networks by leveraging soft-label knowledge from large-scale sleep models, achieving a favorable balance between sleep staging performance and computational efficiency. It provides a feasible approach for developing efficient and lightweight automatic sleep staging models.