基于DeBERTaV3的Token表达与联邦学习的高校图书馆推荐系统
Federated Recommendation System for University Libraries Based on DeBERTaV3 Token Representation
DOI: 10.12677/mos.2025.149587, PDF,    国家自然科学基金支持
作者: 赵 霞:上海理工大学图书馆,上海;鄢 源:上海理工大学光电信息与计算机工程学院,上海
关键词: 联邦学习Token-Based TransformerDeBERTaV3Top-N推荐Federated Learning Token-Based Transformer DeBERTaV3 Top-N Recommendation
摘要: 为提升高校图书馆推荐系统的个性化水平并保护用户隐私,本文提出一种融合联邦学习与Token级Transformer模型的图书推荐方法。该方法以Amazon Kindle Store数据集为基础,模拟多个高校图书馆作为联邦客户端,每个客户端本地训练模型,无需上传原始用户数据,从而有效保护读者隐私。在模型设计方面,本文采用DeBERTaV3作为Token-based Transformer骨干网络,将图书的标题与评论文本编码为多层次语义Token,实现对用户阅读行为和兴趣的深层建模。通过局部训练与中心聚合的联邦机制,构建出具备全局泛化能力的推荐模型。实验在Top-N推荐准确率、隐私保护强度与跨客户端迁移效果等维度进行了系统评估,结果表明所提出方法在保持推荐性能的同时兼顾数据安全性,适用于多校园、多用户异构环境下的个性化推荐任务。
Abstract: To enhance the personalization of university library recommendation systems while preserving user privacy, this paper proposes a book recommendation method that integrates federated learning with a Token-level Transformer model. Based on the Amazon Kindle Store dataset, the method simulates multiple university libraries as federated clients, where each client trains a local model without uploading raw user data, thus effectively safeguarding reader privacy. In terms of model architecture, DeBERTaV3 is adopted as the Token-based Transformer backbone to encode book titles and review texts into multi-level semantic tokens, enabling deep modeling of user reading behaviors and preferences. By combining local training with centralized aggregation through the federated mechanism, a globally generalizable recommendation model is constructed. Extensive experiments are conducted on Top-N recommendation accuracy, privacy preservation strength, and cross-client generalization. The results demonstrate that the proposed method achieves strong recommendation performance while maintaining data security, making it well-suited for personalized recommendation tasks in multi-campus, multi-user heterogeneous environments.
文章引用:赵霞, 鄢源. 基于DeBERTaV3的Token表达与联邦学习的高校图书馆推荐系统[J]. 建模与仿真, 2025, 14(9): 95-104. https://doi.org/10.12677/mos.2025.149587

参考文献

[1] He, X., Liao, L., Zhang, H., Nie, L., Hu, X. and Chua, T. (2017) Neural Collaborative Filtering. Proceedings of the 26th International Conference on World Wide Web, Perth, 3-7 April 2017, 173-182. [Google Scholar] [CrossRef
[2] Zhang, Y. and Lai, G. (2017) DeepCoNN: Deep Cooperative Neural Networks for Sparse and High-Dimensional Recommender Systems. Proceedings of the 10th ACM International Conference on Web Search and Data Mining, Cambridge, 6-10 February 2017, 305-314.
[3] Ammad-Ud-Din, M., Ivannikova, E., Khan, S.A., Oyomno, W., Fu, Q., Tan, K.E. and Flanagan, A. (2019) Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System.
https://arxiv.org/abs/1901.09888
[4] He, P., Gao, J. and Chen, W. (2021) DeBERTav3: Improving DeBERta Using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. [Google Scholar] [CrossRef
[5] McMahan, B., Moore, E., Ramage, D., Hampson, S. and Arcas, B.A. (2017) Communication-Efficient Learning of Deep Networks from Decentralized Data.
https://arxiv.org/abs/1602.05629
[6] 王超, 李斌, 刘金国. 一种改进的Adam优化算法及其在神经网络中的应用[J]. 计算机工程与应用, 2020, 56(24): 136-141.
[7] 张昊, 刘挺. 深度学习中的损失函数综述[J]. 计算机科学, 2020, 47(6): 21-28.
[8] 张海波, 魏秀参, 胡春明. 预训练语言模型的发展综述[J]. 软件学报, 2022, 33(4): 1132-1152.
[9] Song, X., Salcianu, A., Song, Y., Dopson, D. and Zhou, D. (2021) Fast Wordpiece Tokenization. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, November 2021, 2089-2103. [Google Scholar] [CrossRef
[10] Sennrich, R., Haddow, B. and Birch, A. (2016) Neural Machine Translation of Rare Words with Subword Units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, August 2016, 1715-1725. [Google Scholar] [CrossRef
[11] Hand, D.J. and Till, R.J. (2001) A Simple Generalisation of the Area under the ROC Curve for Multiple Class Classification Problems. Machine Learning, 45, 171-186. [Google Scholar] [CrossRef