基于强化学习与动态异构图的电商跨域商品序列推荐方法

A Reinforcement Learning and Dynamic Heterogeneous Graph-Based Method for Cross-Domain Product Sequential Recommendation in E-Commerce

摘要: 本文面向电商领域跨域商品推荐场景，针对目标域商品交互数据稀疏、用户长期偏好难以有效建模与优化的问题，在动态异构图表征与多视图对齐学习的基础上，将推荐任务建模为带迁移代价约束的受限马尔可夫决策过程，提出离线强化学习方法DHGCRL。模型以源域局部图、目标域局部图与跨域全局异构图生成的三视图表征构造状态，并结合候选集约束与SlateQ列表价值近似实现Top-K商品列表决策。奖励函数由目标域商品反馈与跨域对齐相似度塑形项共同构成，以增强迁移表征对目标域商品偏好的刻画能力。同时，采用推荐列表表征与源域稳定兴趣表征之间的余弦距离定义迁移代价，通过联合回报价值与代价价值形成修正价值，从而指导策略学习并满足代价阈值约束。为提升离线训练的稳定性，进一步引入保守价值学习与行为一致性正则。实验结果表明，DHGCRL在HR和NDCG等指标上均优于多种序列推荐与离线强化学习基线方法。

Abstract: In the context of cross-domain product recommendation in e-commerce, this study addresses the sparsity of target-domain interaction data and the difficulty of modeling and optimizing users’ long-term preferences. Building on dynamic heterogeneous graph representations and multi-view alignment learning, the recommendation process is formulated as a constrained Markov decision process with transfer cost constraints, and an offline reinforcement learning method, DHGCRL, is proposed. The model constructs the state from three-view representations derived from the source-domain local graph, target-domain local graph, and cross-domain global heterogeneous graph, and performs Top-K product list decision-making by combining candidate set constraints with Slate Q-based list value approximation. The reward function integrates target-domain feedback with a cross-domain alignment similarity shaping term to strengthen target-domain preference modeling. Meanwhile, the transfer cost is defined as the cosine distance between the recommended list representation and the stable interest representation in the source domain. By jointly modeling return value and cost value, the method learns a revised value function that guides policy optimization under a predefined cost threshold. To improve the stability of offline training, conservative value learning and behavior-consistency regularization are further introduced. Experimental results demonstrate that DHGCRL outperforms a range of sequential recommendation and offline reinforcement learning baselines in terms of HR and NDCG.