基于GRPO强化学习算法的视觉语言模型空间推理研究

Spatial Reasoning in Large Vision-Language Models via GRPO-Based Reinforcement Learning

摘要: 视觉语言模型(VLM)在视觉语义任务上表现出色，但在相对深度、三维定位与多实体空间关系等空间几何推理任务中仍存在不足，根源在于高质量空间数据稀缺以及传统监督式训练难以激发模型的空间推理过程。为此，本文提出一套集自动化标注、数据合成与模型训练于一体的端到端框架，以系统提升VLM的空间认知能力。我们构建了无需人工干预的图片空间标注流水线，通过高召回检测、掩码精修与深度相机参数恢复，实现了对照片空间信息的完整标注。在此基础上，我们设计了任务导向的数据合成模块，生成覆盖定性、定量，单跳、多跳的空间推理数据。进一步地，我们基于GRPO算法设计了一套强化学习训练流程，并结合了定制的奖励函数与课程学习训练策略，实现了大模型在空间任务上的稳定训练。实验结果表明，该框架在多个公开与自建基准上超越包括Qwen2.5-VL与InternVL在内的主流模型，验证了语义–几何一致的数据构建与强化学习训练策略优化对提升VLM空间推理能力的有效性。

Abstract: While Visual Language Models (VLMs) excel in visual semantic tasks, they exhibit significant deficiencies in spatial geometric reasoning tasks, such as relative depth estimation, 3D localization, and multi-entity spatial relationships. These limitations stem primarily from the scarcity of high-quality spatial data and the inability of traditional supervised training to effectively elicit spatial reasoning processes within the models. To address these challenges, this paper proposes an end-to-end framework integrating automated annotation, data synthesis, and model training, designed to systematically enhance the spatial cognitive capabilities of VLMs. We construct a fully automated image spatial annotation pipeline that achieves comprehensive annotation of spatial information through high-recall detection, mask refinement, and the recovery of depth and camera parameters, eliminating the need for human intervention. Building upon this, we design a task-oriented data synthesis module to generate spatial reasoning data encompassing qualitative, quantitative, single-hop, and multi-hop scenarios. Furthermore, we develop a reinforcement learning training pipeline based on the Group Relative Policy Optimization (GRPO) algorithm. By incorporating customized reward functions and curriculum learning strategies, we enable the stable training of large models for spatial tasks. Experimental results demonstrate that the proposed framework outperforms mainstream models, including Qwen2.5-VL and InternVL, across multiple public and self-constructed benchmarks. These findings validate the effectiveness of semantic-geometric consistent data construction and optimized reinforcement learning strategies in elevating the spatial reasoning abilities of VLMs.