面向无标签数据的三维场景语义理解方法研究

Research on Semantic Understanding Methods for 3D Scenes with Unlabeled Data

摘要: 三维场景理解是计算机视觉领域的核心课题，其目标是从二维观测数据中恢复场景的三维结构并赋予语义含义。现有方法通常依赖大量带标签数据，且多采用面向任务的场景表达方式，难以兼顾表达能力的全面性与学习效率。针对上述问题，本文提出一种面向对象的三维场景语义理解框架，旨在从无标签数据中学习以对象为中心的连续场景表达。该方法以三维高斯溅射(3D Gaussian Splatting, 3DGS)为核心表示基元，设计了一种融合自监督与对抗学习的复合训练策略：一方面利用几何与物理一致性提供自监督信号，另一方面通过多模态先验知识引导场景表达的规则化。实验结果表明，该方法在三维重建质量和语义理解能力方面均取得了良好效果，验证了面向对象表达与无标签学习的有效结合。本研究为实现低标注依赖、高泛化能力的三维场景理解提供了可行路径。

Abstract: Three-dimensional scene understanding is a fundamental task in computer vision, aiming to recover the 3D structure of a scene from 2D observations while assigning semantic meaning to it. Existing approaches typically rely heavily on large amounts of labeled data and often adopt task-specific scene representations, which struggle to balance representational expressiveness with learning efficiency. To address these challenges, this paper proposes an object-oriented framework for 3D scene semantic understanding, designed to learn continuous, object-centric scene representations from unlabeled data. The proposed method employs 3D Gaussian Splatting (3DGS) as the core representation primitive and introduces a hybrid training strategy that integrates self-supervised learning with adversarial learning. Specifically, geometric and physical consistency constraints are leveraged to provide self-supervised signals, while multimodal prior knowledge is incorporated to regularize the scene representation. Experimental results demonstrate that the proposed method achieves promising performance in both 3D reconstruction quality and semantic understanding capability, validating the effective combination of object-oriented representation and label-free learning. This study offers a viable pathway toward 3D scene understanding with reduced annotation dependence and enhanced generalization ability.