基于近邻卷积Transformer的视频序列3D人体姿态估计方法

3D Human Pose Estimation Method in Video Sequences Based on Neighbor Convolution Transformer

摘要: 近年来，基于Transformer的方法在单目三维人体姿态估计领域取得了显著进展，其强大的自注意力机制能够有效建模全局特征与长程依赖关系。然而，现有方法大多侧重于构建全局的时空依赖，其交互机制缺乏对局部时空结构(特别是相邻帧之间强相关性)的显式归纳偏置。这可能导致模型对近邻帧间紧密而具结构性的时序关联挖掘不足。为此，本文提出一种新颖的注意力架构——近邻卷积Transformer (NCFormer)，它通过近邻帧卷积与轴向多层感知机显式地建模近邻帧间的依赖关系。具体而言，NCFormer包含三个核心组件：(1) 用于捕获全局时空依赖的多头自注意力模块；(2) 近邻卷积模块，利用时间方向的卷积核提取近邻帧关系；(3) 轴向多层感知机，该模块旨在对时间和空间维度进行独立的特征变换，避免跨维度信息的无差别混合，使模型能够更专注地学习各维度特有的模式。在两个广泛使用的三维人体姿态估计基准数据集——Human3.6M和MPI-INF-3DHP上进行的实验表明，NCFormer在多种评估设定下均取得了具有高度竞争力的性能。

Abstract: In recent years, Transformer-based methods have achieved significant progress in the field of monocular 3D human pose estimation, owing to their powerful self-attention mechanism that effectively capture global representations and long-range dependencies. However, most existing approaches predominantly focus on constructing global spatiotemporal dependencies, and their interaction mechanisms lack explicit inductive bias toward local spatiotemporal structures, particularly the strong correlations between adjacent frames. This may lead to insufficient exploitation of the close and structured temporal relationships among neighboring frames. To address this, this paper proposes a novel attention architecture—the Neighbor Convolution Transformer (NCFormer)—which explicitly models dependencies between neighboring frames through neighbor-frame convolution and axial multi-layer perceptrons. Specifically, NCFormer consists of three core components: (1) A multi-head self-attention module for capturing global spatiotemporal dependencies; (2) A neighbor convolution module, which employs temporal convolution kernels to extract relationships among neighboring frames; and (3) An axial multi-layer perceptron, designed to perform independent feature transformations along the temporal and spatial dimensions, thereby avoiding undifferentiated mixing of cross-dimensional information and enabling the model to focus more on learning dimension-specific patterns. Experiments conducted on two widely used benchmark datasets for 3D human pose estimation—Human3.6M and MPI-INF-3DHP—demonstrate that NCFormer achieves highly competitive performance across various evaluation settings.