基于时序卷积密度回归的跳绳视频计数方法
Jump-Rope Video Counting via Temporal Convolutional Density Regression
DOI: 10.12677/csa.2026.1611333, PDF,    科研立项经费支持
作者: 方睿恒*, 王少波, 智 宇:温州大学计算机与人工智能学院,元宇宙与人工智能研究中心,浙江 温州;陈 昂#:温州大学计算机与人工智能学院,元宇宙与人工智能研究中心,浙江 温州;温州大学元宇宙与人工智能研究院,浙江 温州
关键词: 跳绳视频计数;重复动作计数;姿态特征;计数密度回归;时序卷积;Jump-Rope Video Counting; Repetition Counting; Pose Features; Count-Density Regression; Temporal Convolution
摘要: 针对人工记录跳绳次数易受速度变化、短暂停顿和观察疲劳影响的问题,本文提出基于姿态特征与一维时序卷积的跳绳视频计数方法。在重叠窗口内预测非负逐帧密度,经同帧平均与积分得到完整视频次数。在RopeAnno-134 (134段视频、19,448次跳跃)固定划分上,5个随机种子运行的平均绝对误差(MAE)为1.1925 ± 0.0867次,一跳以内准确率(OBO)为0.5143 ± 0.0762。补充69名被试的留一被试交叉验证后,视频微平均MAE为5.4229 ± 0.3143次,多数被试维持较小误差,少数较大偏差提示跨被试适应性仍有提升空间。计数网络含149,697个参数,含姿态提取的离线视频处理吞吐率为87.06 ± 0.71帧/s。该方法为姿态驱动的跳绳视频计数提供了结构简洁、结果可追踪的基线。
Abstract: Manual jump-rope counting is susceptible to speed changes, brief pauses, and observer fatigue. We propose a method using pose features and one-dimensional temporal convolution. Non-negative frame-wise densities predicted in overlapping windows are averaged at shared frames and integrated to obtain full-video counts. On the fixed split of RopeAnno-134 (134 videos and 19,448 jumps), five random-seed runs achieve a mean absolute error (MAE) of 1.1925 ± 0.0867 jumps and an off-by-one accuracy (OBO) of 0.5143 ± 0.0762. Supplementary leave-one-subject-out cross-validation over 69 subjects yields a video-micro MAE of 5.4229 ± 0.3143 jumps. Most subjects retain relatively small errors, while larger deviations in a minority indicate room to improve cross-subject adaptability. The counting network has 149,697 parameters, and offline video processing including pose extraction reaches 87.06 ± 0.71 frames/s. The method provides a simple and traceable baseline for pose-driven jump-rope video counting.
文章引用:方睿恒, 王少波, 智宇, 陈昂. 基于时序卷积密度回归的跳绳视频计数方法[J]. 计算机科学与应用, 2026, 16(11): 156-168. https://doi.org/10.12677/csa.2026.1611333

参考文献

[1] Soro, A., Brunner, G., Tanner, S. and Wattenhofer, R. (2019) Recognition and Repetition Counting for Complex Physical Exercises with Deep Learning. Sensors, 19, Article 714.
https://doi.org/10.3390/s19030714
[2] Hsu, Y.C., Efstratios, T. and Tsui, K. (2023) Viewpoint-Invariant Exercise Repetition Counting. Health Information Science and Systems, 12, Article No. 1.
https://doi.org/10.1007/s13755-023-00258-3
[3] Abedi, A., Bisht, P., Chatterjee, R., Agrawal, R., Sharma, V., Jayagopi, D.B., et al. (2023) Rehabilitation Exercise Repetition Segmentation and Counting Using Skeletal Body Joints. 2023 20th Conference on Robots and Vision (CRV), Montreal, 6-8 June 2023, 288-295.
https://doi.org/10.1109/crv60082.2023.00044
[4] Levy, O. and Wolf, L. (2015) Live Repetition Counting. 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, 7-13 December 2015, 3020-3028.
https://doi.org/10.1109/iccv.2015.346
[5] Runia, T.F.H., Snoek, C.G.M. and Smeulders, A.W.M. (2019) Repetition Estimation. International Journal of Computer Vision, 127, 1361-1383.
https://doi.org/10.1007/s11263-019-01194-0
[6] Zhang, H., Xu, X., Han, G. and He, S. (2020) Context-Aware and Scale-Insensitive Temporal Repetition Counting. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 14-19 June 2020, 667-675.
https://doi.org/10.1109/cvpr42600.2020.00075
[7] Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P. and Zisserman, A. (2020) Counting Out Time: Class Agnostic Video Repetition Counting in the Wild. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13-19 June 2020, 10384-10393.
https://doi.org/10.1109/cvpr42600.2020.01040
[8] Hu, H., Dong, S., Zhao, Y., Lian, D., Li, Z. and Gao, S. (2022) TransRAC: Encoding Multi-Scale Temporal Correlation with Transformers for Repetitive Action Counting. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, 19-24 June 2022, 18991-19000.
https://doi.org/10.1109/cvpr52688.2022.01843
[9] Newell, A., Yang, K. and Deng, J. (2016) Stacked Hourglass Networks for Human Pose Estimation. In: Leibe, B., Matas, J., Sebe, N. and Welling, M., Eds., Computer Vision—ECCV 2016, Springer, 483-499.
https://doi.org/10.1007/978-3-319-46484-8_29
[10] Cao, Z., Hidalgo, G., Simon, T., Wei, S. and Sheikh, Y. (2021) OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 172-186.
https://doi.org/10.1109/tpami.2019.2929257
[11] Sun, K., Xiao, B., Liu, D. and Wang, J. (2019) Deep High-Resolution Representation Learning for Human Pose Estimation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 5686-5696.
https://doi.org/10.1109/cvpr.2019.00584
[12] Du, Y., Wang, W. and Wang, L. (2015). Hierarchical Recurrent Neural Network for Skeleton Based Action Recognition. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 7-12 June 2015, 1110-1118.
https://doi.org/10.1109/cvpr.2015.7298714
[13] Yan, S., Xiong, Y. and Lin, D. (2018) Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. Proceedings of the AAAI Conference on Artificial Intelligence, 32, 7444-7452.
https://doi.org/10.1609/aaai.v32i1.12328
[14] Shi, L., Zhang, Y., Cheng, J. and Lu, H. (2019) Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 12018-12027.
https://doi.org/10.1109/cvpr.2019.01230
[15] Simonyan, K. and Zisserman, A. (2014) Two-Stream Convolutional Networks for Action Recognition in Videos. Advances in Neural Information Processing Systems, 27, 568-576.
[16] Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., et al. (2016) Temporal Segment Networks: Towards Good Practices for Deep Action Recognition. In: Leibe, B., Matas, J., Sebe, N. and Welling, M., Eds., Computer Vision—ECCV 2016, Springer, 20-36.
https://doi.org/10.1007/978-3-319-46484-8_2
[17] Carreira, J. and Zisserman, A. (2017) Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 June 2017, 4724-4733.
https://doi.org/10.1109/cvpr.2017.502
[18] Lea, C., Flynn, M.D., Vidal, R., Reiter, A. and Hager, G.D. (2017). Temporal Convolutional Networks for Action Segmentation and Detection. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 21-26 June 2017, 1003-1012.
https://doi.org/10.1109/cvpr.2017.113
[19] Farha, Y.A. and Gall, J. (2019) MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, 15-20 June 2019, 3575-3584.
https://doi.org/10.1109/cvpr.2019.00369
[20] Jocher, G., Chaurasia, A. and Qiu, J. (2023) Ultralytics YOLOv8, Version 8.0.0.
https://github.com/ultralytics/ultralytics
[21] Wu, Y. and He, K. (2018) Group Normalization. In: Ferrari, V., Hebert, M., Sminchisescu, C. and Weiss, Y., Eds., Computer Vision—ECCV 2018, Springer, 3-19.
https://doi.org/10.1007/978-3-030-01261-8_1
[22] Lee, M. (2023) Mathematical Analysis and Performance Evaluation of the GELU Activation Function in Deep Learning. Journal of Mathematics, 2023, Article ID: 4229924.
https://doi.org/10.1155/2023/4229924
[23] Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R. (2014) Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research, 15, 1929-1958.
[24] He, K., Zhang, X., Ren, S. and Sun, J. (2016) Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 27-30 June 2016, 770-778.
https://doi.org/10.1109/cvpr.2016.90
[25] Loshchilov, I. and Hutter, F. (2019) Decoupled Weight Decay Regularization. arXiv: 1711.05101.