基于多特征融合及背景建模的微表情Apex帧检测
Micro-Expression Apex Frame Detection Based on Multi Feature Integration and Background Modeling
DOI: 10.12677/CSA.2024.141016, PDF,   
作者: 马韵洁*：合肥综合性国家科学中心数据空间研究院，安徽 合肥;合肥工业大学计算机与信息学院，安徽 合肥；王佐成, 吴艳平, 王 飞：合肥综合性国家科学中心数据空间研究院，安徽 合肥
关键词: 微表情；特征融合；Apex帧；分块方法；计算机视觉；Micro-Expression； Feature Integration； Apex Frame； Chunking Method； Computer Vision
摘要: 微表情是人们在情感在波动时体现在面部的细微变化。根据心理学的研究，微表情在心理治疗等领域有着广泛应用，而Apex帧能够表达微表情的最丰富信息，为了更准确地提取Apex帧，本文研究了图像序列中面部微表情中的Apex帧检测问题。本文提出了一个新的检测Apex帧方法，在频率域中对面部区域采用分块方法进行背景建模，检测出脸部运动区域，随后通过统计运动区域的面积达到检测Apex帧的目的。将提出的方法应用于CASME、CASME II等数据集中，实验结果表明本文提出的方法能够有效地探测定位到Apex帧。
Abstract: Micro-expressions are subtle facial changes that reflect fluctuations in emotions. According to psy-chological research, micro-expressions find widespread application in areas such as psychotherapy. Among these expressions, the Apex frame encapsulates the richest information. In order to accu-rately extract Apex frames, this study investigates the detection problem within facial mi-cro-expressions in image sequences. This paper proposes a novel method for detecting Apex frames. In the frequency domain, a block-based approach is employed for background modeling in facial regions, identifying regions of facial movement. Subsequently, the detection of Apex frames is achieved by statistically analyzing the area covered by the moving regions. The proposed method is applied to datasets such as CASME and CASME II, with experimental results demonstrating its effec-tiveness in detecting and locating Apex frames.
文章引用：马韵洁, 王佐成, 吴艳平, 王飞. 基于多特征融合及背景建模的微表情Apex帧检测[J]. 计算机科学与应用, 2024, 14(1): 147-157. https://doi.org/10.12677/CSA.2024.141016

参考文献

[1] Mehrabia, A. (1968) Commynication without Words. Psychology Today, 2, 53-56.
[2] Li, X., Hong, X., Moilanen, A., et al. (2017) Towards Reading Hidden Emotions: A Comparative Study of Spontaneous Micro-Expression Spotting and Recognition Methods. IEEE Transactions on Affective Computing, 9, 563-577. [Google Scholar] [CrossRef] 
[3] Frank, M.G. and Ekman, P. (1997) The Ability to Detect Deceit Gen-eralizes across Different Types of High-Stake Lies. Journal of Personality and Social Psychology, 72, 1429-1439. [Google Scholar] [CrossRef] [PubMed]
[4] Xu, F. and Zhang, J.P. (2017) Facial Microexpression Recognition: A Survey. Acta Automatica Sinica, 43, 333-348.
[5] Salter, F., Grammer, K. and Rikowski, A. (2005) Sex Difference in Negoti-ating with Powerful Males. Human Nature, 16, 306-321. [Google Scholar] [CrossRef] [PubMed]
[6] Pool, L.D. and Qualter, P. (2012) Improving Emotional Intelligence and Emotional Self-Efficacy through a Teaching Intervention for University Students. Learning and Individual Differences, 22, 306-312. [Google Scholar] [CrossRef] 
[7] Whitehill, J., Serpell, Z., Lin, Y.C., Foster, A. and Movellan, J.R. (2014) The Faces of Engagement: Automatic Recognition of Student Engagementfrom Facial Expressions. IEEE Transactions on Affec-tive Computing, 5, 86-98. [Google Scholar] [CrossRef] 
[8] Haggard, E.A. and Isaacs, K.S. (1966) Micromomentary Facial Ex-pressions as Indicators of Ego Mechanisms in Psychotherapy. In: Haggard, E.A. and Isaacs, K.S., Eds., Methods of Research in Psychotherapy, Springer, Boston, 154-165. [Google Scholar] [CrossRef] 
[9] Ekman, P. and Friesen, W.V. (1969) Nonverbal Leakage and Clues to Deception. Psychiatry Interpersonal & Biological Processes, 32, 88-106. [Google Scholar] [CrossRef] [PubMed]
[10] Ekman, P. and Friesen, W.V. (1974) Nonverbal Behavior and Psychopathology. In: Friedman, R.J. and Katz, M.M., Eds., The Psychology of Depression: Contemporary Theory and Re-search, Halsted Press, Sydney, 3-31.
[11] Matsumoto, D., Leroux, J., Wilson-Cohn, C., et al. (2000) A New Test to Measure Emotion Recognition Ability: Matsumoto and Ekman’s Japanese and Caucasian Brief Affect Recognition Test (JACBART). Journal of Nonverbal Behavior, 24, 179-209. [Google Scholar] [CrossRef] 
[12] Ekman, P. and Friesen, W.V. (1978) Facial Action Coding System Investigator’s Guide. Consulting Psychologists Press, Palo Alto. [Google Scholar] [CrossRef] 
[13] Ekman, P. and Friesen, W.V. (1978) Manual for the Facial Action Coding Sys-tem. Consulting Psychologists Press Palo Alto. [Google Scholar] [CrossRef] 
[14] Pfister, T., Li, X.B., Zhao, G.Y., et al. (2011) Recognising Spontaneous Facial Micro-Expressions. 2011 International Conference on Computer Vision, Barce-lona, 6-13 November 2011, 1449-1456. [Google Scholar] [CrossRef] 
[15] Peng, M., Wang, C.Y., Chen, T., et al. (2017) Dual Temporal Scale Convolutional Neural Network for Micro-Expression Recognition. Frontiers in Psychology, 8, Article 273835. [Google Scholar] [CrossRef] [PubMed]
[16] Peng, M., Wu, Z., Zhang, Z.H., et al. (2018) From Macro to Micro Ex-pression Recognition: Deep Learning on Small Datasets Using Transfer Learning. 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi’an, 15-19 May 2018, 657-661. [Google Scholar] [CrossRef] 
[17] Huang, X., Wang, S.J., Zhao, G., et al. (2015) Facial Micro-Expression Recognition Using Spatiotemporal Local Binary Pattern with Integral Projection. Proceedings of the IEEE International Con-ference on Computer Vision Workshops, Santiago, Chile, 07-13 December 2015, 1-9.
[18] Wang, Y.D., See, J., Phan, R.C.W., et al. (2014) LBP with Six Intersection Points: Reducing Redundant Information in LBP-Top for Micro Expression Recognition. In: Cremers, D., Reid, I., Saito, H. and Yang, M.H., Eds., Computer Vision—ACCV 2014, Springer, Cham, 525-537. [Google Scholar] [CrossRef] 
[19] Li, Y.T., Huang, X.H. and Zhao, G.Y. (2018) Can Micro-Expression Be Recognized Based on Single Apex Frame? 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, 7-10 October 2018, 3094-3098. [Google Scholar] [CrossRef] 
[20] Gan, Y.S., Liong, S.T., Yau, W.C., et al. (2019) OFF-ApexNet on Mi-cro Expressions Recognition System. Signal Processing: Image Communication, 74, 129-139. [Google Scholar] [CrossRef] 
[21] Patel, D., Zhao, G. and Pietikäinen, M. (2015) Spatiotemporal Integra-tion of Optical Flow Vectors for Micro-Expression Detection. In: Battiato, S., Blanc-Talon, J., Gallo, G., Philips, W., Popescu, D. and Scheunders, P., Eds., ACIVS 2015: Advanced Concepts for Intelligent Vision Systems, Springer, Cham, 369-380. [Google Scholar] [CrossRef] 
[22] Liong, S.T., See, J., Wong, K., Le Ngo, A.C., Oh, Y.H. and Phan, R. (2015) Automatic Apex Frame Spotting in Micro- Expression Database. 2015 3rd IAPR Asian Conference on Pattern Recogni-tion (ACPR), Kuala Lumpur, 3-6 November 2015, 665-669. [Google Scholar] [CrossRef] 
[23] Liong, S.T., See, J., Wong, K.S. and Pan, R.C.W. (2018) Less Is More: Micro-Expression Recognition from Video Using Apex Frame. Signal Processing: Image Communication, 62, 82-92. [Google Scholar] [CrossRef] 
[24] Gan, Y.S., Liong, S.T., Yau, W.C., et al. (2019) Off-Apexnet on Micro-Expression Recognition System. Signal Processing: Image Communica-tion, 74, 129-139. [Google Scholar] [CrossRef] 
[25] Xu, W., Zheng, H. and Yang, Z.X. (2021) Apex Frame Micro-Expression Recognition Based Dual Attention Model and Transfer Learning. CAAI Transactions on Intelligent Systems, 16, 1015-1020.
[26] 温杰彬, 杨文忠, 马国详, 张志豪, 李海磊. 基于Apex帧光流和卷积自编码器的微表情识别[J]. 计算机工程与应用, 2021, 57(4): 127-133.
[27] 韦锦涛, 倪建成, 侯杰, 吴俊清, 魏媛媛. 基于卷积神经网络的微表情识别模型[J]. 曲阜师范大学学报, 2021, 47(1): 64-71.
[28] Zhao, Y. and Xu, J. (2019) A Convolutional Neural Net Work for Compound Micro-Expression Recognition. Sensors, 19, Article 5553. [Google Scholar] [CrossRef] [PubMed]
[29] Li, Y., Huang, X. and Zhao, G. (2021) Joint Local and Global Information Learning with Single Apex Frame Detection for Micro-Expression Recognition. IEEE Transactions on Image Processing, 30, 249-263. [Google Scholar] [CrossRef] 
[30] Barnich, O. and Droogenbroeck, M.V. (2011) ViBe: A Universal Back-ground Subtraction Algorithm for Video Sequences. IEEE Transactions on Image Processing, 20, 1709-1724. [Google Scholar] [CrossRef] 
[31] Yan, W.J., Li, X., Wang, S.J., et al. (2014) CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation. PLOS ONE, 9, e86041. [Google Scholar] [CrossRef] [PubMed]
[32] Liong, S.T., See, J., Wong, K.S., et al. (2015) Automatic Apex Frame Spotting in Micro-Expression Database. 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Kuala Lumpur, 3-6 November 2015, 665-669. [Google Scholar] [CrossRef] 
[33] Yan, W.J., Wang, S.J., Chen, Y.H., et al. (2015) Quantifying Mi-cro-Expressions with Constraint Local Model and Local Binary Pattern. In: Agapito, L., Bronstein, M. and Rother, C., Eds., Computer Vision—ECCV 2014 Workshops, Springer, Cham, 296-305. [Google Scholar] [CrossRef] 
[34] Liong, S.T., Phan, C.W., See, J., et al. (2015) Optical Strain Based Recognition of Subtle Emotions. 2014 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), Kuching, 1-4 December 2014, 180-184. [Google Scholar] [CrossRef] 
[35] 马浩原. 微表情特征提取算法研究[D]: [硕士学位论文]. 北京: 北京交通大学, 2018.