基于弯曲敏感石墨烯应变传感器和衍射深度神经网络的小数据手势识别
Small-Data Gesture Recognition Using Bending-Sensitive Graphene Strain Sensors and Diffractive Deep Neural Networks
DOI: 10.12677/mos.2025.145374, PDF,   
作者: 李 瑾, 刘子辰, 陈 希*：上海理工大学智能科技学院，上海；上海理工大学光子芯片研究院，上海
关键词: 手势识别；小数据；应变传感器；衍射深度神经网络；激光还原氧化石墨烯；Gesture Recognition； Small-Data； Strain Sensor； Diffractive Deep Neural Network； Laser-Reduced Graphene Oxide
摘要: 人工智能识别手势在认知神经科学和机械臂技术领域有着广泛的应用。最近，摄像头和应变传感器被用于收集不同手势的图片和随时间变化的响应数据，这些数据被输入人工神经网络进行识别。大量的输入数据会造成算力的浪费、大量的能源消耗和明显的时间延迟。文章介绍了一种基于弯曲敏感石墨烯应变传感器和衍射深度神经网络(Diffraction Deep Neural Network, D2NN)的手势识别方法。应变传感器是通过激光划线技术在氧化石墨烯(Graphene Oxide, GO)薄膜上制备的，在弯曲角度调节下表现出可调的电流响应。10个应变传感器位于手指关节处，用于捕捉不同手势下的电流响应，并将其转换为10像素图像，作为衍射深度神经网络的输入。在3次迭代内，识别0~9数字手势的仿真准确率可达到100%。输入数据量仅为10个，远低于已报道的手势识别设备。石墨烯应变传感器与光学神经形态计算的结合，为实现低成本、高效率、高准确度的人机交互铺平了道路。
Abstract: Artificial intelligence recognition of gestures shows a wide range of applications in cognitive neuroscience and robotic arm technology. Cameras and strain sensors have recently been used to collect photo and time-dependent response data for different gestures. The data are input into artificial neural networks for recognition. The large amount of the input data induces computing power waste, massive energy consumption, and significant time delay. This paper introduces a gesture recognition pathway based on bend-sensitive graphene strain sensors and diffractive deep neural network (D2NN). The strain sensor is fabricated on graphene oxide (GO) film through laser scribing and exhibits tunable current responses under the adjustment of bending angles. 10 strain sensors are located at finger joints to capture the current responses under different gestures and convert them into a 10-pixel image as the input of diffractive deep neural networks. 100% accuracy can be achieved within 3 iterations to recognize gestures representing numbers 0~9. The amount of input data is only ten, much less than those of the reported gesture recognition devices. The pathway combining graphene strain sensors with optical neuromorphic computing paves the way for achieving low-cost, efficient, and high-accuracy human-computer interaction.
文章引用：李瑾, 刘子辰, 陈希. 基于弯曲敏感石墨烯应变传感器和衍射深度神经网络的小数据手势识别[J]. 建模与仿真, 2025, 14(5): 67-82. https://doi.org/10.12677/mos.2025.145374

参考文献

[1] Slade, P., Atkeson, C., Donelan, J.M., Houdijk, H., Ingraham, K.A., Kim, M., et al. (2024) On Human-in-the-Loop Optimization of Human-Robot Interaction. Nature, 633, 779-788. [Google Scholar] [CrossRef] [PubMed]
[2] Tan, P., Han, X., Zou, Y., Qu, X., Xue, J., Li, T., et al. (2022) Self-Powered Gesture Recognition Wristband Enabled by Machine Learning for Full Keyboard and Multicommand Input. Advanced Materials, 34, Article 2200793. [Google Scholar] [CrossRef] [PubMed]
[3] Moin, A., Zhou, A., Rahimi, A., Menon, A., Benatti, S., Alexandrov, G., et al. (2020) A Wearable Biosensing System with In-Sensor Adaptive Machine Learning for Hand Gesture Recognition. Nature Electronics, 4, 54-63. [Google Scholar] [CrossRef] 
[4] Zhou, Z., Chen, K., Li, X., Zhang, S., Wu, Y., Zhou, Y., et al. (2020) Sign-to-Speech Translation Using Machine-Learning-Assisted Stretchable Sensor Arrays. Nature Electronics, 3, 571-578. [Google Scholar] [CrossRef] 
[5] Wang, H., Ding, Q., Luo, Y., Wu, Z., Yu, J., Chen, H., et al. (2023) High-Performance Hydrogel Sensors Enabled Multimodal and Accurate Human-Machine Interaction System for Active Rehabilitation. Advanced Materials, 36, Article 2309868. [Google Scholar] [CrossRef] [PubMed]
[6] Osman, A., Elhakeem, A., Kaytbay, S. and Ahmed, A. (2022) A Comprehensive Review on the Thermal, Electrical, and Mechanical Properties of Graphene-Based Multi-Functional Epoxy Composites. Advanced Composites and Hybrid Materials, 5, 547-605. [Google Scholar] [CrossRef] 
[7] Wu, Y., An, C., Guo, Y., Zong, Y., Jiang, N., Zheng, Q., et al. (2024) Highly Aligned Graphene Aerogels for Multifunctional Composites. Nano-Micro Letters, 16, Article No. 118. [Google Scholar] [CrossRef] [PubMed]
[8] Huang, X., Wei, J., Zhang, Y., Qian, B., Jia, Q., Liu, J., et al. (2022) Ultralight Magnetic and Dielectric Aerogels Achieved by Metal-Organic Framework Initiated Gelation of Graphene Oxide for Enhanced Microwave Absorption. Nano-Micro Letters, 14, Article No. 107. [Google Scholar] [CrossRef] [PubMed]
[9] Song, N., Zhang, F., Cao, D., Wang, P. and Ding, P. (2022) Bicontinuous Laminated Structure Design of Polypropylene/Reduced Graphene Oxide Hybrid Films for Thermal Management. Advanced Composites and Hybrid Materials, 5, 2873-2883. [Google Scholar] [CrossRef] 
[10] Yuan, Y., Huang, J., Li, X., Jiang, L., Li, T., Sun, P., et al. (2023) Laser-Induced Electron Synchronization Excitation for Photochemical Synthesis and Patterning Graphene-Based Electrode. Advanced Materials, 36, Article 2308368. [Google Scholar] [CrossRef] [PubMed]
[11] Yuan, Y., Jiang, L., Li, X., Zuo, P., Zhang, X., Lian, Y., et al. (2022) Ultrafast Shaped Laser Induced Synthesis of Mxene Quantum Dots/Graphene for Transparent Supercapacitors. Advanced Materials, 34, Article 2110013. [Google Scholar] [CrossRef] [PubMed]
[12] Gao, Y., Liu, J. and Yang, S. (2023) Liquid Crystalline Reduced Graphene Oxide Composite Fibers as Artificial Muscles. Materials Today, 69, 19-30. [Google Scholar] [CrossRef] 
[13] Luo, S., Peng, L., Xie, Y., Cao, X., Wang, X., Liu, X., et al. (2023) Flexible Large-Area Graphene Films of 50-600 nm Thickness with High Carrier Mobility. Nano-Micro Letters, 15, Article No. 61. [Google Scholar] [CrossRef] [PubMed]
[14] Wu, J., Zhang, Y., Hu, J., Yang, Y., Jin, D., Liu, W., et al. (2024) 2D Graphene Oxide Films Expand Functionality of Photonic Chips. Advanced Materials, 36, Article 2403659. [Google Scholar] [CrossRef] [PubMed]
[15] Nazari, P., Bäuerle, R., Zimmermann, J., Melzer, C., Schwab, C., Smith, A., et al. (2023) Piezoresistive Free-Standing Microfiber Strain Sensor for High-Resolution Battery Thickness Monitoring. Advanced Materials, 35, Article 2212189. [Google Scholar] [CrossRef] [PubMed]
[16] Kim, E., Kim, S., Jin, H.M., Kim, G., Ha, H., Choi, Y., et al. (2024) Unlocking Novel Functionality: Pseudocapacitive Sensing in Mxene-Based Flexible Supercapacitors. Nano-Micro Letters, 17, Article No. 86. [Google Scholar] [CrossRef] [PubMed]
[17] Yang, D., Zhao, K., Yang, R., Zhou, S., Chen, M., Tian, H., et al. (2024) A Rational Design of Bio-Derived Disulfide Cans for Wearable Capacitive Pressure Sensor. Advanced Materials, 36, Article 2403880. [Google Scholar] [CrossRef] [PubMed]
[18] Kaiser, D., Meyerbroeker, N., Purschke, W., Sell, S., Neumann, C., Winter, A., et al. (2024) Ultrasensitive Detection of Chemokines in Clinical Samples with Graphene-Based Field-Effect Transistors. Advanced Materials, 36, Article 2407487. [Google Scholar] [CrossRef] [PubMed]
[19] Zhou, S., Zhang, X., Wang, Y., et al. (2024) Opto-Electrical Decoupled Phototransistor for Starlight Detection. Advanced Materials (Deerfield Beach, Fla), 2024, e2413247.
[20] Zhao, G., Qian, F., Li, X., Tang, Y., Sheng, Y., Li, H., et al. (2023) Constructing a Continuous Reduced Graphene Oxide Network in Porous Plant Fiber Sponge for Highly Compressible and Sensitive Piezoresistive Sensors. Advanced Composites and Hybrid Materials, 6, Article No. 184. [Google Scholar] [CrossRef] 
[21] Wu, D., Pigou, L., Kindermans, P., Le, N.D., Shao, L., Dambre, J., et al. (2016) Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 1583-1597. [Google Scholar] [CrossRef] [PubMed]
[22] Chen, T., Pang, Z., He, S., Li, Y., Shrestha, S., Little, J.M., et al. (2024) Machine Intelligence-Accelerated Discovery of All-Natural Plastic Substitutes. Nature Nanotechnology, 19, 782-791. [Google Scholar] [CrossRef] [PubMed]
[23] Xie, J., Xiang, X., Xia, S., Jiang, L., Wang, G. and Gao, X. (2024) MGNR: A Multi-Granularity Neighbor Relationship and Its Application in KNN Classification and Clustering Methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46, 7956-7972. [Google Scholar] [CrossRef] [PubMed]
[24] Turkoglu, M.O., Daaronco, S., Wegner, J.D., et al. (2022) Gating Revisited: Deep Multi-Layer Rnns that Can Be Trained. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44, 4081-4092.
[25] Deng, X. and Dragotti, P.L. (2021) Deep Convolutional Neural Network for Multi-Modal Image Restoration and Fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43, 3333-3348. [Google Scholar] [CrossRef] [PubMed]
[26] Zhang, X., Zhang, B., Freddolino, L. and Zhang, Y. (2022) CR-I-TASSER: Assemble Protein Structures from Cryo-Em Density Maps Using Deep Convolutional Neural Networks. Nature Methods, 19, 195-204. [Google Scholar] [CrossRef] [PubMed]
[27] Li, X., Tang, J., Zhang, Q., Gao, B., Yang, J.J., Song, S., et al. (2020) Power-Efficient Neural Network with Artificial Dendrites. Nature Nanotechnology, 15, 776-782. [Google Scholar] [CrossRef] [PubMed]
[28] Zhang, Q., Yu, H., Barbiero, M., Wang, B. and Gu, M. (2019) Artificial Neural Networks Enabled by Nanophotonics. Light: Science & Applications, 8, Article No. 42. [Google Scholar] [CrossRef] [PubMed]
[29] Lin, X., Rivenson, Y., Yardimci, N.T., Veli, M., Luo, Y., Jarrahi, M., et al. (2018) All-Optical Machine Learning Using Diffractive Deep Neural Networks. Science, 361, 1004-1008. [Google Scholar] [CrossRef] [PubMed]
[30] Liu, C., Ma, Q., Luo, Z.J., Hong, Q.R., Xiao, Q., Zhang, H.C., et al. (2022) A Programmable Diffractive Deep Neural Network Based on a Digital-Coding Metasurface Array. Nature Electronics, 5, 113-122. [Google Scholar] [CrossRef] 
[31] Wang, Q., Liu, J., Lyu, D. and Wang, J. (2024) Ultrahigh-Fidelity Spatial Mode Quantum Gates in High-Dimensional Space by Diffractive Deep Neural Networks. Light: Science & Applications, 13, Article No. 10. [Google Scholar] [CrossRef] [PubMed]
[32] Chen, H., Feng, J., Jiang, M., Wang, Y., Lin, J., Tan, J., et al. (2021) Diffractive Deep Neural Networks at Visible Wavelengths. Engineering, 7, 1483-1491. [Google Scholar] [CrossRef] 
[33] Burch, J. and Di Falco, A. (2018) Surface Topology Specific Metasurface Holograms. ACS Photonics, 5, 1762-1766. [Google Scholar] [CrossRef] 
[34] Wu, J., Lin, M., Cong, X., Liu, H. and Tan, P. (2018) Raman Spectroscopy of Graphene-Based Materials and Its Applications in Related Devices. Chemical Society Reviews, 47, 1822-1873. [Google Scholar] [CrossRef] [PubMed]
[35] Chen, X. and Gu, M. (2022) Two-Beam Ultrafast Laser Scribing of Graphene Patterns with 90-nm Subdiffraction Feature Size. Ultrafast Science, 2022, 1-9. [Google Scholar] [CrossRef] 
[36] Liu, X., Fang, L., Zhang, F., Zhang, Q., Wan, Z. and Chen, X. (2024) All-Optical Diffractive Deep Neural Networks Enabled Laser-Reduced Graphene Oxide Tactile Sensor for Braille Recognition. ACS Applied Electronic Materials, 6, 2049-2058. [Google Scholar] [CrossRef] 
[37] Orekhov, N.D., Bondareva, J.V., Potapov, D.O., Dyakonov, P.V., Dubinin, O.N., Tarkhov, M.A., et al. (2022) Mechanism of Graphene Oxide Laser Reduction at Ambient Conditions: Experimental and Reaxff Study. Carbon, 191, 546-554. [Google Scholar] [CrossRef] 
[38] Chen, H., Lou, S., Wang, Q., Huang, P., Duan, H. and Hu, Y. (2024) Diffractive Deep Neural Networks: Theories, Optimization, and Applications. Applied Physics Reviews, 11, Article 021332. [Google Scholar] [CrossRef] 
[39] Goi, E., Chen, X., Zhang, Q., Cumming, B.P., Schoenhardt, S., Luan, H., et al. (2021) Nanoprinted High-Neuron-Density Optical Linear Perceptrons Performing Near-Infrared Inference on a CMOS Chip. Light: Science & Applications, 10, Article No. 40. [Google Scholar] [CrossRef] [PubMed]
[40] Yang, B., Dong, Y. and Chen, X. (2024) Feature-Enhanced Artificial Visual Perception Based on Superlinear Voltage-Reflectance Responses of Electrochromic Arrays. ACS Photonics, 11, 1909-1919. [Google Scholar] [CrossRef] 
[41] Aggarwal, A., Bhutani, N., Kapur, R., Dhand, G. and Sheoran, K. (2023) Real-Time Hand Gesture Recognition Using Multiple Deep Learning Architectures. Signal, Image and Video Processing, 17, 3963-3971. [Google Scholar] [CrossRef] 
[42] Czuszynski, K., Ruminski, J. and Kwasniewska, A. (2018) Gesture Recognition with the Linear Optical Sensor and Recurrent Neural Networks. IEEE Sensors Journal, 18, 5429-5438. [Google Scholar] [CrossRef] 
[43] Zhang, Z., Wang, Q., Shi, G., Ma, Y., Zeng, J. and Liu, G. (2024) Neural Networks Based on In-Sensor Computing of Optoelectronic Memristor. Microelectronic Engineering, 291, Article 112201. [Google Scholar] [CrossRef] 
[44] Lee, H., Lee, S., Kim, J., Jung, H., Yoon, K.J., Gandla, S., et al. (2023) Stretchable Array Electromyography Sensor with Graph Neural Network for Static and Dynamic Gestures Recognition System. npj Flexible Electronics, 7, Article No. 20. [Google Scholar] [CrossRef] 
[45] Wang, M., Yan, Z., Wang, T., Cai, P., Gao, S., Zeng, Y., et al. (2020) Gesture Recognition Using a Bioinspired Learning Architecture that Integrates Visual Data with Somatosensory Data from Stretchable Sensors. Nature Electronics, 3, 563-570. [Google Scholar] [CrossRef] 
[46] Song, Y., Liu, M., Wang, F., Zhu, J., Hu, A. and Sun, N. (2024) Gesture Recognition Based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory Network for a Wearable Wrist Sensor with Multi-Walled Carbon Nanotube/Cotton Fabric Material. Micromachines, 15, Article 185. [Google Scholar] [CrossRef] [PubMed]

为你推荐