基于条件归一化流的并行时空注意力的无监督异常检测模型
Conditional Normalization Flow-Based Unsupervised Anomaly Detection Model with Parallel Spatio-Temporal Attention
摘要: 预测性维护作为提升工业生产效率的重要环节,其核心挑战在于准确识别故障并发出早期预警。在实践中,工业生产中记录的数据具有两大特征:数据规模庞大与标签缺失。因此需建立针对多元时间序列的无监督异常检测模型。本文提出一种新型无监督异常检测模型——基于条件归一化流的并行时空注意力无监督异常检测(PAF)模型。该模型基于“异常数据远少于正常数据”的假设,结合高斯分布理论,将异常数据定位于低概率密度区域。基于此假设,采用条件归一化流变换数据分布并进行密度估计,进而设定异常评分实现有效判别。该模型的核心特征在于运用并行注意力结构捕捉原始数据的时空信息,作为条件归一化流的前置信息条件。相较于现有相关研究,PAF展现出更优异的检测性能。最终,我们通过两个公开数据集验证了PAF模型的先进性和有效性。
Abstract: Predictive maintenance, as a crucial component for improving industrial production efficiency, faces a core challenge in accurately identifying faults and issuing early warnings. In practice, the data recorded in industrial production have two main characteristics: multivariate data and missing labels. Therefore, there is an urgent need to establish unsupervised anomaly detection models for multivariate time series. This paper proposes a novel unsupervised anomaly detection model—the Parallel Spatiotemporal Attention Flow-based (PAF) model. This model is based on the assumption that “anomalous data are far less frequent than normal data” and, combined with Gaussian distribution theory, positions anomalous data in low-probability density regions. Based on this assumption, conditional normalizing flows are used to transform the data distribution and perform density estimation, thereby setting anomaly scores to achieve effective detection. The core feature of this model is the use of a parallel attention structure to capture the spatiotemporal conditions of the original data as prior information for the normalizing flow. Compared with existing related studies, PAF demonstrates superior detection performance. Finally, the effectiveness and advancement of the PAF model are validated using two publicly available datasets.
文章引用:李新宇, 于晋伟, 杨卫华. 基于条件归一化流的并行时空注意力的无监督异常检测模型[J]. 应用数学进展, 2026, 15(4): 170-181. https://doi.org/10.12677/aam.2026.154147

参考文献

[1] Feng, Y., Chen, J., Xie, J., Zhang, T., Lv, H. and Pan, T. (2022) Meta-Learning as a Promising Approach for Few-Shot Cross-Domain Fault Diagnosis: Algorithms, Applications, and Prospects. Knowledge-Based Systems, 235, Article ID: 107646. [Google Scholar] [CrossRef
[2] Zhang, T., Chen, J., Liu, S. and Liu, Z. (2023) Domain Discrepancy-Guided Contrastive Feature Learning for Few-Shot Industrial Fault Diagnosis under Variable Working Conditions. IEEE Transactions on Industrial Informatics, 19, 10277-10287. [Google Scholar] [CrossRef
[3] Zhang, T., Chen, J., He, S. and Zhou, Z. (2022) Prior Knowledge-Augmented Self-Supervised Feature Learning for Few-Shot Intelligent Fault Diagnosis of Machines. IEEE Transactions on Industrial Electronics, 69, 10573-10584. [Google Scholar] [CrossRef
[4] Li, T., Sun, C., Li, S., Wang, Z., Chen, X. and Yan, R. (2022) Explainable Graph Wavelet Denoising Network for Intelligent Fault Diagnosis. IEEE Transactions on Neural Networks and Learning Systems, 35, 8535-8548. [Google Scholar] [CrossRef] [PubMed]
[5] Xu, W., Zhou, Z., Li, T., Sun, C., Chen, X. and Yan, R. (2022) Physics-Constraint Variational Neural Network for Wear State Assessment of External Gear Pump. IEEE Transactions on Neural Networks and Learning Systems, 35, 5996-6006. [Google Scholar] [CrossRef] [PubMed]
[6] Durkan, C., Bekasov, A., Murray, I. and Papamakarios, G. (2019) Neural Spline Flows. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, 8-14 December 2019, 7511-7522.
[7] Ho, J., Chen, X., Srinivas, A., Duan, Y. and Abbeel, P. (2019) Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design. Proceedings International Conference on Machine Learning (ICML), Long Beach, 9-15 June 2019, 2722-2730.
[8] Grathwohl, W., Chen, R.T.Q., Bettencourt, J., Sutskever, I. and Duvenaud, D. (2018) FFJORD: Free-Form Continuous Dynamics for Scalable Reversible Generative Models. arXiv:1810.01367.
[9] Dinh, L., Sohl-Dickstein, J. and Bengio, S. (2016) Density Estimation Using Real NVP. arXiv:1605.08803.
[10] Kim, S., Kim, H., Yun, E., Lee, H., Lee, J. and Lee, J. (2023) Probabilistic Imputation for Time-Series Classification with Missing Data. Proceedings International Conference on Machine Learning (ICML), Honolulu, 23-29 July 2023, 16654-16667.
[11] Zhou, L., Poli, M., Xu, W., Massaroli, S. and Ermon, S. (2023) Deep Latent State Space Models for Time-Series Generation. Proceedings International Conference on Machine Learning (ICML), Honolulu, 23-29 July 2023, 42625-42643.
[12] Raghu, A., Chandak, P., Alam, R., Guttag, J. and Stultz, C. (2023) Sequential Multi-Dimensional Self-Supervised Learning for Clinical Time Series. Proceedings International Conference on Machine Learning (ICML), Honolulu, 23-29 July 2023, 28531-28548.
[13] Li, Y., Chen, W., Chen, B., Wang, D., Tian, L. and Zhou, M. (2023) Prototype-Oriented Unsupervised Anomaly Detection for Multivariate Time Series. Proceedings International Conference on Machine Learning (ICML), Honolulu, 23-29 July 2023, 19407-19424.
[14] Park, S., Park, B., Lee, M. and Lee, C. (2023) Neural Stochastic Differential Games for Time-Series Analysis. Proceedings of the 40th International Conference on Machine Learning, Honolulu, 23-29 July 2023, 27269-27293.
[15] Lai, Z., Liu, M., Pan, Y. and Chen, D. (2022) Multi-Dimensional Self Attention Based Approach for Remaining Useful Life Estimation. arXiv:2212.05772.
[16] Dai, E. and Chen, J. (2022) Graph-Augmented Normalizing Flows for Anomaly Detection of Multiple Time Series. arXiv:2202.07857.
[17] Zhou, Q., He, S., Liu, H., Chen, J. and Meng, W. (2024) Label-Free Multivariate Time Series Anomaly Detection. IEEE Transactions on Knowledge and Data Engineering, 36, 3166-3179. [Google Scholar] [CrossRef
[18] Vaswani, A., et al. (2017) Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 4-9 December 2017, 6000-6010.
[19] Bhatti, A.A. (2009) Reduced Order Multiport Parallel and Multidirectional Neural Associative Memories. Biological Cybernetics, 100, 395-407. [Google Scholar] [CrossRef] [PubMed]
[20] Virbitskaite, I.B., Bozhenkova, E.N. and Erofeev, E. (2015) Space-Time Viewpoints for Concurrent Processes Represented by Relational Structures. Proceedings of the 24th International Workshop on Concurrency, Specification and Programming, Rzeszow, 28-30 September 2015, 222-233.
[21] Pan, J., Lin, C., Nie, L., Liu, M. and Zhao, Y. (2024) Multimodal Spatiotemporal Aggregation for Point Cloud Accumulation. Journal of Visual Communication and Image Representation, 103, Article ID: 104243. [Google Scholar] [CrossRef
[22] Abdulaal, A., Liu, Z. and Lancewicki, T. (2021) Practical Approach to Asynchronous Multivariate Time Series Anomaly Detection and Localization. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 14-18 August 2021, 2485-2494. [Google Scholar] [CrossRef
[23] Hundman, K., Constantinou, V., Laporte, C., Colwell, I. and Soderstrom, T. (2018) Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, 19-23 August 2018, 387-395. [Google Scholar] [CrossRef
[24] Ruff, L., Vandermeulen, R.A., Görnitz, N., Binder, A., Müller, E., Müller, K.-R. and Kloft, M. (2019) Deep Semi-Supervised Anomaly Detection. arXiv:1906.02694.
[25] Goyal, S., Raghunathan, A., Jain, M., Simhadri, H.V. and Jain, P. (2020) DROCC: Deep Robust One-Class Classification. Proceedings International Conference on Machine Learning (ICML), 13-18 July 2020, 3711-3721.
[26] Audibert, J., Michiardi, P., Guyard, F., Marti, S. and Zuluaga, M.A. (2020) USAD: Unsupervised Anomaly Detection on Multivariate Time Series. Proceedings 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 6-10 July 2020, 3395-3404.