一种融合稀疏注意力机制和BlockDiagonal矩阵的xLSTM-Informer深度学习模型

A Deep Learning Model of xLSTM-Informer Fusing Sparse Attention Mechanism and BlockDiagonal Matrices

摘要: 在新能源高渗透率背景下，电力现货价格频繁波动，传统模型在处理这类非线性、非平稳时序数据时存在性能瓶颈。本文提出xLSTM-Informer模型，融合xLSTM、稀疏注意力机制(ProbSparse Attention)及BlockDiagonal结构压缩方法，构建出适应强波动市场特征的预测架构。其中xLSTM顺序堆叠了一个xLSTM乘性门控单元(mLSTM)和一个结构化LSTM (sLSTM)模块，并且将BlockDiagonal结构压缩方法运用到mLSTM中，更好地提取时序局部特征的注意力，旨在处理长序列建模时信息传递受限、参数冗余以及时序特征表达能力不足等问题。模型在输入数据归一化过程中联合采用层归一化与组归一化策略，增强对尺度异构数据的鲁棒性。实验采用甘肃省2024年上半年电力现货市场数据，在多种天气场景(晴天、大风、阴天、无风)下进行测试。结果表明，xLSTM-Informer在五项主要评价指标(RMSE, MAE, MAPE, TIC, R²)上均优于CNN、CNN-LSTM、Informer和iTransformer等基准模型。其中在典型晴天场景下，RMSE仅为8.863，MAPE为4.556%，R²高达0.987，展现出对极端市场变化的强适应能力和稳定性。

Abstract: Against the backdrop of high new energy penetration, electricity spot prices exhibit frequent fluctuations, and traditional models suffer from performance bottlenecks when modeling such nonlinear and non-stationary time series. This paper proposes the xLSTM-Informer model, which integrates xLSTM, ProbSparse Attention, and a BlockDiagonal matrix compression method to construct a prediction framework adapted to highly volatile market characteristics. Specifically, xLSTM sequentially stacks a multiplicative LSTM (mLSTM) unit and a structured LSTM (sLSTM) module, with the BlockDiagonal compression method incorporated into the mLSTM to better capture attention over local time-series features. This design addresses the limitations of long-sequence modeling, including constrained information propagation, parameter redundancy, and insufficient representation of temporal patterns. To enhance robustness against scale-heterogeneous data, the model combines layer normalization and group normalization during input data standardization. Experiments are conducted using electricity spot market data from Gansu Province in the first half of 2024, evaluated across multiple weather scenarios (sunny, windy, cloudy, calm). The results demonstrate that xLSTM-Informer outperforms baseline models such as CNN, CNN-LSTM, Informer, and iTransformer across five key evaluation metrics (RMSE, MAE, MAPE, TIC, R²). In the typical sunny scenario, the model achieves an RMSE of 8.863, MAPE of 4.556%, and R² of 0.987, verifying its strong adaptability and stability under extreme market variations.