面向时间序列预测的注意力池化回声状态网络

Attention-Pooled Echo State Network for Time Series Prediction

摘要: 回声状态网络(ESN)在时间序列预测领域具备优异的应用潜力。为进一步提升其对强非线性时间序列的预测性能，本文提出一种注意力池化回声状态网络(AP-ESN)。该模型在标准ESN的储备池与输出层之间引入注意力池化层，可自适应筛选并聚合储备池中的关键特征信息，有效提升模型预测精度。泄露率、输入尺度因子与谱半径是影响ESN预测性能的核心超参数。为此，本文提出交叉回环机制改进飞蛾扑火优化算法，优化其全局寻优策略，有效克服传统飞蛾扑火算法易陷入局部最优的缺陷。实验选取Mackey-Glass (MG)混沌时间序列和低频正弦序列发生器两组典型强非线性数据集开展验证。结果表明，所提AP-ESN模型在强非线性时间序列预测任务中具备更优越的拟合与预测性能。

Abstract: Echo State Networks (ESNs) exhibit promising potential for applications in the field of time series prediction. To further enhance their predictive performance for strongly nonlinear time series, this paper proposes an attention-pooled echo state network (AP-ESN). The proposed model introduces an attention pooling layer between the reservoir and the output layer of the standard ESN, which adaptively selects and aggregates critical feature information from the reservoir, thereby effectively improving the prediction accuracy of the model. The leakage rate, input scaling factor, and spectral radius are identified as the core hyperparameters that significantly influence the predictive performance of ESNs. To address this issue, this paper incorporates a cross-feedback mechanism to improve the moth-flame optimization algorithm, enhancing its global search strategy and effectively overcoming the drawback of the traditional moth-flame optimization algorithm, which tends to become trapped in local optima. Experimental validation is conducted using two representative strongly nonlinear datasets, namely the Mackey-Glass (MG) chaotic time series and a low-frequency sinusoidal series. The results show that the proposed AP-ESN model has superior fitting and prediction performance in predicting strongly nonlinear time series.