分布式时滞系统LQR最优控制问题的PI算法

PI Algorithm for LQR Optimal Control Problem of Distributed Time-Delay System

摘要: 本文将基于模型的策略迭代方法推广到了分布式时滞系统的线性二次最优控制问题(LQR)的求解，证明了由该迭代方法得到的性能指标是递减的，且控制器收敛于最优控制器。

Abstract: This paper extends the model-based policy iteration method to the solution of the Linear Quadratic Regulator (LQR) problem for distributed delayed systems. It is demonstrated that the performance criterion obtained through this iterative method is monotonically decreasing, and the controller converges to the optimal controller.

1. 引言

最优控制理论自20世纪50年代发展以来，成为了现代控制理论中的一个重要分支。最优控制问题通常涉及在给定的动态系统和性能指标下，寻找一个控制策略，以实现系统性能的最优化。最优控制在航空航天、机器人、自动化、能源管理等领域[1] [2]具有广泛的应用价值。因此，如何设计有效的控制算法，尤其是线性二次最优控制(LQR)问题，一直是学术界和工程界研究的重点。

Kharitonov V [3]对各种时滞系统做出了详细的研究，线性二次最优控制问题[4] [5] (LQR)是一类经典的最优控制问题，目标是使得系统的状态和控制代价最小化。其优点在于通过求解相应的代数里卡蒂方程(Algebraic Riccati Equation, ARE)和最优控制律，可以得到明确且易于实现的控制策略。近年来，随着计算机技术和数学优化算法的发展，LQR的应用已扩展到多个复杂的工程系统中，如无人驾驶、智能电网、航空航天等领域。此外，分布式时滞系统的应用广泛，涵盖了从生物学、工程到经济学等多个领域。其核心特征是当前系统状态受到过去历史状态的影响，目前尤其在火箭发动机室燃料燃烧的稳定和控制方面[6]有着很重要的应用。

最近，基于自适应动态规划(ADP)技术[7]，已经为各种重要类别的线性、非线性或周期性动力系统以及最优稳定化、跟踪和输出调节问题提供了策略迭代(PI)和价值迭代(VI)方法[8] [9]。

受到Kleinman提出的无时滞线性系统基于模型的PI算法[10]的启发，本文为分布式时滞系统提出了一种新的基于模型的PI算法。在给定一个可容许的初始控制器的情况下，证明了每次迭代中更新的次优控制器的稳定性以及学习得到控制器序列收敛于(未知的)最优控制器。

2. 问题描述和预备工作

2.1. 问题模型介绍

考虑线性分布式时滞系统的形式

$\dot{x}\left(t\right)=Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle \underset{-h}{\overset{0}{\int }}E\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)}$ (1)

初值函数$x\left(\theta \right)=\phi \left(\theta \right)$ ，$\theta \in \left[-h,0\right]$ ，$h\in {ℝ}^{+}$ ，$x\left(t\right)\in {ℝ}^n$ ，$u\left(t\right)\in {ℝ}^m$ ，$m\le n$ ，$A,B\in {ℝ}^{n\times n}$ ，$D\in {ℝ}^{n\times m}$ ，$E\left(\theta \right)\in {ℝ}^{n\times n}$ ，并且这里$E\left(\theta \right)$ 是一个连续的矩阵函数当$\theta \in \left[-h,0\right]$ 时，设二次型性能指标如下：

$J={\displaystyle \underset{0}{\overset{\infty }{\int }}{x}^{\text{T}}\left(t\right)Qx\left(t\right)+u^{\text{T}}\left(t\right)Ru\left(t\right)\text{d}t}$ (2)

这里$Q\in {ℝ}^{n\times n}$ ，$R\in {ℝ}^{m\times m}$ ，$Q>0$ ，$R>0$ ，我们的目标是在给定任意一个初始函数$\phi$ 的情况下，寻找一个最优控制$u^{*}$ 使得(2)式的值最小。

定义1 如果一个控制$u\left(t\right)$ 使得系统(1)是全局渐近稳定(GAS)的，并且使得二次型性能指标(2)的值是有限的，对于所有的$x\left(\theta \right)=\phi \left(\theta \right)$ ，$\theta \in \left[-h,0\right]$ ，则称该控制$u\left(t\right)$ 是可容许的。

评论1：(2)式的性能指标是一种非常有效的形式，能够综合考虑系统状态的偏差(如误差)和控制输入的能量(如力度、功率)。这种设计理念非常符合许多实际工程问题的要求，即系统不仅需要保持稳定，还需要在满足性能要求的同时，减少过度的能量消耗。通过这个目标函数，LQR方法使得系统状态趋于理想值的同时，控制输入不会过大，避免了不必要的能量消耗。例如：我们希望系统能够较早地到达0的

平衡状态并且能量损耗不能过高，就可以采取(2)的形式作为性能指标，因为${\displaystyle \underset{0}{\overset{\infty }{\int }}{x}^{\text{T}}\left(t\right)Qx\left(t\right)\text{d}t}$ 可以理解为对系统状态的惩罚，由于Q的正定性，如果系统到达平衡态的时间越长，这个积分的值往往会越大，${\displaystyle \underset{0}{\overset{\infty }{\int }}{u}^{\text{T}}\left(t\right)Ru\left(t\right)\text{d}t}$ 可以理解为能量损耗的计算，因为R的正定性，只要能量在某一时间段内有损耗，这个积

分的值便会记录下这一时间段的损耗，使得(2)的值偏高。所以寻找一个方法来减小(2)的值有很重要的实际意义。

2.2. 主要定理

定理1 (Ross [11]) 如果存在一个可容许的控制$u^{*}=u^{*}\left(x_t\right)$ 和一个非负连续标量函数$V\left(x_t\right)$ (当$x_t=0$ 时$V\left(x_t\right)=0$ )。

满足以下两个条件：

${\dot{V}\left(x_t\right)|}_{u=u^{*}}+g\left(x_t,u^{*}\left(x_t\right)\right)=0,\forall t\ge 0,$ (3)

${\dot{V}\left(x_t\right)|}_{u=u^{*}}+g\left(x_t,u^{*}\left(x_t\right)\right)\le {\dot{V}\left(x_t\right)|}_{u=u_L\left(x_t\right)}+g\left(x_t,u_L\left(x_t\right)\right),\forall t\ge 0,$ (4)

这里g为(2)中被积函数，则$u^{*}$ 是使得二次型性能指标(2)的值最小的最优控制，此外$V\left(\phi \right)=J\left(\phi ,u^{*}\right)$ 。

证：对(3)式两端从0到$\infty$ 进行积分，并且考虑到可容许控制$u^{*}=u^{*}\left(x_t\right)$ 使得系统(1)渐近稳定，这意味着$V\left(\infty \right)=0$ ，从而得到$V\left(\phi \right)={\displaystyle \underset{0}{\overset{\infty }{\int }}g\left(x_t,u^{*}\left(x_t\right)\right)\text{d}t}=J\left(\phi ,u^{*}\right)$ 。

将(3)代入(4)我们可以得到${\dot{V}\left(x_t\right)|}_{u=u_L\left(x_t\right)}\ge -g\left(x_t,u_L\left(x_t\right)\right)$ ，然后对该等式两边从0到$\infty$ 积分，我们可以得到$V\left(\phi \right)\le {\displaystyle \underset{0}{\overset{\infty }{\int }}g\left(x_t,u_L\left(x_t\right)\right)}=J\left(\phi ,u\right)$ ，即$J\left(\phi ,u^{*}\right)\le J\left(\phi ,u\right)$ ，因此$u^{*}$ 是可容许控制中的最优控制。

定理2 线性控制$u^{*}\left(x_t\right)=-R^{-1}{D}^{\text{T}}{P}_{0}x\left(t\right)-R^{-1}{D}^{\text{T}}{\displaystyle \underset{-h}{\overset{0}{\int }}{P}_1}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta ,\forall t\ge 0$ 是最优控制。并且相应的最小二次性能指标为$V^{\ast }\left(x_0\right)=x^{\text{T}}\left(0\right)P_0^{\ast }x\left(0\right)+2x^{\text{T}}\left(0\right){\displaystyle \underset{-h}{\overset{0}{\int }}{P}_1^{\ast }}\left(\theta \right)x\left(\theta \right)\text{d}\theta +{\displaystyle \underset{-h}{\overset{0}{\int }}{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}}}\left(\xi \right)P_2^{\ast }\left(\xi ,\theta \right)x\left(\theta \right)\text{d}\xi \text{d}\theta$ 。

如果

$\begin{array}{l}{A}^{\text{T}}{P}_0+P_{0}A-P_{0}D{R}^{-1}{D}^{\text{T}}{P}_0+P_1^{\text{T}}\left(0\right)+P_1\left(0\right)+Q=0\\ \frac{\text{d}{P}_1\left(\theta \right)}{\text{d}\theta }=\left(A^{\text{T}}-P_{0}D{R}^{-1}{D}^{\text{T}}\right)P_1\left(\theta \right)+P_2\left(0,\theta \right)+P_{0}E\left(\theta \right),\theta \in \left[-h,0\right],\\ \frac{\partial P_2\left(\xi ,\theta \right)}{\partial \xi }+\frac{\partial P_2\left(\xi ,\theta \right)}{\partial \theta }=-P_1^{\text{T}}\left(\xi \right)D{R}^{-1}{D}^{\text{T}}{P}_1\left(\theta \right)+2E^{\text{T}}\left(\xi \right)P_1\left(\theta \right),\xi ,\theta \in \left[-h,0\right],\\ P_1\left(-h\right)=P_{0}B,\\ P_2\left(-h,\theta \right)=B^{\text{T}}{P}_1\left(\theta \right),\theta \in \left[-h,0\right].\end{array}$ (5)

证：我们取

$V\left(x_t\right)=x^{\text{T}}\left(t\right)P_{0}x\left(t\right)+2x^{\text{T}}\left(t\right){\displaystyle \underset{-h}{\overset{0}{\int }}{P}_1}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +{\displaystyle \underset{-h}{\overset{0}{\int }}{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}}}\left(t+\xi \right)P_2\left(\xi ,\theta \right)x\left(t+\theta \right)\text{d}\xi \text{d}\theta .$

这里$P_0$ 是正定矩阵，$P_2^{\text{T}}\left(\xi ,\theta \right)=P_2\left(\theta ,\xi \right)$ 。

$\begin{array}{l}\dot{V}\left(x_t\right)={\left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle \underset{-h}{\overset{0}{\int }}E\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)}\right)}^{\text{T}}{P}_{0}x\left(t\right)\\ +x^{\text{T}}\left(t\right)P_0\left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle \underset{-h}{\overset{0}{\int }}E\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)}\right)\\ +{\left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle \underset{-h}{\overset{0}{\int }}E\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)}\right)}^{\text{T}}{\displaystyle \underset{-h}{\overset{0}{\int }}{P}_1\left(\theta \right)x\left(t+\theta \right)\text{d}\theta }\\ +x^{\text{T}}\left(t\right){\displaystyle \underset{-h}{\overset{0}{\int }}{P}_1}\left(\theta \right)\frac{\partial x\left(t+\theta \right)}{\partial \theta }\text{d}\theta +{\displaystyle \underset{-h}{\overset{0}{\int }}\frac{\partial x^{\text{T}}\left(t+\theta \right)}{\partial \theta }{P}_1^{\text{T}}\left(\theta \right)\text{d}\theta x\left(t\right)}\\ +{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}\left(t+\theta \right)P_1^{\text{T}}}\left(\theta \right)\text{d}\theta \left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle \underset{-h}{\overset{0}{\int }}E\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)}\right)\\ +{\displaystyle \underset{-h}{\overset{0}{\int }}{\displaystyle \underset{-h}{\overset{0}{\int }}\frac{\partial x^{\text{T}}\left(t+\xi \right)}{\partial \xi }{P}_2\left(\xi ,\theta \right)x\left(t+\theta \right)\text{d}\xi \text{d}\theta }}+{\displaystyle \underset{-h}{\overset{0}{\int }}{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}\left(t+\xi \right)P_2\left(\xi ,\theta \right)}}\frac{\partial x\left(t+\theta \right)}{\partial \theta }\text{d}\xi \text{d}\theta \end{array}$

$\begin{array}{l}\dot{V}\left(x_t\right)+x^{\text{T}}\left(t\right)Qx\left(t\right)+u^{\text{T}}\left(t\right)Ru\left(t\right)\\ =x^{\text{T}}\left(t\right)\left(A^{\text{T}}{P}_0+P_{0}A+Q\right)x\left(t\right)+2x^{\text{T}}\left(t-h\right)B^{\text{T}}{P}_{0}x\left(t\right)+2x^{\text{T}}\left(t\right){\displaystyle \underset{-h}{\overset{0}{\int }}{A}^{\text{T}}}{P}_1\left(\theta \right)x\left(t+\theta \right)\text{d}\theta \\ +2x^{\text{T}}\left(t-h\right){\displaystyle \underset{-h}{\overset{0}{\int }}{B}^{\text{T}}}{P}_1\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +2x^{\text{T}}\left(t\right){\displaystyle \underset{-h}{\overset{0}{\int }}{P}_1\left(\theta \right)}\frac{\partial x\left(t+\theta \right)}{\partial \theta }\text{d}\xi \text{d}\theta \\ +{\displaystyle \underset{-h}{\overset{0}{\int }}{\displaystyle \underset{-h}{\overset{0}{\int }}\frac{\partial x^{\text{T}}\left(t+\xi \right)}{\partial \xi }{P}_2\left(\xi ,\theta \right)x\left(t+\theta \right)\text{d}\xi \text{d}\theta }}+{\displaystyle \underset{-h}{\overset{0}{\int }}{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}\left(t+\xi \right)P_2\left(\xi ,\theta \right)}}\frac{\partial x\left(t+\theta \right)}{\partial \theta }\text{d}\xi \text{d}\theta \\ +2{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}}\left(t+\theta \right)E^{\text{T}}\left(\xi \right)P_0\text{d}\theta x\left(t\right)+2{\displaystyle \underset{-h}{\overset{0}{\int }}{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}}}\left(t+\xi \right)E^{\text{T}}\left(\xi \right)P_1\left(\theta \right)x\left(t+\theta \right)\text{d}\xi \text{d}\theta \\ +2u^{\text{T}}\left(t\right){\displaystyle \underset{-h}{\overset{0}{\int }}{D}^{\text{T}}}{P}_1\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +2x^{\text{T}}\left(t\right)P_{0}Du\left(t\right)+u^{\text{T}}\left(t\right)Ru\left(t\right)\end{array}$

由(4)得$\frac{\partial \left(\dot{V}\left(x_t\right)+x^{\text{T}}\left(t\right)Qx\left(t\right)+u^{\text{T}}\left(t\right)Ru\left(t\right)\right)}{\partial u}=0$，

即$2D^{\text{T}}{P}_{0}x\left(t\right)+2D^{\text{T}}{\displaystyle \underset{-h}{\overset{0}{\int }}{P}_1}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +2Ru\left(t\right)=0$ ，

所以

$u^{*}\left(t\right)=-R^{-1}{D}^{\text{T}}{P}_{0}x\left(t\right)-R^{-1}{D}^{\text{T}}{\displaystyle \underset{-h}{\overset{0}{\int }}{P}_1}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta$ (6)

此外$\frac{{\partial }^2\left(\dot{V}\left(x_t\right)+x^{\text{T}}\left(t\right)Qx\left(t\right)+u^{\text{T}}\left(t\right)Ru\left(t\right)\right)}{\partial u^2}=2R>0$，所以我们可以得到$u^{*}\left(t\right)$

是$\dot{V}\left(x_t\right)+x^{\text{T}}\left(t\right)Qx\left(t\right)+u^{\text{T}}\left(t\right)Ru\left(t\right)$的局部最小解。

对于(3)令$t=0$ ，则$x\left(0\right)=\phi \left(0\right)$

可得${\dot{V}\left(\phi \right)|}_{u=u^{\ast }}+{\phi }^{\text{T}}\left(0\right)Q\phi \left(0\right)+u^{{\ast }^{\text{T}}}\left(t\right)R{u}^{\ast }\left(t\right)=0$。

将(6)代入上式得：

$\begin{array}{l}{\phi }^{\text{T}}\left(0\right)\left(A^{\text{T}}{P}_0+P_{0}A-P_{0}D{R}^{-1}{D}^{\text{T}}{P}_0+P_1^{\text{T}}\left(0\right)+P_1\left(0\right)+Q\right){\phi }^{\text{T}}\left(0\right)+2{\phi }^{\text{T}}\left(0\right)\left(P_{0}B-P_1\left(-h\right)\right)\phi \left(-h\right)\\ +2{\phi }^{\text{T}}\left(0\right){\displaystyle {\int }_{-h}^0\left(P_{0}E\left(\theta \right)-P_{0}D{R}^{-1}{D}^{\text{T}}{P}_1\left(\theta \right)+A^{\text{T}}{P}_1\left(\theta \right)-\frac{\text{d}{P}_1\left(\theta \right)}{\text{d}\theta }+P_2\left(0,\theta \right)\right)\phi \left(\theta \right)\text{d}\theta }\\ +2{\phi }^{\text{T}}\left(-h\right){\displaystyle {\int }_{-h}^0\left(B^{\text{T}}{P}_1\left(\theta \right)-P_2\left(-h,\theta \right)\right)\phi \left(\theta \right)\text{d}\theta }\\ +{\displaystyle {\int }_{-h}^0{\displaystyle {\int }_{-h}^0{\phi }^{\text{T}}}}\left(\xi \right)\left(-\frac{\partial P_2\left(\xi ,\theta \right)}{\partial \xi }-\frac{\partial P_2\left(\xi ,\theta \right)}{\partial \theta }-P_1^{\text{T}}\left(\xi \right)D{R}^{-1}{D}^{\text{T}}{P}_1\left(\theta \right)+2E^{\text{T}}\left(\xi \right)P_1\left(\theta \right)\right)\phi \left(\theta \right)\text{d}\xi \text{d}\theta =0\end{array}$

对于任意的$\phi$ 均成立，所以：

$\begin{array}{l}{A}^{\text{T}}{P}_0+P_{0}A-P_{0}D{R}^{-1}{D}^{\text{T}}{P}_0+P_1^{\text{T}}\left(0\right)+P_1\left(0\right)+Q=0\\ \frac{\text{d}{P}_1\left(\theta \right)}{\text{d}\theta }=\left(A^{\text{T}}-P_{0}D{R}^{-1}{D}^{\text{T}}\right)P_1\left(\theta \right)+P_2\left(0,\theta \right)+P_{0}E\left(\theta \right),\theta \in \left[-h,0\right],\\ \frac{\partial P_2\left(\xi ,\theta \right)}{\partial \xi }+\frac{\partial P_2\left(\xi ,\theta \right)}{\partial \theta }=-P_1^{\text{T}}\left(\xi \right)D{R}^{-1}{D}^{\text{T}}{P}_1\left(\theta \right)+2E^{\text{T}}\left(\xi \right)P_1\left(\theta \right),\xi \text{,}\theta \in \left[-h,0\right],\\ P_1\left(-h\right)=P_{0}B,\\ P_2\left(-h,\theta \right)=B^{\text{T}}{P}_1\left(\theta \right),\theta \in \left[-h,0\right].\end{array}$ (7)

证毕。

3. 基于模型的策略迭代

根据定理2，如果我们能求解偏微分方程组(7)，我们就能得到最优控制器。但是由于(7)是非线性偏微分方程组，直接求解往往比较困难。因此我们提出了基于模型的PI算法来简化(7)的求解。首先我们给定一个初始的可容许的控制器$u_1\left(x_t\right)=-K_{0,1}x\left(t\right)-{\displaystyle {\int }_{-h}^0{K}_{1,1}}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta$ ，基于模型的PI算法过程如下：

1. 策略评估

$i\in {\mathbb{N}}_{+},\xi ,\theta \in \left[-h,0\right],P_{0,i}=P_{0,i}^{\text{T}}>0,P_{1,i}\left(\theta \right),P_{2,i}^{\text{T}}\left(\theta ,\xi \right)=P_{2,i}\left(\xi ,\theta \right)$ ，解以下PDEs：

$\begin{array}{l}{A}_i^{\text{T}}{P}_{0,i}+P_{0,i}{A}_i+Q_i+P_{1,i}\left(0\right)+P_{1,i}^{\text{T}}\left(0\right)=0,\\ \frac{\text{d}{P}_{1,i}\left(\theta \right)}{\text{d}\theta }=A_i^{\text{T}}{P}_{1,i}\left(\theta \right)-P_{0,i}D{K}_{1,i}\left(\theta \right)+K_{0,i}^{\text{T}}R{K}_{1,i}\left(\theta \right)+P_{2,i}\left(0,\theta \right)+P_{0,i}E\left(\theta \right),\\ \frac{\partial P_{2,i}\left(\xi ,\theta \right)}{\partial \xi }+\frac{\partial P_{2,i}\left(\xi ,\theta \right)}{\partial \theta }=K_{1,i}^{\text{T}}\left(\xi \right)R{K}_{1,i}\left(\theta \right)-2K_{1,i}^{\text{T}}\left(\xi \right)D^{\text{T}}{P}_{1,i}\left(\theta \right)+2E^{\text{T}}\left(\xi \right)P_{1,i}\left(\theta \right),\\ P_{1,i}\left(-h\right)=P_{0,i}B,\\ P_{2,i}\left(-h,\theta \right))=B^{\text{T}}{P}_{1,i}\left(\theta \right),\end{array}$ (8)

在这里$A_i=A-D{K}_{0,i}$ ，$Q_i=Q+K_{0,i}^{\text{T}}R{K}_{0,i}$ 。

2. 策略改进

$\begin{array}{l}{u}_{i+1}\left(x_t\right)=-K_{0,i+1}x\left(t\right)-{\displaystyle \underset{-h}{\overset{0}{\int }}{K}_{1,i+1}}\left(\theta \right)x_t\left(\theta \right)\text{d}\theta \\ =-R^{-1}{D}^{\text{T}}{P}_{0,i}x\left(t\right)-R^{-1}{D}^{\text{T}}{\displaystyle \underset{-h}{\overset{0}{\int }}{P}_{1,i}}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta .\end{array}$ (9)

用该公式更新控制器，并且里面也隐含了K和P之间的关系。二次性能指标$V_i\left(x_0\right)=J\left(x_0,u_i\right)$ ，

$V_i\left(x_0\right)=x^{\text{T}}\left(0\right)P_{0,i}x\left(0\right)+2x^{\text{T}}\left(0\right){\displaystyle \underset{-h}{\overset{0}{\int }}{P}_{1,i}}\left(\theta \right)x\left(\theta \right)\text{d}\theta +{\displaystyle \underset{-h}{\overset{0}{\int }}{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}}}\left(\xi \right)P_{2,i}\left(\xi ,\theta \right)x\left(\theta \right)\text{d}\xi \text{d}\theta .$

评论2：我们对里卡蒂方程的中$P_1^{\text{T}}\left(\xi \right)D{R}^{-1}{D}^{\text{T}}{P}_1\left(\theta \right)$ 等非线性部分进行了修改，在第一次迭代中，由于我们给定了一个初始的可容许控制器$u_1\left(x_t\right)$ ，这就意味着$K_{0,1},K_{1,1}\left(\theta \right)$ 是已知的，代入到策略评估的过程中，我们可以求解得到$P_{0,1},P_{1,1}\left(\theta \right),P_{2,1}\left(\xi ,\theta \right)$ ，根据K和P的关系(也就是策略改进过程)，我们立刻能得到一组新的$K_{0,2},K_{1,2}\left(\theta \right)$ ，然后继续代入策略评估过程，求解可得一组新的下标 + 1的P，依此一直进行下去，每次策略评估过程中的K都是由上一次迭代得到的计算结果，所以这样就能把难于求解的里卡蒂方程，转变为求解一系列非线性偏微方程组。

定理3给定一个初始的可容许控制器$u_1\left(x_t\right)$ ，对于$\forall$ $i\in {\mathbb{N}}_{+}$ ，通过(8)(9)求解的$P_{0,i}$ ，$P_{1,i}\left(\theta \right)$ ，$P_{2,i}\left(\xi ,\theta \right)$ 和$u_{i+1}\left(x_t\right)$ ，有以下性质成立。

1) $V^{*}\left(x_0\right)\le V_{i+1}\left(x_0\right)\le V_i\left(x_0\right)$ ；

2) $u_{i+1}$ 是可容许的控制；

3) $V_i\left(x_0\right),u_i\left(x_t\right)$ 分别收敛于$V^{*}\left(x_0\right)$ 和$u^{*}\left(x_t\right)$ 。

证明见附录。

4. 结论和思考

本文对分布式时滞系统的线性二次最优控制问题中最优控制器的求解进行了详细的推导，并得到了最优控制器中参数所满足的里卡蒂方程的表达式，而且这类里卡蒂方程的求解往往相当复杂，一般在只有一些特殊情况才能得到解析解。所以本文为了避免此类方程的求解，构造了一个与此方程相关的策略迭代，这极大地简化了求解的过程，并且多次迭代后得到的控制器是收敛于我们理论上的最优控制器。此类策略迭代能为各种时滞系统LQR最优控制器的求解简化提供一个参考。

基金项目

山东省自然科学基金(面上基金) ZR2024MA097。

附 录

${ℳ}_2={ℝ}^n\oplus L_2\left(\left[-h,0\right],{ℝ}^n\right)$ ，$ℒ\left(X\right)$ 表示从X映射到X的有界线性算子， $z_i=\left[r_i,f_i(·)\right],\langle z_1,z_2\rangle =r_1^{\text{T}}{r}_2+{\displaystyle \underset{-h}{\overset{0}{\int }}{f}_1^{\text{T}}\left(\theta \right)f_2}\left(\theta \right)\text{d}\theta$ 。

定理3的证明

$V_i\left(x_t\right)=x^{\text{T}}\left(t\right)P_{0,i}x\left(t\right)+2x^{\text{T}}\left(t\right){\displaystyle \underset{-h}{\overset{0}{\int }}{P}_{1,i}}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +{\displaystyle \underset{-h}{\overset{0}{\int }}{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}}}\left(t+\xi \right)P_{2,i}\left(\xi ,\theta \right)x\left(t+\theta \right)\text{d}\xi \text{d}\theta$

首先我们计算${\dot{V}}_i\left(x_t\right)$ 的展开式

$\begin{array}{l}{\dot{V}}_i\left(x_t\right)={\left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle {\int }_{-h}^{0}E}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)\right)}^{\text{T}}{P}_{0,i}x\left(t\right)\\ +x^{\text{T}}\left(t\right)P_{0,i}\left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle {\int }_{-h}^{0}E}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)\right)\\ +2{\left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle {\int }_{-h}^{0}E}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)\right)}^{\text{T}}{\displaystyle {\int }_{-h}^0{P}_{1,i}}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta \\ +2x^{\text{T}}\left(t\right){\displaystyle {\int }_{-h}^0{P}_{1,i}}\left(\theta \right)\frac{\text{d}x\left(t+\theta \right)}{\text{d}t}\text{d}\theta +{\displaystyle {\int }_{-h}^0{\displaystyle {\int }_{-h}^0\frac{\text{d}{x}^{\text{T}}\left(t+\xi \right)}{\text{d}t}{P}_{2,i}\left(\xi ,\theta \right)x\left(\theta \right)\text{d}\xi \text{d}\theta }}\\ +{\displaystyle {\int }_{-h}^0{\displaystyle {\int }_{-h}^0{x}^{\text{T}}}}\left(t+\xi \right)P_{2,i}\left(\xi ,\theta \right)\frac{\text{d}x\left(t+\theta \right)}{\text{d}t}\text{d}\xi \text{d}\theta \end{array}$

$\begin{array}{l}={\left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle {\int }_{-h}^{0}E}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)\right)}^{\text{T}}{P}_{0,i}x\left(t\right)\\ +x^{\text{T}}\left(t\right)P_{0,i}\left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle {\int }_{-h}^{0}E}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)\right)\\ +2{\left(Ax\left(t\right)+Bx\left(t-h\right)+{\displaystyle {\int }_{-h}^{0}E}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta +Du\left(t\right)\right)}^{\text{T}}{\displaystyle {\int }_{-h}^0{P}_{1,i}}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta \\ +{2x^{\text{T}}\left(t\right)P_{1,i}\left(\theta \right)x\left(t+\theta \right)|}_{\theta =-h}^{\theta =0}-2x^{\text{T}}\left(t\right){\displaystyle {\int }_{-h}^0\frac{\text{d}{P}_{1,i}\left(\theta \right)}{\text{d}\theta }x\left(t+\theta \right)\text{d}\theta }\\ +{{\displaystyle {\int }_{-h}^0{x}^{\text{T}}}\left(t+\xi \right)P_{2,i}\left(\xi ,\theta \right)x\left(t+\theta \right)\text{d}\theta |}_{\xi =-h}^{\xi =0}-{\displaystyle {\int }_{-h}^0{\displaystyle {\int }_{-h}^0{x}^{\text{T}}}}\left(t+\xi \right)\frac{\partial P_{2,i}\left(\xi ,\theta \right)}{\partial \xi }x\left(t+\theta \right)\text{d}\xi \text{d}\theta \end{array}$

$\begin{array}{l}=x^{\text{T}}\left(t\right)\left(A^{\text{T}}{P}_{0,i}+P_{0,i}A+P_{1,i}\left(0\right)+P_{1,i}^{\text{T}}\left(0\right)\right)x\left(t\right)+2x^{\text{T}}\left(t\right)\left(P_{0,i}B-P_{1,i}\left(-h\right)\right)x\left(t-h\right)\\ +2x^{\text{T}}\left(t\right){\displaystyle {\int }_{-h}^0\left(P_{0,i}E\left(\theta \right)+A^{\text{T}}{P}_{1,i}\left(\theta \right)+P_{2,i}\left(0,\theta \right)-\frac{\text{d}{P}_{1,i}\left(\theta \right)}{\text{d}\theta }\right)x\left(t+\theta \right)\text{d}\theta }\\ +2x^{\text{T}}\left(t-h\right){\displaystyle {\int }_{-h}^0\left(B^{\text{T}}{P}_{1,i}\left(\theta \right)-P_{2,i}\left(-h,\theta \right)\right)x\left(t+\theta \right)\text{d}\theta }\\ -{\displaystyle {\int }_{-h}^0{\displaystyle {\int }_{-h}^0{x}^{\text{T}}}}\left(t+\xi \right)\left(\frac{\partial P_{2,i}\left(\xi ,\theta \right)}{\partial \theta }+\frac{\partial P_{2,i}\left(\xi ,\theta \right)}{\partial \xi }+2E^{\text{T}}\left(\xi \right)P_{1,i}\left(\theta \right)\right)x\left(t+\theta \right)\text{d}\xi \text{d}\theta \\ +2u^{\text{T}}\left(t\right)D^{\text{T}}{P}_{0,i}x\left(t\right)+2u^{\text{T}}\left(t\right){\displaystyle {\int }_{-h}^0{D}^{\text{T}}}{P}_{1,i}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta \end{array}$

$\begin{array}{l}=x^{\text{T}}\left(t\right)\left(-Q-K_{0,i}^{\text{T}}R{K}_{0,i}+K_{0,i}^{\text{T}}{D}^{\text{T}}{P}_{0,i}+P_{0,i}D{K}_{0,i}\right)x\left(t\right)\\ +2x^{\text{T}}\left(t\right){\displaystyle {\int }_{-h}^0\left(P_{0,i}D{K}_{1,i}\left(\theta \right)-K_{0,i}^{\text{T}}R{K}_{1,i}\left(\theta \right)+K_{0,i}^{\text{T}}{D}^{\text{T}}{P}_{1,i}\left(\theta \right)\right)x\left(t+\theta \right)\text{d}\theta }\\ -{\displaystyle {\int }_{-h}^0{\displaystyle {\int }_{-h}^0{x}^{\text{T}}}}\left(t+\xi \right)\left(K_{1,i}^{\text{T}}\left(\xi \right)R{K}_{1,i}\left(\theta \right)-2K_{1,i}^{\text{T}}\left(\xi \right)D^{\text{T}}{P}_{1,i}\left(\theta \right)\right)x\left(t+\theta \right)\text{d}\xi \text{d}\theta \\ +2u^{\text{T}}\left(t\right)D^{\text{T}}{P}_{0,i}x\left(t\right)+2u^{\text{T}}\left(t\right){\displaystyle {\int }_{-h}^0{D}^{\text{T}}}{P}_{1,i}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta \end{array}$

记${\displaystyle \underset{-h}{\overset{0}{\int }}{R}^{-1}}{D}^{\text{T}}{P}_{1,i}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta ={\displaystyle \underset{-h}{\overset{0}{\int }}{K}_{1,i+1}}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta =m_{i+1}$ ，

则$u_{i+1}=-R^{-1}{D}^{\text{T}}{P}_{0,i}x\left(t\right)-m_{i+1}=-K_{0,i+1}x\left(t\right)-m_{i+1}$ ，并且注意到$R^{-1}R=E$ ，所以我们可以在适当的地方添加$R^{-1}R$ 。

则可得到：

$\begin{array}{l}{x}^{\text{T}}\left(t\right)\left(-Q-K_{0,i}^{\text{T}}R{K}_{0,i}+K_{0,i}^{\text{T}}{D}^{\text{T}}{P}_{0,i}+P_{0,i}D{K}_{0,i}\right)x\left(t\right)\\ =-x^{\text{T}}Qx+{\left(u_i+m_i\right)}^{\text{T}}R\left(-u_i-m_i\right)+{\left(-u_i-m_i\right)}^{\text{T}}R{R}^{-1}{D}^{\text{T}}{P}_{0,i}x+x^{\text{T}}{K}_{0,i}^{\text{T}}R{R}^{-1}{D}^{\text{T}}{P}_{0,i}x\\ =-x^{\text{T}}Qx+{\left(u_i+m_i\right)}^{\text{T}}R\left(-u_i-m_i\right)+{\left(-u_i-m_i\right)}^{\text{T}}R\left(-u_{i+1}-m_{i+1}\right)+{\left(-u_i-m_i\right)}^{\text{T}}R\left(-u_{i+1}-m_{i+1}\right)\\ =-x^{\text{T}}Qx-u_i^{\text{T}}R{u}_i-m_i^{\text{T}}R{m}_i-2u_i^{\text{T}}R{m}_i+2u_i^{\text{T}}R{u}_{i+1}+2u_i^{\text{T}}R{m}_{i+1}+2m_i^{\text{T}}R{u}_{i+1}+2m_i^{\text{T}}R{m}_{i+1}\end{array}$

$\begin{array}{l}2x^{\text{T}}\left(t\right){\displaystyle {\int }_{-h}^0\left(P_{0,i}D{K}_{1,i}\left(\theta \right)-K_{0,i}^{\text{T}}R{K}_{1,i}\left(\theta \right)+K_{0,i}^{\text{T}}{D}^{\text{T}}{P}_{1,i}\left(\theta \right)\right)x\left(t+\theta \right)\text{d}\theta }\\ =2x^{\text{T}}{P}_{0,i}B{\left(R^{\text{T}}\right)}^{-1}R{m}_i-2x^{\text{T}}{K}_{0,i}^{\text{T}}R{m}_i+2x^{\text{T}}{K}_{0,i}^{\text{T}}R{\displaystyle \underset{-h}{\overset{0}{\int }}{R}^{-1}{D}^{\text{T}}{P}_{1,i}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta }\\ =-2{\left(u_{i+1}+m_{i+1}\right)}^{\text{T}}R{m}_i+2{\left(u_i+m_i\right)}^{\text{T}}R{m}_i-2{\left(u_i+m_i\right)}^{\text{T}}R{m}_{i+1}\\ =-2u_{i+1}^{\text{T}}R{m}_i-2m_{i+1}^{\text{T}}R{m}_i+2u_i^{\text{T}}R{m}_i+2m_i^{\text{T}}R{m}_i-2u_i^{\text{T}}R{m}_{i+1}-2m_i^{\text{T}}R{m}_{i+1}\end{array}$

$\begin{array}{l}-{\displaystyle {\int }_{-h}^0{\displaystyle {\int }_{-h}^0{x}^{\text{T}}}}\left(t+\xi \right)\left(K_{1,i}^{\text{T}}\left(\xi \right)R{K}_{1,i}\left(\theta \right)-2K_{1,i}^{\text{T}}\left(\xi \right)D^{\text{T}}{P}_{1,i}\left(\theta \right)\right)x\left(t+\theta \right)\text{d}\xi \text{d}\theta \\ =-{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}}\left(t+\xi \right)K_{1,i}^{\text{T}}\left(\xi \right)\text{d}\xi R{\displaystyle \underset{-h}{\overset{0}{\int }}{K}_{1,i}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta }\\ +2{\displaystyle \underset{-h}{\overset{0}{\int }}{x}^{\text{T}}}\left(t+\xi \right)K_{1,i}^{\text{T}}\left(\xi \right)\text{d}\xi R{R}^{-1}{D}^{\text{T}}{\displaystyle \underset{-h}{\overset{0}{\int }}{P}_{1,i}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta }\\ =-m_i^{\text{T}}R{m}_i+2m_i^{\text{T}}R{m}_{i+1}\end{array}$

$\begin{array}{l}2u^{\text{T}}\left(t\right)D^{\text{T}}{P}_{0,i}x\left(t\right)+2u^{\text{T}}\left(t\right){\displaystyle {\int }_{-h}^0{D}^{\text{T}}}{P}_{1,i}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta \\ =2u^{\text{T}}\left(t\right)R{R}^{-1}\left[D^{\text{T}}{P}_{0,i}x\left(t\right)+{\displaystyle {\int }_{-h}^0{D}^{\text{T}}}{P}_{1,i}\left(\theta \right)x\left(t+\theta \right)\text{d}\theta \right]\\ =-2u^{\text{T}}\left(t\right)R{u}_{i+1}\end{array}$

将以上得到的4处结果合并可得

${\dot{V}}_i\left(x_t\right)=-x^{\text{T}}\left(t\right)Qx\left(t\right)-u_i^{\text{T}}\left(t\right)R{u}_i\left(t\right)+2u_{i+1}^{\text{T}}R{u}_i-2u^{\text{T}}\left(t\right)R{u}_{i+1}$ (10)

当$i=1$ 时，系统(1)被可容许控制$u_1\left(x_t\right)$ 驱动时，根据(10)，我们有${\dot{V}}_1\left(x_t\right)=-x^{\text{T}}Qx-u_1^{\text{T}}R{u}_1$，

对上式从0到$\infty$ 积分得：$V_1\left(x_0\right)={\displaystyle \underset{0}{\overset{\infty }{\int }}{x}^{\text{T}}}\left(t\right)Qx\left(t\right)+u_1^{\text{T}}\left(t\right)R{u}_1\left(t\right)\text{d}t=J\left(x_0,u_1\right)<\infty$ 。

接下来我们使用$u_2\left(x_t\right)$ 驱动系统(1)，根据(10)式，可得：

$V_1\left(x_t\right)=-x^{\text{T}}Qx-u_1^{\text{T}}R{u}_1-2u_2^{\text{T}}R\left(u_2-u_1\right)=-x^{\text{T}}Qx-u_2^{\text{T}}R{u}_2-{\left(u_2-u_1\right)}^{\text{T}}R\left(u_2-u_1\right)$

对上式两端从0到$\infty$ 积分得：

$J\left(x_0,u_2\right)=V_1\left(x_0\right)-V_1\left(x_{\infty }\right)-{\displaystyle {\int }_0^{\infty }{\left(u_2-u_1\right)}^{\text{T}}}R\left(u_2-u_1\right)\text{d}t\le V_1\left(x_0\right)<\infty$ (11)

所以$u_2\left(x_t\right)$ 是全局渐近稳定的控制器，所以$u_2\left(x_t\right)$ 是可容许的控制器。

令(10)中的$i=2$ 并把$u^{\text{T}}\left(t\right)$ 替换为$u_2^{\text{T}}\left(t\right)$ 可得：

${\dot{V}}_2\left(x_t\right)=-x^{\text{T}}Qx-u_2^{\text{T}}R{u}_2$

对该式从0到$\infty$ 积分得：$J\left(x_0\text{,}{u}_2\right)=V_2\left(x_0\right)$ 。根据(11)我们有$V_2\left(x_0\right)\le V_1\left(x_0\right)$ 。

当$i>1$ 时，假设1)和2)成立。当系统(1)被$u_i$ 驱动时，${\dot{V}}_i\left(x_t\right)=-x^{\top }Qx-u_i^{\top }R{u}_i$ ，对该式从0到$\infty$ 积分得：

$V_i\left(x_0\right)={\displaystyle {\int }_0^{\infty }{x}^{\top }}\left(t\right)Qx\left(t\right)+u_i^{\top }\left(t\right)R{u}_i\left(t\right)\text{d}t=J\left(x_0,u_i\right)<\infty .$

当系统(1)被$u_i$ 驱动时，并把$u^{\text{T}}\left(t\right)$ 替换为$u_{i+1}^{\text{T}}$ 可得：

${\dot{V}}_i\left(x_t\right)=-x^{\top }Qx-u_{i+1}^{\top }R{u}_{i+1}-{\left(u_{i+1}-u_i\right)}^{\top }R\left(u_{i+1}-u_i\right).$

对两端从0到$\infty$ 积分得：$J\left(x_0,u_{i+1}\right)=V_i\left(x_0\right)-V_i\left(x_{\infty }\right)-{\displaystyle {\int }_0^{\infty }{\left(u_{i+1}-u_i\right)}^{\top }}R\left(u_{i+1}-u_i\right)\text{d}t\le V_i\left(x_0\right)<\infty$ 。

所以$u_{i+1}$ 是全局渐近稳定的控制器，$u_{i+1}$ 是可容许的控制。2)得证。

当系统(1)被$u_{i+1}$ 驱动时，通过(10)可得${\dot{V}}_{i+1}\left(x_t\right)=-x^{\text{T}}Qx-u_{i+1}^{\text{T}}R{u}_{i+1}$。

同理可得$V_{i+1}\left(x_0\right)\le V_i\left(x_0\right)$ ，因为$V^{*}\left(x_0\right)=J\left(x_0,u^{*}\right)$ 是最小值，所以$V^{*}\left(x_0\right)\le V_i\left(x_0\right)$ 。

因此1) 2)得证。

所以我们有$V^{*}\left(x_0\right)\le \cdots \le V_i\left(x_0\right)\le \cdots \le V_1\left(x_0\right)$ 。

定义$z_0=\left[\begin{array}{c}{x}_0\left(0\right)\\ x_0(·)\end{array}\right]$ ，${{\rm P}}_i{z}_0=\left[\begin{array}{c}{P}_{0,i}x\left(0\right)+{\displaystyle \underset{-h}{\overset{0}{\int }}{P}_{1,i}\left(\theta \right)x_0\left(\theta \right)\text{d}\theta }\\ {\displaystyle \underset{-h}{\overset{0}{\int }}{P}_{2,i}\left(·,\theta \right)x_0\left(\theta \right)\text{d}\theta +P_{1,i}^{\text{T}}(·)x\left(0\right)}\end{array}\right]$ 。

易知${{\rm P}}_i$ 是对称并且是正定的，并且$V_i\left(x_0\right)=\langle z_0,{{\rm P}}_i{z}_0\rangle$ ，此外$\forall i\in {\mathbb{N}}_{+}$ ，${{\rm P}}_{i+1}\le {{\rm P}}_i\le {{\rm P}}_{i-1}$ ，通过[12]定理6.3.2存在${{\rm P}}_p={{\rm P}}_p^{\text{T}}>0$ ，$\forall z_0\in {ℳ}_2$ ，有$\underset{i\to \infty }{\lim }{{\rm P}}_i{z}_0={{\rm P}}_p{z}_0$ 。因此$P_{0,i},P_{1,i}\left(\theta \right),P_{2,i}\left(\xi ,\theta \right)$ 逐点收敛于 $P_{0,p},P_{1,p}\left(\theta \right),P_{2,p}\left(\xi ,\theta \right)$ 。根据收敛的性质，我们有

$\underset{i\to \infty }{\lim }{P}_{0,i}=\underset{i\to \infty }{\lim }{P}_{0,i+1}=P_{0,p},\underset{i\to \infty }{\lim }{P}_{1,i}\left(\theta \right)=\underset{i\to \infty }{\lim }{P}_{1,i+1}\left(\theta \right)=P_{1,p}\left(\theta \right),\underset{i\to \infty }{\lim }{P}_{2,i}\left(\xi ,\theta \right)=\underset{i\to \infty }{\lim }{P}_{2,i+1}\left(\xi ,\theta \right)=P_{2,p}\left(\xi ,\theta \right)$

由(8)可得：

$\begin{array}{l}{\left(A-D{K}_{0,p}\right)}^{\text{T}}{P}_{0,p}+P_{0,p}\left(A-D{K}_{0,p}\right)+Q+K_{0,p}^{\text{T}}R{K}_{0,p}+P_{1,p}(0)+P_{1,p}^{\text{T}}\left(0\right)=0,\\ \frac{\text{d}{P}_{1,p}\left(\theta \right)}{\text{d}\theta }={\left(A-D{K}_{0,p}\right)}^{\text{T}}{P}_{1,p}\left(\theta \right)-P_{0,p}D{K}_{1,p}\left(\theta \right)+K_{0,p}^{\text{T}}R{K}_{1,p}\left(\theta \right)+P_{2,p}\left(0,\theta \right)+P_{0,p}E\left(\theta \right),\\ \frac{\partial P_{2,p}\left(\xi ,\theta \right)}{\partial \xi }+\frac{\partial P_{2,p}\left(\xi ,\theta \right)}{\partial \theta }=K_{1,p}^{\text{T}}\left(\xi \right)R{K}_{1,p}\left(\theta \right)-2K_{1,p}^{\text{T}}\left(\xi \right)D^{\text{T}}{P}_{1,p}\left(\theta \right)+2E^{\text{T}}\left(\xi \right)P_{1,p}\left(\theta \right),\\ P_{1,p}\left(-h\right)=P_{0,p}B,\\ P_{2,p}\left(-h,\theta \right)=B^{\text{T}}{P}_{1,p}\left(\theta \right),\end{array}$ (12)

$K_{0,p}=R^{-1}{D}^{\text{T}}{P}_{0,p},K_{1,p}\left(\theta \right)=R^{-1}{D}^{\text{T}}{P}_{1,p}\left(\theta \right)$ (13)

将(13)代入(12)，我们可以发现方程组的形式与(7)一致，所以可得$P_{0,p},P_{1,p}\left(\theta \right),P_{2,p}\left(\xi ,\theta \right)$ 是(7)的解，由于(7)解的唯一性，并且(7)的解为$P_0^{*},P_1^{*}\left(\theta \right),P_2^{*}\left(\xi ,\theta \right)$ ，所以$P_{0,i},P_{1,i}\left(\theta \right),P_{2,i}\left(\xi ,\theta \right)$ 逐点收敛于 $P_0^{*},P_1^{*}\left(\theta \right),P_2^{*}\left(\xi ,\theta \right)$ ，所以3)得证。在算法收敛速度方面，如[13]所示，PI算法的收敛率在希尔伯特空间中是二次的，因此，对分布式时滞系统提出的基于模型的PI具有相同的二次收敛率。

NOTES

^*通讯作者。