离散时间时变Markov跳变系统的线性二次最优控制

Linear Quadratic Optimal Control for Discrete Time Time-Varying Markov Jump Systems

摘要: 研究了离散时间时变Markov跳变系统在有限时域上的最优控制问题。证明了任何离散时间时变Markov跳变系统的最优控制问题都可以通过广义代数Riccati方程的解来确定；也证明了最优控制问题的可达性是广义代数Riccati方程解的存在性的充分条件；最后通过数值举例来验证结果的正确性。

Abstract: The optimal control of discrete-time time-varying Markov jump systems in infinite horizon is stud-ied. It is proved that the optimal control problem of any discrete time time-varying Markov jump system can be determined by the solution of generalized algebraic Riccati equation. It is also proved that the attainablility of the optimal control problem is a sufficient condition for the existence of so-lutions of generalized algebraic Riccati equations. Finally, numerical examples are given to verify the correctness of the results.

1. 引言

最优性是我们生活生产中所追求的目标之一。研究控制系统时，若系统是线性的，二次指标表示成状态变量 $x\left(t\right)$ 和输入控制变量 $u\left(t\right)$ 的二次函数，则把求得系统所对应的二次指标达到最优的控制问题，称为最优控制问题 [1] 。线性二次最优控制问题最早实在1958年由Bellman，Glicksberg，Gross等人研究的 [2] 。在此基础之上，R.E. Kalman针对二次最优控制建立了状态反馈 [3] ，并且将Riccati方程引入控制理论中，建立了最优线性反馈调节器 [4] [5] [6] [7] 。参考文献 [8] - [15] 中针对线性系统的系数问题作出假设，并且研究了任何的线性系统的稳定解可以用广义Riccati方程的可解性来表示 [16] [17] [18] 。在文献 [19] 中，为了保证二次线性控制问题是适定的和反馈稳定控制的存在性，要定义一些稳定性概念。

本文与参考文献 [20] 不同之处在于，本文考虑了具体非齐次马尔可夫切换的无限时域离散时间时变Markov跳变系统的线性二次最优控制，介绍了与KKT定理相关的最优线性反馈的存在性，且证明了广义Riccati方程的可解性与二次最优问题的适定性与可达性是等价的。要保证线性二次最优问题是适定的，则对应的广义Riccati方程是可解的，并且存在最优控制。在文章的第二节介绍了一些相关的定义和后续文章中所需要的一些引理；第三节对所研究的模型进行详细的描述，对模型中所出现的符号进行说明。

2. 预备知识

为了能够方便在后续文章中对一些相关性定理引理的应用，先对文章中将要用到的定理引理进行简单的介绍：

数学规划：

$\begin{array}{l}\min f\left(x\right)\\ \text{s}\text{.t}\text{.}\{\begin{array}{c}g\left(x\right)\le 0\\ h\left(x\right)=0\end{array}\end{array}$

其中：

$\begin{array}{l}g\left(x\right)=\left(g_1\left(x\right),g_2\left(x\right),\cdots ,g_m\left(x\right)\right)\\ h\left(x\right)=(h_1\left(x\right),h_2\left(x\right),\cdots ,h_m(\; x\; ))\end{array}$

定义1 (正则点)：对于数学规划中的可行点 $x^{\ast }$ ，令 $I^{\ast }=\left\{i|g_1\left(x^{\ast }\right)=0\right\}$ ，如果梯度向量 $\nabla g_i\left(x^{\ast }\right),i\in I^{\ast }$ 和 $\nabla h_j\left(x^{\ast }\right),j=1,2,\cdots ,n$ 都是线性独立的，则 $x^{\ast }$ 成为约束正则点。

定义2 (正则性条件)：令 $I^{\ast }=\left\{i|g_1\left(x^{\ast }\right)=0\right\}$ ，如果梯度向量 $\nabla g_i\left(x^{\ast }\right),i\in I^{\ast }$ 和 $\nabla h_j\left(x^{\ast }\right),j=1,2,\cdots ,n$ 都是线性独立的，则称之为正则性条件。

定义3 (KKT定理)：设正则点 $x^{\ast }$ 是数学规划的局部最优解，且目标函数 $f\left(x\right)$ 以及约束函数 $g\left(x\right)=\left(g_1\left(x\right),g_2\left(x\right),\cdots ,g_m\left(x\right)\right)$ ， $h\left(x\right)=\left(h_1\left(x\right),h_2\left(x\right),\cdots ,h_m\left(x\right)\right)$ ，在点 $x^{\ast }$ 处是连续可微的，则存在 $\lambda \in {ℝ}^m,\lambda \ge 0$ 及 $\mu \in {ℝ}^n$ 叫做KKT乘数，使得下列等式成立：

$\{\begin{array}{l}{\nabla }_x\mathcal{L}\left(x^{\ast },{\lambda }^{\ast },{\mu }^{\ast }\right)=0\\ {\lambda }^{\text{T}}g\left(x^{\ast }\right)=0\end{array}$

其中拉格朗日函数为： $\mathcal{L}\left(x,\lambda ,\mu \right)=f\left(x\right)+{\lambda }^{\text{T}}g\left(x\right)+{\mu }^{\text{T}}h(\; x\; )$

引理1 [12] ：给定矩阵 $Q\in {ℝ}^{m\times n}$ ，矩阵 $Q^{†}\in {ℝ}^{n\times m}$ 叫做Q的广义逆矩阵，则：

$\{\begin{array}{l}Q{Q}^{†}Q=Q,Q^{†}Q{Q}^{†}=Q^{†}\\ {\left(Q{Q}^{†}\right)}^{\text{T}}=Q{Q}^{†},{\left(Q^{†}Q\right)}^{\text{T}}=Q^{†}Q\end{array}$

引理2 [12] ：给定对称矩阵Q，则：

$\{\begin{array}{l}{\left(Q^{†}\right)}^{\text{T}}={\left(Q^{\text{T}}\right)}^{†},Q^{†}Q=Q{Q}^{†}\\ Q\ge 0,当且仅当Q^{†}\ge 0\end{array}$

引理3 (舒尔补引理) [13] ：设具有适当维数的矩阵 $Q^{\text{T}}=Q,N=N^{\text{T}},R=R^{\text{T}}$ ，则下面的条件是等价的：

(I) $Q-N{R}^{†}{N}^{\text{T}}\ge 0,R\ge 0,N\left(I-R{R}^{†}\right)=0$

(II) $\left[\begin{array}{cc}Q& N\\ N^{\text{T}}& R\end{array}\right]\ge 0$

(III) $\left[\begin{array}{cc}R& N^{\text{T}}\\ N& Q\end{array}\right]\ge 0$

引理4 [14] ：给定矩阵 $Q,K,L$ ，当且仅当 $Q,K,L$ ，当且仅当 $Q{Q}^{†}LK{K}^{†}=L$ 时，矩阵方程 $QXK=L$ 有一个解X，另外，X满足 $X=Q^{†}L{K}^{†}+Y-Q^{†}QK{K}^{†}$ ，其中Y是一个适当维数的矩阵。

3. 系统描述

本节主要考虑以下随机离散时间系统：

$\begin{array}{c}x\left(t+1\right)=A_0\left(t,{\theta }_t\right)x\left(t\right)+{\displaystyle \underset{k=1}{\overset{r}{\sum }}{A}_k\left(t,{\theta }_t\right)x\left(t\right)\omega \left(t\right)}+B_0\left(t,{\theta }_t\right)u\left(t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{B}_k\left(t,{\theta }_t\right)u\left(t\right)\omega \left(t\right)}+\omega \left(t\right)\end{array}$ (2-1)

$a_{i1}{x}_1\left(T\right)+a_{i2}{x}_2\left(T\right)+\cdots +a_{i1}{x}_n\left(T\right)\le {\xi }_i$

其中 $A_0\left(t,{\theta }_t\right)\in {ℝ}^n,A_k\left(t,{\theta }_t\right)\in {ℝ}^n$ ， $B_0\left(t,{\theta }_t\right)$ 和 $B_K\left(t,{\theta }_t\right)$ 是具有适当维数的向量， $x\in R^n$ 是系统的输出状态， $u\in R^m$ 是系统的输入控制， $x_0\in R^n$ 是给定的系统的初始状态， ${\left\{\omega \left(t\right)\right\}}_{t\ge 0}$ 是定义在完备概率空间 $\left(\Omega ,\mathcal{F},\mathcal{P}\right)$ 上的一维独立随机变量。 $\omega \left(t\right)$ 是噪声， $t\in \left\{t_0,t_0+1,\cdots ,T-1\right\},N_T\in \left\{0,1,\cdots ,T\right\}$ 。令 $N_{r\times n}={\left(a_{ij}\right)}_{r\times n},\xi ={\left({\xi }_1,{\xi }_2,\cdots ,{\xi }_r\right)}^{\text{T}}$ ，其中 $a_{ij}$ 是常数。

系统(2-1)所对应的二次指标为：

$\begin{array}{l}\mathcal{J}\left(t_0,x_0;u\left(t_0\right),u\left(t_0+1\right),\cdots ,u\left(T-1\right)\right)\\ =E\left\{{{\displaystyle \underset{t=t_0}{\overset{T-1}{\sum }}\left(\begin{array}{c}x\left(t\right)\\ u\left(t\right)\end{array}\right)}}^{\text{T}}\left(\begin{array}{cc}Q\left(t,{\theta }_t\right)& L\left(t,{\theta }_t\right)\\ {\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}& R\left(t,{\theta }_t\right)\end{array}\right)\left(\begin{array}{c}x\left(t\right)\\ u\left(t\right)\end{array}\right)+x^{\text{T}}\left(T\right)Mx\left(T\right)\right\}\end{array}$ (2-2)

其中， $\left(\begin{array}{cc}Q\left(t,{\theta }_t\right)& L\left(t,{\theta }_t\right)\\ {\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}& R\left(t,{\theta }_t\right)\end{array}\right)$ 与M均为对称矩阵。值函数定义为：

$V\left(t_0,x_0\right)=\underset{u\left(t_0\right),u(t_0+1),\cdots ,u\left(T-1\right)}{\inf }\mathcal{J}\left(t_0,x_0;u(\cdot )\right)$ (2-3)

假设1：为方便后续研究，现作出以下假设：

1) 上述矩阵都是有界的矩阵值序列。

2) 对于任意的 $t\ge 0$ ， $P_t={\left(p_t\left(i,j\right)\right)}_{i,j\in \Im \times \Im }$ 是一个非退化的随机矩阵，对于任意的 $\forall i,j\in \Im \times \Im$ 满足：

$\{\begin{array}{c}0\le p_t\left(i,j\right)\le 1\\ {\displaystyle \underset{k=1}{\overset{N}{\sum }}{p}_t\left(i,k\right)=1}\\ {\displaystyle \underset{k=1}{\overset{N}{\sum }}{p}_t\left(k,j\right)>0}\end{array}$ (2-4)

假设2： ${\left\{\omega \left(t\right)\right\}}_{t\ge 0}$ 是定义在完备概率空间 $\left(\Omega ,\mathcal{F},\mathcal{P}\right)$ 上的一维独立随机变量，与初值条件 $x\left(t_0\right)=x_0$ 是相互独立的，且具有以下性质：

$\begin{array}{l}E\left[\omega \left(t\right)\right]=0,\\ E\left[\omega \left(s\right){\left(\omega \left(t\right)\right)}^{\text{T}}\right]={\delta }_{st},\\ E\left[{\left(\omega \left(t\right)\right)}^2\right]=1\end{array}$ (2-5)

其中：

${\delta }_{st}=\{\begin{array}{c}{I}_r,s=t\\ 0,s\ne t\end{array}$ (2-6)

假设3：对于任意的 $t\ge 0$ ， $\sigma$ -代数 ${\mathcal{F}}_t$ 与 $\sigma$ -代数 ${\mathcal{G}}_t$ 是相互独立的，其中， ${\mathcal{F}}_t=\sigma \left\{\omega \left(s\right),0\le s\le t\right\}$ ， ${\mathcal{G}}_t=\sigma \left\{\vartheta \left(s\right),0\le s\le t\right\}$ 。

定义 4：无限随机线性二次问题(2-1)、(2-2)、(2-3)是适定的，若对于任意的 $\left(t_0,x_0\right)\in N_T\times R^n$ ， $V\left(t_0,x_0\right)>-\infty$ 。

定义 5：无限随机线性二次问题(2-1)、(2-2)、(2-3)是可达的，若对于任意的 $\left(t_0,x_0\right)\in N_T\times R^n$ ，存在一个序列 $\left\{u\left(t\right),t=t_0,t_0+1,\cdots ,T-1\right\}$ 为系统的最优控制，使得 $V\left(t_0,x_0\right)=\mathcal{J}\left(t_0,x_0;u\left(t_0+1\right),\cdots ,u\left(T-1\right)\right)$ 。

注记 1：若无限随机线性二次问题(2-1)、(2-2)、(2-3)的一个线性反馈控制是最优的，则必存在一个最优线性反馈控制具有以下形式：

$u\left(t\right)=K\left(t,{\theta }_t\right)x\left(t\right),t\in N_T$

其中， $K\left(t\right)$ 是一个矩阵值函数。

4. 主要成果

在研究中发现，系统(2-1)、(2-2)、(2-3)的最优控制问题可以通过广义Riccati差分方程的解来表示。

4.1. 随机线性系统与Riccati方程

定理1：如果系统(2-1)、(2-2)、(2-3)的最优控制问题是可达的， $u\left(t\right)=K\left(t,{\theta }_t\right)x\left(t\right),t\in N_T$ ，而且正则点 $\left(u^{\ast }\left(t\right),x^{\ast }\left(t\right)\right)$ 是系统(2-1)、(2-2)、(2-3)的局部最优解，则下列广义Riccati方程(GDRE)有解 $\left(P\left(k\right),\mu \right),0\le \mu \in R^1,t\in N_T$ ：

$\begin{array}{c}P\left(t\right)={\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\times A_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(A_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)A_k\left(t,{\theta }_t\right)+Q\left(t,{\theta }_t\right)\\ -{\left(T\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)\end{array}$ (2-7)

$\begin{array}{c}T\left(t,{\theta }_t\right)={\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\times A_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)A_k\left(t,{\theta }_t\right)\end{array}$ (2-8)

$\begin{array}{c}W\left(t,{\theta }_t\right)={\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\times B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)+R\left(t,{\theta }_t\right)\end{array}$ (2-9)

$W\left(t,{\theta }_t\right)\ge 0$

$\mu =\frac{Tr\left\{P\left(T\right)-M\right\}}{Tr\left\{N^{T}N\right\}}$ (2-10)

此外，

$\begin{array}{l}{u}^{*}\left(t\right)=\left[-{\left(W\left(t,{\theta }_t\right)\right)}^{†}T\left(t,{\theta }_t\right)+Y\left(t,{\theta }_t\right)-{\left(W\left(t,{\theta }_t\right)\right)}^{†}W\left(t,{\theta }_t\right)Y\left(t,{\theta }_t\right)\right]x\left(t\right)\\ u^{*}\left(t\right)\in R^{m\times n},t\in N_T\end{array}$ (2-11)

$V\left(x_0,{\theta }_t\right)={\displaystyle \underset{t=1}{\overset{T-1}{\sum }}Tr\left\{V\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\right\}}+x_0^{\text{T}}P\left(0,{\theta }_t\right)x_0-\mu Tr\left\{M\right\}$ (2-12)

证明：对于任意的 $t\in N_T$ ，令 $X\left(t,{\theta }_t\right)=E\left({\left[x\left(t\right)x{\left(t\right)}^{\text{T}}\right]}_{\left\{{\theta }_{t_0}=i\right\}}\right)$ ，其中 $u\left(t\right)=K\left(t,{\theta }_t\right)x\left(t\right),t\in N_T$ ，则线

性二次控制问题(2-1)、(2-2)、(2-3)可以转化成下面的最优化问题：

$\begin{array}{l}J\left(t_0,x_0;u\left(t_0\right),u\left(t_0+1\right),\cdots ,u\left(\infty \right)\right)\\ =E\left\{{\displaystyle \underset{t=t_0}{\overset{\infty }{\sum }}{\left(\begin{array}{c}x\left(t\right)\\ u\left(t\right)\end{array}\right)}^{\text{T}}\left(\begin{array}{cc}Q\left(t,{\theta }_t\right)& L\left(t,{\theta }_t\right)\\ {\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}& R\left(t,{\theta }_t\right)\end{array}\right)\left(\begin{array}{c}x\left(t\right)\\ u\left(t\right)\end{array}\right)+\underset{T\to \infty }{\lim }{x}^{\text{T}}\left(T\right)Mx\left(T\right)}\right\}\\ ={\displaystyle \underset{t=t_0}{\overset{\infty }{\sum }}E\{{\left(x\left(t\right)\right)}^{\text{T}}Q\left(t,{\theta }_t\right)x\left(t\right)+{\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(x\left(t\right)\right)}^{\text{T}}{\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}x\left(t\right)+{\left(x\left(t\right)\right)}^{\text{T}}L\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)x\left(t\right)}\\ +{\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(x\left(t\right)\right)}^{\text{T}}R\left(t,{\theta }_t\right)x\left(t\right)K\left(t,{\theta }_t\right)+\underset{T\to \infty }{\lim }{x}^{\text{T}}\left(T\right)Mx\left(T\right)\}\\ ={\displaystyle \underset{t=t_0}{\overset{\infty }{\sum }}Tr}\{Q\left(t,{\theta }_t\right)X\left(t\right)+{\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}K\left(t,{\theta }_t\right)X\left(t\right)+{\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}X\left(t\right)\\ +L\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)X\left(t\right)+\underset{T\to \infty }{\lim }MX\left(T\right)\}\end{array}$ (2-13)

s.t.下式均成立：

$\begin{array}{c}X\left(t+1\right)=A_0\left(t,{\theta }_t\right)X\left(t\right){\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}+A_0\left(t,{\theta }_t\right)X\left(t\right){\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}\\ +B_0\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)X\left(t\right){\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}\\ +B_0\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)X\left(t\right){\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(A_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}X\left(t\right)A_k\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(A_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}X\left(t\right){\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{B}_k\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)}X\left(t\right){\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}+V\left(t\right)\end{array}$ (2-14)

$\begin{array}{l}X\left(t_0\right)=E\left[x_0{x}_0^{\text{T}}\right]=x_0{x}_0^{\text{T}}\\ Tr\left(X\left(T\right)N^{\text{T}}N\right)\le TrS,S=E\left[\xi {\xi }^{\text{T}}\right]\end{array}$

即：

$\underset{t\in N_{T-1}}{\min }f\left[X\left(t\right),K\left(t,{\theta }_t\right)\right]$

$\begin{array}{l}\text{s}\text{.t}.\mathcal{H}\left[X\left(t\right),K\left(t,{\theta }_t\right)\right]=0\\ \mathcal{G}\left(X\left(T\right)\right)\le 0\end{array}$

其中， $K\left(1,{\theta }_t\right),K\left(2,{\theta }_t\right),\cdots ,K\left(T-1,{\theta }_t\right)$ 为待定控制项。

$f\left[X\left(t\right),K\left(t,{\theta }_t\right)\right]=J\left(t_0,x_0;u\left(t_0\right),u\left(t_0+1\right),\cdots ,u\left(T-1\right)\right)$ (2-15)

$\begin{array}{l}\mathcal{H}\left[X\left(t\right),K\left(t,{\theta }_t\right)\right]-X\left(t+1\right)=0\\ \mathcal{G}\left(X\left(T\right)\right)=Tr\left(X\left(T\right)N^{\text{T}}N\right)-TrS\end{array}$

根据KKT定理，设拉格朗日函数为：

$\begin{array}{l}\mathcal{L}[X\left(t\right),K\left(t,{\theta }_t\right),P\left(t+1,{\theta }_t\right),\mu ]\\ =f\left[X\left(t\right),K\left(t,{\theta }_t\right)\right]+{\displaystyle \underset{t=t_0}{\overset{\infty }{\sum }}Tr\left\{P\left(t+1,{\theta }_t\right)H\left[X\left(t\right),K\left(t,{\theta }_t\right)\right]\right\}}+\mu g\left(X\left(T\right)\right)\end{array}$ (2-16)

对 $\mathcal{L}\left[X\left(t\right),K\left(t,{\theta }_t\right),P\left(t+1,{\theta }_t\right),\mu \right]$ 中的 $X\left(t\right)$ 进行求导，且根据KKT定理得：

$\frac{\partial \mathcal{L}\left[X\left(t\right),K\left(t,{\theta }_t\right),P\left(t+1,{\theta }_t\right),\mu \right]}{\partial X\left(t\right)}=0$

通过计算可得：若 $X\left(0\right)=0$ ，则有

$TrM\Delta X\left(T\right)-Tr\Delta X\left(T\right)P\left(T,{\theta }_t\right)+Tr\left(\Delta X\left(T\right)N^{\text{T}}N\right)=0$

即：

$P\left(T,{\theta }_t\right)=M+\mu N^{\text{T}}N$ (2-17)

其中， $T\to \infty$ 。

又由于 $\Delta X\left(T\right)$ 与 $\Delta X\left(t\right)$ 相互独立，则有：

$\begin{array}{l}P\left(t,{\theta }_t\right)=Q\left(t,{\theta }_t\right)+{\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}[R\left(t,{\theta }_t\right)+{\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}K\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)]K\left(t,{\theta }_t\right)\\ +{[\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}+{\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(A_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)]K\left(t,{\theta }_t\right)\end{array}$

$\begin{array}{l}+{\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}[L\left(t,{\theta }_t\right)+{\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)A_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)A_k\left(t,{\theta }_t\right)]\\ +A_0\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right){\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{B}_k}\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right){\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}\end{array}$ (2-18)

同理，对 $K\left(t,{\theta }_t\right)$ 求导可得：

$\begin{array}{l}[R\left(t,{\theta }_t\right)+{\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}K\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)}]K\left(t,{\theta }_t\right)\\ +{\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}+{\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0(t,{\theta }_t)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(A_k\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)}=0\end{array}$ (2-19)

取：

$\begin{array}{c}W\left(t,{\theta }_t\right)=R\left(t,{\theta }_t\right)+{\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}K\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)\end{array}$ (2-20)

$\begin{array}{c}T\left(t,{\theta }_t\right)={\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}+{\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(A_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)\end{array}$ (2-21)

综合上述(2-20)、(2-21)则第(2-18)、(2-19)式可写做：

$\begin{array}{c}P\left(t\right)=Q\left(t,{\theta }_t\right)+{\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}W\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)+T\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)\\ +{\left(K\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(T\left(t,{\theta }_t\right)\right)}^{\text{T}}+A_0\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right){\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{B}_k\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right){\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}\end{array}$ (2-22)

$W\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)+T\left(t,{\theta }_t\right)=0$ (2-23)

根据引理4可得：若 $W\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)+T\left(t,{\theta }_t\right)=0$ 有解 $K\left(t,{\theta }_t\right)$ ，当且仅当 $W\left(t,{\theta }_t\right){\left(W\left(t,{\theta }_t\right)\right)}^{†}K\left(t,{\theta }_t\right)=-T\left(t,{\theta }_t\right)$ ，则此时方程的解 $K\left(t,{\theta }_t\right)$ 表示如下：

$K\left(t,{\theta }_t\right)=-{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)+Y\left(t,{\theta }_t\right)-{\left(W\left(t,{\theta }_t\right)\right)}^{+}W\left(t,{\theta }_t\right)Y\left(t,{\theta }_t\right)$ (2-24)

将 $K\left(t,{\theta }_t\right)$ 带入(2-22)式得以下广义Riccati方程：

$\begin{array}{c}P\left(t\right)=Q\left(t,{\theta }_t\right)+A_0\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right){\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{B}_k\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right){\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}\\ +{\left(T\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)\end{array}$ (2-25)

即为(2-7)式。

$\begin{array}{c}W\left(t,{\theta }_t\right)=R\left(t,{\theta }_t\right)+{\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}K\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)\end{array}$ (2-26)

即为(2-9)式。

$\begin{array}{c}T\left(t,{\theta }_t\right)={\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}+{\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(A_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)\end{array}$ (2-27)

即为(2-10)式。

$\mu =\frac{Tr\left\{P\left(T,{\theta }_t\right)-M\right\}}{Tr\left\{N^{T}N\right\}}$

且得到：

$K\left(t,{\theta }_t\right)=-{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)+Y\left(t,{\theta }_t\right)-{\left(W\left(t,{\theta }_t\right)\right)}^{+}W\left(t,{\theta }_t\right)Y\left(t,{\theta }_t\right)$ (2-28)

由于 $u\left(t\right)=K\left(t,{\theta }_t\right)x\left(t\right),t\in N_T$ ，故有：

$\begin{array}{l}{u}^{*}\left(t\right)=\left[-{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)+Y\left(t,{\theta }_t\right)-{\left(W\left(t,{\theta }_t\right)\right)}^{+}W\left(t,{\theta }_t\right)Y\left(t,{\theta }_t\right)\right]x\left(t\right)\\ u^{*}\left(t\right)\in R^{m\times n},t\in N_T\end{array}$ (2-29)

即为(2-11)式。

假设 $P\left(t+1,{\theta }_t\right)$ 是一个对称矩阵，若 $P\left(t+1,{\theta }_t\right)$ 不是对称矩阵，则取：

$\tilde{P}\left(t+1,{\theta }_t\right)=\frac{P\left(t+1,{\theta }_t\right)+{\left(P\left(t+1,{\theta }_t\right)\right)}^{\text{T}}}{2}$

显然， $\tilde{P}\left(t+1,{\theta }_t\right)$ 也是一个对称矩阵。

$\begin{array}{l}{\displaystyle \underset{t=0}{\overset{T-1}{\sum }}E\left[{\left(x\left(t+1\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)x\left(t+1\right)-{\left(x\left(t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t\right)\right]\left({\theta }_t\right)x\left(t\right)\right]}\\ =E\left[{\left(x\left(T\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(T\right)\right]\left({\theta }_T\right)x\left(T\right)-{\left(x\left(0\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(0\right)\right]\left({\theta }_0\right)x\left(0\right)\right]\end{array}$ (2-30)

将第(2-30)式变形得到：

$\begin{array}{l}{\displaystyle \underset{t=0}{\overset{T-1}{\sum }}E\left[{\left(x\left(t+1\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)x\left(t+1\right)-{\left(x\left(t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t\right)\right]\left({\theta }_t\right)x\left(t\right)\right]}\\ -E\left[{\left(x\left(T\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(T\right)\right]\left({\theta }_T\right)x\left(T\right)+{\left(x\left(0\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(0\right)\right]\left({\theta }_0\right)x\left(0\right)\right]\\ =0\end{array}$ (2-31)

将第(2-31)式与第(2-2)式相加，得：

$\begin{array}{l}J\left(t_0,x_0;u\left(t_0\right),u\left(t_0+1\right),\cdots ,u\left(T-1\right)\right)\\ =E\left\{{\displaystyle \underset{t=t_0}{\overset{T-1}{\sum }}{\left(\begin{array}{c}x\left(t\right)\\ u\left(t\right)\end{array}\right)}^{\text{T}}\left(\begin{array}{cc}Q\left(t,{\theta }_t\right)& L\left(t,{\theta }_t\right)\\ {\left(L\left(t,{\theta }_t\right)\right)}^T& R\left(t,{\theta }_t\right)\end{array}\right)\left(\begin{array}{c}x\left(t\right)\\ u\left(t\right)\end{array}\right)+x^{\text{T}}\left(T\right)Mx\left(T\right)}\right\}\\ +{\displaystyle \underset{t=0}{\overset{T-1}{\sum }}E\left[{\left(x\left(t+1\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)x\left(t+1\right)-{\left(x\left(t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t\right)\right]\left({\theta }_t\right)x\left(t\right)\right]}\\ -E\left[{\left(x\left(T\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(T\right)\right]\left({\theta }_T\right)x\left(T\right)-{\left(x\left(0\right)\right)}^{\text{T}}P\left(0,{\theta }_0\right)x\left(0\right)\right]\end{array}$

$\begin{array}{l}={\displaystyle \underset{t=t_0}{\overset{T-1}{\sum }}E\left\{{\left(u\left(t\right)+{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)x\left(t\right)\right)}^{\text{T}}W\left(t,{\theta }_t\right)\left(u\left(t\right)+{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)x\left(t\right)\right)\right\}}\\ +{\displaystyle \underset{t=t_0}{\overset{T-1}{\sum }}Tr\left[V\left(t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\right]}+E\left[{\left(x\left(T\right)\right)}^{\text{T}}\left(M-P\left(T,{\theta }_t\right)\right)x\left(T\right)\right]\\ +{\left(x\left(0\right)\right)}^{\text{T}}P\left(0,{\theta }_0\right)x\left(0\right)\end{array}$ (2-32)

此时，必须证明 $W\left(t,{\theta }_t\right)\ge 0,t\in N_T$ 。假设 $W\left(t,{\theta }_t\right)$ 有负的特征值 $\alpha$ ，令 $v_{\alpha }$ 是 $\alpha$ 所对应得唯一的特征向量，并且有：

$\{\begin{array}{l}{\left(v_{\alpha }\right)}^{\text{T}}{v}_{\alpha }=1\\ W\left(t,{\theta }_t\right)v_{\alpha }=-\alpha v_{\alpha }\end{array}$

对任意的 $\delta \ne 0$ ，设控制列：

$\tilde{u}\left(t\right)\{\begin{array}{l}-{\left(W\left(t,{\theta }_t\right)\right)}^{†}T\left(t,{\theta }_t\right)x\left(t\right),t\ne k\\ \delta {\left|\alpha \right|}^{\frac{1}{2}}{v}_{\alpha }-{\left(W\left(t,{\theta }_t\right)\right)}^{†}T\left(t,{\theta }_t\right)x\left(t\right),t=k\end{array}$

相应的目标函数为：

$\begin{array}{l}J\left(t_0,x_0;\tilde{u}\left(t_0\right),\tilde{u}\left(t_0+1\right),\cdots ,\tilde{u}\left(T-1\right)\right)\\ ={\displaystyle \underset{t=t_0}{\overset{T-1}{\sum }}E\left\{{\left(u\left(t\right)+{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)x\left(t\right)\right)}^{\text{T}}W\left(t,{\theta }_t\right)\left(u\left(t\right)+{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)x\left(t\right)\right)\right\}}\\ +{\displaystyle \underset{t=t_0}{\overset{T-1}{\sum }}Tr\left[V\left(t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\right]}+E\left[{\left(x\left(T\right)\right)}^{\text{T}}\left(M-P\left(T,{\theta }_t\right)\right)x\left(T\right)\right]\\ +{\left(x\left(0\right)\right)}^{\text{T}}P\left(0,{\theta }_0\right)x(\; 0\; )\end{array}$

$\begin{array}{l}={\left(\delta {\left|\alpha \right|}^{\frac{1}{2}}v\alpha \right)}^{\text{T}}W\left(t,{\theta }_t\right)\delta {\left|\alpha \right|}^{\frac{1}{2}}v\alpha +{\displaystyle \underset{t=t_0}{\overset{T-1}{\sum }}Tr\left[V\left(t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\right]}\\ +E\left[{\left(x\left(T\right)\right)}^{\text{T}}\left(M-P\left(T,{\theta }_t\right)\right)x\left(T\right)\right]+{\displaystyle \underset{k=1}{\overset{\infty }{\sum }}Tr\left[V\left(t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\right]}\\ +E\left[{\left(x\left(T\right)\right)}^{\text{T}}\left(M-P\left(T,{\theta }_t\right)\right)x\left(T\right)\right]+{\left(x\left(0\right)\right)}^{\text{T}}P\left(0,{\theta }_0\right)x\left(0\right)\end{array}$ (2-33)

当 $\delta \to 0$ 时， $\mathcal{J}\left(t_0,x_0;\tilde{u}\left(t_0\right),\tilde{u}\left(t_0+1\right),\cdots ,\tilde{u}\left(T-1\right)\right)\to \infty$ ，这与线性二次控制问题(2-1)、(2-2)、(2-3)矛盾，故 $W\left(t,{\theta }_t\right)\ge 0$ 是成立的。

根据 $P\left(T,{\theta }_t\right)=M+\mu N^{\text{T}}N$ 可知：

$\begin{array}{c}V\left(x_0\right)=J\left(t_0,x_0;u\left(t_0\right),u\left(t_0+1\right),\cdots ,u\left(T-1\right)\right)\\ ={\displaystyle \underset{t=t_0}{\overset{T-1}{\sum }}Tr\left[V\left(t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\right]}-\mu TrS+{\left(x\left(0\right)\right)}^{\text{T}}P\left(0,{\theta }_0\right)x\left(0\right)\end{array}$ (2-34)

故定理1证明完毕。

4.2. Riccati方程的相关性质研究

定理2：如果系统(2-1)、(2-2)、(2-3)的最优控制问题是可达的， $u\left(t\right)=K\left(t,{\theta }_t\right)x\left(t\right),t\in N_T$ ，而且正则点 $\left(u^{\ast }\left(t\right),x^{\ast }\left(t\right)\right)$ 是系统(2-1)、(2-2)、(2-3)的局部最优解，则下列广义Riccati方程(GDRE)有解 $\left(P\left(k\right),\mu \right),0\le \mu \in R^1,t\in N_T$ ：

$\begin{array}{c}P\left(t\right)={\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)A_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(A_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)A_k\left(t,{\theta }_t\right)+Q\left(t,{\theta }_t\right)\\ -{\left(T\left(t,{\theta }_t\right)\right)}^{\text{T}}{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)\end{array}$ (2-35)

$\begin{array}{c}T\left(t,{\theta }_t\right)={\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)A_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)A_k\left(t,{\theta }_t\right)+{\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}\end{array}$ (2-36)

$\begin{array}{c}W\left(t,{\theta }_t\right)={\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)+R\left(t,{\theta }_t\right)\end{array}$ (2-37)

$W\left(t,{\theta }_t\right)>0$

$\mu =\frac{Tr\left\{P\left(T,{\theta }_t\right)-M\right\}}{Tr\left\{N^{\text{T}}N\right\}}$

此外，

$\begin{array}{l}{u}^{*}\left(t\right)=\left[-{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)\right]x\left(t\right)\\ u^{*}\left(t\right)\in {\mathcal{R}}^{m\times n},t\in N_T\end{array}$

$V\left(x_0,{\theta }_t\right)={\displaystyle \sum Tr\left\{V\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\right\}}+x_0^{\text{T}}P\left(0,{\theta }_t\right)x_0-\mu Tr\left\{M\right\}$ (2-38)

证明：根据定理 可得，系统(2-1)、(2-2)、(2-3)所对应得广义 方程为式(2-23)，其中 $W\left(t,{\theta }_t\right)>0$ ，且有：

$\begin{array}{l}[R\left(t,{\theta }_t\right)+{\left(B_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(B_k\left(t,{\theta }_t\right)\right)}^{\text{T}}K\left(t,{\theta }_t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)}]K\left(t,{\theta }_t\right)\\ +{\left(L\left(t,{\theta }_t\right)\right)}^{\text{T}}+{\left(A_0\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]B_0\left(t,{\theta }_t\right)\\ +{\displaystyle \underset{k=1}{\overset{r}{\sum }}{\left(A_k\left(t,{\theta }_t\right)\right)}^{\text{T}}E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)B_k\left(t,{\theta }_t\right)}=0\end{array}$ (2-39)

即： $W\left(t,{\theta }_t\right)K\left(t,{\theta }_t\right)+T\left(t,{\theta }_t\right)=0$ ，由于 $W\left(t,{\theta }_t\right)>0$ ，故存在 ${\left(W\left(t,{\theta }_t\right)\right)}^{-1}$ ，使得：

$K\left(t,{\theta }_t\right)=-{\left(W\left(t,{\theta }_t\right)\right)}^{-1}T\left(t,{\theta }_t\right)$ (2-40)

故：

$u^{*}\left(t\right)=K\left(t,{\theta }_t\right)x\left(t\right)=-{\left(W\left(t,{\theta }_t\right)\right)}^{-1}T\left(t,{\theta }_t\right)x\left(t\right)$ (2-41)

根据定理1同样可得：

$\begin{array}{l}J\left(t_0,x_0;\tilde{u}\left(t_0\right),\tilde{u}\left(t_0+1\right),\cdots ,\tilde{u}\left(\infty \right)\right)\\ ={\displaystyle \underset{t=t_0}{\overset{\infty }{\sum }}E\left\{{\left(u\left(t\right)+{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)x\left(t\right)\right)}^{\text{T}}W\left(t,{\theta }_t\right)\left(u\left(t\right)+{\left(W\left(t,{\theta }_t\right)\right)}^{+}T\left(t,{\theta }_t\right)x\left(t\right)\right)\right\}}\\ +{\displaystyle \underset{t=t_0}{\overset{T-1}{\sum }}Tr\left[V\left(t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\right]}+E\left[{\left(x\left(T\right)\right)}^{\text{T}}\left(M-P\left(T,{\theta }_t\right)\right)x\left(T\right)\right]\\ +{\left(x\left(0\right)\right)}^{\text{T}}P\left(0,{\theta }_0\right)x\left(0\right)\end{array}$ (2-42)

将(2-41)代入(2-42)式，且 $T\to \infty$ 可得：

$\begin{array}{c}V\left(x_0,{\theta }_t\right)={\displaystyle \underset{t=t_0}{\overset{\infty }{\sum }}Tr\left[V\left(t\right)E\left(t\right)\left[P\left(t+1\right)\right]\left({\theta }_t\right)\right]}-\mu Tr\left\{M\right\}\\ +{\left(x\left(0\right)\right)}^{\text{T}}P\left(0,{\theta }_0\right)x\left(0\right)\end{array}$ (2-43)

即定理2证明完毕。

5. 数值举例

考虑系统(2-1)、(2-2)、(2-3)的特殊形式，当 $N=1,\mathcal{Z}=\left\{1\right\},r=1$ 时，对应的系数为：

$\begin{array}{l}{A}_0=\left(\begin{array}{cc}1& 0\\ 0& 2\end{array}\right),A_1=\left(\begin{array}{cc}-1& 0\\ 0& 1\end{array}\right)\\ B_0=\left(\begin{array}{c}2\\ 1\end{array}\right),B_1=\left(\begin{array}{c}1\\ -1\end{array}\right)\\ x_0=\left(\begin{array}{cc}1& 2\\ 3& 4\end{array}\right),M=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)\end{array}$

$\begin{array}{l}L\left(0\right)=\left(\begin{array}{cc}0& 0\end{array}\right),R\left(0\right)=-13\\ V\left(0\right)=\left(\begin{array}{cc}2& 1\\ 1& 1\end{array}\right),Q\left(0\right)=\left(\begin{array}{cc}3& 0\\ 0& 0\end{array}\right)\\ a_{11}=a_{22}=0,a_{12}=a_{21}=0\end{array}$

$\begin{array}{l}{\xi }_1={\xi }_2=1\\ Y=(\; 0\; 0\; )\end{array}$

根据定理1可直接计算得：

$P\left(0\right)=\left(\begin{array}{cc}-65& -64\\ -64& -52\end{array}\right),P\left(1\right)=\left(\begin{array}{cc}2& 0\\ 0& 2\end{array}\right)$

$\begin{array}{l}H\left(0\right)=\left(\begin{array}{cc}8& 8\end{array}\right),G\left(0\right)=1\\ \mu =1\\ u=\left(\begin{array}{cc}8& 8\end{array}\right)x(\; t\; )\end{array}$

6. 结论

本文将离散时间无限时域时变非齐次Markov跳变系统的线性二次最优控制问题转化为广义Riccati方程的可解性。在后续的研究中具有重大的意义，通过广义Riccati方程的可解性可以更好的研究对应系统的稳定性，更是引导未来用算子理论与随机分析的方法来对离散时间无限时域时变非齐次Markov跳变系统的均方稳定性与可探测性等问题进行研究。

基金项目

重庆理工大学研究生教育高质量发展行动计划资助成果，项目编号：gzlcx20223308，项目类型：校级联合资助项目；重庆理工大学研究生教育高质量发展行动计划资助成果，项目编号：gzlcx20223304，项目类型：校级联合资助项目。

NOTES

^*通讯作者。