TODO

• 相关系数矩阵
• 协方差矩阵
• Hermitan矩阵
• 共轭矩阵
• Hessian 矩阵

## Hessian Matrix

海森矩阵是一个多变量实值函数的二阶偏导数组成的方块矩阵，假设一个实数函数 ${\displaystyle \textstyle f(x_{1},x_{2},\dots ,x_{n})}$，如果 $f$ 所有的二阶偏导数都存在，那么 $f$ 的黑森矩阵为：

$H(f)=\left[ \begin{array}{cccc}{\frac{\partial^{2} f}{\partial x_{1}^{2}}} & {\frac{\partial^{2} f}{\partial x_{1} \partial x_{2}}} & {\cdots} & {\frac{\partial^{2} f}{\partial x_{1} \partial x_{n}}} \\ {\frac{\partial^{2} f}{\partial x_{2} \partial x_{1}}} & {\frac{\partial^{2} f}{\partial x_{2}^{2}}} & {\dots} & {\frac{\partial^{2} f}{\partial x_{2} \partial x_{n}}} \\ {\vdots} & {\vdots} & {\ddots} & {\vdots} \\ {\frac{\partial^{2} f}{\partial x_{n} \partial x_{1}}} & {\frac{\partial^{2} f}{\partial x_{n} \partial x_{2}}} & {\cdots} & {\frac{\partial^{2} f}{\partial x_{n}^{2}}}\end{array}\right]$

若一元函数 $f(x)$ 在 $x=x_0$ 点的某个邻域内具有任意阶导数，则 $f(x)$ 在 $x=x_0$ 点处的泰勒展开式：

$f(x)=f\left(x_{0}\right)+f^{\prime}(x) \Delta x+\frac{f^{\prime \prime}(x)}{2 !} \Delta x^{2}+\cdots$

其中，${\displaystyle \Delta x=x-x_{0}}$。

同理，二元函数 ${\displaystyle f(x_{1},x_{2})}$ 在 ${\displaystyle x_{0}(x_{10},x_{20})}$ 点处的泰勒展开式为

${\displaystyle f(x_{1},x_{2})=f(x_{10},x_{20})+f_{x_{1}}(x_{0})\Delta x_{1}+f_{x_{2}}(x_{0})\Delta x_{2}+{\frac {1}{2}}[f_{x_{1}x_{1}}(x_{0})\Delta x_{1}^{2}+2f_{x_{1}x_{2}}(x_{0})\Delta x_{1}\Delta x_{2}+f_{x_{2}x_{2}}(x_{0})\Delta x_{2}^{2}]+\cdots }$

其中， ${\displaystyle \Delta x_{1}=x_{1}-x_{10}}$， ${\displaystyle \Delta x_{2}=x_{2}-x_{20}}$，${\displaystyle f_{x_{1}x_{1}}={\frac {\partial ^{2}f}{\partial x_{1}^{2}}}}$， ${\displaystyle f_{x_{2}x_{2}}={\frac {\partial ^{2}f}{\partial x_{2}^{2}}}}$， ${\displaystyle f_{x_{1}x_{2}}={\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}={\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}}$。

将上述展开式写成矩阵形式，则有

${\displaystyle f(x)=f(x_{0})+\nabla f(x_{0})^{T}\Delta x+{\frac {1}{2}}\Delta x^{T}H(x_{0})\Delta x+\cdots }$

其中， ${\displaystyle \Delta x=(\Delta x_{1},\Delta x_{2})}$，${\displaystyle \Delta x^{T}={\begin{bmatrix}\Delta x_{1}\\\Delta x_{2}\end{bmatrix}}}$ 是 ${\displaystyle \Delta x}$ 的转置（此处“转置”用上角标 ${\displaystyle T}$ 表示），${\displaystyle \nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}\\{\frac {\partial f}{\partial x_{2}}}\end{bmatrix}}}$ 是函数 ${\displaystyle f(x_{1},x_{2})}$ 的梯度，矩阵

${\displaystyle H(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{bmatrix}}_{x_{0}}}$

即函数 ${\displaystyle f(x_{1},x_{2})}$ 在 ${\displaystyle x_{0}(x_{10},x_{20})}$ 点处的二阶黑塞矩阵。它是由函数 ${\displaystyle f(x_{1},x_{2})}$ 在 ${\displaystyle x_{0}(x_{10},x_{20})}$ 点处的二阶偏导数所组成的方阵。由函数的二次连续性，有

${\displaystyle {\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}={\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}} {\displaystyle {\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}={\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}}$

所以，黑塞矩阵 ${\displaystyle H(x_{0})}$ 为对称矩阵。

将二元函数的泰勒展开式推广到多元函数时，${\displaystyle f(x_{1},x_{2},\cdots ,x_{n})}$ 在 $x_{0}$ 点处的泰勒展开式为

${\displaystyle f(x)=f(x_{0})+\nabla f(x_{0})^{T}\Delta x+{\frac {1}{2}}\Delta x^{T}H(x_{0})\Delta x+\cdots }$

其中，

${\displaystyle \nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}&{\frac {\partial f}{\partial x_{2}}}&\cdots &{\frac {\partial f}{\partial x_{n}}}\end{bmatrix}}_{x_{0}}^{T}}$

为函数 ${\displaystyle f(x)}$ 在 ${\displaystyle x_{0}}$ 点的梯度。

${\displaystyle H(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}}$

为函数 ${\displaystyle f(x)}$ 在 ${\displaystyle x_{0}}$ 点的 ${\displaystyle n}$ 阶黑塞矩阵。若函数有 ${\displaystyle n}$ 次连续性，则函数的 ${\displaystyle n}$ 阶黑塞矩阵也是对称矩阵。