matrix analysis brief overview

Introduction

r (A + B) ⩽ r (A) + r (B) r (A B) ⩽ r (A), r (B) i f A B = O, t h e n r (A) + r (B) ⩽ n r (A B) ⩾ r (A) + r (B) - n

$r(A+B) \leqslant r(A) + r(B) \\ r(AB) \leqslant r(A),\ r(B) \\ if\quad AB=O, \ then \ r(A)+r(B)\leqslant n \\ r(AB) \geqslant r(A) + r(B) - n$

Image and Kernel (subspace)

Definition. The image of a function consists of all the values the function assumes. If $f:X\rightarrow Y$ is a function from $X$ to $Y$ , then

i m (f) = {f (x) : x \in X}

$\mathrm{im}(f)=\{f(x):x\in X\}$

Notice that im( $f$ ) is a subset of $Y$ .

Definition. The kernel of a function whose range is $\mathbb{R}^n$ consists of all the values in its domain at which the function assumes the value $0$ . If $f:X\rightarrow \mathbb{R}^n$ is a function from $X$ to $\mathbb{R}^n$ , then

k e r (f) = {x \in X : f (x) = 0}

$\mathrm{ker}(f) = \{x\in X : f(x) = 0\}$

Notice that ker( $f$ ) is a subset of X. Also, if $T(x)=Ax$ is a linear transformation from $\mathbb{R}^m$ to $\mathbb{R}^n$ , then ker( $T$ ) (also denote ker( $A$ )) is the set of solutions to the equation $Ax= 0$ .

Kernel and Image

Theorem. Let $A$ be an $n \times n$ matrix. Then the following statements are equivalent.

$A$ is invertible
The linear system $Ax = b$ has a unique solution $x$ for every $b\in \mathbb{R}^n$
rref( $A$ ) = $I_n$
rank( $A$ ) = $n$
im( $A$ ) = $\mathbb{R}^n$
ker( $A$ ) = $\{0\}$

linear mapping

V i s a m d e m e n s i o n l i n e a r s p a c e . l e t V_{m \times m}^{'} a s a g r o u p o f b a s i s v e c t o r s (m a y b e n o t s t a n d a r d) . V_{m \times m}^{'} = {v_{1}, v_{2}, \dots, v_{m}}, v_{i} i s o n e o f a g r o u p o f b a s i c v e c t o r s . W i s a n d e m e n s i o n l i n e a r s p a c e . l e t W_{n \times n}^{'} a s a g r o u p o f b a s i s v e c t o r s (m a y b e n o t s t a n d a r d) . W_{n \times n}^{'} = {w_{1}, w_{2}, \dots, w_{n}}, w_{i} i s o n e o f a g r o u p o f b a s i c v e c t o r s . l i n e a r m a p p i n g : V ⟼ W, F (v_{m \times 1}) = w_{n \times 1}, i t c a n m a p f r o m V t o W .

$V \mathrm{\ is\ a\ } m \mathrm{\ demension\ linear\ space.}\quad \mathrm{let\ } V_{m \times m}' \mathrm{\ as\ a\ group\ of\ basis\ vectors(maybe\ not\ standard).} \\ V_{m \times m}' = \{v_1,v_2,\dots,v_m\}, \ v_i \mathrm{\ is\ one\ of\ a\ group\ of\ basic\ vectors.} \\ W \mathrm{\ is\ a\ } n \mathrm{\ demension\ linear\ space.}\quad \mathrm{let\ } W_{n \times n}' \mathrm{\ as\ a\ group\ of\ basis\ vectors(maybe\ not\ standard).} \\ W_{n \times n}' = \{w_1,w_2,\dots,w_n\}, \ w_i \mathrm{\ is\ one\ of\ a\ group\ of\ basic\ vectors.} \\ linear\ mapping:\quad V \longmapsto W, \quad \mathcal{F}(v_{m \times 1})=w_{n \times 1},\ \mathrm{it\ can\ map\ from\ V\ to\ W.}$

F (v_{i}) = {w_{1}, w_{2}, \dots, w_{n}} [a_{1 i}, a_{2 i}, \dots, a_{n i}]^{T} m a t r i x r e p r e s e n t a t i o n : A_{n \times m}, F (V_{m \times m}^{'})_{n \times m} = W_{n \times n}^{'} A_{n \times m} s o, F (V_{m \times m}^{'})_{n \times m} x = F (V_{m \times m}^{'} x)_{n \times m} = W_{n \times n}^{'} A_{n \times m} x C o o r d i n a t e m a p p i n g : x_{(i n V)} ⟼ {A_{n \times m} x}_{(i n W)}

$\mathcal{F}(v_i)=\{w_1,w_2,\dots,w_n\}[a_{1i},a_{2i},\dots,a_{ni} ]^T \\ matrix\ representation:\quad A_{n\times m},\ \mathcal{F}(V'_{m\times m})_{n \times m}=W'_{n \times n}A_{n\times m} \\ so,\quad \mathcal{F}(V'_{m\times m})_{n \times m} x = \mathcal{F}(V'_{m\times m}x)_{n \times m} =W'_{n \times n}A_{n\times m} x \\ Coordinate\ mapping: \quad x_{(\mathrm{in}\ V)} \longmapsto \{A_{n\times m}x\}_{(\mathrm{in}\ W)}$

Matrix equivalence

A, B \in R^{m \times n} M a t r i x e q u i v a l e n c e : B = Q^{- 1} A P, Q a n d P i s i n v e r t i b l e m a t r i x .

$A,B\in \mathcal{R}^{m \times n} \\ Matrix\ equivalence: \quad B=Q^{-1}AP,\ Q \mathrm{\ and\ } P \mathrm{\ is\ invertible\ matrix.}$

Matrix similarity

A, B \in R^{n \times n} s i m i l a r : \exists i n v e r t i b l e P, B = P^{- 1} A P

$A,B\in \mathcal{R}^{n\times n} \\ similar:\quad \exists \ \mathrm{invertible}\ P,\ B=P^{-1}AP$

Invariant subspace

Consider a linear mapping T.

T : R^{n} \to R^{n} W i s s u b s p a c e o f R^{n}

$T:\mathbb {R} ^{n}\to \mathbb {R} ^{n} \\ W \mathrm{\ is\ subspace\ of\ } \mathbb{R}^{n}$

An invariant subspace W of T has the property that all vectors v \in W are transformed by T into vectors also contained in W. This can be stated as

I n v a r i a n t s u b s p a c e : \forall v \in W ⟹ T (v) \in W

$Invariant\ subspace: \quad \forall\ \mathbf {v} \in W\implies T(\mathbf {v} )\in W$

Vector Space

For having a vector space these two operations must satisfy the eight axioms, which are listed in the following table below, where the equations must be satisfied for every $u$ , $v$ and $w$ in $V$ , and $a$ and $b$ in $F$ .

Associativity of vector addition

$u + (v + w) = (u + v) + w$
Commutativity of vector addition

$u + v = v + u$
Identity element of vector addition

There exists an element $0 \in V$ , called the zero vector, such that $v + 0 = v$ for all $v \in V$ .
Inverse elements of vector addition

For every $v ∈ V$ , there exists an element $−v ∈ V$ , called the additive inverse of $v$ , such that $v + (−v) = 0$ .
Compatibility of scalar multiplication with field multiplication

$a(bv) = (ab)v$
Identity element of scalar multiplication

$1v = v$ , where $1$ denotes the multiplicative identity in $F$ .
Distributivity of scalar multiplication with respect to vector addition

$a(u + v) = au + av$
Distributivity of scalar multiplication with respect to field addition

$(a + b)v = av + bv$

Exercise:

$V$ is the set of positive real numbers, that is $V= \{x \in \mathbb{R} | x > 0 \}$ and $F=\mathbb{R}$ where vector addition is defined as $x\oplus y = xy$ and scalar multiplication is defined as $\alpha \otimes x = x^\alpha$ . Prove the above is vector spaces.

lambda matrix and Jordan

lambda matrix

F (λ) = a_{0} λ^{0} + a_{1} λ^{1} + \dots λ m a t r i x : A (λ)^{m \times n} = [\begin{matrix} F_{11} (λ) & \dots & F_{1 n} (λ) \\ ⋮ & ⋱ & ⋮ \\ F_{m 1} (λ) & \dots & F_{m n} (λ) \end{matrix}]

$\mathcal{F}(\lambda)=a_0\lambda^0+a_1\lambda^1+\cdots \\ \lambda\ \mathrm{matrix}:\quad A(\lambda)^{m\times n}= \left[\begin{matrix} \mathcal{F}_{11}(\lambda) & \cdots & \mathcal{F}_{1n}(\lambda) \\ \vdots & \ddots & \vdots \\ \mathcal{F}_{m1}(\lambda) & \cdots & \mathcal{F}_{mn}(\lambda) \end{matrix}\right]$

unitary matrix

U^{*} U = U U^{*} = I U i s i n v e r t i b l e w i t h U^{- 1} = U^{*} | det (U) | = 1

$U^{*}U=UU^{*}=I \\ U \mathrm{\ is\ invertible\ with\ }U^{-1}=U^{*} \\ \left|\det(U)\right|=1$

Smith normal form

(\begin{matrix} α_{1} & 0 & 0 & \dots & 0 \\ 0 & α_{2} & 0 & \dots & 0 \\ 0 & 0 & ⋱ & 0 \\ ⋮ & α_{r} & ⋮ \\ 0 \\ ⋱ \\ 0 & \dots & 0 \end{matrix})

$\begin{pmatrix}\alpha _{1}&0&0&&\cdots &&0\\0&\alpha _{2}&0&&\cdots &&0\\0&0&\ddots &&&&0\\\vdots &&&\alpha _{r}&&&\vdots \\&&&&0&&\\&&&&&\ddots &\\0&&&\cdots &&&0\end{pmatrix}$

α_{i} = \frac{d_{i} (A)}{d_{i - 1} (A)}, d_{i} (A) i s i - t h d e t e r m i n a n t d i v i s o r d_{0} (A) := 1

$\alpha _{i}={\frac {d_{i}(A)}{d_{{i-1}}(A)}}, \quad d_i(A)\mathrm{\ is\ }i\mathrm{-th\ determinant\ divisor} \\ d_{0}(A):=1$

Where di(A) (called i-th determinant divisor) equals the greatest common divisor of all i x i minors of the matrix A

Jordan normal form

J_{i} = [\begin{matrix} λ_{i} & 1 \\ λ_{i} & ⋱ \\ ⋱ & 1 \\ λ_{i} \end{matrix}] J o r d a n n o r m a l f o r m : J = [\begin{matrix} J_{1} \\ ⋱ \\ J_{p} \end{matrix}]

${J_i = \begin{bmatrix} \lambda_i & 1 & \; & \; \\ \; & \lambda_i & \ddots & \; \\ \; & \; & \ddots & 1 \\ \; & \; & \; & \lambda_i \end{bmatrix}}\\ Jordan\ normal\ form:\quad J = \begin{bmatrix} J_1 & \; & \; \\ \; & \ddots & \; \\ \; & \; & J_p\end{bmatrix}$

In general, a square complex matrix A is similar to a block diagonal matrix

\exists P, P^{- 1} A P = J

$\exists P,\ P^{−1}AP = J$

Inner product

porperty

symmetry:

< v_{1}, v_{2} >=< v_{2}, v_{1} >

$<v_1,v_2>=<v_2,v_1>$

Linear:

< v_{1}, v_{2} k + v_{3} l >=< v_{1}, v_{2} > k + < v_{1}, v_{3} > l

$<v_1,v_2k+v_3l>=<v_1,v_2>k+<v_1,v_3>l$

positive definitenes
s :

v \neq 0 (< v, v >)_{i n n e r p r o d u c t} > 0

$v\ne 0\\ (<v,v>)_{inner\ product}>0$

unitary spaces

v \in C

$v\in\mathbb{C}$

Conjugate symmetry or Hermitian symmetry:

⟨ v_{1}, v_{2} ⟩ = \bar{⟨ v_{2}, v_{1} ⟩}

$\langle v_1,v_2\rangle ={\overline {\langle v_2,v_1\rangle }}$

Linear:

< v_{1}, v_{2} k + v_{3} l >=< v_{1}, v_{2} > k + < v_{1}, v_{3} > l

$<v_1,v_2k+v_3l>=<v_1,v_2>k+<v_1,v_3>l$

positive definiteness :

v \neq 0 (< v, v >)_{i n n e r p r o d u c t} > 0

$v\ne 0\\ (<v,v>)_{inner\ product}>0$

example:

< v_{1} k, v_{2} l >= \bar{k} < v_{1}, v_{2} > l

$<v_1k,v_2l>=\overline{k}<v_1,v_2>l$

normal unitary spaces:

< v_{1}, v_{2} >= {\bar{v_{1}}}^{T} v_{2}

$<v_1,v_2>=\overline{v_1}^{T}v_2$

Gram matrix

G (x_{1}, \dots, x_{n}) = | \begin{matrix} ⟨ x_{1}, x_{1} ⟩ & ⟨ x_{1}, x_{2} ⟩ & \dots & ⟨ x_{1}, x_{n} ⟩ \\ ⟨ x_{2}, x_{1} ⟩ & ⟨ x_{2}, x_{2} ⟩ & \dots & ⟨ x_{2}, x_{n} ⟩ \\ ⋮ & ⋮ & ⋱ & ⋮ \\ ⟨ x_{n}, x_{1} ⟩ & ⟨ x_{n}, x_{2} ⟩ & \dots & ⟨ x_{n}, x_{n} ⟩ \end{matrix} |

${ G(x_{1},\dots ,x_{n})={\begin{vmatrix}\langle x_{1},x_{1}\rangle &\langle x_{1},x_{2}\rangle &\dots &\langle x_{1},x_{n}\rangle \\\langle x_{2},x_{1}\rangle &\langle x_{2},x_{2}\rangle &\dots &\langle x_{2},x_{n}\rangle \\\vdots &\vdots &\ddots &\vdots \\\langle x_{n},x_{1}\rangle &\langle x_{n},x_{2}\rangle &\dots &\langle x_{n},x_{n}\rangle \end{vmatrix}}}$

Hermite:

{\bar{G}}^{T} = G

$\overline{G}^{T}=G$

positive semidefinite:

{\bar{x}}^{T} G x = \sum_{i, j} {\bar{x}}_{i} ⟨ v_{i}, v_{j} ⟩ {\bar{x}}_{j} = \sum_{i, j} ⟨ v_{i} x_{i}, v_{j} x_{j} ⟩ = ⟨ \sum_{i} v_{i} x_{i}, \sum_{j} v_{j} x_{j} ⟩ = {‖ \sum_{i} x_{i} v_{i} ‖}^{2} ⩾ 0

$\overline{x}^{\textsf {T}}\mathbf {G} x=\sum _{i,j}\overline{x}_i\left\langle v_{i},v_{j}\right\rangle\overline{x}_j =\sum _{i,j}\left\langle v_{i}x_{i},v_{j}x_{j}\right\rangle =\left\langle \sum _{i}v_{i}x_{i},\sum _{j}v_{j}x_{j}\right\rangle =\left\|\sum _{i}x_{i}v_{i}\right\|^{2}\geqslant 0$

positive definite:

\forall x \neq 0, v_{1} x_{1} + v_{2} x_{2} + \dots + v_{n} x_{n} \neq 0 \Rightarrow {\bar{x}}^{T} G x > 0

$\forall x\ne 0,\ v_1x_1+v_2x_2+\cdots+v_nx_n \ne 0\\ \Rightarrow \overline{x}^T\mathbf{G}x > 0$

rank:

r a n k (G (x_{1}, \dots, x_{n})) = r a n k ([x_{1}, \dots, x_{n}]) A^{'} s i m a g e : {A x | x \in R^{n}} \in R^{m}

$rank(\mathbf{G}(x_{1},\dots ,x_{n}))=rank([x_{1},\dots ,x_{n}]) \\ A's\ image: \quad \{Ax\ |\ x\in \mathcal{R}^{n} \} \in \mathcal{R}^{m}$

Work on geometric space

‖ x ‖ = \sqrt{< x, x >}

$\|x\|=\sqrt{<x,x>}$

\begin{aligned} 1. x = 0, ‖ x ‖ = 0; x \neq 0, ‖ x ‖ \neq 0 \\ 2. ‖ λ x ‖ = | λ | \cdot ‖ x ‖ \\ 3. ‖ x + y ‖ ⩽ ‖ x ‖ + ‖ y ‖ \\ 4. | < x, y > | ⩽ ‖ x ‖ \cdot ‖ y ‖ \\ 5. < x, y >= 0, x ⊥ y \end{aligned}

$\begin{align} &1.\quad x=0,\|x\|=0;\ x\ne 0,\|x\|\ne 0\\ &2.\quad \|\lambda x\| = |\lambda|\cdot\|x\| \\ &3.\quad \|x+y\|\leqslant\|x\|+\|y\| \\ &4.\quad |<x,y>| \leqslant \|x\|\cdot\|y\| \\ &5.\quad <x,y>=0,x\perp y \end{align}$

projection matrices

β \in V^{n}, W_{s u b s p a c e}^{s} \in V^{n}, n ⩾ s q u e s t i o n : α = \arg min_{x \in W} (‖ β - x ‖)

$\beta \in V^{n},\ W_{subspace}^{s} \in V^{n},\ n\geqslant s \\ question:\ \alpha=\arg{\min\limits_{x\in W}{(\|\beta-x\|)}}$

\forall v \neq 0, β_{1} v_{1} + β_{2} v_{2} + \dots + β_{s} v_{s} \neq 0 (b a s e) α = β_{1} k_{1} + \dots + β_{s} k_{s} = [β_{1} β_{2} \dots β_{s}] k k = G (β_{1}, β_{2}, \dots, β_{s})_{s \times s}^{- 1} \cdot {[\begin{matrix} < β_{1}, β > \\ < β_{2}, β > \\ ⋮ \\ < β_{s}, β > \end{matrix}]}_{s \times 1}

$\forall v \ne 0,\ \beta_1v_1+\beta_2v_2+\cdots+\beta_sv_s \ne 0\quad (\mathrm{base}) \\ \alpha=\beta_1k_1+\cdots+\beta_sk_s=[\beta_1\ \beta_2\ \cdots\ \beta_s]k \\ k = \mathbf{G}(\beta_1,\beta_2,\dots,\beta_s)_{s\times s}^{-1}\cdot {\left[\begin{matrix} <\beta_1,\beta> \\ <\beta_2,\beta> \\ \vdots \\ <\beta_s,\beta> \end{matrix}\right]_{s\times 1}}$

W = i m A α = A ({\bar{A}}^{T} A)^{- 1} {\bar{A}}^{T} β p r o j e c t i o n m a t r i c e s : P_{A} = A ({\bar{A}}^{T} A)^{- 1} {\bar{A}}^{T}

$W=imA \\ \alpha = A(\overline{A}^{T}A)^{-1}\overline{A}^{T}\beta \\ projection\ matrices:\quad P_{A} = A(\overline{A}^{T}A)^{-1}\overline{A}^{T}$

porperty:

\begin{aligned} 1. {\bar{P_{A}}}^{T} = P_{A} \\ 2. P_{A}^{2} = P_{A} \\ 3. r a n k (P_{A}) = r a n k (A) \end{aligned}

$\begin{align} & 1.\quad \overline{P_{A}}^{T}=P_{A} \\ & 2.\quad P_{A}^{2} = P_{A} \\ & 3.\quad rank{(P_{A})} = rank{(A)} \end{align}$

Orthonormal basis

Fourier:

f, g = C ([0, 2 π], R^{1}) < f, g >= \int_{0}^{2 π} f (t) g (t) d t

$f,g=\mathfrak{C}([0, 2\pi], \mathbb{R}^{1}) \\ <f,g>=\int_{0}^{2\pi}f(t)g(t) dt$

unitary matrix

{\bar{A}}^{T} A = I_{n}

$\overline{A}^{T}A=I_n$

property:

\begin{aligned} 1. < A x, A y >=< x, y > \\ 2. ‖ A x ‖ = ‖ x ‖ \\ 3. | det (A) | = 1 \end{aligned}

$\begin{align} & 1.\quad <Ax,Ay>=<x,y> \\ & 2.\quad \|Ax\|=\|x\| \\ & 3.\quad \left|\det(A)\right|=1 \\ \end{align}$

QR decomposition

Any real square matrix A may be decomposed as

A = Q R

$A=QR$

Q is a unitary matrix.

where Q is an orthogonal matrix (its columns are orthogonal unit vectors and R is an upper triangular matrix (also called right triangular matrix). If A is invertible, then the factorization is unique if we require the diagonal elements of R to be positive.

Schur decomposition

The Schur decomposition reads as follows: if A is an n × n square matrix with complex entries, then A can be expressed as

{\bar{A}}^{T} A = A {\bar{A}}^{T} A = Q U Q^{- 1}

$\overline{A}^{T}A=A\overline{A}^{T} \\ A = Q U Q^{-1}$

where Q is a unitary matrix, and U is an upper triangular matrix, which is called a Schur form of A. Since U is similar to A, it has the same spectrum, and since it is triangular, its eigenvalues are the diagonal entries of U.

Hermitian matrix

A = {\bar{A}}^{T} ∴ \exists U_{u n i t a r y m a t r i x}, U^{- 1} A U = {\bar{U}}^{T} A U = [\begin{matrix} λ_{1} \\ λ_{2} \\ ⋱ \\ λ_{n} \end{matrix}]

$A=\overline{A}^{T} \\ \therefore \exists U_{unitary\ matrix},\ U^{-1}AU=\overline{U}^{T}AU={ \left[ \begin{matrix} \lambda_1 & & & \\ & \lambda_2 & & \\ & & \ddots & \\ & & & \lambda_{n} \\ \end{matrix} \right] }$

porperty:

The eigenvalue of A is belong to real number.

positive semidefinite

{\begin{aligned} A = {\bar{A}}^{T} \\ {\bar{x}}^{T} A x ⩾ 0 \end{aligned}

$\left\{\begin{align} A=\overline{A}^{T}\\ \overline{x}^{T}Ax \geqslant 0 \\ \end{align}\right.$

necessary and sufficient condition:

λ_{i} ⩾ 0

$\lambda_i \geqslant 0$

Other:

λ_{m a x} (A) = max_{‖ x ‖ = 1} {{\bar{x}}^{T} A x}

$\lambda_{max}(A)=\max\limits_{\|x\|=1}\{\overline{x}^{T}Ax\}$

SVD

Question:

A \in C^{m \times n}, f i n d V_{u n i t a r y} \in C^{n \times n}, U_{u n i t a r y} \in C^{m \times m} A V = U Q, m a k e Q a s s i m p l e a s p o s s i b l e .

$A\in \mathbb{C}^{m\times n }, find \quad V_{unitary}\in \mathbb{C}^{n \times n},U_{unitary}\in \mathbb{C}^{m\times m} \\ AV=UQ,\mathrm{\ make\ Q\ as\ simple\ as\ possible.}$

S V D : Q = {[\begin{array}{ccccc} σ_{1} & \dots & 0 \\ ⋮ & ⋱ & ⋮ & 0 \\ 0 & \dots & σ_{r} \\ 0 & 0 \end{array}]}_{m \times n}

$SVD: Q= {\left[ \begin{array}{ccc|cc} \sigma_1 & \cdots & 0 & & \\ \vdots & \ddots & \vdots & \ \ 0 \\ 0 & \cdots & \sigma_r & \\ \hline & 0 & & 0 \\ \end{array} \right]_{m\times n}} \\$

singular value:

σ_{i} > 0, i = 1, 2, \dots, r r = r a n k (A) σ_{i} = \sqrt{λ_{i} ({\bar{A}}^{T} A)}, i = 1, 2, \dots, r

$\sigma_i >0,\ i=1,2,\dots,r \\ r = rank(A) \\ \sigma_i=\sqrt{\lambda_i(\overline{A}^TA)},\ i = 1,2,\dots,r$

demonstrate:

A = [a_{1}, a_{2}, \dots, a_{n}], a_{i} \in C^{m \times 1} {\bar{A}}^{T} A = G (a_{1}, a_{2}, \dots, a_{n}) L e t : H = {\bar{A}}^{T} A

$A = [a_1,a_2,\cdots,a_n],\ a_i \in \mathbb{C}^{m\times 1} \\ \overline{A}^{T}A = \mathbf{G}(a_1,a_2,\dots,a_n) \\ Let:\quad H=\overline{A}^{T}A$

{\bar{V}}^{T} ({\bar{A}}^{T} A) V = {[\begin{array}{ccccc} λ_{1} & \dots & 0 \\ ⋮ & ⋱ & ⋮ & 0 \\ 0 & \dots & λ_{r} \\ 0 & 0 \end{array}]}_{n \times n} = {\bar{A V}}^{T} A V

$\overline{V}^{T}\left(\overline{A}^{T} A\right)V={\left[ \begin{array}{ccc|cc} \lambda_1 & \cdots & 0 & \\ \vdots & \ddots & \vdots & 0 \\ 0 & \cdots & \lambda_r & \\ \hline & 0 & & 0 \\ \end{array} \right]_{n\times n}} = \overline{AV}^TAV$

We can get eigenvectors (V) and eigenvalues by A.

l e t : B = A V = [b_{1}, b_{2}, \dots, b_{r} ⋮ b_{r + 1}, \dots, b_{n}] B_{1} = [b_{1}, b_{2}, \dots, b_{r}] B_{2} = [b_{r + 1}, \dots, b_{n}]

$let:\quad B = AV = [b_1,b_2,\dots,b_r\vdots\ b_{r+1},\dots,b_{n}] \\ B_1 = [b_1,b_2,\dots,b_r] \\ B_2 = [\ b_{r+1},\dots,b_{n}]$

{\bar{A V}}^{T} A V = [\begin{matrix} {\bar{B_{1}}}^{T} \\ {\bar{B_{2}}}^{T} \end{matrix}] \cdot [\begin{matrix} B_{1} & B_{1} \end{matrix}] = [\begin{matrix} {\bar{B_{1}}}^{T} B_{1} & {\bar{B_{1}}}^{T} B_{2} \\ {\bar{B_{2}}}^{T} B_{1} & {\bar{B_{2}}}^{T} B_{2} \end{matrix}]

$\overline{AV}^TAV = {\left[\begin{matrix} \overline{B_1}^{T} \\ \overline{B_2}^{T} \end{matrix}\right]} \cdot {\left[\begin{matrix} B_1 & B_1 \end{matrix}\right]} = {\left[\begin{matrix} \overline{B_1}^{T}B_1 & \overline{B_1}^{T}B_2 \\ \overline{B_2}^{T} B_1 & \overline{B_2}^{T} B_2 \end{matrix}\right]}$

∵ [\begin{matrix} {\bar{B_{1}}}^{T} B_{1} & {\bar{B_{1}}}^{T} B_{2} \\ {\bar{B_{2}}}^{T} B_{1} & {\bar{B_{2}}}^{T} B_{2} \end{matrix}] = {[\begin{array}{ccccc} λ & \dots & 0 \\ ⋮ & ⋱ & ⋮ & 0 \\ 0 & \dots & λ_{r} \\ 0 & 0 \end{array}]}_{n \times n} ∴ B_{2} = 0, {\bar{B_{1}}}^{T} B_{1} = {[\begin{matrix} λ & \dots & 0 \\ ⋮ & ⋱ & ⋮ \\ 0 & \dots & λ_{r} \end{matrix}]}_{r \times r}

$\because {\left[\begin{matrix} \overline{B_1}^{T}B_1 & \overline{B_1}^{T}B_2 \\ \overline{B_2}^{T} B_1 & \overline{B_2}^{T} B_2 \end{matrix}\right]} = {\left[ \begin{array}{ccc|cc} \lambda & \cdots & 0 & \\ \vdots & \ddots & \vdots & 0 \\ 0 & \cdots & \lambda_r & \\ \hline & 0 & & 0 \\ \end{array} \right]_{n\times n}} \\ \therefore B_2=0,\quad \overline{B_1}^{T}B_1 = {\left[ \begin{matrix} \lambda & \cdots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \cdots & \lambda_r \end{matrix} \right]_{r\times r}}$

∴ b_{1}, b_{2}, \dots, b_{r} i s i r r e l e v a n t .

$\therefore b_1,b_2,\dots,b_r \mathrm{\ is\ irrelevant.}$

normalization:

\tilde{b_{i}} = b_{i} \frac{1}{‖ b_{i} ‖} = b_{i} \frac{1}{\sqrt{λ_{i}}}

$\widetilde{b_i} = b_i \frac{1}{\|b_i\|} = b_i \frac{1}{\sqrt{\lambda_i}}$

Expand:

F i n d : β_{r + 1}, \dots . β_{m} L e t : \tilde{b_{1}}, \tilde{b_{2}}, \dots, \tilde{b_{r}}, β_{r + 1}, \dots . β_{m} i s i r r e l e v a n t . ∴ U = [\tilde{b_{1}}, \tilde{b_{2}}, \dots, \tilde{b_{r}}, β_{r + 1}, \dots . β_{m}]

$Find:\quad \beta_{r+1},\dots.\beta_{m} \\ Let: \quad \widetilde{b_1},\widetilde{b_2},\dots,\widetilde{b_r},\beta_{r+1},\dots.\beta_{m} \mathrm{\ is\ irrelevant.} \\ \therefore U=\left[\widetilde{b_1},\widetilde{b_2},\dots,\widetilde{b_r},\beta_{r+1},\dots.\beta_{m}\right]$

End:

∵ B_{2} = 0 ∴ B = [b_{1}, b_{2}, \dots, b_{r} ⋮ 0] ∵ U \cdot {[\begin{array}{ccccc} \sqrt{λ_{1}} & \dots & 0 \\ ⋮ & ⋱ & ⋮ & 0 \\ 0 & \dots & \sqrt{λ_{r}} \\ 0 & 0 \end{array}]}_{m \times n} = [b_{1}, b_{2}, \dots, b_{r} ⋮ 0]

$\because B_2=0\\ \therefore B=\left[b_1,b_2,\dots,b_r\vdots \ 0\right] \\ \because U \cdot{\left[ \begin{array}{ccc|cc} \sqrt{\lambda_1} & \cdots & 0 & & \\ \vdots & \ddots & \vdots & \ \ 0 \\ 0 & \cdots & \sqrt{\lambda_r} & \\ \hline & 0 & & 0 \\ \end{array} \right]_{m\times n}} = \left[b_1,b_2,\dots,b_r\vdots \ 0\right] \\$

∵ B = A V ∴ A V = U \cdot {[\begin{array}{ccccc} \sqrt{λ_{1}} & \dots & 0 \\ ⋮ & ⋱ & ⋮ & 0 \\ 0 & \dots & \sqrt{λ_{r}} \\ 0 & 0 \end{array}]}_{m \times n}

$\because B=AV \\ \therefore AV=U \cdot{\left[ \begin{array}{ccc|cc} \sqrt{\lambda_1} & \cdots & 0 & & \\ \vdots & \ddots & \vdots & \ \ 0 \\ 0 & \cdots & \sqrt{\lambda_r} & \\ \hline & 0 & & 0 \\ \end{array} \right]_{m\times n}}$

Application: (linear mapping),(decoupling)