[Axler] Orthonormal Basis and Shur's Theorem

Orthonormal Bases (Part 3)

Based on Linear Algebra Done Right by Sheldon Axler

This post covers the concept of Orthonormal Bases, the Gram-Schmidt procedure, and important theorems like Schur's Theorem and the Riesz Representation Theorem.

(Assumption: $V$ denotes an inner product space over $F$, where $F$ is $\mathbf{R}$ or $\mathbf{C}$.)

---

1. Orthonormal Lists

Definition

A list of vectors in $V$ is called orthonormal if:

1. Each vector has norm 1.
2. Each vector is orthogonal to all other vectors in the list.

Mathematically, a list $e_1, \dots, e_m$ is orthonormal if:

$$ \langle e_j, e_k \rangle = \begin{cases} 1 & \text{if } j = k \\ 0 & \text{if } j \ne k \end{cases} $$

Key Properties

1. Norm of a Linear Combination:
If $e_1, \dots, e_m$ is an orthonormal list, then for any scalars $a_1, \dots, a_m$:

$$ \| a_1 e_1 + \dots + a_m e_m \|^2 = |a_1|^2 + \dots + |a_m|^2 $$

2. Linear Independence:
Every orthonormal list of vectors is linearly independent.

Proof Idea: If $\sum a_j e_j = 0$, then $\|\sum a_j e_j\|^2 = \sum |a_j|^2 = 0$, which implies all coefficients $a_j$ must be 0.

---

2. Orthonormal Bases

An orthonormal basis of $V$ is an orthonormal list of vectors that is also a basis of $V$.

Why are they important?

Orthonormal bases make finding coefficients in a linear combination extremely simple.

If $e_1, \dots, e_n$ is an orthonormal basis of $V$ and $v \in V$, then:

$$ v = \langle v, e_1 \rangle e_1 + \dots + \langle v, e_n \rangle e_n $$

And the norm of $v$ is given by:

$$ \|v\|^2 = |\langle v, e_1 \rangle|^2 + \dots + |\langle v, e_n \rangle|^2 $$

This means the coefficient of each basis vector is simply the inner product of the vector $v$ with that basis vector.

---

3. Gram-Schmidt Procedure

This is a standard algorithm for turning any linearly independent list into an orthonormal list with the same span.

• Input: A linearly independent list $v_1, \dots, v_m$.
• Output: An orthonormal list $e_1, \dots, e_m$ such that: $$ \operatorname{span}(v_1, \dots, v_j) = \operatorname{span}(e_1, \dots, e_j) \quad \text{for } j=1, \dots, m $$

Existence Theorems

Using the Gram-Schmidt procedure, we can prove:

1. Every finite-dimensional inner product space has an orthonormal basis.
2. Every orthonormal list of vectors in $V$ can be extended to an orthonormal basis of $V$.

---

4. Advanced Theorems

Schur's Theorem

Suppose $V$ is a finite-dimensional complex inner product space and $T \in \mathcal{L}(V)$. Then $T$ has an upper-triangular matrix with respect to some orthonormal basis of $V$.

Riesz Representation Theorem

Suppose $V$ is finite-dimensional. Then for every linear functional $\varphi$ on $V$ (a linear map from $V$ to $F$), there exists a unique vector $u \in V$ such that:

$$ \varphi(w) = \langle w, u \rangle $$

for every $w \in V$.

---

This concludes the video on orthonormal bases.