Linear Algebra: Key Theorems and Proofs

Abstract

This article surveys foundational theorems in linear algebra, focusing on the structure of vector spaces, matrix properties, and diagonalization. We examine the bases of column and null spaces, determinants, eigenvalues, and conditions for diagonalizability, with emphasis on how these concepts interconnect to provide insight into linear transformations.

Background

Linear algebra provides the mathematical framework for understanding linear transformations and vector spaces. At its core are questions about invertibility, solvability of systems, and the geometric meaning of matrix operations. The theorems discussed here form the backbone of computational and theoretical linear algebra, with applications spanning data science, physics, and engineering.

A matrix $A$ encodes a linear transformation, and its properties—whether it is invertible, how it scales space, which vectors it leaves unchanged—determine the behavior of systems it governs. Understanding these properties requires tools that extract structural information from the matrix itself.

Key Results

Column Space and Basis

The column space of a matrix $A$, denoted $\text{Col}(A)$, is the subspace spanned by all linear combinations of its columns ^{[basis-of-column-space]}. The basis of this space consists of the linearly independent columns that generate it.

A practical method for finding this basis is to perform row reduction on $A$ and identify the pivot columns. The columns of $A$ corresponding to pivot positions in the row echelon form constitute a basis for $\text{Col}(A)$ ^{[basis-of-column-space]}. Importantly, the dimension of the column space—the rank of $A$—equals the number of pivot columns.

This result is essential for understanding whether a system $Ax = b$ has a solution: the system is consistent if and only if $b$ lies in $\text{Col}(A)$.

Null Space and Basis

The null space of $A$, denoted $\text{Null}(A)$, is the set of all vectors $x$ satisfying the homogeneous equation $Ax = 0$ ^{[basis-of-null-space]}. Finding a basis for the null space requires solving this equation and expressing the general solution in terms of free variables.

The dimension of the null space equals the number of free variables in the solution ^{[basis-of-null-space]}. A non-trivial null space (dimension greater than zero) indicates that the transformation represented by $A$ collapses some directions to zero, revealing linear dependence among the columns.

The relationship between these two spaces is formalized by the Rank-Nullity Theorem: for an $m \times n$ matrix $A$, $\text{rank}(A) + \text{nullity}(A) = n$ where rank is the dimension of the column space and nullity is the dimension of the null space.

Determinant

The determinant of a square matrix $A$, denoted $\det(A)$, is a scalar that encodes critical information about the matrix ^{[determinant-of-a-matrix]}. For a $2 \times 2$ matrix, $\det\begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad - bc$

For larger matrices, the determinant can be computed via cofactor expansion or row reduction ^{[determinant-of-a-matrix]}.

The determinant serves multiple purposes: it indicates invertibility (a matrix is invertible if and only if $\det(A) \neq 0$), it represents the signed volume scaling factor of the linear transformation, and it is used to compute eigenvalues. A zero determinant signals that the transformation collapses space into a lower dimension, corresponding to linear dependence among rows or columns ^{[determinant-of-a-matrix]}.

Eigenvalues and Eigenvectors

For a square matrix $A$, an eigenvalue $\lambda$ is a scalar such that there exists a non-zero vector $v$ (called an eigenvector) satisfying $Av = \lambda v$

Eigenvalues are found by solving the characteristic equation ^{[eigenvalues-of-a-matrix]}: $\det(A - \lambda I) = 0$

This equation, obtained by rearranging $Av = \lambda v$ as $(A - \lambda I)v = 0$, yields a polynomial in $\lambda$ whose roots are the eigenvalues ^{[eigenvalues-of-a-matrix]}.

Eigenvalues reveal how the transformation stretches or compresses directions in space. They appear in stability analysis, oscillation frequencies, and growth rates across applications ^{[eigenvalues-of-a-matrix]}.

Diagonalization

A matrix $A$ is diagonalizable if it can be written as $A = PDP^{-1}$ where $D$ is a diagonal matrix and $P$ is an invertible matrix whose columns are eigenvectors of $A$ ^{[diagonalizable-matrix]}.

Diagonalization is powerful because it simplifies computation: raising $A$ to a power becomes $A^k = PD^kP^{-1}$ and since $D^k$ is diagonal, this is easy to compute ^{[diagonalizable-matrix]}.

A sufficient condition for diagonalizability is that $A$ has $n$ linearly independent eigenvectors (where $A$ is $n \times n$). This occurs when the eigenvectors form a basis for the vector space ^{[diagonalizable-matrix]}.

Worked Example

Consider the matrix $A = \begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix}$

Finding eigenvalues: Compute $\det(A - \lambda I)$: $\det\begin{pmatrix} 2-\lambda & 1 \\ 0 & 3-\lambda \end{pmatrix} = (2-\lambda)(3-\lambda) = 0$

So $\lambda_1 = 2$ and $\lambda_2 = 3$ ^{[eigenvalues-of-a-matrix]}.

Finding eigenvectors: For $\lambda_1 = 2$, solve $(A - 2I)v = 0$: $\begin{pmatrix} 0 & 1 \\ 0 & 1 \end{pmatrix}\begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = 0 \implies v_2 = 0$

An eigenvector is $v_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$.

For $\lambda_2 = 3$, solve $(A - 3I)v = 0$: $\begin{pmatrix} -1 & 1 \\ 0 & 0 \end{pmatrix}\begin{pmatrix} v_1 \\ v_2 \end{pmatrix} = 0 \implies v_1 = v_2$

An eigenvector is $v_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$.

Diagonalization: Since we have two linearly independent eigenvectors, $A$ is diagonalizable ^{[diagonalizable-matrix]}: $P = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}, \quad D = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$

and $A = PDP^{-1}$.

References

• ^{[basis-of-column-space]}
• ^{[basis-of-null-space]}
• ^{[determinant-of-a-matrix]}
• ^{[eigenvalues-of-a-matrix]}
• ^{[diagonalizable-matrix]}

AI Disclosure

This article was drafted with AI assistance from class notes (Zettelkasten). All mathematical claims are grounded in cited notes. The article was structured, written, and edited to ensure technical accuracy and clarity.