What Is an Invariant Subspace?

A subspace S of \mathbb{C}^n is an invariant subspace for A\in\mathbb{C}^{n \times n} if AS \subseteq S, that is, if x\in S implies Ax \in S.

Here are some examples of invariant subspaces.

  • \{0\} and \mathbb{C}^n are trivially invariant subspaces of any A.
  • The null space \mathrm{null}(A) = \{ x: Ax = 0\} is an invariant subspace of A because x\in\mathrm{null}(A) implies Ax = 0\in\mathrm{null}(A).
  • If x is an eigenvector of A then \mathrm{span}(x) = \{\, \alpha x: \alpha\in\mathbb{C} \,\} is a 1-dimensional invariant subspace, since A\alpha x = \lambda \alpha x \in S, where \lambda is the eigenvalue corresponding to x.

  • The matrix

    \notag \begin{bmatrix} 1 & 1 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{bmatrix}

    has a one-dimensional invariant subspace \mathrm{span}(e_1) and a two-dimensional invariant subspace \mathrm{span}(e_1, e_2), where e_i denotes the ith column of the identity matrix.

Let x_1,x_2, \dots,x_p\in\mathbb{C}^n be linearly independent vectors. Then S = \mathrm{span}(x_1,x_2, \dots,x_p) is an invariant subspace of A if and only if Ax_i\in S for i=1\colon p. Writing X = [x_1,x_2,\dots,x_p]\in\mathbb{C}^{n \times p}, this condition can be expressed as

\notag AX = XB, \qquad(1)

for some B\in\mathbb{C}^{p \times p}.

If p = n in (1) then AX = XB with X square and nonsingular, so X^{-1}AX = B, that is, A and B are similar.

Eigenvalue Relations

We denote by \Lambda(A) the spectrum (set of eigenvalues) of A and by A^+ the pseudoinverse of A.

Theorem.

Let A\in\mathbb{C}^{n\times n} and suppose that (1) holds for some full-rank X\in\mathbb{C}^{n\times p} and B\in\mathbb{C}^{p\times p}. Then \Lambda(B)\subseteq\Lambda(A). Furthermore, if (\lambda,x) is an eigenpair of A with x\in\mathrm{range}(X) then (\lambda,X^+x) is an eigenpair of B.

Proof. If (\lambda,z) is an eigenpair of B then AXz = XBz = \lambda Xz, and X z\ne 0 since the columns of X are independent, so (\lambda,Xz) is an eigenpair of A.

If (\lambda,x) is an eigenpair of A with x\in\mathrm{range}(X) then x = Xz for some z\ne0, and z = X^+x, since X being full rank implies that X^+X = I. Hence

\notag \lambda x = Ax = AXz = XBz.

Multiplying on the left by X^+ gives \lambda z = Bz, so (\lambda,z) is an eigenpair of B.

Block Triangularization

Assuming that X in (1) has full rank p we can choose Y\in\mathbb{C}^{p \times (n-p)} so that W = [X,\,Y] is nonsingular. Let W^{-1} = \left[\begin{smallmatrix} G \\ H \end{smallmatrix}\right] and note that W^{-1}W = I implies GX = I and HX = 0. We have

\notag W^{-1}AW = \begin{bmatrix} G \\H \end{bmatrix} [AX,\, AY] = \begin{bmatrix} G \\H \end{bmatrix} [XB,\, AY] = \begin{bmatrix} B & GAY\\ 0 & HAY \end{bmatrix}, \qquad (2)

which is block upper triangular. This construction is used in the proof of the Schur decomposition with p=1, x an eigenvector of unit 2-norm, and W chosen to be unitary.

The Schur Decomposition

Suppose A\in\mathbb{C}^{n \times n} has the Schur decomposition Q^*AQ = R, where Q is unitary and R is upper triangular. Then AQ = QR and writing Q = [Q_1,\,Q_2], where Q_1 is n\times p, and

\notag R = \begin{bmatrix} R_{11} & R_{12} \\ 0 & R_{22} \\ \end{bmatrix},

where R_{11} is p\times p, we have A Q_1 = Q_1 R_{11}. Hence Q_1 is an invariant subspace of A corresponding to the eigenvalues of A that appear on the diagonal of R_{11}. Since p can take any value from 1 to n, the Schur decomposition provides a nested sequence of invariant subspaces of A.

Notes and References

Many books on numerical linear algebra or matrix analysis contain material on invariant subspaces, for example

  • David S. Watkins. Fundamentals of Matrix Computations Third edition, Wiley, New York, USA, 2010.

The ultimate reference is perhaps the book by Gohberg, Lancaster, and Rodman, which has an accessible introduction but is mostly at the graduate textbook or research monograph level.

  • Israel Gohberg, Peter Lancaster, and Leiba Rodman, Invariant Subspaces of Matrices with Applications, Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2006 (unabridged republication of book first published by Wiley in 1986).

Related Blog Posts

This article is part of the “What Is” series, available from https://nhigham.com/index-of-what-is-articles/ and in PDF form from the GitHub repository https://github.com/higham/what-is.

Leave a comment