A norm on is unitarily invariant if for all unitary and and for all . One can restrict the definition to real matrices, though the term unitarily invariant is still typically used.
Two widely used matrix norms are unitarily invariant: the -norm and the Frobenius norm. The unitary invariance follows from the definitions. For the -norm, for any unitary and , using the fact that , we obtain
For the Frobenius norm, using ,
since the trace is invariant under similarity transformations.
More insight into unitarily invariant norms comes from recognizing a connection with the singular value decomposition
Clearly, , so depends only on the singular values. Indeed, for the 2-norm and the Frobenius norm we have and . Here, and throughout this article, . Another implication of the singular value dependence is that for all for any unitarily invariant norm.
There is a beautiful characterization of unitarily invariant norms in terms of symmetric gauge functions, which are functions such that is an absolute norm on and for any permutation matrix and all . An absolute norm is one with the property that for all , and this condition is equivalent to the monotonicity condition that implies for all and .
A norm on is unitarily invariant if and only if for all for some symmetric gauge function , where are the singular values of .
The matrix -norm and the Frobenius norm correspond to being the vector -norm and the -norm, respectively. More generally, we can take for any vector -norm, obtaining the class of Schatten -norms:
The Schatten -norm is the sum of the singular values, , which is called the trace norm or nuclear norm. It can act as a proxy for the rank of a matrix. The trace norm can be expressed as , where is a polar decomposition.
Another class of unitarily invariant norms is the Ky Fan -norms
We have and .
Among unitarily invariant norms, the -norm and the Frobenius norm are widely usd in numerical analysis and matrix analysis. The nuclear norm is used in problems involving matrix rank minimization, such as matrix completion problems.
The benefit of the concept of unitarily invariant norm is that one can prove certain results for this whole class of norms, obtaining results for the particular norms of interest as special cases. Here are three important examples.
- For , the matrix is the nearest Hermitian matrix to in any unitarily invariant norm.
- For (), the unitary polar factor is the nearest matrix with orthonormal columns to in any unitarily invariant norm.
- The best rank- approximation to in any unitarily invariant norm is obtained by setting all the singular values beyond the th to zero in the SVD of . This result, which generalizes the Eckart-Young theorem, which covers the 2- and Frobenius norm instances, is an easy consequence of the following result of Mirsky (1960).
Let have SVDs with diagonal matrices , where the diagonal elements are arranged in nonincreasing order. Then for every unitarily invariant norm.
We also give a useful matrix norm inequality. For any matrices , , and for which the product is defined,
holds for any unitarily invariant norm, and in fact, any two of the norms on the right-hand side can be 2-norms.
This is a minimal set of references, which contain further useful references within.
- Roger Horn and Charles Johnson, Topics in Matrix Analysis, Cambridge University Press, 1991. Section 3.5.
- Roger A. Horn and Charles R. Johnson, Matrix Analysis, second edition, Cambridge University Press, 2013. Sections 5.6, 7.4. My review of the second edition.
- L. Mirsky, Symmetric Gauge Functions and Unitarily Invariant Norms, Quart. J. Math. 11, 50–59, 1960.
- Benjamin Recht, Maryam Fazel, and Pablo Parrilo, Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization, SIAM Rev. 52 (3), 471–501, 2010.
Related Blog Posts
- What Is the Nearest Symmetric Matrix? (2020)
- What Is the Polar Decomposition? (2020)
- What Is the Singular Value Decomposition? (2020)