The matrix inverse is defined only for square nonsingular matrices. A generalized inverse is an extension of the concept of inverse that applies to square singular matrices and rectangular matrices. There are many definitions of generalized inverses, all of which reduce to the usual inverse when the matrix is square and nonsingular.
A large class of generalized inverses of an matrix can be defined in terms of the Moore–Penrose conditions, in which is :
Here, the superscript denotes the conjugate transpose. A 1-inverse is any satisfying condition (1), a (1,3)-inverse is any satisfying conditions (1) and (3), and so on for any subset of the four conditions.
Condition (1) implies that if then , so solves the equation, meaning that any 1-inverse is an equation-solving inverse. Condition (2) implies that if .
A (1,3) inverse can be shown to provide a least squares solution to an inconsistent linear system. A (1,4) inverse can be shown to provide the minimum 2-norm solution of a consistent linear system (where the 2-norm is defined by ).
There is not a unique matrix satisfying any one, two, or three of the Moore–Penrose conditions. But there is a unique matrix satisfying all four of the conditions, and it is called the Moore-Penrose pseudoinverse, denoted by or . For any system of linear equations , minimizes and has the minimum 2–norm over all minimizers.
The pseudoinverse can be expressed in terms of the singular value decomposition (SVD). If is an SVD, where the matrix and matrix are orthogonal, and with (so that ), then
In MATLAB, the function
pinv computes using this formula. If then the concise formula holds.
For square matrices, the Drazin inverse is the unique matrix such that
where . The first condition is the same as the second of the Moore–Penrose conditions, but the second and third have a different flavour. The index of a matrix of is the smallest nonnegative integer such that ; it is characterized as the dimension of the largest Jordan block of with eigenvalue zero.
If then is also known as the group inverse of and is denoted by . The Drazin inverse is an equation-solving inverse precisely when , for then , which is the first of the Moore–Penrose conditions.
The Drazin inverse can be represented explicitly as follows. If
where and are nonsingular and has only zero eigenvalues, then
Here is the pseudoinverse and the Drazin inverse for a particular matrix with index :
The Moore–Penrose pseudoinverse is intimately connected with orthogonality, whereas the Drazin inverse has spectral properties related to those of the original matrix. The pseudoinverse occurs in all kinds of least squares problems. Applications of the Drazin inverse include population modelling, Markov chains, and singular systems of linear differential equations. It is not usually necessary to compute generalized inverses, but they are valuable theoretical tools.
This is a minimal set of references, which contain further useful references within.
- Adi Ben-Israel, The Moore of the Moore–Penrose Inverse, Electron. J. Linear Algebra 9, 150–157, 2002.
- Adi Ben-Israel and Thomas N. E. Greville, Generalized Inverses: Theory and Applications, second edition, Springer-Verlag, New York, 2003
- Stephen Campbell and Carl Meyer, Generalized Inverses of Linear Transformations, Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2009. published (Originally published by Pitman in 1979.)
- Stephen J. Kirkland and Michael Neumann, Group Inverses and -Matrices and their Applications, Chapman and Hall/CRC, 2013
- Guorong Wang, Yimin Wei and Sanzheng Qiao, Generalized Inverses: Theory and Computations, second edition, Springer-Verlag, Singapore, 2018.
Related Blog Posts
- What is a Matrix? (2020)
This article is part of the “What Is” series, available from https://nhigham.com/category/what-is and in PDF form from the GitHub repository https://github.com/higham/what-is.
3 thoughts on “What Is a Generalized Inverse?”
Beautiful “What is” about the pseudoinverse. Well written, concise, accurate. It is an art to write an article about anything — a news story, for example — that is both concise and accurate.
I have one quibble. As you probably know, I am interested in mathematical typography. I see that you are not using MathJax or MathML, Why not? You don’t have mathematics, you have pictures of mathematics. All of the math, even the A’s and X’s, are little .png files. The inline math doesn’t have the proper baseline. The displayed math looks OK in my browser, but it is pixelated when you print it or enlarge it. I know you are interested in this topic as well. How have you decided to use whatever mathematical typesetting you are using?
Should I submit this, minus this question, as a comment.
I hope you and your family are well,
FWIW on easy docs with Math (not suggesting you change your blog) this might be of interest to some people: https://casual-effects.com/markdeep/
Nice idea for a blog series. Thanks for doing it.