Normal matrices - unitary/orthogonal vs hermitian/symmetric
August 13, 2021 9 min read
Both orthogonal and symmetric matrices have orthogonal eigenvectors matrices. If we look at orthogonal matrices from the standpoint of outer products, as they often do in quantum mechanics, it is not immediately obvious, why they are not symmetric. The demon is in complex numbers - for symmetric matrices eigenvalues are real, for orthogonal they are complex.
In quantum mechanics you would sometimes run into normal matrices, which basically means a matrix that has unitary eigenvectors matrices. This definition includes both hermitian/symmetric and unitary/orthogonal matrices, as these 2 kinds of matrices both have unitary matrices of eigenvectors.
Let us prove that eigenvectors of symmetric/hermitian and unitary/orthogonal matrices are unitary/orthogonal:
Symmetric/hermitian matrices have orthogonal eigenvalues
For symmetric/hermitian matrix we have:
Unitary/orthogonal matrices have orthogonal eigenvalues
For orthogonal/unitary matrices the proof is different:
Suppose that is orthogonal/unitary, and are different eigenvalues with corresponding eigenvectors and . Then, as shown here, let us take their inner product:
Hence, either , so that eigenvectors are orthogonal, or .
Also, for any eigenvector we have , so absolute values of all the eigenvalues equals to 1, or , where is an arbitrary value.
As , and . Thus, (for instance, a special case of this is or ).
Thus, either our eigenvalues are identical and share the same eigenspace, and this is a degenerate case of eigenvalue multiplicity > 1), or the eigenvectors are orthogonal.
Strangely, the concept of outer product is not popular in Soviet/Russian mathematical literature; when I used to refer to it in my university days, my PIs (who had a strong background in linear algebra and functional analysis, being students of Israel M. Gelfand) were unaware of it. This is surprising, because I find it very helpful and elegant, and it is often used in quantum mechanics.
A reminder: if and are two vectors, their inner product is a scalar (a number):
Their outer product, however, is a matrix .
Now, if you have 2 matrices, and , their product is normally viewed as an outer product of inner products:
, where - row-vectors and - column-vectors.
But sometimes it can be useful to view it the other way around, as an inner product of outer products of their respective column-vectors by row-vectors:
, where - column-vectors and - row-vectors.
We’ll use the latter representation to interpret eigen decomposition using it and see, why symmetric matrix is symmetric, and why orthogonal matrix is not symmetric.
Eigendecomposition as an outer product: how comes that orthogonal matrix is not symmetric?
Let us consider a normal matrix , which can be either symmetric or orthogonal.
, where is a diagonal matrix of eigenvalues and is the orthogonal matrix of eigenvectors: , where , and are eigenvectors, so that their values are .
From the outer product standpoint we see:
It is obvious that this matrix is symmetric. So, how comes that orthogonal matrices are not symmetric?
Well, I lied here! Recall, that your eigenvectors (and eigenvalues) can be complex numbers, not real. So, in fact instead of just transposing the eigenvectors matrix we also have to take its complex conjugate, so in fact:
which makes the matrix A hermitian, if eigenvectors are real, i.e. for instance . However, if eigenvalues are complex-valued, there is no such symmetry in .
So, the main difference between orthogonal and symmetric matrices is that for symmetric matrix eigenvalues are real, and for orthogonal matrix they are complex.
Written by Boris Burkov who lives in Moscow, Russia and Cambridge, UK, loves to take part in development of cutting-edge technologies, reflects on how the world works and admires the giants of the past. You can follow me in Telegram