Normal matrices - unitary/orthogonal vs hermitian/symmetric
August 13, 2021 9 min read
Both orthogonal and symmetric matrices have orthogonal eigenvectors matrices. If we look at orthogonal matrices from the standpoint of outer products, as they often do in quantum mechanics, it is not immediately obvious, why they are not symmetric. The demon is in complex numbers - for symmetric matrices eigenvalues are real, for orthogonal they are complex.
In quantum mechanics you would sometimes run into normal matrices, which basically means a matrix that has unitary eigenvectors matrices. This definition includes both hermitian/symmetric and unitary/orthogonal matrices, as these 2 kinds of matrices both have unitary matrices of eigenvectors.
Let us prove that eigenvectors of symmetric/hermitian and unitary/orthogonal matrices are unitary/orthogonal:
Symmetric/hermitian matrices have orthogonal eigenvalues
For symmetric/hermitian matrix A we have:
Unitary/orthogonal matrices have orthogonal eigenvalues
For orthogonal/unitary matrices the proof is different:
Suppose that A is orthogonal/unitary, λ1 and λ2 are different eigenvalues with corresponding eigenvectors X and Y. Then, as shown here, let us take their inner product:
Hence, either XTY=0, so that eigenvectors are orthogonal, or λ1ˉλ2=1.
Also, for any eigenvector X we have 1=XTX=XT⋅(AT⋅A)⋅X=λi2XTX=λi2⟹∣λi∣=1, so absolute values of all the eigenvalues equals to 1, or λi=eit, where t is an arbitrary value.
As λ1ˉλ2=1, λ1ˉ=eit and λ2=e−it. Thus, λ1=λ2 (for instance, a special case of this is λ1=λ2=1 or λ1=λ2=−1).
Thus, either our eigenvalues are identical and share the same eigenspace, and this is a degenerate case of eigenvalue multiplicity > 1), or the eigenvectors are orthogonal.
Strangely, the concept of outer product is not popular in Soviet/Russian mathematical literature; when I used to refer to it in my university days, my PIs (who had a strong background in linear algebra and functional analysis, being students of Israel M. Gelfand) were unaware of it. This is surprising, because I find it very helpful and elegant, and it is often used in quantum mechanics.
A reminder: if X=⎝⎛x1x2x3⎠⎞ and Y=⎝⎛y1b2y3⎠⎞ are two vectors, their inner product is a scalar (a number):
Their outer product, however, is a matrix XYT=⎝⎛x1x2x3⎠⎞⋅(y1y2y3)=⎝⎛x1y1x2y1x3y1x1y2x2y2x3y2x1y3x2y3x3y3⎠⎞.
Now, if you have 2 matrices, A and B, their product is normally viewed as an outer product of inner products:
A⋅B=⎝⎛A1A2A3⎠⎞⋅(B1B2B3)=⎝⎛A1B1A2B1A3B1A1B2A2B2A3B2A1B3A2B3A3B3⎠⎞, where Ai=(a1,ia2,ia3,i) - row-vectors and Bi=⎝⎛bi,1bi,2bi,3⎠⎞ - column-vectors.
But sometimes it can be useful to view it the other way around, as an inner product of outer products of their respective column-vectors by row-vectors:
A⋅B=(A1A2A3)⋅⎝⎛B1B2B3⎠⎞=A1B1+A2B2+A3B3=i∑⎝⎛ai,1b1,iai,2b1,iai,3b1,iai,1b2,iai,2b2,iai,3b2,iai,1b3,iai,2b3,iai,3b3,i⎠⎞, where Ai=⎝⎛ai,1ai,2ai,3⎠⎞ - column-vectors and Bi=(b1,ib2,ib3,i) - row-vectors.
We’ll use the latter representation to interpret eigen decomposition using it and see, why symmetric matrix is symmetric, and why orthogonal matrix is not symmetric.
Eigendecomposition as an outer product: how comes that orthogonal matrix is not symmetric?
Let us consider a normal matrix A, which can be either symmetric or orthogonal.
A=EΛET, where Λ is a diagonal matrix of eigenvalues and E is the orthogonal matrix of eigenvectors: (E1E2E3), where E1, E2 and E3 are eigenvectors, so that their values are Ei=⎝⎛ei,1ei,2ei,3⎠⎞.
It is obvious that this matrix is symmetric. So, how comes that orthogonal matrices are not symmetric?
Well, I lied here! Recall, that your eigenvectors (and eigenvalues) can be complex numbers, not real. So, in fact instead of just transposing
the eigenvectors matrix we also have to take its complex conjugate, so in fact:
which makes the matrix A hermitian, if eigenvectors λi are real, i.e. for instance ∑iλiei,1ei,2∗=(∑iλiei,1∗ei,2)∗. However, if eigenvalues are complex-valued, there is no such symmetry in A.
So, the main difference between orthogonal and symmetric matrices is that for symmetric matrix eigenvalues are real, and for orthogonal matrix they are complex.
Written by Boris Burkov who lives in Moscow, Russia, loves to take part in development of cutting-edge technologies, reflects on how the world works and admires the giants of the past. You can follow me in Telegram