Eigenvalues and Eigenvectors

18.1 Eigenvalues and Eigenvectors

The usual function of a matrix is to act on a vector like a function, i.e. in goes a vector \(x\), out comes a vector \(Ax\). For a specific matrix \(A\), if \(Ax\) is parallel to \(x\) i.e. \(Ax = \lambda x\), the vectors \(x\) are called as eigenvectors. The constant \(\lambda\) is called as eigenvalue.

The vector \(x\) and constant \(\lambda\) satisfying the equation \(Ax = \lambda x\) are eigenvectors and eigenvalues. The eigenvalues and eigenvectors for some of the commonly used matrices are as follows:

• Projection Matrix: A projection matrix \(P\) takes a vector \(b\) and projects it as \(Pb\), where \(Pb\) is in the plane of \(P\). If the vector \(x\) is in the plane of \(P\), then it’s projection \(Px (=x)\) will be the same. This means that all the vectors in the plane of projection matrix \(P\) are eigenvectors with eigenvalues equal to \(\lambda = 1\). Apart from this, any \(x \perp P\) will have a \(0\) projection and hence an eigenvesctor with \(\lambda = 0\) eigenvalue.

• Permutation Matrix: Let us take an example of a permutation matrix \(A = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\), the eigenvalues and eigenvectors are \(\lambda_1 = 1, x_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix}; \lambda_2 = -1, x_2 = \begin{bmatrix} -1 \\ 1 \end{bmatrix}\).

• Rotation Matrix: Let \(Q = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\), then \(\begin{vmatrix} -\lambda & -1 \\ 1 & -\lambda \end{vmatrix} = 0 \implies \lambda^2 + 1 = 0 \implies \lambda = \pm i\). Here eigenvalues are not real. It should be noted that the eigenvalues are complex conjugate of each other. Intutively, finding the eigenvectors of a rotation matrix is like finding vectors which when rotated by \(90^\circ\) comes out to be the same. Only vector having this property is the \(0\) vector.

• Traiangular Matrix: Let \(Q = \begin{bmatrix} 3 & 1 \\ 0 & 3 \end{bmatrix}\), then \(\begin{vmatrix} 3-\lambda & 1 \\ 0 & 3-\lambda \end{vmatrix} = 0 \implies (3-\lambda)^2 = 0 \implies \lambda_1 = 3,\lambda_2 = 3\). We get \(x_1 = x_2 = \begin{bmatrix} 1\\ 0 \end{bmatrix}\). Hence, if we get repeated eigenvalues, we can have shortage of eigenvectors.

One of the most important fact about eigenvalue is: Sum of the eigenvalues of a matrix is the sum of the elements across it’s diagonal (called as trace) and the product of the eigenvalues is the value of the determinant.

18.2 How to solve \(Ax = \lambda x\)

We can rewrite the equation \(Ax = \lambda x\) as \((A - \lambda I)x = 0\). For this equation to have any other solution apart from \(x=0\), the matrix \(A - \lambda I\) has to be singular, i.e. \(|A - \lambda I| = 0\). Now to find \(x\) is like finding the null space of \((A - \lambda I)\).

Example: Take \(A = \begin{bmatrix} 3 & 1 \\ 1 & 3 \end{bmatrix}\). Then, \(|A - \lambda I| = \begin{vmatrix} 3-\lambda & 1 \\ 1 & 3-\lambda \end{vmatrix} = (3-\lambda)^2 - 1 = \lambda^2 - 6\lambda + 8\). In the equation \(\lambda^2 - 6\lambda + 8\), \(6\) is the trace (sum of diagonal elements) and \(8\) is the determinant of matrix \(A\). On solving this equation, we get the eigenvalues as \(\lambda_1 = 4,\lambda_2 = 2\). The eigenvector \(x_1\) can be obtained by solving the equation \((A - \lambda_1I)x_1 = 0 \implies (A - 4I)x_1 = 0 \implies \begin{bmatrix} -1 & 1 \\ 1 & -1 \end{bmatrix}x_1 = 0 \implies x_1 = \begin{bmatrix} 1 \\ 1 \end{bmatrix}\). Similarly, for \(\lambda_2\), \(x_2 = \begin{bmatrix} -1 \\ 1 \end{bmatrix}\).

If we look at the matrix in the example, it is \(3I\) away from the permutation matrix discussed above. It should be noted that the eigenvalues of the new matrix \(A\) can be obtained by adding \(3\) in the eigenvalues of the permutation matrix with eigenvectors remaining the same. This fact can be demonstrated mathematically as well.

If \(Ax = \lambda x\), then \((A + 3I)x = \lambda x + 3x = (\lambda + 3)x\). Hence for the new matrix \(A + 3I\), eigenvalues are \(\lambda + 3\) with eigenvectors remaining unchanged.