Positive Definite Matrices

24.1 Positive Definite Matrices

These are the complete tests for a \(2 \times 2\) matrix \(A = \begin{bmatrix} a & b \\ b & c \end{bmatrix}\) for being Positive Definite:

• Both the eigenvalues should be positive: \(\lambda_1 > 0;\lambda_2 > 0\)
• All the sub-determinants should be positive: \(a > 0; ac - b^2 > 0\)
• Pivots should be positive: \(a>0;\frac{ac-b^2}{a} > 0\)
• \(x^TAx > 0;\forall x\)

The matrix for which any of these conditions holds with equality instead are called as positive semi-definite matrices. For example, the matrix \(A = \begin{bmatrix} 2 & 6 \\ 6 & 18 \end{bmatrix}\) is a positive-semidifinite matrix. Let us run the \(x^TAx > 0\) test on this matrix. Let \(x = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}\), then

The matrix \(A\) is not positive definite as \(x^TAx \not>0;\forall x\). If we take \(A = \begin{bmatrix} 2 & 6 \\ 6 & 7 \end{bmatrix}\), this matrix is definitely not positive definite. The expression \(x^TAx = 2x^2 + 12xy + 7y^2\) where \(x = \begin{bmatrix} x \\ y \end{bmatrix}\). For \(A = \begin{bmatrix} 2 & 6 \\ 6 & 20 \end{bmatrix}\), the matrix is positive definite as \(x^TAx = 2x^2 + 12xy + 20y^2 > 0\) for all values. The plot of \(x^TAx\) for two matrices is shown in below figure.

from mpl_toolkits import mplot3d
import numpy as np
import matplotlib.pyplot as plt

def f(x, y):
    return 2*(x ** 2) + 12*x*y + 7*(y ** 2)

x = np.linspace(-100, 100, 100)
y = np.linspace(-100, 100, 100)
  
X, Y = np.meshgrid(x, y)
Z = f(X, Y)
 
fig = plt.figure(figsize=(16, 30))

ax = fig.add_subplot(1, 2, 1, projection='3d')
ax.plot_wireframe(X, Y, Z, color ='green')
ax.view_init(10, -120)
plt.xlabel('x', fontsize=12)
plt.ylabel('y', fontsize=12)
ax.set_title('Shape of the Curve:c=7');

def f(x, y):
    return (2*(x ** 2)) + (12*x*y) + (20*(y ** 2))

X, Y = np.meshgrid(x, y)
Z = f(X, Y)

ax = fig.add_subplot(1, 2, 2, projection='3d')
ax.plot_wireframe(X, Y, Z, color ='green')
ax.view_init(10, -120)
plt.xlabel('x', fontsize=12)
plt.ylabel('y', fontsize=12)
ax.set_title('Shape of the Curve:c=20');

Another way to find whether \(x^TAx\) is always positive or not is by completing the square. For example, \(x^TAx = 2x^2 + 12xy + 20y^2 = 2(x+3y)^2 + 2y^2\), and hence \(x^TAx\) is always positive. One of the important thing to note that the numbers which come in completing the square come from the row elimination steps with the pivot \(2\) outside and the row-multiplier \(3\) inside.

This can seen be one of the reasons for the pivots being positive for a positive definite matrix as positive pivots will give us positive completed square.

Let us take a \(3 \times 3\) matrix \(A = \begin{bmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{bmatrix}\). Is this matrix positive definite? The values of sub-determinants of this matrix are: \(2,3,4\) and hence it is positive definite. We can find the pivots using the fact that product of pivots give us the determinants. Hence first pivot is \(2\), second pivot is \(\frac{3}{2}\) and the third pivot is \(\frac{4}{2 \times \frac{3}{2}} = \frac{4}{3}\). Pivots are also positive and hence the matrix is positive definite.