Linear Algebra on Amit Rajan

Left, Right and Pseudo Inverses

Sun, 15 May 2022 14:07:28 +0100

28.1 Left and Right Inverses

For any matrix $A$, there are four fundamental subspaces: Row Space, Null Space, Column Space, Null Space of $A^T$. Row Space and Null Space combined span entire $\mathbb{R}^n$ for a $m \times n$ matrix $A$. Column Space and Null Space of $A^T$ combined span entire $\mathbb{R}^m$ for a $m \times n$ matrix $A$. For the inverse of the matrix to exist, it’s rank should be $r=m=n$ whtere the dimension of the matrix is $m \times n$.

Linear Transformations, Change of Basis and Image Compression

Thu, 12 May 2022 14:07:28 +0100

27.1 Linear Transformations

When we are dealing with coordinates, every linear transformation leads us to a matrix. For any vector $v$ and $w$, linear transformation $T$ follows following properties:

$T(v+w) = T(v) + T(w)$
$T(cv) = cT(v)$
$T(0) = T(0)$, derived from tha above two properties

Below are some examples and non-examples of linear transformation:

Example 1: Projection In a two-dimensional space $\mathbb{R}^2$, a projection of a vector $v$ on a line is linear transformation and can be denoted as $T:\mathbb{R}^2 \to \mathbb{R}^2$.

Singular Value Decomposition

Mon, 09 May 2022 14:07:28 +0100

26.1 Singular Value Decomposition

If any matrix $A$ can be decomposed as $A = U\Sigma V^T$, where $\Sigma$ is a diagonal matrix and $U,V$ are orthogonal matrices, this is called as Singular Value Decomposition (SVD). One of the examples of SVD is for a symmetric positive definite matrix $A$, we know that $A = Q\Lambda Q^T$, where $\Lambda$ is diagonal and $Q$ is orthogonal.

For a matrix $m \times n$ matrix $A$, let the row-space be entire $\mathbb{R}^n$ and the column-space be entire $\mathbb{R}^m$. Any vector $v_1$ in the row-space can be transformed to a vector $u_1$ in the column-space as $u_1 = Av_1$. The goal of SVD is to find an orthogonal basis in row-space which can be transformed to an orthogonal basis in column-space. Let for a rank $r$ matrix $A$, the orthonormal basis (unit vectors) in the row-space consists of $v_1, v_2, ..., v_r$. Their transformations in the column-space are $\sigma_1u_1 = Av_1, \sigma_2u_2 = Av_2, ..., \sigma_ru_r = Av_r$ where $u_1, u_2, ..., u_r$ are also orthonormal (unit vectors and hence a factor of corresponding $\sigma$ for each $u$). In the matrix form, this can be represented as $AV=U\Sigma$ where $V$ and $U$ are the matrices whose columns are $v_1, v_2, ..., v_r$ and $u_1, u_2, ..., u_r$, and $\Sigma$ is a diagonal matrix of $\sigma_i$. Hence, $A = U\Sigma V^{-1} = U\Sigma V^{T}$ as $V$ has orthonormal columns. On further exploration, $A^TA = V\Sigma^{T}U^TU\Sigma V^T = V\Sigma^{T}(U^TU)\Sigma V^T = V\Sigma^{T}I\Sigma V^T = V\Sigma^{T}\Sigma V^T = V\Sigma^2V^T$, as $\Sigma$ is a daigonal matrix and hence $\Sigma^T \Sigma$ will have $\sigma_i^2$ at it’s diagonal. $A^TA = V\Sigma^2 V^T$ can be interpreted as symmetric positive definite matrix $A^TA$ decomposed into multiple of eigenvectors and eigenvalues, where $V$ is the eigenvector matrix of $A^TA$ and $\Sigma^2$ is the diagonal matrix having eigenvalues of $A^TA$. Similarly, $AA^T = U\Sigma^2U^T$ and above mentioned facts hold for it as well. We can find $U,V$ and $\Sigma$ using this method.

Similar Matrices

Fri, 06 May 2022 14:07:28 +0100

25.1 Similar Matrices

One of the major generator of a positive definite matrix is when we square one. In a nutsheel, the matrix $A^TA$ is always positive definite. To prove this, let $A$ be a $m \times n$ rectangular matrix. Then $A^TA$ is a square symmetric matrix. If we evaluate the expression $x^T(A^TA)x$, we get $x^T(A^TA)x = (x^TA^T)(Ax) = (Ax)^T(Ax) = |Ax|^2 > 0$ if $Ax$ is non-zero. Another important thing to note is if matrix $A, B$ are positive definite, $A+B$ is positive definite.

Positive Definite Matrices

Mon, 02 May 2022 14:07:28 +0100

24.1 Positive Definite Matrices

These are the complete tests for a $2 \times 2$ matrix $A = \begin{bmatrix} a & b \\ b & c \end{bmatrix}$ for being Positive Definite:

Both the eigenvalues should be positive: $\lambda_1 > 0;\lambda_2 > 0$
All the sub-determinants should be positive: $a > 0; ac - b^2 > 0$
Pivots should be positive: $a>0;\frac{ac-b^2}{a} > 0$
$x^TAx > 0;\forall x$

The matrix for which any of these conditions holds with equality instead are called as positive semi-definite matrices. For example, the matrix $A = \begin{bmatrix} 2 & 6 \\ 6 & 18 \end{bmatrix}$ is a positive-semidifinite matrix. Let us run the $x^TAx > 0$ test on this matrix. Let $x = \begin{bmatrix} x_1 \\ x_2 \end{bmatrix}$, then

Complex Matrices and Fourier Transform

Fri, 29 Apr 2022 14:07:28 +0100

23.1 Complex Vectors/Matrices

Let $z = \begin{bmatrix} z_1 \\ z_2 \\ ... \\ z_n \end{bmatrix}$ be a complex vector in $C^n$. The expresson $z^Tz$ doesn’t represent the length of the vector $z$. Instead it’s length is represented by $\overline{z}^Tz$. $\overline{z}^T$ is also called as $z^H$ and is called as Hermitian of a matrix. Hence, the length of a complex vector is $z^Hz = |z_1|^2 + |z_2|^2 + ... + |z_n|^2$. Similarly, for a complex matrix $A$ to be symmetric, $A^H = A$ with diagonal elements being real. In a complex domain, we call symmetric matrices as Hermitian Matrices. For two vectors $x$ and $y$ in a complex plane are perpendicular to each other if and only if $y^Hx = 0$. Hence, for an orrhogonal matrix (matrix with orthonormal columns) $Q$ in complex plane, $Q^HQ = I$. These orthogonal matrices in the complex plane are called as Unitary Matrices.

Symmetric Matrices and Positive Definiteness

Tue, 26 Apr 2022 14:07:28 +0100

22.1 Symmetric Matrices

For a Symmetric Matrix $A$, $A = A^T$. Eigenvalues of real Symmetric Matrices are real and eigenvectors are perpendicular. Any matrix $A$ can be written as $A = S\Lambda S^{-1}$. For a symmetric matrix $A$, this relationship reduces to $A = Q \Lambda Q^{-1}$ as $S$, which is the eigenvector matrix has orthonormal eigenvectors. For an orthonormal matrix, $Q^{-1} = Q^T$ and hence $A = Q\Lambda Q^T$. This is called as the Spectral Theorem in mathematics.

Markov Matrices and Fourier Series

Fri, 22 Apr 2022 14:07:28 +0100

21.1 Markov Matrices

Markov Matrices have the following properties:

All entries $\geq 0$
Sum of the entries in a column equal $1$

A markov matrix will always have an eigenvalue of $1$. Apart from this, all other eigenvalues will be $\leq 1$.

Let us consider the difference equation $u_k = A^ku_0$ where we represent $u_0$ as the combinations of eigenvectors, i.e. $u_0 = c_1x_1 + c_2x_2 + ... + c_nx_n = Sc$. Then $Au_0 = c_1Ax_1 + c_2Ax_2 + ... + c_nAx_n = c_1\lambda_1x_1 + c_2\lambda_2x_2 + ... + c_n\lambda_nx_n = \Lambda Sc$. Similarly, $A^ku_0 = c_1\lambda_1^kx_1 + c_2\lambda_2^kx_2 + ... + c_n\lambda_n^kx_n = \Lambda^kSc$.

Differential Equations and Matrix Exponentials

Wed, 20 Apr 2022 14:07:28 +0100

20.1 Differential Equations $\frac{du}{dt} = Au$

Example: Let the system of differential equation to be solved is: $\frac{du_1}{dt} = -u_1 + 2u_2; \frac{du_2}{dt} = u_1 - 2u_2$ with initial condition of $u(0) = \begin{bmatrix} 1 \\ 0 \end{bmatrix}$. The matrix $A$ representing the coefficients of the equation is $A = \begin{bmatrix} -1 & 2 \\ 1 & -2 \end{bmatrix}$. The eigenvalues of the matrix $A$ satisfies the equation $\lambda_1 + \lambda_2 = -3; \lambda_1 \times \lambda_2 = 0$, i.e. $\lambda_1 = 0, \lambda_2 = -3$ with the eigenvectors $x_1 = \begin{bmatrix} 2 \\ 1 \end{bmatrix}; x_2 = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$. The general solution of the set of differential equation is given as: $u(t) = c_1e^{\lambda_1t}x_1 + c_2e^{\lambda_2t}x_2$. Individual pure solutions can be checked by plugging in $e^{\lambda_1t}x_1$ and $e^{\lambda_2t}x_2$ to the equation $\frac{du}{dt} = Au$ and verifying the outcome.

Diagonalization and Powers of a Matrix

Sun, 17 Apr 2022 14:07:28 +0100

19.1 Diagonalization of a Matrix

Suppose for a given matrix $A$, we have $n$ linearlly independent eigenvectros and we put them in a matrix $S$. Then $AS = A \begin{bmatrix} x_1 & x_2 & ... & x_n \end{bmatrix}$, where each $x_i$ is the eigenvector. But for each of the eigenvectors, $Ax_i = \lambda_ix_i$. Hence, $AS = A \begin{bmatrix} x_1 & x_2 & ... & x_n \end{bmatrix} = \begin{bmatrix} \lambda_1x_1 & \lambda_2x_2 & ... & \lambda_nx_n \end{bmatrix} = \begin{bmatrix} x_1 & x_2 & ... & x_n \end{bmatrix}diag(\lambda_i) = S\Lambda$. Hence, $AS = S\Lambda$, where $\Lambda$ is a diagonal matrix containing eigenvalues and $S$ is the matrix of eigenvectors as the columns. If $S$ is invertible, i.e. the eigenvectors are independent, the equation can be rewritten as $S^{-1}AS = \Lambda$ or $A = S \Lambda S^{-1}$.

Eigenvalues and Eigenvectors

Thu, 14 Apr 2022 14:07:28 +0100

18.1 Eigenvalues and Eigenvectors

The usual function of a matrix is to act on a vector like a function, i.e. in goes a vector $x$, out comes a vector $Ax$. For a specific matrix $A$, if $Ax$ is parallel to $x$ i.e. $Ax = \lambda x$, the vectors $x$ are called as eigenvectors. The constant $\lambda$ is called as eigenvalue.

The vector $x$ and constant $\lambda$ satisfying the equation $Ax = \lambda x$ are eigenvectors and eigenvalues. The eigenvalues and eigenvectors for some of the commonly used matrices are as follows:

Formula for $A^{-1}$ and Cramer's Rule

Sun, 10 Apr 2022 14:07:28 +0100

17.1 Formula for $A^{-1}$

$A^{-1}$ can be given as $A^{-1} = \frac{1}{|A|}C^T$, where $C^T$ is the matrix of cofactors transposed. To verify this formula, we have to check that $AA^{-1}=I$, or $AC^T=|A|I$. If we expand the left hand side, we get

\[\begin{align} \begin{bmatrix} a_{11} & ... & a_{1n}\\ : & : & :\\ a_{n1} & ... & a_{nn} \end{bmatrix}\begin{bmatrix} C_{11} & ... & C_{n1}\\ : & : & :\\ C_{1n} & ... & C_{nn} \end{bmatrix}=\begin{bmatrix} |A| & 0 & 0\\ 0 & |A| & 0\\ 0 & 0 & |A| \end{bmatrix}=|A|I \end{align}\]

17.2 Cramer’s Rule

The solution for the equation $Ax=b$ can be given as $x=A^{-1}b=\frac{1}{|A|}C^Tb$. Different components of $x$ can be given as: $x_1 = \frac{|B_1|}{|A|}; x_2 = \frac{|B_2|}{|A|};...;x_j = \frac{|B_j|}{|A|};...$, where $B_1$ is the matrix $A$ with column 1 replaced by $b$. Hence, $B_j$ is the matrix $A$ with column $j$ replaced by $b$.

Determinant and Cofactors

Thu, 07 Apr 2022 14:07:28 +0100

16.1 Formula for $|A|$

For a $2 \times 2$ matrix $A$, the formula for $|A|$ can be derived as follows:

\[\begin{align} |A| = \begin{vmatrix} a & b \\ c & d \end{vmatrix}= \begin{vmatrix} a & 0 \\ c & d \end{vmatrix}+\begin{vmatrix} 0 & b \\ c & d \end{vmatrix} \end{align}\]\[\begin{align} =\begin{vmatrix} a & 0 \\ c & 0 \end{vmatrix}+\begin{vmatrix} a & 0 \\ 0 & d \end{vmatrix}+\begin{vmatrix} 0 & b \\ c & 0 \end{vmatrix}+\begin{vmatrix} 0 & b \\ 0 & d \end{vmatrix} \end{align}\]\[\begin{align} =0+ad-bc+0=ad-bc \end{align}\]

For a $3 \times 3$ matrix, first row can be seperated into $3$ pieces as demonstrated above. For each of the individual separated matrices, the second row will be separated into $3$ pieces giving a total of $9$ matrices. Finally, for each of these $9$ matrices, the third row will be separated into $3$ pieces, giving a total of $27$ matrices. Out of these $27$ matrices, a lot will have $0$ determinant. Matrices with non-zero determinant will have one entry from each row and column. The splitted matrices with non-zero determinant for a $3 \times 3$ matrix $A$ is shown below. The sign of individual determinants is derived based on number of row exchanges needed to get a diagonal matrix.

Determinant

Tue, 05 Apr 2022 14:07:28 +0100

15.1 Determinant

Every square matrix has a number assosiated with it can we call this number as determinant, often denoted as $det(A) = |A|$ for matrix $A$.

Prperties of determinant is as follows:

$|I|=1$
Row exchange reverses the sign of the determinant:

Permutation Matrices are derived by row exchange of Identity Matrix. Hence, $|P|= \pm1$.

(a) For any square matrix $A$, $\begin{vmatrix} ta & tb \\ c & d \end{vmatrix}=t\begin{vmatrix} a & b \\ c & d \end{vmatrix}$
(b) For any square matrix $A$, $|A|$ behaves like a linear function of a row if all the other rows are keep fixed. $\begin{vmatrix} a+a^{'} & b+b^{'} \\ c & d \end{vmatrix}=\begin{vmatrix} a & b \\ c & d \end{vmatrix}+\begin{vmatrix} a^{'} & b^{'} \\ c & d \end{vmatrix}$
If two rows of a square matrix $A$ are equal, $|A| = 0$: This can be proved using property 2. Exchanging the rows changes the sign of the determinant, but for a matrix $A$ which has two equal rows, the matrix obtained by exchanging these equal rows is same as $A$, i.e. $|A| = -|A| \implies |A|=0$.
Subtracting a multiple of one row from another, doesn’t change the determinant:

Orthonormal Vectors, Orthogonal Matrices and Gram-Schmidt Method

Sat, 02 Apr 2022 14:07:28 +0100

14.1 Orthonormal Vectors & Orthogonal Matrices

A set of Orthonormal Vectors can be defined as:

\[\begin{align} q_i^Tq_j = \begin{cases} 0 ,& \text{if } i \neq j \\ 1 ,& \text{if } i=j \end{cases} \end{align}\]

When these set of $n$ orthonormal vectors are put into a matrix $Q$ such that $Q = \begin{bmatrix} q_1 & q_2 & ... & q_n \end{bmatrix}$, then we get $Q^TQ = I$. It should be noted that $Q$ doesn’t have to be a square matrix. When $Q$ is a square matrix, we can call it Orthogonal. If $Q$ is a square matrix, $Q^TQ=I$ gives us $Q^T=Q^{-1}$. Hence, a square matrix whose columns are $\perp$ to each other and are of unit length is called as orthogonal matrix. One of the example of orthogonal matrix is given below.

Projection Matrices and Least Squares

Wed, 30 Mar 2022 14:07:28 +0100

13.1 Projection Matrices

Let us look at the two extreme cases while taking the projection of vector $b$ onto the plane represented by matrix $A$. The projection matrix $P$ is given as: $P = A(A^TA)^{-1}A^T$.

Case 1: When $b$ is $\perp$ to the column space of $A$, it’s projection $p=0$. This means that $b$ lies in the null space of $A^T$, i.e. $A^Tb=0$. Hence, $p = Pb = A(A^TA)^{-1}A^Tb = 0$.
Case 1: When $b$ is in the column space of $A$, it’s projection $p = b$. Any vector in the column space of $A$ will be linear combination of it’s columns i.e. $b = Ax$. Hence, $p = Pb = A(A^TA)^{-1}A^TAx = A(A^TA)^{-1}(A^TA)x = AIx = Ax = b$.

Projection of a Matrix

Sat, 26 Mar 2022 14:07:28 +0100

12.1 Projections in one-dimensional Space

Given two vectors $a,b$ in a plane, the projection of $b$ onto $a$ is shown below. The projection $p$ will be a multiple of $a$ and the error vector $e$ will be orthogonal to $a$. Error vector can be denoted as: $e = b - p$. As $e \perp a$, we can say that: $a^Te =0 \implies a^T(b - p) = 0 \implies a^T(b - xa) = 0 \implies xa^Ta = a^Tb \implies x = \frac{a^Tb}{a^Ta}$. Hence, $p = ax = a\frac{a^Tb}{a^Ta}$.

Orthogonal Vectors and Orthogonal Subspaces

Wed, 23 Mar 2022 14:07:28 +0100

11.1 Orthogonal Vectors

Orthogonal means perpendicular. For two vectors $x,y$, they are orthogonal if and only if $x^Ty=0$. As per Pythagoras Theorem, the test for orthogonality is: $\lVert x \rVert^2 + \lVert y \rVert^2 = \lVert x+y \rVert^2$.

We can conclude the dot product condition from Pythoagoras Theorem as follows:

\[\begin{align} \lVert x \rVert^2 + \lVert y \rVert^2 = \lVert x+y \rVert^2 \\ x^Tx + y^Ty = (x+y)^T(x+y) \\ x^Tx + y^Ty = x^Tx + y^Ty + x^Ty + y^Tx \\ x^Ty + y^Tx = 0 \\ 2x^Ty = 0 \\ x^Ty = 0 \end{align}\]

One importnt thing to note is: zero vector is orthogonal to all the other vectors.

Graphs, Networks and Incidence Matrices

Mon, 21 Mar 2022 14:07:28 +0100

10.1 Graphs, Networks and Incidence Matrices:

Graphs consist of nodes and edges. For example, in the attached figure, the graph has $n=4$ nodes and $m=5$ edges. The graph can be interpreted as a circuit where nodes represent the points from which current flows and edges with the arrow represent the direction of its flow.

The above graph can be represented using a matrix, called as Incidence Matrix. The edges of the graph are represented using rows where entry at each column (one columnn for each of the node) index represents the start and end of the edge. An entry of $-1$ marks the start node of the edge and that of $1$ marks the end node of the edge. For example, $edge 1$ between node $1$ and $2$ can be represented by the row $\begin{bmatrix}-1 & 1 & 0 & 0\end{bmatrix}$. The overall Incidence Matrix where rows are numbered as per the edges is shown below.

Matrix Spaces

Thu, 17 Mar 2022 14:07:28 +0100

9.1 Matrix Spaces

The idea of vector spaces can be extended to matrices as well as far as they follow the following properties

If $A \in S$ then $cA \in S$
If $A \in S; B \in S$ then $A+B \in S$

It should be noted that the matrix multiplication doesn’t need to belong to the same matrix space.

For a matrix $M$, some of the examples of matrix spaces are: Upper Triangular Mtrices, Symmetric Matrices, Diagonal Matrices etc. For a $3 \times 3$ matrix $M$, the basis can be given as ($dim(M) = 9)$:

Four Fundamental Subspaces

Tue, 15 Mar 2022 14:07:28 +0100

8.1 Four Fundamental Subspaces

The four fundamental subspaces for a $m \times n$ matrix $A$ are as follows:

Column Space $C(A)$ in $\mathbb{R}^m$
Null Space $N(A)$ in $\mathbb{R}^n$: Solution to $Ax=0$
Row Space $C(A^T)$ in $\mathbb{R}^n$: All combinations of the rows of $A$ or we can say that all combinations of the columns of $A^T$
Left Null Space of $A^T$ $N(A^T)$ in $\mathbb{R}^m$: Solution to $A^Ty=0$ and is also called as Left Null Spcae of $A$

The pictorial representation of these spaces with their dimension and basis is as follows:

Matrix Independence, Span, Basis & Dimension

Sun, 30 Jan 2022 14:07:28 +0100

7.1 Independence

Vectors $x_1,x_2, ..., x_n$ are linearly independent if no combinations of these vectors gives zero vector, except the zero combination. i.e.

\[\begin{align} c_1x_1 + c_2x_2 + ... + c_nx_n \neq 0; except \ \forall c_i=0 \end{align}\]

As we know that, for a $m \times n$ matrix $A$, if $m < n$, then we will have at least one non-zero solution to $Ax=0$. It implies that we will always find some non-zero combinations of $c_i$ that will make satisfy the above condition. This means that whenever we are in a m-dimensional space with $n$ vectors in it such that $m < n$, these $n$ vectors will always be linearly dependent.

Algorithm for solving $Ax=b$

Tue, 25 Jan 2022 14:07:28 +0100

6.1 Algorithm for solving $Ax=b$

The idea behind this exercise is to come up with the solution for $Ax=b$. Let us start by taking a matrix $A$ and vector $b$.

\[\begin{align} A = \begin{bmatrix} 1 & 2 & 2 & 2 \\ 2 & 4 & 6 & 8 \\ 3 & 6 & 8 & 10 \end{bmatrix}; b = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \end{bmatrix} \end{align}\]

The augumented matrix $\begin{bmatrix} A & b \end{bmatrix}$ can be formed and elimination steps can be performed on it as follows.

Algorithm for solving $Ax=0$

Thu, 20 Jan 2022 14:07:28 +0100

5.1 Algorithm for solving $Ax=0$

The idea behind this exercise is to come up with an algorithm to find the nullspace of a matrix $A$. Let us start by taking a matrix $A$.

\[\begin{align} A = \begin{bmatrix} 1 & 2 & 2 & 2 \\ 2 & 4 & 6 & 8 \\ 3 & 6 & 8 & 10 \end{bmatrix} \end{align}\]

One of the first thing to notice in $A$ is that $row_3$ is a linear combination of $row_1$ and $row_2$ ($row_3 = row_1 + row_2$). The way to solve a system of linear equations is by elimination. Different row elimination steps can be used to transform $A$ into an upper triangular matrix with the same steps repeated on the right hand side. As the right hand side is $0$ here, the elimination steps can be skipped for it.

Vector Space and Subspace

Mon, 17 Jan 2022 14:07:28 +0100

4.1 Vector Space and Subspace

Vector Space is a set of vectors whose linear combinations belong to the same set. This means that whenever we pick $2$ vectors from a vector space and find thier linear combination, the resultant vector will be in the vector space. A vector space in two and three dimensions is $R^2$ and $R^3$.

We can make some common observations about vector spaces. Let $u,v$ be two vectors, then their linear combination can be described as $w=au+bv$ where $a,b$ are scalars. If $a=b=0$, $w=au+bv=0$. This means that for any vector space (which contains vectors $u,v$), the zero vector will always be in it. This observation gives us a powerful tool to deduce whether a set is a vector space or not. If zero vector is not in the set, it’s not a vector space.

Inverse of a Matrix & Factorization into $A=LU$

Wed, 12 Jan 2022 14:07:28 +0100

3.1 Inverse of a Matrix & Factorization into $A=LU$

Given a square matrix $A$, $A^{-1}$ is it’s inverse if $AA^{-1} = A^{-1}A = I$, where $I$ is the Identity Matrix and we say that $A$ is invertible or nonsingular.

For a singular matrix $A$, the inverse does not exist and it’s determinant is $0$. Also, we can find a vector $x$ such that $Ax=0$. This means that one or more than one columns of $A$ is linear combination of other columns. For example, below $2 \times 2$ matrix is not invertible.

Elimination & Permutation with Matrices

Sat, 08 Jan 2022 14:07:28 +0100

2.1 Elimination & Permutation with Matrices

Elimination is the method which is used to solve a system of linear equations. Let $Ax=b$ is a system of linear equation where

\[\begin{align} A = \begin{bmatrix} 1 & 2 & 1\\ 3 & 8 & 1\\ 0 & 4 & 1 \end{bmatrix}, b = \begin{bmatrix} 2 \\ 12 \\ 2 \end{bmatrix} \end{align}\]

The idea of elimination is to transform $A$ into a matrix where all the entries in the cell below the diagonal is 0. The transformed matrix is called as upper triangular matrix (denoted by $U$). Once we have the upper triangular matrix, the system of equation can be easly solved using back substitution.

Geometry of Linear Equations & Matrix Multiplications

Mon, 03 Jan 2022 14:07:28 +0100

1.1 The Geometry of Linear Equations

The fundamental goal of linear algebra is to solve a system of linear equations. Let us look at an example of a set of $2$ linear equations in $2$ unknowns:

\[\begin{align} 2x-y = 0 \\ -x+2y = 3 \end{align}\]

The above system of linear equation when written in matrix multiplication form can be represented as $Ax = b$ where $A$ is a $2 \times 2$ matrix, $x$ and $b$ are column vectors.