The Geometry of Linear Equations

May 20, 2019

One of the fundamental problem in linear algebra is to solve a system of equation.

A normal, nice case(for now) is to have N equations and N unknowns.

An exmaple,

$\begin{aligned} 2x-y & =0\\ -x+2y & =3 \end{aligned}\tag{1}$

writing in the form of a product of coefficient matrix and unknown vector,

$\begin{bmatrix} 2 & -1\\ -1 & 2 \end{bmatrix} \begin{bmatrix} x\\ y \end{bmatrix}=\begin{bmatrix} 0\\ 3 \end{bmatrix}\tag{2}$

Let $A=\begin{bmatrix} 2 & -1\\ -1 & 2 \end{bmatrix}, \mathbf x=\begin{bmatrix} x\\ y \end{bmatrix}, \mathbf b=\begin{bmatrix} 0\\ 3 \end{bmatrix}$ , then the above equation can be shorten as

$A\mathbf{x}=\mathbf{b}$

which is a common expression for a system of equations. $A$ is the coefficient matrix, $\mathbf x$ is the unknown vector, and $\mathbf b$ is the constant vector. If you fonud confusing on matrix multiplication, checkout the preliminary note

Row picture:

Draw the two equations in $(1)$ into a 2D coordinates:

$\begin{tikzpicture}[scale=1.0544]\small \begin{axis}[axis line style=gray, samples=120, width=6.0cm,height=6.4cm, xmin=-2.5, xmax=2.5, ymin=-2.5, ymax=2.5, %restrict y to domain=-2:2, ytick={-2,-1,0,1,2}, xtick={-2,-1,0,1,2}, axis equal, axis x line=center, axis y line=center, xlabel=$x$,ylabel=$y$] %\addplot[red,domain=-2:1,semithick]{exp(x)}; \addplot[color = {rgb:red,181;green,23;blue,23},line width = 1pt]{x/2+3/2}; \addplot[color = {rgb:red,16;green,133;blue,152}, line width = 1pt]{2*x}; \addplot[] coordinates {(2,3)} node{$2x-y=0$}; \addplot[] coordinates {(-1.5,1.5)} node{$-x+2y=3$}; \addplot[color = {rgb:red,146;green,98;blue,74},mark=*] coordinates {(1,2)}; %zero node \path (axis cs:0,0) node [anchor=north west,yshift=-0.07cm] {0}; \end{axis} \end{tikzpicture}$

By finding a line for the first equation and for second equation we can see an intersection point $(1,2)$ that can solve both equations.

Column Picture:

Rewriting the equation $(2)$ to expand $x$ and $y$ ,

$x\left[\begin{matrix} 2\\ -1 \end{matrix}\right] +y\left[\begin{matrix} -1\\ 2 \end{matrix}\right] =\left[\begin{matrix} 0\\ 3 \end{matrix}\right]$

right now the equation is asking us to find the right amount of $x$ and $y$ to combine the two vectors on the right of them to produce the correct output $\left[\begin{matrix}0\\3\end{matrix}\right]$ . This is equivalent to find the right linear combination of the columns of $A$ . Finding a linear combination of columns is a fundemental operation in linear algbera. View it in picture:

$\begin{tikzpicture}[scale=1.0544]\small \begin{axis}[axis line style=gray, samples=120, width=6.0cm,height=6.4cm, xmin=-2.5, xmax=2.5, ymin=-1.5, ymax=3.5, %restrict y to domain=-2:2, ytick={-1,0,1,2,3}, xtick={-2,-1,0,1,2}, axis equal, axis x line=center, axis y line=center, xlabel=$x$,ylabel=$y$] \addplot[->,color = {rgb:red,181;green,23;blue,23}, line width = 1pt] coordinates {(0,0) (2,-1)}; \addplot[->,color = {rgb:red,16;green,133;blue,152}, line width = 1pt] coordinates {(0,0) (-1,2)}; \addplot[dashed, ->,color = {rgb:red,16;green,133;blue,152}, line width = 1pt] coordinates {(2,-1) (1,1)}; \addplot[dashed, ->,color = {rgb:red,16;green,133;blue,152}, line width = 1pt] coordinates {(1,1) (0,3)}; \addplot[->,color = {rgb:red,66;green,177;blue,8}, line width = 1pt] coordinates {(0,0) (0,3)}; \addplot[color = {rgb:red,66;green,177;blue,8}] coordinates {(0.7,3)} node{$\left[\begin{array}{@{}c@{}} 0 \\ 3 \\ \end{array} \right]$}; \addplot[color = {rgb:red,181;green,23;blue,23}] coordinates {(2.5,-1)} node{$\left[\begin{array}{@{}c@{}} 2 \\ -1 \\ \end{array} \right]$}; \addplot[color = {rgb:red,16;green,133;blue,152}] coordinates {(-1,2.5)} node{$\left[\begin{array}{@{}c@{}} -1 \\ 2 \\ \end{array} \right]$}; %zero node \path (axis cs:0,0) node [anchor=north west,yshift=-0.07cm] {0}; \end{axis} \end{tikzpicture}$

As we have already found the solution to $x$ and $y$ which is 1 and 2, we can plug in and get the combination of column vectors to produce the final output. This is, 1 of column vector 1( $\left[\begin{matrix}2\\-1\end{matrix}\right]$ ), 2 of column vector 2 then we can get $\left[\begin{matrix}0\\3\end{matrix}\right]$ . We will discuss how to solve system of equations more generally using elimination in later notes.

We also see why this is a “column picture” because the coefficient matrix is splited into columns, instead of rows.

Let’s do a more complex example with three unknowns:

$A=\left[\begin{matrix} 2 & -1 & 0\\ -1 & 2 & -1\\ 0 & -3 & 4 \end{matrix}\right] ,b=\left[\begin{matrix} 0\\ -1\\ 4 \end{matrix}\right]$

if we write out the row picture, we will have:

$\begin{aligned} 2x-y&=0\\ -x+2y-z&=-1\\ -y+4z&=4 \end{aligned}$

The first and third equation will be a line. The second one will be a plane. It’s much harder to visualize in 3-D where the intersection will be. You can try draw it out. Then the advantages of column picture comes out:

$x\left[\begin{matrix}2\\-1\\0\end{matrix}\right]+y\left[\begin{matrix}-1\\2\\-3\end{matrix}\right]+z\left[\begin{matrix}0\\-1\\4\end{matrix}\right]=\left[\begin{matrix}0\\-1\\4\end{matrix}\right]$

It’s easy to see that the righthand side column is equal to 1 of the z’s column vector. The solution point will be $(0,0,1)$ .

Linear Independence Preview:

Can we solve $A\mathbf{x}=\mathbf{b}$ for every righthand side? In other words, do the linear combination of the columns fill 3-D space? The answer is yes in this case, because the three vectors are independent. Any two of them cannot constitute the third one. This reasoning is same in high dimensional case. If there’s some of the vectors, in general $n-1$ , can compose another vector in the matrix, then this matrix cannot span the whole space.

It’s okay if you do not grasp this for now, this topic will come out again and again in later videos and notes.