## Subtitles section Play video

• As you can probably tell by now, the bulk of this series is on understanding matrix

• and vector operations

• through that more visual lens of linear transformations.

• This video is no exception, describing the concepts of inverse matrices,

• column space, rank, and null space through that lens.

• A forewarning though: I'm not going to talk about the methods for actually computing these

• things,

• and some would argue that that's pretty important.

• There are a lot of very good resources for learning those methods outside this series.

• Keywords: "Gaussian elimination" and "Row echelon form."

• I think most of the value that I actually have to add here is on the intuition half.

• Plus, in practice, we usually get software to compute this stuff for us anyway.

• First, a few words on the usefulness of linear algebra.

• By now, you already have a hint for how it's used in describing the the manipulation of

• space,

• which is useful for things like computer graphics and robotics,

• but one of the main reasons that linear algebra is more broadly applicable,

• and required for just about any technical discipline,

• is that it lets us solve certain systems of equations.

• When I say "system of equations," I mean you have a list of variables, things you don't

• know,

• and a list of equations relating them.

• In a lot of situations, those equations can get very complicated,

• but, if you're lucky, they might take on a certain special form.

• Within each equation, the only thing happening to each variable is that it's scaled by some

• constant,

• and the only thing happening to each of those scaled variables is that they're added to

• each other.

• So, no exponents or fancy functions, or multiplying two variables together; things like that.

• The typical way to organize this sort of special system of equations

• is to throw all the variables on the left,

• and put any lingering constants on the right.

• It's also nice to vertically line up the common variables,

• and to do that you might need to throw in some zero coefficients whenever the variable

• doesn't show up in one of the equations.

• This is called a "linear system of equations."

• You might notice that this looks a lot like matrix vector multiplication.

• In fact, you can package all of the equations together into a single vector equation,

• where you have the matrix containing all of the constant coefficients, and a vector containing

• all of the variables,

• and their matrix vector product equals some different constant vector.

• Let's name that constant matrix A,

• denote the vector holding the variables with a boldface x,

• and call the constant vector on the right-hand side v.

• This is more than just a notational trick to get our system of equations written on

• one line.

• It sheds light on a pretty cool geometric interpretation for the problem.

• The matrix A corresponds with some linear transformation, so solving Ax = v

• means we're looking for a vector x which, after applying the transformation, lands on

• v.

• Think about what's happening here for a moment.

• You can hold in your head this really complicated idea of multiple variables all intermingling

• with each other

• just by thinking about squishing and morphing space and trying to figure out which vector

• lands on another.

• Cool, right?

• To start simple, let's say you have a system with two equations and two unknowns.

• This means that the matrix A is a 2x2 matrix,

• and v and x are each two dimensional vectors.

• Now, how we think about the solutions to this equation

• depends on whether the transformation associated with A squishes all of space into a lower

• dimension,

• like a line or a point,

• or if it leaves everything spanning the full two dimensions where it started.

• In the language of the last video, we subdivide into the case where A has zero determinant,

• and the case where A has nonzero determinant.

• Let's start with the most likely case, where the determinant is nonzero,

• meaning space does not get squished into a zero area region.

• In this case, there will always be one and only one vector that lands on v,

• and you can find it by playing the transformation in reverse.

• Following where v goes as we rewind the tape like this,

• you'll find the vector x such that A times x equals v.

• When you play the transformation in reverse, it actually corresponds to a separate linear

• transformation,

• commonly called "the inverse of A"

• denoted A to the negative one.

• For example, if A was a counterclockwise rotation by 90º

• then the inverse of A would be a clockwise rotation by 90º.

• If A was a rightward shear that pushes j-hat one unit to the right,

• the inverse of a would be a leftward shear that pushes j-hat one unit to the left.

• In general, A inverse is the unique transformation with the property that if you first apply

• A,

• then follow it with the transformation A inverse,

• you end up back where you started.

• Applying one transformation after another is captured algebraically with matrix multiplication,

• so the core property of this transformation A inverse is that A inverse times A

• equals the matrix that corresponds to doing nothing.

• The transformation that does nothing is called the "identity transformation."

• It leaves i-hat and j-hat each where they are, unmoved,

• so its columns are one, zero, and zero, one.

• Once you find this inverse, which, in practice, you do with a computer,

• you can solve your equation by multiplying this inverse matrix by v.

• And again, what this means geometrically is that you're playing the transformation in

• reverse, and following v.

• This nonzero determinant case, which for a random choice of matrix is by far the most

• likely one,

• corresponds with the idea that if you have two unknowns and two equations,

• it's almost certainly the case that there's a single, unique solution.

• This idea also makes sense in higher dimensions,

• when the number of equations equals the number of unknowns.

• Again, the system of equations can be translated to the geometric interpretation

• where you have some transformation, A,

• and some vector, v,

• and you're looking for the vector x that lands on v.

• As long as the transformation A doesn't squish all of space into a lower dimension,

• meaning, its determinant is nonzero,

• there will be an inverse transformation, A inverse,

• with the property that if you first do A,

• then you do A inverse,

• it's the same as doing nothing.

• And to solve your equation, you just have to multiply that reverse transformation matrix

• by the vector v.

• But when the determinant is zero, and the transformation associated with this system

• of equations

• squishes space into a smaller dimension, there is no inverse.

• You cannot un-squish a line to turn it into a plane.

• At least, that's not something that a function can do.

• That would require transforming each individual vector

• into a whole line full of vectors.

• But functions can only take a single input to a single output.

• Similarly, for three equations in three unknowns,

• there will be no inverse if the corresponding transformation

• squishes 3D space onto the plane,

• or even if it squishes it onto a line, or a point.

• Those all correspond to a determinant of zero,

• since any region is squished into something with zero volume.

• It's still possible that a solution exists even when there is no inverse,

• it's just that when your transformation squishes space onto, say, a line,

• you have to be lucky enough that the vector v lives somewhere on that line.

• You might notice that some of these zero determinant cases feel a lot more restrictive than others.

• Given a 3x3 matrix, for example, it seems a lot harder for a solution to exist

• when it squishes space onto a line compared to when it squishes things onto a plane,

• even though both of those are zero determinant.

• We have some language that's a bit more specific than just saying "zero determinant."

• When the output of a transformation is a line, meaning it's one-dimensional,

• we say the transformation has a "rank" of one.

• If all the vectors land on some two-dimensional plane,

• We say the transformation has a "rank" of two.

• So the word "rank" means the number of dimensions in the output of a transformation.

• For instance, in the case of 2x2 matrices, rank 2 is the best that it can be.

• It means the basis vectors continue to span the full two dimensions of space, and the

• determinant is nonzero.

• But for 3x3 matrices, rank 2 means that we've collapsed,

• but not as much as they would have collapsed for a rank 1 situation.

• If a 3D transformation has a nonzero determinant, and its output fills all of 3D space,

• it has a rank of 3.

• This set of all possible outputs for your matrix,

• whether it's a line, a plane, 3D space, whatever,

• is called the "column space" of your matrix.

• You can probably guess where that name comes from.

• The columns of your matrix tell you where the basis vectors land,

• and the span of those transformed basis vectors gives you all possible outputs.

• In other words, the column space is the span of the columns of your matrix.

• So, a more precise definition of rank would be that

• it's the number of dimensions in the column space.

• When this rank is as high as it can be,

• meaning it equals the number of columns, we call the matrix "full rank."

• Notice, the zero vector will always be included in the column space,

• since linear transformations must keep the origin fixed in place.

• For a full rank transformation, the only vector that lands at the origin is the zero vector

• itself,

• but for matrices that aren't full rank, which squish to a smaller dimension,

• you can have a whole bunch of vectors that land on zero.

• If a 2D transformation squishes space onto a line, for example,

• there is a separate line in a different direction,

• full of vectors that get squished onto the origin.

• If a 3D transformation squishes space onto a plane,

• there's also a full line of vectors that land on the origin.