## Subtitles section Play video

• Okay.

• This lecture is mostly about the idea of similar matrixes.

• I'm going to tell you what that word similar means

• and in what way two matrixes are called similar.

• But before I do that, I have a little more

• to say about positive definite matrixes.

• You can tell this is a subject I think is really important and I

• told you what positive definite meant --

• it means that this --

• this expression, this quadratic form, x transpose I

• x is always positive.

• But the direct way to test it was with eigenvalues

• or pivots or determinants.

• So I -- we know what it means, we know how to test it,

• but I didn't really say where positive definite matrixes come

• from.

• And so one thing I want to say is that they come from least

• squares in -- and all sorts of physical problems start with

• a rectangular matrix -- well, you remember in least squares

• the crucial combination was A transpose A.

• So I want to show that that's a positive definite matrix.

• Can -- so I --

• I'm going to speak a little more about positive definite

• matrixes, just recapping --

• so let me ask a question.

• It may be on the homework.

• Suppose a matrix A is positive definite.

• I mean by that it's all --

• I'm assuming it's symmetric.

• That's always built into the definition.

• So we have a symmetric positive definite matrix.

• Is the inverse of a symmetric positive definite matrix also

• symmetric positive definite?

• So you quickly think, okay, what do I

• know about the pivots of the inverse matrix?

• Not much.

• What do I know about the eigenvalues

• of the inverse matrix?

• Everything, right?

• The eigenvalues of the inverse are

• one over the eigenvalues of the matrix.

• So if my matrix starts out positive definite,

• then right away I know that its inverse is positive definite,

• because those positive eigenvalues --

• then one over the eigenvalue is also positive.

• What if I know that A -- a matrix A and a matrix B are

• both positive definite?

• But let me ask you this.

• Suppose if A and B are positive definite, what about --

• what about A plus B?

• In some way, you hope that that would be true.

• It's -- positive definite for a matrix is kind of like positive

• for a real number.

• But we don't know the eigenvalues of A plus B.

• We don't know the pivots of A plus B.

• So we just, like, have to go down this list of, all right,

• which approach to positive definite

• can we get a handle on?

• And this is a good one.

• This is a good one.

• Can we -- how would we decide that --

• if A was like this and if B was like this,

• then we would look at x transpose A plus B x.

• I'm sure this is in the homework.

• Now -- so we have x transpose A x bigger than zero,

• x transpose B x positive for all -- for all x,

• guy.

• And of course, you just add that and that

• and we get what we want.

• If A and B are positive definites, so is A plus B.

• So that's what I've shown.

• So is A plus B.

• Just -- be sort of ready for all the approaches through

• eigenvalues and through this expression.

• And now, finally, one more thought about positive definite

• is this combination that came up in least squares.

• Can I do that?

• So now -- now suppose A is rectangular, m by n.

• I -- so I'm sorry that I've used the same letter A

• for the positive definite matrixes in the eigenvalue

• chapter that I used way back in earlier chapters when

• the matrix was rectangular.

• Now, that matrix -- a rectangular matrix,

• no way its positive definite.

• It's not symmetric.

• It's not even square in general.

• But you remember that the key for these rectangular ones

• was A transpose A.

• That's square.

• That's symmetric.

• Those are things we knew --

• we knew back when we met this thing

• in the least square stuff, in the projection stuff.

• But now we know something more --

• we can ask a more important question, a deeper question --

• is it positive definite?

• And we sort of hope so.

• Like, we -- we might --

• in analogy with numbers, this is like --

• sort of like the square of a number, and that's positive.

• So now I want to ask the matrix question.

• Is A transpose A positive definite?

• Okay, now it's -- so again, it's a rectangular A that

• I'm starting with, but it's the combination A transpose A

• that's the square, symmetric and hopefully positive definite

• matrix.

• So how -- how do I see that it is positive definite,

• or at least positive semi-definite?

• You'll see that.

• Well, I don't know the eigenvalues of this product.

• I don't want to work with the pivots.

• The right thing -- the right quantity to look at is this,

• x transpose Ax --

• A -- x transpose times my matrix times x.

• I'd like to see that this thing --

• that that expression is always positive.

• I'm not doing it with numbers, I'm doing it with symbols.

• Do you see -- how do I see that that expression comes out

• positive?

• I'm taking a rectangular matrix A and an A transpose --

• that gives me something square symmetric,

• but now I want to see that if I multiply --

• that if I do this --

• I form this quadratic expression that I

• get this positive thing that goes upwards when I graph it.

• How do I see that that's positive,

• or absolutely it isn't negative anyway?

• We'll have to, like, spend a minute on the question

• could it be zero, but it can't be negative.

• Why can this never be negative?

• The argument is --

• like the one key idea in so many steps in linear algebra --

• put those parentheses in a good way.

• Put the parentheses around Ax and what's the first part?

• What's this x transpose A transpose?

• That is Ax transpose.

• So what do we have?

• We have the length squared of Ax.

• We have -- that's the column vector Ax that's the row vector

• Ax, its length squared, certainly greater than

• or possibly equal to zero.

• So we have to deal with this little possibility.

• Could it be equal?

• Well, when could the length squared be zero?

• Only if the vector is zero, right?

• That's the only vector that has length squared zero.

• So we have -- we would like to --

• I would like to get that possibility out of there.

• So I want to have Ax never -- never be zero,

• except of course for the zero vector.

• How do I assure that Ax is never zero?

• The -- in other words, how do I show that there's no null space

• of A?

• The rank should be --

• so now remember -- what's the rank when there's no null

• space?

• By no null space, you know what I mean.

• Only the zero vector in the null space.

• So if I have a -- if I have an 11 by 5 matrix --

• so it's got 11 rows, 5 columns, when is there no null space?

• So the columns should be independent -- what's the rank?

• n 5 -- rank n.

• Independent columns, when -- so if I --

• then I conclude yes, positive definite.

• And this was the assumption -- then A transpose A is

• invertible --

• the least squares equations all work fine.

• And more than that -- the matrix is even positive definite.

• And I just to say one comment about numerical things,

• with a positive definite matrix, you never

• have to do row exchanges.

• You never run into unsuitably small numbers or zeroes

• in the pivot position.

• They're the right -- they're the great matrixes to compute with,