Name: StatQuest: Principal Component Analysis (PCA), Step-by-Step
Uploaded: 2022-07-28T06:28:37.000Z
Duration: 21 min 58 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

StatQuest breaks it down into bite-sized pieces hooray

Hello, I'm Josh stormer and welcome to stat quest in this StatQuest

We're going to go through principle component analysis PCA one step at a time using singular value decomposition

How it does it and how to use it to get deeper insight into your data

We've measured the transcription of two genes gene 1 and gene 2 in 6 different mice

think of the mice as individual samples and

The genes as variables that we measure for each sample

For example the samples could be students in high school and the variables could be test scores in math and reading

Or the samples could be businesses, and the variables could be market capitalization and the number of employees

Ok now we're back to mice and genes because I'm a geneticist and I work in a genetics department if

We only measure one gene. We can plot the data on a number line

mice 1 2 & 3 have relatively high values and

Even though it's a simple graph it shows us that mice 1 2 & 3 are more similar to each other than they are to

If we measured two genes then we can plot the data on a two-dimensional XY graph

Gene 1 is the x-axis and spans one of the two dimensions in this graph

Gene - is the y-axis and spans the other dimension

we can see that mice 1 2 & 3 cluster on the right side and

mice 4 5 & 6 cluster on the lower left-hand side

if we measured three genes we would add another axis to the graph and make it look 3d ie

The smaller dots have larger values for gene three and are further away

The larger dots have smaller values for gene three and are closer

If we measured for jeans however we can no longer plot the data

So we're going to talk about how PCA can take four or more Jean measurements and thus

four or more dimensions of data and make a two dimensional PCA plot

This plot will show us that similar mice cluster together

We'll also talk about how PCA can tell us which gene or variable is the most valuable for clustering the data?

For example PCA might tell us that gene 3 is responsible for separating samples along the x axis

Lastly, we'll talk about how PCA can tell us how accurate the 2d graph is?

To understand what PCA does and how it works let's go back to the dataset that only had two genes

Then we'll calculate the average measurement for gene 1 and

With the average values we can calculate the center of the data

From this point on we'll focus on what happens in the graph we no longer need the original data

Now we'll shift the data so that the center is on top of the origin in the graph

Note shifting the data did not change how the data points are positioned relative to each other

Now that the data are centered on the origin

We can try to fit a line to it to do this

We start by drawing a random line that goes through the origin

Then we rotate the line until it fits the data as well as it can given that it has to go through the origin

But I'm getting ahead of myself first we need to talk about how PCA decides if a fit is good or not

So let's go back to the original random line that goes through the origin

To quantify how good this line fits the data PCA projects the data onto it

And then it can either measure the distances from the data to the line and try to find the line that minimizes those distances

Or it. Can try to find the line that maximizes the distances from the projected points to the origin

If those options don't seem equivalent to you

We can build intuition by looking at how these distances shrink when the line fits better

While these distances get larger when the line fits better

Now to understand what is going on in a mathematical way, let's just consider one data point

This point is fixed and so is its distance from the origin in

Other words the distance from the point to the origin

Doesn't change when the red dotted line rotates

We get a right angle between the black dotted line and the red dotted line

that means that if we label the sides like this a

Then we can use the Pythagorean theorem to show how B and C are inversely related

Since a and thus a squared doesn't change

Likewise if C gets bigger, then B must get smaller

Thus PCA can either minimize the distance to the line or

Maximize the distance from the projected point to the origin

The reason I'm making such a fuss about this is that

Intuitively, it makes sense to minimize B. And the distance from the point to the line

But it's actually easier to calculate C the distance from the projected point to the origin

Maximizing the sum of the squared distances from the projected points to the origin

Then measures the distance from this point to the origin

Note I'm going to keep track of the distance as we measure up here and

Then PCA measures the distance from this point to the origin. We'll call that D 2

Here are all six distances that we measured

The next thing we do is Square all of them

The distances are squared so that negative values don't cancel out positive values

Then we sum up all these squared distances and that equals the sum of the squared distances

For short. We'll call this SS distances or sum of squared distances

Then sum up the squared distances from the projected points to the origin

And we repeat until we end up with the line with the largest sum of square

distances between the projected points and the origin

Subtitles ListPlay Video

StatQuest: Principal Component Analysis (PCA), Step-by-Step

ultimately

figure

access

slightly