Name: Flat and Hierarchical Clustering | The Dendrogram Explained
Uploaded: 2020-03-09T01:00:15.000Z
Duration: 8 min 27 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

We will have a short lecture about… clustering of clustering!

Adverbs of manner

Originally, cluster analysis was developed by anthropologists aiming to explain the origin

Later it was adopted by psychology, intelligence and other areas.

Nowadays, there are 2 broad types of clustering: Flat and hierarchical.

K-means is a flat method in the sense that there is no hierarchy, but rather we choose

the number of clusters, and the magic happens.

And that’s what we are going to discuss in this lecture.

Historically, hierarchical clustering was developed first, so it makes sense to get

An example of clustering with hierarchy is taxonomy of the animal kingdom.

For instance, there is the general term: animal.

Sub-clusters are fish, mammals, and birds, for instance.

There are birds which can fly, and those that can’t.

We can continue in this way, until we reach dogs and cats.

Even then we can divide dogs and cats into different breeds.

There are two types of hierarchical clustering: agglomerative or ‘bottom up’ and divisive

With divisive clustering we start from a situation where all observations are in the same cluster.

Then we split this big cluster into 2 smaller ones.

Then we continue with 3, 4, 5, and so on, until each observation is its separate cluster.

However, in order to find the best split, we must explore all possibilities at each

Therefore, faster methods have been developed, such as k-means.

With k-means, we can simulate this divisive technique.

When it comes to agglomerative clustering, the approach is bottom up.

We start from different dog and cat breeds, cluster them into dogs and cats respectively,

and then we continue pairing up species, until we reach the animal cluster.

Agglomerative and divisive clustering should reach similar results, but agglomerative is

This is also the other clustering method we will explore – agglomerative hierarchical

In order to perform agglomerative hierarchical clustering, we start with each case being

Second, using some similarity measure like Euclidean distance, we group the two closest

clusters together, reaching an ‘n minus 1’ cluster solution.

Then we repeat this procedure, until all observations are in a single cluster.

The end result looks like this animal kingdom representation.

The name for this type of graph is: a ‘dendrogram’.

Then the two closest clusters are combined, then another two, and so on, until we are

Note that all cluster solutions are nested inside the dendrogram.

Let’s explore a dendrogram and see how it works.

Here is the dendrogram created on our ‘Country cluster.

You can see the names of the countries at the beginning of those lines.

This is to show that, at the start, each country is a separate cluster.

The first two lines that merge are those of Germany and France.

According to the dendrogram, these two countries are the closest in terms of the features considered.

At this point, there are 5 clusters: Germany and France are 1, and each other country is

From this point on, going up, Germany and France will be considered one cluster.

The next two lines that merge are those of the Germany and France cluster, and the UK.

At this point there are 4 clusters: Germany, France and the UK are 1, and the rest are

At the next stage of the hierarchy, Canada and the US join forces.

The next step is to unite the Germany, France, UK cluster with the Canada-US one.

Finally, all countries become one big cluster, representing the whole sample.

What other information can we get from the dendrogram?

Well, the bigger the distance between two links, the bigger the difference in terms

As you can see, Germany, France and the UK merged into 1 cluster very quickly.

This shows us that they are very similar in terms of ‘longitude’ and ‘latitude’.

Moreover, Germany and France are closer than Germany and UK, or France and UK.

The USA and Canada came together not long after.

However, it took half of the dendrogram to join these 5 countries together.

This indicates the Europe cluster and the North America cluster are not so alike.

Finally, the distance needed for Australia to join the other 5 countries was the other

half of the dendrogram, meaning it is extremely different from them.

To sum up, the distance between the links shows similarity, or better: dissimilarity

Next on our list is the choice of number of clusters.

If we draw a straight line, piercing these two links, we will be left with two clusters,

Australia in one, and all the rest in the other.

Instead, if we pierce them here, we will get three clusters: North America, Europe, and

The general rule is: when you draw a straight line, you should count the number of links

In this case, we have broken 3 links, so we will be left with 3 clusters, because the

links were coming out of those 3 clusters.

Should we break the links here, there will be 4 clusters, and so on.

Finally, how should we decide where to draw the line?

Well, there is no specific rule, but after solving several problems, you kind of develop

When the distance between two stages is too big, it is probably a good idea to stop there.