Name: 17. Learning: Boosting
Uploaded: 2016-02-09T10:12:32.000Z
Duration: 51 min 40 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

PATRICK WINSTON: We've now almost completed our journey.

Future simple

the venerable kind, that's the nearest neighbors and

Still useful, still the right thing to do if there's no

All kinds of problems with local maxima and overfitting

and oscillation, if you get the rate constant too big.

Like neural nets, both are very naive in their attempt to

So maybe they work on a class of problems.

They surely do each have a class of problems for which

But as a general purpose first resort, I don't recommend it.

But now the theorists have come out and done some things

And in the end, you have to say, wow, these are such

I wonder if nature has discovered them, too?

Or given the nature of evolution, is it just random

junk that is the best ways for doing anything?

But today, we're going to talk about an idea that I'll bet is

in there somewhere, because it's easy to implement, and

it's extremely powerful in what it does, and it's the

essential item in anybody's repertoire of learning

It's also a mechanism which, if you understand only by

formula, you will never be able to work the problems on

Because on the surface, it looks like it'd be very

But once you understand how it works and look at a little bit

of the math and let it sing songs to you, it turns out to

So it's about letting multiple methods work in your behalf.

So far, we've been talking about using just one method to

And what we're going to do now is we're looking to see if a

crowd can be smarter than the individuals in the crowd.

But before we get too far down that abstract path, let me

just say that the whole works has to do with classification,

Am I holding a piece of chalk in my hand, or a hand grenade?

Those are binary classification problems.

And so we're going to be talking today strictly about

We're not going to be talking about finding the right letter

in the alphabet that's written on the page.

So we assume that there's a set of classifiers

And it produces either a minus 1 or a plus 1.

So that's how the classification is done.

Now, too bad for us, normally the world doesn't give us very

So if we look at the error rate of this classifier or any

other classifier, that error rate will range from 0 to 1 in

So you'd like your error rate to be way down here.

Just a little bit better than flipping a coin.

If it's just a little bit better than flipping a coin,

And the question is, can you make a classifier that's way

over here, like there, a strong classifier, by

combining several of these weak classifiers, and

You might say, well, let us make a big classifier capital

H, that works on some sample x, and has its output produces

something that depends on the sum of the outputs of the

Let's say three of them, just to start us off.

And now let's add those guys up, and take

So if two out of the three of those guys agree, then we'll

Subtitles ListPlay Video

17. Learning: Boosting

guy

decision

rate

wrong