Name: Training Neural Networks: Crash Course AI #4
Uploaded: 2020-04-13T17:16:20.000Z
Duration: 12 min 29 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

Hey, I'm Jabril and welcome to Crash Course AI!

One way to make an artificial brain is by creating a neural network, which can have

millions of neurons and billions (or trillions) of connections between them.

Nowadays, some neural networks are fast and big enough to do some tasks even better than

humans can, like for example playing chess or predicting the weather!

But as we've talked about in Crash Course AI, neural networks don't just work on their

They need to learn to solve problems by making mistakes.

using an algorithm called backpropagation to make sure all the neurons that contributed

to an error get their math adjusted, and we'll unpack this a bit later.

And neural networks have two main parts: the architecture and the weights.

The architecture includes neurons and their connections.

And the weights are numbers that fine-tune how the neurons do their math to get an output.

So if a neural network makes a mistake, this often means that the weights aren't adjusted

correctly and we need to update them so they make better predictions next time.

The task of finding the best weights for a neural network architecture is called optimization.

And the best way to understand some basic principles of optimization is with an example

Say that I manage a swimming pool, and I want to predict how many people will come next

week, so that I can schedule enough lifeguards.

A simple way to do this is by graphing some data points, like the number of swimmers and

the temperature in fahrenheit for every day over the past few weeks.

Then, we can look for a pattern in that graph to make predictions.

A way computers do this is with an optimization strategy called linear regression.

We start by drawing a random straight line on the graph, which kind of fits the data

To optimize though, we need to know how incorrect this guess is.

So we calculate the distance between the line and each of the data points, add it all up,

We're quantifying how big of a mistake we made.

The goal of linear regression is to adjust the line to make the error as small as possible.

We want the line to fit the training data as much as it can.

The result is called the line of best fit.

We can use this straight line to predict how many swimmers will show up for any temperature,

For example, super cold days have a negative number, while dangerously hot days have way

To get more accurate results, we might want to consider more than two features, like for

example adding the humidity which would turn our 2d graph into 3d.

And our line of best fit would be more like a plane of best fit.

But if we added a fourth feature, like whether it's raining or not, suddenly we can't

So as we consider more features, we add more dimensions to the graph, the optimization

problem gets trickier, and fitting the training data is tougher.

This is where neural networks come in handy.

Basically, by connecting together many simple neurons with weights, a neural network can

learn to solve complicated problems, where the line of best fit becomes a weird multi-dimensional

Let's give John Green-bot an untrained neural network.

To stick with the same example, the input layer of this neural network takes features

like temperature, humidity, rain, and so on.

And the output layer predicts the number of swimmers that will come to the pool.

We're not going to worry about designing the architecture of John Green-bot's neural

He'll start, as always, by setting the weights to random numbers, like the random line on

Only this time, it's not just one random line.

Because we have lots of inputs, it's lots of lines that are combined to make one big,

Overall, this neural network's function resembles some weird multi-dimensional shape

To train this neural network, we'll start by giving John Green-bot a bunch of measurements

from the past 10 days at the swimming pool, because these are the days where we also

We'll start with one day, where it was 80 degrees Fahrenheit, 65% humidity, and not

The neurons will do their thing by multiplying those features by the weights, adding the

results together, and passing information to the hidden layers until the output neuron

John Green-bot: 145 people were at the pool!

Just like before, there is a difference between the neural network's output and the actual

swimming pool attendance -- which was recorded as 100 people.

Because we just have one output neuron, that difference of 45 people is the error.

In some neural networks though, the output layer may have a lot of neurons.

So the difference between the predicted answer and the correct answer is more than just one

In these cases, the error is represented by what's known as a loss function.

Moving forward, we need to adjust the neural network's weights so that the next time

we give John Green-bot similar inputs, his math and final output will be more accurate.

Basically, we need John Green-bot to learn from his mistakes, a lot like when we pushed

a button to supervise his learning when he had the perceptron program.

But this is trickier because of how complicated neural networks are.

To help neural networks learn, scientists and mathematicians came up with an algorithm

called backpropagation of the error, or just backpropagation.

The basic goal is to look at the loss function and then assign blame to neurons back in the

Some neurons' calculations may have been more to blame for the error than others, so

This information is fed backwards, which is where the idea of backpropagation comes from.

So for example, the error from our output neuron would go back a layer and adjust the

weights that get applied to our hidden layer neuron outputs.

And the error from our hidden layer neurons would go back a layer and adjust the weights

Remember: our goal is to find the best combination of weights to get the lowest error.

Subtitles ListPlay Video

Training Neural Networks: Crash Course AI #4

bunch

strategy

basically

tricky