Name: Neural Networks and Deep Learning: Crash Course AI #3
Uploaded: 2020-03-30T03:08:21.000Z
Duration: 12 min 23 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

Hi, I'm Jabril, and welcome to CrashCourse AI!

In the supervised learning episode, we taught John Green-bot to learn using a perceptron,

But our brains make decisions with 100 billion neurons, which have trillions of connections

We can actually do a lot more with AI if we connect a bunch of perceptrons together, to

create what's called an artificial neural network.

Neural networks are better than other methods for certain tasks like, image recognition.

The secret to their success is their hidden layers, and they're mathematically very

Both of these reasons are why neural networks are one of the most dominant machine learning

Not that long ago, a big challenge in AI was real-world image recognition, like recognizing

a dog from a cat, and a car from a plane from a boat.

Even though we do it every day, it's really hard for computers.

That's because computers are good at literal comparisons, like matching 0s and 1s, one

It's easy for a computer to tell that these images are the same by matching the pixels.

But before AI, a computer couldn't tell that these images are of the same dog, and

had no hope of telling that all of these different images are dogs.

So, a professor named Fei-Fei Li and a group of other machine learning and computer vision

researchers wanted to help the research community develop AI that could recognize images.

The first step was to create a huge public dataset of labeled real-world photos.

That way, computer scientists around the world could come up with and test different algorithms.

It has 3.2 million labeled images, sorted into 5,247 nested categories of nouns.

Like for example, the “dog” label is nested under “domestic animal,” which is nested

Humans are the best at reliably labeling data.

But if one person did all this labeling, taking 10 seconds per label, without any sleep or

snack breaks, it would take them over a year!

So ImageNet used crowd-sourcing and leveraged the power of the Internet to cheaply spread

Once the data was in place, the researchers started an annual competition in 2010 to get

people to contribute their best solutions to image recognition.

Enter Alex Krizhevsky, who was a graduate student at the University of Toronto.

In 2012, he decided to apply a neural network to ImageNet, even though similar solutions

His neural network, called AlexNet, had a couple of innovations that set it apart.

He used a lot of hidden layers, which we'll get to in a minute.

He also used faster computation hardware to handle all the math that neural networks do.

AlexNet outperformed the next best approaches by over 10%.

It only got 3 out of every 20 images wrong.

In grade terms, it was getting a solid B while other techniques were scraping by with a low

Since 2012, neural network solutions have taken over the annual competition, and the

Plus, AlexNet sparked an explosion of research into neural networks, which we started to

apply to lots of things beyond image recognition.

To understand how neural networks can be used for these classification problems, we have

All neural networks are made up of an input layer, an output layer, and any number of

There are many different arrangements but we'll use the classic multi-layer perceptron

The input layer is where the neural network receives data represented as numbers.

Each input neuron represents a single feature, which is some characteristic of the data.

Features are straightforward if you're talking about something that's already a number,

But, really, just about anything can be converted to a number.

Sounds can be represented as the amplitudes of the sound wave.

So each feature would have a number that represents the amplitude at a moment in time.

Words in a paragraph can be represented by how many times each word appears.

So each feature would have the frequency of one word.

Or, if we're trying to label an image of a dog, each feature would represent information

So for a grayscale image, each feature would have a number representing how bright a pixel

But for a color image, we can represent each pixel with three numbers: the amount of red,

green, and blue, which can be combined to make any color on your computer screen.

Once the features have data, each one sends its number to every neuron in the next layer,

Then, each hidden layer neuron mathematically combines all the numbers it gets.

The goal is to measure whether the input data has certain components.

For an image recognition problem, these components may be a certain color in the center, a curve

near the top, or even whether the image contains eyes, ears, or fur.

Instead of answering yes or no, like the simple Perceptron from the previous episode, each

neuron in the hidden layer does some slightly more complicated math and outputs a number.

And then, each neuron sends its number to every neuron in the next layer, which could

be another hidden layer or the output layer.

The output layer is where the final hidden layer outputs are mathematically combined

So, let's say we're just trying to label an image as a dog.

We might have a single output neuron representing a single answer - that the image is of a dog

But if there are many answers, like for example if we're labeling a bunch of images, we'll

Each output neuron will correspond to the probability for each label -- like for example,

And then we can pick the answer with the highest probability.

The key to neural networks -- and really all of AI -- is math.

A neural network kind of seems like a black box that does math and spits out an answer.

I mean, those middle layers are even called hidden layers!

But we can understand the gist of what's happening by working through an example.

Let's give John Green-bot a program with a neural network that's been trained to

When we show him this photo first, every feature will contain a number between 0 and 1 corresponding

Subtitles ListPlay Video

Neural Networks and Deep Learning: Crash Course AI #3

specific

process

recognize

imitate