- Hello, and welcome to another Coding Train video

about neuroevolution.

So this video, what I'm going to do is,

I'm going to take my previous, finished,

sort of finished, version of neuroevolution,

the process of evolving the optimal weights

of a neural network to solve some kind of,

perform some kind of machine learning task.

In this case, I'm making a guess

based on the environment

of this very crude Flappy Bird game.

I'm not making this,

the neural network is making a guess

whether it should jump or not.

So this is a very simple, actually, scenario to solve.

And so it's a nice demonstration of the idea.

I've been meaning to return to this topic for quite a while

so I can try to look at some more complex scenarios.

So neuroevolution can be applied to a lot of problems,

at different,

reinforcement learning algorithms are also applied too,

but it's a pretty different technique.

There's like five, or six, or seven videos

all about neuroevolution

that I would recommend you check out,

or you can start right here.

Because what I'm going to do in this video is,

I am going to take the existing version that I made,

which used a toy neural network JavaScript library

that I implemented in yet another video series

to, kind of, learn more about neural networks

and spin up a very, very basic one

in JavaScript without other dependencies.

But I'm going to replace that now with TensorFlow.js.

What you're seeing here

is the last example from Part Five,

where I saved a trained model

and I'm loading it into a version of the game

and watching it play out.

What I did previous to that,

is what you're seeing right here,

which is, this is launching 500 random neural networks

to try to make guesses.

I can use this slider to sort of speed up the system,

and over time those neural networks

are going to perform the processes

of crossover and mutation.

It's not actually doing crossover,

it's doing crossover which would be the act of combining

two agents' genetic material together into one.

I'm just taking one and copying it,

but then I am mutating it

to optimally search for the configuration of weights

that will work the best.

And just to remind you, when I'm talking about the weights,

this is what I'm talking about.

So we have the Flappy Bird game,

this is the agent that needs to make a decision.

In a way it's a classification problem.

Should I go up or should I not go up, right?

These are the only two possibilities,

a very simple classification problem.

Two categories.

I have done, I'm the human being,

have basically done the feature extraction.

I could use the environment, the input to the neural network

as this image, like all the pixels of the game,

that would be a wonderful thing to try

and I would love to do that,

especially once I have convolutional layers

with TensorFlow.js,

which is not something I have

in my toy neural network library.

You might not know what a convolutional layer is.

Don't worry, if I use it, I will explain it.

But, I have done that feature extraction.

So I have decided that the features

that I want to use as inputs into my neural network

are, I think it was like, the bird y position,

the bird, y velocity,

the top pipe location, the bottom pipe location.

And then, I'll call this x,

the distance to the nearest pipe.

So these are what I've decided

might be all the important values to use from this game,

to feed into the neural network.

So that means the neural network

has one, two, three, four, five inputs.

These all get normalized

into a value with a range between zero and one,

and fed into the neural network.

Then the outputs is a classification problem.

So there are just two outputs,

and they would each output a number.

If I got something like 0.8 here and 0.2 here,

that means there is an 80%,

basically, an 80% probability confidence score

that I should jump, so I will jump.

Now I could actually pick random numbers

and that kind of thing,

but I'm just going to take the highest one,

the arg max, so to speak, and go.

So, neural networks are able to learn

a sophisticated amount of information through hidden layers.

So the inputs don't pass directly to the outputs,

but the inputs pass through a hidden layer.

And I don't recall what I picked

as the configuration of the hidden layer,

but I think it was something like eight.

So if there were eight, I think that's what it is.

We'll look.

One, two, three, four, five, six, seven, eight,

these are what are known as dense layers.

Meaning, every single node is connected to every node.

I will not sit here and draw all of them,

but I will start that process.

And then all of these, sorry, are connected to the outputs,

like this.

So a neural network's core data,

the configuration of that data is in a matrix.

'Cause every one of these connections has a weight.

So if there are five here and eight here,

that is 40 weights.

If there are eight here and two here, that's 16 weights.

Now the truth of the matter is, there's also a bias.

So there's a bias for each one of these

and a bias for each one of these.

So the configuration is all of the weights,

all of the biases for each of these nodes.

But these are the details that I cover a lot more

in the series about building a neural network

from scratch in JavaScript.

So if you want to learn about that, you can go back.

But all you need to know for here,

is that I have a way of creating this neural network.

I have a way of feeding these values into it

and looking at the stuff that comes out.

So let's go look at that code.

So if I go look at the code, we can see that here,

there's this idea of a bird object.

And the bird object makes a neural network

with five inputs, eight hidden nodes, and two outputs.

I don't know what I've said here, but this is known

as a feedforward neural network.

Multilayer Perceptron, it's two layers.

The inputs, it looks like a layer,

but these are just numbers coming in.

There is a hidden layer in the output layer.

That's important terminology.

So that's happening here, and you can see in the bird,

the bird sort of like, takes all of those properties,

its y position, the closest pipe,

the closest pipe's x position, its velocity,

makes that into an array,

feeds that into a predict function,

and then I just sort of like,

if the first output is greater then the second output, jump.

So this is that process.

And all of this happens

in this neural network library.

So there is neuralnetwork.js and matrix.js.

This is like if TensorFlow.js was made by one person

who really didn't know what they were doing (laughs).

That's what's in here.

So what I want to do, is I am going to delete this folder.

Right, I'm deleting it, it's gone.

Gone, move to trash, boom!

So now when I go over here and I hit refresh,

of course, it doesn't work.

Because we have no neuralnetwork.js.

Can I get this, I don't have a watch on,

in however long I have right now,

in the course of this video,