Placeholder Image

Subtitles section Play video

  • (bell dings)

  • - Hello, and welcome to another Coding Train video

  • about neuroevolution.

  • So this video, what I'm going to do is,

  • I'm going to take my previous, finished,

  • sort of finished, version of neuroevolution,

  • the process of evolving the optimal weights

  • of a neural network to solve some kind of,

  • perform some kind of machine learning task.

  • In this case, I'm making a guess

  • based on the environment

  • of this very crude Flappy Bird game.

  • I'm not making this,

  • the neural network is making a guess

  • whether it should jump or not.

  • So this is a very simple, actually, scenario to solve.

  • And so it's a nice demonstration of the idea.

  • I've been meaning to return to this topic for quite a while

  • so I can try to look at some more complex scenarios.

  • So neuroevolution can be applied to a lot of problems,

  • at different,

  • reinforcement learning algorithms are also applied too,

  • but it's a pretty different technique.

  • There's like five, or six, or seven videos

  • all about neuroevolution

  • that I would recommend you check out,

  • or you can start right here.

  • Because what I'm going to do in this video is,

  • I am going to take the existing version that I made,

  • which used a toy neural network JavaScript library

  • that I implemented in yet another video series

  • to, kind of, learn more about neural networks

  • and spin up a very, very basic one

  • in JavaScript without other dependencies.

  • But I'm going to replace that now with TensorFlow.js.

  • What you're seeing here

  • is the last example from Part Five,

  • where I saved a trained model

  • and I'm loading it into a version of the game

  • and watching it play out.

  • What I did previous to that,

  • is what you're seeing right here,

  • which is, this is launching 500 random neural networks

  • to try to make guesses.

  • I can use this slider to sort of speed up the system,

  • and over time those neural networks

  • are going to perform the processes

  • of crossover and mutation.

  • It's not actually doing crossover,

  • it's doing crossover which would be the act of combining

  • two agents' genetic material together into one.

  • I'm just taking one and copying it,

  • but then I am mutating it

  • to optimally search for the configuration of weights

  • that will work the best.

  • And just to remind you, when I'm talking about the weights,

  • this is what I'm talking about.

  • So we have the Flappy Bird game,

  • this is the agent that needs to make a decision.

  • In a way it's a classification problem.

  • Should I go up or should I not go up, right?

  • These are the only two possibilities,

  • a very simple classification problem.

  • Two categories.

  • I have done, I'm the human being,

  • have basically done the feature extraction.

  • I could use the environment, the input to the neural network

  • as this image, like all the pixels of the game,

  • that would be a wonderful thing to try

  • and I would love to do that,

  • especially once I have convolutional layers

  • with TensorFlow.js,

  • which is not something I have

  • in my toy neural network library.

  • You might not know what a convolutional layer is.

  • Don't worry, if I use it, I will explain it.

  • But, I have done that feature extraction.

  • So I have decided that the features

  • that I want to use as inputs into my neural network

  • are, I think it was like, the bird y position,

  • the bird, y velocity,

  • the top pipe location, the bottom pipe location.

  • And then, I'll call this x,

  • the distance to the nearest pipe.

  • So these are what I've decided

  • might be all the important values to use from this game,

  • to feed into the neural network.

  • So that means the neural network

  • has one, two, three, four, five inputs.

  • These all get normalized

  • into a value with a range between zero and one,

  • and fed into the neural network.

  • Then the outputs is a classification problem.

  • So there are just two outputs,

  • and they would each output a number.

  • If I got something like 0.8 here and 0.2 here,

  • that means there is an 80%,

  • basically, an 80% probability confidence score

  • that I should jump, so I will jump.

  • Now I could actually pick random numbers

  • and that kind of thing,

  • but I'm just going to take the highest one,

  • the arg max, so to speak, and go.

  • So, neural networks are able to learn

  • a sophisticated amount of information through hidden layers.

  • So the inputs don't pass directly to the outputs,

  • but the inputs pass through a hidden layer.

  • And I don't recall what I picked

  • as the configuration of the hidden layer,

  • but I think it was something like eight.

  • So if there were eight, I think that's what it is.

  • We'll look.

  • One, two, three, four, five, six, seven, eight,

  • these are what are known as dense layers.

  • Meaning, every single node is connected to every node.

  • I will not sit here and draw all of them,

  • but I will start that process.

  • And then all of these, sorry, are connected to the outputs,

  • like this.

  • So a neural network's core data,

  • the configuration of that data is in a matrix.

  • 'Cause every one of these connections has a weight.

  • So if there are five here and eight here,

  • that is 40 weights.

  • If there are eight here and two here, that's 16 weights.

  • Now the truth of the matter is, there's also a bias.

  • So there's a bias for each one of these

  • and a bias for each one of these.

  • So the configuration is all of the weights,

  • all of the biases for each of these nodes.

  • But these are the details that I cover a lot more

  • in the series about building a neural network

  • from scratch in JavaScript.

  • So if you want to learn about that, you can go back.

  • But all you need to know for here,

  • is that I have a way of creating this neural network.

  • I have a way of feeding these values into it

  • and looking at the stuff that comes out.

  • So let's go look at that code.

  • So if I go look at the code, we can see that here,

  • there's this idea of a bird object.

  • And the bird object makes a neural network

  • with five inputs, eight hidden nodes, and two outputs.

  • I don't know what I've said here, but this is known

  • as a feedforward neural network.

  • Multilayer Perceptron, it's two layers.

  • The inputs, it looks like a layer,

  • but these are just numbers coming in.

  • There is a hidden layer in the output layer.

  • That's important terminology.

  • So that's happening here, and you can see in the bird,

  • the bird sort of like, takes all of those properties,

  • its y position, the closest pipe,

  • the closest pipe's x position, its velocity,

  • makes that into an array,

  • feeds that into a predict function,

  • and then I just sort of like,

  • if the first output is greater then the second output, jump.

  • So this is that process.

  • And all of this happens

  • in this neural network library.

  • So there is neuralnetwork.js and matrix.js.

  • This is like if TensorFlow.js was made by one person

  • who really didn't know what they were doing (laughs).

  • That's what's in here.

  • So what I want to do, is I am going to delete this folder.

  • Right, I'm deleting it, it's gone.

  • Gone, move to trash, boom!

  • So now when I go over here and I hit refresh,

  • of course, it doesn't work.

  • Because we have no neuralnetwork.js.

  • Can I get this, I don't have a watch on,

  • in however long I have right now,

  • in the course of this video,

  • working again without my library,

  • but with TensorFlow.js instead.

  • That's the thought experiment here.

  • So here is my html file,

  • and you can see that it has the P5 libraries,

  • it's loading my neural network library,

  • and then these are all the files

  • for the Flappy Bird game itself,

  • as well as the file that includes some information

  • about how the genetic algorithm works.

  • So what I want to do, I need to take these out,

  • and then I want to import TensorFlow.js.

  • So I have the TensorFlow.js website up

  • and this is the script tag for importing TensorFlow.js.

  • So I'm going to add that here, and actually I am pretty sure

  • that the most current version is 1.0.4, so let's add that.

  • And now I'm going to go back here and hit refresh.

  • Now, of course, ah, NeuralNetwork is not defined,

  • so we have fewer error messages here now, which is good.

  • I just want to make sure tf is loaded,

  • so, for example, I can call tf memory,

  • and there is no memory being used,

  • but I can see that tf js is loaded,

  • 'cause I can call tf functions in the console now,

  • which I'm going to need to do to figure all this out.

  • All right, so now the thing I think I want to do is that,

  • actually, I don't need to do this,

  • but I think it would be nice for me to create,

  • I'm going to create a file called nn.js,

  • and I'm going to make a little wrapper for a neural network.

  • And actually one of the reasons why I want to do this is,

  • as you know, I've made a lot of videos

  • using a library called ML5,

  • which is a wrap around TensorFlow.js,

  • which allows you to work

  • with some of the machine learning algorithms

  • without having to manipulate

  • the lower level details of TensorFlow.js,

  • and so this is, kind of, a little bit,

  • of maybe a preview of that.

  • Because this ultimately,

  • this idea of neuroevolution is something

  • that I would like to work on putting into the ML5 library.

  • So if I make a neural network class,

  • and I write a constructor,

  • we know here, that in the bird object

  • what it's doing is, it's making a neural network

  • with five inputs, eight hidden nodes, and two outputs.

  • So there's no reason why I can't actually

  • just keep that same structure,

  • and I'm going to have three arguments,

  • I'm just going to call them a, b, and c,

  • because at some point I might need

  • to call the constructor in different ways,

  • and I'm going to then say,

  • this.input_nodes equals a.

  • this.hidden_nodes equals b,

  • and this.output_nodes equals c.

  • Then I'm going to put, I already,

  • by the way, I did this in a coding class

  • that I'm teaching, last week,

  • so I kind of have quite a bit of the sense of the plan here.

  • So I'm also going to write a function

  • called this.createModel

  • 'cause I might need to do that in different places

  • and then I'm going to make a separate function

  • called createModel.

  • So this now is the function,

  • where I want to create my neural network,

  • using TensorFlow.js.

  • And I'm going to use the layers API.

  • The truth of the matter is,

  • this particular architecture, right,

  • this is perhaps one of the simplest, the most basic,

  • vanilla, so to speak, neural network architectures.

  • It's got two layers, one hidden layer, very few nodes,

  • there's no convolutions, no fancy stuff,

  • just the very basic.

  • So I could probably do this

  • with just simple TensorFlow operations itselves,

  • but I'm going to use something called tf.layers

  • which is actually based on Keras,

  • which is a Python library for TensorFlow,

  • and Python, so many things,

  • I've talked about these in videos.

  • But basically tf.layers is an API

  • that allows me to create a model

  • from a slightly higher level perspective.