Placeholder Image

Subtitles section Play video

  • Hello, and welcome to another ML5 neural network video

  • tutorial.

  • I'm following up on what I did in the previous video

  • where I built this example.

  • This example has this interaction

  • where you click the mouse all over this canvas and press keys

  • to assign each xy point a label, C, D, E--

  • and I added a bunch of notes since the previous video.

  • So I have the full scale C, D, E, F, G, A, B.

  • Then I trained the model with the inputs

  • being the xy of all these points and the target

  • being the actual label.

  • And once the model is trained, it can make guesses.

  • So in theory, I just collected this data

  • set, trained the model.

  • When I click into it, when I click over here,

  • I should hear the musical note D.

  • I should hear the musical note E, G, A.

  • And in between, it's sort of interesting to see what I get.

  • But it works as expected.

  • I ran into like a pretty significant problem

  • while working on this.

  • Because once I've collected the data set and trained the model,

  • if I had a bug in the code or something I needed to fix

  • or I wanted to try a different parameter,

  • I have to stop the sketch and run it again

  • and sit there and do this.

  • This highly manual process of clicking, clicking, clicking

  • to collect the data set.

  • So in this video, I want to look at saving the data.

  • And I also want to look at saving

  • the model, which, those are two pretty different things.

  • It might seem like the same idea.

  • I want to save the data.

  • I want to save the model.

  • Why would I do one versus the other?

  • Let's pause for a second and examine all the steps

  • of a machine learning project and where

  • we might want to save the data versus save the model and why.

  • Step 1, collect the data.

  • Now, this could be a really big, complicated step.

  • But in my scenario, in my example,

  • it's just clicking the mouse a whole bunch of times

  • and pressing keys on the keyboard.

  • Step 2, train the model.

  • Once the model has been trained, the idea

  • is to use that model in some scenario.

  • So that we can call step 3, deploy the model or prediction

  • inference.

  • So now the question is, where along the way

  • might you want to save the state of what you're doing?

  • So in the most traditional machine learning sense,

  • once you've done all of this and your model is trained,

  • you don't ever need to look back.

  • You've got a trained model.

  • You can save that model.

  • So right here in between steps 2 and 3

  • is a point where we might want to save the model if we're done

  • and our model is exactly the way we

  • want it and we're ready to just use it in a project.

  • However, you might want to try training the model

  • a variety of different ways.

  • And this is where you might want to, in between these two steps,

  • save the data.

  • We also might collect a lot of data want to take a break,

  • reload that data, collect more data.

  • There's a lot of different reasons

  • why we might want to stop in between steps 1 and 2

  • and save where we are.

  • And the functions in the ML5 neural network

  • class that we want to use are saveData and save.

  • So just save is saving the model.

  • SaveData is saving the data.

  • There are also functions for loading it back,

  • which we'll look at, loadData, and load.

  • Let's begin by just looking at saveData.

  • So in this particular example, all of the interaction

  • happens with key presses.

  • And certainly as I've mentioned before,

  • you might want to think about a more thoughtful interface

  • for doing all this work.

  • But for me, I'm just going to add another key press, s,

  • for saveData.

  • So I'm going to say elseif the key is s,

  • then I'm going to call model.saveData.

  • I can look up more about how the saveData function works

  • by looking at the ML5 website.

  • And we can see there's two optional arguments.

  • So one argument is a file name, which

  • I want to use because I want to set the file name.

  • It'll just pick a date if you don't.

  • And then a callback to know that it's done.

  • I don't actually really need to worry about that.

  • Because I'll know that it's done when the file is there

  • and downloaded to the downloads directory.

  • I'm going to give it the name mouse-notes and run the sketch

  • and collect some data.

  • So I'm just going to do a little bit just

  • to make sure it's working.

  • So now I can hit s.

  • And look, a file has been downloaded.

  • I can take a look at this file in Visual Studio Code.

  • And here's what the file looks like.

  • So I've got a data property with an array

  • that has all the data in it.

  • x, ys with a, label, x, y with a label.

  • And if I reformat the JSON, you can actually see it here,

  • and it's much more legible what's going on.

  • So this is all of the data that I've collected.

  • Not very much data, but there it is.

  • So now that I've done that, I might as well

  • take the time to collect a lot more data,

  • knowing that I can save it.

  • Methodically collected a large data set.

  • Now I'm going to press s to save it.

  • And here's what it looks like.

  • Almost 400 data samples.

  • Let's see how it performs.

  • I going to train the model.

  • Try doing some inference.

  • It works pretty well.

  • So now, the next thing that I want to try to do

  • is hit stop and run the sketch again but have

  • all of my data reappear.

  • Let's see if I can make that happen.

  • Now, instead of just creating the neural network,

  • I can create the neural network and load data into it.

  • And that's as easy as saying, model.loadData

  • mouse-notes.json.

  • The only thing here is that you have

  • to remember that I'm working in client-side JavaScript only.

  • So if I run this right now, well, it's

  • giving me this nice error here because it's

  • looking for a JSON object with an array called data.

  • But it can't find it because that JSON file doesn't exist.

  • It doesn't exist because I downloaded it

  • to the downloads directory.

  • And so I need, for my p5 sketch to be able to access it,

  • I have to manually upload it back to the p5 web editor.

  • If I were writing my own server, maybe with Node.js,

  • I could do something where I could save the data

  • and have it reload back automatically.

  • But that's another example for another time.

  • Let me do Add File and drag mouse-notes.json in here.

  • Now, we can see that that file is part of my p5 sketch.

  • And I should be able to run the sketch now.

  • All right, I think the data was loaded.

  • I don't see it, because I'm not drawing it.

  • So this might be something I want to add in a moment,

  • be able to draw the data that it's loaded.

  • But in theory, there's no reason why I couldn't train the model.

  • All right, the model is trained.

  • And you can see, I'm not seeing the data.

  • I'm not seeing those clusters.

  • But it's clearly been trained based on that data.

  • To show you how this can be useful,

  • one thing that I might want to do

  • is change some property that affects the training process.

  • So I could try it multiple times.

  • And an obvious one might be to try learning rate.

  • So let's say I make a smaller learning rate, 0.01,

  • and I run the sketch again.

  • I've got to click in here so that my key press gets

  • activated.

  • So I'm going to add one more piece of data and hit T.

  • So you can see, with a small learning rate,

  • the loss is going down very, very, very, very slowly.

  • So in this case, having a small learning rate

  • is not super helpful.

  • But I can say, OK, that learning rate wasn't good.

  • Let me try a much larger learning rate, like 0.5,

  • run it again, hit T for train.

  • And then you can can see, with this high learning rate,

  • that loss is going down really, really quickly.

  • Now, I don't mean to suggest here

  • that universally a high learning rate is better

  • than a low learning rate.

  • There's a lot of it depends here.

  • But just to show you how you can now retrain the model changing

  • all of the different kinds of options,

  • and you could look at the ML5 neural network documentation

  • and see what other kind of options

  • you might want to play around with or change.

  • You might be finding this example a little bit

  • tricky to follow because you can't actually see the data.

  • So let's add that feature of, once I've loaded the data, also

  • drawing it to the canvas.

  • So in this case, having a callback

  • for when the data is ready would be very useful.

  • I'm going to say, dataLoaded.

  • I'm going to write my dataLoaded function.

  • And let me just look at where I'm drawing stuff.

  • I'm going to grab all this drawing code,

  • bring it back up here.

  • Let me comment it out.

  • And let's actually look at what the data

  • looks like in the neural network model.

  • So I think I should be able to just console.log

  • the model's data, I think.

  • So this is what the ML5 data object looks like.

  • And for me, the important bit here is under data under raw.

  • This raw data is all of the actual xy

  • coordinate with the target label.

  • I can make a variable called data.

  • And I can iterate over it.

  • So here, I'm looking at the raw data,

  • iterating over every single element,

  • pulling out the x's into the inputs,

  • the y's into the target.

  • And then I can add the drawing code back in.

  • Only here, I'm saying, inputs.x, inputs.y.

  • And this is target.label.

  • So I believe if I run it--

  • aha, I figured out what my mistake is.

  • This is confusing, because in model--

  • is what I'm seeing here is model.data,

  • and I want to look at model.data.data.raw.

  • This leads me to think that maybe it

  • would make sense to have something like a getData

  • function.

  • Because in many cases, you just want the raw data.

  • ML5 is storing a lot of information

  • about the data set additionally to help it

  • when it loads it later.

  • But for looking at it again, it might be easier just

  • to have a function that just grabs that rather

  • than saying data.data.raw.

  • But take a look.

  • Maybe by the time you're watching this,

  • this will have been added to the ML5 library.

  • Let's see if this works.

  • There we go.

  • Ooh, so it drew all the circles, but I have the label wrong.

  • Why do I have the label wrong?

  • Oops, I Copy/Pasted mouseX, mouseY in there.

  • But I've got to get the actual xy coordinate.

  • Always making silly mistakes.

  • Let's run this again.

  • And there we go.

  • Now, it's loading that data.

  • And maybe actually what I want to do also

  • once the data is loaded is just automatically call training.

  • So I could do all this stuff right here.

  • I could run the sketch.

  • And it's immediately going to train the model.

  • And I'm ready for inference.

  • None of this is working, I might offer an exercise suggestion.

  • And I will try to remember to, in the video's description,

  • linked to code that does this in addition

  • to just the code that's here right now.

  • So remind me in the comments if I haven't.

  • But how could you take this example, which

  • now loads the data set and immediately trains the model,

  • to allow you to change the state back to data collection?

  • So I trained the model.

  • Then I want to add some more data, retrain the model,

  • resave the data.

  • How could you have a workflow that allows you to load

  • the data you previously had and add new data,