Placeholder Image

Subtitles section Play video

  • what's going on?

  • Everybody and welcome to yet another deep learning with python tensorflow and Kare aas tutorial.

  • In this tutorial, we're gonna continue with last tutorial where we're attempting to predict future price movements of a certain crypto currency based on the sequence in the historical prices and volume of that Cryptocurrency as well as other major crypto currencies.

  • And we're trying to do this with the recurrent neural network.

  • So, uh, let's continue.

  • The next thing that we want to do here is we're gonna import times.

  • We're gonna use that just in a moment and we need Thio create a few more Constance here.

  • The first thing we're gonna do is iPAQ So how many blocks we want to train this model for?

  • Then we're gonna go with a batch size and we're gonna go with 64 to start weaken.

  • Tinker with that later.

  • If we wanted.

  • And then finally, we're gonna go with a name on.

  • We're gonna make this an F string, and what you want is a name that is descriptive of the model because generally you're gonna tinker a little bit with the model, tweak it a little bit here and there re run it again, and then you're gonna do the same thing again.

  • Hopefully, you don't have itchy advises.

  • I do.

  • Anyway, um, so you want to have a unique name, both for the model that you save as well as intense or bored.

  • So later you can compare the results of a bunch of different models and you don't have to be like I don't know what model that was so or worse, you overwrite the other models.

  • So anyways, uh, yeah.

  • So let's come up with a good name.

  • So I'm gonna go with sequence, length, dash sequence, dash, future period.

  • Predict dash, predict Dash.

  • And then we'll just throw in and int time that time here.

  • And that should be good enough.

  • That'll give us a nice, unique model name.

  • So now we're gonna do is import all the tents airflow stuff that we need.

  • So import tensor flow as t f as t f Stop it sublime from tensor float.

  • Kare aas dot models were gonna import in poor sequence show.

  • There we go.

  • Uh ah, You're such a It's a Quintus D'oh!

  • Oh, this This might be a hard one to get through from 10 serve tens or float.

  • Kare aas dot layers were going to import dense dropout.

  • L s t m probably also coud cou d n n l s t Do you know I had it right.

  • L s t m and then batch batch normalization.

  • Um, all of these we've seen, except for I don't remember if we've done batch normalization.

  • I apologize if I've already covered it.

  • Basically, just normalization, but between the layers.

  • So for the same reason you want to normalize your input data batch normalization can be useful from layer to layer because because really, you can think of each layer.

  • If it's going to another layer, that layer, its output is kind of like your it's it's a new input for a new layer.

  • So you would want it if you could normalize that data.

  • So we're gonna do batch normalization there.

  • Okay, Once we've got that more imports from tensor float up Keira stock callbacks.

  • We want to import 10.

  • Sore board tense.

  • Why can't I live right now?

  • Uh, anyway, tensor board and model checkpoint, I think be intense or board probably needs to be capitalized.

  • It does.

  • Okay.

  • Tense aboard.

  • We've really already seen this callback model checkpoint is a fancy dancy little call back where basically, you can set various parameters as to when you want to save certain checkpoints.

  • So I like to use validation accuracy.

  • I think it's pretty good one.

  • A validation loss would be another good one.

  • Basically, each time it reaches a new max, you could have you could have the model actually save that checkpoint That way, you know, you can run for 100 epoxy now, private the end of 100 box before you got there.

  • The model over fit, and it kind of went crappy.

  • Um, and you could save every single iPAQ, but this might wait space or whatever.

  • Um, anyway, this is makes a nice way to always save the best one.

  • Okay, Uh, once we have that, we're gonna scroll on down.

  • We've got a train.

  • External validation explanation.

  • Why?

  • Ready to build the model?

  • So make some space here, Uh, and we're just gonna There's a lot of this coded stuff we've seen before.

  • So again, model is a sequential model.

  • We've seen that one model, that ad, and we're gonna add, uh, I'm gonna use Cody and else tm If you're on CPU version in tensorflow, use a regular l S T M.

  • It's gonna be 1 28 nodes in this layer and input shape and something bangs on this desk, and it makes me mad that what is that noise I could never find What that noise is striking drives me nuts.

  • And then sometimes it's, like, stops happening for, like, months.

  • And then it comes back anyway.

  • Input shape trained ex, uh, train exact shape, one cold.

  • That was about to be an error.

  • That was not gonna be fun to debug.

  • Uh, figure out where we are now.

  • So we're inside the cootie in an L s t m Still.

  • Yes, it's yes s.

  • So then return underscore er sequences equals true.

  • So you should already know that what the next layer is gonna be.

  • Are we good?

  • Why does it do it?

  • What is it?

  • Because, uh, we don't.

  • But new line, this is maybe Pepe.

  • Yeah.

  • Okay.

  • All right.

  • So then model that ad and we will throw in a drop outs, and we're gonna d'oh 0.2.

  • Then we'll add a batch normalization.

  • So model that ad batch normalization and no parameters there.

  • And then I am going to copy this and then paste it twice paste and paste, and then we're gonna have a dense layer.

  • So because we're gonna have a dense layer, I'm gonna remove the return sequences.

  • And so now it's a 1 28 to another 1 28 and then to a final 1 28 Now, for whatever reason, in my notes, I have this as a dropout of 0.1.

  • So I'm gonna leave that there.

  • But I'm pretty confident your 0.2 would probably work.

  • Justus.

  • Good.

  • Uh, anyway, love movie back.

  • Simple one, Uh, and I think we'll leave everything else as the same.

  • Now we're gonna add eight dense layer, but not the output layer.

  • Just a dense layer model That ad, Yes, that one dense 32 activation will be rectified Linear.

  • Which reminds me, if you can't use Cudi, an analyst him make sure you throw in some activations.

  • I would do either tan H because that's what Cody N N L C M's using, or you could just use rectified linear.

  • So we've got that dense layer.

  • Cool model dot ad will throw in a dropout 0.2.

  • I really wonder if this was like a typo or what everybody else is 0.2, except for this layer.

  • Um, so anyway, um and then finally, we need the final dense layer.

  • Uh, there's this is a binary choice, so it should be only two options there activation because it's the output layer Soft max.

  • Okay, now that we've got that we're ready to specify the optimizer will go A t f got Cara stott optimizers dot Adam with a learning rate of 0.1 23 and a d k of one e negative six and then we'll do the model dot Compile, and we'll go with loss is sparse.

  • Cat, Categorical Cross and Truby, You could also go with, like, binary cross entropy.

  • Uh, next, What we're gonna do is optimizer is Theo optimizer that we defined and then metrics we will with accuracy.

  • Okay, cool.

  • The next thing is, we need to define our callbacks and I guess we can fix this make makes sublime happy s So we have to call back.

  • So we want to dio that one is tensor board which we've really already seen.

  • And so that's just gonna be a tensor board.

  • Objects laundered.

  • Eagles longs, Uh, and then we'll do some formatting here and in my text based version, I haven't made this an F strings trying to move completely to F strings.

  • They do.

  • They are better.

  • It's just I'm so used to, you know, doing the format.

  • But it's so much nicer to use F strings, to be honest, like build it out and then not have to remember what order things Aaron.

  • And then and then later.

  • The worst thing is like when you build it for the first time, it's really no big deal.

  • But then, if you want to, like, go in and move things around or like at a bunch more, it it really does kind of.

  • It's kind of challenging with string formatting, whereas with, like the F strings, just it's just a a breeze.

  • Okay, so that's intense aboard object.

  • The next thing we're ready to do is do the checkpoint objects.

  • So here a file path.

  • And to be quite honest with you, this is not my code.

  • I just I basically searched for the check like an example for the checkpoint.

  • Things so this might be from, like, Carris documentation.

  • I don't even remember where I found this, but basically, we'll do.

  • Orrin and Final obviously didn't have this exact name, but then it's iPAQ, um, and then zero to D.

  • And the reason why I'm doing this is, um I kind of wanted just, like, copy and paste this in so I can talk about it and not waste a bunch of time.

  • Um, here paste.

  • So, like, I'm not sure how how your formatting, Because obviously this is this here is a string formatting object to this file dot model, and then we got format the file path.

  • But then it's like monitor, I guess I just understand how these values are eventually getting passed into here with a dot format.

  • I don't know.

  • Anyway, what I want is a pock invalidation accuracy in this code works.

  • And again, I don't because, obviously, you know, format file path, it throws in the file path.

  • But then it's like these other things in the format.

  • Anyway, I don't get it.

  • I don't know how it works, but it works anyways s.

  • So now we're going to do a history equals modeled outfits.

  • Uh, we're gonna fit train X train.

  • Why, Uh, the batch size is just batch signs.

  • A pox is going to equal a pox Capitals.

  • Let's fix this.

  • They are parameters.

  • Therefore, no spaces.

  • After a pox, we have the validation.

  • Valid validation underscored data.

  • And then in here we have validation.

  • Ex validation.

  • Why?

  • And then finally, we have our callbacks that we want to run.

  • So that's tensor board and checkpoint.

  • Okay?

  • And then once that's done, um, I do run a value, uh, and evaluating the text based version.

  • But really, every pock should be running validation data anyways, So at the end, I guess we could just do a model, not save, although even here, Not really sure what?

  • The point is because the model checkpoint is going to be saved.

  • So I guess we're done at this point.

  • Okay?

  • Save that.

  • And now let's go ahead and run this thing.

  • Time Industry six crypto aren't in tight.

  • Let's see if we have any errors.

  • Faster, please.

  • Impatient.

  • Nothing yet.

  • Come on.

  • Okay.

  • Starts training already at 51 ish accuracy.

  • Uh, the pox are not too too slow.

  • I I think I'll probably positive just chop this out.

  • But we can also watch it as it trains with a tensor.

  • No.

  • What have you done?

  • It went to save it in that.

  • Ah, probably because we don't have a model's directory.

  • Let's just throw one in real quick models during it.

  • Well, here's a perfect example.

  • Why you want to throw in time to your name because you're gonna fix the bug.

  • You're gonna rerun it and then you're going to, like, gang it.

  • I didn't change the name, and then it's gonna overwrite it in the logs.

  • And now it doesn't matter Now there's two of them by a time stamp, although actually, why, that's Oh, no, it's not the same earth dude I can't see at all clearly.

  • Anyway, those look the same, you know, give me a break.

  • Their three versus eight anyways.

  • Okay.

  • I need some some alcohol.

  • Anyways, that would definitely help the situation.

  • Okay, So, back to our tents or bored tensor board logger equals logs.

  • So why is this Dash, Dash blogger with no underscore?

  • Uh and then this here is Oh, shoot.

  • Yeah.

  • This here has an underscore.

  • Drives me nuts because it really always confuses me every time.

  • Okay, so then we can go to a browser, go to our PC.

  • You could always go to the 1 27 001 Colon 6006 if you don't know.

  • But anyways, HPC for me is my PC.

  • His name.

  • Okay, so the blue line is is what's, um Oh, we might I might might see GPU money just crashed.

  • Hopefully this recording isn't abort.

  • What is happening?

  • Like, what is this?

  • You guys can't see it, but my entire does top just, like went black and all my all of my icons went bonkers.

  • Anyway, hopefully hopefully still recording?

  • Uh, okay.

  • Uh, we're currently five ipaq Sandy, refresh the page.

  • Hopefully, we get proper scaling here.

  • There we go.

  • Okay, So this is our model, Um, and we're actually doing pretty good.

  • Validation loss has fallen for the 1st 5 pox.

  • At least validation accuracy has gone up again.

  • We are seeing what appears to be in general, better validation accuracy than in sample accuracy.

  • Uh, which probably shouldn't be the case, because this is always going up.

  • We'll see if that remains the case.

  • But like I said before, a lot of times you will have a higher, um, validation accuracy.

  • If the models toe has a lot of learn or it's learning a lot per epoch, you might find yourself having, at least initially, a sly, rhe, slightly higher validation accuracy, because the validation accuracy is calculated at the end of the park.

  • Whereas the accuracy here is for the entire iPAQ.

  • So the initial bid of the park is kind of weighing down anyways, uh, accuracy in sample and out of sample.

  • I'm just gonna do this.

  • Um, both look pretty good on dhe then Lost looks really great here.

  • Uh, and validation, accurate or validation loss.

  • Rather looks pretty good for the first.

  • Really, like 60 pox.

  • And then it kind of looks like it's turning, turning around, and then you can even see validation.

  • Accuracy is kind of doing the same thing.

  • After about 60 pox, it starts falling.

  • But we had about, um, as high as 57.28% accuracy, which is pretty darn good.

  • So the other thing we probably should have in our our name here is the thing that we're predicting.

  • So I would also throw in ratio to predict um So now what we could do is like, let's change this to a theory.

  • Um, do ethereum usd and then come over.

  • You're way we run that.

  • Oh, my g b won't run.

  • Cry too hard.

  • I probably should have said GPU usage to, like, 75%.

  • My mistake.

  • Let's see.

  • Hold on.

  • Do we have even less data here?

  • Oh, no, no.

  • We should have more data for this.

  • This one.

  • So this is for a theory.

  • Um, by watching the other prices already have 51% is pretty good.

  • Okay, so here is a theory.

  • Um, done.

  • So Ethereum is our red line here.

  • Blue is the light coin.

  • So at the end, the validation accuracy is about the same, but for the most part before, it's like things started turning around a little bit below, but pretty, pretty similar results.

  • Validation loss Similar.

  • They kind of diverge in a lot of places, but still, it's pretty similar trends.

  • After about the sixth epoch, things just kind of go south.

  • Um, anyways, I am now running bit queen cash.

  • We'll see how that doesn't.

  • Then after that, I'll run the big Queen, and then we can kind of see how everything relates.

  • Okay, Bitcoin cash is done.

  • And actually, it's the best performing one of all of them in terms of validation, accuracy and loss on both That has the least loss in the highest accuracy, ese.

  • Pretty cool.

  • Uh, finally, we're going to run Bitcoin and see how that does.

  • I'm interested to see Bitcoin.

  • Now again, I actually talked to other results.

  • I probably should've found this out before, but, uh, pretty good results made you keep this all to myself.

  • Okay?

  • And Bitcoin is done.

  • Very interesting results on Bitcoin.

  • Initially, it was just kind of 50 50 50 just 50% the whole way.

  • Then obviously, somewhere around the fifth iPAQ it figured something else, something out started dropping in.

  • Then obviously, validation accuracy started going up.

  • Validation law started going down.

  • Um, although briefly did bike up.

  • That's actually interesting.

  • That's we better validation accuracy than I thought bit Quinn would ever get just because Bitcoin there's probably just a whole lot less Alfa involved with Bitcoin nowadays.

  • But also I think Bitcoin is more of the trendsetter.

  • But then again, maybe not.

  • I mean, I think a lot of these other crypto coins have a lot more of an established presence than just relying on the price of Bitcoin.

  • So anyways, um, pretty cool results that were able to seemingly predict forward movement to a pretty good degree.

  • I mean, I kind of draw threshold around 60% for, like, a pretty impressive classifier when it comes to predicting finance.

  • And in this case, this is predicting just simply an upward or downward movement.

  • So I think you could probably tweet this model.

  • I mean, I didn't spend too long on the model.

  • All I did was I just wanted to find a model that started to learn from here.

  • I would probably write a script to just kind of automate the rest of everything, but I'm pretty confident we could get probably all of these over 60% and that's just off of pricing and volume features.

  • So then you'd price start bringing in other features besides that, but anyways, pretty cool.

  • I will just say, though this is seemingly good.

  • As I said before, probably a few times.

  • Um, it's so easy to make mistakes on projects like this.

  • So obviously, uh, use this at your own risk I'm not responsible for anybody who actually uses this and destroys their trading accounts.

  • Uhm, I'd go so far as to say there is almost certainly a bug in here, so, uh, yeah, educational use only.

  • Please.

  • So, anyways, pretty cool results.

  • Let me know if you guys tweak it and it can get better results and all that, or if you don't want to talk about it, I guess that's fine, too.

  • Anyway, keep it all to yourself.

  • Um, pretty cool.

  • Uh, that's it for this project.

  • For now, I don't really know what I want to do next in this series.

  • I'm probably done with, like, basic scare us because I'm running out of things to do besides getting, like, super complicated like we did here.

  • Uh, the next thing I'd really like to get into a sw far as just kind of playing around with things is audio.

  • Um, either with songs or speech or something like that, I want to do audio.

  • So, uh, anyway, let me know what you guys think about doing audio with this stuff.

  • Also, if you have anything else you really want me to get into, um, let me know.

  • Down below.

  • If you have questions about anything in here, feel free to ask.

  • You can also come and join.

  • The discord is just discord dot g slash Centex.

  • Um, if you spot an error, have something you'd like to improve upon the script.

  • You can let us know in the comments section or by discord again.

  • Otherwise that is all for now.

  • I will see you guys in another video.

what's going on?

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it