Name: Cloud vs Local GPU Hosting (what to use and when?)
Uploaded: 2020-03-31T18:23:43.000Z
Duration: 14 min 16 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

The imperative

And welcome to a video about cloud of verse, local GP use and when should you use which one or which one should you use?

I think it's pretty clear that one that you should use is most likely cloud or local.

We're both so so in which circumstances should you use which one and so on.

So I think you know, most of the hosts are gonna vary in price at least a little bit.

But in Shana roll, a good average number to look at is gonna be about $2 an hour per $4000 of GPU.

So you're gonna pay $2 an hour for $4000 of GPU that you rent.

So it's kind of regardless of which GPU it is that's gonna be in general, the ratios a point.

So whether you rent a smaller Jeep, you or even more power than a $4000 GPU, that's a close to the same ratio, right?

Is gonna very significantly as we've seen, we can see hosts literally cost twice as much for the same amount of V.

So anyways, keep that in mind, but that's the number I'm gonna be working off of.

So let's say it's $2 an hour for a $4000 Jeep.

How many hours is going to take Thio equal?

So that's going to be about 83 days, which is like two point eight months.

So once we also factor into so basically, we take 2.832 point eight months to equal $4000.

So let's just say no matter what the GPU is that you're going to rent, let's assume you were going to consider buying it.

You would have to spend 2000 hours on that Jeep.

You constantly training with that GPU to equal the cost of that GP if you were to buy it, you know locally, as opposed to hosting it.

Of course, there's other considerations, like if you buy the hardware, you could later sell the hardware, so there would be usually, I don't know.

I try to budget about a 50% depreciation per year just because one it's old, but also like one it's used, but then to usually by the time you're ready to sell it, there's new or bed or ones out.

So it also gets depreciated for that reason.

So generally, a 50% hit per year tends to be the case on high end GPU.

Some of the mid range ones holder price better but anyway s.

But the other thing you have to consider if you run locally is it's not just the cost of the GPU, right.

You could in theory, take that GPU and put it in your main machine.

And so that's what I did when I first started out.

Andi, I'm actually gonna recommend against doing that if you're actually starting out because it's not cost efficient.

The only reason I did that is there were no cloud GPU hosts when I first was getting into machine.

Learning these GPS in the cloud are actually kind of a new thing that hasn't really been around for very long, so I think if you're just starting out, you can't possibly fathom actually training models for 2000 hours, so just rent in the in the cloud.

I will throw in one more kind of caveat in that is and slash warning to watch out for these, like value added hosts.

So I see them recommended or people ask about them all the time.

I think they have a decent enough service.

Uh, it's just that I think some people mistakenly go to them when they shouldn't.

So if you're trying to learn how to do machine learning and actually program at all yourself and run it yourself, go into production yourself.

Ah, place like Floyd Hub kind of becomes a trap again.

But you'll get stuck there because you get used to that specific platform and leaving becomes a significant hardship because you've gotta learn howto actually do hosting rather than using their pretty Gu Ys and their pre built models and all this stuff.

So you you want to be careful about those Service is also the problem, and the reason why it becomes a trap is Floyd Hub or other sites like it or going to just always be a little bit more expensive, their value added.

So like comparing the cost of Floyd hub to other hosts, even some of the more expensive ones you're still gonna pay, like a 33% premium over what you would pay if you hosted it on a more traditional host.

That does not scale well, so I would recommend against making that mistake.

There's also like free tears that you can get honestly when you're just learning and you don't even want to rent a GPU or you don't want to buy a GP, or you don't have one powerful enough, just go off of your CPU.

You start needing a GPU when data sets are like million's large, and obviously you've got how many features per sample or something like that.

You need a G P u T actually produce things.

So anyway, and you also don't need a GPU for, like production like once you actually go into production with a specific model.

Chances are you're doing it on a CPU just because you don't need the GPU.

We use GPS so we can feed in batches of these samples.

Unless you're running a big business or you're processing things for Facebook, who needs to classify it just a ton of stuff.

Every second, um, you can just run out on a CPU.

I mean, every model I've ever put into production ran on the CPU, even though we trained on the GPU.

So anyway, getting back to our let's, say, or $4000 jeep you for $2 an hour.

You know the other costs that you have when your local is just electricity.

So let's say you are running a $4000 jeep you 24 hours a day.

It's going to draw like the entire machine, maybe 50 to $70 per month, just in power for the machine.

The machine also is going to generate heat, which has to be accounted for with air conditioning, usually unless it's like in the winter.

But here in the south, I tend to budget about $100 per machine per month that I run If I'm running a 24 7 training models, so again, that adds up.

So so again, it kind of basically becomes about three months.

So if you were training the model nonstop for three months straight or a cumulative off three months and let's say three months over the course of a year, so 25% of the time you are training some sort of model on that GPU, Uh, then it starts to make sense to buy this GPU and have it local.

But there's other reasons you might want to buy a GP and have it local.

First of all, computer hardware is just cool, like I understand that.

So sometimes you know, you might be trying to justify buying it locally because it's going to save you money.

Just be, you know, realistic and honest to yourself.

Just admit you wanna buy cool stuff and that's okay.

Like if you just want to buy it toe own it.

But you're not saving any money like the cloud just makes more sense because you have options, right?

At any point, you can change to a different cloud provider.

You could at any point decide I'll just go local now, right?

But if you go local, you spend $5000 in here and then you train for only 100 hours over the course of a year because I'm gonna be honest, like it's actually pretty difficult to hit that, um, 2000 hours of training mark in a year, I've done it.

But I've also gone long periods of time where I don't do it, and it really depends, like the only reason I'm able to do it is because I have clients.

And then I also do like tutorials, and so some always like working on something.

Subtitles ListPlay Video

Cloud vs Local GPU Hosting (what to use and when?)

opportunity

recommend

basically

average