Name: Graphing/visualization - Data Analysis with Python and Pandas p.2
Uploaded: 2020-03-31T18:23:57.000Z
Duration: 27 min 44 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

Everybody welcome back to another data analysis last night.

It's science tutorial with python and pandas were gonna be continuing our work with the avocado data set before we leave it for a new and exciting data set in the next tutorial.

Just a few more things I want to show with the avocado data set.

Uh, especially just issues you might run into overtime.

So the first thing we're gonna do is we're just gonna basically recreate where we were import pandas as p d d f equals p D ri PT Read C S v data sets of kado dot c S v Albany D f equals D f eagles DF where d f region is all Benny.

Not all Pandey often e d f dot set index, as dates in place is, he will be true in there.

Finally, uh, Albany DF don Actually, we're just gonna do average price price dot plot And I didn't look up the command, uh, soldiers from this twice and there we have it.

But there's a few issues that we're having right out of the gate.

The first issue is these, like, these dates are running over each other.

And when you see something like this, it probably means pandas doesn't actually realize it's a date.

So the first thing that I would do is convert that to a date time.

So when we read in the C S V, if you actually have a date, the pride of the big thing you'd want to do is go ahead and just say, D f date or whatever the column name is equals p dot to daytime, Uh, and then d f date.

So, um, this will probably try to think we've done done one in the 1st 1 But I don't think so.

Is our first time where we're actually reassigning the value of that column.

This is No, this is no different than any other function.

You can just map functions to the to the column here like that.

The other thing you could use, like map and apply and stuff like that, and we'll talk a little more about that.

But this is a nice built in pan dysfunction that's gonna convert this column.

Um, and then we're just reassigning that value because this is actually just gonna return a bunch of values for us.

Anyway, uh, now what if we go to graph that we could see?

Okay, they're nice and slanted, and it appears that pandas actually knows.

The next thing is, this graph is crazy, crazy, busy looking.

So the first thing I would think of to smooth it out is to use some sort of moving average.

So, um, if you don't know what, like a moving averages, it's like any point along the way.

Let's say we're gonna do a 25 moving average, so we'll take every point we're gonna say, Okay at this point.

So this point and the previous 24 points, what's the average?

And then we go one point over and do the exact same thing.

That and what that's gonna wind up doing is convincing these like crazy fluctuations, and it will hopefully smooth things out a little bit.

So the way the entity that is Albany d f on, then we'll do average prize and then dot rolling.

Then you would pass a number for the window and then what kind of rolling?

So you could say it rolling some, or in this case, we're gonna say a ruling mean and then we're gonna plot it Cool, except not cool, because this is pretty ugly.

So the first thing I would think of when I would see a chart like this is that possibly things were out of order.

So what if we said all Benny D f don't let's do a head, uh, 25.

So when we look here, we could see Okay, we start.

So it's in reverse order, and then we scroll down.

And at least in this case, it does look like everything is in proper order.

Uh, let's do dot index, but you can see here at least right out of the gate.

Um, the index does start at late December and then goes down and reverse.

So you would think, Oh, this is reverse chronological.

But apparently it's not because then we have 2018 which again goes in reverse chronological order.

So the first thing that I would do is like, Okay, let's make sure these dates are improper.

So what I would say is, um let's do all Benny D f dot sort index in place evils troop.

You can also sort data frames by specific columns and that will also adjust the index.

Just for the record, we do get a little bit of a warning here.

Uh, I will talk about that warning in a little bit.

So all I want to do now is actually just sort that thing.

And now we get a pretty graph, and I guess the warning doesn't show.

I don't know what you've already kind of sort of the index and stuff.

Anyway, we'll see the warning soon enough.

So now we get a nice, smooth out graft That actually is a little different than the other graph.

There are kind of like these two humps, but the second hump is quite a bit higher than the first.

It was clearly like, this one is clearly wrong.

And then even this one looks to be pretty darn wrong.

So the next thing that I want to dio is what if we actually want this to be a column in our data frame?

Because we can later output this data frame to a C S V again.

And it doesn't have to look exactly the same as it came in.

So we decided, Hey, this is a valuable meaningful column.

So, actually, what I want to do is copy this column, come down here and I say Albany D f.

I'm gonna say price 25 m A is equal to that value.

I will explain that momentarily Albany DF dot head.

And then we'll just do like three here and we can see him initially.

That's because tthe e rolling periods required 25.

So the 1st 25 are gonna be nan because there's nothing to calculate later you can set.

I think it's literally men periods or something like that, and you can overcome that.

Otherwise, what we could say is all been e d f dot tale three and boom.

We've actually got values alternatively, um, will almost certainly use this again in the future.

But you can say Albany DF dot dropping a for example on.

And in this case, what dropping is gonna do for us is remove any rose that have any values and a not any or any Anais.

Anyways, if removes, um, Vienna's so later if you have, like, gaps for whatever reason, or you just want to drop missing data for what N a n a N is not a number.

So anything that doesn't have values, you can just drop them.

If you want, I'm not gonna drop him here, but just know that you can.

So now that we've done that, let's talk about this air.