Name: 數據--用Python和Pytorch進行深度學習和神經網絡 p.2 (Data - Deep Learning and Neural Networks with Python and Pytorch p.2)
Uploaded: 2021-01-14T10:43:40.000Z
Duration: 28 min 17 s
Description: 【看影片學英語】數萬部 YouTube 影片，搭配英漢字典即點即查，輕鬆掌握單字發音與用法，長久累積看電影不必再看字幕。

現在進行式

Welcome to part two of the deep Learning with python and pint torch tutorials in this video, we're gonna be talking about data and really, that's gonna be it.

Now, I think a lot of times everyone wants to focus on the neural network itself.

But actually, the step of acquiring data pre processing your data and how you were going to generate over your data pretty much consumes, I would say, 90% of your time and energy.

Uh, at least in terms of thinking about your model.

Obviously, training time when you're just waiting for the model learn stuff can take a long time.

But you the work that you are going to be putting in this is the step where, uh, where you're gonna be doing probably the majority of your work.

So to begin, what we're gonna do is work with a kind of toy data set.

This is a really popular data set to use for beginners one, because it's a machine learning, learning ble data set like it's really simple.

There's no question we can definitely learn this week in tinker with it.

Um, it just it's a good starting data set.

So that's what we're gonna use that we're going to use a package called Torch Vision, which you should already have installed.

That should just be Pippen stole installed torch vision like then you can get it s o fun fact.

I also looked tried to look up how to change font size in Jupiter Lab.

Uh, I found so many responses about various things you could do.

So just curious for my future videos by the time you anybody tells me to be over.

What do I need to modify to modify just the font size.

I even tried editing some of the CSS files that dictate Jupiter Lab, and it didn't work, so Okay, anyway, totally baffled.

So please, if you actually know away, don't just link me to opposed.

So under the do Dorial you guys got to give me You've got to give back to me sometimes.

So what, we're gonna do first is we're gonna import torch, and then we're going to import torch vision and just my microphone here.

Uh, the other thing we're gonna do is from torch vision.

We're going to import transforms and data sets.

So Torch comes with a bunch of data sets, and I can't remember now if other data sets exist now in torch vision that weren't vision tasks.

But basically it's a It's a collection of data that is used for vision.

So most training data sets with neural networks have something to do with vision just because vision seems to be the big thing that we benchmark against.

I wished there were more data sets for other tasks.

I mean, there's some, but vision is clearly like the main interest that people are working with you because really, neural networks, air soft solving vision tasks and other machine learning algorithms just haven't been able to do that.

So just in terms of just money from like investments and business interests, it tends to be vision tasks is that's like a low hanging fruit that we can do something with right now.

Um, but obviously there are other tests as well, Especially when it comes.

Like advertising is a huge market where we want to be able to predict what people are going to do, Uh, and funneling and all that.

But anyways, um, Torch Vision just has a bunch of vision data for us, and as you're going to see as we continue on the Siri's is, uh, this is basically cheating to use their their built in data.

You are like I said, most of your time is going to be spent getting data preparing your data formatting it in such a way that's gonna work within our own network, right?

Like cause you again, you gotta convert things like categories or, you know, words have to be converted to numbers, all that kind of stuff.

So you're gonna spend a long time doing that kind of thing here.

It's already done for us, and then the other thing we typically need to do is batch ing, which we'll talk about in a minute and again, it's gonna be done for us Now, I will say in the next uh, not the next to tour because we'll probably be working on this.

But in in the next kind of model that we build the next day to set, we're going to use our completely our own data set.

Because I know this isn't what most people want to do.

Most people are gonna want to probably apply this to their own problem.

We're going to do this so it makes other things easier to learn.

And like I said, we're going to slowly increase complexity as we go through the Siri's so anyways transforms and data sets.

And so now what we're gonna do is we're gonna define our two major data sets.

And when it comes to just machine learning in general, you're gonna have two different things.

You're gonna have a training data set, and then you're gonna have a testing day to say, and it's important that you separate these out as soon as possible before you forget, because in orderto validate your data or your model, rather you want tohave what's called out of sample testing data.

This is the most realistic test that we can have.

Basically, it's data that has never been seen before by your machine because if you use in sample data.

If it has learned toe over fit, it will do really good on your end sample data and very poorly on your out of sample data.

So it's essential that we have truly out of sample data meaning data that the machine has just simply never seen before.

Because when you actually go to use this model in practice, it's going to be, you know, out of sample, right?

It's gonna be new data that has never seen you're hoping that it performs well and with neural networks, like I said before, because you've got millions of trainable tunable parameters, it is highly likely that if you train long enough, your machine will over fit.

So as we progress through the Siri's, we'll be talking a little bit more about how long to train what stats, tow, watch and stuff like that.

But anyways, you need train and testing data.

So for this I am just going to write out This is a bunch of stuff, but basically we're just working with this data sets stuff.

So we're gonna say data sets dot capital M n'est, and then you're gonna specify where you want the data to go.

Someone do open and close parenthesis Farah quotes.

And we're going to say this is trained equals true download equals true.

And then any transform so transformed equals transforms dots, Capital C composed.

And then in here you would paste all the transforms.

These are things we want to apply to the data, and I don't know why, but the data is not even though we're grabbing it from torch vision.

The data is not natively already intense.

Er, um So we have to convert it to attend sir's.

So the one of the transforms that we're going to apply here is going to be transforms dot too tensor.

Uh, this one is already built in, obviously the torch vision because we imported it here.

But later on you, you can actually write, like, if you really enjoy this, you can write your own data set and you can use this kind of Samos intact because because as you'll see it, there's a lot of things like, especially in this tutorial, won't become his obvious.

But in the coming tutorials, it will start to become pretty obvious how tedious Iterating over a data set, for example, can be I almost think it would be just as tedious to convert it to one of these, like data sets as well.

But anyway, just know that you can do that.

And then you can also write your own kind of custom transforms.

But for now, we'll just use the one that's built in here, and I believe that's a valid line.

So, actually, I'm going to go ahead and take this copy paste train will be set toe false.

So let's go ahead and run that real quick and you can see it's downloading.

And now, while that's going, I think we'll just start writing, uh, writing our next two lines.

So basically, this gets us our data, um, downloaded and kind of in a sort of variable here.

The next thing we want to do is actually load this into another type of object that's going to help us iterated over that data.

So let me write these two lines and then I'll explain why we're doing that because it might not seem obvious Why, Okay, we've got our training and testing data.

Why do we need to talk about how to reiterate over it?

So the next thing I'm just going to go ahead and right here is I did not think it was going to do.

We're going to say that is equal to data sets dot and then em, uh, to write, rewrite their grabbing the data.

Anyway, torch dot you tills dot data dot data loader And what we want to do is load in train.

So we're just gonna load in the data that's currently added to or is currently this train variable here.

字幕列表影片播放

數據--用Python和Pytorch進行深度學習和神經網絡 p.2 (Data - Deep Learning and Neural Networks with Python and Pytorch p.2)

stuff

stick

obvious

bunch