Hello, and welcome to another Beginner's

Guide to Machine Learning with ml5js video.

Now in this video, I am going to unlock something for you.

It's already unlocked for you, but I'm

going to show it to you, that is incredibly powerful for what

you could do now with ml5 that you

couldn't do before but many of you

asked about in the comments.

And what is that?

It is the Save/Load feature extractor.

This is a new feature that was added

to ml5 just five days ago.

You need to make sure that you are using ml5 0.1.3,

or whatever number in the future past that.

But certainly, this is the version of the library

that I'm using in this video.

Now what does it do?

So the last example, if you've been

watching these video series in order, was this example.

What this example does is it loads

a pre-trained image classification

model called MobileNet.

And MobileNet is trained on 1,000 different kinds of things

and recognizes puppies and dogs and birds

and different kinds of objects.

Transfer learning is the process by which

we take that pre-trained MobileNet model

and basically disconnect it from all those labels

and reconnect it to our own labels.

For example, I'm going to make up a label called Happy

and a label called Sad.

I could certainly have more than just two.

And I'm going to show it things like the train whistle

is me being happy.

I'm going to show it that train whistle a bunch of times.

Say, happy, happy, happy, happy, happy.

Now no train whistle is very sad.

I'm sad.

No train whistle is sad.

I'm spending way too much time on this,

because I haven't implemented the thing

that I want to implement.

Now I'm going to say train.

And it's going to train.

And then once it's done, happy.

Sad.

Happy train whistle.

Sad.

OK, so it works.

It is now learning to classify images in real time

according to those two categories.

But I'm a very big spaz.

And I'm going to just be over hitting Refresh.

And I have now lost that forever.

I no longer have that model.

It's gone.

The new feature is ability to save that custom trained model

and then reload it.

So if you're using this for an installation,

and you're going to take down the computer

and set it up every day, you can save that model.

You can imagine, there's lots of possibilities here.

So there's only two things that I really

need to add to the code.

There's a save function and a load function.

There's a bunch of pieces there.

But that's what I'm going to do right now.

So I'm going to go here into the code.

And I'm going to just add another button.

I have a Happy button, a Sad button, and a Train button.

I'm going to add a Sad button.

No, not a sad.

A Save button.

I'm going to call it Save.

And Save button, when the mouse is pressed--

I'm just going to say classifier.save.

That's it.

All I have to do is say classifier.save.

Let's see what happens.

So I'm not going to train it very--

actually, no.

I am going to train it.

Let's do a really good, solid training this time.

Because this is the one we're going to save.

Plus, it works.

So let's do the same thing.

Happy, happy, happy, happy.

Train whistle is a happy thing.

A happy, happy, happy, thing.

Just me is very sad.

There's no train whistle.

I'm so sad.

I'm very sad.

I'm very sad.

And now I'm going to train this.

Weird how the loss is 0.

I'm just going to not worry about that too much.

I'm going to hit Save.

You can see that, down here, by the way,

that I did this a couple of times practicing.

Now what it did is it downloaded to my download directory

two files model.json and model.weights.bin.

So those files will end up wherever the default downloads

directory of your browser is.

And the next step is just to load those files in.

But before we load them, let's talk about what's

in those files.

So those two files, I said model.json, model.weights.bin.

So what is a neural network?

What is a machine learning model?

What is the thing that we're saving?

Well, in this case, it's actually

saving the configuration of a neural network.

Now if you want to know what a neural network is,

I have some videos about that.

But I would refer you to the Three Blue One Brown video,

What is a Neural Network?

I will link to that in this video's description.

That will give you a much bigger deep dive into those details.

But if you look at that video, what you'll see

is there's basically a big diagram.

And the diagram has a bunch of inputs.

It has some outputs.

By the way, in this case, we could actually

say the outputs are just two.

A happy and a sad.

And what the neural network, what the machine learning model

outputs, is a probability.

Maybe 80% of it being happy, of that image being happy,

and 20% that image is sad.

So the whole point of this is to feed in an image.

It's the image, and maybe all the pixels of the image,

that are actually these inputs.

It goes through this magic neural network thing,

which isn't really magic.

It's a thing that you can learn about.

And then out the other end comes a guess

as to whether it's happy or sad.

Now what is all this stuff in the middle?

The stuff in the middle is typically referred

to-- and there many different styles and flavors and kinds

of neural network--

but in the zoomed-out view, in general terms,

is what's known as a hidden layer or hidden layers.

So every input is connected to the output but not directly.

There are some amount of nodes, maybe two hidden layers, each

with four nodes.

And every input is connected to every node.

And then every node is connected to every node.

And then every node is connected every output,

and so on and so forth.

So I could be here all day trying to do this diagram

and draw every connection between everything.

I'm not going to do that.

But all of the information about here

is what is saved in these two files.

Model.json is a file that just explains all of these pieces.

The layers, the outputs, the inputs, all of that stuff.

That is what is in model.json.

In a moment, I will just open up that file and look at it.

Model.weights is an interesting thing.

So the magic of a neural network,

what makes a neural network work,

is a number that's associated with every single one

of these connections known as a weight.

You can think of it as a whole bunch of dials.

So I'm tuning the dials, or I'm trying

to get the dials in the right position

so that it really makes good guesses about happy versus sad.

That's the training process.

Once that training process is done,

I want to save where all those dials are.

All of those numbers are in this file.

This is a binary format file, because there's

a lot of numbers.

Millions upon millions of connections,

potentially, between a lot of pixels and a lot of labels,

and a lot of hidden layers.

So this, you'll notice the file that we

saved is five megabytes.

Because it's tons and tons of numbers.

But this is just a very small file

with a little bit of text information about how

this is configured.

I spent a lot of time on that.

Hopefully, that's some helpful background to you.

Let's go back and actually look at those files.

So now I've got those files.

What I'm going to do is, I'm just

going to drag them into Visual Studio Code, which is what I'm

using to code this right now.

But you could be using any environment.

They didn't make it into the right place.

Let me try that again.

I'll clean this up later.

But I want them in this directory.

So you can see that they're there.

Model.json, model.weights.bin.

If I click on this, you can start

to see all the stuff in it.

There's information about the input shape.

And is it a sequential model?

And what kind of algorithm is it using?

And is it dense?

And it uses something called softmax.

All this stuff.

So this is way beyond the scope of what

I'm doing in these videos.

But if you're interested in more about these details,

you could look at some of my videos

that use TensorFlow.js natively to understand more pieces here.

But you can see here, this is where--

looking for the weights file, et cetera.

And this is really important.

This is really just what TensorFlow.js

would do natively.

But ml5 is helping with a little bit on top of it

by adding these happy and sad labels.

So now all we have to do is load the model now.

So we're going to go-- we saved that model.

And so the steps are--

the first thing we have to do is load the MobileNet model.

So we're not actually saving that original pre-trained image

classifier.

We're just saving the bits and pieces that are hooked into it.

So we can't hook into it until MobileNet is ready.

So once MobileNet is ready, we can then

say, classifier.load model.json.

Now there are two files, model.json

and model.weights.bin.

But ml5 is set up that, if you just give it one file,

it'll look automatically for the other file in the same place.

There are ways of customizing the file names and their paths

and all of that, but that, you can

look into in the documentation.

But the easiest thing for just to do this.

And then I'm going to say, custom model ready.

So I'm going to write another event function.

CustomModelReady.

And there, I'm going to say, custom model is ready.

So it's a two-step process.

We have to load MobileNet.

MobileNet is ready.

Then load model.json with the weights.

Custom model is ready.

Let's just run this.

Zoom back out.

And there we go.

Everything's loaded.

But I don't see any results.

I don't see any results.

Why?

Well, this sketch was written originally with code to train.