- Hello, and welcome to another Coding Train video

about neuroevolution.

So this video, what I'm going to do is,

I'm going to take my previous, finished,

sort of finished, version of neuroevolution,

the process of evolving the optimal weights

of a neural network to solve some kind of,

perform some kind of machine learning task.

In this case, I'm making a guess

based on the environment

of this very crude Flappy Bird game.

I'm not making this,

the neural network is making a guess

whether it should jump or not.

So this is a very simple, actually, scenario to solve.

And so it's a nice demonstration of the idea.

I've been meaning to return to this topic for quite a while

so I can try to look at some more complex scenarios.

So neuroevolution can be applied to a lot of problems,

at different,

reinforcement learning algorithms are also applied too,

but it's a pretty different technique.

There's like five, or six, or seven videos

all about neuroevolution

that I would recommend you check out,

or you can start right here.

Because what I'm going to do in this video is,

I am going to take the existing version that I made,

which used a toy neural network JavaScript library

that I implemented in yet another video series

to, kind of, learn more about neural networks

and spin up a very, very basic one

in JavaScript without other dependencies.

But I'm going to replace that now with TensorFlow.js.

What you're seeing here

is the last example from Part Five,

where I saved a trained model

and I'm loading it into a version of the game

and watching it play out.

What I did previous to that,

is what you're seeing right here,

which is, this is launching 500 random neural networks

to try to make guesses.

I can use this slider to sort of speed up the system,

and over time those neural networks

are going to perform the processes

of crossover and mutation.

It's not actually doing crossover,

it's doing crossover which would be the act of combining

two agents' genetic material together into one.

I'm just taking one and copying it,

but then I am mutating it

to optimally search for the configuration of weights

that will work the best.

And just to remind you, when I'm talking about the weights,

this is what I'm talking about.

So we have the Flappy Bird game,

this is the agent that needs to make a decision.

In a way it's a classification problem.

Should I go up or should I not go up, right?

These are the only two possibilities,

a very simple classification problem.

Two categories.

I have done, I'm the human being,

have basically done the feature extraction.

I could use the environment, the input to the neural network

as this image, like all the pixels of the game,

that would be a wonderful thing to try

and I would love to do that,

especially once I have convolutional layers

with TensorFlow.js,

which is not something I have

in my toy neural network library.

You might not know what a convolutional layer is.

Don't worry, if I use it, I will explain it.

But, I have done that feature extraction.

So I have decided that the features

that I want to use as inputs into my neural network

are, I think it was like, the bird y position,

the bird, y velocity,

the top pipe location, the bottom pipe location.

And then, I'll call this x,

the distance to the nearest pipe.

So these are what I've decided

might be all the important values to use from this game,

to feed into the neural network.

So that means the neural network

has one, two, three, four, five inputs.

These all get normalized

into a value with a range between zero and one,

and fed into the neural network.

Then the outputs is a classification problem.

So there are just two outputs,

and they would each output a number.

If I got something like 0.8 here and 0.2 here,

that means there is an 80%,

basically, an 80% probability confidence score

that I should jump, so I will jump.

Now I could actually pick random numbers

and that kind of thing,

but I'm just going to take the highest one,

the arg max, so to speak, and go.

So, neural networks are able to learn

a sophisticated amount of information through hidden layers.

So the inputs don't pass directly to the outputs,

but the inputs pass through a hidden layer.

And I don't recall what I picked

as the configuration of the hidden layer,

but I think it was something like eight.

So if there were eight, I think that's what it is.

We'll look.

One, two, three, four, five, six, seven, eight,

these are what are known as dense layers.

Meaning, every single node is connected to every node.

I will not sit here and draw all of them,

but I will start that process.

And then all of these, sorry, are connected to the outputs,

like this.

So a neural network's core data,

the configuration of that data is in a matrix.

'Cause every one of these connections has a weight.

So if there are five here and eight here,

that is 40 weights.

If there are eight here and two here, that's 16 weights.

Now the truth of the matter is, there's also a bias.

So there's a bias for each one of these

and a bias for each one of these.

So the configuration is all of the weights,

all of the biases for each of these nodes.

But these are the details that I cover a lot more

in the series about building a neural network

from scratch in JavaScript.

So if you want to learn about that, you can go back.

But all you need to know for here,

is that I have a way of creating this neural network.

I have a way of feeding these values into it

and looking at the stuff that comes out.

So let's go look at that code.

So if I go look at the code, we can see that here,

there's this idea of a bird object.

And the bird object makes a neural network

with five inputs, eight hidden nodes, and two outputs.

I don't know what I've said here, but this is known

as a feedforward neural network.

Multilayer Perceptron, it's two layers.

The inputs, it looks like a layer,

but these are just numbers coming in.

There is a hidden layer in the output layer.

That's important terminology.

So that's happening here, and you can see in the bird,

the bird sort of like, takes all of those properties,

its y position, the closest pipe,

the closest pipe's x position, its velocity,

makes that into an array,

feeds that into a predict function,

and then I just sort of like,

if the first output is greater then the second output, jump.

So this is that process.

And all of this happens

in this neural network library.

So there is neuralnetwork.js and matrix.js.

This is like if TensorFlow.js was made by one person

who really didn't know what they were doing (laughs).

That's what's in here.

So what I want to do, is I am going to delete this folder.

Right, I'm deleting it, it's gone.

Gone, move to trash, boom!

So now when I go over here and I hit refresh,

of course, it doesn't work.

Because we have no neuralnetwork.js.

Can I get this, I don't have a watch on,

in however long I have right now,

in the course of this video,

working again without my library,

but with TensorFlow.js instead.

That's the thought experiment here.

So here is my html file,

and you can see that it has the P5 libraries,

it's loading my neural network library,

and then these are all the files

for the Flappy Bird game itself,

as well as the file that includes some information

about how the genetic algorithm works.

So what I want to do, I need to take these out,

and then I want to import TensorFlow.js.

So I have the TensorFlow.js website up

and this is the script tag for importing TensorFlow.js.

So I'm going to add that here, and actually I am pretty sure

that the most current version is 1.0.4, so let's add that.

And now I'm going to go back here and hit refresh.

Now, of course, ah, NeuralNetwork is not defined,

so we have fewer error messages here now, which is good.

I just want to make sure tf is loaded,

so, for example, I can call tf memory,

and there is no memory being used,

but I can see that tf js is loaded,

'cause I can call tf functions in the console now,

which I'm going to need to do to figure all this out.

All right, so now the thing I think I want to do is that,

actually, I don't need to do this,

but I think it would be nice for me to create,

I'm going to create a file called nn.js,

and I'm going to make a little wrapper for a neural network.

And actually one of the reasons why I want to do this is,

as you know, I've made a lot of videos

using a library called ML5,

which is a wrap around TensorFlow.js,

which allows you to work

with some of the machine learning algorithms

without having to manipulate

the lower level details of TensorFlow.js,

and so this is, kind of, a little bit,

of maybe a preview of that.

Because this ultimately,

this idea of neuroevolution is something

that I would like to work on putting into the ML5 library.

So if I make a neural network class,

and I write a constructor,

we know here, that in the bird object

what it's doing is, it's making a neural network

with five inputs, eight hidden nodes, and two outputs.

So there's no reason why I can't actually

just keep that same structure,

and I'm going to have three arguments,

I'm just going to call them a, b, and c,

because at some point I might need

to call the constructor in different ways,

and I'm going to then say,

this.input_nodes equals a.

this.hidden_nodes equals b,

and this.output_nodes equals c.

Then I'm going to put, I already,

by the way, I did this in a coding class

that I'm teaching, last week,

so I kind of have quite a bit of the sense of the plan here.

So I'm also going to write a function

called this.createModel

'cause I might need to do that in different places

and then I'm going to make a separate function

called createModel.

So this now is the function,

where I want to create my neural network,

using TensorFlow.js.

And I'm going to use the layers API.

The truth of the matter is,

this particular architecture, right,

this is perhaps one of the simplest, the most basic,

vanilla, so to speak, neural network architectures.

It's got two layers, one hidden layer, very few nodes,

there's no convolutions, no fancy stuff,

just the very basic.