Everybody.

And welcome to part three of our tensorflow G a s tutorial.

Siri's in this video in the next video, we're gonna be talking about incorporating tensorflow dot Js into a much more complex applications.

So he saw, like, a really basic application of doing linear regression and doing, like, a best fit line to some input data.

But this is a pretty boring application.

So now we're gonna do is something a little more exciting and that's going to be having an A I play pong against you before we get into.

I just want to give a quick shout out.

And thanks to the channels to most recent sponsors Leonard M and the Golden Square.

Thank you guys so much for your support.

You're awesome.

Okay, so, uh, first of all, as I've been saying, like, a 1,000,000 times, I am not a Java script programmer.

So, um, I knew what I wanted to do was do pong, but I I couldn't Code Pong and JavaScript pride to save my life.

So I used the following I put a link.

You shouldn't need it.

All the code I have in my text based right out basically, because we've modified it pretty significantly.

But this is the original code.

And if you wanna learn, like how to actually make it, I would go here.

Um, the link is in the text based version of the editorial, though if you want to check it out, Um, and coming on down here, basically, at the end of the day, this is this is what you wind up with some sort of pong.

Actually, that's not quite it.

I think they have a Js fiddle.

Yeah, where you can actually play pong over here.

So you move the paddle back and forth, and then the computer, This is like the computer than this is the player here, and you can play pong with it.

So anyways, what I wanted to do is take this code and adapt it to use tensorflow just to play Pong.

So that's what we're gonna be doing.

I'm gonna get this stuff out of the way now, and we're gonna get into it.

Also, um, you will definitely want to head to the text based version of this tutorial.

Part three.

There's going to be a significant amount of copy pasta because what I just don't see the point of us like writing this stuff out and then also writing out a lot of the code that's coming from the pond game tutorial.

The whole point is adapting JavaScript code to include JavaScript.

So this is definitely not a Java script tutorial.

I'm just not someone who should be teaching Java script.

So so anyway, uh, yeah.

So first we're gonna start off with just a pawn game dot html.

So I'm just gonna do new call this, uh, pong game dot html.

Cool.

Open that up and zoom and Ellen in, and here we go.

So all we're gonna have in here is the following.

So you've got your main content.

Div were bringing intense or floated Js and then finally, um, reloading this pong game dot Js.

And that's where basically, all our code is going to go.

So before we do that, um, we probably should understand.

Like, what would be How would you even do this?

Like, how will be trained in a I to play Pong in the first place kind of model are we going to use?

So my plan is to basically feed into the like there are features for the network are just gonna be like, where is what's the balls?

Location?

What's the your paddle location?

What's the enemy's paddle location?

And then, like all the previous things of those.

So that was the previous location of the ball.

Previous location of the paddle previous location of, um, the enemy basically.

So, um yeah.

So now we're gonna do is go.

Uh, I guess we're gonna make another.

Let's make another and we're gonna call this one pong.

What do we call this thing?

It's like pong game dot jazz.

Yeah.

Pong game dot Js?

Yes.

And then we'll open that again in sublime, and then we're just gonna start going through it.

And again, I'm just gonna be copying and pasting this and just kind of running through the text based version of the tutorial.

So first is the star model.

This actually isn't the final model end up going with, but this will do for now.

So we're just defining the model that we that we want to use.

So again, um, here, the input shape that we're defining, and in fact, what is it?

This actually came through, and I already want to mention that, OK, I'm just going to leave that for now.

Um okay.

Just herself.

For some reason, this still works.

And I have no idea why.

Why would the input shape unless it, like, slices it or something, which would be nuts.

Anyway, this actually works.

But I'm gonna delete that and go with this, which is better.

I can't believe that made it through.

That really confused me.

Initially.

I have no idea why that works, but it actually does.

So again, um, the eight inputs Not in this specific order, necessarily.

But where is the players?

Paddle.

Where's the computer paddle?

What's the ball?

Ex bolt?

Why?

Previous Bull X previous ball.

Why previous paddle, Ex previous paddle.

Why?

So those are all the things now?

The next major chunk of code here is all original from the pong code contrato Auriol that I'm linking to anyways, so I'm just gonna paste all of that in.

So again, you'll have to go to the Text Bay's freshest story because we're not going through all that.

Um, the next thing is, now we've got some custom code because, um, just paste it in.

So So this this is for the actual thing to move.

So when you're playing this version, that is from the tutorial, the computer player that we're gonna replace with an A I, um The way it moves is by following the ball, it just follows the ball around, and that's it.

So it's not like the greatest a I in the world, but it's it's somewhat challenging to defeat, but it's actually pretty simple.

Um, once you learned the simple way to defeat it.

Now, um, because we're not always gonna be using Peter Greenaway to update the paddle for our A I so actually are a eyes.

Logic is much, much simpler.

So are our neural network model is gonna return basically the move.

So we feed in, um, here we feed in those eight features, pass it through the all these beautiful hidden layers in the output is three units, and it's gonna be a one hot output.

So the output is gonna be something like 100 Um, it could be 010 or it could be 001 and this is corresponds to, like move left.

Move.

Don't move it all hold still.

And then move right.

And then what we're gonna do is apply an arg Max to this.

So then you're gonna say, probably like TF tard Max, for example.

I think it's camel cased.

I forget we'll get there in a second anyway.

And in this case, the art Max would be 012 Right.

So, too.

Then what we're gonna do is we're gonna subtract one no matter what.

No matter what the output is, we subtract one.

So in this case, the art Max would be zero.

We subtract one.

So subtracting one.

Assuming it was this one, that answer would be one.

So one is the move that's going to get past to this function here.

So one and then we're gonna move that paddle four times, whatever that move is.

So, in this case, 01 art Max is one minus one is zero times four, still zero.

And then, in this case, it was zero minus one negative.

One times four minus four.

So either it moves four pixels left, right?

Or nothing at all.

Okay, so that's basically the moving logic.

Now, the next bit, uh, next batch of code is almost mixed bit in batch and said a wordy dirt.

Now, uh, yes.

So the next batch of code is just from the, you know, the pong game tutorial.

So I'm not really gonna talk about that cop be pasta.

Okay, this is just some custom code to start collecting data for the A.

I also I guess there's a bunch of, like, this stuff here.

If you're not a job script programmer like meat, this is seems to be very, very similar to self in Python, where it's like just this, like, abstract object.

The only real difference between this and self that I've notices.

It works and functions.

So it's just like this overarching object, I guess, anyways, moving on someone composed below a much better description, I'm sure otherwise you can like Wikipedia or stack overflow.

Um, I think there's probably koro stuff, too, for you know, this for self or just What is this?

Um, yeah, Okay, now we have some code for it's a little more custom, and all we're doing here is, um this basically is.

You're saving data per frame.

So at the end of the day, what do we need to train a model?

We need training data, so we have to collect that data.

So in this case, what we're going to be doing is we're gonna play against the computer for n games.

We're gonna decide however many games we're gonna play.

So for end games, we're gonna play against the computer player.

And then after that, we're gonna take all the training data, hopefully balanced the training data, and then train the computer player to be an A I deep learning I to play against us.

So in order to do that, the other thing that we need to do is basically the computer players gonna learn based on what we do.

So in order to make that apple kable to the computer, we have to, like, flip the table.

Um, so that's what's going on here, speeder, Which I didn't mention it before.

Big thanks to Daniel Cookie L.

A.

For, uh, for helping you with this JavaScript code.

He's actually the one that wrote most of the the Java script here because if I didn't mention I'm not drive a script programmer.

Although neither is he.

I don't think.

Anyways, um, so there's probably airs here, and, uh, we apologize profusely.

So, um, now we want to do is, um we need to switch.

Like I was saying before, after playing so many games, we want to switch to that.

Aye, aye, player.

So that's what this is doing.

So, basically, every time, um, every time a score is made, we need to start a new turn.

And then, in this case, if the turn is greater than two, so after to hold games, we're gonna train in a You could change this because obviously two games is nowhere near enough games actually training out of play against you 10 is getting there.

I found some success after about 20 games.

The A I was like starting to learn some stuff, but you're definitely gonna need some more stuff to get here.

Ah, the other option.

I don't think it's unreasonable to expect players to play against this boring computer for 20 games.

Um, just to play against the Aye.

Aye, So what's the more likely scenario is later on, we'll load a model that's already pre trained, and then you just do transfer learning, and that makes a big difference.

So anyways, I'm gonna stick with one, just so we can get through this in the editorial in a timely manner.

So after every two games, we're gonna just clear out the data so doesn't get super bloated.

Um, you either could do this, or you don't necessarily have to.

That way, every time you train your training with all of the data, that's just a example.

For if you did want to clear out the data, there's a nice little function for you to do that.

Um, Now, we actually would after those two games and after we actually want a train, we need to actually train.

So in this case, we're gonna go ahead and train, and then all this is doing is it's splitting up the data, pushing it to our exes and our wise, um, defining the X's and y's is tense, sirs.

And then we train the model right here.

Okay, so, um, once we've done that, uh, come down, shoot.

I got lost.

Yes.

Okay.

So, um, then we need to be able to actually make the predictions themselves.

So paste that in, um and again very similar to what we had before.

As long as we actually have training data to make a prediction.

Uh, we make the prediction.

And then again, it's just the Arcamax grab that data via Data Sync minus one, like I was talking about before.

And then And then that gets past that.

The move function multiplied by four.

And that's how many pixels we end up moving.

Okay.

And then the last bit of code is just the pong, um, just from pong itself.

So let's go ahead and test this really quick.

I'll just open with chrome.

Pull it over.

All right, So now let's open up.

Consul, just in case we do hit some sort of air and I'm actually moving.

You do this.