Everybody.

And welcome to part nine.

I think of the deep learning with the highlight A I competition 3 2018 All right, let's go.

So, uh, I've been working on is doing machine learning with Hal, like three and a two East where we left off.

We've got some code here and we figured out how to at least save the, you know, the frame basically from the game, at least of the information that we're curious and we visualized it we saw Okay, this is looking like what we want except for the flippy stuff.

But then we figured out what's going on there.

Um, now we're ready to actually create and save the data.

So we need to come up with some sort of principle to why we would want to save or not some data.

Now, Historically, I've done this before with, like, an evolutionary based model.

And if you just go off of, let's say the best between your model that allows for the model to actually get worse over time.

So if you train a model in that new model actually sucks for whatever reason, maybe over fit or whatever and then you go back and start playing some games.

You're still gonna have winning gains.

But actually, let's say the total hell I collected or whatever is going to be worse than before and we don't want that.

So instead, what I think I would rather go with is rather than an evolutionary based model, as in you know, who wins.

Instead, what we should do is go off of some sort of threshold of how light that's what this entire game is about.

Anyways, it's all about the how light.

So, um so that's what I'd like to do now, Um, a couple of things like So, for example, um, it's a little challenging to for the bite like the, but never actually knows when the game is over because of the way Hal itis set up.

And it's kind of unfortunate, but we can know in advance how many turns there will be given a game size.

So, for example, let me just zoom in a little bit here.

I guess what I'll do is I'll just make because later we could query so the body can query the game map and then we can get the size of the game map, which will always be square.

And then what we can do is have some sort of variable here called like maps, settings that will be a dictionary.

And then let's say the map is a 32 by 32.

That means there's going to be 400 turns.

Then we could just continue this.

So at 40 40 by 40 that means it's gonna be 425.

We have a 48 groups, not 49 48 which is gonna be a 4 50 Wow, I can't type.

I hope this doesn't turn out like yesterday's videos.

Anyways, 56 becomes a 475 and finally 64 is 500 turns.

So we could query the game map and we can know.

Okay, how many turns air there This way.

On the last turn, we could query and say how much I like, Do we have?

Doesn't meet a certain threshold.

Yea, nay, uh, if it does now will save the data.

So in theory, all of the bots in any given game could hit that threshold, and that would be good.

Now there's a couple of downsides with this one is each map is seated randomly.

So some maps If you just look at replays on like hal i dot io Some maps have an insane abundance of Hal I TTE everywhere, especially close to the to the drop off.

So these games are likely to produce higher How?

Light numbers.

Because ships didn't have to travel this far to get the bulk of the Hal I TTE Also, it allows you to create many more ships immediately.

So, um, this is somewhat problematic.

So, you know, maybe there should be some sort of combination of Winter and Hal i tor.

Maybe ships produced Hal I TTE or even better, some sort of maybe combination of, uh, like a percentage of how light.

And I think, you know, like the least on hala dot io.

When they visualize it, they do say what percentage of the how light was collected.

And I think instead probably eventually will go that route and, like, just go based on ah threshold of percent collected rather than how light collected because that might wind up confusing a model.

But for now, what I'd like to dio is continue on this trek that we've got going on.

So first of all, we want to set some sort of minimum threshold.

Um, so I'm gonna say threshold and let's just say I don't know, 1000 How light?

I don't think we're gonna be able to collect 1000 How light?

Because it's gonna be random, but I just want to set it there just to have a variable there.

Um, And then, um in fact, let's call this, uh, save save threshold, so it will require 1000.

How light?

Also, Just for now, I'm gonna throw in another variable called Total Turns.

So for now, we're gonna limit it to 50 turns.

Because when you call the highlights, you know, command line interface thing, there is a flag for, uh, I think it's turned ash limit.

And then you could say 50.

Uh, let me check real quick on, make sure that is true.

I think it's turn like, yeah, turn.

It's just a dash dash, turn limit, and then you can set so it doesn't matter.

The game size term limits 50.

Now, why do we want to do that?

Um, we'd like to do that because, um, the first models are gonna be totally random.

So the quote unquote reason why one day I was able to collect 1000.

How light, Let's say, And the other one, not when both of them moved 100% randomly is maybe one move out of all of the moves that was made by any one of the ships.

So if we allow this to go for 500 freakin turns, the results are probably it's gonna be almost untradeable cause the little kernel of what the A I did good during 500 turns is going to be obstructed by insane amounts of noise.

So instead, if we can lower the total turn number and then maybe lower the threshold like 50 turns to collect 1000 how light is going to be quite challenging?

But if we could do that, that would be beneficial.

Also, probably safe threshold.

Maybe we make that well, because we're gonna I can't decide what I want to do If I want to produce all of the shit cause you start with 5000.

How light?

So you might want to even save the threshold is 6000 or you know something like that I can't really decide what I want to do there.

Uh, because we're only gonna make one.

She was actually not 6000 but you'll start with 5000.

You make one ship, you'll have 4000.

So let's say anything about 4000 is good.

That means we at least dropped off something.

So in theory, we could save safe threshold.

It's 5000 if we want to collect 1000.

Anyway, we can We can hash that one out in a little bit, but I'll leave it at 5000 for now.

So, um okay, so we've gotta save threshold.

We better total number of turns.

Now what we want to do is let's add a max Max.

Keep forgetting my caps.

Lock doesn't work, Max ships equals.

Let's go with one for now.

Again, Um, overtime would like more ships, but it just like I was saying before, with the total turns, let's just do one ship and because those model is a ship only model.

So it's a very micro model.

So we'd like first to train a single ship to do a smart thing, just to see if that can worry, this will be the easiest version of this challenge.

Eventually, we do need the model it take the other ships into account.

So we do need a model that's training on the presence of other friendly ships.

That way, if a friendly ship is closer to a bigger plot of how light we don't waste our time trying to go to that spot because our other ships probably doing the same thing.

Okay, so Max ships one.

Let's go ahead and scroll down and fix that eso less than max ships.

Okay, so now what we want to do is we will also rather than n p dot save.

Ah, here.

What I'd like to do is I'm gonna comment that out.

We're in it.

Come over here and let's make a new directory.

And I'm gonna call this not in all caps.

I'm gonna call this training Underscored data.

So this is where all our training date is going to go.

If we meet a certain threshold, I'm gonna throw another if statement down here and want to say if game dot turn number equals, um what was the thing?

Mac, uh, turns total turns?

That's got to be it.

Total turns um So if that's the case, okay, so if we are on the final turn, because again R A I does not get information from the game that the game is over, so we have no way of knowing, but we do have a way of knowing what is the final turn.

And then we can kind of surmise.

Okay, Okay, it's the end of the game.

So?

So, if game dot turn number equals the total turns, what do we want to do?

Well, if me dot hallie amount is greater than or equal to that save threshold that we set, then we want to save.

So say, n p dot save whoops, and then where do we want to save it?

Let's make this enough sharing again.

Uh, training data, right?

Training, data training Did it yet?

Training data slash.

And then let's save it.

Let's save it by two variables.

Let's say, uh, the allied amount because later we might we might set an initial threshold, and then suddenly we've got, like, up bricking 20 games.

Uh, and then we still might wantto like somehow, uh, set some sort of limit there.

So for now, let's just say me dot Hal I TTE amount.

So that'll be the first segment of this file name.

So then later we could go in via python and os dot lister or something.

And then we could filter by that first thing with just a simple split string by dash and then get the zero with, uh, anyways Mita hala him out.

And then let's get a int er time dot time and, uh, that should be fine, but let's go ahead and go down to that like millisecond, just to be certain that it's a unique family.

Don't overwrite some other file, but I really think the chances pretty much none.

But okay, and then n p y.

So also, I'm not sure I brought in time.

I have not.

So we shall import time.

Um, now, later, total turns like once we don't set a turn limit.

Total turns would be equal to like you Price set total turns.

Um, somewhere down here, like around line 39 we would do Len, I think you have game map dot width and game map dot height.

Or maybe it's not sighs.

We'll figure that out.

Um, so you get that number.

And then you would say total turns equals that, you know, map settings, that number with the key.

So are as the key.

But in this case, we're gonna hard code 50 for now because we want the game to stop a 50.

Okay.

Um impeded Aussie Bob.

Bob.

Bob, Bob, I think we're all set.

So let's go ahead.

Actually, let me change that safe threshold.

I just feel it.

I feel dirty asking for that.

Let's just say the safe threshold is 4000 and 50.

So anything I don't even know I just don't know how many will collect that random.

So we'll just save it at 4000 eso anything actually do 4000 won anything Any drop off we're happy with?

Um, okay.

And then, um okay, secrets up.

Choice direction, or so they're already move randomly.

Um, I'm forgetting anything else, or are we ready?

Oh, I think we're ready to rumble.

Uh, so let's go into the run game.

Ah, Okay.

So now we don't want to versions.

This won't cause that same problem is before this problem before was, uh, that, like over Because we were using, like, the same directory where is in this case, it will create its own little unique file.

Uh, okay, so run game.

I kind of got rid of it.

Okay, so this is the command to run the game now.

We could just run one game really quickly, like, let's make sure it works.

Would be nice.

Um, and also, actually, let me just throw in replayed directory.

Let's just throw in a dash, dash, turn, dash, limit 50.

Okay, copy.

Save.

Uh, break that running again.

Let's see what happens if we hit the errors.

Okay.

Looks good.

Um, zero highlight was collected.

Let's run it again.

Still zero.

How light.

This could get ugly, man.

No.

Hello, Really?

I figured, like with random It can't be that hard toe accidentally go left then.

Right?

You know, left, right, right.

Oh, wait.

Oh, no, no, no.

Um, we're hitting an error, first of all, And because of that error, we kill the player.

I was missing that.

So we passed the file name, but not the array.

Okay.

Idiot.

Okay.

Um uh, what are we saving here?

Surroundings.

Surroundings?

Yeah.