Everybody.

And welcome to part two of the Karl, a self driving car likely with reinforcement learning Siri's in this video.

We're going to just carry on with what we were doing.

The last video we just made sure Carl, that was working.

Connections were being made.

Scripts were running.

And now what we want to do is actually start working on our own, um, python code.

So my goal for the Siri's is to use reinforcement learning to train a self driving core.

Now, in order to do that like any big problem, we have to break these problem down into little itty bitty junks.

So the next small chunk that we need to be able to do is we need to be able to figure out how to spawn this car in the environment, how to control this car in the environment, and then how to get sensory data back from that car.

We need basically like at least to start two types of sensor data.

The first is like a hood camera, basically so, just like some sort of forward facing camera, and then we also need to sense whenever that car has been in an accident.

So to do reinforcement, learning pretty much all we want to do is drive around.

If we get an occlusion, that's bad.

If we drive around at decent pace, that's good.

Cool.

Done.

So this isn't gonna be a it's kind of gonna kind of be like the self driving car in Grand Theft Auto.

I'm not actually gonna try to honor road rules or speed limits or anything of anything.

I just find it much more entertaining to watch a car drive really, really fast.

So So take note if you want later, you can, ADM or rules and how you just determine that that reward function on and you can get the car to do whatever you want it to.

D'oh!

So anyway, what we're gonna do here is figure out.

How do we actually interface with Carla using python?

So if you go to the get started bit and scroll down to the python a p I tutorial, we've been on this page briefly in the previous video.

If I recall correctly, it's been a while for me.

It's probably just a few moments ago for you, but so be it.

And we basically don't forget, we've got actor Blueprint World actor.

This is both your vehicle.

Other vehicles, pedestrians as well as your sensors thes air.

All actors then blueprints these air the attributes of those actors of the vehicle Pedestrian sensors and the world.

This is your environment.

Basically the server.

Uh, we need to connect.

Set a time out.

This way we don't end up hanging forever and ever and ever we grab the world Once we have that information, basically, these will be your first few lines of code every time as well as this one.

You're pretty much gonna always reference the blueprint library.

And then here right away we have at least this one sensor, right?

This is for your collision sensor, which we will definitely be using.

Um And then this basically is going to show you maybe how to attach that sensor.

I don't really see it here.

Transform, Spawn Actor Blueprint transform.

This is probably doing it.

This little bit is kind of confusing to me as faras Like how we actually add things to things.

But I'll show you guys hopefully it will make sense when I get through it.

Uh, okay, so I think we're ready to rumble.

Uh, what I'm gonna do is we can leave this up for reference, I think.

And what I'd like to do is I'm gonna write everything in the inn.

That examples directory.

If you want, you can move.

What would it be, Carla than dissed?

You could move this into your site packages of python 3.7 and imported.

You could do that.

I'm just gonna keep it nice and simple and just keep it in the same package here.

So what I'm gonna do is just take, like, any one of these files.

I'm just gonna copy pasta just so you can not write a few lines on, we'll call this tutorial video to what a great name.

Uh and then we will open in sublime text.

Move this aside, and now we are ready.

Cool.

So what I'm gonna do is basically delete everything from there to here on the import.

Carla, that's fine.

And then everything else I'm just gonna delete from this script basically just didn't want to write that like, Well, this this bit, it just changes that system path so we can import that Carla egg easy enough So now what I want to do is begin talking about some fundamentals of what's going on here.

So, first of all, any time you've got a client server, we need to also probably clean up what we're doing.

So the first order of business that we're going to attend to is going to be We're gonna have this actor underscore list, and that'll be an empty list.

And then what we're gonna do is attempt to basically do our connection to the server, all that fun stuff, and then when we're all done, we need to clean up.

And what the way we clean up is four Actor in actor underscore list actor dot Destroy.

Uh, and then Prince all cleaned up.

Okay, so this'll probably be the starting script for pretty much any Carla code that you're right.

Honestly?

Well, actually, well, we would first need to connect, so let's do that as well.

So we're gonna take client equals, carla dot Kalai in, uh and then we're connecting local host again.

That could be some sort of remote public I p, which is super awesome.

In the case of reinforcement learning one of the hardships of doing it that way would be the Leighton see tune from.

So with reinforcement learning it's a particularly challenging one because each step needs to know what the result of that step was.

So if you have a server and you've got, like, 100 millisecond ping, it's not like everything's just kind of shifted over 100 milliseconds.

Instead, you have to take a step and then send that out and then wait for it to come back so you would get 200 milliseconds per steps.

So five frames per second, which probably isn't good enough.

So we're gonna keep everything local running on.

So in theory, this would all have to run inside the car.

Um, luckily, you would only need to run the model.

The model would actually predict pretty quick.

Like you probably wouldn't be doing exploration in reality.

Anyways, I digress.

So continue on client dot set underscore.

Time out.

We're gonna set this to two seconds in the docks.

They set a time out.

I think of 10.

I hope we don't take that long.

If we take more in two seconds to do something, there's probably something wrong.

So anyway, we'll keep to their They're going to say world equals equals client dot get underscore world And then we're gonna do that.

Blueprint library, Eagles, World war world dot Get underscore.

Blueprint underscore library.

Okay, this is probably more along the lines of the code that will start every Carla endeavor.

Um, then we're gonna pick a car.

This might also be pretty common, but we're gonna say BP for basically blueprint is going to be equal to that blueprint library dot filter.

And we're gonna filter by things that say model three.

So we're gonna grab a Tesla model three here, and I'm gonna say the item at the zero with index.

That's what we want.

So there's only one model three.

So this will load us in a blueprint.

That is the blueprint default for the Tesla model three.

So what?

I'm thinking about it.

I probably should just fire up Carla.

Um And then while that's firing up, I'll continue.

Hopefully nothing glitches out on me.

Well, that's doing it, but we'll find out.

So while we wait on that, um, we're gonna set once we've got the blueprint for the vehicle, the next thing I could hear the GPU must have it already.

Yep.

Um okay, So we've got our world, and what I'm gonna do is I'm just gonna look down and scroll up again.

Kind of center over here.

This thing looks like a strange smiley face with the goofy knows, huh?

I wish I could scroll out a little quicker, but anyway, this way, when we do spawn a car will be able to actually see it in the environment.

That's good enough.

Ah, the only place the car could spawn and we don't see it is if it's ponds in this, like tunnel area.

So cool.

All right, Now, uh, coming over here, basically, we've got blueprints.

OK, so what we want to do is if we spawn a car, we want to, actually, um, to do that, we need to spawn it somewhere.

So we're gonna say Spawn point equals, uh, we're gonna use random dot choice, and we have not yet imported random.

So let's go ahead, import random and come back down here.

Spawn point eagles, random dot choice for world get underscore.

Map dot Get spawn points.

So get spawn points has I believe, like, 200 spawn points that we uh, we can choose from, So we're just choosing randomly one of those spawn points and at least one is in the tunnel, so just take notes.

Sometimes you won't see it, but that's okay.

Again.

We're gonna only look down upon our our world here, Uh, a handful of times before we're, uh we're done looking at it this way, except for debugging purposes only.

So Okay, so once we've done, we've got the spawn point.

What we want to do is we want to say our vehicle is equal to world dot spawn underscore actor.

The actor that we want to spawn is the blueprint that we just set up here again.

We can change attributes of that blueprint before we spawn it.

If we so choose in this case, we're just gonna go with whatever the default is.

Um, respond the blueprint.

Where do we want to spawn the spawn?

The blueprint at the spawned point.

Okay, So that will spawn our car.

Now, one thing that we can say there's kind of two options here, So if you want to spawn n peces orjust computerized vehicles, what you could do is vehicle dot set underscore auto pilot to true.

Now this is funny because even in their docks, they they're like and we're stressing that this is not a self driving car code.

This is just rule based because has connected to our environment.

And to me, it's just crazy that we have to explain this.

But it was super common in this self driving car in Grand Theft Auto.

Siri's, where people were like the taxi's drive.

So much better than this and the other.

You know, cars and grand theft auto drive better than this.

Why don't you just use that code?

Well, that's hard coded that it's connected to the game engine so it knows where the things like lanes are.

It knows where other drivers are.

It's all rule based, um, because it's connected to the game engine, whereas in reality we're not connected to some game engine.

Otherwise, self driving cars would be so simple.

So anyway, I wish I didn't have to make that clear, and I wish they didn't either.

But you do so if you just want the car to drive around and act, sort of like a self driving car are not a self driving car, just a car that's driving on the road.

You can do this.

Um, I don't think they take erratic actions or do anything unexpected.

Those I wouldn't use this to replace human drivers or, like, simulate human drivers.

I suspect it's nothing like a human driver.

Anyway, we don't want to set autopilot.

We actually want to control the vehicle ourselves.

So to do that, Um, although one thing I would say is you probably could set autopilot and really quickly train a model to drive like autopilot drives.

We could do that again.

I don't want to do that, because it just happens to be the case that we have access to, uh, the game engine.

I don't really necessarily want toe training that way, but you probably good.

Anyway, um, vehicles are a pilot.

Okay.

So rather than setting the autopilot instead, we want to control the car.

And the way that we do that is just a vehicle.

Don't apply, underscore control.

And then it's carla dot vehicle control.

And we're gonna set throttle equal to full throttle and steer for now.

We'll just say 0.0.

We're just gonna make this car go straight now.

Uh, every time we spawn anything.

Um, we want to actor underscore list on upend and we want to upend that vehicle.

Now we're gonna say Blueprint is going to be equal to Well, I think what I'll do for now.

Uh, let's just make sure that runs.

Actually, that's probably a good idea.

So we need a couple other things.

One, we need the camera.

And then, too, we need the collision sensor.

Um, but for now, let me just run this.

Let's just see where we are right now.

Uh, where Carla is this?

Yeah.

Okay, Uh, what do we call it to Touro via?

So pie Dash 37 tutorial video to dot pie.

And what I'll do is I'll just put this over here.

I'll pull up the environment.