as good as the features you provide.

That means coming up with good features

is one of your most important jobs in machine learning.

But what makes a good feature, and how can you tell?

If you're doing binary classification,

then a good feature makes it easy to decide

between two different things.

For example, imagine we wanted to write a classifier

to tell the difference between two types of dogs--

greyhounds and Labradors.

Here we'll use two features-- the dog's height in inches

and their eye color.

Just for this toy example, let's make a couple assumptions

about dogs to keep things simple.

First, we'll say that greyhounds are usually

taller than Labradors.

Next, we'll pretend that dogs have only two eye

colors-- blue and brown.

And we'll say the color of their eyes

doesn't depend on the breed of dog.

This means that one of these features is useful

and the other tells us nothing.

To understand why, we'll visualize them using a toy

dataset I'll create.

Let's begin with height.

How useful do you think this feature is?

Well, on average, greyhounds tend

to be a couple inches taller than Labradors, but not always.

There's a lot of variation in the world.

So when we think of a feature, we

have to consider how it looks for different values

in a population.

Let's head into Python for a programmatic example.

I'm creating a population of 1,000

dogs-- 50-50 greyhound Labrador.

I'll give each of them a height.

For this example, we'll say that greyhounds

are on average 28 inches tall and Labradors are 24.

Now, all dogs are a bit different.

Let's say that height is normally distributed,

so we'll make both of these plus or minus 4 inches.

This will give us two arrays of numbers,

and we can visualize them in a histogram.

I'll add a parameter so greyhounds are in red

and Labradors are in blue.

Now we can run our script.

This shows how many dogs in our population have a given height.

There's a lot of data on the screen,

so let's simplify it and look at it piece by piece.

We'll start with dogs on the far left

of the distribution-- say, who are about 20 inches tall.

Imagine I asked you to predict whether a dog with his height

was a lab or a greyhound.

What would you do?

Well, you could figure out the probability of each type

of dog given their height.

Here, it's more likely the dog is a lab.

On the other hand, if we go all the way

to the right of the histogram and look

at a dog who is 35 inches tall, we

can be pretty confident they're a greyhound.

Now, what about a dog in the middle?

You can see the graph gives us less information

here, because the probability of each type of dog is close.

So height is a useful feature, but it's not perfect.

That's why in machine learning, you almost always

need multiple features.

Otherwise, you could just write an if statement

instead of bothering with the classifier.

To figure out what types of features you should use,

do a thought experiment.

Pretend you're the classifier.

If you were trying to figure out if this dog is

a lab or a greyhound, what other things would you want to know?

You might ask about their hair length,

or how fast they can run, or how much they weigh.

Exactly how many features you should use

is more of an art than a science,

but as a rule of thumb, think about how many you'd

need to solve the problem.

Now let's look at another feature like eye color.

Just for this toy example, let's imagine

dogs have only two eye colors, blue and brown.

And let's say the color of their eyes

doesn't depend on the breed of dog.

Here's what a histogram might look like for this example.

For most values, the distribution is about 50/50.

So this feature tells us nothing,

because it doesn't correlate with the type of dog.

Including a useless feature like this in your training

data can hurt your classifier's accuracy.

That's because there's a chance they might appear useful purely

by accident, especially if you have only a small amount

of training data.

You also want your features to be independent.

And independent features give you

different types of information.

Imagine we already have a feature-- height and inches--

in our dataset.

Ask yourself, would it be helpful

if we added another feature, like height in centimeters?

No, because it's perfectly correlated with one

we already have.

It's good practice to remove highly correlated features

from your training data.

That's because a lot of classifiers

aren't smart enough to realize that height in inches

in centimeters are the same thing,

so they might double count how important this feature is.

Last, you want your features to be easy to understand.

For a new example, imagine you want

to predict how many days it will take

to mail a letter between two different cities.

The farther apart the cities are, the longer it will take.

A great feature to use would be the distance

between the cities in miles.

A much worse pair of features to use

would be the city's locations given by their latitude

and longitude.

And here's why.

I can look at the distance and make

a good guess of how long it will take the letter to arrive.

But learning the relationship between latitude, longitude,

and time is much harder and would require many more

examples in your training data.

Now, there are techniques you can

use to figure out exactly how useful your features are,

and even what combinations of them are best,

so you never have to leave it to chance.

We'll get to those in a future episode.

Coming up next time, we'll continue building our intuition

for supervised learning.

We'll show how different types of classifiers

can be used to solve the same problem and dive a little bit

deeper into how they work.

Thanks very much for watching, and I'll see you then.