to one more Authors at Google Talk.

Today, our guest speaker is Pedro Domingos,

whose new book is called "The Master Algorithm."

We have it here and you can buy copies outside.

So one definition of machine learning

is "the automation of discovery."

Our guest, Pedro Domingos, is at the very forefront

of the search for the master algorithm, a universal learner

capable of deriving all knowledge, past, present

and future, from data.

Pedro Domingos is a professor of Computer Science

and Engineering at the University of Washington.

He's the co-founder of the International Machine Learning

Society.

Pedro received his MS in Electrical Engineering

and Computer Science from IST in Lisbon,

his Master's of Science and PhD in Information

and Computer Science from the University of California

at Irvine.

He spent two years as an assistant professor at IST

before joining the faculty of the University of Washington

in 1999.

Pedro is the author or co-author of over 200

technical publications in machine learning, data mining,

and other areas.

He is the winner of the SIGKDD Innovation Award, the highest

honor in data science.

He's an AAAI Fellow and has received the Sloan Fellowship

and NSF Career Award, a Fulbright scholarship, an IBM

Faculty Award, several best paper awards,

and other distinctions.

He's a member of the editorial board of "The Machine Learning

Journal."

Please join me in welcoming Pedro, today, to Google.

[APPLAUSE]

PEDRO DOMINGOS: Thank you.

Let me start with a very simple question--

where does knowledge come from?

Until very recently, it came from just three sources, number

one, evolution-- that's the knowledge that's

encoded in your DNA-- number two,

experience-- that's the knowledge that's

encoded in your neurons-- and number three, culture,

which is the knowledge you acquire

by talking with other people, reading books, and so on.

And everything that we do, right,

everything that we are basically comes from these three sources

of knowledge.

Now what's quite extraordinary is just, only recently,

there's a fourth source of knowledge on the planet.

And that's computers.

There's more and more knowledge now that comes from computers,

is discovered by computers.

And this is as big of a change as the emergence

of each of these four was.

Like evolution, right, well, that's life on earth.

It's the product of evolution.

Experience is what distinguishes us mammals from insects.

And culture is what makes humans what we are

and as successful as we are.

Notice, also, that each of these forms of knowledge discovery

is orders of magnitude faster than the previous one

and discovers orders of magnitude more knowledge.

And indeed, the same thing is true of computers.

Computers can discover knowledge orders of magnitude

faster than any of these things that went before

and that co-exist with them and orders of magnitude more

knowledge in the same amount of time.

In fact, Yann LeCun says that "most

of the knowledge in the world in the future

is going to be extracted by machines

and will reside in machines."

So this is a major change that, I think, is not just for us

computer scientists to know about and deal

with, it's actually something that everybody

needs to understand.

So how do computers discover new knowledge?

This is, of course, the province of machine learning.

And in a way, what I'm going to try to do in this talk

is try to give you a sense of what machine learning is

and what it does.

If you're already familiar with machine learning,

this will hopefully give you a different perspective on it.

If you're not familiar with machine learning already,

this should be quite fascinating and interesting.

So there are five main paradigms in machine learning.

And I will talk about each one of them in turn

and then try to step back and see, what is the big picture

and what is this idea of the master algorithm.

The first way computers discover knowledge

is by filling gaps in existing knowledge.

Pretty much the same way that scientists work, right?

You make observations, you hypothesize

theories to explain them, and then

you see where they fall short.

And then you adapt them, or throw them away

and try new ones, and so on.

So this is one.

Another one is to emulate the brain.

Right?

The greatest learning machine on earth

is the one inside your skull, so let's reverse engineer it.

Third one is to simulate evolution.

Evolution, by some standards, is actually an even greater

learning algorithm than your brain

is, because, first of all, it made your brain.

It also made your body.

And it also made every other life form on Earth.

So maybe that's something worth figuring out how it works

and doing it with computers.

Here's another one.

And this is to realize that all the knowledge that you learn

is necessarily uncertain.

Right?

When something is induced from data,

you're never quite sure about it.

So the way to learn is to quantify that uncertainty using

probability.

And then as you see more evidence,

the probability of different hypotheses evolves.

Right?

And there's an optimal way to do this using Bayes' theorem.

And that's what this approach is.

Finally, the last approach, in some ways,

is actually the simplest and maybe even the most intuitive.

It's actually to just reason by analogy.

There's a lot of evidence in psychology

that humans do this all the time.

You're faced with a new situation,

you try to find a matching situation in your experience,

and then you transfer the solution

from the situation that you already

know to the new situation that you're faced with.

And connected with each of these approaches to learning,

there is a school of thought in machine learning.

So the five main ones are the Symbolists, Connectionists,

Evolutionaries, Bayesians, and Analogizers.

The Symbolists are the people who

believe in discovering new knowledge

by filling in the gaps in the knowledge

that you already have.

One of the things that's fascinating about machine

learning is that the ideas in the algorithms

come from all of these different fields.

So for example, the Symbolists, they have their origins

in logic, philosophy.

And they're, in some sense, the most "computer-sciency"

of the five tribes.

The Connectionists, their origins

are, of course, in neuroscience, because they're

trying to take inspiration from how the brain works.

The Evolutionaries, well, their origins

are, of course, in evolutionary biology,

in the algorithm of evolution.

The Bayesians come from statistics.

The Analogizers actually have influences

from a lot of different fields, but probably the single most

important one is psychology.

So in addition to being very important for our lives,

machine learning is also a fascinating thing,

I think, to study, because in the process of studying machine

learning, you can actually study all of these different things.

Now each of these "tribes" of machine learning, if you will,

has its own master algorithm, meaning its own general purpose

learner that, in principle, can be used to learn anything.

In fact, each of these master algorithms

has a mathematical proof that says,

if you give it enough data, it can learn anything.

OK?

For the Symbolists, the master algorithm is inverse deduction.

And we'll see, in a second, what that is.

For the Connectionists, it's backpropagation.

For the Evolutionaries, it's genetic programming.

For the Bayesians, it's probabilistic inference

using Bayes' theorem.

And for the Analogizers, it's kernel machines, also known

as support vector machines.

OK?

So let's see what just the key ideas in each one of these are.

So the Symbolists-- here are some

of the most prominent Symbolists in the world.

There's Tom Mitchell at Carnegie Mellon,

Steve Muggleton in the UK, and Russ Quinlan in Australia.

And their idea is actually a very interesting one.

It's to think of deduction-- sorry,

it's to think of learning as being the inverse of deduction.

Learning is induction of knowledge.

Right?

Deduction is going from general rules to specific facts.

Induction is the opposite.

It's going from specific facts to general rules.

So in some sense, one is the inverse of the other.

And so maybe we can figure out how

to do induction in the same way that people in mathematics

figure out how to do other inverse operations.

Like, for example, subtraction is the inverse of addition,

or integration is the inverse of differentiation, and so forth.

So as a very, very simple example,

addition gives you the answer to the question,

if I add 2 and 2, what do I get.

Subtraction-- and the answer, of course, is 4.

And this is the deepest thing I'll say in this whole talk.

And subtraction, of course, gives you the answer

to the inverse question, which is, what do I

need to add to 2 in order to get 4, the answer, of course,

being 2.

Now inverse deduction works in a very similar way.

So here's a simple example of deduction.

You know that Socrates is human and you

know that humans are mortal.

And the question is, what can you infer from that.

Well, of course, the answer is that, from that,

you can infer that Socrates, too, is mortal.

Now the inverse of this-- and that's

when it becomes induction-- is, if I know that Socrates

is human, what else do I need to know in order to be

able to infer that he's mortal.

And of course, what I need to know is that humans are mortal.

And so in this way, I have just introduced a new general rule

that humans are mortal.

Of course, in general, I wouldn't just

do it from Socrates, I would do it

from Socrates and a bunch of other people.

But that's the general way that this works.

OK?

And then once I've induced rules like this,

I can now combine them in all sorts of different ways

to answer questions that I may never even have thought of.

And this kind of flexibility and composition

is actually something that, of all the five tribes,

only the Symbolists have.

Now of course, these examples are in English

and computers don't understand natural language yet.

So what they use is something like first order logic.

So these things, both the facts and the rules

that are discovered, are represented

in first order logic.

And then questions are answered by chaining those rules,

by reasoning with them.

OK?

But whether it's in logic or natural language,

the principle is the same.

OK?

And as I said, of all the five paradigms,

this is the one that is most like scientists at work.

Right?

They figure out, where are the gaps in my knowledge.

Let me enunciate a general principle

that will fill that gap.