Hold on to your papers, because this work on AlphaGo is absolute insanity.

In the game of Go, the players put stones on a table where the objective is to surround

more territory than the opponent.

This is a beautiful game that is particularly interesting for AI research, because the space

of possible moves is vastly larger than in chess, which means that using any sort of

exhaustive search is out of question and we have to resort to smart algorithms that are

able to identify a small number of strong moves within this stupendously large search

space.

The first incarnation of DeepMind's Go AI, AlphaGo uses a combination of a policy network

that is responsible for predicting the moves, and a value network that predicts the winner

of the game after it plays it to the end against itself.

These are both deep neural networks and they are then combined with a technique called

Monte Carlo Tree Search to be able to narrow down the search in this this large search

space.

This algorithm started out with a bootstrapping process where it was shown thousands of games

that were used to learn the basics of Go.

Based on this, it is clear that such an algorithm can learn to be as good as formidable human

players.

But the big question was, how could it possibly become even better than the professionals

that it has observed?

How could the disciple become better than its master?

The solution is that after it has learned what it can from these games, it plays against

itself many-many times to improve its skills.

This second phase is the main part of the training that takes the most time.

Let's call this base algorithm AlphaGo Fan, which was used to play against Fan Hui a 2-dan

European Go champion, who was defeated 5 to 0.

This was a historic moment and the first time an AI beat a professional Go player without

a handicap.

Fan Hui described his experience as playing against a very strong and stable player and

he also mentioned that the algorithm felt very human-like.

Some voiced their doubts within the Go community and noted that the algorithm would never be

able to beat Lee Sedol, a 9-dan world champion, and winner of 18 international titles.

Just to give you an intuition of the difference, based on their Elo points, Lee Sedol is expected

to beat Fan Hui 97 times out of 100 games.

So a few months later, DeepMind organized a huge media event where they would challenge

him to play against AlphaGo.

This was a slightly modified version of the base algorithm that used a deeper neural network

with more layers and was trained using more resources than the previous version.

There was also an algorithmic change to the policy networks, the details on this are available

in the paper in the description, it is a great read, make sure to have a look.

Let's call this algorithm AlphaGo Lee.

This event was watched all around the world and can perhaps be compared to Kasparov's

public chess games against Deep Blue.

I have the fondest memories of waking up super early in the morning, jumping out of the bed

in excitement to watch all these Go matches.

And in a long and nailbiting series, Lee Sedol was defeated 4 to 1 by the AI.

With significantly less media attention, the next phase came bearing the name AlphaGo Master,

which used around ten times less tensor processing units than the AlphaGo Lee and became an even

stronger player.

This algorithm played against human professionals online in January 2017 and won all 60 matches

it had played.

This is insanity, but if you think that's it, well, hold on to your papers now.

In this newest work, AlphaGo has reached its next form, AlphaGo Zero.

This variant does not have access to any human played games in the first phase and learns

completely through self-play.

It starts out from absolutely nothing, with just the knowledge of the rules of the game.

It was trained for 40 days, and by day 3, it reached the level of AlphaGo Lee, this

is above World champion level.

Around day 21, it hits the level of AlphaGo Master, which is practically unbeatable to

all human beings.

And get this, at 40 days, this version surpasses all previous AlphaGo versions and defeats

the previously published worldbeater version 100-0.

This has kept me up for several nights now and I am completely out of words.

In this version, the two neural networks are fused into one, which can be trained more

efficiently.

It is beautiful to see these curves as they show this neural network starting from a random

initialization.

It knows the rules, but beyond that, it is completely clueless about the game itself,

and it rapidly becomes practically unbeatable.

And I left the best part for last - it uses only one single machine.

I think it is fair to say that is history unfolding before our eyes.

What a time to be alive!

Congratulations to the DeepMind team for this remarkable achievement.

And, for me, I love talking about research to a wider audience and it is a true privilege

to be able to tell these stories to you.

Thank you very much for your generous support on Patreon and making me able to spend more

and more time with what I love most.

Absolutely amazing.

And now, I know it's a bit redundant, but from muscle memory, I'll sign out the usual

way.

Thanks for watching and for your generous support, and I'll see you next time!