DANIEL SITUNAYAKE: Hey, everybody.

So my name's Daniel.

LAURENCE MORONEY: And I'm Lawrence.

DANIEL SITUNAYAKE: And we have an awesome talk

for you, this afternoon.

So I've been super excited watching

the keynote, because there's just been so much stuff

that is relevant to what we're going

to be talking about today, which is running machine

learning on devices.

So we've seen the kind of amazing things

that you can do if you're running ML at the edge

and on device.

But there are a ton of options that developers have

for doing this type of stuff.

And it can be a little bit hard to navigate.

So we decided to put this session together

to give you an overview.

We're going to be showing you all the options,

walking you through some code, and showing you

some awesome demos.

So hopefully, you'll enjoy.

So first, we're going to walk through using TensorFlow

to train models and then export saved models, which

you can then convert to deploy on devices.

We're then going to see a number of different ways

that you can deploy to Android and iOS devices.

And finally, we're going to talk about

some new and super exciting hardware devices that you

can use to run your models.

So first of all, I want to give you

an overview of some different technologies and the device

types they each allow you to support.

So to begin with, we have ML Kit,

which is designed to make it super easy to deploy

ML inside of mobile apps.

We then have TensorFlow.js, which basically

lets you target any device that has a JavaScript interpreter,

whether that's in browser or through Node.js.

So that even supports embedded platforms.

And finally, TensorFlow Lite gives you

high performance inference across any device or embedded

platform, all the way from mobile phones

to microcontrollers.

So before we get any further, let's

talk a little bit about TensorFlow itself.

So TensorFlow is Google's tool chain for absolutely everything

to do with machine learning.

And as you can see, there are TensorFlow tools

for basically every part of the ML workflow,

from loading data through to building models

and then deploying them to devices and servers.

So for this section of the talk, we're

going to focus on building a model with TensorFlow

and then deploying it as a TensorFlow Lite model.

There are actually tons of ways to get up

and running with TensorFlow on device.

So the quickest way is to try out our demo apps and sample

code.

And we also have a big library of pretrained models

that you can drop into your apps that are ready to use.

You can also take these and retrain them

based on your own data using transfer learning.

You can, as you've seen this morning,

use Federated Learning to train models

based on distributed data across a pool of devices.

And you can finally build models from scratch,

which is what Laurence is now going to show off.

LAURENCE MORONEY: Thank you, Daniel.

Quick question for everybody.

How many of you have ever built a machine learn model?

Oh, wow.

DANIEL SITUNAYAKE: Wow.

LAURENCE MORONEY: Oh, wow.

Big round of applause.

So hopefully, this isn't too basic for you,

what I'm going to be showing.

But I want to show just the process

of building a model and some of the stops

that you can then do to prepare that model

to run on the mobile devices that Daniel was talking about.

Can we switch to the laptop, please?

Can folks at the back read that code?

Just wave your hands if you can.

OK, good.

Wave them like this if you need it bigger.

OK, some do, or you just want to stretch.

Let's see.

How's that?

OK, cool.

So I'm just going to show some very basic TensorFlow

code, here.

And I wanted to show the simplest

possible neural network that I could.

So for those of you who've never built something in machine

learning or have never built a machine learn model,

the idea is like with a neural network,

you can do some basic pattern matching

from inputs to outputs.

We're at Google I/O, so I'm going

to talk about inputs and outputs a lot.

And in this case, I'm creating the simplest

possible neural network I can.

And this is a neural network with a single layer

and a single neuron in that neural network.

And that's this line of code, right here.

The Keras.layers.Dense units equal 1.

Input shape equals 1.

And I'm going to then train this neural network on some data,

and that's what you can see in the line

underneath-- the Xs and the Ys.

Now, there is a relationship between these data points.

Can anybody guess what that relationship is?

There's a clue in the 0 and 32.

AUDIENCE: Temperature.

LAURENCE MORONEY: Yeah, a temperature conversion, right?

So the idea is I could write code that's like 9 over 5

plus whatever, plus 32.

But I want to do it as a machine learn model, just

to give as an example.

So in this case, I'm going to create this model.

And with this model, I'm just training it

with six pairs of data.

And then what it will do is it will start then

trying to infer the relationship between these data points.

And then from that, going forward,

it's a super simple model to be able to do a temperature

conversion.

So how it's going to work is it's going to make a guess.

And this is how machine learning actually works.

It just makes a wild guess as to what the relationship

between these data is.

And then it's got something called a loss function.

And what that loss function is going to do

is it's going to see how good or how bad that guess actually is.

And then based on the data from the guess

and the data from the loss function,

it then has an optimizer, which is this.

And what the optimizer does is it creates another guess,

and then it will measure that guess to see how well

or how badly it did.

It will create another guess, and it will measure that,

and so on, and so on, until I ask it to stop

or until it does it 500 times, which is what this line of code

is actually doing.

So if I'm going to create this model quite simply,

we'll see it's going to train.

There was one that I created earlier,

so my workbook's taking a moment to get running.

My network connection's gone down.

Hang on.

Let me refresh and reload.

You love it when you dry run a demo, and it works great.

We get that warning.

I'll run that.

I'll do that.

And now, it starts training, hopefully.

There we go.

It's starting to train, now.

It's going through all these epochs.

So it's going to do that 500 times.

And then at the end of the 500 times,

it's going to have this trained model.

And then this trained model, I'm just

going to ask it to predict.

So for example, if I give it 100 degrees centigrade,

what's that going to be in Fahrenheit?

The real answer is 212, but it's going

to give me 211 and something, because this

isn't very accurate.

Because I've only trained it on six points of data.

So if you think about it, there's a linear relationship

between the data from Fahrenheit to centigrade on those six,

but the computer doesn't know that.

It doesn't go linearly forever.

It could go like this, or it could change.

So it's giving me a very high probability

that for 100 degrees centigrade, it would be 212 Fahrenheit.

And that comes out as 211 degrees centigrade.

So I've just built a model.

And what we're going to take a look at next

is, how do I get that model to work on mobile?

Can we switch back to the slides, please?

So the process is pretty simple.

The idea is like using Keras or using

an estimator in TensorFlow.

You build a model.

You then save that model out in a file format

called SavedModel.

And in TensorFlow 2, we're standardizing on that file

format to make it easier for us to be able to go across

different types of runtimes, like JavaScript,

TFX in the web, or TensorFlow Lite.

By the way, the QR code on this slide is to the workbook

that I just showed a moment ago.

So if you want to experiment with that workbook

for yourself, if you're just learning, please go ahead

and do so.

It's a public URL, so feel free to have fun with it.

And I put a whole bunch of QR codes

in the rest of the slides.

Now, once you've done that, in TensorFlow Lite,

there's something called the TensorFlow Lite Converter.

And that will convert our SavedModel

into a TensorFlow Lite model.

So the process of converting means

it's going to shrink the model.

It's going to optimize the model for running on small devices,

for running on devices where battery life is

a concern, and things like that.

So out of that process, I get a TensorFlow Lite model,

which I can then run on different devices.

And here's the code to actually do that.

So we've got a little bit of a breaking change

between TensorFlow 1 and TensorFlow 2.

So in the workbook that was on that QR code,

I've put both pieces of code on how to create the SavedModel.

And then once you've done that, the third line from the bottom

here is the TF Lite Converter.

And all you have to do is say here's

the SavedModel directory.

Run the TF Lite Converter from SavedModel in that directory,

and it will generate a .tflite file for me.

And that .tflite file is what I can then use on mobile.

So let's take a look at that in action,

if we can switch back to the laptop.

So all I'm going to do within the same workbook

is I'm going to run that code that I just saw.

And I'm using TensorFlow 1.x and Colab here.

And we shall see that it actually

has saved out a model for me in this directory.

And I need that in the next piece of code,

because I have to tell it the directory that it got saved to.

So I'll just paste that in, and then I'll run this out.

And we can see the TF Lite Converter is what

will do the conversion for us.

So if I run that, it gives me the number 612.

Can anybody guess why it gives me the number 612?

It's not an HTTP code.

I thought it was that, at first, too.

That's actually just the size of the model.

So the model that I just trained off those six pieces of data,

when that got compiled down, it's a 612 byte model.

So if I go in there, you can see I saved it

in /tmp/model.tflite.

And if in my Colab, if I go and I look at /tmp directory,