Placeholder Image

Subtitles section Play video

  • [MUSIC PLAYING]

  • ELIE BURZSTEIN: If you are new to our session

  • on cutting edge TensorFlow, my name is Elie.

  • And today with Josh, Mike, and Sophien,

  • we have four exciting projects for you.

  • First, I will talk about Keras Tuner.

  • Then, Josh will talk about probability programming.

  • Next, Mike will talk about TF-Ranking.

  • And finally, Sofien will show you how to use TensorFlow.

  • Graphics

  • Let me start by telling you about Keras Tuner, which

  • is a framework we initially developed internally

  • to bring new [INAUDIBLE] model to faster to production.

  • Before getting started, let me ask you a question.

  • How many of you have ever spent time optimizing your model

  • performances.

  • Right, I see quite a few hands raised.

  • This is not surprising because getting the best model

  • performances is not easy.

  • Getting the best model performances

  • is hard because there are many parameters such as the learning

  • rate, the batch size, and the number of layers

  • which influence the model performances.

  • Moreover, those parameters are interdependent,

  • which make finding the optimal combinations by hand

  • very challenging.

  • This is why we rely hypertuning to automatically find

  • the best combinations.

  • However so far, hypertuning have been

  • known to be not easy to use.

  • So if hypertuning is essential to get

  • the optimal performances, this begs the question,

  • can we make it easy as one, two, three?

  • And the answer is yes.

  • This is why today I am happy to introduce

  • to Keras Tuner which is a tuning framework made for humans.

  • Keras Tuner is a tuning framework

  • designed to make the life of AI practitioner

  • as easy as possible.

  • It also helps hypertuner algorithm creators and model

  • designers by providing them with a clean and easy to use API.

  • For AI practitioner, which is most of us,

  • Keras Tuner makes moving from a base model

  • to a hypertuned model quick and easy.

  • Let me show you how this is done by converting a basic MNIST

  • model to a hypertuned one.

  • We will only have to change a few lines of code.

  • Here's our basic MNIST model that is a TensorFlow

  • Keras controlled API.

  • This is something I'm sure all of you have seen already.

  • As you can set on the side, all parameter are fixed.

  • For example, our learning rate is set to 0.001.

  • So let's transition it to a hypertunable model

  • in three steps.

  • First, we wrap up our model in a function.

  • Second, we define hyper-parameter ranges

  • for the parameter that we would like to optimize.

  • Here for example, we're going to optimize the learning

  • rate which is one of the most important parameters

  • to hypertune.

  • Finally as the last step, we replace our fixed parameters

  • with our hyper-parameter range.

  • And we're done.

  • Our model is ready to be hypertuned.

  • This is as easy.

  • Besides offering an intuitive API Keras Tuner,

  • we'll also provide you with state of the art hypertuning

  • algorithm, tuneable architectures which

  • are ready to go, and an automatic experimental

  • recording which make it easy for you to analyze, share,

  • and reproduce your results.

  • Originally, I started developing Keras Tuner to quickly try it

  • on new models for [INAUDIBLE] purposes

  • including the one we developed to protect Gmail

  • against malware and phishing.

  • Band detection is one of the core building

  • block we need to protect your inbox against phishing email

  • that impersonates the brand you love.

  • This is why today, our [INAUDIBLE] example

  • would be to show you how to build a simple and yet

  • accurate brand logo classifier as logo identification is

  • one of the critical components to detect brand spoofing email.

  • The first thing we need to do is to load our dataset.

  • In that example, we have about the 150 icons

  • from various brands, including the ones displayed on the side.

  • We also need to set a few variables

  • such as batch size and the number of icons

  • we're going to use [INAUDIBLE].

  • Next, our data set is very small so we'll

  • rely on data augmentation to create enough training data.

  • This slide shows you a few examples of the augmented icons

  • we're going to feed to the classifier as training

  • input, the output being the brand names.

  • We are going to save the real icon as our validation dataset

  • to make sure that our classifier degenerates well.

  • To establish a baseline, let's first

  • train a ResNet101v2 which is one of the most common and well

  • known model architectures.

  • As you can see on the [INAUDIBLE] graph,

  • our model did converge but the actuality on real icon

  • is not great.

  • We barely reached 79% accuracy.

  • And it's also quite big with 44 million parameters.

  • Well, that's OK.

  • We can use Keras Tuner to find a better model which

  • is smaller and more accurate.

  • So to do that, the first thing we need to do

  • is to create, as the MNIST model a model function and input

  • TunableResNet.

  • TunableResNet is a tunable version of ResNet

  • that we will provide with Keras Tuner

  • as one of the architectures which are ready to tune.

  • Next, you add a few layers on top of it

  • and combine the model.

  • Then well, we initialize the tuner

  • and we give it $500 to spend on tuning to find the best model.

  • And we ask it to maximize evaluation accuracy.

  • Finally, we have to launch the tuning

  • and wait for the results.

  • So did it work?

  • Well, yes it did.

  • Actually, Keras Tuner found a way better model.

  • Our new model have now 100% accuracy

  • and only takes 24 million parameters.

  • So we get a faster and more accurate

  • model thanks to hypertuning.

  • Keras Tuner works with many of the tools you love,

  • including TensorBoard, Colab, and BigQuery,

  • and many more to come.

  • One more thing.

  • Hypertuning takes a long time so to make everything

  • more convenient, we will be releasing

  • alongside with Keras Tuner an optional cloud service that

  • will allow you to monitor your tuning on the go

  • whether it's from your phone or from your laptop.

  • Here is a screenshot of an early version of the mobile dashboard

  • to give you a sense of what to expect.

  • So the design is not final, but the UI

  • will show you how long before you're tuning complete,

  • as well as offer you a visual summary of the model trained

  • so far so you can know how your tuning is going.

  • Thank you for attending today.

  • We are really excited about Keras Tuner.

  • And you can sign up today for the early access program

  • by heading to g.co/research/kerastunereap.

  • [APPLAUSE]

  • Josh is now going to talk about [INAUDIBLE] programming

  • and TensorFlow.

  • JOSH DILLON: OK.

  • Hi, I'm Josh.

  • And today, we'll be talking about everyone's

  • favorite topic-- probability.

  • So let's just start with a simple example.

  • So suppose we're trying to predict these blue dots--

  • that is, the Y-coordinate from the X-coordinate.

  • And Keras makes this pretty easy to do.

  • As you can see here, we have a dense neural network

  • with one hidden layer outputting one float.

  • And that float is the predicted Y-coordinate.

  • And we've chosen mean squared error as a loss, which

  • is a good default choice.

  • But the question is, how do we make our loss function better?

  • What does a better loss function look like?

  • And how would we even know?

  • How would we evaluate the fit of this model?

  • And so we would like to be able to specify the negative log

  • likelihood as a generic loss, but then

  • encode the distributional assumptions in the model.

  • And furthermore, if we're doing that,

  • then wouldn't it be nice to get back

  • an object for which we can query--

  • ask for the mean, the variance, the entropy, et cetera.

  • And the answer is we can do this.

  • Using a TensorFlow probability distribution layer,