Placeholder Image

Subtitles section Play video

  • [MUSIC PLAYING]

  • NICK KREEGER: Hey, everybody.

  • My name's Nick, and this is my colleague, Yannick.

  • We're going to talk today about JavaScript and ML.

  • TensorFlow.js is a library we launched last year.

  • It's a library for training and deploying ML models

  • in the browser and on Node.js.

  • We want to showcase what you can do today with the platform

  • and where we're going.

  • One of the great parts about the library

  • is there's really no drivers to install.

  • If you run it in the browser, you can get out-of-the-box GPU

  • acceleration.

  • The browser itself tends to be very interactive by nature,

  • which builds really great applications

  • and demos for using ML.

  • And privacy is a very important part to the library.

  • You can run inference and training locally

  • on the client, which works around all sorts of privacy

  • issues you might have with doing server-side inference

  • or training.

  • And what can you do today with the library?

  • Well, we have a collection of pre-trained, off-the-shelf

  • models that you can use without any knowledge of ML.

  • We also have the ability to take existing Python models

  • and convert them and run them in TensorFlow.js.

  • We also have a full stack for training, inference,

  • and low-level linear algebra.

  • And that runs in the browser and Node.js.

  • And we also have a bunch of platforms

  • that JavaScript can run on outside of just the browser.

  • The first thing I want to showcase

  • is some of our new, off-the-shelf models

  • we've launched.

  • The first one is a bunch of pre-trained-- or off-the-shelf

  • models are bunch of pre-trained models.

  • They are image, audio, and text classification models.

  • And the APIs are all user-friendly.

  • You don't have to worry about converting images.

  • The tensors are resizing.

  • They're very high-level, easy-to-use APIs.

  • These are available on NPM for local application development.

  • Or we also have pre-compiled hosted scripts as well.

  • We'll also be working on this a lot in the upcoming year.

  • We'll have more and more models as we go forward.

  • The first model is BodyPix.

  • We were actually showcasing this at our booth.

  • And I want to show you how easy it is to use this model.

  • So the first thing we need do is include the library

  • and our BodyPix model.

  • This can be done with our pre-compiled scripts.

  • So it's two simple imports.

  • And the next step is to--

  • we'll just create a image in the DOM.

  • So this is a body image detection thing.

  • So I have a picture of my toddler trying to do yoga.

  • It's kind of a funny picture that

  • isn't so much a human just like this and finding arms and legs.

  • So this is Frank.

  • He's trying to do something on the couch.

  • But I want to actually load the model

  • and find body parts on Frank.

  • So the first thing to do is to load the BodyPix model--

  • just a simple, one-line call.

  • And the next step is to call one of the methods we expose,

  • which is estimatePersonSegmentation.

  • And I can pass in a DOM element.

  • This returns a JavaScript object with the width and height

  • of the object or the image and a value for every pixel that

  • was in the image, if it's an arm, or a leg,

  • or a head, et cetera.

  • There's also a bunch of really easy-to-use methods for doing

  • filtering on the image.

  • So I can take the results of that

  • and render it directly on the DOM.

  • So it shows head, body, arm, and so on.

  • Another model we just launched just a couple of weeks

  • ago is the Toxicity model.

  • It's a out-of-the-box text classification model.

  • Again, to use this model, we'll use the pre-hosted scripts--

  • two lines of code.

  • And then I'll load the Toxicity model

  • and ask the model to classify just

  • some really lovely text, pretty PG-- you suck.

  • And I'll get a result back again as a JavaScript object

  • that has the seven labels that we identify

  • different types of toxic type text, and the probabilities,

  • and if it matches.

  • We also have the ability to take pre-trained Python models

  • and run them directly in browsers.

  • So if you have a pre-trained model today that's already

  • been trained in Python world, we have a command line tool

  • that makes it very easy to serialize the model as a JSON

  • object, and the weights, and distribute them

  • in a web format.

  • We support Saved Model, TFHub, and Keras models.

  • The converter itself right now supports

  • over 170 and counting ops.

  • And we will be TensorFlow 2.0 compatible.

  • I want to walk through how simple it is to use this.

  • I have a Python model.

  • I run it through the command line tool,

  • and then I can easily just a load that in my JavaScript

  • application.

  • Very simple.

  • And with that, I want to hand it off to Yannick

  • to walk through our training APIs.

  • YANNICK ASSOGBA: Thanks, Nick.

  • So in addition to working with pre-trained models,

  • TensorFlow.js also allows you to author and train models

  • directly in JavaScript, both in the browser and in Node.

  • The primary tool for this is the Layers API,

  • which is a Keras-compatible API for authoring models.

  • There's also a lower level op-driven API,

  • if you need fine control over model architecture

  • or execution.

  • And we're going to take a quick look

  • at what TFJS code for training looks like.

  • And the main takeaway is that it's

  • pretty similar to using Keras and Python,

  • but follows JavaScript conventions.

  • So the first step is to import the library.

  • And when working in Node,js, you can also use the Node.js

  • bindings, which execute the TensorFlow operations using

  • native compiled C++ code.

  • If you're on a system that supports CUDA,

  • you can import tfjs-node-gpu to get

  • CUDA-accelerated performance when

  • doing training or inference.

  • And this is what creating a convolutional model

  • for a classification task looks like in JavaScript.

  • As you can see, it's very similar to Keras code

  • and Python.

  • We start by instantiating a model.

  • We add our convolutional layers, and we finish our model

  • definition by adding a flatten operation and a dense layer

  • with a number of output classes.

  • Similar to Python, we use model.compile to get

  • it ready for training.

  • And here, we specify our loss function and our optimizer.

  • And model.fit is the function that drives the train loop.

  • In JavaScript, it's an async function.

  • So here, we want to wait for the result,

  • or wait for it to be done.

  • Once the model is done training, we can save the model.

  • And here, we're saving it to the browser's local storage.

  • We support saving to a number of different targets,

  • both on the client and on the server.

  • And finally, just like you're used to,

  • you can use model.predict to get a result from the model.

  • So over the past year, we've also heard feedback

  • from the community on ways we can improve the experience

  • of training with TensorFlow.js.

  • And two particular requested areas

  • are that of data management and data visualization.

  • So we'd like to show you some of the progress we've

  • made in those areas.

  • So first up is tf.data.

  • And it's an API for managing data

  • pipelines to drive training.

  • It's a JS analog to Python's tf.data

  • and provides a whole set of utility functions

  • for data set transformation.

  • And finally, it works with streams.

  • And the lazy evaluation allows you

  • to work with data that doesn't fit in memory,

  • which can be quite important.

  • So let's take a look at a simple example.

  • So here, we load up a CSV file, using tf.data.csv loader.

  • And we specify that we want to predict the price column, using

  • the isLabel attribute.

  • So this is going to set it as a label

  • in future transformations.