Name: Building a Visual Debugging Tool for ML - TF.js in Interactive Visual Analytics (TF Dev Summit '19)
Uploaded: 2020-03-25T04:21:04.000Z
Duration: 5 min 19 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

My name's Lezhi, and I'm a software engineer at Uber.

So today I'm going to talk about how we build a visual debugging

tool for machine learning using TensorFlow.js.

that they spend the majority of their time

on iterating and debugging of the existing model.

to improve efficiency of the 80% of the time.

Traditionally, the only guidance for model developers

into how to improve on a model or why a model performs

So given the intrinsic opacity of machine learning algorithms,

The idea here is that we can transform a model space problem

of asking what went wrong with the model,

we look at on which data did this model make mistakes.

And instead of asking why a model makes certain mistakes,

So based on those two ideas, we developed

Manifold, which is a model agnostic visual debugger

The user will connect Manifold to the output data

set of several machine learning outcomes,

and Manifold will automatically segment these data sets

data points with similar performance with each other.

The users would choose the subset of their interest

to compare against each other, and Manifold

would highlight the feature distribution difference

of these two different subsets, and helping

them to diagnose the behavior of the performance outcome.

So while we developed these ideas into production,

And among them there's a performance challenge and also

So traditionally, it is the model training backend's job

to handle the performance metric calculation.

to our visual interface because of the latency introduced

by the recalculation in response to the user interaction.

And also, if you want to connect Manifold

to another machine learning training back end,

there are two pieces of code we need to port out, the back end

But in reality, the metrics calculation logic actually

Those two reasons shows why we put this computation logic

And because this computation could get intensive as a data

TensorFlow.js to help us increase the competition

we compute and perform scores for each data point

And in the future attribution view, for each feature

we compute the distribution histograms of the two

histograms to compute KL-divergence to rank

that feature importance for model developers

And in all of those scenarios, TensorFlow.js

compared to plain JavaScript implementation.

And in some cases, the performance boosting

can be as high as 100 times, for the per instance model metrics

So to conclude, complex tasks such as machine learning

diagnosis can benefit a lot from numerical computation capacity

And TensorFlow.js opens up new opportunities for developers

Subtitles ListPlay Video

Building a Visual Debugging Tool for ML - TF.js in Interactive Visual Analytics (TF Dev Summit '19)

opportunity

majority

pattern

boost