Name: Fairness Indicators for TensorFlow (TF Dev Summit '20)
Uploaded: 2020-03-31T07:02:03.000Z
Duration: 6 min 17 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

I'm a developer advocate within the research and machine

And today, I've got Thomas and Sean from ML Fairness team

at Google to talk to us about their new updates.

But first of all, what does the ML Fairness team actually do?

THOMAS GREENSPAN: So our team is an infrastructure team

And our mission is to help lower the barrier

And what do you have in store for us today?

to be presenting on Fairness Indicators and TensorFlow

that help developers regularly evaluate for fairness concerns.

And constrained optimization is a technique

where you can introduce a constraint into the model

So today, what we're going to do is first train

a simple model to predict if a subject is smiling or not

We're going to run fairness Indicators to evaluate

We're going to retrain using constraint optimization

to fix the fairness concerns that we may find.

So first, the data set that we'll be using

It has about 200,000 images with about 40 attributes,

such as smiling, not smiling, young, not young, that kind

We use keras, and we keep it very simple,

because that's not what we're concentrating on.

When we evaluate it, we see that we get 86% accuracy, which

So to do that, we specify how to set up the fairness indicators.

Specifically, we tell it that we're using various indicators

In this case, it's going to be young or not young.

In this case, we're going to be looking at false positive rates

As you can see, there's actually a really large gap

between the young slice, which, with the false positive rate,

is around 4%, and the not young slice, where it's about 11%.

We are going to apply constraint optimization on this model.

So we tell it that we're looking at smiling, not smiling,

and we tell it what slice we're interested in, young, not

Next, we ask the model to keep the false positive rate

And, finally, we define the optimizer that

Next, we train the model and we re-evaluate.

As we can see, the false positive rate for the overall

But on the not young slice, the false positive rate

has decreased by about 5%, which is really big.

It's worth noting that on the young slice,

the false positive rate has increased a little bit,

Well, thank you for that, and what's up next?

SEAN O'KEEFE: Within this next case study,

we'll follow on from Thomas's previous case study

to understand fairness indicators within TFX pipeline.

The tools we will be using will be many of the same tools

But with the addition of TensorFlow Extended,

or TFX, which is a Google production grade

And then also, ML Metadata, which will allow us to help

track and understand the underlying performance

And this is two years of actual criminal record

Now, it is important to note that using this data

of predicting that anyone might commit another crime.

I would encourage anybody that follows along with this

to also take a look at the AI Partnership and their article

Subtitles ListPlay Video

Fairness Indicators for TensorFlow (TF Dev Summit '20)

awesome

present

technique

improve