Name: Neural Structured Learning - Part 3: Training with synthesized graphs
Uploaded: 2020-03-25T09:24:40.000Z
Duration: 5 min 16 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

and welcome to episode 3 of Neural Structured Learning.

In the previous episode, we learned about natural graphs

and how they can be used in neural structured learning.

are many machine learning tasks where the input data does not

For instance, if you recall the document classification task

from the previous episode, we used citations

In the absence of citations, we wouldn't have had a graph.

Similarly, if you're doing simple image classification

or text classification where the input data contains

we can apply neutral structured learning to such tasks.

So what do we do if we don't have a natural graph

The title of the video might have given this away.

But the main idea is to build or synthesize

Building a graph can be done in many ways.

of similarity between instances to build a graph.

whether they are documents, text, or images,

to corresponding embeddings or dense representations.

We can do this using pretrained embedding models, such as those

Once we convert raw instances to their embeddings,

such as the cosine similarity, to compare

If the similarity score is greater than the threshold,

then we add a corresponding edge in the resulting graph.

Repeating this process to cover the entire data set

And once we have a graph, using neutral structured learning

is straightforward, as we saw in the previous episode.

Let's illustrate this workflow for the task of sentiment

And the task is to classify them as good or bad.

Let's see what the code looks like to build

a neural structured learning model for this task.

Here again, we use Keras for illustration.

But neutral structured learning also supports estimators.

The first step is to load the IMDb data set.

For simplicity, we use a version of it that is part of Keras.

the raw text in the movie reviews to embeddings.

We use swivel embeddings in this example.

But any other embedding model, such as Bert, Word2Vec,

we can build a graph using those embeddings.

Neural structured learning provides an API

Notice that it accepts the similarity threshold

below which edges are dropped from the resulting graph.

In this example, we use a threshold of 0.8.

Once we have the graph, we define the features of interest

with the graph using the partNeighbours API

In this example, we use a maximum of three neighbors

Now that we have the augmented training data,

the next step is to create a graph regularized model.

This part is similar to what we did in the previous episode.

can be any type of Keras model, whether it's

a sequential model, a functional API-based model,

It can also have an arbitrary architecture.

Then we define a graph regularization configuration

Once this configuration object is created,

you can draft the base model with the graph regularization

model whose training includes a graph regularization term.

What's left is then just compiling, training,

and evaluating the graph regularized model.

This example is also available as a colab-based tutorial

You can find that link in the description below.

In summary, we looked at how to build a graph regularized

model when the input data does not form a natural graph.

This technique can be applied to all kinds of input data,

Subtitles ListPlay Video

Neural Structured Learning - Part 3: Training with synthesized graphs

episode

approach

aspect

technique