Name: Neural Structured Learning - Part 2: Training with natural graphs
Uploaded: 2020-03-25T02:32:59.000Z
Duration: 5 min 12 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

I'm Arjun Gopalan, and welcome to episode 2

and how this new learning paradigm can be used to improve

In this episode, we'll discuss how Neural Structured Learning

can be used to train neural networks with natural graphs.

Before we begin, let's define what a natural graph is.

that have an inherent relationship with each other.

The nature of this relationship can vary based on the context.

are classic examples that we interact with on a daily basis.

in data that is commonly used for many machine learning

For instance, if we are trying to capture user behavior based

might make sense to model the data as a co-occurrence graph.

Alternatively, if we are working with articles or documents that

contain references or citations to other documents or articles,

then we can model the data set as a citation graph.

Finally, for natural language applications,

we can define a text graph where nodes represent

entities and edges represent relationships

Now that we understand what natural graphs are,

let's look at how we can use them to train a neural network.

Consider the task of document classification.

As an example, machine learning practitioners

might be interested in categorizing machine learning

papers based on a specific topic such as computer vision

or natural language processing or even reinforcement learning.

And often, we have a lot of these documents or papers

to classify, but very few of them have labels.

So how can we use Neural Structured Learning

The key idea is to use citation information

whose existence is what makes the data set a natural graph.

What this means is that, if one paper cites another paper,

then both papers likely share the same label.

from the citation graph leverages both labeled as well

This can help compensate for the insufficiency

You might be wondering, well, all this sounds great,

a Neural Structured Learning model for this task?

Since we are dealing with natural graphs here,

we expect the graph to already exist in the input.

The first step then is to augment the training data

This involves combining the input citation

to produce an augmented training data set.

The pack neighbors API in Neural Structured Learning

And notice that it allows you to specify the number of neighbors

In this example, we use up to three neighbors.

The next step is to define a base module.

In this example, we've used Keras for illustration,

The base module can be any type of Keras model,

whether it's a sequential model, a functional API-based model,

It can also have an arbitrary architecture.

Once we have a base model, we define a graph regularization

configuration object, which allows you to specify

Once this configuration object is created,

you can wrap the base model with the graph regularization

model whose training loss includes a graph regularization

You can then compile, train, and evaluate the graph Keras model,

just as you would with any other Keras model.

As you can see, creating a graph Keras model is really simple.

It requires just a few extra lines of code.

document classification also exists on our website.

You can find that in the description below.

Feel free to check it out and experiment with it.

In summary, we looked at how we can use natural graphs

for document classification using Neural Structured

The same technique can also be applied to other machine

Subtitles ListPlay Video

Neural Structured Learning - Part 2: Training with natural graphs

episode

essentially

process

context