Name: AI YouTube Comments - Computerphile
Uploaded: 2020-04-15T17:08:21.000Z
Duration: 15 min 21 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

Michael is YouTube comments you've got on your screen.

Gerund

No, these are completely fake comments that I have generated using a neural network, so I've trained it on YouTube.

So basically, I've read a lot of computer file comments on now generating semi plausible new comments.

But talk about Lennox on DDE floppies and job, for example, all kinds of semi plausible things, but not really.

Let's talk a bit about recurrent Newell networks.

Are videos on machine learning deep learning convolution?

Your nets Google deep Dream in this sort of thing.

A lot of people are doing research in new networks now on dhe, some of them are doing things like natural language processing.

So things like trying to predict sentences or workout sentiment analysis from sentences, things like is this sentence of happy centers or sad sentence?

If it's an angry tweet or relax tweet, I'm gonna very briefly recap, but I would advise people to go back to these other videos first.

So if you've got a new or network on dure trying to make some predictions and tech or anything sequential.

You know, Linc's going up here and like this, and we put in something in this side.

Let's say a sentence and we want to predict something about that sentence.

I may be what the next word is going to be, or whether or not but be sentences, a positive or negative one or so you know, this sort of thing.

The problem is that this is a very fixed structure.

I might put the first letter in here and the second letter in here and the third letter in here.

And as long as all my sentences have the exact same length in the exact same structure, that kind of thing work.

But as soon as things start to move around a bit like words, but we're looking for, like the first name of the person in the sentence of the subject, it's gonna become much more difficult.

So let's imagine, you know, using that example that I'm trying to predict the subject of a sentence based on some sentences, right?

So I've collected together lots of lots of sentences, and I've got lots of lots of the subject of that sentence, and I put all the sentences in and try and learn the subject.

The problem is that the subject moves around in a sentence.

So this network has got quite a difficult job to learn not only how to determine what the subject is, but also how to do that when it could be anywhere.

So you you might say, You know, I went to the park on Tuesday, but you might also say, on Tuesday I went to the park, So if you're trying to predict when you went to the park, you gotta learn until it's like it's more complicated than you packed first think So What we really want to try and do is put in sequential data.

We want to put in one word or one letter at a time and then try and make decisions on that.

So we need some way off examining this overtime.

So if we simplify this into a sort of a different, slightly different structures, let's imagine we're training a very simple your network to take characters.

Let's say T H E and produce a guess as to what the Nets character will be.

That way we can work out things like what?

We could also, for example, have an output.

But that major sentiment analysis, because it's starting to generate and work out structure of words.

So let's say our input is t and then h and many in a row, right?

So we have our input nose here of our network.

We have some kind of learned mapping between these two things.

And then we have an output, which is our prediction of what the next character will be right now in a normal new network situation.

Although, you know, this is, I've turned this now on its side of it.

Essentially, what we're trying to do is say, Well, look, I don't put here is h or it should be a tch.

So we go forward to the network, we see whether it gets his age and we essentially calculate a loss and say, Well, that wasn't very good.

You thought it would be said, And so we then adjust these weights here to try and make that a little bit better.

The problem with this sort of structure is that we're only ever trying to make a decision based on one character input.

So you know that the letter that follows is going to differ depending on what the other letters are, you know, isn't always going to be an A or a tea or something else.

This is inherently gonna struggle with this kind of problem.

So what we do is we also want to bear in mind what we've seen before.

We want to say we've seen an H, but we've also seen a team previously.

And we do that by basically making more weights and connecting up the seven layers like this.

This is essentially a current Newell network.

This is gonna be expanded unrolled structure.

But it's the same kind of principle when we first start this h zero, we put in the tea and it goes, Well, I'm predicting, given a t what the most likely letter is.

It could be in a could be lots of different things.

All right, so we make it, we make an attempt to guessing here.

We put it in a day and we say, Well, we've already seen it t and we know about because it's stored somewhere in this hidden area.

So you know, it could be any next for the what could be an A for that, like that Kind of I kind of guessed.

It's unlikely to be a Zed, you know, unless people would make lots of typos, we can predict maybe that this could be any next.

They won't actually do see any now and so we can put in here and say We looked We saw t we saw on H.

Can we make it really any more informed decision as to what's going on there?

And it turns out we can so we can see what it could be a space because it maybe we're at the end of the word.

Or it could be in a because it's the word is theater or an end.

Because the word is then there's lots of different ideas it could be.

But you can imagine if this was very long, we could be bringing in words from previously as well.

So we could say, Well, look on DDE isn't what makes more sense than and then I see that's not true.

It depends on you Gotta have more sentence.

The more sentence you have, the more powerful your your your prediction is going to be on.

What this will actually predict is basically the likelihood of the next letter.

So it will say, given th e, I think the next output has a 10% chance of being in a but a 50% chance of being a space right and some other chances of being other characters maybe get the idea.

The only other interesting thing about becoming your networks is that the weights here and here and here are shared.

So they are the same because otherwise this one here would always be learning what it is to see the first character.

And this one will be, you know, seeing the 1st 2 characters.

So what we do is when we when we train this network up.

We actually train these hidden neurons to predict in the general case of wherever they are in the sentence, and that makes him really good at varying blank V input.

So this works really well for YouTube comments, for example, because they often different lengths.

They might have a similar structure, but they're very, very different lengths.

So I thought have moved off my sentiment analysis and, you know, word prediction kind of thing to kind of character level, low level thing.

But you could imagine if you just ignored these outputs here and we have only one output at the end of a sentence, you could try and predict something like the sentiment of a sentence, whether it's a happy one or predict the subject of his engines or something like that.

Processing lends itself quite well to this kind of idea.

So I've trained one of these up, and I thought, Well, what I want to do is do some word.

Prediction is lots of fun examples online, and we'll link a blogger really good brought post in the comments are from really fun examples of predicting text.

You could basically Creighton your network that does this and then basically just start generating random text.

Subtitles ListPlay Video

AI YouTube Comments - Computerphile

sort

essentially

process

content