Learning From Machines by Ashi Krishnan | JSConf.Asia 2019 - VoiceTube: Learn English through videos!

Subtitles section Play video

Yes.
So this talk is that it's not actually that scary.
It just has some unusual things.
Some flickering moments, a few unsettling pictures, nothing graphic, nothing where you'll be looking at it and you'll say, like, Wow, this is disturbing I and I know why.
It's disturbing.
No, it's it'll be more like I don't know why, but this looks.
This is weird to me and we'll be asking why, Um and we'll also be talking about some things some of the kinds of things that we don't usually talk about, and all sorts of stuff kind of wants to pour down river when you open locks like that.
And so I want you to be here for it.
But if you can't be here for it than take care of yourselves Otherwise, hello, my name is Sashi, and today I want to talk with you about a hobby.
I want to share some of the things that I've learned at the intersection of computational neuroscience and machine learning.
For years, I've been fascinated by how we think, how we perceive how the machinery of our bodies results in qualitative experiences and why we have the kinds of experiences we d'oh why they're shaped like this and not like that, Why we suffer for years I've also been fascinated by a I and I think we all have.
We're watching these machines begin.
Thio approximate the tasks of her cognition, sometimes in unsettling ways.
And so today I want to share with you some of what I've learned.
Some of it is solid research.
Some of it is solid speculation and all if it speaks to a truth that I have come to believe, which is that we are computations, our worlds created on an ancient computer, powerful beyond imagining.
So let's begin part one hallucinations.
This person is Michaela Perry Oh Nieto and he has something to show us.
It starts with the simple patterns and splashes of light and dark.
I kind of like images from the first eyes.
These give way to lines and colors and then curves and more complex shapes.
What's happening is that we're diving through the layers of the inception image classifier, and it seems that there are whole worlds in here shaded, multi chromatic hatches, the crystalline farm fields of an alien world.
The cells of plants toe understand where these visuals air coming from Let's take a look inside the job of an image classified areas to reshape its input, which is the square of pixels into its output, which is a probability distribution.
So the probability.
But this image is of a cat.
The probability of a dog, a person, a banana, a toaster.
It performs this reshaping through a series of convolution, all filters convolution.
All filters are basically Photoshopped filters.
Each neuron in a convolution.
A layer has a receptive field, some small patch of the previous layer that it's looking at, and each convolution a layer applies a filter.
Specifically, it applies an image.
Colonel, A colonel is a matrix of numbers.
Were each number represents the weight of the corresponding input neuron.
So each pixel in a neurons receptive field is multiplied by its weight, and then we some, all of them to produce the output neurons value.
We apply that same filter across every neuron in a layer, and the values, not filter, are learned during training.
So we feed the classifier labeled image that something where we know what's in it.
It outputs predictions, and then we math to figure out how wrong those predictions were, and then we math again to figure out how to change the values in this filter to produce a better result.
So the term for that is greedy int descend.
The deep dream process, which is what's creating these visuals, inverts that So this visualization is recursive.
I didn't feel the next frame we feed the previous week.
We feed the current frame into the network.
We run it through the networks many layers until we find the letter that we're interested in.
And then we math.
What could we do to the input layer to make that layer activate more?
And then we had just the input image in that direction.
So the term for this process is great and sent.
Finally, we scale the image up very slightly before feeding it back into the network for the next frame.
That kind of keeps the network from fixating on the same details in the same places, and it also creates this really wild zooming off it.
Every 100 frames or so we moved to a deeper layer or layer that's off to the side.
Inception has ah whole lot of players, and they're not all arranged in a neat linear stack and that gives us this.
So we started with these rudiments of light and shadow.
And now, down here, we kind of have, ah City of Cagamas situation happening.
But then we're about to enter the spider observation Area in which spiders observed you.
But it's okay because soon the spiders will become Courtney's and the core G's are gonna become the seventies.
Later, we've got this, ah, space of nearly human eyes, which will transform into dogs, slugs and then dog bird slugs deeper.
Unfortunately, we had a saxophonist teleporter accident and finally, the flesh zones with a side of lizard.
So when I first saw, this was like, should I tell the story?
When I told the story when I first saw this, I I thought it looked like nothing so much as U.
S.
President Donald Trump and and I resolved to never tell anyone that certainly not on stage until my best friend was watching the same video.
And she said, You know, this is just kind of reminds me of, and I think that actually says more about the state of our neural networks than this when I think the lizard juxtaposition has something to do with it.
But I do want you to notice and think about what it means that all of the flesh in this network is so very pale.
So this is pretty trippy.
Yeah, why is that?
What does it mean for something to be trippy?
To figure that out?
Let's take a look inside ourselves.
Meet Scully's.
Scully doesn't need all this craft.
Were just looking at Scalia's visual A system which starts here in the retina.
Now Scalia's retina.
You're right now, our retinas.
They're actually pretty weird.
Light comes into them, and it immediately hits a membrane.
There's a layer of ganglia ions, which are not actually particularly photo receptive that they are a little.
Then there's a layer of more stuff that does important things, presumably and at the back.
Or the photo receptors, the rods and cones, which sends luminous and color.
So when light comes in, it has to go through these four layers of tissue hit a photo receptor.
That photo receptor is going to get excited.
It'll send out a signal to its ganglia NHS, which then have to send it to the brain.
Somehow, through the optic nerve, which has drilled through the center of your eye, which means the scent, the sensors and our eyes are mounted backwards, and there's a hole in the center of them, and it's all okay because we patch it up later in software, a couple of other problems with our eyes to one.
We have about 120 million photo receptors, and there's 10 times you're gangly aunts than that, so it can't be a 1 to 1.
Mapping and to the band with of our optic nerve is 10 megabits, which is not, you know, like a lot.
I don't know when.
The last time you tried toe watch video over a 10 megabit connection is but much slower than WiFi.
Our cameras air about 100 mega pixels.
It's not gonna work.
And so our retinas do what you would do if you were given those design constraints.
They compress the data.
Each gang Leon is connected to a receptive field.
That's 100 or so photo receptors that is that are divided into a central disk and the surrounding region.
So when there's no light on any of them, the gang Leon doesn't fire.
When the whole field is illuminated, the gangly on fires very weakly.
When on Lee the strand is eliminated and the center is dark.
Half the gang liens in your retina fire release strongly, and the other half don't fire it all in the same situation.
But those gangly ons behave the opposite way.
They fire when the center is bright and the surround a star.
So these different species of gang liens, they're not actually different species.
They're just kindly owns that naturally behaved this way.
They're scattered, they're distributed evenly throughout your retina.
And if you think about what this does, creates an edge detection filter.
So we're doing processing even in our eyeballs in order to down sample the image coming in from our photo receptors 100 times while retaining vitally important information, namely, where the boundaries of objects are.
Okay, so then the signal is gonna go into the brain.
It's gonna hit the optic hi, Asma, where the data streams from our left and right eyes cross where we extract stereo vision.
It's gonna get processed by the thalamus, which is a switching center for all kind of signals in your brain.
It's responsible amongst other things for running our eyes.
Auto focus.
Each step in the signal pathway is doing a little bit of processing for extracting a little bit of information.
And that's all.
Before we even get to here, the visual cortex all the way around back.
Our visual cortex was arranged into a sack of neuronal layers.
The signal stay is actually pretty especially coherent throughout the visual cortex, so there's some slice of tissue in the back of your brain that's responsible for pulling faces out of this particular chunk of your visual field.
I mean, more or less.
Your brain is very squishy.
Each neuron in a layer of our visual cortex has a receptive field.
Some chunk of the entire visual field that it's looking at, and neurons in a given layer tend to respond to signals in the same way, and that operation distributed over a layer of neurons, it extracts futures from the visual signal.
Early layers extract simple features like lions and curves and edges, and then later, Blair's extract more complex ones like radiance and surfaces and objects, eyes, faces and movement.
It's no accident that we see very similar behavior and inception because convolution all neural networks like inception were inspired by the design of our visual cortex.
Of course, our visual cortex is different from inception.
In many ways, inception is a straight shot through it has branches, but no cycles.
Our visual cortex contains feedback loops, these pyramidal neurons that connect deeper layers to earlier ones.
Those feedback loops allow the results of deeper layers to inform the behavior of earlier layers so we might turn up the gain for edge detection along the edge of what is later detected to be an object.
This lets our visual system adapt and focus not optically, but intentionally.
It gives us the ability to ruminate on visual input well, because before we become consciously aware if it improving our predictions over time, you know this feeling.
I think you think you see one thing and then you realize that something else and these loop back pyramidal cells in our visual cortex are covered in serotonin receptors.
Different kinds of pre middle cells respond to certain and differently, but generally they find it exciting.
And don't we all.
You might be familiar with serotonin in its starring role as the target of typical antidepressants, which our serotonin re uptake inhibitors.
When serotonin gets released into your brain, they make it stick around longer there.
By treating depression, some side effects may occur.
Most serotonin in your body is actually located in your gut, where it controls bowel movement.
It signals to your gut that it's got food in it, and it should go on and do whatever it does to food.
And that seems to be with the molecule signals throughout your body.
Resource availability.
And for animals like us with complex societies, resource is could be very abstract.
Social resource is as well as energetic ones that your pyramidal cells respond excitedly to.
Serotonin suggests that we focus on that which we believe will nourish us now.
It's not correct as a blanket statement to say that pyramidal cells are excited by serotonin.
They're different kinds of serotonin receptors, and their binding produces different effects.
So five ht one day receptors tend to be inhibitory, somewhat drowsiness inducing five HT.
Three receptors in the brain, their associative sensations of queasiness and anxiety.
And in the cut, they make it run backwards.
So anti nausea drugs are frequently five ht three antagonised, so there's another serotonin receptor one that the pyramidal cells in your visual cortex find particularly exciting.
This is the five ht to a receptor.
This is the primary target of every known psychedelic drug.
This is what enables our brains to create psychedelic experiences.
So you go to a show, you eat a little piece of paper, and that piece of paper makes its way down into her stomach, where dissolves releasing molecules of lysergic acid.
Ayatollah my into your gun.
Now LSD doesn't bind to five HT three receptors, particularly.
So if you feel butterflies in your stomach, it's likely just because you're excited because you know what's gonna happen.
And what's gonna happen is this.
LSD will diffuse into your blood.
It has no trouble crossing the blood brain barrier because it's tiny but powerful.
Like you, it will diffuse deep into your brain into your visual cortex, where it finds a pure middle of five ht to a receptor and locks into place.
It will stay bound there for around 221 minutes.
That's four hours, which is a very long time.
They think that a couple of proteins snap into place and form a lid over top of the receptor.
Trapping LSD inside, which would help explain why it's so very potent with typical dose, is about 1000 times less than most other drugs.
So while it's rattling around in there, but little molecule of LSD is stimulating a feedback loop in your visual cortex, it's sending the signal.
Pay attention what you're looking at, maybe nourishing the pattern.
Finding machinery in your brain then starts to run over time and at different rates in one moment, the pattern and a tapestry seems to extend into the world beyond it.
In the next, it's the trees that are growing and breathing, the perception of movement of visual hypothesis that's been allowed to grow wild with Deep Dream.
We asked what would excite some layer if inception, and then we adjusted the input image in that direction.
There's no comparable Grady Int ascent process in the biological psychedelic experience.
That's because we're not looking at a source image.
We're looking at the output of the network.
We are the output of the network.
The output of your visual cortex is a signal carrying visual perceptions.
These, like proto Kuala, which will be integrated by other circuits in your brain into your next moment of conscious experience.
Inception.
Poor thing never gets that far.
We never even run it all the way to the classifications stage.
We never asked what it sees in all this, but we could.
We could perform the amplification process on a final result rather than an intermediate one.
Maybe we ask, What would it take for you to see this banana as a toaster?
Or say, Don't thes skiers look like a dog?
So these air adversarial examples images that have been tuned to give class if IRS frank hallucinations, the confident belief that they're seeing something that just isn't there and they're not completely wild, these robot delusions.
I mean, that sticker does look quite a lot like a toaster, and it's very shiny.
And those skiers do kind of look like a dog If you squint and I mean, there's the head, there's the body.
And if you were far away and tired and drunk, you might think that it was a big dog, but you probably wouldn't conclude that it's a big dog.
The recurrent properties of our visual cortex, not to mention the whole rest of our brains mean that our sense of the world is state ful.
It's a continuously refined hypothesis whose state is held by the state of our neurons.
Laying the groundwork for capsule networks.
Sarasa bore Nicholas Frost and Geoffrey Hinton, right?
A parse tree is carved out of a fixed, multi layer, and neural network like a sculpture is carved from wrong.
Our perceptions are process of continuous refinement, which may point the way towards more robust recognition architectures, recurrent convolution, all neural networks that ruminate upon images, making better classifications or at least providing some kind of signal.
But something is off about an input.
There are adversarial examples for the human visual system, after all, and we call them optical Visions, and they usually feel pretty weird to look at.
In this image, we can feel our sensory impression of the scene, flipping between three alternatives.
The box.
A little box in front of a big one, a box in a corner and a box missing one.
And this monk or illusion.
There is something scintillating about the color of the dots, which are all the same and are all brown.
If we designed conditional neural networks with recurrence they could exhibit such behavior as well, which maybe doesn't sound like such a good thing on the face.
Fit like let's make our image class fires vast, slating and uncertain and then put them in charge of driving cars around.
But we drive cars around, and it's our ability to hem and haw and reconsider our own perceptions at many levels.
That gives our perceptual systems such tremendous robustness.
Paradoxically, being able to second guess ourselves allows us greater confidence in our predictions.
We're doing science in every moment, the cells of our brain continuously reconsidering and refining, shifting hypotheses about the state of the world.
And this gives us the ability to adapt and operate within a pretty extreme range of conditions even while we're tripping face or while asleep.
Two dreams.
These are not real people like the one people Monica showed us yesterday.
These are the photos of fake celebrities trumped up a generative adversarial network, a pair of networks which her particularly creative.
The networks get better through continuous mutual refinement, and it works like this.
On one side we have the creator.
This is a deep, deep learning network, not unlike inception but trained to run in reverse.
This network we feed with noise, literally just a bunch of random numbers, and it learns to generate images.
But it has no way to learn to play this game.
In the technical parlance, it lacks ingredient without another opponent, the adversary.
The adversary is an image classifier again like inception but trained to run on only two classes, riel and fake.
Its job is to distinguish the creator's forgeries from true faces.
We feed this network with the ground truth with actual examples of celebrity faces, and the adversary learns.
And then we use those results to train the creator.
So if it makes a satisfying forgery, it's doing well.
If it's forgeries, air detected, we back propagate the failure so that it can learn.
I should tell you that the technical names for these networks are the generator and the discriminator.
I changed the names because names are important and also meaningless.
They don't change this underlying structure of the training methodology, which is a game.
These two neural circuits are playing with each other, and competition is inspiring.
When we spar, our opponent creates for us the tactical landscape that we must traverse Justus.
We do the same for that.
Together our movements ruminate on a space of possibility is much larger than any fixed example set Genz contrarian remarkably well on relatively small amounts of data.
And it seems likely that this kind of adversarial process could be helpful for neural circuits of all kinds.
It was not without its quirks.
Genz are not particularly great.
A global structure.
Uh, this is fallout cow.
It is a cow with an extra body, just a CZ.
You may have spent the night walking through a house that is your house, but with many extra rooms.
These networks are also not super grated counting.
So this monkey has eight eyes because sometimes science goes too far.
Do something for me.
Next time you think you're away, which I think is now, count your fingers just to be sure, but even awkward if I didn't have five Now, if you find that you have more or fewer than you expected, please don't wake up just yet.
We're not quite done.
Another interesting thing about this training technique is that the generator is being said noise or vector of noise.
Some random point in a very high dimensional latent space.
And so it learns a mapping from this space onto its generation target in this case faces.
If we take a point in that space and we just kind of drag it around, we got this, which is also pretty trippy now.
I mean, this resembles things.
I've seen things someone who isn't me has seen on asset.
This resembles the sorts of things that you may have seen in long forgotten dreams.
Now I don't have a magic school bus ride to take us on to understand why that is.
But I do have a theory.
When we see a face, there's a bunch of neurons in our brain that light up and begin resonating the signal that is the feeling of looking at that particular face.
Taken together, all of the neurons implicated in face detection produce a vector, embedding some mapping from faces, two positions in a high dimensional space.
And so as we drag around, the generators vector here we are also dragging around our own, which is a novel and kind of unsettling sensation.
So that's a wild theory, but it's not totally without neurocognitive precedent.
Here we have a rat in a cage.
So we've hooked an electrode up to a particular neuron in the rat's brain, and these pink dots are the locations where it's firing.
If we speed this whole thing up and we're going to start to see a pattern emerge, this neuron is a grid cell, so named because the centers of its firing fields produce a triangular grid.
There's lots of different grid cells in your brain, each aligning to a different grid, and they collect data from your visual system from Head Direction cells, which encode position of your head.
And together these cells, they construct an encoding of our position in two D Euclidean space.
This operates even in our sleep.
If earlier you discovered that you were dreaming and you want to see the end of this talk, which are having trouble not waking up, Oh, narrow knots recommend Spinning around this detach is your perceived body, the one with 12 fingers and three extra bedrooms from your physical body, which is lying in bed.
This positioning system is something which, on some level, you always knew existed.
After all, you know where you are in space, you have a sense of space as you move through it.
And it's likely even necessary if we believe that cognition is computation, that our qualitative sense of position has a neuro cognitive precursor.
Some signal in the Web that tells us where we're at.
In many senses of the word for three sticks and stones, they say you can't tickle yourself because you know it's coming specifically when your brain sons an action.
Commend her muscles called in reference when an references sent your brain makes copy Now makes a copy of sound so planned, so engineered your brain is this big, messy evolved signal processing mash.
So another way to think of reference copies is those reflections.
We take the reference, and we send it out to the peripheral nerves where presumably it's going to make some muscles contract.
Meanwhile, from the reference copy, we predict how our bodies state will change, and we use that to update our brains model off our bodies stayed.
Now, if we didn't do this, then we would have to wait for sense data to come back to tell us what happened.
I mean, where is our hand right now?
And then we would face the same problem is trying to play a twitchy video game over a crap connection signals take about 10 milliseconds to travel from our brain to the periphery and about 10 milliseconds to come back.
It's just not that low Leighton see or high band with our nervous system.
And so to enable smooth, coordinated movements are brain has to make a prediction.
So life goes on.
But in a moment we have another problem.
See, we still receive sense data from our nerves, and if we update our model's skin, they would fall out of sing.
And so we attenuate this signal and keep the model insing.
This attenuation applies even to our sense of touch when that touch is an expected consequence of our own movement now expected consequence.
That's quite a complicated model.
An aspect of fit are likely distributed throughout our brain, but there is one place that is particularly important in maintaining it.
The cerebellum, the cerebellum is very special.
It contains half the neurons in our nervous system, all action commands from the brain to the body, right through it, and all sensations from the body to the brain as well.
It has long been recognized as vitally important to motor coordination like this.
People with Sir Beller Damage have trouble performing this action smoothly.
With Sarah Biller damage, it's our movements become jerky and Laghi.
It's there as that the cerebellum acts as a Smith predictor.
Our brands controller for Leighton see distant bodies able to estimate the body's current state integrate sensory feedback toe update that model and decomposed gross action commands generated elsewhere in the brain into a fine tune, continuously varying control signal.
And once you've evolved it, nothing like that has many uses.
There's a growing body of evidence implicating the cerebellum and language, which makes sense.
Utterance is a kind of movement and language, she said.
Kind of gesticulating is not limited to utter ends, the work of moving words.
It's not so different from the work of moving the body.
They're both transformations, from the space of internal states of reference and ideas to the space of world co ordinates, an external sense impressions and back again.
And what happens when this predictor encounters a problem when there is an irreconcilable discontinuity in the model?
But uh huh huh These are not extremely different.
They're both visceral gut Earl to shake our bones and jokes, too, are shaped like trauma.
They're both shattering Sze the illuminations of discontinuities paradoxes which cannot be and yet are things that we must revisit again and again.
Water smoothing the edges of cutting stone, the machinery of our brains trying to make sense of the world that resists these last few months have been very difficult for me.
The world.
It's heavy.
My heart is heavy.
We built cages for kids.
To Diane, we live in drowning cities built by slaves.
Meanwhile, I spend my days trying to make numbers into bigger numbers.
There are days when I opened my email and every subject line is a stone, and I think I should put these all in my dress and walk into the sea.
But I don't because I remind myself, because I remember that I am a process of creation.
I am a song singing myself.
We're Story is telling ourselves a C understanding itself, our churning waves, creating every moment of exquisite agony and exquisite joy and everything else.
It's you.
It's all you.
You aren't everything.
Everything you have ever seen, every place you have ever bean every song you've ever sung Every God you have ever prayed to every person you've ever loved And this boundaries between you and them and the sea and the stars are all in your head.
Thank you.