I thought we would talk a bit about logical induction paper out of the Machine Intelligence Research Institute.

Very technical paper, very mathematical and can be a bit hard to get your head around, and we're not going to get too far into it for computer file.

I just want to explain, like why it's cool and why it's interesting and people who are into it and read the paper themselves and find out about it.

One of the things about logical induction as a paper is that it's not immediately obvious why it's ending.

I safety paper.

Um, but like my perception off the way that married the way that the mission intelligence recessions to shoot is thinking about this is they're saying they're thinking like we're going to be producing at some point artificial general intelligence.

And as we talked about in previous videos, there are hundreds of ways really weird, subtle, difficult to predict ways that he's this kind of thing could go very badly wrong, and so we want to be confident about the behavior of our systems before we turn them on.

And that means ideally, we want to be making systems that we can, um, actually in the best case that we could actually prove theorems about write things that are well specified enough that we could actually write down and formally proved that will have certain characteristics.

And if you look at, like, current machine learning stuff old like date neural networks and that kind of thing, they're just very opaque.

Do you mean by that?

Because we don't necessarily know exactly why it's doing what it's doing.

Yeah, and also just that the the system itself is very is very complex and very contingent on a lot of specifics about the training data on dhe, the architecture and like, there's just yet effectively.

We don't understand it well enough, but it's it's like, not formally specified enough, I guess.

And so they're trying to come up with the sort of mathematical foundations that we would need to be able to prove important things about powerful A i systems.

Before we were talking about hypothetical Future A.

I systems, we've got time to print more stamps, so maybe hijacks the world's stamp printing factories.

And when we were talking about the stamp collector, it was really useful tohave.

This framework of an agent and say this is what an agent is.

It's a formerly specified thing on dhe.

We can reason about the behavior of agents in general.

And then all we need to do is make these fairly straightforward assumptions about our A i system that it will behave like an agent.

So we have idealized forms of reasoning, like we have probability theory, which tells you the rules that you need to follow to have good beliefs about the state of the world.

Right?

And we have, like, like, rational choice theory about, uh, what what rules you need to follow.

And it may not be.

It may not be like, actually possible to follow those rules, but we can at least formally specify.

Like if the thing has these properties, it will do this well.

And all we're trying to do is that they're thinking if we're building very powerful A.

I systems, the one thing we can expect from them is there going to be able to do thinking well.

And so if we can come up with some formalized method of of exactly what we mean by doing thinking, well, then if we do, if we reason about that.

That should give us some insight into how these systems will behave.

That's the idea.

So let's talk about probability theory.

Then this is where we should have a demo.

Pretty sure about Damon as nice.

Yeah, well, this is a fun one.

How's it, too?

I don't know how to play backgammon.

Anyway, here's to die.

So basic probability theory, right?

I roll the die.

Now it's under the cup.

And I can ask you was the probability that that is 51 in six, right?

One in six because you don't know it could be anything.

There was a time when people reasoned about the probabilities of things happening in a very sort of fuzzy way.

You'd be playing cards and you'd be like, Oh, you know, that card has shown up here.

So I guess it's less likely that he has this hand and people would, um, into it.

Yeah, and like there was some sense of, like, clearly we are doing something.

What we're doing here is not random.

It's meaningful to say that this this outcome or that outcome is more or less likely by a bit, or buy a lotto or whatever But people didn't have a really good explanation of how that all worked until we had probability theory, right?

We didn't It wasn't assistant, right?

Right.

And like practically speaking, you often can't do the full probability theory stuff in your head for a game of cards, especially when you're trying to read people's faces and like stuff that's very hard to quantify.

But now we have this understanding that, like even when we can't actually do it, what's going on underneath this probability theory?

So that's one thing, right?

Straightforward probability.

Now let's do a different thing.

And now I'm getting now I'm gonna need a pen.

Now suppose I give you a sentence?

That's like the 10th digit after the decimal point off the square root of 17 is a five.

What's the probability that that's true?

Well, it's much more difficult Problem.

One in 10 right?

One in 10 seems like a totally reasonable thing to say, right.

It's gonna be one of the digits.

We don't know which one, but if I You've got the paper here, you know you could do like division.

You could do this if you had long enough.

Like if I gave you say an hour with the paper and you could sit here and figure it out on Dhe.

Let's say you do it and you you come up and it seems to be like three.

I don't know what actually is.

I haven't done this calculation.

Let's do the calculation about expression.

Don't roam around.

On 23456789 Tenets of six.

It's a six.

OK, I picked that out of nowhere.

So I suppose you didn't have a calculator you were doing on paper I gave you, you know, half an hour an hour to do the like, long divisional.

Whatever is you would have to do to figure this out.

What's the probability?

That statement is true Now what do you say?

I'm gonna say zero right.

But you've just done this in a big hurry on paper, right?

You might have screwed up.

So what's the probability that you made a mistake you forgot to, like, carry the one or something could be one.

So it's not zero this yet.

There's, like, some smaller probability.

Um, and then if I left you in the room again, still with a piece of paper for 100 years.

1000 years, infinite time eventually, assuming that was correct.

Eventually you say zero.

Right?

Um, you come up with some, like, formal proof that this is This is false.

Um, whereas you imagine, if I leave you for infinite time with the cup and you can't look at you got look under the cup, it's one sick.

Then it's gonna stay 1/6 forever, right?

When you're playing cards, you've got these probabilities for which cards you think it depending on the game.

And as you observe new things, you're updating your probabilities for these different things as you observe new evidence, and then it may be eventually you'll actually see the thing, and then you can go to 100% 0.

Um, whereas in this case, all that's changing is your thinking.

But you're doing a similar sort of thing.

You are still updating your probabilities for things, But probability theory kind of doesn't have anything to say about this scenario.

Like probability is how you update when you've seen your evidence, and here you're not seeing any new evidence.

So in principle, whatever the probability of this is, it's one or a zero just as a direct, logical consequence of things that you already know, right?

So your uncertainty comes from the fact that you are not logically initiate In order for you to figure things out.

It takes you time.

And this turns out to be really important because it's so most of the time.

What you have is actually kind of a mixture of these types of uncertainty.

Right?

So let's imagine 1/3 scenario, right?

Suppose you're like an A I system, and that's your eyes.

Because you have a camera.

I do the same thing again.

And now I ask you, what's the probability that it's a five?

You would say one sick because you don't know.

But on the other hand, you have recorded video footage of it going under the thing.

The video is your memory of what happened, right?

So you're not observing new information.

You're still observing the same thing you observe the first time, but you can look at that and say, you know, it looked like with the amount of energy, it hard in the way that it was rotating on which numbers were where it looked like.

It probably wasn't gonna land on a five if I asked you on a millisecond deadline.

You know what?

What was it?

What's the probability that it was a five?

You're going to give the same number that you gave it the beginning Here, Right, One in six.

But you can look at the data.

You already have the information, the observations that you've already made.

And you can do logical reasoning about them, You can think.

Okay.

Based on the speed it was going, the angle it was turned out in which which piece is which faces will wear and so on.

It seems like, you know, maybe you can run some simulations internally.

Something like that.

Say, it seems like actually less likely than one in six.

Right?

If before I thought it was like 60.16 recurring.

Now I think it's like 0.15 because all around a 1,000,000 simulations and it seems like that the longer you think about it, yeah, more precise you could be.

Maybe you keep thinking about it again, and you get it down to like, you think it's actually 0.13 right?

Something like that.

But you don't You can't.

You don't actually know, right, Because there's still things you don't know.

Like you don't know exactly the physical properties of the paper or what, exactly?

The inside of the cup is like all the waiting of die like you still have uncertainty left because you haven't seen which way it landed.

But by doing some thinking, you're able to take some of your you're able to you, like, reduce your a logical uncertainty.

The point is that probability theory, in order to do probability theory properly, to modify your beliefs according to the evidence you observe in a way that satisfies the laws of probability, you have to be logically omniscient.

You have to be able to immediately see all of the logical consequences of everything you observe and propagate them throughout your beliefs.

Um, and this is like, not really practical in the real world.

Like in the real world, observations do not form an orderly queue and come at you one at a time and give you enough time after each one to fully integrate the consequences of each observation.

Um, because human beings are bounded, right?

We have limited processing ability and limited speed with this kind of logical uncertainty.

It feels very intuitive that we can do probabilities based on our logical uncertainty that we can.

We can think about it in this way.

It makes perfect sense to say that one in 10 is the probability.

Until you've done your thinking.

One in 10 seems like a perfectly reasonable number.

But, like why use one intent?

A good answer and, like 50% is not a good answer, because you might look at it and say, Well, this is a logical statement.

Um, it's either true or false.

That's two possibilities, you know, 50 per cent.

Why it weighs one in 10.

More sensible.

And in fact, you had to do a bit of thinking to get there, right?

You had to say, Oh, yeah, it's going to be some digit.