Placeholder Image

Subtitles section Play video

  • [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY

  • JOHANN SEBASTIAN BACH]

  • PROFESSOR: Well, there's one bit of mystery left, which I'd

  • like to get rid of right now.

  • And that's that we've been blithely doing things like

  • cons assuming there's always another one.

  • That we've been doing these things like car-ing and

  • cdr-ing and assuming that we had some idea

  • how this can be done.

  • Now indeed we said that that's equivalent to having

  • procedures.

  • But that doesn't really solve the problem, because the

  • procedure need all sorts of complicated mechanisms like

  • environment structures and things like that to work.

  • And those were ultimately made out of conses in the model

  • that we had, so that really doesn't solve the problem.

  • Now the problem here is the glue the data

  • structure's made out of.

  • What kind of possible thing could it be?

  • We've been showing you things like a machine, a computer

  • that has a controller, and some

  • registers, and maybe a stack.

  • And we haven't said anything about, for

  • example, larger memory.

  • And I think that's what we have to worry about right now.

  • But just to make it perfectly clear that this is an

  • inessential, purely implementational thing, I'd

  • like to show you, for example, how you can do it

  • all with the numbers.

  • That's an easy one.

  • Famous fellow by the name of Godel, a logician at the end

  • of the 1930s, invented a very clever way of encoding the

  • complicated expressions as numbers.

  • For example--

  • I'm not saying exactly what Godel's scheme is, because he

  • didn't use words like cons.

  • He had other kinds of ways of combining to make expressions.

  • But he said, I'm going to assign a number to every

  • algebraic expression.

  • And the way I'm going to manufacture these numbers is

  • by combining the numbers of the parts.

  • So for example, what we were doing our world, we could say

  • that if objects are represented by numbers, then

  • cons of x and y could be represented by 2 to the x

  • times 2 to the y.

  • Because then we could extract the parts.

  • We could say, for example, that then car of, say, x is

  • the number of factors of 2 in x.

  • And of course cdr is the same thing.

  • It's the number of factors of 3 in x.

  • Now this is a perfectly reasonable scheme, except for

  • the fact that the numbers rapidly get to be much larger

  • in number of digits than the number of

  • protons in the universe.

  • So there's no easy way to use this scheme other than the

  • theoretical one.

  • On the other hand, there are other ways of representing

  • these things.

  • We have been thinking in terms of little boxes.

  • We've been thinking about our cons structures as looking

  • sort of like this.

  • They're little pigeon holes with things in them.

  • And of course we arrange them in little trees.

  • I wish that the semiconductor manufacturers would supply me

  • with something appropriate for this, but actually what they

  • do supply me with is a linear memory.

  • Memory is sort of a big pile of pigeonholes,

  • pigeonholes like this.

  • Each of which can hold a certain sized object, a fixed

  • size object.

  • So, for example, a complicated list with 25 elements won't

  • fit in one of these.

  • However, each of these is indexed by an address.

  • So the address might be zero here, one here, two here,

  • three here, and so on.

  • That we write these down as numbers is unimportant.

  • What matters is that they're distinct as a way to get to

  • the next one.

  • And inside of each of these, we can stuff something into

  • these pigeonholes.

  • That's what memory is like, for those of you who haven't

  • built a computer.

  • Now the problem is how are we going to impose on this type

  • of structure, this nice tree structure.

  • Well it's not very hard, and there have been numerous

  • schemes involved in this.

  • The most important one is to say, well assuming that the

  • semiconductor manufacturer allows me to arrange my memory

  • so that one of these pigeonholes is big enough to

  • hold the address of another I haven't made.

  • Now it actually has to be a little bit bigger because I

  • have to also install or store some information as to a tag

  • which describes the kind of thing that's there.

  • And we'll see that in a second.

  • And of course if the semiconductor manufacturer

  • doesn't arrange it so I can do that, then of course I can,

  • with some cleverness, arrange combinations of these to fit

  • together in that way.

  • So we're going to have to imagine imposing this

  • complicated tree structure on our nice linear memory.

  • If we look at the first still store, we see a classic scheme

  • for doing that.

  • It's a standard way of representing Lisp structures

  • in a linear memory.

  • What we do is we divide this memory into two parts.

  • An array called the cars, and an array called the cdrs.

  • Now whether those happen to be sequential addresses or

  • whatever, it's not important.

  • That's somebody's implementation details.

  • But there are two arrays here.

  • Linear arrays indexed by sequential indices like this.

  • What is stored in each of these

  • pigeonholes is a typed object.

  • And what we have here are types which begin with letters

  • like p, standing for a pair.

  • Or n, standing for a number.

  • Or e, standing for an empty list. The end of the list. And

  • so if we wish to represent an object like this, the list

  • beginning with 1, 2 and then having a 3 and a 4 as its

  • second and third elements.

  • A list containing a list as its first part and then two

  • numbers as a second and third parts.

  • Then of course we draw it sort of like this these days, in

  • box-and-pointer notation.

  • And you see, these are the three cells that have as their

  • car pointer the object which is either 1, 2 or 3 or 4.

  • And then of course the 1, 2, the car of this entire

  • structure, is itself a substructure which contains a

  • sublist like that.

  • What I'm about to do is put down places which are--

  • I'm going to assign indices.

  • Like this 1, over here, represents the

  • index of this cell.

  • But that pointer that we see here is a reference to the

  • pair of pigeonholes in the cars and the cdrs that are

  • labeled by 1 in my linear memory down here.

  • So if I wish to impose this structure on my linear memory,

  • what I do is I say, oh yes, why don't we drop

  • this into cell 1?

  • I pick one.

  • There's 1.

  • And that says that its car, I'm going to

  • assign it to be a pair.

  • It's a pair, which is in index 5.

  • And the cdr, which is this one over here, is a pair which I'm

  • going to stick into place 2.

  • p2.

  • And take a look at p2.

  • Oh yes, well p2 is a thing whose car is the number 3, so

  • as you see, an n3.

  • And whose cdr, over here, is a pair, which lives in place 4.

  • So that's what this p4 is.

  • p4 is a number whose value is 4 in its car and whose cdr is

  • an empty list right there.

  • And that ends it.

  • So this is the traditional way of representing this kind of

  • binary tree in a linear memory.

  • Now the next question, of course, that we might want to

  • worry about is just a little bit of implementation.

  • That means that when I write procedures of the form

  • assigned a, [UNINTELLIGIBLE] procedures--

  • lines of register machine code of the form assigned a, the

  • car of [UNINTELLIGIBLE]

  • b, what I really mean is addressing these elements.

  • And so we're going to think of that as a abbreviation for it.

  • Now of course in order to write that down I'm going to

  • introduce some sort of a structure called a vector.

  • And we're going to have something which will reference

  • a vector, just so we can write it down.

  • Which takes the name of the vector, or the--

  • I don't think that name is the right word.

  • Which takes the vector and the index, and I have to have a

  • way of setting one of those with something called a vector

  • set, I don't really care.

  • But let's look, for example, at then that kind of

  • implementation of car and cdr.

  • So for example if I happen to have a register b, which

  • contains the type index of a pair, and therefore it is the

  • pointer to a pair, then I could take the car of that and

  • if I-- write this down--

  • I might put that in register a.

  • What that really is is a representation of the assign

  • to a, the value of vector reffing--

  • or array indexing, if you will-- or

  • something, the cars object--

  • whatever that is--

  • with the index, b.

  • And similarly for cdr. And we can do the same thing for

  • assignment to data structures, if we need to do that sort of

  • thing at all.

  • It's not too hard to build that.

  • Well now the next question is how are we going to do

  • allocation.

  • And every so often I say I want a cons.

  • Now conses don't grow on trees.

  • Or maybe they should.

  • But I have to have some way of getting the next one.

  • I have to have some idea of if their memory is unused that I

  • might want to allocate from.

  • And there are many schemes for doing this.

  • And the particular thing I'm showing you

  • right now is not essential.

  • However it's convenient and has been done many times.

  • One scheme's was called the free list allocation scheme.

  • What that means is that all of the free memory that there is

  • in the world is linked together in a linked list,

  • just like all the other stuff.

  • And whenever you need a free cell to make a new cons, you

  • grab the first, one make the free list be the cdr of it,

  • and then allocate that.

  • And so what that looks like is something like this.

  • Here we have the free list starting in 6.

  • And what that is is a pointer-off to say 8.

  • So what it says is, this one is free and the

  • next one is an 8.

  • This one is free and the next one is in 3, the next one

  • that's free.

  • That one's free and the next one is in 0.

  • That one's free and the next one's in 15.

  • Something like that.

  • We can imagine having such a structure.

  • Given that we have something like that, then it's possible

  • to just get one when you need it.

  • And so a program for doing cons, this is what

  • cons might turn into.

  • To assign to a register A the result of cons-ing, a B onto

  • C, the value in this containing B and the value

  • containing C, what we have to do is get the current

  • [? type ?] ahead of the freelist, make the free list

  • be its cdr. Then we have to change the cars to be the

  • thing we're making up to be in A to be the B, the thing in B.

  • And we have to make change the cdrs of the thing that's in A

  • to be C. And then what we have in A is the right new frob,

  • whatever it is.

  • The object that we want.

  • Now there's a little bit of a cheat here that I haven't told

  • you about, which is somewhere around here I haven't set that

  • I've the type of the thing that I'm cons-ing up to be a

  • pair, and I ought to.

  • So there should be some sort of bits here are being set,

  • and I just haven't written that down.

  • We could have arranged it, of course, for the free lift to

  • be made out of pairs.

  • And so then there's no problem with that.

  • But that sort of-- again, an inessential detail in a way

  • some particular programmer or architect or whatever might

  • manufacture his machine or Lisp system.

  • So for example, just looking at this, to allocate given

  • that I had already the structure that you saw before,

  • supposing I wanted to allocate a new cell, which is going to

  • be representation of list one, one, two, where already one

  • two was the car of the list we were playing with before.

  • Well that's not so hard.

  • I stored that one and one, so p1 one is the

  • representation of this.

  • This is p5.

  • That's going to be the cdr of this.

  • Now we're going to pull something off the free list,

  • but remember the free list started at six.

  • The new free list after this allocation is eight, a free

  • list beginning at eight.

  • And of course in six now we have a number one, which is

  • what we wanted, with its cdr being the pair starting in

  • location five.

  • And that's no big deal.

  • So the only problem really remaining here is, well, I

  • don't have an infinitely large memory.

  • If I do this for a little while, say, for example,

  • supposing it takes me a microsecond to do a cons, and

  • I have a million cons memory then I'm only going to run out

  • in a second, and that's pretty bad.

  • So what we do to prevent that disaster, that ecological

  • disaster, talk about right after questions.

  • Are there any questions?

  • Yes.

  • AUDIENCE: In the environment diagrams that we were drawing

  • we would use the body of procedures, and you would

  • eventually wind up with things that were no longer useful in

  • that structure.

  • How is that represented?

  • PROFESSOR: There's two problems here.

  • One you were asking is that material becomes useless.

  • We'll talk about that in a second.

  • That has to do with how to prevent ecological disasters.

  • If I make a lot of garbage I have to somehow be able to

  • clean up after myself.

  • And we'll talk about that in a second.

  • The other question you're asking is how you represent

  • the environments, I think.

  • AUDIENCE: Yes.

  • PROFESSOR: OK.

  • And the environment structures can be represented in

  • arbitrary ways.

  • There are lots of them.

  • I mean, here I'm just telling you about list cells.

  • Of course every real system has vectors of arbitrary

  • length as well as the vectors of length, too, which

  • represent list cells.

  • And the environment structures that one uses in a

  • professionally written Lisp system tend to be vectors

  • which contain a number of elements approximately equal

  • to the number of arguments-- a little bit more because you

  • need certain glue.

  • So remember, the environment [UNINTELLIGIBLE]

  • frames.

  • The frames are constructed by applying a procedure.

  • In doing so, an allocation is made of a place which is the

  • number of arguments long plus [? unglue ?]

  • that gets linked into a chain.

  • It's just like algol at that level.

  • There any other questions?

  • OK.

  • Thank you, and let's take a short break.

  • [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY

  • JOHANN SEBASTIAN BACH]

  • PROFESSOR: Well, as I just said, computer memories

  • supplied by the semiconductor manufacturers are finite.

  • And that's quite a pity.

  • It might not always be that way.

  • Just for a quick calculation, you can see that it's possible

  • that if [? memory ?]

  • prices keep going at the rate they're going that if you

  • still took a microsecond second to do a cons, then--

  • first of all, everybody should know that there's about pi

  • times ten to the seventh seconds in a year.

  • And so that would be ten to the seventh plus ten to the

  • sixth is ten to the thirteenth.

  • So there's maybe ten to the fourteenth conses in the life

  • of a machine.

  • If there was ten to the fourteenth words of memory on

  • your machine, you'd never run out.

  • And that's not completely unreasonable.

  • Ten to the fourteenth is not a very large number.

  • I don't think it is.

  • But then again I like to play with astronomy.

  • It's at least ten to the eighteenth centimeters between

  • us and the nearest star.

  • But the thing I'm about to worry about is, at least in

  • the current economic state of affairs, ten to the fourteenth

  • pieces of memory is expensive.

  • And so I suppose what we have to do is make

  • do with much smaller.

  • Memories

  • Now in general we want to have an illusion of infinity.

  • All we need to do is arrange it so that whenever you look,

  • the thing is there.

  • That's really an important idea.

  • A person or a computer lives only a finite amount of time

  • and can only take a finite number of looks at something.

  • And so you really only need a finite amount of stuff.

  • But you have to arrange it so no matter how much there is,

  • how much you really claim there is, there's always

  • enough stuff so that when you take a look, it's there.

  • And so you only need a finite amount.

  • But let's see.

  • One problem is, as was brought up, that there are possible

  • ways that there is lots of stuff that we make that we

  • don't need.

  • And we could recycle the material out

  • of which its made.

  • An example is the fact that we're building environment

  • structures, and we do so every time we call a procedure.

  • We have built in it a environment frame.

  • That environment frame doesn't necessarily

  • have a very long lifetime.

  • Its lifetime, meaning its usefulness, may exist only

  • over the invocation of the procedure.

  • Or if the procedure exports another procedure by returning

  • it as a value and that procedure is defined inside of

  • it, well then the lifetime of the frame of the outer

  • procedure still is only the lifetime of the procedure

  • which was exported.

  • And so ultimately, a lot of that is garbage.

  • There are other ways of producing garbage as well.

  • Users produce garbage.

  • An example of user garbage is something like this.

  • If we write a program to, for example, append two lists

  • together, well one way to do it is to reverse the first

  • list onto the empty list and reverse that onto the second

  • list. Now that's not terribly bad way of doing it.

  • And however, the intermediate result, which is the reversal

  • of the first list as done by this program, is never going

  • to be accessed ever again after it's copied back on to

  • the second.

  • It's an intermediate result.

  • It's going to be hard to ever see how anybody would ever be

  • able to access it.

  • In fact, it will go away.

  • Now if we make a lot of garbage like that, and we

  • should be allowed to, then there's got to be some way to

  • reclaim that garbage.

  • Well, what I'd like to tell you about now is a very clever

  • technique whereby a Lisp system can prove a small

  • theorem every so often on the [? forum, ?] the following

  • piece of junk will never be accessed again.

  • It can have no affect on the future of the computation.

  • It's actually based on a very simple idea.

  • We've designed our computers to look sort of like this.

  • There's some data path, which contains the registers.

  • There are things like x, and env, and val, and so on.

  • And there's one here called stack, some sort which points

  • off to a structure somewhere, which is the stack.

  • And we'll worry about that in a second.

  • There's some finite controller, finite state

  • machine controller.

  • And there's some control signals that go this way and

  • predicate results that come this way, not

  • the interesting part.

  • There's some sort of structured memory, which I

  • just told you how to make, which may contain a stack.

  • I didn't tell you how to make things of arbitrary shape,

  • only pairs.

  • But in fact with what I've told you can simulate a stack

  • by a big list. I don't plan to do that, it's not a

  • nice way to do it.

  • But we could have something like that.

  • We have all sorts of little data structures in here that

  • are hooked together in funny ways.

  • They connect to other things.

  • And so on.

  • And ultimately things up there are pointers to these.

  • The things that are in the registers are pointers off to

  • the data structures that live in this Lisp structure memory.

  • Now the truth of the matter is that the entire consciousness

  • of this machine is in these registers.

  • There is no possible way that the machine, if done

  • correctly, if built correctly, can access anything in this

  • Lisp structure memory unless the thing in that Lisp

  • structure memory is connected by a sequence of data

  • structures to the registers.

  • If it's accessible by legitimate data structure

  • selectors from the pointers that are

  • stored in these registers.

  • Things like array references, perhaps.

  • Or cons cell references, cars and cdrs.

  • But I can't just talk about a random place in this memory,

  • because I can't get to it.

  • These are being arbitrary names I'm not allowed to

  • count, at least as I'm evaluating expressions.

  • If that's the case then there's a very simple theorem

  • to be proved.

  • Which is, if I start with all lead pointers that are in all

  • these registers and recursively chase out, marking

  • all the places I can get to by selectors, then eventually I

  • mark everything they can be gotten to.

  • Anything which is not so marked is

  • garbage and can be recycled.

  • Very simple.

  • Cannot affect the future of the computation.

  • So let me show you that in a particular example.

  • Now that means I'm going to have to append to my

  • description of the list structure a mark.

  • And so here, for example, is a Lisp structured memory.

  • And in this Lisp structured memory is a Lisp structure

  • beginning in a place I'm going to call--

  • this is the root.

  • Now it doesn't really have to have a root.

  • It could be a bunch of them, like all the registers.

  • But I could cleverly arrange it so all the registers, all

  • the things that are in old registers are also at the

  • right moment put into this root structure, and then we've

  • got one pointer to it.

  • I don't really care.

  • So the idea is we're going to cons up stuff until our free

  • list is empty.

  • We've run out of things.

  • Now we're going to do this process of proving the theorem

  • that a certain percentage of the memory has got crap in it.

  • And then we're going to recycle that to grow new

  • trees, a standard use of such garbage.

  • So in any case, what do we have here?

  • Well we have some data structure which starts out

  • over here one.

  • And in fact it has a car in five, and its cdr is in two.

  • And all the marks start out at zero.

  • Well let's start marking, just to play this game.

  • OK.

  • So for example, since I can access one from the root I

  • will mark that.

  • Let me mark it.

  • Bang.

  • That's marked.

  • Now since I have a five here I can go to five and see, well

  • I'll mark that.

  • Bang.

  • That's useful stuff.

  • But five references as a number in its car, I'm not

  • interested in marking numbers but its cdr is seven.

  • So I can mark that.

  • Bang.

  • Seven is the empty list, the only thing that references,

  • and it's got a number in its car.

  • Not interesting.

  • Well now let's go back here.

  • I forgot about something.

  • Two.

  • See in other words, if I'm looking at cell one, cell one

  • contains a two right over here.

  • A reference to two.

  • That means I should go mark two.

  • Bang.

  • Two contains a reference to four.

  • It's got a number in its car, I'm not interested in that, so

  • I'm going to go mark that.

  • Four refers to seven through its car, and is empty in its

  • cdr, but I've already marked that one so I don't have to

  • mark it again.

  • This is all the accessible structure from that place.

  • Simple recursive mark algorithm.

  • Now there are some unhappinesses about that

  • algorithm, and we can worry about that a second.

  • But basically you'll see that all the things that have not

  • been marked are places that are free, and I could recycle.

  • So the next stage after that is going to be to scan through

  • all of my memory, looking for things that are not marked.

  • Every time I come across a marked thing I unmark it, and

  • every time I come across an unmarked thing I'm going to

  • link it together in my free list.

  • Classic, very simple algorithm.

  • So let's see.

  • Is that very simple?

  • Yes it is.

  • I'm not going to go through the code in any detail, but I

  • just want to show you about how long it is.

  • Let's look at the mark phase.

  • Here's the first part of the mark phase.

  • We pick up the root.

  • We're going to use that as a recursive procedure call.

  • We're going to sweep from there, after when we're done

  • with marking.

  • And then we're going to do a little couple of instructions

  • that do this checking out on the marks and changing the

  • marks and things like that, according to the algorithm

  • I've just shown you.

  • It comes out here.

  • You have to mark the cars of things and you also have to be

  • able to mark the cdrs of things.

  • That's the entire mark phase.

  • I'll just tell you a little story about this.

  • The old DEC PDP-6 computer, this was the way that the

  • mark-sweep garbage collection, as it was, was written.

  • The program was so small that with the data that it needed,

  • with the registers that it needed to manipulate the

  • memory, it fit into the fast registers of the machine,

  • which were 16.

  • The whole program.

  • And you could execute

  • instructions in the fast registers.

  • So it's an extremely small program, and it could run very

  • fast.

  • Now unfortunately, of course, this program, because the fact

  • that it's recursive in the way that you do something first

  • and then you do something after that, you have to work

  • on the cars and then the cdrs, it requires auxiliary memory.

  • So Lisp systems--

  • those requires a stack for marking.

  • Lisp systems that are built this way have a limit to the

  • depth of recursion you can have in data structures in

  • either the car or the cdr, and that doesn't work very nicely.

  • On the other hand, you never notice it if it's big enough.

  • And that's certainly been the case for most Maclisp, for

  • example, which ran Macsyma where you could deal with

  • expressions of thousands of elements long.

  • These are algebraic expressions with thousand of

  • terms. And there's no problem with that.

  • Such, the garbage collector does work.

  • On the other hand, there's a very clever modification to

  • this algorithm, which I will not describe, by Peter Deutsch

  • and Schorr and Waite--

  • Herb Schorr from IBM and Waite, who I don't know.

  • That algorithm allows you to build-- you do can do this

  • without auxiliary memory, by remembering as you walk the

  • data structures where you came from by reversing the pointers

  • as you go down and crawling up the reverse

  • pointers as you go up.

  • It's a rather tricky algorithm.

  • The first time you write it-- or in fact, the first three

  • times you write it it has a terrible bug in it.

  • And it's also rather slow, because it's complicated.

  • It takes about six times as many memory references to do

  • the sorts of things that we're talking about.

  • Well now once I've done this marking phase, and I get into

  • a position where things look like this, let's look--

  • yes.

  • Here we have the mark done, just as I did it.

  • Now we have to perform the sweep phase.

  • And I described to you what this sweep is like.

  • I'm going to walk down from one end of memory or the

  • other, I don't care where, scanning every cell that's in

  • the memory.

  • And as I scan these cells, I'm going to link them together,

  • if they are free, into the free list. And if they're not

  • free, I'm going to unmark them so the marks become zero.

  • And in fact what I get-- well the program is not very

  • complicated.

  • It looks sort of like this-- it's a little longer.

  • Here's the first piece of it.

  • This one's coming down from the top of memory.

  • I don't want you to try to understand this at this point.

  • It's rather simple.

  • It's a very simple algorithm, but there's pieces of it that

  • just sort of look like this.

  • They're all sort of obvious.

  • And after we've done the sweep, we get an answer that

  • looks like that.

  • Now there are some disadvantages with mark-sweep

  • algorithms of this sort.

  • Serious ones.

  • One important disadvantage is that your memories get larger

  • and larger.

  • As you say, address spaces get larger and larger, you're

  • willing to represent more and more stuff, then it gets very

  • costly to scan all of memory.

  • What you'd really like to do is only scan useful stuff.

  • It would even be better if you realized that some stuff was

  • known to be good and useful, and you don't have to look at

  • it more than once or twice.

  • Or very rarely.

  • Whereas other stuff that you're not so sure about, you

  • can look at more detail every time you want to do this, want

  • to garbage collect.

  • Well there are algorithms that are organized in this way.

  • Let me tell you about a famous old algorithm which allows you

  • only look at the part of memory which

  • is known to be useful.

  • And which happens to be the fastest known garbage

  • collector algorithm.

  • This is the Minsky-Feinchel-Yochelson

  • garbage collector algorithm.

  • It was invented by Minsky in 1961 or '60 or something, for

  • the RLE PDP-1 Lisp, which had 4,096 words of list memory,

  • and a drum.

  • And the whole idea was to garbage collect

  • this terrible memory.

  • What Minsky realized was the easiest way to do this is to

  • scan the memory in the same sense, walking the good

  • structure, copying it out into the drum, compacted.

  • And then when we were done copying it all out, then you

  • swap that back into your memory.

  • Now whether or you not use a drum, or another piece of

  • memory, or something like that isn't important.

  • In fact, I don't think people use

  • drums anymore for anything.

  • But this algorithm basically depends upon having about

  • twice as much address space as you're actually using.

  • And so what you have is some, initially, some mixture of

  • useful data and garbage.

  • So this is called fromspace.

  • And this is a mixture of crud.

  • Some of it's important and some of it isn't.

  • Now there's another place which is hopefully big enough,

  • if we recall, tospace, which is where we're copying to.

  • And what happens is-- and I'm not going to go

  • through this detail.

  • It's in our book quite explicitly.

  • There's a root point where you start from.

  • And the idea is that you start with the root.

  • You copy the first thing you see, the first thing that the

  • root points at, to the beginning of tospace.

  • The first thing is a pair or something

  • like, a data structure.

  • You then also leave behind a broken heart saying, I moved

  • this object from here to here, giving the place

  • where it moved to.

  • This is called a broken heart because a friend of mine who

  • implemented one of these in 1966 was a very romantic

  • character and called it a broken heart.

  • But in any case, the next thing you do is now you have a

  • new free pointer which is here, and you start scanning.

  • You scan this data structure you just copied.

  • And every time you encounter a pointer in it, you treat it as

  • if it was the root pointer here.

  • Oh, I'm sorry.

  • The other thing you do is you now move the

  • root pointer to there.

  • So now you scan this, and everything you see you treat

  • as it were the root pointer.

  • So if you see something, well it points

  • up into there somewhere.

  • Is it pointing at a thing which you've not copied yet?

  • Is there a broken heart there?

  • If there's a broken heart there and it's something you

  • have copied, you've just replaced this pointer with the

  • thing a broken heart points at.

  • If this thing has not been copied, you copy it to the

  • next place over here.

  • Move your free pointer over here, and then leave a broken

  • heart behind and scan.

  • And eventually when the scant pointer hits the free pointer,

  • everything in memory has been copied.

  • And then there's a whole bunch of empty space up here, which

  • you could either make into a free list, if that's what you

  • want to do.

  • But generally you don't in this kind of system.

  • In this system you sequentially

  • allocate your memory.

  • That is a very, very nice algorithm, and sort of the one

  • we use in the scheme that you've been using.

  • And it's expected--

  • I believe no one has found a faster algorithm than that.

  • There are very simple modifications to this

  • algorithm invented by Henry Baker which allow one to run

  • this algorithm in real time, meaning you don't have to stop

  • to garbage collect.

  • But you could interleave the consing that the machine does

  • when its running with steps of the garbage collection

  • process, so that the garbage collector's distributed, and

  • the machine doesn't have to stop, and garbage

  • collecting can start.

  • Of course in the case of machines with virtual memory

  • where a lot of it is in inaccessible places, this

  • becomes a very expensive process.

  • And there have been numerous attempts to

  • make this much better.

  • There is a nice paper, for those of you who are

  • interested, by Moon and other people which describes a

  • modification to the incremental

  • Minsky-Feinchel-Yochelson algorithm, and modification

  • the Baker algorithm which is more efficient for virtual

  • memory systems.

  • Well I think now the mystery to this is sort of gone.

  • And I'd like to see if there are any questions.

  • Yes.

  • AUDIENCE: I saw one of you run the garbage collector on the

  • systems upstairs, and it seemed to me to run extremely

  • fast. Did the whole thing take--

  • does it sweep through all of memory?

  • PROFESSOR: No.

  • It swept through exactly what was needed to

  • copy the useful structure.

  • It's a copying collector.

  • And it is very fast. On the whole, I suppose to copy--

  • in a Bobcat--

  • to copy, I think, a three megabyte thing or something is

  • less than a second, real time.

  • Really, these are very small programs. One thing you should

  • realise is that garbage collectors have to be small.

  • Not because they have to be fast, but because no one can

  • debug a complicated garbage collector.

  • A garbage collector, if it doesn't work, will trash your

  • memory in such a way that you cannot figure out what the

  • hell happened.

  • You need an audit trail.

  • Because it rearranges everything, and how do you

  • know what happened there?

  • So this is the only kind of program that it really,

  • seriously matters if you stare at it long enough so you

  • believe that it works.

  • And sort of prove it to yourself.

  • So there's no way to debug it.

  • And that takes it being small enough so you can

  • hold it in your head.

  • Garbage collectors are special in this way.

  • So every reasonable garbage collector has gotten small,

  • and generally small programs are fast. Yes.

  • AUDIENCE: Can you repeat the name of this

  • technique once again?

  • PROFESSOR: That's the Minsky-Feinchel-Yochelson

  • garbage collector.

  • AUDIENCE: You got that?

  • PROFESSOR: Minsky invented it in '61 for the RLE PDP-1.

  • A version of it was developed and elaborated to be used in

  • Multics Maclisp by Feinchel and Yochelson in somewhere

  • around 1968 or '69.

  • OK.

  • Let's take a break.

  • [MUSIC: "JESU, JOY OF MAN'S DESIRING" BY

  • JOHANN SEBASTIAN BACH]

  • PROFESSOR: Well we've come to the end of this subject, and

  • we've already shown you a universal machine which is

  • down to evaluator.

  • It's down to the level of detail you could imagine you

  • could make one.

  • This is a particular implementation of Lisp, built

  • on one of those scheme chips that was talked about

  • yesterday, sitting over here.

  • This is mostly interface to somebody's memory with a

  • little bit of timing and other such stuff.

  • But this fellow actually ran Lisp at a fairly reasonable

  • rate, as interpretive.

  • It ran Lisp as fast as a DEC PDP-10 back in 1979.

  • And so it's gotten pretty hardware.

  • Pretty concrete.

  • We've also downed you a bit with the

  • things you can compute.

  • But is it the case that there are things we can't compute?

  • And so I'd like to end this with showing you some things

  • that you'd like be able to compute that you can't.

  • The answer is yes, there are things you can't compute.

  • For example, something you'd really like is--

  • if you're writing [UNINTELLIGIBLE], you'd like a

  • program that would check that the thing you're

  • going to do will work.

  • Wouldn't that be nice?

  • You'd like something that would catch infinite loops,

  • for example, in programs that were written by users.

  • But in general you can't write such a program that will read

  • any program and determine whether or not it's an

  • infinite loop.

  • Let me show you that.

  • It's a little bit of a minor mathematics.

  • Let's imagine that we just had a mathematical

  • function before we start.

  • And there is one, called s, which takes a procedure and

  • its argument, a.

  • And what s does is it determines whether or not it's

  • safe to run p on a.

  • And what I mean by that is this: it's true if p applied

  • to a will converge to a value without an error.

  • And it's false if p of a loops forever or makes an error.

  • Now that's surely a function.

  • There is some for every procedure and for every

  • argument you could give it that is either true or false

  • that it converges without making an error.

  • And you could make a giant table of them.

  • But the question is, can you write a procedure that compute

  • the values of this function?

  • Well let's assume that we can.

  • Suppose that we have a procedure called "safe" that

  • computes the value of s.

  • Now I'm going to show you by several methods that

  • you can't do this.

  • The easiest one, or the first one, let's define a procedure

  • called diag1.

  • Given that we have safe, we can define diag1 to be the

  • procedure of one argument, p, which has the following

  • properties.

  • If if it's safe to apply p to itself, then I wish to have an

  • infinite loop.

  • Otherwise I'm going to return 3.

  • Remember it was 42.

  • What's the answer to the big question?

  • Where of course we know what an infinite loop is.

  • Infinite loop, to be a procedure of no arguments,

  • which is that nice lambda calculus loop.

  • Lambda of x, x of x, applied to lambda of x, x of x.

  • So there's nothing left to the imagination here.

  • Well let's see what the story is.

  • I'm supposing it's the case that we worry about the

  • procedure called diag1 applied to diag1.

  • Well what could it possibly be?

  • Well I don't know.

  • We're going to substitute diag1 for p in the body here.

  • Well is it safe to compute diag1 of diag1?

  • I don't know.

  • There are two possibilities.

  • If it's safe to compute diag1 of diag1 that means it

  • shouldn't loop.

  • That means I go to here, but then I

  • produce an infinite loop.

  • So it can't be safe.

  • But if it's not safe to compute diag1 of diag1 then

  • the answer to this is 3.

  • But that's diag1 of diag1, so it had to be safe.

  • So therefore by contradiction you cannot produce safe.

  • For those of you who were boggled by that one I'm going

  • to say it again, in a different way.

  • Listen to one more alternative.

  • Let's define diag2.

  • These are named diag because of Cantor's diagonal argument.

  • These are instances of a famous argument which was

  • originally used by Cantor in the late part of the last

  • century to prove that the real numbers were not countable,

  • that there are too many real numbers to

  • be counted by integers.

  • That there are more points on a line, for example, than

  • there are counting numbers.

  • It may or may not be obvious, and I don't want to

  • get into that now.

  • But diag2 is again a procedure of one argument p.

  • It's almost the same as the previous one, which is, if

  • it's safe to compute p on p, then I'm going to produce--

  • then I want to compute some other things

  • other than p of p.

  • Otherwise I'm going to put out false.

  • Where other then it says, whatever p of p, I'm going to

  • put out something else.

  • I can give you an example of a definition of other than which

  • I think works.

  • Let's see.

  • Yes.

  • Where other than be a procedure of one argument x

  • which says, if its eq x to, say, quote a, then the answer

  • is quote b.

  • Otherwise it's quote a.

  • That always produces something which is not what

  • its argument is.

  • That's all it is.

  • That's all I wanted.

  • Well now let's consider this one, diag2 of diag2.

  • Well look.

  • This only does something dangerous, like calling p of

  • p, if it's safe to do so.

  • So if safe defined at all, if you can define such a

  • procedure, safe, then this procedure is always defined

  • and therefore safe on any inputs.

  • So diag2 of diag2 must reduce to other than diag2 of diag2.

  • And that doesn't make sense, so we have a contradiction,

  • and therefore we can't define safe.

  • I just waned to do that twice, slightly differently, so you

  • wouldn't feel that the first one was a trick.

  • They may be both tricks, but they're at

  • least slightly different.

  • So I suppose that pretty much wraps it up.

  • I've just proved what we call the halting theorem, and I

  • suppose with that we're going to halt.

  • I hope you have a good time.

  • Are there any questions?

  • Yes.

  • AUDIENCE: What is the value of s of diag1?

  • PROFESSOR: Of what?

  • AUDIENCE: S of diag1.

  • If you said s is a function and we can

  • [INTERPOSING VOICES]

  • PROFESSOR: Oh, I don't know.

  • I don't know.

  • It's a function, but I don't know how to compute it.

  • I can't do it.

  • I'm just a machine, too.

  • Right?

  • There's no machine that in principle--

  • it might be that in that particular case you just

  • asked, with some thinking I could figure it out.

  • But in general I can't compute the value of s any better than

  • any other machine can.

  • There is such a function, it's just that no machine can be

  • built to compute it.

  • Now there's a way of saying that that should not be

  • surprising.

  • Going through this--

  • I mean, I don't have time to do this here, but the number

  • of functions is very large.

  • If there's a certain number of answers possible and a certain

  • number of inputs possible, then it's the number of

  • answers raised to the number inputs is the number of

  • possible functions.

  • On one variable.

  • Now that's always bigger than the thing you're raising to,

  • the exponent.

  • The number of functions is larger than the number of

  • programs that one can write, by an

  • infinity counting argument.

  • And it's much larger.

  • So there must be a lot of functions that can't be

  • computed by programs.

  • AUDIENCE: A few moments ago you were talking about

  • specifications and automatic generation of solutions.

  • Do you see any steps between specifications and solutions?

  • PROFESSOR: Steps between.

  • You mean, you're saying, how you go about constructing

  • devices given that have specifications for the device?

  • Sure.

  • AUDIENCE: There's a lot of software engineering that goes

  • through specifications through many layers of design and then

  • implementation.

  • PROFESSOR: Yes?

  • AUDIENCE: I was curious if you think that's realistic.

  • PROFESSOR: Well I think that some of it's realistic and

  • some of it isn't.

  • I mean, surely if I want to build an electrical filter and

  • I have a rather interesting possibility.

  • Supposing I want to build a thing that matches some power

  • output to the radio transmitter, to some antenna.

  • And I'm really out of this power--

  • it's output tube out here.

  • And the problem is that they have different impedances.

  • I want them to match the impedances.

  • I also want to make a filter in there which is going to get

  • rid of some harmonic radiation.

  • Well one old-fashioned technique for doing this is

  • called image impedances, or something like that.

  • And what you do is you say you have a basic

  • module called an L-section.

  • Looks like this.

  • If I happen to connect this to some resistance, r, and if I

  • make this impedance x, xl, and if it happens to be q times r,

  • then this produces a low pass filter with a q square plus

  • one impedance match.

  • Just what I need.

  • Because now I can take two of these, hook them

  • together like this.

  • OK, and I take another one and I'll hook them

  • together like that.

  • And I have two L-sections hooked together.

  • And this will step the impedance down to one that I

  • know, and this will step it up to one I know.

  • Each of these is a low pass filter

  • getting rid of some harmonics.

  • It's good filter, it's called a pie-section filter.

  • Great.

  • Except for the fact that in doing what I just did, I've

  • made a terrible inefficiency in this system.

  • I've made two coils where I should have made one.

  • And the problem with most software engineering art is

  • that there's no mechanism, other than peephole

  • optimization and compilers, for getting rid of the

  • redundant parts that are constructed when doing top

  • down design.

  • It's even worse, there are lots of very important

  • structures that you can't construct at all this way.

  • So I think that the standard top down design is a rather

  • shallow business.

  • Doesn't really capture what people want to do in design.

  • I'll give you another electrical example.

  • Electrical examples are so much clearer than

  • computational examples, because computation examples

  • require a certain degree of complexity to explain them.

  • But one of my favorite examples in the electrical

  • world is how would I ever come up with the output stage of

  • this inter-stage connection in an IF amplifier.

  • It's a little transistor here, and let's see.

  • Well I'm going to have a tank, and I'm going to hook this up

  • to, say, I'm going to link-couple that to the input

  • of the next stage.

  • Here's a perfectly plausible plan--

  • well except for the fact that since I put that going up I

  • should make that going that way.

  • Here's a perfectly plausible plan for a--

  • no I shouldn't.

  • I'm dumb.

  • Excuse me.

  • Doesn't matter.

  • The point is [UNINTELLIGIBLE]

  • plan for a couple [UNINTELLIGIBLE]

  • stages together.

  • Now what the problem is is what's this hierarchically?

  • It's not one thing.

  • Hierarchically it doesn't make any sense at all.

  • It's the inductance of a tuned circuit, it's the primary of a

  • transformer, and it's also the DC path by which bias

  • conditions get to the collector of that transistor.

  • And there's no simple top-down design that's going to produce

  • a structure like that with so many overlapping uses for a

  • particular thing.

  • Playing Scrabble, where you have to do triple word scores,

  • or whatever, is not so easy in top-down design strategy.

  • Yet most of real engineering is based on getting the most

  • oomph for effort.

  • And that's what you're seeing here.

  • Yeah?

  • AUDIENCE: Is this the last question?

  • [LAUGHTER]

  • PROFESSOR: Apparently so.

  • Thank you.

  • [APPLAUSE]

  • [MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY

  • JOHANN SEBASTIAN BACH]

[MUSIC-- "JESU, JOY OF MAN'S DESIRING" BY

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it