It's probably worth diving down and actually looking at how a CPU actually executes the code be right

I mean, we've touched on this before we did a video on pipelining we did a video on caching, but also delve down and see

What happens below the surface when we actually get our CPU to execute our code?

Let's start by having a simple example:

A line of code that we might want to look at what happens. Let's take a line of code that takes a variable

Let's take a line of code. It's gonna add up A plus B plus C plus D

Times e so I've written this in this sort of see like language

So we're gonna do this calculation now as I'm sure most of us are aware

When we take that and put into our C compiler run it it gets converted into the machine code that the CPU executes

so we take that client of code, and then we'd have to

Convert that into the machine code, and then the CPU

Executes that machine code so a program like this would end up looking and I'm going to use arm assembly here

Just because I know it better than the anything else perhaps for the first instruction. We would load the value for memory of a

Into registers, let's pick our zero. We've got 14 or so of them

We can use the 16 of them

but some of them get used for different things that we don't really use so although the value of a

Into our zero next thing we want to do is you want to add that to the value of B

Then after make sure we'll get the operator precedence right so we can load the value of B into a register

So let's loading the value of C

Here into another register

And we might as well do D. And E. As well so load or three come on D. And

We'll load our four

With E as well, and now we can start

Adding these things up multiplying them to produce

The actual result we want now we're going to make sure we get the precedence right

But we could either start by adding a and B together then add on C. And then

Multiply D. And E and have them together or we could do that one first

I'm just going to start going from left to right as long as the math is right

We'll get the right result so we'll add together a and B now

I put those two values in r0 and r1 and we need to store the results somewhere

We are going to need the value of a again after this, so we'll reuse the register R

0 so we're saying put into R 0

the value of R 0

plus R

1 so this is adding together storing the result in R. 0 so we now added a and B together

We want to add on C. And so we could do the same thing add

to R 0

The value in R. 0 which is now because of this instruction a plus B want to add on the value in R

2 there's now about a plus B plus C in

Our 0 now we need to do the multiplication

And we need to do that separately before we add it on so we get the right result so we'll multiply

And we'll see we've got an arm, too cheap here, so we've got the multiply instruction there

And we need to put the results on whether it's use our 5 D. Which we put in R. 3 and E

Which we put in R?

4 and then we want to add the result of that onto the value

In our 0 and now our 0 contains the result of a plus B. Plus C plus D times E. And

We could then store that

back into X

So that line of code there at one line of C code would become what 1 2 3 4 5 6 7 8 9 10

different lines an assembler and I've numbered them because I'm going to

Refer to them at different times so we can say searching one instruction 5 etc to refer to the different ones now

We might expect that our CPU will just xu instruction 1 the new instruction 2 instruction 3 instruction 405 and so on in order

To generate the result and some cpus do in fact work exactly like that, but actually if you think about

What the cpus and what these descriptions are actually doing you might think well actually?

when I get this first one I've got to go an access memory and

As we talked about in the caching video many years ago, cache is perhaps a an old-fashioned English word

but it basically just means a small place where we can store things so you might use it to store your hidden treasure if you're

a pirate or to store

Your food for winter on a modern CPU probably say around 200

Nanoseconds to actually go and get the value out of your main memory and load it into the register now of course

If these are already cached in the same bit of memory, then you may find that these all execute very quickly

We don't know that this isn't the only way we could write this program because if we take this instruction here instruction 6

Where we do the add of r0 and r1 to add up a and B. Well. We've got those two values here

They're already in the registers at this point in the program

So there's nothing to stop us moving this instruction up there

and it would still have exactly the same effect so instruction 6 could be moved to me between instructions 2 & 3

And then we do the next instruction which was the same as instruction 3 here?

which would be LDR

R to come of the values in memory that's representing the letter the variable see how exactly the same effect. We just moved that

Instruction earlier so you could rewrite this program in various different ways now

Why is that interesting?

well when we think about how a CPU is designed and that you will have

various different what impress be termed execution units within there now one of them is what's generally referred to as the

ALU or the arithmetic and logic unit and that's the bit of your CPU that does

Addition it does subtraction it does sort of logical operators and or and so on

But you also have other bits inside there

And one of the bits you'll often have in a modern CPU is it part of your CPU that handles loading and storing

Values from memory sometimes interact sometimes they don't now

Assuming that they are separate parts of the CPU if we look back at our instructions here. We execute instruction 1

It uses a load store. You need to get a value for memory we execute instruction 2

It uses the load store unit to get a value for memory instruction 3

It uses a load store unit to get a value for memory for uses the load store unit to get a value for memory

5 uses the load store unit to get a value for a memory 6

changes and uses the ALU as 2 7 8 & 9

before insertion turn uses the load store unit so we've got a pretty sequential series the first 5

instructions all execute using the load store part of the CPU the next four instructions execute using the ALU and

The final instruction again uses the load store unit but as we said we can reorder that

into this version here using instructions w x y and z

Differentiate them and we execute the first instruction instruction w uses a load store unit instruction X

Uses a load store unit instruction Y uses the ALU restrictions ed uses the load store unit

Okay, what difference does that make well let's think about what's happening when we're using the load store unit

the ALU isn't being used that part of the CV is just sitting there not being used and

When we're using the ALU the load store units sitting there not being used, that's what we saw there

But does that have to be the case could we actually design it, and you probably guess the answer is that yes?

We can so that

While the load store unit say is being used that we can run the instructions on the ALU part as well

I'd turn the paper round and I'm going to draw

This as a sort of timeline so these are our two units and we've got time running along this side as well

I'm using the computer for our paper in a

Radically different orientation, but never mind, so we're going to execute the instructions

On here and the first thing that happens is that we execute instruction W

No problem

That's going to take certain amount of time to actually that's using the load store unit to execute it

These are being fetched and decoded and sort of executed by the different execution units we then execute the next instruction

which is

X and we couldn't execute this any earlier because the load store unit was being used to execute that one so no difference than what?

We had before we're using this one after the other we now come to

execute

The add instruction now we can't execute this any earlier than this point in time

Because this depends on the value of registers r0 and r1 which aren't set?

until this point so we need those two values so we can start doing

instruction why here now actually

It's an ADD it's not going to take as long as fetching things from memory because it's all inside the CPU so we can use

A smaller box and we can put instruction Y there and this depends on the value being fetched from there

And I'm just going to show this as an arrow here, but the next instruction load

r2 comma C

well

I doesn't depend on anything except the value in there Marie and our load/store units not being used

So if we build our CPU right? There's nothing to start that

Instruction being executed at the same time and that means that actually when we come to the next instruction

Which would be which will be the best instruction to execute next in this example. Let's go back to our program

We've executed instructions one to six and three already

That's w x y&z we've rewritten the mass let's put instruction seven here

What was instruction seven and this is now going to become?

I'm gonna have to use it's gonna become instruction a I'll hopefully remember to say instruction a but

You can guess the colonics are referring to a on its own is probably the variable if not is probably the instruction so we can

now execute

instruction a and again instruction a depends on two things

It depends on the value of R. 0 which is going to come from this instruction so we have to have that ready

But it also depends on the value of R

2 which is coming from this instruction so we have to have that ready as well so it can actually happen any point before

This point in time so this would be the LDR R 2 comma dot and this is the add R

0 and this is the next add, but again we can start trying to leave more the instructions because I okay well

That's what instruction for here at the same time. We'll call this instruction B

And so on we put that at that point we can execute

Instruction B at the same time as we do way and I'm really confusing myself with pens here and so again

We've saved some time because rather than having to execute that in the same thing we can do these two things at the same time

Now to be able to do this we need these instructions need to execute on different execution units we couldn't for example

Execute to add instructions at the same time because we haven't got to Al use well, though

There's no reason why you can build a CPU with two Al use if you look at modern

CPU designs from Intel AMD arm and cetera they all have often have multiple Al used or allow you to do just that

but because the different types we can execute them at the same time and the reason we can do that is because

They don't depend

on the results of one to work out the other so they're working on different things and they're using different parts of the CPU and

The CPU that enables you to do this is what's known as a superscalar

CPU because it can run multiple instructions at the same time will you continue doing this and we'd end up we execute instruction

B then we've got to execute instruction C

instruction D

uses a

Multiply and actually on a CPU probably got a separate execution unit which does and multiplies because you can actually do them faster that way

So you have a multiply unit as well so we can execute that multiply D up there

We think well okay?

Can we do the other at the same time well no because we need the result of that as well so we can then execute

the ad down here before finally, and it just fits on the paper like that so we can actually squash things up and

we're going to save some time because if you think about it you have the original order of the program and

Here's one. I made earlier

All right, or as in I'm just about to draw and Shawn will do some very clever

Cutting so even if we had a superscalar processor. We've only got one load store unit we've only got one

Al you really got one multiply unit we wouldn't have any opportunities with this program

To run two instructions at the same time so this version of the program would

Still take ten instructions this one still takes ten instructions, but with a superscalar processor we have the opportunity to sort of

execute two instructions at the same time because they use different bits of the CPU now you need to design the CPU to allow that

but that enables us to

Speed things up a little bit because while this is working to get the value for memory. We can execute some more

instructions

Now that's all very well and superscalar processors started to appear in

the mid 90s things like the six eight thousand and sixty the

Pentium I think was superscalar

But they require the code to be written in a way

That enables this to happen so this program wouldn't have been able to do anything

This one would but as we said when we were developing this we could work out which

Instructions we could move around to get that speed up based on

What those instructions depended on so this instruction?

We said what what six became why only depended on the values of R

0 and R 1 which has been set by instructions 1 & 2 so we can move that earlier

Without affecting anything in our program because it only depended on those 2 values

so we can either do this in the compiler or by hand if you write in the assembly yourself like we just did here or

It's possible to let the CPU work it out, and so what a modern CPU does what's called an out of water CPU is

Reorders the instructions without

supposedly breaking the rules of

What each instruction does so it'll still execute it as if it was written like this?

And it won't change break any of the rules of that

but it will say well hang on it will spot that this instruction could happen earlier and

So move it earlier to gain some of that parallelism in fact then execute them together at the same time

And that works generally get well

But as we saw with things like Spector and meltdown if you allow things to happen too far earlier and start doing what's called speculative

Evaluation where you say okay?

I've got the stuff. I need to execute it now

I don't if I need the result but I might do so I'll execute it anyway, and then if I need it

I've already done it and if I don't need it while I was still waiting for this to come in anyway

So it doesn't matter that I've done it. I've not lost