Subtitles section Play video Print subtitles Mind reading? Of course not. I love reading. Look, mind reading might sound like pseudoscientific-- pardon my language-- bullshoot. But its scientific counterpart, thought identification, is very much a real thing. It's based in neuroimaging and machine learning, and what's really cool is that experiments in mind reading aren't just about spying on what someone is thinking. They're about figuring out what thoughts are even made of. I mean, when I think of something, what does that mental picture actually look like? What resolution is it in? How high fidelity is a memory, and how do they change over time? Well, in this episode, I'm going to look at how reading people's minds can help us answer these questions. My journey begins right here at the University of Oregon. I'm meeting with Dr. Brice Kuhl from the Kuhl lab. He's a neuroscientist who uses neuroimaging and machine learning to figure out what people are thinking without them telling him. So tell me what you're doing here. Well, I'm in the cognitive neuroscience program here, and I study human memory. My lab primarily uses neuroimaging methods, so we do a lot of work using functional magnetic resonance imaging, or fMRI. And how do you use fMRI to investigate memories? We're looking at the pattern of neural activity. When you form a memory, there's a certain pattern. And we can record that pattern and then test whether that pattern is reinstated or reactivated at a later point, like when you're remembering it. Does that mean we can look at the patterns of brain activity and deduce what it is that is being remembered, or recalled, or even just thought? Yes, and so we call that decoding. So it basically takes your input pattern as some pattern of activity that we record while you're remembering something. And we make a prediction about what you're remembering. You can see how this sounds like mind reading. [laughs] Yes. It sounds like that. So, Brice, what are you going to do to me today? So, what we're going to be doing today is uncharted territory for us. So we're going to be trying out a kind of new variant of the experiment on you. So I can't guarantee any particular results. But it represents where the field is and where we're trying to go. Today, you're going to participate in an experiment where you'll be studying faces. So we're going to have you study 12 pictures of celebrities. People I already am familiar with. -People that you know, yeah. -Okay. And you're going to try to remember those pictures. Then we're going to have you go into the MRI scanner. Try to bring that picture to mind as vividly as possible. And we're going to be recording your brain activity as you try to imagine these pictures. We're going to try to build the face. Essentially draw a picture of what you're remembering. -A picture? -A picture. An actual picture that we can print out and I could, like, hang on my wall. [laughs] If you wanted. [Michael] The first step is for me to memorize the 12 specific celebrity photographs Brice will later try to detect me thinking about. I sat down to do this graduate student, Max. The success of his predictions depend, in part, on my ability to recall these faces as vividly as possible while inside the fMRI. All right, so... [sighs] I think I have a pretty good memory of all of those. -Great. -I feel the stakes are high. With the celebrity faces hopefully memorized, it's time for the next step: going through the metal detector and into the fMRI, where Brice will record and monitor my brain activity, and then later feed it into his algorithm to rebuild the faces. This will be the first time he's attempted to reconstruct faces from long-term memory, which is very difficult, because we're relying on how clearly I can remember the celebrity photos I saw an hour ago. I love its eyes. Look at that. [woman] Wouldn't the kid be like, "It's going to eat me"? An fMRI monitors the activity within the brain by dividing it up into thousands of small cubes called voxels, or volumetric pixels. Each of these voxels contains hundreds of thousands of neurons. Using fMRI, we are able to detect blood flow within these voxels, which means that that part of the brain is active. If I'm shown several pictures of people with mustaches, my brain will react to the features for each face. But there will be a common area of my brain that is engaged throughout. That may be the area of my brain that reacts to mustaches. So later, when I imagine a face, if Brice notices that area is engaged, he can predict that I am thinking about a mustache. So right now Michael's in the scanner, and he's seeing words appear on the screen one at a time, and he's trying to visualize the face, remember the face in as much detail as possible. What you can see here are the images that we're acquiring. We get one of these brain volumes every two seconds. So these are refreshing in real time as we collect the images. [Michael] With part one of the fMRI session over, it's time for part two, where Brice and his team will learn the language of my brain activity, so they can later decode by brain scans. Hi, Michael. You doing okay still? [Michael] Yup. They'll show me hundreds of unique faces, and record how my brain reacts to certain facial characteristics. They will then use this information to reconstruct the celebrity faces I thought about during the first phase of the scan. Really, the more faces that we can show Michael, the better. So we're going to basically keep him in there as long as he's comfortable. [Michael] Two hours was the maximum time we could get in the fMRI. But I was able to look at over 400 faces, which should be enough to get some pretty interesting results. Hey, Michael, you did it. That was great. We're going to come get you out. [Michael] All right. Yeah, so these just show some of the pictures that we were taking while you were in there. Some images of your brain. Now we are going to crunch some numbers. Max is going to analyze your data. We'll meet up again tomorrow, where we'll look at the results, where we try to actually reconstruct the face images from the brain data that we just collected. All right. Well, see you tomorrow. All right. Thanks a lot. Max, thank you as well. I can't wait. You better pull an all-nighter. I want this data to be perfect. All right, so I am back at Dr. Kuhl's lab. Overnight, his team crunched the data, and I can't wait to see what they think they saw me thinking. How are my results? I think they look good. We're going to take a look in just a moment here. All right, I can't wait. -So can I just take a seat? -Yeah, have a seat. All right, so... first of all... what am I seeing? Oh, okay, well, these are the pictures I actually memorized. -That's right. -And this is what you've reconstructed from my imagination. -That's right. -Oh, wow. Okay. [Brice] Okay, so this is one of the reconstructions that was generated. [Michael] Interesting. [Max] So that's John Cho. [Michael] Not bad. Not bad. -Can we see the side by side? -Yeah. [Michael] I see, you know, similarities in the kind of facial expressions in general. You know, you could almost see the hairline matching here. The shape of the face I also thought was-- It kind of had a square shape to it. -Yes. Yes. -So those are the things that came out to me. And so when I was visualizing this image of John Cho, the squareness of the face was the first, most salient thing. I just kept thinking, he was the square guy. Excellent, all right. [Brice] So that's Megan Fox. [Michael] Mm-hmm. You're going to show us the-- side by side. [Michael] The side by side. Right. [Brice] You can see the picture you actually saw, and that's the reconstruction we generated. I'll you this. Megan Fox, I was not able to have a really clear picture in my mind. For some reason, this image of her was really hard for me to bring back into my mind. The sternness in the face was something that I did pick up on. So I did sense that there was-- It looked feminine. And you picked up on the sternness. And so together, that produces a match. [Michael] Keep in mind that Brice and his team have read these from my memory. But when I remember a face, do I picture every detail simultaneously with photographic accuracy? Or do I just attend to a few at a time? By reading my mind, they may be seeing how bad my memory is, and how it works. -Me! Me! -[Brice laughs] Okay, so that is your reconstruction of me thinking about this image of myself. [Brice] That's right. Where'd the beard go? [Brice] I don't know. I was hoping you could tell me. [Michael] For instance, this is a picture of me remembering my own face. It really doesn't look like me, but the question is: how good am I at picturing myself? I don't think of my own face that often, so the strangeness in the result may be as much about flaws in my own memory and mental picture of myself as flaws in the technology. So that's Jennifer Lawrence, I believe. [Michael] That's Jennifer Lawrence? It looks like it's Jennifer Lawrence's much older uncle. [all chuckle] Nothing here was too mind-blowingly close. But this is something that you're just starting out trying these sort of long-term memories. What Brice and his team read in my mind might have been more accurate if they'd shown me thousands rather than hundreds of images in the fMRI, because then the algorithm would have learned the language of my brain more thoroughly. But regardless, the quality of my memories would have still been an issue. I mean, look what happens when memory is cut out of the equation entirely. Brice also read my brain activity when I was looking at faces in the fMRI. not just imagining them. And those results were much closer than those reconstructed from my memory. Okay, so, what am I looking at right here? [Brice] Okay, so what you're seeing here in the top row, these are images that you saw while you were in the scanner.