Placeholder Image

Subtitles section Play video

  • So guys, just yesterday we saw what is perhaps the largest competition to OpenAI's incredible Sora video generator model yet.

  • A lot of you probably have already seen videos about it, but I want to bring you some new and interesting use cases and value propositions that you might not see in other places.

  • So for those of you who don't know, this is Gen 3, which is produced by RunwayML up here.

  • Now, RunwayML is special in the AI video space because they were the first ones to make the actual first commercial video generation model.

  • And now they have Gen 3 Alpha, so the third iteration.

  • It's a step towards building general world models.

  • And yeah, right off the bat, you can see this thing is damn impressive.

  • I mean, really, really good.

  • Edges close to Sora.

  • I think in the motion department, it does struggle a tad.

  • But I mean, all of these examples are really, really quite impressive.

  • Take a look at this one, for example.

  • I think it doesn't have necessarily the same fidelity that Sora has, but it looks really, really cool and the motion is super impressive.

  • Or even something like this.

  • This has a little bit better detail in my mind, and it's really cool to see how temporally consistent the buildings are as they pass by the camera.

  • And I mean, you can do very special effect-esque style things as well.

  • The possibilities, as always with AI video, are truly endless.

  • So apparently, it's been trained with highly descriptive, temporally dense captions, enabling imaginative transitions like, you know, a street being flooded with water.

  • Not something you're going to see every day.

  • Or, you know, we have like this drone shot that moves through a castle.

  • This looks pretty cool.

  • Looks like real GoPro footage.

  • I mean, that is really impressive.

  • That almost looks legit.

  • Like, I would believe that that is an actual video, not some AI generated thing.

  • Here we can see some water glistening on a window through a train as it passes by.

  • Again, wouldn't really be able to tell right off the bat that that is AI.

  • Here is another one that people found really, really impressive.

  • This is clearly, you know, some more GoPro footage of someone walking through some tunnels, and then it pans over and you see a runway.

  • Look at the effect of the flashlight as it moves over the graffiti on the runway.

  • I mean, you can tell it's a different texture there.

  • It's really impressive stuff.

  • I mean, no doubt in my mind, all of these examples blow me away, and I cannot wait until we get access.

  • That's the thing.

  • We still don't have access to this, but it appears that access is going to be coming to Gen 3 very, very soon.

  • And I can imagine people are going to be willing to pay a lot for access to this thing because it is the closest thing to Sora that we've seen so far in terms of like prompt following, coherency, and temporal stability.

  • It is something else.

  • Photorealistic humans, that's obviously a really big deal when you're trying to tell stories.

  • Humans are most often a part of a lot of stories that are told in filmmaking, TV shows, etc.

  • So there's some really great-looking humans in here, and obviously that is some bias in the training data there.

  • They purposefully set out to make sure that it's going to be able to produce some pretty realistic-looking people.

  • And they absolutely have done that.

  • I mean, look at how cinematic this scene looks, and look at the glare of the sun and everything.

  • So impressive.

  • One thing I will say that I am seeing a lot is pretty much every one of these videos, no matter what it is, if it's a person or like this over here, everything looks like it's being shot in slow motion, which isn't necessarily a downside because, think about it, you can always speed up a video.

  • If for some reason it produces videos that look like they're in slow motion, well, we can always just speed up the video.

  • Like, that totally looks like it's in slow motion, no?

  • Maybe they purposefully trained it on slow motion video?

  • Like, we could just download these videos and speed them up and they would look completely normal.

  • I mean, shockingly realistic, but I just find that really intriguing.

  • I mean, there we have like a monster walking around.

  • That looks like a shot from a movie legitimately.

  • I mean, it's so impressive.

  • The way that the light bounces and reflects off of the fur, it is so darn cool.

  • Here's like some, I don't know, rock person walking through the woods.

  • Like, this thing has an understanding of physics and the world in a way that reminds me of the Sora demos.

  • This is a true competitor to Sora, I think, more so than anything else we've seen.

  • And that's really, I think, what is causing all of the stir around Gen 3.

  • I keep seeing people talk about this.

  • I mean, yeah, check that out.

  • I mean, that's some, that's a pretty good understanding of physics and paper and how it's going to interact in this alleyway with the wind tunnel.

  • It's super cool.

  • Here's another one that again looks like it's in slow motion.

  • I mean, why does it look like it's so in slow motion?

  • I don't get that part about it.

  • But the way he like moves his thumb and everything, that just looks so legit.

  • And these are obviously going to be cherry picked here.

  • Detailed close-up of bacteria.

  • That's so cool.

  • I mean, that looks so legit, man.

  • That's just some crazy, crazy concepts.

  • It's exciting.

  • It is exciting, no doubt.

  • See different styles.

  • Here's like an anime, animated film art style.

  • Really cool stuff.

  • They've really gotten close to solving motion at this point.

  • I mean, it's just, again, here's another one that looks like it's in slow motion.

  • Super cool though.

  • I mean, what would you guys create if you had access to this?

  • It seems like access is going to be coming.

  • At least it was hinted in like a few days.

  • We're going to be able to get access to this.

  • So I'd love to do like a live stream, testing it out, hanging out with you guys, doing some prompts.

  • I don't mind paying for credits to use something like this because it's groundbreaking technology.

  • Like I said, biggest Sora competitor that we've seen so far, at least in my mind.

  • Anyways, that's sort of the TLDR on Gen 3.

  • Now let's get into some of the more interesting use cases.

  • Some of the bigger value propositions that you might not see in other places.

  • So Cristobal here on Twitter brings up that there's a lot of unexplored latent horror to discover.

  • So the applications for horror movies or horror genres are very, very much already alive, as we can see by this terrifying generation.

  • He does a few other generations.

  • Like we've got a monkey here who is learning to play guitar on the street.

  • And again, it's just a really nice smooth shot.

  • Looks like it's in slow motion.

  • I don't know what it is with runway and the slow-mo stuff, but it's super weird.

  • Oh yeah, he works for runway, so that's why he's got access to it.

  • And you can see drink water over here.

  • So the text is also really impressive.

  • It's able to do like these animated things with text popping up on the screen.

  • I mean, that's really darn cool.

  • Some more examples of this.

  • Look at the reflections on the stones, though.

  • That's not something that is easy to replicate in a 3D animation environment.

  • That's really, really crazy.

  • And that's one of like the impressive things about AI is that those more complex things that you would normally do in 3D animation are actually a lot easier in AI.

  • And the simple stuff in animation is the most difficult part in the AI.

  • And here we go.

  • We've got like some animation with snow in the mountains and text comes through and says, ice, ice, baby.

  • And yeah, I mean, he is really, really showing off the fact that you can produce really good looking text.

  • Can't sleep coming down there in sand.

  • And here we have another example where it just says, Timmy.

  • And it's like this blasting dirt flying into view.

  • It's really awesome stuff.

  • I would love to try some MatVidPro ones.

  • I mean, most of these are pretty simple words that he's trying in here.

  • So I'd like to see how it does with more complex stuff like a username.

  • And here you can see it does text in novel situations, not just where you would expect to see it.

  • So we have a giant maybe on top of, you know, an exit sign on the highway or something.

  • It looks like there's some interesting car accidents and really, really weird craziness going on in the corner here.

  • So these models, keep in mind, are by no means perfect.

  • But this absolutely competes with the best of the best, which is Sora.

  • And the list kind of just goes on for you guys.

  • I mean, you can see on top of a train, giant danger sign.

  • It obviously looks edited, but it gets the point across.

  • This was incredible.

  • I mean, this is clearly like a bowl of soup or something.

  • And then you can see these little letters drop in.

  • And the way that the physics and the letters move with like the liquid, it just looks so realistic.

  • And it looks like it's not been created in a machine or a physics environment.

  • It looks like it was created or actually filmed in real life.

  • And that's really one of the more interesting parts about AI generated video that you're going to notice is that since it is trained on real footage, it gets a lot more close to realism in the physics department than you'll see in like actual 3D animation, because it's something that is really hard to replicate.

  • Again, we have more of that latent horrors type of thing.

  • We got Pytorch written on a wall here, but it's someone who is walking through the woods and then comes across something.

  • Here we've got coffee pouring into a mug.

  • This is by Emily Golden, another member of the runway staff.

  • So it seems like they all have access to it, but there's just so many crazy things that can go on.

  • Here, folks, is yet another impressive display of Gen 3's knowledge capabilities.

  • We take this ice cube and we drop it on a searing hot pan, and immediately it starts to, well, melt into water, which shows that the model has a very intelligent and nuanced understanding of how physics work and interact with our natural world.

  • So the ice isn't necessarily melt, but it definitely produces the water.

  • We're getting a lot closer to a model that can generalize and think about physical interactions in our world very well.

  • So again, coming straight from runway ML here, we do have some specs.

  • So it takes about 90 seconds to get a 10-second video, which is not too shabby at all.

  • That's very quick generation, can generate multiple videos at once.

  • They also are going to be adding motion brush, advanced camera controls, and director mode to this.

  • And they're also going to have more fine-grained control over structure, style, and motion.

  • I mean, again, we didn't really get this kind of announcement with Sora.

  • Sora is very much kept under wraps as an AI video generator, and we're already getting all of these crazy other generators cropping up, and Gen 3 seems to be the most fully-fledged of the bunch.

  • Here is a nice example of some 3D animation of an anime girl.

  • This one's a little bit creepy.

  • The eyes and the face are very consistent, and the hair and everything, it looks good, but it's creepy.

  • It looks like a doll or something.

  • Here we've got some footage of some fire burning in the distance.

  • Again, this looks super real, almost scary real.

  • People might think that this is a genuine actual video that was recorded.

  • Lots of really great cinematic stuff as well.

  • Again, some of it does delve into the more creepy side of things, like we've got giant ears on someone.

  • This just looks weird.

  • I don't know.

  • It's just a strange generation.