Placeholder Image

Subtitles section Play video

  • [TRAIN HORN]

  • Hello.

  • And welcome to a coding challenge, Generating StyleGAN

  • Rainbows With Runway ML and passing those to P5 JS

  • to display them in the browser for our delight.

  • And so this is basically making a version of software

  • that I built for the I/O Festival.

  • The talk that I gave at the I/O Festival

  • will be out on the internet at some point soon.

  • And if it is, I will include a link

  • to it in this video's description

  • where I had people play a game on stage.

  • And when they finished the game, an AI rainbow

  • was generated and tweeted to the I/O Rainbow's on Twitter

  • account.

  • So I'm going to build a little mini version of this today.

  • And so the primary tool that's going

  • to do the generating of the rainbows

  • is something called Runway ML.

  • So this is the first coding challenge

  • that I'm using with Runway.

  • And I've talked in a couple other videos

  • in a livestream more about what Runway

  • is, how to download and install it,

  • how to sign up for an account, how

  • to get some free credits, and that sort of thing.

  • So I'll refer you to that video if you want to find out more.

  • But if you have Runway downloaded--

  • find a link to download it in the description.

  • If you've signed up for an account,

  • and you're on the Browse Models page,

  • you will find yourself here.

  • And where you want to go next is you want to look for StyleGAN.

  • Now, I see StyleGAN right here.

  • But just in case you don't, I could type it in up here

  • under Browse Models.

  • I could click here.

  • This is now giving me, "Generate photorealistic images

  • of faces, landscape, and more."

  • There's more information about the license for StyleGAN,

  • credits about who created and authored the original StyleGAN

  • paper that you can find out about.

  • But I just want to use StyleGAN.

  • So I'm going to click Add to Workspace.

  • And if you don't have a workspace already,

  • you can click New Workspace.

  • I have one called Coding Train Livestream.

  • And then here I am.

  • Now, I can generate a variety of types of things with StyleGAN.

  • And these are known as checkpoints.

  • So there are checkpoints, you can see,

  • for cars, and landscapes, and portrait.

  • Wouldn't it be nice if there was a checkpoint for rainbows?

  • And guess what.

  • There is.

  • So I'm going to click the rainbows checkpoint.

  • Then I'm going to choose the input source, which

  • is just going to be a vector.

  • And I'll talk about what that is in a little bit

  • as I get to writing some code to do this.

  • And then what I want to do is click Run Remotely.

  • So this is very important for me to be clear.

  • This requires cloud credits that you have to pay for.

  • If you sign up through the link in this video's description,

  • you'll get free $10 in credits.

  • And there's actually a coupon code, CODINGTRAIN,

  • which you can get an additional $10 in credits as well.

  • So that's certainly enough to run this example.

  • Some models in Runway you can run locally on your computer

  • without using cloud credits.

  • But this one in particular only runs remotely at the moment.

  • So I'm going to click Run Remotely.

  • It's running.

  • So I'm going to give it some time to start up.

  • So now we see it's starting to populate.

  • And this is what's known up here as the latent

  • space, this sort of space of imaginary rainbows

  • that this generative model is producing.

  • And this is one of the reasons why

  • I love using Runway is that I can actually just kind

  • of browse around this space, and kind of

  • have this 2D view of this multi-dimensional world of all

  • of these rainbows.

  • And I can do things like, oh, I really like this one.

  • And I could just change the output here to preview.

  • And so I can see it here.

  • And I could download this one.

  • And I now have my beautiful StyleGAN generated rainbows.

  • But what if I want to have these rainbows in my own software?

  • If I wanted to show them on a web page?

  • Or if I want to tweet them from my Twitter bot?

  • Or any other types of thing that you

  • might be making, whether it's JavaScript, processing,

  • open frameworks, or some other piece of software

  • that you want to connect to Runway.

  • The way that you do that is through talking to Runway

  • over the network.

  • So over here in the bottom--

  • I'm sorry.

  • In the top right, there is a Network tab.

  • If I click on this, it's showing me

  • a variety of different options.

  • I can communicate with Runway over OSC.

  • This is something I did in another video tutorial,

  • communicating with processing and Runway over OSC

  • to demonstrate the PoseNet model for detecting

  • human skeletal poses.

  • I could also use Socket I/O for real time web sockets.

  • But really what I want to do is just an HTTP connection.

  • I want to make an HTTP request.

  • I want to post some data to Runway.

  • And I want to receive the image back.

  • And, in fact, there's JavaScript code right here out of the box

  • that I could copy and paste.

  • So I encourage you to just actually

  • go and grab this JavaScript code,

  • and make your own example.

  • But I am going to do this using the built in P5

  • function, HTTP Post.

  • So I'm going to write my own code

  • for doing this, referencing everything that's

  • here under the HTTP option.

  • My next step then is to go to the browser.

  • And I am going to write this code in the P5 web editor.

  • So first thing that I might do is

  • let's make sure this is running, if this works.

  • OK.

  • Great.

  • Let's make a button.

  • Create Button.

  • And I'm going to call that button rainbow.

  • And I'm going to attach that button to an event called--

  • whoops.

  • Let me hit stop here for a second--

  • Generate Rainbow.

  • So when I run it, a button will appear.

  • Rainbow.

  • And then presumably when I click the button,

  • a function called Generate Rainbow will be executed.

  • So I need to write that function.

  • And in that function, this is where

  • I want to send my request to Runway itself.

  • So I need to make an HTTP post.

  • So I probably want to look at the HTTP post.

  • This is a P5 specific function.

  • You could use fetch.

  • I have some video tutorials about how

  • to use the fetch function to make a post request.

  • And you could do that here.

  • But I'm going to use P5 in this example.

  • So I'm going to go to the HTTP post reference page on P5.

  • And this is going to show me the stuff that I need

  • to include in the post request.

  • So here's the stuff that I need.

  • I need a path.

  • Where am I posting this to?

  • I need a data type, which is what kind of data

  • am I sending along with this post request?

  • And then any parameters, data that

  • needs to go along with the post request, as well

  • as the callback and the error callback for getting

  • information back.

  • So if I could just take this and bring this into my code,

  • I'm just going to put this in the comments

  • right here as a reminder.

  • So what are the things?

  • The path is--

  • Runway's telling me this.

  • It's actually this.

  • The server address.

  • Local host port 8000.

  • So I'm going to paste that in here.

  • The data type is JSON.

  • That's the kind of data.

  • The data I'm sending is what?