Placeholder Image

Subtitles section Play video

  • Automated Performance Budgeting Into Your Pipeline With Sitespeed.io

  • - Daniel Lopez

  • Hi, thank you.

  • Mic check.

  • Hello.

  • All right.

  • So the official title for this is automated performance budgeting into your pipeline with

  • Sitespeed.io.

  • It's kind of a mouthful and also I put W there for some horrible reason, apologies for that.

  • and I normally don't do public speaking, I'm very happy that we have the HPE track for

  • those who are beginners, or yeah, and I just really wanted to share this tool, because

  • I think it's awesome.

  • I think it really doesn't get enough credit, and it's been around for an extremely long

  • time, and I just wanted to share it with you.

  • So that's what I'll be doing with you here today.

  • So a quick overview, so we're going to talk about my own personal performance journey,

  • some -- I guess just like performance budgets in general, what they are, what they're for,

  • and we're going to talk about Sitespeed.io specifically, and performance budgets within

  • Sitespeed.io specifically.

  • We're going to talk about, there's a lot of different tools that do performance budgeting,

  • Sitespeed.io is one of them and there's a lot of other really amazing tools that I found

  • with in my journey.

  • So yeah, we'll be talking about other tools and things to consider, and I wanted to throw

  • out some stuff that I've been experimenting with, and what I see as my future for front-end

  • performance, what I'd like to see, and what I've experimented with.

  • So first: A little bit about me.

  • There's me behind my pizza curtains, that I sewed myself and put up in my office and

  • some things that I like.

  • I go by the name of debug xyz on the interweb and that's because I have way too many hobbies

  • for my own good.

  • I'm not good at any of them.

  • So this list here is all things I'm really bad at.

  • So I'm into growing Pequin peppers, tech minimalism, is hard.

  • Accessible is hard.

  • Skateboarding was really hard.

  • turntablism is insane, I'm horrible, horrible at that one, but I love learning about it.

  • Front-end engineering is insanely hard.

  • Right now I'm learning NixOS which is an operating system that is configuration as code.

  • I've been doing, I don't even want to say how long I've been taking front-end seriously

  • but I'm relearning CSS and investing a lot of time in that, as well.

  • So I'm going to quickly go through two of my hobbies, don't worry, I'm not going to

  • go through my whole list of hobbies.

  • One of my brothers gave me chilly Pequin seeds.

  • He suggested I give it a shot.

  • They have a 15% germination rate.

  • Normally birds only eat them and they kind of eat away the hull of the seed so you kind

  • of have to emulate that to make sure that they're happy and keep them O

  • Thrasher, so this is a '90s thing.

  • I'm see if I can resize my slide here.

  • There we go.

  • Yeah.

  • Here we have the pose of the year, so at this point I'd been skateboarding for a very long

  • time.

  • I had -- it was an absolute obsession of mine.

  • It takes tons of practice, and is -- it's difficult.

  • And so when I was skateboarding, I would go all around downtown, I'd grind the -- you

  • know, destroy the curbs at my local library, and it would get really hot and the (speaks

  • Spanish) when I would get really hot and exhausted from skateboarding, I would go into my local

  • library, and cool off, and play with a computer.

  • And that's where I actually learned to code.

  • And I would print out -- actually JavaScript, even though I gave up really quickly, and

  • take it home for studying, and I did that, because it was hard.

  • So when I knew I was going to be a programmer, I decided I'd go to boot camp.

  • That was hard.

  • So this is me at my graduation photo Parris Island and there's my father and mother supporting

  • me at my graduation.

  • It was hard.

  • Frontend is hard.

  • It was very difficult and achieving web performance can be very difficult.

  • It's a discipline -- like front-end web performance is a discipline within front-end and maintaining

  • web performance is even more difficult.

  • You can really work really hard, you know, getting your site to get that time to first

  • interactive super-low, but at the pace that we jump from job to job, and from project

  • to project, you're going to have a lot of people from with varying skill levels, and

  • everyone within front-end performance has their specialties.

  • There's amazing front-end developers who are super knowledgeable about accessibility, but

  • may not know about front-end performance, so it's important to share your skillset with

  • your peers and make sure that you're having front-end discipline meetings to kind of share

  • your expertise within the field.

  • So you have to do all of the things for front-end performance: Capture metrics, observe trends,

  • provide alerting, fixing actual issues on the website, because that's what, you know,

  • -- what we're talking about here is that's the kind of context that I come in from is

  • the web, and front-end development.

  • So fixing issues on the website, raising awareness with your peers.

  • You know, you may go through this same struggle if you're working on things like security,

  • maintainability, maybe you're super into testing, you've got to be sure that you're raising

  • awareness within your peers, as well as consulting, tools, analysis, for analysis, providing -- supporting

  • platforms.

  • One of the first things I did when I started focusing front-end performance within my organization

  • is I just started capturing PSI data because I thought that was really important for reasons

  • that I'll talk about later.

  • And just all of the things you have to do.

  • And it's -- and it's pretty intense, you know, especially if you're a smaller shop, and you

  • may have all of this within your organization, but you still need to provide that consulting

  • to your peers to make sure they're aware of the tooling and of the structure that's there.

  • Performance is critical on the web.

  • I'm not going to talk too much about this, but CPU are getting really fast.

  • We have blazing fast mobile devices, but you know, not everyone has that, and most of the

  • cell phone adoption right now is happening with like new people buying cell phones who

  • haven't had cell phones before, are buying low to medium-tier devices, so it's super

  • important going into the future.

  • And as the web advances, expectations are getting higher.

  • People want snappier experiences.

  • Users gret frustrated, they tend to get less likely to come back and nobody likes a slow

  • site.

  • This is us today.

  • I actually like I think that was funny and stuff, but I like I literally do this.

  • I literally, you know, do this to my salad, like it's really gross.

  • [laughter]

  • So yeah, most of you are probably concerned about profit on your websites.

  • This is it.

  • So Step 1, Step 2, performance, profit.

  • Check it out.

  • No, seriously, there's amazing resources by Google and there's this book that I read called

  • time is money, the business value of web performance that if you're trying to sell web performance

  • to your organization, you gotta read this book.

  • It's amazing.

  • It gave me tons of case studies to show to my leadership.

  • And, you know, you know, you could -- don't do an A/B test for it.

  • Look to other studies, seriously, because it's very difficult.

  • I don't recommend it, And you're ignoring a key part of the equation.

  • There's a lot to it, it's very complex, but, you know, SEO is a big factor to revenue,

  • so where you're, you know, with Google having tools like Lighthouse and PHP insights being

  • directly factored into your page rank score, this is one of the key pieces of advice that

  • you know, Google lets you know is a big impact.

  • So I forgot some things.

  • In that big list of all the things and that's regression.

  • So it's -- it's easy to accidentally, you know, you have a project where you need to

  • convert a timezone, so you pull in a moment and then you realized that you need moment

  • timezone plugin and then that pulls in literally all of the geographical latitude and longitude

  • information about the timezone lines that you didn't know about, and wasn't documented.

  • So it can be an easy mistake to make, and you need tools to support you throughout that.

  • And also, it's about scalability, as well, it's kind of a weird term to use in this sense,

  • but I think about organizational scalability.

  • You can't be there for everything as, you know, maybe that one or maybe there's a couple

  • performance experts and you have this large organization, or a large project, so I tend

  • to speak within the context of organizations.

  • But you may be working on an open source project where performance is absolutely critical.

  • It may be for Mobile Web.

  • And I'm still probably forgetting tons of things.

  • So what are performance budgets?

  • It is what it sounds like.

  • You define something like a budget.json, you can specify some things.

  • Typically you might want to failure build.

  • It's a bit of a touchy subject to roll out to your organization, but I'll talk about

  • that a little bit later.

  • How can performance budgets help?

  • First of all, it gets the conversation started.

  • When you introduce performance budgets into your organization or project, you're immediately

  • going to start talking about performance, which is immediate when?

  • But what's more important, and one of the keys to performance budgeting is it keeps

  • the conversation going, so this is, you know, when you're introducing a new feature and

  • then you roll it out and then it fails to build, you're like, oh, well, lesson learned,

  • I'm going to make sure that I bring up this conversation a little bit early, because it's

  • kind of pushed our project a little bit out.

  • You're making explicit tradeoffs.

  • Instead of implicitly kind of making that tradeoff, you're being direct and more -- you're

  • being more direct about it.

  • There has been a conversation, there has been a commit, maybe you even bump your budget.

  • That's OK.

  • It works like that in real life.

  • If you've ever tried budgeting and like going on a hard core budget, the first thing you

  • do is overrestrict yourself.

  • And you adjust, and that's OK.

  • But it's being explicit about it that kind of helps out with the situation and it prevents

  • accidents.

  • This is one of the biggest things for me.

  • So there's a lot of different things that you can budget on.

  • So there's score-based metrics.

  • If you're using something like Lighthouse you might be familiar with, it gives you an

  • overall performance score, so you can assert on that.

  • So if your SEO is critical to your organization and you need that traffic coming in to make

  • your revenue, you know, that score is going to be important, so your SEO team might even

  • demand that and then there's quantitative.

  • So this is about numbers.

  • It's about how many kilobytes of JavaScript do we think we really need?

  • And kind of asserting against that.

  • It can be how many images you have on your home page?

  • And maybe it will remind you that oh, I need to lazy load.

  • That's one of the things that, like, I do as consulting within my organization.

  • Lazy load.

  • Lazy load, lazy load.

  • So timings, as well.

  • So timings can be difficult.

  • In fact, this is one of the things that I really wanted to touch on, because when you

  • roll out performance budgets into your organization, the last thing you want is false positives,

  • especially at the beginning, so that's one of the key advices -- pieces of advice that

  • I have, is to make sure that if you're budgeting, make sure that your timings or that what you're

  • budgeting on doesn't have false positives and that your timings are stable.

  • Before I forget, you know, because this is an important part, I have some advice on keeping

  • your timing stable, so within the type of websites that I work on, there's lots of third

  • parties, and that can cause a lot of volatility in your metrics, so you don't want to include

  • that within the metrics that you're budgeting on.

  • It is important that you keep an eye on those things, but make it separate and budget for

  • them separately.

  • Accessibility budgets!

  • This is about performance budgeting, but it's an easy thing to do, so don't forget to check

  • the accessibility box within your performance budgeting tool.

  • A lot of them come with that so you can assert against that, and it's an easy win for accessibility.

  • Just a caveat here: Automated accessibility detection only catches like an extremely small

  • amount of actual accessibility issues.

  • I've seen percentages as low as like 3%.

  • But that doesn't mean that you can't take advantage of some of those easy wins of making

  • some simple mistakes.

  • So don't forget to check the check box and don't forget to use your label tags.

  • So back to performance: How do you roll out performance budgets within your organization?

  • I mentioned that it can be easy to be too aggressive, and it can be difficult to sell

  • it.

  • One of my main pieces of advice is when you -- when it comes to rolling out in your organization,

  • is to don't immediately jump to the slowest item on your page.

  • Go to the fastest.

  • Because really this is about regression and keeping a conversation going, and I'd recommend

  • if you have a fast area already, do that first.

  • Let's see here.

  • No, you really have to get everyone on board, so that's actually why I'm here.

  • This is my sprint demo.

  • So yeah, you have -- you know, I'm going to be giving this presentation to my organization,

  • as well.

  • Another thing, you know, make sure you plant those seeds early, so I've been talking about

  • this for a long time within my organization, and people are asking me for it now.

  • And that's a wonderful thing.

  • Some places to focus, places, you know, areas that are really fast, new projects can be

  • a great place to start.

  • Just for like my context, the happy path is a great place to focus.

  • This is the areas of your website that the users are going to be kind of jumping through.

  • Some ideas of things that you can budget on, so things like time to interactive, speed

  • index is an amazing visual metric.

  • And I was exposed to that by another insanely web tool, web page tests.

  • People look at it all weird, but it's really an incredible project.

  • So just wanted to mention that tool.

  • Yeah, bundle size, basically anything within the performance API, so like, sitespeed,io

  • specifically, it has incredible metrics.

  • We'll talk about it more.

  • Sitespeed.io is very pluggable.

  • It's been that way for a while, and yeah, you can, you know, I want this checkout button

  • to display within this amount of seconds.

  • It's pretty awesome.

  • How do you figure out what your budget is?

  • At Google I/O 2018 I think they released a performance budget.

  • It's a slider to kind of estimate how fast you want your page to load.

  • And then slide like how many kilobytes of JavaScript and whatnot.

  • So that's an amazing introduction of a nice little tool.

  • Analyze your competitors.

  • You know, you can actually -- there's a little SEO trick to find your competitors.

  • I actually don't remember.

  • Yeah, you know, just look at your competitors, see how they're doing.

  • Another thing is that when you're working against competitors, you know, just a slight

  • difference between your competitor doesn't really make that huge of an impact.

  • You got to be way faster than your competitor to actually notice.

  • There's statistics on this, and studies on this.

  • What happens when you make a mistake?

  • What happens when your budget fails in your pipeline?

  • You can, you know, obviously optimize the feature that you're introducing.

  • You can remove other features, you can redesign the feature, you may even catch this early

  • as you're, you know, talking about it early in the design development process, but it

  • can just as easily be redesigned afterwards.

  • Or just don't do the feature at all.

  • It's an option.

  • Less is more, seriously.

  • It's so easy to forget.

  • So here's a screenshot of Sitespeed.io budgeting results.

  • I don't think I'm going to have enough time to do an end-up demo, but the innovation within

  • Jenkins is pretty awesome.

  • The documentation is incredible on that.

  • And yeah, we'll talk a little bit Mr. Sitespeed.io.

  • It has something called Coach.

  • This is the thing that gives you advice.

  • It has a philosophy of trying not to give you any bad advice.

  • If you follow the Coach, you will win.

  • Browsertime is the thing that kind of actually opens up the web page and tries to capture

  • measurements.

  • It's actually very cool technically if you deep dive into it.

  • There's the thing called XVXB it's like an X display server, which is super-cool.

  • It uses a neat protocol.

  • And to dig into Sitespeed.io even more, it has this tool called pageXray, shout out to

  • the designer these are all little icons in the project.

  • There's this PageXray.

  • It's used on the compare.sitespeed.io which is a waterfall comparison tool.

  • It's amazing work.

  • I just love this project.

  • And, yeah, I definitely want to deep dive into that at some point.

  • Here's an example of some of the format here.

  • This is where if you wanted to write a deep dive you would write your own information.

  • The compare.sitespeed.io, I'm obsessed with waterfall comparisons, I'll have to jump through

  • some of these slides because I want to talk about the future here, especially when this

  • comes to water fall comparisons.

  • I really love Sitespeed.io because it fell within the technologies that I use at work.

  • So we use Graphite and Graphana, you can directly import to have insanely nice displays of all

  • of your metrics.

  • If you're using Sitespeed.io not just for budgeting, but for tracking metrics separately.

  • The you know, there's this cool thing called Throttle.

  • I haven't deep dived into it, but there's some really interesting stuff about this.

  • A lot of the volatility.

  • Again, volatility within your timing metrics is critical, there's this kind of network

  • replay thing that is experimental right now, that it would capture what, during the task

  • what the network activity was, and it would replay that activity, which is a pretty insane

  • idea.

  • And Throttle is inspired by the funniest GitHub project name I've ever heard in my life.

  • So we're talking about a tool that slows down your website.

  • It's called Comcast.

  • [laughter]

  • It's just incredible.

  • I jokingly say read the manual, because the documentation is insanely good.

  • There's tons of integration, instructions for -- I implemented it using it my own personal

  • GitLab instance and it wasn't too bad.

  • I didn't read the manual, don't recommend not reading the manual.

  • I kind of did it on my own, and yeah, and I also integrated work on Jenkins instance.

  • These are documented integrations for GitHub Action, Jenkins, Travis, CircleCI, GitLab.

  • The concepts are basically the same across platforms, but difficulty varies.

  • For me, setting it up on Jenkins was difficult, because at work, we use just a plug for our

  • blog, that our DevOps engineers are amazing, so engineering.rei.com, they write a lot about

  • infrastructure as code there, so I had to deep dive into Jenkins DSL, which is, you

  • know, a little subset of groovy, in order to implement this pipeline as infrastructure

  • as code.

  • So it seemed hard, so I gave it a shot.

  • And you know, I don't expect you to be able to read this and I wish I could pull it up

  • a little bit more, but yeah, this is what Jenkins DSL looks like.

  • I specified what I recommend if you're rolling this out to a large organization, is to provide

  • some default, sane, and generous budgets and so that's what I tried to do, but you want

  • to have that flexibility so your different teams who have different requirements and

  • different needs can override those budgets.

  • So that's what I did here.

  • So I pulled in some default budgets, merged it with the project budget, and then did an

  • assertion which basically just ran Sitespeed.io as a Docker image.

  • And then I let -- we use Bitbucket now about that, and the Jenkins integration is awesome

  • because it uses the HTML publisher plugin, which Sitespeed.io will output like a full-blown

  • website with a video, showing the page load, illustrating the visual metrics and when your

  • build fails, you know, you can output some text that will link you to a nice little,

  • you know, whole suite of HTML that you can kind of view. and kind of figure out what's

  • going on with your budget.

  • GitLab should be pretty straightforward, follow the docs.

  • There's lots of other budgeting tools and lots of amazing, absolutely incredible tools.

  • So I've -- one thing is that WebPack actually has performance budgeting built in, you just

  • may not know it so if you want to roll out some sort of basic performance budgeting,

  • into your organization there's a flag within WebPack that you can flip to make sure that

  • it fails if your bundle size is over some insane amount.

  • So you can make that assertion there.

  • Lighthouse is obviously amazing.

  • It has Sitespeed.io actually has Lighthouse integration, so you can use Sitespeed.io to

  • run Lighthouse and assert against Lighthouse scores as well as other Lighthouse metrics.

  • But Lighthouse is a great place to get started if you at all care about SEO, which a lot

  • of people do.

  • Speedcurve, I've experimented with this, it's incredible, as well.

  • And the future!

  • So I think regression analysis is kind of empty.

  • Maybe I just don't know much about the toolsets here.

  • But I've done some basic experimentation.

  • We've set up a private webpagetest instance, within our organization that we can run tests

  • against and I just ran a job that ran webpagetest private instance on real mobile devices and

  • started plotting that information out on a graph using D3 and what I haven't seen done

  • before, and this may exist is you know, you see the graph go up, but OK, what happened?

  • Why?

  • So what we did, you'd select a low point of the item that was ran by webpagetest and you

  • select a high point and that took you to the wonderful waterfall comparison tool where

  • I've caught seriously so many issues while we have this experimental prototype up.

  • It's, you know, the ease of being able to observe regressions is important.

  • Within my own personal journey, setting up webpagetest is not straightforward.

  • And especially with physical devices, and I think it could be easier if tools like webpagetest

  • and other tools could start using the cloud.

  • So there's a lot of cloud services that are providing device forms, and if these tools

  • can start integrating with those device forms, instead of plugging in your real phones, you

  • can start using services like G cloud or alias Device Farm to connect and run your tests,

  • but right now these are mostly focused around actual mobile app tests, so I don't really,

  • you know, I don't really see a straightforward way to kind of run those tests.

  • In the browser. and integrated in the tools.

  • I've spent a ton of time kind of doing research.

  • Here's a couple of links, and the Google I/O talk is basically the same as this.

  • I highly recommend checking that out.

  • There's a Google has put out so many amazing tools and guides and guidance, so I'll actually

  • publish these slides and put even more links and credits here.

  • That's my talk.

  • Thank you everyone.

  • I appreciate it.

  • [applause]

Automated Performance Budgeting Into Your Pipeline With Sitespeed.io

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it