Placeholder Image

Subtitles section Play video

  • Um, hello, your regale Cuban oak.

  • Try.

  • Um, so Good morning.

  • Um, my name is Courtney, and today I'm going to talk about testing in production with JavaScript.

  • What, Come two years come put up its and thanks for coming to my talk.

  • Hayden's life doesn't matter.

  • Um, I promised the talks.

  • Not gonna be too heavy on dhe.

  • Let's get started so quick about me.

  • My name is called.

  • I am an engineer.

  • I'm from Spain, and I'm currently working a Bristol.

  • Started working on autonomous indoor navigation.

  • Your baby's on Manny area.

  • Baker's fronts on, then got back to the close pay for years working at I start back, called me that just got a quiet about being worse.

  • So I could just do with my team on now currently working at Dyson.

  • So I always like the star that talks with a quick pour to know how the audience is doing on this topic.

  • Um, I want to know how many of you know what Javascript ms.

  • Okay.

  • Thank you.

  • Erasing the house over there.

  • Um, are here for the city, the food, the party, and just a bit of javascript.

  • Okay, that's fine.

  • Um, no.

  • What destinies.

  • We're raising the bar.

  • No.

  • About the test empire made or any other way of classifying the different testing levels.

  • Yeah, most of you.

  • Thank you.

  • Um, who has a production environment again?

  • Every one of you.

  • Amazing.

  • Who is afraid off testing in production?

  • Who is not afraid of testing in production.

  • Give give.

  • This is coming.

  • So, um, are you testing your services in production?

  • No.

  • So many about quite a few.

  • That's great for the final.

  • Prefers spaces, two tops.

  • Yes, space is right.

  • So they still what's this talk about the stock is about.

  • I'm gonna try to answer some frequently asked questions about testing on your production were close.

  • I'm not going to be too technical.

  • I'm not going to deep inside any JavaScript kill on its whole base.

  • From the experience I gained working, I dance a Dyson.

  • I've been working a Dyson for almost two years now.

  • Um, and there has been great and we have learned a lot about testing in production.

  • So this is a quick summary of what I'm gonna I'm going to answer.

  • Why do you need testing intestine production with the white tested level?

  • Um, how can we ever scrape or no, no.

  • J s work help with this, um, helped a boy disrupting the real uses.

  • Because if you're testing in production, you're gonna be part off your users.

  • You're gonna be using protection systems.

  • How to keep, um, how to keep your test out of the statistics reports out of metrics.

  • So when you report your to your money Oh, too high, get the business.

  • Um, given Guess que resource on.

  • Then why you should clean up.

  • Andi, what happens if you don't?

  • Because that happened to us.

  • So yep.

  • Let's dive in a bit.

  • Why did you need testing in production?

  • Well, that's a good question, but I hope most of you can answer that.

  • But really, What do I mean?

  • Way?

  • What do I mean by testing in production?

  • When you deploy your application, I could be an application in the surveys.

  • Could be a lambda.

  • Whatever thing you deploy to production and has s o the public has access to is, um, you need to guarantee its continues to work over the days and over the hours.

  • And it's not just it passed.

  • The unit ties it past the service level, tears or whatever.

  • And then I deploy and I don't care about my service.

  • The service is running in production, and if something happens to it, you want to get notified?

  • No, but they users, but by some automated system so they d about.

  • This is to have some kindof testing tool that runs on a schedule basis.

  • So it's testing your service like it was a real user.

  • Andi, if something happened so the test fail, it means that a real use it wasn't able to do one off.

  • The actions you have defined for your production service's on.

  • Do you get a notification?

  • So let's let's dive into a lobby.

  • Why, I really do.

  • You need a testing?

  • You need testing in production.

  • So here's a quick simple in my same dumb.

  • But it can happen.

  • So imagine you have a London function that returns the current date right on.

  • Dhe doesn't cut it for the year 2038 problem, which is when you're storing your date in a sign it 32 bits Bible do run out off space to store the date, and then you overflow right on the 16th off generate 2038.

  • Um, so yet you do your deployment.

  • You right, you need test for you issue, which, on a date, if you query for the date again, he should return a different number.

  • Should return a different date.

  • Yeah, you really tested fine.

  • Do you deploy it on after a handful of years?

  • It's tough failing Andi.

  • It's because it has dynamic being put out, but it depends on own on the current time, on how the users are interactive winter service or what the service is returning dynamically but going to a more Rio situation.

  • Imagine one of your state full service is so it's a service that store some estate on your users on your data vase or on the state of your system.

  • Then suddenly by a buck or whatever becomes inconsistent, and then it starts failing for all the customers.

  • Then, when you will have it's a heart off customers chasing you, angry customers saying this is not working.

  • I pay for it, But you don't want that.

  • You want something that it's not gonna get angry after you.

  • Andi.

  • It's a vote on automated process that's gonna behave like a user is gonna test your things.

  • Your service is in production and tell you when something bad happens so you don't get the feedback from the uses.

  • Um, so if you compare these two situations so imagine you're in the first situation where you don't have an ultimate test system in production, and then something bad happens and all your customers come to you saying this is no working on dhe claiming back the money.

  • Um, it's gonna take some time for the user's to report to you that I never is happening and they have to go to your service, help this or logon issue, or look a ticket or send you an email or whatever.

  • So it's a slow you don't know.

  • Your service is failing until you get the feedback from the users.

  • Uh, but if you have an automated system that once every minute, every 10 minutes, every 20 minutes, every 30 minutes, you get notifications about your system.

  • You're working like this, Andi, also you.

  • If you're in the first situation, you're losing money because the more time you last in detecting that your systems aren't working that morning that you're losing on dhe and also your pain.

  • The support the supporting Ah, a lot for dealing with those things Where you go will have catch the beforehand and also the final.

  • The final comparison is about the company reputation.

  • So if you have all the uses coming back at you and say this is not working, is that I pay for the love of your company's reputation is over there over the floor.

  • And and then, if you haven't automate the assistant that and notifies you right away often Eastern your production systems, nobody needs to, um, notice it and then you fix it quickly, and then customers are happy.

  • So that's the cooperation on why do you need testing in production?

  • Um, so what?

  • Released the right level of testing?

  • You want to perform in production because you don't want to mess with really later.

  • But you're using Rio Systems.

  • Yeah.

  • So this is the test empire.

  • Mid one of the many testing pirates are, um but let me focus on let me focus on this one because it is one of the simplest that I found.

  • It comes from from an article in From Ultimate from their website Ultimate your own automation Panda.

  • On the bottom.

  • You have you need test than integration test or service level tests, however you want to call them and then the end 20.

  • So what happens when you have the unit tested you Wonder Unit test before you deployed your application.

  • So you test the single functions off your application for your service, and then you deploy your application because you are confident that your application was working Fine, because you have you tested it.

  • Um, but then when you deploy, um, you perform some integration tests or service level tests, So you're focusing on a single service.

  • That single service is the mini moon unit after deployment for us, the engineers, you, the police evidence service is on.

  • So these anto and Tess eso this integration types or seven level tests off for those single units from their engineer inside.

  • But what the user's say's is like the whole system on the user has used the intense user actions or sickle use of journeys are testing things across all the system.

  • Of course, all your service is so that's the kind off a level I'm referring when I'm talking about testing in production, you want to become a real you sir.

  • You want to be in the skin off a user on.

  • Try all the possible actions I use I can do to make sure your system still works.

  • Us expected.

  • How can javascript or annoyed?

  • Yes, help with that.

  • Well, um, this is where my example comes into into show.

  • Um, so at the moment I'm working a Dyson on.

  • We have more than one million connected mission performing actions against the cloud on calling.

  • Our service is calling our where FBI's Orlando's blah, blah, blah and also uses all around the globe doing actually in some performing, actually, just like renaming your robot from Dyson 360 I to make a 2nd 9000 Whatever, Um and then, um, what happens is that it's a big system.

  • It's a form off, several moving parts on.

  • I want to make sure that users can change the name off the robot and don't get the never right.

  • So I need a tool.

  • That's Fass.

  • That's easy to van in production.

  • So when I want to verify the all the use of actions, all the use of Uranus are working in production.

  • I wonder, too, like this locket containing or whatever it is not difficult to deploy.

  • I wanted to be extendable also So when I write, new service is I write new web ap eyes Whatever.

  • I cannot new libraries or new files on having those testing situations on DDE.

  • Also, I would like to have the ability to write behavior driving development test.

  • You want to call them leather?

  • So really, what?

  • I want to find what the s now you're for real User is what a real user's.

  • Gonna, however, really uses gonna interact with my systems on DDE and then a tried to reproduce that automated simulating those big 12 user assimilating everything but no emulating the clouds.

  • No emulating my service is the solution that we come to A place was to everything in in JavaScript using know Js on the main libraries that were used in combat area Js on the request library.

  • Also a bit off the library on basically what we tried to define a lot off singular steps off single actions and then using cucumber Yes, on Ben putting older sections together into a group to form an ex scenario you're gonna see now, um and yeah, then using the request library just make requests over the Internet, a city or its to be as requests on the library toe, sir, that the bodies were getting back are the ones that we expect.

  • This is an example off off the cooking bear off a scenario ricin in Cucamonga?

  • Yes, or Nixon.

  • Are you really?

  • I want to test that, given an existing user with this configuration on the use of locks in.

  • And they used to change the robot named to whatever.

  • Then the one the robot name has changed.

  • I want to test that.

  • That is unusual action.

  • I'm not testing one service.

  • I'm not test.

  • I'm not testing one function on testing the use of your own intestine.

  • They use their intent on.

  • That's what we want.

  • We want to and try and do what the uses will be doing in in production with our systems.

  • This is a very simple scheme.

  • Off Howie words.

  • We have the user's the uses at the moment interact with a smartphone application.

  • Um, that's my iPhone application, sends commands to the cloud, and then the cloud communicates that comments back to the rubber to the cleaning robot, for example.

  • And I was testing two amazing tested toe, a k a t t hospital users because there are no real users.

  • Have a bitch with a smartphone app on That's making the course to the cloud on back.

  • And it's also emulating the robot.

  • So we're simulating all the points in production except the cloud our service is in production.

  • That's we want to behave like uses.

  • But And if you test in production, um, basically, you're stressing your production systems.

  • You're becoming a user, and if you run your testing production very fires or you do a lot of factions, you become like a 1,000,000 users.

  • Talking to your service is at a given time, and you don't want that.

  • So you need to keep a balance between what you're testing in production on what you want to assert.

  • Um, do you know what?

  • It's good that you are very close today to the user and to the re elections, but you need to violence it right so far as an example of this, Um, when we were writing the test and we started doing tests like every five minutes, I guess if I recall, what happened is that we started Gatling getting a slowdown on our weary P s because we were quite in the way of the ice every five minutes, doing a lot of tests on then it was a slowing down.

  • Our productions service is and we don't want We don't want that for the user's.

  • And also we experience it some dynamodb throat Lee so that have a startling as our testing was doing coast to the A P.

  • I bury fires and today and reads on right to dynamodb or that vase.

  • Then we got proto on that was affecting users.

  • So you need to be careful about that.

  • You need to a space a bit your 20 s, um so the more often you run them, the faster you're gonna realize that you have any suit if there's an issue, but then you get more noise because you get more noise on looks.

  • You get more noise on the metrics.

  • You were motion more noise everywhere on slowdowns.

  • It slows down your production system on, so you want to keep a balance about that And just this another another quick meet.

  • Um, when for this testing tool, we are simulating this users.

  • So, um, we have code in JavaScript.

  • How user will behave.

  • What a PS.

  • Will they use a hit on DDE?

  • What does the user want to do?

  • Like, for example, Ching in the name of the war.

  • But, um yes.

  • As I said, we want to test the really user behavior.

  • Okay.

  • Um so basically, yeah, again one.

  • Do you want your tests in production?

  • You're becoming, Ah, another user.

  • You're becoming, um, part off off, off the system on DDE equal askew.

  • Your metrics.

  • Um, for example, the number off user interacting with your system.

  • You want to report to your mind a year or two Their business.

  • How many users are using your Your production service is.

  • But if you have this anto and test running in production than that skewing your resource also, if you want to know how many calls to an A p I, um and also the ever account.

  • So, for example, you are very interested on how many times you're a P I switch on enough 500 or returning our 400 if you have the entire contest, that's going to skew your results.

  • So you need a way off keeping those test that you know, about them that you are generating them, keeping them out of the study sticks.

  • And you could also get mixed looks.

  • So if you have a loving utility, that's put the puts all the locks together like, um, it's plank or elasticsearch.

  • How do you do this turn?

  • Which locks were part off the entire intestine, which looks where path really uses doing re elections.

  • Um, so you need to be careful with that.

  • How can we, um how can we keep our test out of the Thai sticks?

  • We could use a correlation, i d.

  • So with every request, we will send a spot of the request A correlation I D.

  • Or we will generate a correlation idea that's gonna be passed between the different service is internally, so every service, if they see the correlation idea starting by anto an f f f f f, they're gonna know it's part of the test.

  • Um, they shouldn't behave differently, but at least no longing metrics, for example, or logging metrics to a different name space.

  • We could also change the user again in case off way GPS.

  • Um yeah, specifying instead of being Mozilla gecko, whatever you have your application went toe in.

  • Tests on diversion on.

  • Finally, you can lose that.

  • New Http Heather's that nobody's gonna, um, freak about that.

  • You just put test.

  • Oh, bash in 1.1 point you or whatever.

  • Then you treat them differently.

  • Do not treat them differently.

  • You don't look the metrics.

  • Obviously that had a commune in into your production systems, right?

  • So after everything, um, this, uh, this was a quick introduction to how to run into intestine production.

  • You now have you and toe and test running in production.

  • They are executing every 15 minutes.

  • 20 minutes, 30 minutes.

  • Um, they're passing.

  • That's great.

  • But what about all the fate that they have generated?

  • So they they were beautiful.

  • Uses orbital machines, beautiful smart phone taps that we're creating new uses that we're creating new cleans New new later to our production systems.

  • Andi, that's fake data with them.

  • Really wanted into in the system on DDE, it's making something.

  • It's gonna cause a problem in the long run.

  • So, for example, we were log in, and when you've reduced their machine into the cloud into the service is we store a row in the database saying this matching is now.

  • Doesn't matter.

  • Six.

  • Andi belongs to this user.

  • Um So as we were executing the entire intestine production, we were creating new machines with new user random machines, random users, blah, blah, blah, blah.

  • But we were filling their database.

  • What happened was that our service is in production work.

  • Wearing that that race on dhe getting all those users on DDE, the service is were expecting to find between one on 90 entries, different entries in production.

  • But we were generating a lot off entries every single day.

  • So the entries ended up like being in the order off 1000 entries.

  • What happened at our service was not prepared to do any pagination.

  • And then it started failing for the customers because we couldn't find their entries because they were in their page six or pay seven.

  • Because all the tests, all the testing production, we're generating more later.

  • I'm filling up the database, and that's a problem.

  • Um, so, yeah, you need to clean that data is very important on Do you want to do that automatically if possible?

  • After every single test run?

  • Um, so it helps to do keep your environment clean it helps to a start a test the next test to start them from a clean state.

  • Avoid clutter in your your databases or clever in your your services or exhausting any other limited resources like, um, Ivy's or any any any of the limited resources.

  • You you go having your production system, um, and also avoids.

  • As an example, I just put a Boyd's making real data more difficult to query to retrieve on, to search Cem as every cop tested in production in school and necessary place to wait.

  • If you're not doing it, you need to think a surreal Use it to put yourself in the skin off a user on act like user so that you have really testing your your system, then, using the framework of combat.

  • Yes, it's a good start on.

  • We found that it worked for us, and it's it's amazing.

  • You need to think also about your system capacity.

  • So if your system cannot handle 1000 request per second, then you shouldn't be doing down many and 20 space them a beat.

  • Every health an hour every 45 minutes.

  • Whatever fits you, um, you also want to mark your test intense to differentiate them from there from the production low from the real users.

  • With the correlation I d or with the issue to be Heather or any other any other solution on after that?

  • You want to clean your test data on dhe?

  • We set every connections or every any status you might have changed after after the stuff.

  • Um, and that's so basically, um, thank you for for listening.

  • Um, one more thing.

  • I would like to take a big boy in top.

  • I wish my father, Graham Sophie, donated by now because I have plenty off for time.

  • So now a smile.

  • I hope I can feed everyone.

  • Yeah.

  • Amazing on way.

  • Thank you, but just wait.

  • Um are you brave enough to testing productions?

  • Are you?

  • Can I tell you what?

  • Um, yes.

  • And now the applause.

  • Thank you.

Um, hello, your regale Cuban oak.

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it