Placeholder Image

Subtitles section Play video

  • Thanks to Brilliant for supporting this episode of SciShow.

  • You can check out Brilliant.org/SciShow to learn more

  • and get 20% off an annual premium subscription.

  • This episode was filmed on March 19th, 2021.

  • For up-to-date information on the COVID-19 pandemic,

  • check out our playlist linked in the description.

  • [♪ INTRO]

  • If I offered you a choice between two things,

  • and one of them was 95% effective and the other was 66% effective,

  • you might wonder why I was even bothering.

  • This is playing out right now with our vaccines for COVID-19.

  • For the sake of ease, we're going to compare the three vaccines

  • authorized for use here in the USA, but this basic logic applies to all of them,

  • like Sputnik V or the vaccine made by AstraZeneca.

  • The vaccine made by BioNTech and distributed here in the US by Pfizer

  • has a 95% efficacy rate.

  • And the one by Moderna clocks in at a similar-sounding 94%.

  • Meanwhile, the shot from Janssen,

  • the pharmaceutical division of Johnson & Johnson,

  • has a 66% efficacy rate, though their data is still awaiting peer review.

  • But this is not a matter of you having a 5% chance of getting sick

  • compared to a 34% chance.

  • That's just not how the numbers are calculated.

  • So let's look at how it does work, why we can't compare these numbers,

  • and why that's ok -- it's still a good thing that we have these vaccines.

  • This efficacy number comes from phase III clinical trials.

  • Specifically, it's from their primary endpoints,

  • meaning the criteria they set ahead of time to determine a vaccine's efficacy.

  • Phase three trials recruit thousands of volunteers,

  • some who receive the vaccine and some a placebo.

  • To calculate the overall efficacy of the vaccine,

  • they take the number of cases of illness in the placebo group

  • and calculate how much less likely the vaccinated participants were to get sick.

  • So, let's look at the BioNTech-Pfizer numbers as an example.

  • The vaccine and placebo groups had about 21,000 participants each.

  • There were 8 positive COVID-19 cases in the vaccine group -- that's not 5% of all participants!

  • Meanwhile, 162 of those who didn't get the vaccine got sick.

  • So 162 minus 8 equals 154 more unvaccinated people

  • who got sick compared to the vaccine group.

  • And if you divide that out, you get 95%.

  • So it's pretty simple, but the complications arise

  • when you start comparing one trial to another.

  • Because each clinical trial arrived at that efficacy number slightly differently.

  • Moderna and BioNTech looked for any symptomatic COVID-19 after two doses

  • of vaccine or placebo shots, and counted those as cases in their trials.

  • By contrast, the Janssen trial looked at efficacy

  • against moderate to severe cases.

  • And they also had different criteria for when a case counted.

  • Like, Moderna started counting cases that were contracted

  • 14 days after participants got the second shot, while it was only 7 days for Pfizer.

  • They had similar efficacy numbers, but under the hood,

  • the counting doesn't match up.

  • Not only that, but the trials all happened in different times,

  • in different places, with different strains of the virus going around.

  • This is why you can't compare the numbers from one clinical trial to another,

  • unless they were designed for it,

  • which is hard to do when all the vaccines are all being invented at the same time.

  • Talking about these numbers in the same breath is comparing apples to oranges,

  • or durians to... something else.

  • BioNTech's 95% is different from Janssen's 66%...

  • and different from Moderna's 94%.

  • So, if you're thinking, “but 95% still sounds better,”

  • well, there are a few more important points.

  • All three of these vaccines were 100% efficacious at preventing death in the trials.

  • Less death and less burden on our hospitals are among the main reasons

  • health officials are urging people to get whichever vaccine they're offered.

  • At least as long as they don't have related allergy or other health concerns.

  • And finally, there's a difference between what happens

  • in a controlled trial setting and the real world.

  • Think of efficacy as how well the vaccine works in trials,

  • and effectiveness as how it works in the real world.

  • We know one of those, but not the other -- yet.

  • There are several variables that are likely to affect how things play out.

  • Like higher or lower proportions of vaccinated people in various communities,

  • whether more contagious variants become dominant,

  • and racial and other inequities in vaccination rates.

  • Officials are keeping an eye on things.

  • And in the meantime, I'll be on the edge of my seat,

  • and wearing a mask if that seat is in public.

  • But who knew you needed so much math to understand clinical trials?

  • Well, it turns out if you want a more complex understanding of the world,

  • you need an appreciation for the numbers that underlie it all.

  • And Brilliant has everything you need.

  • From fundamentals to stats to complex numbers,

  • they've got a course to help you learn all about it.

  • If you want to check it out, you can go to Brilliant.org/SciShow.

  • And if you sign up, you'll get 20% off an annual Premium subscription.

  • [♪ OUTRO]

Thanks to Brilliant for supporting this episode of SciShow.

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it