Name: Peer Review and the Quest for Truth: Crash Course Scientific Thinking #4
Uploaded: 2026-02-17T22:31:33.000Z
Duration: 10 min 44 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

Modal verbs of deduction & speculation

Alien life may have been discovered right here on Earth.

Or at least that's what some headlines seemed to indicate in 2010 after a NASA backed study was released.

Well, the truth is out there, but as always, it's more complex.

Hi, I'm Hank Green and this is Crash course scientific Thinking.

Now, I'm not saying I don't believe in aliens.

The universe is vast and full of opportunities for life.

Also, I would never say that because they might be listening.

But let's clear up a big misconception right at the top.

This study, which was published in the prestigious journal Science in 2010 and reported by lots of news outlets, never actually claimed to find aliens.

That was an unfortunate miscommunication on the part of the media.

What the study actually claimed to find was arsenic based life.

Which admittedly sounds a lot more boring than aliens, but

if it were true, it would be very revolutionary.

Unfortunately, the claim did not hold up under closer examination of the evidence.

After the original announcement, new headlines ran stating that actually maybe we didn't find anything all that strange as a science person on the Internet, but

also like a regular person trying to navigate my own media feed.

It's frustrating when we are given contradictory science stories, but

there are lots of reasons why stories like this one are more nuanced than they come off in the news.

We'll talk a lot more about how science gets communicated in the news in a later episode.

But today I want to tell you about how this study was received by the scientific community and why it serves as a great example of one of science's biggest self correction.

The research was done in California's salty, arsenic filled Mono Lake, which for a poison loving microorganism is like the Amalfi Coast.

Researchers found this bacterium, GFAJ1, which seemed like it might replace phosphorus with arsenic in its molecular makeup, which would be super weird and cool because the fundamental building blocks of life as we know it are carbon, hydrogen, nitrogen, oxygen, sulfur and phosphorus.

This would be like making a smoothie with yogurt, berries, greens and glass.

So the team set up an experiment where they put that little GFAJ1 bacteria in a salty lake water environment that had very little phosphorus and lots of arsenic.

That arsenic laced water shouldn't have supported life.

It grew and grew, seemingly on arsenic alone.

After all their hypothesizing observing and experimentation, the team felt like they had made a breakthrough so they wrote about their findings in a super exciting study called a bacterium that can grow by using arsenic instead of phosphorus.

But here's the the scientific process doesn't stop with the writing.

In order to get published in a real academic journal, articles typically go peer review, a system where experts check one another's work to make sure it meets the standards for scientific quality.

Reviewers assess whether a study's methods are sound and if the findings actually support the conclusions.

The peer review process makes good work better by bringing up ideas and concerns researchers might never think of on their own.

But what actually happens when a paper is undergoing peer review?

Like let's say that I did my own experiment and wrote a paper about it.

In my secret laboratory in my basement, I tested out a new kind of eco friendly sunscreen and found that it works better than all the other sunscreens on the market.

How do I get that paper into a scientific journal and share my legitimized findings with the wider world?

I think that it might be time for some sage advice.

Well, there are a few things that need to happen.

Then the journal will send your article to a team of scientists who work in your same field for your study.

There'll probably be people like photobiologists and dermatologists, people who know sunscreen and skin disease.

It wouldn't make sense to send a paper about sunscreen to an astronomer.

Your peer's job is to answer a big but basic question.

Is what I'm reading actually supported by the evidence provided?

They'll also ask questions like is this study relevant to the field?

Does it effectively build on the current state of research?

Or does it overlook known sunscreen related issues?

They'll also want to find out if you use sound methods.

Did you measure the results in trustworthy ways?

And do the results really imply what you say they do?

So like, if hypothetically my sunscreen wasn't quality controlled, or my sample size was too small, or my statistical analysis was whack, which of course it never would be, but

if it was, then that's the reviewer's job to catch those mistakes.

Yes, but again, only as it relates to the very specific field of sunscreen research.

Like if you mixed up two very common sunscreen ingredients in your tests, they'd want you to know about it because they're experts in sunscreen.

And peer review is all about field specific experts reviewing each other's work.

The whole process can take some time, so the world may have to wait for the latest by Hank Green et al.

And even then, there will be more peer review that happens after it's published.

So it's hard to get past that initial peer review.

Scientists have to make sure they've checked all the right boxes and that their research is solid.

Which matters because the big collaborative effort of science depends on trust and really good scrutiny.

New researchers base their work on earlier discoveries and peer review helps make sure those discoveries are supported by evidence.

But sometimes, despite our best efforts, papers do make it into publication with flawed data.

Which is what happened with the arsenic based life study.

As Sage mentioned, peer review doesn't end with journals.

And in fact, it's only after a study is published that the most intense peer review begins.

A published study is one of the main ways scientists present their work to other scientists in their field.

Those other scientists then approach the same question from different angles in order to test the findings themselves.

After a study is published, scientists might try to replicate its results using the same data as the original study to see if they arrive at the same conclusions.

Others might try a different experiment using the same methodology, but

with all new data basically asking if someone else does this study, will they come to the same conclusion?

And in the case of groundbreaking arsenic based life, they didn't.

Other scientists tested that same strain of GFAJ1 and tried to grow it just like the original experiment on arsenic instead of phosphorus.

The new studies found that while the bacteria could grow in a low phosphorus environment, it couldn't grow with no phosphorus at all.

Which, while interesting, was not revolutionary.