Ninety five percent. NINETY FIVE PERCENT!

This is part 2 in a series covering the story of the COVID vaccines in the US, looking back at how well they have worked and why the communication around them has been confusing. Read part 1 here.

It was late November 2020, and we were just about to start rounds at the hospital when the news broke. A press release had just come out, reporting the final results of the phase III trial for the first COVID vaccine: 95% efficacy.

NINETY FIVE PERCENT! It cannot be overstated just how good this news was, and just how much it exceeded the scientific community’s expectations.

We didn’t know if COVID vaccines were going to work at all (other attempts at making vaccines against different coronaviruses had proven difficult in the past). We didn’t know how long it would take to get any sort of effective vaccine. And, for the first ever COVID vaccine created, the scientific and medical communities were expecting one with much lower efficacy. The FDA had set a threshold of 50% efficacy for a COVID vaccine to be considered for public use. With over a quarter million people dead from COVID in the US alone, we would have been thankful for a vaccine that reduced death by even just half. Many doctors and scientists were expecting efficacy in the 50-70% range. Dr. Fauci said that a vaccine with 70% or 75% efficacy would be “terrific.”

Instead, all these expectations were blown out of the water. Not 50%, not 75%, but 95%. That’s ridiculously good.

And just a few weeks later, Moderna reported a similar vaccine efficacy for their mRNA vaccine. Not just one, but two highly effective vaccines in less than a year since the pandemic started. What?!?

Very, very good. But not perfect.

The results of the Moderna and Pfizer COVID vaccine trials were truly exceptional – better than doctors and scientists dared to hope. However, in the midst of the fully justified enthusiasm about these vaccines, groundwork was laid for a communication blunder that later left many people feeling confused and betrayed. To understand what happened, let’s do a quick refresh on where that 95% number came from.

Vaccine efficacy is a measure of how much a vaccine reduces the risk of a certain outcome. It’s not the probability you won’t get the disease, nor is it a measure of how many people out of 100 are guaranteed not to get sick. It’s a measure of risk reduction. If an unvaccinated person has a 10% chance of developing a particular outcome from a particular virus, a vaccine with 95% efficacy will reduce that risk to 0.5%. (10%-0.5%/10% = 95% risk reduction).

The first question you should ask yourself whenever you see efficacy reported is efficacy against what? Whenever vaccine efficacy is measured, it is measured for a specific clinical outcome. That could be symptomatic infection (a person has tested positive for the disease and has symptoms from the disease), it could be asymptomatic infection (they tested positive but have no symptoms), or it could be severe disease (they were hospitalized) or it could be death. For the same vaccine, the efficacy may be different depending on which outcome you’re looking at. (And it can vary depending on the variant of the virus, but we’ll get to that later.) The COVID vaccine clinical trials were designed with a specific outcome in mind: symptomatic infection (and that’s what the 95% efficacy number is referring to).

A second outcome was also recorded: severe disease (e.g. hospitalization). In the Moderna trial, 30 people developed severe COVID (and one died): all of them were in the placebo group. Nobody developed severe disease in the vaccine group. This lead to a statistic that became widely circulated: 100% efficacy against severe disease and death.

Source here.

Soon this became a talking point for all the COVID vaccines available in the United States: “no matter which vaccine you get, all three vaccines are 100% effective in preventing severe disease and death.”

100% sounds really good. Perfect, in fact. If I were not a doctor or scientist and heard this statistic, I would assume that this statistic means that if I get vaccinated, I’m essentially guaranteed not to get severe COVID. And that is what many people thought. 

To add to the confusion, some people seemed to mix up different efficacies, saying that if you got vaccinated, you wouldn’t get COVID period (implying 100% efficacy against even mild COVID, something the studies definitely did not support.)

You can see how miscommunication started to happen. The scientific community was overjoyed with these results, having expected something far less exceptional. The vaccines truly were (and are) extremely good. But some of the messaging around them started to make them sound perfect. And no vaccine is ever perfect.

Where did the '100% effective' statistic come from?

It is now obvious that while the COVID vaccines have reduced the risk of severe disease and death tremendously, they are not 100% effective. Vaccinated people have been hospitalized with COVID, and vaccinated people have died of COVID. Vaccines reduce the risk significantly, but they do not completely stamp it out.

So what happened? Was this statistic wrong? Were the studies wrong? Was this an exaggeration to convince people to get vaccinated?

In the next post, we’ll dig into where this number came from and why it later caused so much trouble.

This is the second post in a series covering the story of the COVID vaccines in the US and why their effectiveness has been so confusing. To see future posts, subscribe here.