How well do the COVID vaccines work? (and why has this been so confusing?)

The COVID vaccines are undoubtedly among the most impressive medical feats in history. One model estimated they have saved millions of lives in the United States alone. As a physician-scientist, watching the scientific world come together to produce not one but multiple vaccines in a matter of months in the midst of a global pandemic has been truly awe-inspiring.

But just how well do they work? This question is not as simple to answer as it may seem. Why have there been so many breakthrough infections? Why have some vaccinated people died of COVID? Why are vaccinated people wearing masks? Do the vaccines work at all against omicron? If they really work, then why do the vaccinated care if other people don’t get vaccinated? Are they really even vaccines?

“Unity” is probably not the word that comes to mind when we think about the pandemic, but there is one thing that most people seem to agree on: the communication during the pandemic has been confusing. This is particularly true for vaccine effectiveness.

So let’s dig into it. I am both a scientist and a science communicator, and over the past two years I’ve had a foot in both worlds: what are the scientists saying, and how is the public perceiving it? (Hint: they are often not the same). In this series of post, I’m going to tell you the story of the COVID vaccines, from both sides, and dissect where the communication broke down. And along the way, we’ll answer the question: just how well do the vaccines work?

Expectation Management

“Expectation management” is one of my husband’s favorite phrases. He says it’s essential to success in both professional and personal relationships (and he’s right). Expectation management means telling people what to expect ahead of time, so they aren’t disappointed by unspoken (and perhaps unrealistic) expectations down the road. When our expectations are set appropriately, we’re far more likely to be satisfied with the outcome, even when it’s not perfect.

So what were the expectations set for the COVID vaccines?

To start, let’s look at the bar set by the FDA, back in 2020. On June 30, 2020 the FDA released guidance on what standards COVID vaccines must meet in order to gain FDA authorization. Vaccine development had already begun for many different COVID vaccines, and the FDA was establishing the bar these vaccines would have to meet in order to be considered for use in the general public.

In addition to strict standards around vaccine safety, they issued a minimum requirement for vaccine effectiveness: “the FDA would expect that a COVID-19 vaccine would prevent disease or decrease its severity in at least 50% of people who are vaccinated.”

So that was the bar: 50% efficacy. And that’s not just talking about efficacy against infection. Even if a vaccine didn’t reduce the risk of infection at all, but reduced the risk of severe disease or death by 50%, it would have met the FDA’s threshold for efficacy.

Isn't 50% efficacy kind of low?

If you’re thinking this seems low compared to other vaccines, you’re both right and wrong. The flu vaccine, which is updated every year to match circulating flu strains, has an efficacy of 40-60%, so fairly similar to the minimum bar set by the FDA for COVID vaccines.  But if we compare to many of the vaccines we get in childhood, then 50% is quite a bit lower. The two dose MMR vaccine series is 97% effective against measles, and 88% effective against mumps. At least 3 doses of the polio vaccine is 99-100% effective against polio. That’s much higher than 50%. 

But if you consider that we were at the beginning of a global pandemic that was killing thousands of people every day, then reducing the risk of death by 50% would be a major win. It may seem low relative to the efficacy of many of our childhood vaccinations, but even a vaccine that reduced disease severity by half would save thousands of lives.

So the bar was set. And the scientists and doctors who were following vaccine development had their expectations managed: anything above 50% efficacy would be considered by the FDA, and if authorized, we would have a new major tool to help us fight SARS-CoV-2.

Were the public's expectations set?

Did the public know that 50% efficacy was the bar set by the FDA? While this press release was covered in a few news outlets, in June 2020, there really wasn’t a lot of discussion about what vaccine efficacy meant. When it came to the pandemic, there was so much other news drawing our attention: states were “reopening,” people were arguing about hydroxychloroquine, videos were going viral about how the pandemic was planned, and debates about masks abounded, just to name a few. Vaccine efficacy, and the nuances of what it meant, wasn’t really on the public’s mind yet. We didn’t know if we were going to get a vaccine in six months or three years, never mind the details about how well it works. Some scientists were talking about it, but it wasn’t a primary focus of attention.

And this, I think, illustrates one of the major challenges with communication during the pandemic. There has been so much new information coming so fast, that to cover the current news is challenging enough. To anticipate future challenges (like setting expectations about how well a new vaccine might work for current and future unknown variants) is even more difficult. Even when scientists have the foresight to do this, when the public is going through so much in such a short period of time (recall how much of rollercoaster 2020 was), getting their attention for future unknowns is pretty challenging. It’s hard to set people’s expectations for next year when there is crisis after crisis happening right now.

Only a few months later, that 50% number was quickly forgotten. In fall 2020, the first Phase III COVID vaccine trials began reporting their results. I still remember where I was when we heard the news. We were on morning rounds at the hospital, and the press release came out for the first mRNA vaccine reporting efficacy against symptomatic infection: it was WAY higher than 50%, far surpassing our expectations.

After a truly horrible year, finally some hope was on the horizon.

Continued in part 2.

This is the first post in the series covering the story of the COVID vaccines in the US and why their effectiveness has been so confusing. To see future posts, subscribe here.