What do we really know about vaccine effectiveness?

Reviewed by Emily Henderson, B.Sc.Nov 2 2021

The politicization of covid vaccines — and, well, just about everything else having to do with the pandemic — has led to confusion, if not utter fatigue.

And some posts circulating on social media — this slickly edited piece on YouTube, for example — seem to build on these feelings, attempting to cast doubt on the effectiveness of the vaccines. This one intersperses comments from White House medical adviser Dr. Anthony Fauci extolling their protectiveness with screenshots of news headlines, starting with those citing 100% effectiveness, then moving through others reporting sharply lower percentages. Set to the rapidly increasing tempo of the orchestral piece "In the Hall of the Mountain King," the video ends with headlines about drug company profits.

But slowing the video to parse the headlines reveals more complexity. Some are reporting on studies that looked only at infection rates; others, more serious outcomes, including hospitalization and death. Some are about vaccines not offered in the U.S.

In short, the video fosters misperceptions by mixing together dissimilar data points and leaving out key details.

Still, one can't help but wonder what's really going on with effectiveness — and is any of it a surprise?

If you don't read any further, know this: No vaccine is 100% effective against any disease. The covid shots are no exception. Effectiveness in preventing infection — defined as a positive test result — appears in some studies to wane sharply the more time that goes by after completing the one- or two-shot regimen. But on key measures — prevention of serious illness, hospitalization and death — real-world studies from the U.S. and abroad generally show protection weakening slightly, particularly in older or sicker people, but remaining strong overall, even with the rise of the more infectious delta variant of the covid virus.

The bottom line? Getting vaccinated with any of the three vaccines available in the U.S. reduces the chance of getting infected in the first place, and significantly cuts the risk of hospitalization or death if you do contract covid-19. The Centers for Disease Control and Prevention recently published a study showing fully vaccinated people were more than 10 times less likely to die or be hospitalized than the unvaccinated.

"When it comes to what matters, vaccines hold up really well," said Dr. Amesh Adalja, an infectious-disease physician and senior scholar at the Johns Hopkins Center for Health Security. "They were designed to tame the virus."

So, what do "efficacy" and "effectiveness" mean, anyway?

Before a drug or vaccine is greenlighted by federal regulators, it is tested on volunteers randomly assigned to get either the product or a placebo. Then researchers compare how the groups fare. In the case of a vaccine, they look at how well it prevents infection, and whether it protects against serious illness, hospitalization or death. Those clinical trial results are often referred to as efficacy measures.

In the real world, however, a drug or vaccine's performance is affected by numerous factors, including a much larger population receiving it, some of whom have underlying conditions or socioeconomic circumstances different from those in the clinical trial. That real-world performance measure is called effectiveness.

When authorized for emergency use following clinical trials, both the Pfizer-BioNTech and Moderna two-dose vaccines reported efficacy against symptomatic illness in the mid-90% range. The Johnson & Johnson single-dose shot — which was tested later, when there were more variants — reported overall efficacy in the high 60% range. Those numbers exceeded the 50% threshold health officials sought as a minimum for covid vaccine efficacy. Keep in mind, also, that the annual influenza vaccine's real-world effectiveness is often 40% to 50%.

Another point: 95% effectiveness doesn't mean 95% of vaccinated people will never get infected. What it means is that a fully vaccinated person exposed to the virus faces only 5% of the risk of infection compared with an unvaccinated person.

Have the effectiveness numbers changed?

Yes, decline in effectiveness against infection is seen in some studies. A few have also raised concerns that protection against serious illness may also be diminished, particularly in older people and patients with underlying medical conditions.

Reasons for the decline vary.

First, when the vaccines were authorized, much of the U.S. was under tighter pandemic-related stay-at-home rules. Nearly a year later, restrictions — including mask rules — have loosened in many areas. More people are traveling and going into situations they would have avoided a year ago. So, exposure to the virus is higher.

Some studies from the U.S. and abroad show that time elapsed since vaccination also plays a role.

The Lancet recently published a study of more than 3.4 million Kaiser Permanente members, both vaccinated and not, reviewing the effectiveness of the Pfizer vaccine. It showed an overall average 73% effectiveness against infection during the six months after inoculations, and an overall 90% effectiveness against hospitalization.

But protection against infection declined from 88% in the month after full vaccination to 47% at five to six months. Time since vaccination played a larger role than any changes in the virus itself, the researchers concluded.

"It shows vaccines are highly effective over time against severe outcomes," said report lead author Sara Tartof, an epidemiologist with the Department of Research and Evaluation for Kaiser Permanente Southern California. "Against infection, it does decline over time, something that is not unexpected. We have boosters for many other vaccines."

The virus, too, has mutated.

"Along came delta," said Dr. William Schaffner, a professor of preventive medicine at Vanderbilt University School of Medicine. "Because this virus was so highly contagious, it changed the outcomes slightly."

And some vaccinated people can fall seriously ill with covid, or even die, especially if they have an underlying medical problem, as was the case with Gen. Colin Powell. He died of covid complications even though he was fully vaccinated — likely because he also had a blood cancer called multiple myeloma, which can lower the body's response to an invading virus as well as to vaccination.

What should we make of these changing numbers and the recent authorization of booster shots?

Most scientists, researchers and physicians say the vaccines are working remarkably well, especially at preventing serious illness or death.

And it's not unusual to need more than one dose.

Vaccines for shingles and measles both require two shots, while people need to be revaccinated against tetanus every 10 years. Because influenza varies each year, flu shots are annual.

Immune response is often better when vaccines are spaced apart by a few months. But during the rollout of the covid vaccines, so many people were falling ill and dying of covid each day that the Food and Drug Administration and CDC decided not to delay, but to authorize the first and second doses within about a month of each other.

"We learn as we go along," said Schaffner. "It was always anticipated there might have to be follow-up doses."

Now, the recommendations call for a second dose for anyone who received a J&J shot at least two months prior. For those who received the two-dose Pfizer or Moderna vaccine, the recommendation is to wait six months after the second dose to get a booster, which is currently recommended for those who are 65 and older; have any of a variety of underlying health conditions; live in congregate settings, such as nursing homes; or have jobs that put them at higher risk. The booster recommendations may expand in the coming months.