Online reviews of health care facilities drop significantly after COVID-19 pandemic

After the COVID-19 pandemic struck, online reviews of health care facilities dropped significantly, and they have not yet fully recovered, according to a new analysis led by researchers at the Perelman School of Medicine at the University of Pennsylvania. More than half of reviews on the online platform, Yelp, now are negative, flipping the pre-COVID picture. The findings are published today in JAMA Network Open. 

Online reviews can tell us information about the patient experience that traditional reporting metrics, like hospital-administered patient experience surveys, might miss. These reviews can help hospitals understand what matters most to patients and their support networks in near real time." 

Neil Sehgal, ME, study's lead author, associate fellow in the Penn Leonard Davis Institute of Health Economics

By analyzing all reviews of health care facilities in the United States on the online platform Yelp dated from 2014 through 2023, Sehgal, co-author Anish Agarwal, MD, an assistant professor of Emergency Medicine, and their team saw that the percentage of positive-;four- and five-star-;reviews dropped from 54.3 percent before March 2020 (marked as the beginning of the COVID pandemic in the United States) to 47.9 percent after. 

In fact, from the latter half of 2021 on, the researchers found that positive reviews were never more than 50 percent. 

"We analyzed Yelp reviews of health facilities nationwide-;which includes hospitals, urgent care centers, doctors' offices, and more," said Sehgal. "Our analysis shows how public perception of health care changed after COVID-19. And with this information, hopefully, health care professionals can work to improve upon what's been driving these opinions."

Feedback ups and downs

In analyzing the reviews, the researchers used a language processing technique to tease out the most common topics from the reviews, organized them into themes, and measured how they changed over time. 

The themes that saw the greatest change in mentions between the pre- and post-COVID periods were "insurance and billing issues" and "customer service and staff behavior." 

One theme mentioned less in negative reviews after the arrival of COVID-;and a renewed, supercharged focus on hygiene: Facility cleanliness. 

Trends emerge along ethnic and geographic lines 

While positive reviews of health care facilities on Yelp, as a whole, declined over time, health care facilities in rural areas already had lower ratings at the start of the time period examined, and the differences became more significant after COVID struck. Post-COVID, rural facilities were 23 percent less likely to have positive reviews, compared to health facilities in urban areas, which were 7 percent less likely to have positive reviews. 

Looking at data on race, researchers found that health facilities in the sample of neighborhoods that they analyzed that had higher proportions of Black or white residents were more likely to have significant declines in positive reviews.

But when it came to areas with higher-than-average Hispanic populations, facilities still saw a decline in positive ratings, though the dips were not as pronounced.

"At the moment, we cannot understand what is driving these changes among different populations," Agarwal said. "But this could help us think critically on what questions to ask next and how to better deliver care around the nation in the future."

Researchers also identified ethnic correlations around the most mentioned complaints. Reviews for facilities in areas with higher Black and/or Hispanic populations criticized "insurance and billing issues" more often, while the facilities in areas with higher white populations found fault with "wait times" more often.

"Many of the themes have clear overlap but are distinct across geographic areas, which is interesting and begins to highlight a growing need for health systems and facilities to be more intentional and tailored to their local communities' needs-;and opinions," said Agarwal.

Moving forward, the researchers hope to look deeper into the content of the reviews to better understand how attitudes and opinions break down along socio-ethnographic lines, giving health care professionals better tools for discussing issues that are important to individuals and assuaging concerns. 

This study was partially funded by National Institutes of Health grants (NIH NHLBI R01HL1-141844, NIH/DHHS R01 MH127686, NIH-NIMHD:R01MD018340, and NIH K24 HL157621).