Study: Magazine's rankings of hospitals reflect reputation, not necessarily quality

Apr 21 2010

The Cleveland Plain Dealer: "Hospital rankings shouldn't be a popularity contest. But U.S. News & World Report's annual Best Hospitals issue is just that -- and not a reflection of the quality of care that is provided -- says a study conducted by a local physician. Curious to find out how much of the magazine's rankings came from objective measures and how much from subjective measures, Dr. Ashwini Sehgal [of Case Western Reserve University] spent three months analyzing the data" on 12 speciality rankings.  

"Seventy-five percent of the ranking is based on reputation, Sehgal said. As a result, nearly 100 percent of a top-five hospital's ranking is based on reputation, not from objective measures, he said. ... 'There is virtually no relationship between reputation and quality of care. If we used that approach for professional football, we'd say, 'Let's just award the Super Bowl to the Dallas Cowboys because they have the reputation of being 'America's Team'" (Townsend, 4/19).

Bloomberg BusinessWeek: The "objective components of the total score for ranking the hospitals -- death rates, nurse-to-patient ratios and patient safety -- were taken from the American Hospital Association or from Medicare, the federal health program for people age 65 or disabled, and other data sources. The objective scores showed less variation than reputation did among all hospitals in the survey, Sehgal said."  

Avery Comarow, the U.S. News editor for the hospital rankings, said "the magazine has taken steps to reduce the influence of reputation in the rankings. 'We get criticized a lot for putting emphasis on reputation. Since we're looking at hospitals that do really well with the most critical patients, there's a very good case to be made that the reputational survey is a valid and legitimate form of peer review,' Comarow said. 'We will take a further step with the rankings that come out this year. But I would not propose to eliminate it'" (Ostrow, 4/19).

The study appeared in the Annals of Internal Medicine and the author notes a limitation: "The findings apply primarily to interpretations about the relative standings of the 50 top-ranked hospitals in each specialty and not necessarily to the hundreds of unranked hospitals" (Sehgal, 4/20).