Publicly released report cards do not appear to improve hospitals' performance, says new study

An analysis of quality of cardiac care following the public release of data on measures of care at hospitals in Ontario, Canada, did not result in significant systemwide improvement in hospitals' performance on most quality of care indicators, according to a study to be published in the December 2 issue of JAMA. The study is being released early online because of its presentation at an American Heart Association scientific conference.

"Public release of hospital performance data is increasingly being mandated by policy makers with the goal of improving the quality of care. Advocates of report cards believe that publicly releasing performance data on hospitals will stimulate hospitals and clinicians to engage in quality improvement activities and increase the accountability and transparency of the health care system. Critics argue that publicly released report cards may contain data that are misleading or inaccurate and may unfairly harm the reputations of hospitals and clinicians," the authors write. "Although there has been considerable debate, few empirical data exist to determine whether publicly released report cards on hospital performance improve the overall quality of care provided."

Jack V. Tu, M.D., Ph.D., of the Institute for Clinical Evaluative Sciences, Toronto, and colleagues conducted the Enhanced Feedback for Effective Cardiac Treatment (EFFECT) trial to determine whether publicly released report card data could improve the quality of cardiac care delivered. The study included 86 hospitals in Ontario, Canada, with patients admitted for acute myocardial infarction (AMI; heart attack) or congestive heart failure (CHF). The researchers chose to focus on hospitals that treat patients with these conditions because of considerable evidence of a large gap between actual and ideal practice patterns. Participating hospitals were randomized to early (January 2004) or delayed (September 2005) feedback of a public report card on their performance at the beginning of the study (between April 1999 and March 2001) on a set of 12 process-of-care indicators for AMI and 6 for CHF. Follow-up performance data (between April 2004 and March 2005) also were collected.

The researchers found that after the public release of the results for the early feedback group the composite AMI process-of-care indicator did not improve significantly in the early feedback group compared with the delayed feedback group (absolute change, 1.5 percent). They note that only the percentage of patients receiving fibrinolytic therapy prior to transfer to a coronary care or intensive care unit improved significantly more in the early feedback group.

There was also no significant improvement in the composite CHF process-of-care indicator (absolute change, 0.6 percent) in the early feedback group after the public release of the report card. "The absolute rate of angiotensin-converting enzyme (ACE) inhibitor and angiotensin II receptor blocker use in patients with left ventricular dysfunction increased by 5.9 percent, but this was the only indicator that improved significantly more in the early feedback group," the authors write.

They add that during the follow-up period, the average 30-day AMI mortality rates were 2.5 percent lower in the early feedback group compared with the delayed feedback group. The hospital mortality rates for CHF were not significantly different.

"The process-of-care findings suggest that public release of hospital-specific performance data may not be a particularly effective systemwide intervention for measurably improving processes of care for either AMI or CHF," the researchers write. "- the EFFECT study data likely stimulated some important local, hospital-specific changes in delivery of care that may have contributed to the better outcomes observed at the early feedback hospitals. Policy makers and clinicians may wish to consider the findings from the EFFECT study in the design and evaluation of future public reporting initiatives. Greater attention to developing common strategies across hospitals for addressing report card results might enhance the systemwide effectiveness of future report cards."