Patient safety measures work at children’s hospitals

In recent years, children's hospitals have joined in the national push to improve patient safety and avoid preventable problems. But it has been hard to measure progress, because of uncertainty about whether standard patient safety measurement tools apply to their patients, who are younger, smaller and on average sicker than those at other hospitals.

But now, a new study of data from 67 children's hospitals in 31 states shows that many of the same indicators used in adult hospitals can be used to measure preventable complications and problems in children's hospital patients, from post-procedure infections and dangerous blood clots to bedsores.

The measurement tools, from the federal Agency for Healthcare Research and Quality, are formally called AHRQ Patient Safety Indicators, or PSIs.

However, the researchers find that two of the AHRQ PSIs are inaccurate for measuring children's care. And they say that none of the measures should be used to compare children's hospitals with one another, though the indicators can be useful in pinpointing individual patients' charts that need to be reviewed.

The study, published online early by the journal Pediatrics, evaluated information about 1.92 million children's hospital stays over four years, compiled by the National Association of Children's Hospitals and Related Institutions, or NACHRI. The team was led by a University of Michigan Medical School faculty member.

The analysis shows that children's hospitals could do a better job in areas such as preventing hospital-acquired infections, clots in intravenous lines and bedsores. These three potentially avoidable problems affected 3.5, 5.7 and 17 patients, respectively, out of every 1,000 applicable children treated at the hospitals in the study. The numbers are risk-adjusted to take into account patient characteristics.

In two other categories, the analysis appeared to reveal a disturbingly high preventable death rate among young patients. But when the researchers dug deeper into the data and looked at each case individually, they found that the standards, not the hospitals, were to blame, and that the death rates were inaccurate.

In these two categories — deaths due to complications and deaths among patients with low-risk diagnoses — the safety standards didn't take into account the very complex and risky nature of serious illness among kids. In other words, most of the deaths were brought on by factors relating to a child's underlying condition, such as cystic fibrosis or a heart defect, not by a preventable incident or mistake.

“This is the first time that the national patient safety indicators have been applied to children's hospitals specifically, and we find both encouraging news for patients and a chance to refine the analytical tools so that they fit children's care even better,” says lead author Aileen Sedman, M.D., professor emerita of pediatrics and former associate chief of clinical affairs at the University of Michigan Health System. She led the analysis with colleagues from the U-M C.S. Mott Children's Hospital, NACHRI, AHRQ, and the Children's Hospital of Wisconsin.

The AHRQ developed the standard patient safety indicators, or PSIs, in response to the 1999 report on patient safety and medical mistakes released by the prestigious Institute of Medicine. They have been applied to adult hospitals as a way to measure quality and safety of care, adjusted for the severity of illness among the patients in a particular hospital and for other specific criteria.

Based on the new results, the researchers are working with AHRQ to refine the PSIs so they more accurately reflect truly preventable incidents and deaths among sick children. But for now, based on the new study, the authors warn that the “failure to rescue” and “death in low-mortality diagnosis-related groups” standards should not be used to measure the safety of children's care.

The team also strongly cautions that their analysis showed that the rates of patient safety issues in children's hospitals are low, so that statistically valid comparison of hospitals is not always possible.

But the new analysis does show the need for very specific studies on, for example, measures to prevent hospital-acquired infections and bedsores among hospitalized children.

The authors are already working with a collaboration of children's hospitals that will share the “best practices” — such as consistent use of antibiotic-coated central lines — that have allowed them to keep their rates of preventable infections or other problems lower than average. The analysis shows that of the 43 hospitals providing data for all four years of the study period, 19 had in-hospital infection rates far below the average, and 11 had rates above the average.

Sedman notes that the new study complements one published last year by a team that looked at the safety of hospital care for newborns and children treated in general hospitals. That study, by researchers from the Johns Hopkins Children's Center and AHRQ, raised questions about the safety of certain care, and noted in particular that hospitals had high levels of birth trauma and other safety concerns for newborns and infants.

The new study used data from the NACHRI Case Mix Comparative Database, which contains information on patients treated at NACHRI member hospitals. Risk-adjustment was done by an AHRQ formula. The Hopkins-AHRQ study was based on one year of data from 27 state databases.

As a result, the two study populations were different: One-third of the children in the new study were under the age of 1 year, as opposed to 71 percent of children in the Hopkins-AHRQ study, reflecting the high level of newborns in general hospitals. Most children's hospitals do not have a maternity service. And 94 percent of children in the new study were treated at teaching hospitals, compared to 50 percent of children in the other study.

The new study's design also allowed the researchers to examine the complete coded diagnoses of each case in which the initial analysis showed a patient safety issue. This allowed them to see what other conditions a child had, and what might have contributed to a death, or made a child vulnerable to infection.

“That in-depth analysis was what revealed that children who died of sepsis or acute renal failure, for example, were almost always children with multiple severe conditions that together would be fatal,” Sedman says. “This suggests that the deaths were due to a combination of factors, and might not be preventable no matter what a hospital did.”

In the same way, she notes, the in-depth analysis revealed that many of the children who got infections as a result of their hospital care were kids with abnormal gastrointestinal systems — who are likely to need prolonged intravenous therapy.

And of the children who developed bedsores (called decubitus ulcers by doctors), many were found to have low blood flow to their skin because they were on heart-lung machines or had had heart surgery, or to have limited movement due to neurological problems. This suggests an opportunity to develop specific approaches to prevent bedsores from developing in these types of young patients.

“The bottom line is that the AHRQ patient safety indicators give us a good start toward assessing the safety of care at children's hospitals, allowing children's hospitals to pull specific charts to review cases, and allow us to test patient safety interventions,” says Sedman. “Now, the collaborative team is working with AHRQ to continue to improve the indicators for children.”

In addition to Sedman, the research team included J. Mitchell Harris, Kristine Schulz, Ellen Schwalenstocker from NACHRI, Denise Remus from AHRQ, Matthew Scanlon of the Children's Hospital of Wisconsin, and Vinita Bahl of U-M.