Mining a new data set to pinpoint critical staffing issues in skilled nursing facilities

The Centers for Medicare & Medicaid Services is not known for linguistic playfulness. Nonetheless, at least one person there must have been chuckling when it named its rich new data source for nursing home staffing levels the Payroll-Based Journal, or PBJ.

Like that classic sandwich, the PBJ data set is irresistible. CMS created it to fulfill a requirement of the Affordable Care Act to improve the accuracy of its five-star staffing ratings on Medicare's Nursing Home Compare website. The data set contains payroll records that nursing homes are required to submit to the government.

In April, Medicare began using them to rate staffing for more than 14,000 skilled nursing facilities (SNFs). The PBJ data gives a much better look at the how staffing relates to quality of care than the less precise — and too easy to inflate — staffing data Medicare had been using since 2008, which were based on two-week snapshots of staffing homes provided to inspectors. The data show staffing and occupancy on every day — an unprecedented degree of granularity that allows for new levels of inquiry.

Low staffing is a root cause of many injuries in nursing homes. As I wrote in the article published in The New York Times based on the data: "When nursing homes are short of staff, nurses and aides scramble to deliver meals, ferry bedbound residents to the bathroom and answer calls for pain medication. Essential medical tasks such as repositioning a patient to avert bedsores can be overlooked when workers are overburdened, sometimes leading to avoidable hospitalizations."

Analyzing the files took time. To start with, the files were too large to analyze in Excel — a challenge with more and more health care data sets. I previously had taken classes in SQL at the National Institute for Computer-Assisted Reporting, but this was the first project in which I had to use SQL. Fortunately, KHN has two great data journalists who steered, taught and backstopped me: data editor Elizabeth Lucas and data correspondent Sydney Lupkin.

There were a lot of decisions to make in the analysis. Should we count nursing administrators in assessing staffing levels? How should we define best-staffed and worst-staffed days? How do we handle the fact that the data in its raw form wasn't risk-adjusted — that is, that it didn't tell you how comparatively frail and sick each nursing home's residents were, making it dangerous to make comparisons. (You can read our decisions in our methodology, which accompanies the downloadable data.)

I guided myself with two intersecting principles: to reflect residents' lived experience as accurately as possible, and to be fair to the facilities. When in doubt, I erred on the side of caution.

I bounced approaches off not only Liz and Sydney, but two of the top health care researchers who have published studies on staffing: David Stevenson at Vanderbilt University Medical Center and David Grabowski at Harvard Medical School. Both are straight shooters and rigorous in their methods. It was also invaluable to talk early on in the project to the nursing home industry's trade group, the American Health Care Association, to solicit its views about the strengths and weaknesses of the data set.

I've been writing about nursing homes for a while, but until this story I didn't know the source value of SNF resident and family councils. These councils are authorized by CMS, and facilities must let them meet on their premises and invite to their meetings anyone they want, including journalists.

The leaders of the council at Beechtree Center for Rehabilitation & Nursing, the Ithaca, N.Y., home that provided the anecdotes for the story, were a reporter's dream: smart, dedicated and reasonable. They were able to see things from the facility administration's position, but also were firm advocates for good care. They had educated themselves about the facility and the rules and regulations. Adam Chandler, whose mom was in Beechtree and whom I quoted in the story, had already been studying the PBJ data. The council's president, Stan Hugo, carried around a tan notebook where he jotted down incidents of staffing problems he came across. He also kept a daily log of how many employees he observed when visiting his wife and how many the administration said were on duty in the sign Medicare requires them to post publicly.

Our story ultimately focused on the wide fluctuations in nursing home staffing day to day, with staffing consistently plummeting on weekends. Caitlin Hillyard, one of KHN's web producers, created a terrific interactive graphic that reporters can use to see staffing patterns in nursing homes in your state. We encourage other reporters to download and use the data for their own stories. There's no cost — just please credit KHN and provide a link back to us.

KHN's coverage of these topics is supported by John A. Hartford Foundation and The SCAN Foundation