New book suggests ways to reduce medical reversals

University of ChicagoOct 8 2015

Medical reversal—when accepted medical interventions are abandoned because they are found to be ineffective—is the "most important problem in medicine today," according to the authors of a new book: Ending Medical Reversal.

It is "hardly the only problem," say the authors, Vinayak K. Prasad of the National Cancer Institute and Adam S. Cifu of the University of Chicago, "but it is a common channel through which many of the problems run."

Medical reversal occurs when interventions, such as medications, procedures or diagnostic tests, are adopted without a robust evidence base. Reversal is distinct from replacement, when a good therapy is replaced by a better one.

Reversals are distressingly common. In previously published research, Cifu and Prasad combed through the most prestigious American medical publication, the New England Journal of Medicine. They found that from 2001 through 2010, 40 percent of the articles about a new or recently adopted medical practice described a "clear-cut reversal." Only 38 percent of the articles affirmed the benefit of a new practice.

"It is not because a drug or procedure worked and then stopped working or because someone discovered a harm that no one else had noticed," Cifu said. "It's because the practice never worked. We were wrong all along."

Sometimes—when faced with limited evidence and a genuine need for better therapies—doctors are tempted to adopt a promising procedure or prescribe a new drug before they have robust clinical data. "This is one reason," the authors wrote, "why health care costs are soaring without improving people's health."

In just over 200 pages, plus a 30-page list of 146 recent reversals, Prasad and Cifu describe and assess the scope of the reversal phenomenon. They suggest ways to prevent, or at least reduce the frequency of such inadvertent, confidence-shattering blunders, by offering a series of seemingly sensible reforms.

Their suggestions address medical education, the funding of research, and drug approval. They also outline how patients might do a better job discussing treatment recommendations with their doctors.

They provide a lucid, accessible and often humorous description of reversals and why they continue to occur in the age of "precision medicine." They explain the hierarchy of medical evidence and the many types of clinical trials that can either advance, or sidetrack, medical progress.

The authors have opinions. They display strong feelings about research that relies on surrogate endpoints for trials and aggressive promotion of various screening tests.

Surrogate end points can be found at the heart are many reversals. These are stand-ins, accessible substitutes for more substantive, but difficult to collect, clinical data. An example of a surrogate endpoint is relying on bone density in a trial of a medication for osteoporosis as a stand in for bone fractures. People do not benefit from increases in bone density while they do benefit from avoiding fractures. Surrogate endpoints make perfect biological sense and they often correlate with a genuine clinical endpoint, but "recent history has given us ample reason to be skeptical," Cifu said.

One type of reversal that particularly concerned the authors is those that involve screening tests. Since these tests are performed before people get sick, an ineffective test can convert millions of healthy people into patients. Screening tests for breast or prostate or colon cancer are designed to find cancer early, lower the rate of dying from that cancer and, most importantly, improve overall survival.

The authors argue that although the tests do well finding cancers, the evidence that these tests prevent cancer death, or death from any cause, is often "pretty weak." So far, the only screening test that scores well on overall survival is the newest, CT scanning of heavy smokers for lung cancer.

Doctors are not the only ones prone to recommending therapies that do not work. Most Americans, for example, take some sort of dietary supplement to ward off a vitamin or mineral shortage. The fact that we seldom, if ever, see rickets, scurvy, pellagra, or other vitamin-deficiency syndromes in healthy people does not dissuade people from assuming supplements will somehow help.

"We tend to begin these therapies without evidence but only a good story," Cifu said. "And the story is often one suggested by someone likely to profit from the therapy's use."

We put our faith in medical science even when there is "no good science to put our faith in," the authors wrote. The harms caused by the consequent reversals are threefold. When a therapy is widely deployed before there is good evidence that it works, patients are put at risk. If the therapy turns out to be ineffective or worse, patients suffer. Third, reversals undermine our faith in medical science.

To help readers understand how reversals happen, and how to better understand the evidence that physicians ought to rely on, the authors include an "interlude on evidence," two chapters devoted to explaining the strengths and weaknesses of different types of clinical trials. They stress the importance of randomized controlled clinical trials as the gold standard and suggest that the RCT is "arguably the most important medical technology of the 20th century."

Even such trials are not tamper-proof, however. The rise of the pharmaceutical industry—which relies on such trials for drug approvals—as a major sponsor of clinical research has forced physicians to examine trial results with greater scrutiny.

Drug and device makers, for example, find ways to manipulate studies. A 2008 study showed that industry-funded studies were more likely to reach positive conclusions, largely because negative studies, which could limit a drug's sales, seldom get published. Treatment guidelines are not always evidence based, direct-to-consumer marketing can be misleading, and the approval process for new therapies "often sets the bar too low."

The topic that tips Cifu and Prasad briefly out of academic mode, however, is what they label "useless research," studies that tell us nothing new. "Hundreds of hours are spent studying the effects of exercise in observational data sets," they wrote. "Let us say: please stop. Should you exercise? Yes. Of course! …Exercise is associated with lower rates of heart disease, and diabetes, and pancreatic cancer, and whatever. One more study on the benefits of exercise among the healthy is not needed. We have reached consensus." No one is "waiting for one more piece of evidence to start."

Reforms are needed and overdue, the authors conclude. They outline ways to enhance medical education about clinical trials, improve the quality of clinical research and tighten up the process for approving new drugs by increasing the burden of proof—requiring drug or device developers to demonstrate that an innovation clearly works prior to its adoption and widespread use.

"It is time," they insist," to make sure we get things right in medicine while the horse is still in the barn."