Study reports flaws in design, analysis, and interpretation of Lyme disease

Statistical analysis questions evidence discouraging retreatment, coverage

Most doctors treat Lyme disease with antibiotics for two to four weeks after diagnosis, but if symptoms persist after that, medical guidelines recommend against antibiotic retreatment. That recommendation may not be warranted. A newly published statistical review of the four studies upon which those guidelines are based reports flaws in design, analysis, and interpretation that call into question the strength of the evidence against retreatment.

Allison DeLong, a biostatistician at Brown University's Center for Statistical Sciences and lead author of the study published online Aug. 19, 2012, in Contemporary Clinical Trials, said the four studies do not prove that retreatment does not work. That questionable interpretation, however, has led doctors to forgo treatment and insurance companies to withhold reimbursement.

"The goal of the paper is to clarify what was actually found from these clinical trials and what could be said and what couldn't be said," DeLong said. "A lack of evidence should not be used to deny treatment when the studies have serious flaws."

Evidence in the trials is most often inconclusive, she and three co-authors found. Two studies even found some statistically significant benefits from antibiotics.

DeLong has been curious about Lyme disease retreatment for more than a decade since a friend of hers seemed to benefit from therapy. Her friend paid for the treatment out-of-pocket. Statisticians would call that anecdote an "n of 1," but the example stuck with DeLong as more people, including journalists, began to question whether retreatment really was ineffective.

In 2009 and 2010, DeLong and her colleagues decided to look into the matter with full statistical rigor. Their analysis started by scanning the medical literature for any randomized clinical trials offering evidence of the efficacy of antibiotic retreatment for Lyme disease. Careful review of more than 100 studies ultimately whittled the field down to the four studies on which the Infectious Diseases Society of America and the American Academy of Neurology are based their guidelines.

The most influential studies were conducted by Klempner et al., and published together in the New England Journal of Medicine in 2001. The multicenter trials enrolled chronic Lyme sufferers with positive or negative blood serum results for Immunoglobulin G, an antibody that might indicate active infection. In each of the IgG positive and negative groups, patients either received IV antibiotics followed by oral antibiotics or IV placebo followed by oral placebo. Their symptoms were measured along the way using a subjective set of health quality-of-life measures called the SF-36.

Although Klempner et al. found no significant benefit to retreatment, findings from subsequent SF-36 studies in chronic illnesses provide evidence that the Klempner study was looking for unrealistically large differences.

"The trials, as designed, called for treatment effects considerably larger than the minimum clinically important differences (MCID) identified in other chronic illnesses, suggesting that the sample sizes were inadequate and the trials were very likely underpowered to detect the true underlying MCIDs," DeLong and her co-authors wrote in the journal.

Klempner's statistics showed that treatment might or might not have been effective given the broad range of a statistical measure known as the confidence interval, DeLong said.

In another of the four trials conducted by Krupp et al., researchers found that retreatment produced a significant benefit for fatigue, but the authors of the study mistakenly discounted that result, DeLong said.

The authors became concerned that their results were tainted by too many subjects realizing that they were receiving real treatment instead of the placebo. The measure of fatigue is subjective and could be influenced by that realization. But DeLong found that the subjects weren't likely to have realized anything. Here's why: If the members of each group have a blindly optimistic seven in 10 chance of believing that they received real medicine, then the people who really were would be right seven out of 10 times and the people receiving the placebo would only be right 3 out of 10 times. The people receiving the medicine would seem like they had discovered their status, but in reality they were only making a lucky, optimistic guess.

While the Krupp study was adequately powered to measure a significant benefit from fatigue, it had less power to measure the two other treatment effects it considered: improvements in cognitive processing and clearance of a potential Lyme disease biomarker, DeLong said.

The last of the four studies, by Fallon et al., had a very small sample size. It found hints of some benefits from retreatment but nothing definitive either positively or negatively.

Ultimately, DeLong said, the best evidence to support or refute antibiotic retreatment will come when scientists develop a definitive test for active Lyme disease infection. In the interim, it is possible that chronic Lyme disease patients harbor an ongoing infection that antibiotics could treat.

"The interpretation of the trials goes too far," she said. "You can't say it's been shown that retreatment is not beneficial. You can't then jump to the conclusion that this shows there is no persistence of infection."