Study finds flaws in cancer trials

Feb 1 2007

Cancer research and drug development are yielding more sophisticated candidate therapies, but investigators' methods to test them haven't kept pace, according to researchers at Memorial Sloan-Kettering Cancer Center. That could explain why so many experimental drugs fail in the final large and costly phase of testing, they say.

In the February 1 issue of Clinical Cancer Research the researchers found that only nine of the 70 Phase II studies they examined clearly defined measures by which an experimental drug could be judged to offer benefit to patients.

“We are facing a new and growing problem in clinical trial testing, and that is while the drugs have changed, researchers are still using the same old methods to gauge how effective they are," said the study's lead author, Andrew Vickers, Ph.D., a research methodologist.

The problem, Vickers said, is that for so long, therapies (usually chemotherapy) were tested by seeing if tumors would shrink in patients with advanced cancer. Measuring that reduction was an accepted way to gauge benefit, he said. But today's new treatments, which can include targeted therapies that slow tumor progression, are often tested in less advanced cancer and in combinations “and it can be hard to answer the question of whether patients are doing better than expected," he said.

In their study, Vickers, Howard Scher, M.D., Chief of the Genitourinary Oncology Service, and medical student Vennus Ballen, examined Phase II clinical trials reported from June 2003 - June 2005 in the Journal of Clinical Oncology or in Cancer, two major journals in cancer research. These studies, which usually enroll 30 to 50 patients, aim to provide a “go/no go" decision on whether the therapies studied should be evaluated in a large Phase III clinical trial, the ultimate test of whether a drug should be given to cancer patients.

They specifically looked at 70 studies whose design required “historical data" to determine whether a drug was promising enough to justify a Phase III trial. “When a novel agent is added to an existing standard in the hope of increasing response rates over and above those expected from the standard treatment alone, historical data on the response rates to the standard treatment are required," Vickers said. “Similarly, some agents are thought to slow disease progression, rather than lead to rapid tumor regression, necessitating an endpoint such as progression-free survival or overall survival at one year. That survival target clearly needs to be developed by reference to historical data."

For example, if two chemotherapy drugs used in combination lead to a 30 percent survival rate at one year, and researchers are interested in knowing whether an addition of a third drug is of benefit, the three-drug combination has to meet that 30 percent hurdle and jump over it, Vickers said. “So we have to be pretty certain that the 30 percent target is correct," he said.

Of the 70 studies they examined, however, nearly half (46 percent) did not give any justification for the historical target. And of the studies that did refer clearly to prior data, only a few (nine, or 13 percent), did so properly. Furthermore, trials that failed to report a rationale for the historical bar were much more likely to conclude that the new therapy was “active" and therefore worthy of further study or a Phase III clinical trial, Vickers said.

The researchers could not find a single study that used advanced statistical techniques to adjust for differences between patients studied in older clinical trials that were used as the historical bar and patients treated in the new trial, who may be at an early stage of cancer.

“These studies could have been done better", Vickers said. “Phase II studies are all about seeing whether patients on a new treatment are doing better than expected; if so, we should investigate the new treatment in a really big trial."

“However, to know whether we are 'doing better than expected' we need some kind of benchmark of what we should expect from standard treatment," Vickers said. “That benchmark assessment is what we find is missing from these studies."