Differences in reporting methods can over- or under-estimate the safety of drugs

Reviewed by James Ives, M.Psych. (Editor)Apr 29 2020

An important goal of early-phase clinical trials is to discover a drug's possible side effects. But despite FDA guidelines seeking to standardize this reporting, a University of Colorado Cancer Center study finds significant variation in how drug side effects are reported, potentially making some drugs seem safer or less safe than they really are.

"Without standardizing how we present such data, there is the potential for misinformation about these side effects," says Emily Simons, MD, MPH, senior resident at University of Colorado School of Medicine and the first author of the study.

The group including senior author D. Ross Camidge, MD, PhD, got interested in the topic after seeing how differently the side effects of new drugs for treating lung cancer were being presented - and realizing that differences in reporting methods could over- or under-estimate the apparent safety of such drugs.

Sometimes you only report an adverse event that happens in, say, 10 percent or more of patients on a trial. However, if you split related side-effects into lots of little sub-groups, perhaps no one event reaches this 10 percent threshold and nothing gets reported,"

In contrast, if you combine rationally grouped events - like confusion and memory issues - then this group passes 10 percent and you could clearly see the side effects as, for example, very broadly affecting higher functions of the brain."

Emily Simons, MD, MPH, Study First Author Senior and Resident, University of Colorado School of Medicine

These observations led Simons to look into all phase I and II clinical trials involving lung cancer patients reported at the American Society for Clinical Oncology (ASCO) annual meetings from 2017-2019, the major outlet for early-phase trial results, finding 209 trials in all.

What the group found was wide-ranging variation in the ways trial investigators report drug side effects.

For example, the study found six different thresholds used to decide when a side effect should be reported. Simons and colleagues also found that when patients were treated with multiple doses of a trial drug, side effects were sometimes reported only across all drug dosages.

Combining side effect data across dosages could make a drug look more toxic due to the inclusion of side effects that occur only at very high doses. On the other hand, including side effects from especially low doses of a drug could make the drug seem more benign that it is at the recommended dose.

Additionally, patients are supposed to report all symptoms when they are on a clinical trial and then it's up to trial investigators to decide, in their opinion, whether a symptom is likely due to the drug or just happens to be another symptom the patient is experiencing at the time.

"Determining if a side effect is treatment-related or not is subjective. If you rely on this, you get rid of some of the background noise of coincidental symptoms. However, you can also miss more subtle side effects," Simons says.

Simon's study found that the reasons patients in some clinical trials stop taking study drugs or reduce the doses are also usually not documented.

Similarly, some patients received blood tests to evaluate possible side effects, but it was often not documented whether everyone on a trial received these tests, or if only those patients already identified to be at risk were tested.

"If you don't know if everyone or only a fraction of people got a specific blood test, you don't know if the ten people with an abnormal result should be documented as 100 percent or 10 percent of the trial population. Some trials require patients to reduce the dose of a drug after an abnormal blood tests, even without other symptoms. Without knowing the details of why dose changes occurred leaves a lot open to speculation," Simons says.

Simons suggests there is a tremendous opportunity for conferences and academic journals to standardize their requirements for the reporting of side effects to help accurately communicate risk.

"Given the increased speed of drug licensing, early phase trial data is essential in helping us form accurate impressions of a new drug. Recognizing and addressing the variation in how side effects are reported would improve the accuracy of these impressions in the future," Simons says.

The study is published today in the Journal of Thoracic Oncology.