SLU biostatistician encourages people to interpret scientific and medical research

Sep 29 2015

Thanks to the internet, access to research studies, medical news and health websites is greater than ever, offering instant admission to the world of medical knowledge.

Interpreting research studies can be daunting, however, whether you're gathering information about a recent diagnosis, deciding if a report on the evening news means you should change your diet, or wondering whether a new research study is a game-changer or a complicating new wrinkle.

The truth is that research also can be difficult for scientists and doctors to interpret: Studies can contradict each other, promising leads remain years from becoming a cure in the clinic, and findings sometimes apply only to a small subset of people rather than to everyone.

Nevertheless, Saint Louis University biostatistician Travis Loux, Ph.D., encourages everyone to learn more about interpreting scientific and medical research.

"While research news might seem confusing, asking just a few key questions about the studies you are hearing about can cut through the confusion and really help you place the news in context," said Loux, who is assistant professor of biostatistics at SLU's College for Public Health and Social Justice.

Here are six key pieces of information that experts look for when they read research news that can help you quickly put the details you are reading into perspective:

ONE: What kind of study is it?
One of the first questions experts ask is whether a study was done in the laboratory (in an animal model or cells grown in a petri dish) or in a clinical trial (studying people).

This is a key distinction that can tell you how far along in the process toward finding a new drug or treatment a scientific concept may be.

"Research breakthroughs in the laboratory represent important science news and are cause for optimism," Loux said. "However, there are many more hurdles they must pass before we'll know if they will be good treatments for people."

Clinical trials, on the other hand, are broken into phases: Phase I (a small group of people to gauge safety and side effects), Phase II (a bigger group looking at safety and effectiveness), or Phase III (a bigger group monitoring effectiveness, side effects and best dosage). If a Phase III trial is successful, the drug may be considered by the Food and Drug Administration (FDA) for approval to be used by doctors.

There are other types of studies, as well. An epidemiological study, for example, looks at data sets from lots of people to find patterns.

Research reviews take several studies on a topic and consider them together, drawing conclusions and giving readers a more complete sense of which studies have been conducted and what they show overall.

TWO: Will this study change the recommendations a doctor makes right now?
As you're taking stock of the research timeline, the main question you may have is whether this research is complete enough to draw conclusions that will change the recommendations a doctor makes right now.

A study must provide enough evidence that experts believe will improve on current treatments or recommendations, before they will make a change to health advice. If you believe a study may make the case for changing your diet or medication, ask your doctor if he or she agrees before making a change.

THREE: Where was the research published and who funded it?
These two details will tell you a lot about a study. If a study is published in a high profile, peer-reviewed journal, such as the New England Journal of Medicine or the Journal of American Medical Association, it is a signal that doctors and researchers believe the findings may be particularly significant.

Funding sources will tell you who has evaluated, prioritized and funded a particular research study. Funding tends to come from government sources (the National Institutes of Health or the Department of Defense), from industry (a pharmaceutical company that is developing a drug to bring to market), or non-profit entities (like the American Heart Association).

FOUR: What's the sample size?
Researchers often start small in order to test an idea, and frequently they will identify a statistically significant result. However, investigators would need to repeat the study, ideally with a larger group, to be sure that they are, in fact, measuring a real effect.

While the pitfalls of a small sample size are well known, there also are challenges with very large data sets.

"This is also a problem with a really big study. You can find small relationships that creep in because there are so many things to look at," Loux said. "To determine whether they are significant, you'll need to repeat the experiment in another group."

FIVE: Does the data show correlation or causation?
If there is one concept from statistics worth taking the time to understand, this is it.

If a study shows a correlation, it means there is a connection between two factors, though we don't necessarily know how the two factors are related. If a study shows causation, it means there is a cause-and effect relationship between one factor and another.

For example, data shows that areas with higher educated populations also have higher crime rates. While the numbers show a connection -- a correlation -- between these two factors, it doesn't mean that higher education causes crime. Rather, it is likely that both factors are linked to a third factor. In this case, both higher education and higher crime rates are associated with the higher population concentrations of cities.

"So, when we see a correlation between two factors, we perk up and pay attention because it signals that something interesting is happening," Loux said. "But, we also know we need more data before we can really understand what is going on.

"It is very important not to mistake correlation for proof of cause-and-effect."

SIX: Where does this fit in to other research?
Finally, when experts read a study, they attempt to put it in context by seeing how it relates to previous research on the same topic.

"You can't look at a single study in isolation," Loux said. "It's always part of a bigger conversation."

So, what do you do when the evidence is muddy, changes direction or is inconclusive?

"The truth is that all of these scenarios - breakthroughs, complications, contradictions, baby steps forward, dead ends, promising leads, accumulation of enough data that we can make recommendations -- these all are a part of the scientific process."

"We use the best information we have today. And, if tomorrow we find new information, we incorporate that, and if we need to adjust our course, that's what we do. This is science."

Professor Travis Loux, Ph.D., has expertise in causal inference, experimental design, and missing data, and a passion for statistics literacy.

Source: Saint Louis University