New AI system aims to improve real-world application of clinical trial results

A new study led by Winship Cancer Institute of Emory University and Abramson Cancer Center of University of Pennsylvania researchers demonstrates that a first-of-its-kind platform using artificial intelligence (AI) could help clinicians and patients assess whether and how much an individual patient may benefit from a particular therapy being tested in a clinical trial. This AI platform can help with making informed treatment decisions, understanding the expected benefits of novel therapies and planning future care.

The study, published in Nature Medicine, was led by board-certified medical oncologist Ravi B. Parikh, MD, MPP, medical director of the Data and Technology Applications Shared Resource at Winship Cancer Institute of Emory University and associate professor in the Department of Hematology and Medical Oncology at Emory University School of Medicine, who develops and integrates AI applications to improve the care of patients with cancer. Qi Long, PhD, a professor of Biostatistics, and Computer and Information Science, and founding director of the Center for Cancer Data Science at the University of Pennsylvania, and associate director for Quantitative Data Science of the Abramson Cancer Center of Penn Medicine, was co-senior author. The study's first author was Xavier Orcutt, MD, a trainee in Parikh's lab. Other study authors included Kan Chen, a PhD student training in Long's lab, and Ronac Mamtani, associate professor of medicine at the University of Pennsylvania.

Parikh and his fellow researchers developed TrialTranslator, a machine learning framework to "translate" clinical trial results to real-world populations. By emulating 11 landmark cancer clinical trials using real-world data, they were able to recapitulate actual clinical trial findings, thus enabling them to identify which distinct groups of patients may respond well to treatments in a clinical trial, and those that may not.

"We hope that this AI platform will provide a framework to help doctors and patients decide if the results of a clinical trial can apply to individual patients," Parikh says. "Furthermore, this study may help researchers identify subgroups in whom novel treatments do not work, spurring newer clinical trials for those high-risk groups."

"Our work demonstrates the enormous potential of leveraging AI/ML to harness the power of rich, yet complex real-world data to advance precision medicine at its best," adds Long.

Limited generalizability of trial results

Parikh explains that clinical trials of potential new treatments are limited because less than 10% of all patients with cancer participate in a clinical trial. This means clinical trials often do not represent all patients with that cancer. Even if a clinical trial shows a novel treatment strategy has better outcomes than the standard of care, "there are many patients in whom the novel treatment does not work," Parikh says.

"This framework and our open-source calculators will allow patients and doctors to decide whether results from phase III clinical trials are applicable to individual patients with cancer," he says, adding that "this study offers a platform to analyze the real-world generalizability of other randomized trials, including trials that have had negative results."

How they did their analysis

Parikh and colleagues used a nationwide database of electronic health records (EHR) from Flatiron Health to emulate 11 landmark randomized controlled trials (studies that compare the effects of different treatments by randomly assigning participants to groups) that investigated anticancer regimens considered standard of care for the four most prevalent advanced solid malignancies in the United States: advanced non-small cell lung cancer, metastatic breast cancer, metastatic prostate cancer and metastatic colorectal cancer.

What they found

Their analysis revealed that patients with low- and medium-risk phenotypes, which are machine learning-based traits used to assess the underlying prognosis of a patient, had survival times and treatment-associated survival benefits similar to those who were observed in the randomized controlled trials. In contrast, those with high-risk phenotypes showed significantly lower survival times and treatment-associated survival benefits compared to the randomized controlled trials.

Their findings suggest that machine learning can identify groups of real-world patients in whom randomized controlled trial results are less generalizable. This means, they add, that "real-world patients likely have more heterogeneous prognoses than randomized controlled trial participants."

Why this is important

The research team concludes that the study "suggests that patient prognosis, rather than eligibility criteria, better predicts survival and treatment benefit." They recommend that prospective trials "should consider more sophisticated ways of evaluating patients' prognosis upon entry, rather than relying solely on strict eligibility criteria."

What's more, they cite recommendations by the American Society of Clinical Oncology and Friends of Cancer Research that efforts should be made to improve the representation of high-risk subgroups in randomized controlled trials "considering that treatment effects for these individuals might differ from other participants."

As to the role of AI in studies such as this one, Parikh says, "Soon, with appropriate oversight and evidence, there will be an increasing tide of AI-based biomarkers that can analyze pathology, radiology or electronic health record information to help predict whether patients will or will not respond to certain therapies, diagnose cancers earlier or result in better prognoses for our patients."

This research was supported by grants from the National Institute of Health: K08CA263541, P30CA016520 and U01CA274576.