Most oncology AI research overlooks critical patient safety and privacy

Flinders University researchers were part of an international team that conducted a scoping review that identified 2,441 records and found just 32 studies examined how cancer patients, survivors and carers feel towards generative AI being used in their care.

People affected by cancer were generally favourable when generative AI improved the accessibility and readability of health information. However, most studies focussed on providing cancer information or simplifying clinical reports, leaving many other potential uses of generative AI in cancer care unexplored.

"Designing AI for patients is not the same as designing it with them," says Dr Bradley Menz, a Research Fellow in the College of Medicine and Public Health at Flinders University.

"If these tools are going to influence cancer care, then patients, survivors and carers need a meaningful role in shaping how they are designed, governed and used.

"In addition, future research needs to broaden the application contexts for responsible and effective AI implementation."

The research revealed that important aspects of Generative AI engagement were assessed only intermittently in previous studies, including the willingness of cancer patients to use AI, and concerns regarding privacy, alignment with expectations and familiarity.

The review found that trust in Generative AI often depended on whether its responses were personalised, emotionally appropriate and perceived as accurate.

Major gaps in what researchers had asked people affected by cancer were also revealed. Of the 32 studies, accountability perspectives were examined in only three studies, and no studies assessed perspectives on safety, transparency, equity, or the preservation of human autonomy.

"Gaining perspectives from cancer patients might help inform the design and implementation of generative AI that bests serve oncology patients, including improved interface design, patient education, consent processes, workflow integration, governance, monitoring and feedback mechanisms," says Associate Professor Ashley Hopkins from the College of Medicine and Public Health at Flinders University.

"Future studies need to assess perspectives with greater conceptual breadth and depth. The current literature focused most often on comprehension, usefulness, relevance, trust, accuracy and clarity."

The researchers hope their review paves the way for more specific and detailed examination of generative AI in oncology – including how the perspectives of people affected by cancer may influence uptake, engagement, effectiveness and concerns about the broader impacts of AI use.