Artificial intelligence uses facial features to better diagnose rare genetic diseases

Reviewed by Emily Henderson, B.Sc.Feb 10 2022

Rare genetic diseases can sometimes be recognized through facial features, such as characteristically shaped brows, nose or cheeks. Researchers at the University of Bonn have now trained software that uses portrait photos to better diagnose such diseases. The improved version "GestaltMatcher" can now also detect diseases that are not yet known to it. It also manages to diagnose known diseases with very small numbers of patients. The study has now been published in the journal "Nature Genetics".

Many sufferers of rare diseases endure an odyssey until the correct diagnosis is made. "The goal is to detect such diseases at an early stage and initiate appropriate therapy as soon as possible," says Prof. Dr. Peter Krawitz from the Institute for Genomic Statistics and Bioinformatics (IGSB) at the University Hospital Bonn. The researcher is a member of the Cluster of Excellence ImmunoSensation2 and the Transdisciplinary Research Area "Modelling" at the University of Bonn.

The majority of rare diseases are genetic. The underlying hereditary mutations often cause varying degrees of impairment in different areas of the body. In most cases, these hereditary changes are also expressed by characteristic facial features: for example, because eyebrows, the base of the nose or the cheeks are shaped in a distinctive way. However, this varies from disease to disease. Artificial intelligence (AI) uses these facial characteristics, calculates the similarities, and automatically links them to clinical symptoms and genetic data of patients. "The face provides us with a starting point for diagnosis," says Tzung-Chien Hsieh of Krawitz's team. "It is possible to calculate what the disease is with a high degree of accuracy."

"GestaltMatcher" requires only a few patients

The AI system "GestaltMatcher" described in the current publication is a continued development of "DeepGestalt", which the IGSB team trained with other institutions a few years ago. While DeepGestalt still required about ten non-related affected persons as a reference for training, its successor "GestaltMatcher" requires significantly fewer patients for feature matching. This is a great advantage in the group of very rare diseases, where only a few patients are reported worldwide. Furthermore, the new AI system also considers similarities with patients who have also not yet been diagnosed, and thus combinations of characteristics that have not yet been described. GestaltMatcher therefore also "recognizes" diseases that were previously unknown to it and suggests diagnoses based on this.

This means we can now classify previously unknown diseases, search for other cases and provide clues as to the molecular basis."

Prof. Dr. Peter Krawitz, Institute for Genomic Statistics and Bioinformatics (IGSB), University Hospital Bonn

The team used 17,560 patient photos, most of which came from digital health company FDNA, which the research team worked with developing the web service through which the AI can be used. Around 5,000 of the photos and patient data were contributed by the research team at the Institute of Human Genetics at the University of Bonn, along with nine other university sites in Germany and abroad. The researchers focused on disease patterns that were as diverse as possible. They were able to consider a total of 1,115 different rare diseases. "This wide variation in appearance trained the AI so well that we can now diagnose with relative confidence even with only two patients as our baseline at best, if that's possible," Krawitz says.

"We are very happy to finally have a phenotype analysis solution for the ultra-rare cases, which can help clinicians solve challenging cases, and researchers to progress rare disease understanding," says Aviram Bar-Haim of FDNA Inc. in Boston, USA. In Germany, too, the application in doctors' offices, for example, is not far off, adds Krawitz. Doctors can already use their smartphones to take a portrait photo of a patient and use AI to make differential diagnoses, he says. "GestaltMatcher helps the physician make an assessment and complements expert opinion."

Peter Krawitz and his team turned over the data they collected themselves to the non-profit Association for Genome Diagnostics (AGD), to provide researchers with access. "The GestaltMatcher Database (GMDB) will improve the comparability of algorithms and provide the basis for further development of artificial intelligence for rare diseases, including other medical image data such as X-rays or retinal images from ophthalmology," Krawitz says.