AI used to detect sexual orientation from facial images

By Dr. Ananya Mandal, MDSep 10 2017

New research has used a novel technology to successfully determine the sexual orientation of people from facial images or photographs. The accuracy of these techniques is reportedly more than those detected by humans. The study details are to be published n the Journal of Personality and Social Psychology. At present the study is available in a draft form.

Authors of the study Yilun Wang and Michal Kosinsk from the Stanford University, Stanford, USA, in their study entitled, “Deep neural networks are more accurate than humans at detecting sexual orientation from facial images” show that faces are more informative about sexual orientation than previously thought. They used deep neural networks – a form of technology, to extract certain facial features from the facial images. The computer would then assess the facial features and detect if the person is hetero or homosexual based on just the facial features.

Image Credit: Ramon Espelt Photography / Shutterstock

For the study the team included 130,741 images of 36,630 men and 170,360 images of 38,593 women. These were obtained from an American dating website that has public profiles. All images were first detected using a facial-detection technology to check for those with most clarity. This left 35,326 pictures of 14,776 people. There was a fair representation of both homosexual and heterosexual men and women.

The technology used was called the VGG-Face that first detects the face and gives out a string of numbers that characterize the face and is called the “faceprint”. Next this was analyzed using statistical tests to detect if a person was gay or straight.

For each image the technology was successful in differentiating between homo and heterosexual men in 81 percent of cases and between homo and heterosexual women in 74 percent cases. This was compared to the humans who were given the same photos and asked to guess which of them belonged to gay or lesbian individuals. Human assessors scored 61 percent for men and 54 percent for women respectively.

Results of this study revealed that the accuracy of the algorithm that the technology used increased to 91% and 83% when five facial images per person were fed to it. This included two types of facial features –

• Fixed type such as shape of the nose or other parts of the face that are unalterable
• Transient type such as grooming style that could change as per wishes of the individual

There have been studies previously that show that sexual orientation has deep rooted basis on the hormonal make up of the individual. Hormones determine not only the facial morphology or shape of the faces but also expression, and grooming styles. The algorithms could detect gay males with 57 percent accuracy and lesbian females with 58 percent accuracy. It was seen that for male sexuality the features to be noted included the shapes of the noses, eyes, eyebrows, cheeks, chin and hairline. For female sexuality the facial features taken into account by the software were the shapes of the nose, corners of the mouth, hair and neckline.

Authors add as a conclusion that the ability of this technique to accurately detect gay persons could be a potential threat to the privacy as well as safety of the gay men and women in future. Dr Kosinski says that this type of AI systems can in future also detect other intimate and private information such as IQ or political views. He explained that humans missed a lot of information with their naked eyes that the machines would not. Dr Kosinski is well known for his previous work on psychometric profiling using information from social networking sites such as Facebook to draw conclusions regarding personality. Both Donald Trump’s campaign and Brexit supporters used this to target voters. This has significantly raised concerns about what is private and personal and what is not.

Reference: Wang, Y., & Kosinski, M. (in press). Deep neural networks are more accurate than humans at detecting sexual orientation from facial images. Journal of Personality and Social Psychology. https://osf.io/zn79k/