A smart system helps children with autism improve socialization skills

A team of NIH-funded researchers at Stanford University Medical School has found that children with autism improved measurably on a test of socialization and learning when their therapy included an at-home intervention with Google Glass. Google Glass is a headset worn like eyeglasses that provides augmented reality on a miniature screen, with sound. The smart system of eye wear and mobile-phone-based games helped the children with autism understand emotions conveyed in facial expressions.

Autism is a complex neurological and developmental spectrum disorder that affects how children interact with others, communicate, and learn. Many children affected by autism spectrum disorder (ASD) are unable to discern facial expressions, so miss out on important cues that aid in learning and socialization.

The Stanford team, led by Dennis P. Wall, Ph.D., associate professor of Pediatrics, Psychiatry and Biomedical Data Sciences, used Google Glass's built-in camera along with software customized to run on a smart phone as an intervention that care givers used with their children at home to supplement clinic-based therapy. The study was published March 25, 2019, in JAMA Pediatrics.

Technology can be a terrific asset to the therapy process, for both physical and neurodevelopmental gains. This is a heartwarming achievement and a promising example using a bioengineering approach. The innovative software and hardware solution coupled with the therapeutic component meets a dire need for many children and their parents."

Tiffani Lash, Ph.D., director of the NIBIB programs in Connected Health (mHealth and Telehealth), and Point-of-Care Technologies

Google Glass is lens-less, non-invasive, and peripheral-;sitting off to the right side of view for the child. "The system acts as a true augmentation to their reality, keeping them in their natural social world, as opposed to taking them out of it," Wall said. "In contrast to virtual or mixed reality, augmented reality is potentially a powerful vehicle through which we can teach children social skills to rescue some of these deficits early in their development."

A camera in the device captures the facial expression of family members in the glasses' field of view, reinforcing what the child sees by providing an image and audio prompt. It detects up to eight emotions: happy, sad, angry, scared, surprised, disgusted, "meh," and neutral. The glasses are wirelessly connected to a smartphone device that may be operated in three different play modes. There is 'find the smile' game, where the child is prompted to say something that prompts an expression in the family member's face; the 'guess the emotion' game, where the family member asks the child to guess the emotion from the family member's face; and free play, an unstructured mode of identifying facial expressions.

The device also records a video that parents can observe at a later time to monitor the progress that their child makes with the activities. "Our hope was that the video playback would be a good source of reinforcement learning with the children," Wall said. "It provides the opportunity for the learner to focus in on certain human emotions that they may or may not be getting right, so they might become more adept at detecting those emotions in real time."

The researchers recruited 71 children between the ages of six and 12 who all had been enrolled in a program of applied behavioral analysis therapy-;the standard care for most children with ASD. Experts recommend 20 hours per week of the standard therapy, in which the child interacts with a therapist who leads learning activities to improve social, motor, and verbal behaviors, as well as reasoning skills through observation and positive reinforcement. The authors cite the current cost for this standard therapy to be between $40,000 and $60,000 per year, noting that parents can often wait up to 18 months for their child to gain access to the therapy.

Experts suggest that it is important for children with ASD to receive a diagnosis early-;which can be assessed by the age of two-;so that children can begin treatment as early as possible. According to the authors, learning aids such as the type tested in the study could begin to address this difficult challenge of accessing therapy more immediately, outside the clinic.

Of the 71 enrolled in the study, 40 children also received the augmented reality device to play the programmed games or freely play during 20-minutes sessions, four times per week. After six weeks, the team assessed all 71 children on a standard socialization scale.

The researchers found that children receiving the smart-glasses intervention along with standard therapy scored significantly better in the post-study assessment than those in the control group. Children who used the smart glasses improved 4.58 points on the standard scale above those who did not use the Google Glass intervention. Authors unrelated to the work by Wall and colleagues recently published in Autism Research that changes of 2 to 3.75 points on the scale represent a clinically important difference.

"This is based on a statistically rigorous approach to the analysis of the data," Wall said. "We should be excited about the result. While the overall effect is modest, the positive change seen in the treated children is significant and points to a new direction that could help more children get the care they need, when they need it."

Wall noted that the device represents a short-term learning aid and predicts that in the not-too-distant-future there will be a wider array of available augmented reality wearables. "After a period of time, they take the glasses off and they grow on their own into more complex social scenarios."

Though a playful intervention, families in the treatment group missed a portion of the prescribed hours in which to practice with Google Glass, and most preferred the structured games over the unstructured free-play option. But the activity was positively received to the point that children who participated in the study created a new name for the tool, calling it Superpower Glass, a moniker the authors adopted in writing their report on the study. The researchers have begun to plan for a larger, follow up study.