Smart device-based electronic platform can continuously monitor mental health

Reviewed by Emily Henderson, B.Sc.May 27 2021

To help patients manage their mental wellness between appointments, researchers at Texas A&M University have developed a smart device-based electronic platform that can continuously monitor the state of hyperarousal, one of the signs of psychiatric distress. They said this advanced technology could read facial cues, analyze voice patterns and integrate readings from built-in vital signs sensors on smartwatches to determine if a patient is under stress.

Furthermore, the researchers noted that the technology could provide feedback and alert care teams if there is an abrupt deterioration in the patient's mental health.

Mental health can change very rapidly, and a lot of these changes remain hidden from providers or counselors. Our technology will give providers and counselors continuous access to patient variables and patient status, and I think it's going to have a lifesaving implication because they can reach out to patients when they need it. Plus, it will empower patients to manage their mental health better."

  Dr. Farzan Sasangohar, Assistant Professor in Wm Michael Barnes '64 Department of Industrial and Systems Engineering

The researchers' integrated electronic monitoring and feedback platform is described in the Journal of Psychiatric Practice.

Unlike some physical illnesses that can usually be treated with a few doctor visits, people with mental health needs can require an extended period of care. Between visits to a health care provider, information on a patient's mental health status has been lacking. Hence, unforeseen deterioration in mental health has a limited chance of being addressed. For example, a patient with anxiety disorder may experience a stressful life event, triggering extreme irritability and restlessness, which may need immediate medical attention. But this patient may be between appointments. On the other hand, health care professionals have no way to know about their patients' ongoing struggle with mental health, which can prevent them from providing the appropriate care.

Hence, patient-reported outcomes between visits are critical for designing effective health care interventions for mental health so that there is continued improvement in the patient's wellbeing. To fill in this gap, Sasangohar and his team worked with clinicians and researchers in the Department of Psychiatry at Houston Methodist Hospital to develop a smart electronic platform to help assess a patient's mental wellbeing.

"The hospital has the largest inpatient psychiatry clinic in the Houston area," said Sasangohar. "With this collaboration, we could include thousands of patients that had given consent for psychiatric monitoring."

Sasangohar's collaborators at Houston Methodist Hospital were already using an off-the-shelf patient navigation tool called CareSense. This software can be used to send reminders and monitoring questions to patients to better assess their wellbeing. For instance, individuals at risk for self-harm can be prompted to take questionnaires for major depressive disorder periodically.

Rather than solely relying on the patients' subjective assessment of their mental health, Sasangohar and his team also developed a whole suite of software for automatized hyperarousal analysis that can be easily installed on smartphones and smartwatches. These programs gather input from face and voice recognition applications and sensors already built in smartwatches, such as heart rate sensors and pedometers. The data from all of these sources then train machine-learning algorithms to recognize patterns that are aligned with the normal state of arousal. Once trained, the algorithms can continuously look at readings coming from the sensors and recognition applications to determine if the individual is in an elevated arousal state.

"The key here is triangulation," said Sasangohar. "Each of these methods on their own, say facial sentiment analysis, show promise to detect the mental state, albeit with limitations. But when you combine that information with the voice sentiment analysis, as well as physiological indicators of distress, the diagnosis and inference become much more powerful and clearer."

Sasangohar noted that both the subjective evaluation of mental state and the objective evaluation from the machine-learning algorithms are integrated to make a final assessment of the state of arousal for a given individual.

While their technology's prototype is ready, the researchers said they still need to improve the battery life of smartphones carrying their software since the algorithms guzzle a lot of power. Further, they noted that they have to address usability issues, that is, any issues that prohibit patients from using their technology, such as difficulty in navigating their application.

"Because of the stigmatization that surrounds mental illness, we wanted to build a mental health monitoring device that was very discreet," said Sasangohar. "So, we chose off-the-shelf products, like smartphones, and then build sophisticated applications that operate within these devices to make monitoring mental health discreet."