Online tool analyzes at-home videos to predict musculoskeletal health

Reviewed by Emily Henderson, B.Sc.Apr 21 2023

Musculoskeletal conditions-;which encompass impairments of the joints, bones, or muscles-;affect more than 1.5 billion people around the world. Like most health problems, catching these issues early could potentially prevent serious disorders, but few quantitative and objective tests to assess musculoskeletal health are available.

There is a significant need for inexpensive, scalable tools to objectively measure and monitor changes in physical function, especially as the global population ages. Finding ways to remotely evaluate musculoskeletal health could assist diagnoses, guide treatment decisions, and potentially improve the quality of life of countless people."

Melissa Boswell, Ph.D., Postdoctoral Fellow at Stanford University

To tackle this problem, Boswell and colleagues developed an online tool that can analyze self-collected, at-home videos taken with a smartphone. The tool, which was deployed in a nationwide study, was sensitive enough to predict physical health (as evaluated using a standardized survey tool) and osteoarthritis of the knee or hip. Results of the study were recently described in npj Digital Medicine (a publication of Nature).

The tool is based on the sit-to-stand test, an evaluation that is fairly self-explanatory: starting from a seated position with arms crossed, a participant stands up and sits back down a total of five times while being timed. Slower times could indicate reduced lower body strength, or a condition that affects the musculoskeletal system, or simply aging, explained Boswell, noting that the test by itself is not a precise measure of physical function. This is where kinematic analyses (which evaluate motion) come into play. By assessing things like the angles and accelerations of different parts of the body during movement, more definitive predictions of musculoskeletal health can be made.

Here's how their tool works-;using a smartphone, someone takes a video of the participant performing the sit-to-stand test, which is then uploaded into the app. The tool combines an open-source algorithm for estimation of the participant's pose with customized algorithms that calculate the time it takes for the participant to complete the test along with kinematic factors, such as joint angles during movement. Participants also take a survey to input demographic information and complete a questionnaire to evaluate factors like fatigue, pain, emotional distress, and overall physical function. Participants were also asked if they had a previous clinical diagnosis of osteoarthritis.

Because the study was accessible to many people, participation was markedly higher than traditional biomechanics trials. "Investigations of human movement have typically been limited to biomechanics laboratories equipped with expensive and technical equipment," Boswell explained. In their nationwide study, Boswell and colleagues incorporated data from 405 participants from 35 U.S. states, with individuals submitting videos in just over a month's time. "Our study had more than 35 times the median number of subjects compared with traditional biomechanics studies," she noted. The age of the participants ranged from 18 to 96 years, with an average age of 37.5 years.

Similar to previous laboratory and clinical assessments, the researchers found that a larger maximum trunk angle-;that is, leaning further forward-;when standing could predict osteoarthritis, even when controlling for a variety of factors, such as age, sex, or body mass index (BMI). Like previous biomechanical studies, the tool found that longer times to complete the sit-to-stand test were associated with a lower physical health score, a higher BMI, and older age. These results suggest that at-home smartphone-based assessments may be feasible for future biomechanical studies.

Because their trial had a diverse participant population, the researchers were able to evaluate how demographic factors might be associated with biomechanical differences. When they compared the two largest ethnic groups in their study, they found that the 243 white participants had significant differences in trunk angle during the sit-to-stand test compared with the 103 Asian participants, even after taking other variables into account. "This finding indicates that we can't assume similarities across racial and ethnic groups and highlights the importance of more diverse samples in biomechanical studies," said Boswell. "These functional differences are likely due to underlying factors like mobility or strength, or even health inequities, that weren't captured in this study."

The researchers also found a biomechanical feature (specifically, leaning forward more quickly when standing) that was significantly associated with a higher mental health score. "It is exciting to see potential relationships between mental health and how we move," Boswell said. She stressed, however, that the relationships between biomechanical differences and race/ethnicity or mental health status observed in their study are mainly hypothesis-generating and will require follow-up research.

"This study highlights that self-collected data can be used to assess physical function and musculoskeletal health, allowing patients to participate in a biomechanical study from the comfort of their own home," said Moria Bittmann, Ph.D., a program director in the division of Discovery Science & Technology at the National Institute of Biomedical Imaging and Bioengineering (NIBIB). "Studies like these provide additional evidence that smartphone-based applications can help facilitate decentralized clinical trials and can empower individuals to take an active role in their health."

"New and improved machine learning tools are constantly being developed to estimate human motion (and much more), which have the potential to have a profound impact on health care through mobile health monitoring," said senior study author Scott Delp, Ph.D., a professor at Stanford University and director of the NIH-funded Mobilize Center. "But merely building new AI tools is not enough. To truly make an impact, we need to translate these new tools into the world, make them usable for the average person and for vulnerable populations, and rigorously evaluate their ability to give insights into important clinical measures."

This study was supported by the Mobilize Center, which is funded by a grant from NIBIB and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD; P41EB027060) and the Restore Center, a Medical Rehabilitation Research Resource that is supported by NICHD and the National Institute of Neurological Disorders and Stroke (NINDS; P2CHD101913). This study also received support from the National Science Foundation (DGE-1656518).