Study finds wearable fitness trackers accurately measure heart rate but not energy expenditure

According to a study published online in the Journal of Personalized Medicine, fitness activity trackers measure heart rates precisely but not the number of calories burned.

Credit: SFIO CRACHO/Shutterstock.com

Millions of people wear fitness activity trackers to monitor their exercise and health and often share the device data with their physician. The team of researchers at the Stanford University School of Medicine has done a study to find out the accuracy of the data.

The authors of the study have reported that people who use these wearables can be assured that with regards to heart rate, the devices are doing a good job. However if it measures energy expenditure, it is probably off by a significant amount.

Among the diverse group of 60 volunteers wearing devices, evaluation showed six out of the seven devices, measured heart rate with an error rate of less than 5 percent. Some devices were very precise compared with others and factors such as color of the skin and body mass index influenced the measurements.

The study found that none of the seven devices measured energy expenditure precisely. The error rate was 27 percent even in the most accurate device and 93 percent in the least accurate device.

Euan Ashley, DPhil, FRCP, professor of cardiovascular medicine, of genetics and of biomedical data science at Stanford said "People are basing life decisions on the data provided by these devices,". However, consumer devices do not hold the same standards as medical-grade devices. He said physicians find it difficult to know what to make of heart-rate data and other data from the device that the patient is wearing.

Ashley  comments that manufacturers may test the accuracy of activity devices extensively, however, it is difficult for the consumers to know how accurate such information is or the process used by the manufacturers in testing the accuracy of the devices. The research team has set out to independently assess the activity trackers meeting the criteria such as measuring both heart rate and energy expenditure and being available commercially.

Anna Shcherbina, one of the authors of the paper, said: "For a lay user, in a non-medical setting, we want to keep that error under 10 percent."

Sixty volunteers comprising of 31 women and 29 men, wore the seven devices while walking or running on treadmills or using stationary bicycles. A medical-grade electrocardiograph was used to measure the heart rate of each volunteer. Metabolic rate was estimated with an instrument to measure the oxygen and carbon dioxide in breath— a good alternative for metabolism and energy expenditure. Results from the wearable devices were then compared with the measurements from the two "gold standard" instruments.

"The heart rate measurements performed far better than we expected," said Ashley, "but the energy expenditure measures were way off the mark. The magnitude of just how bad they were surprised me."

He said the most important point to remember is that a user can pretty much rely on the heart rate measurements from the fitness tracker, however, basing the amount of food you eat on the number of calories burned as per the device is not a good idea.

Ashley and Shcherbina said each of the devices uses its own proprietary algorithm for calculating energy expenditure and they were not sure why the energy-expenditure measures were so far off.

My take on this is that it's very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone's fitness level, height and weight, etc."

 

Graduate student Anna Shcherbina, Stanford University Medical Center

Shcherbina mentioned heart rate is measured directly, whereas energy expenditure must be measured indirectly through proxy calculations and there is a possibility the algorithms are making assumptions that do not fit individuals very well.

The research team saw a need to make their assessment of wearable devices open to the research community. So they created a website displaying their own data and welcomed others to upload data related to device performance.  

In the next stage of the study that the team is already working on, they are assessing the devices while volunteers wear them as they go about a normal day, including exercising in the open, instead of walking or running on a laboratory treadmill. "In phase two," said Shcherbina, "we actually want a fully portable study. So volunteers' ECG will be portable and their energy calculation will also be done with a portable machine."

The work is an example reflecting the goal of Stanford Medicine's focus on precision health which is to foresee and prevent disease in the healthy and precisely diagnose and treat disease in the ill.