New regulations requiring certain female athletes to medically lower their testosterone levels in order to compete internationally are based on "fatally flawed" data, according to new research led by the University of Colorado Boulder.
The paper, published this month in the Asser International Sports Law Journal, comes as South African Olympic sprinter Caster Semenya prepares to challenge the controversial new rules in an international court in Switzerland.
The authors have called for a retraction of the original research and asked the International Association of Athletics Federations - the global governing body for track and field - to reconsider the rule change.
"In almost any other setting of science, errors of this magnitude would lead to a paper being retracted," said lead author Roger Pielke Jr., director of the Center for Sports Governance at CU Boulder. "And it certainly would not be the basis for broad regulations that have a profound impact on people's lives."
In April 2018, the IAAF announced new regulations requiring certain female athletes with naturally high testosterone levels to take testosterone-lowering hormones if they want to continue to compete in the women's category for the 400-meter, the 400-meter hurdles, the 800-meter, the 1,500-meter and the mile.
The rule, which applies to IAAF-sanctioned international competitions, requires that they maintain serum testosterone levels below 5 nanomoles per liter (nmol/L) for at least six months prior to competition. Most females have testosterone levels ranging from 1.12 to 1.79 nmol/L while the normal adult male range is 7.7 - 29.4 nmol/L. About seven in every 1,000 elite female athletes have high testosterone levels, according to IAAF.
The association had attempted to put forth similar regulations in 2011 , but that rule was thrown out when the Swiss-based Court of Arbitration for Sport (CAS) - the highest court for international sport - concluded in 2015 that there was a lack of evidence linking high testosterone to "a real competitive advantage" in women.
In 2017, the IAAF came back with that research, publishing a paper in the British Journal of Sports Medicine (BJSM), which claimed that elite women runners with the highest testosterone levels performed as much as 3 percent better than those with the lowest levels in several events.
Pielke and co-authors Erik Boye, a professor emeritus of molecular biology at the University of Oslo, and Ross Tucker, a University of Cape Town exercise physiologist, challenge those results.
"We found problematic data throughout the study and consequently, the conclusions can't be seen as reliable," Pielke said.
When the three tried to replicate the original findings using data from the study's authors and publicly available results from four of the races included, they uncovered "significant anomalies and errors."
For instance, they found performance times that were erroneously duplicated and "phantom times" that did not exist in official IAAF competition results. In addition, some athletes disqualified for doping were included in the study dataset - a fact that could confound the results.
In all, from 17 to 32 percent of the data used in the study was found to be in error. The researchers also note that IAAF researchers themselves conducted the BJSM study.
"We would not find it appropriate for cigarette companies to provide the scientific bases for the regulation of smoking, or oil companies to provide the scientific bases for regulation of fossil fuels. Sport regulation should be held to the same high standards," they write.
The IAAF researchers did correct what they characterized as "data capture errors" and re-ran their analysis in a subsequent letter to the journal. But flaws remain in that revision, Pielke said.
The research will be at issue later this month when Pielke is expected to serve as an expert witness at the Court of Arbitration for Sport, where Semenya and Athletics South Africa have brought a case against the IAAF calling the rules "discriminatory, irrational, and unjustifiable."
Under the new regulations, those who decline to medically reduce their testosterone levels must relinquish their right to compete as females.
Originally set to take effect in November, implementation of the rules has been postponed until after the outcome of the case.
"Fundamentally, the issues that we raise with our paper are about the integrity of science in regulation," said Pielke. "Any agency, in sport or beyond, should be expected to produce science that can withstand scrutiny and which actually supports the justification for proposed regulations. That simply did not happen here."
BMI's influence on disease pathogenesis uncovered in new research