New analysis highlights need for rigorous evidence to back up 'The Global Fund' impact claims

Reviewed by James Ives, M.Psych. (Editor)Jan 25 2019

The Global Fund to Fight AIDS, Tuberculosis and Malaria ('The Global Fund') asserts they and their partners have saved 27 million lives—but more rigorous evidence and data is needed to back up that claim, according to a new analysis published in The Lancet this week.

The analysis, by Dr Rocco Friebel of The London School of Economics and Political Science (LSE) and Rachel Silverman, Amanda Glassman, and Dr Kalipso Chalkidou of the Center for Global Development (CGD), calls on The Global Fund to publish more robust data and be more transparent in its official impact reports, and highlights ways the financing organization can improve its evidence base.

The authors note The Global Fund is likely to have had a positive impact, having distributed over $39 billion of aid funding around the world, but suggest improvements the organization and others could adopt for their impact measurement and reporting, including:

• More rigorous performance evaluations: Rather than relying on theoretical, modeled results alone, more rigorous performance evaluations would help assess whether The Global Fund's grant recipients are delivering the services and products being financed.
• Structuring grants to prioritize evidence from the start: Taking this step would help ensure the most effective projects and products are being funded.
• Expanding the scope of impact evaluation: As well as expanding evaluations, The Global Fund should refocus on areas where evidence is most needed, such as particularly large projects and programs using new and untested strategies.
• Publishing the data and models used to estimate impact: Where empirical evidence isn't available, The Global Fund should at least make the data and models used to estimate impact available to researchers and the public.

The analysis also details current shortcomings of The Global Fund's evaluations, including:

• Opaque methodology: Without further details in the public domain, the methodology used by The Global Fund in calculating the lives it has saved cannot be verified or reproduced by external researchers. More transparency in its modeling methodology, the authors argue, would build confidence in the organization's claims among funders and external researchers. (Since a draft of the analysis was shared, The Global Fund has revised its methodology statement on its website, but the authors call for further improvements).
• Distinguishing The Global Fund's institutional impact from the impact of the broader partnership:The analysis suggests The Global Fund results report conflates the impact of The Global Fund "partnership"—encompassing other donors, NGOs, and country governments—with the impact of The Global Fund as a standalone institution with an annual budget of roughly $4bn. The authors argue that distinctly separating the activities of The Global Fund from activities carried out by partner organizations would make its impact much clearer. For example, domestic policies in large countries such as India and South Africa have led to significant investments to tackle tuberculosis and HIV;in addition, economic growth is estimated to be responsible for a significant reduction in maternal and child deaths from 1990-2010. The impact of both of these external factors is not distinguished in The Global Fund's claim of saving 27 million lives.
• Unreliable data: Much of the data is from countries where statistical systems are weak or non-existent. The authors acknowledge the difficulties of working with these restraints but reiterate their call for data to be collected in rigorous performance evaluations to supplement the results of modeling exercises.

Commenting on the analysis, Dr Rocco Friebel, lead author and Assistant Professor of Health Policy at LSE, said: "There is no doubt organizations like The Global Fund do great work but to ensure continuous donor investment they need to be more open and honest with their reporting,"

"The methods and underlying data of the modeling exercise conducted by The Global Fund and others should be released and subject to public scrutiny. The organization should be clear about its methodology, share relevant data and open itself up to peer review. Taking these steps toward openness will instill confidence in partners and lead to more sustainable fundraising for aid relief."

Amanda Glassman, chief operating officer at the Center for Global Development, said: "Funders have tough choices to make, and a more grounded and evidence-based assessment of The Global Fund's actual impacts would help the organization make its case in the difficult replenishment cycle ahead."

"Our concerns are not new," added Kalipso Chalkidou, director of global health policy at the Center for Global Development. "For years, we've called for The Global Fund to stand on firmer ground when estimating the lives its work has saved. The Global Fund does important work, and in order to make sure everyone has confidence in that work, it's time for the organization to take our calls for transparency seriously."

Rachel Silverman, policy fellow at the Center for Global Development, said ."Global health partnerships, including The Global Fund, hold a position of immense responsibility as the stewards of scarce aid dollars. To best serve the populations in need of that assistance—and to attract additional resources in the upcoming replenishment cycle—we hope that The Global Fund and other funders will embrace the highest standards of evidence and accountability."