Automated method shows predictive power for COVID-19 in-hospital mortality

By Colin Lightfoot, M.Sc. Infection and ImmunityReviewed by Danielle Ellis, B.Sc.Nov 4 2021

Depending on many factors, including concurrent medications, patient histories, and comorbidities, the effectiveness of medical treatments can vary from patient to patient in coronavirus disease 2019 (COVID-19). Due to this, a wide variety of statistical tools have been established to robustly estimate heterogeneous treatment effects (HTE) and individualized treatment effects (ITE) as functions of patient features. However, it requires a large number of pairwise tests to test multiple treatments for heterogeneous effects.

Study: Automated Interpretable Discovery of Heterogeneous Treatment Effectiveness: A Covid-19 Case Study. Image Credit: Corona Borealis Studio/ Shutterstock

This news article was a review of a preliminary scientific report that had not undergone peer-review at the time of publication. Since its initial publication, the scientific report has now been peer reviewed and accepted for publication in a Scientific Journal. Links to the preliminary and peer-reviewed reports are available in the Sources section at the bottom of this article. View Sources

In a research paper posted to the medRxiv* server, researchers from the Massachusetts Institute of Technology (MIT), NYU Langone Health, and Microsoft Research focussed on the S-learner strategy of estimating the conditional average treatment effect (CATE). This strategy uses a single model that models mortality risk as a function of treatment and patient risk factors, allowing for the comparison of mortality risk with and without the treatment for different patient risk factors.

The authors validated the proposed method on the MIMIC-IV dataset to test its recovery of HTEs. This method was then applied to a dataset containing over three thousand hospitalized COVID-19 patients, and evidence was found of HTEs which were largely predicted by indicators of inflammation and thrombosis risk. Furthermore, patients who benefited most from inflammation treatments were those with few indicators of thrombosis risk, while those who benefited the most from thrombosis treatments were those with few indicators of inflammation risk.

The study

The baseline mortality risks associated with comorbidities and pre-admission outpatient medications are depicted in Figure 1. The effects observed of comorbidities and outpatient medications coincide with mechanisms of mortality related to inflammation and/or thrombosis. Valve replacement is the most protective association, for which the typical treatment is long-term anticoagulants. Conversely, the most harmful association are hypertension, heart failure, and myocardial infarction.

Architecture to estimate latent personalized treatment benefits. Gray boxes indicate observed data, orange boxes are learned models, and blue boxes are latent variables. We first train an Explainable Boosting Machine (EBM) to predict mortality risk from pre-treatment features to generate baseline mortality risk. This model, based on gradient boosted trees, captures discontinuities in risk, but is not differentiable and so must be trained in isolation from the rest of the architecture. After the EBM is trained, we train the single-task Neural Additive Model (NAM) and the multi-task NAM in parallel to decompose mortality risk into untreated mortality risk and personalized treatment benefits. This framework provides state-of-art mortality risk prediction and decomposes the risk into interpretable additive factors contributing to underlying risk and treatment benefits.

From laboratory tests conducted, it was observed that the most significant effect is for neutrophil/lymphocyte ratio, which is used to measure inflammation and to indicate the severity of COVID-19. The likelihood of death I also increased by the presence of procalcitonin and c-reactive protein. Serum calcium levels were also noteworthy, as no elevation in levels is related to increased risk, and levels of creatinine, which indicate a step-function drop in risk at 4 mg/dL, a common threshold for dialysis decisions. Also shown in the laboratory results was that hematocrit displays opposite effects when estimated in isolation or following correction for other factors.

The HTEs of six treatments were assessed in this study (heparin, non-steroidal anti-inflammatories [NSAIDs], hydroxychloroquine, azithromycin, zinc replacement, and glucocorticoids). All of these treatments display diminished effectiveness with increasing patient age, which is associated with severe outcomes related to COVID-19.

Homogeneous treatment effects of in-patient medications, calculated as the adjusted risk difference (left) and the adjusted risk ratio (right). For each treatment, the number in parentheses is the number of patients treated with this medication in the dataset.

The HTEs of three of these treatments are associated with neutrophil/lymphocyte ratio, which is a marker of inflammation and severe COVID-19. With an increase in neutrophil/lymphocyte rate, the statistical benefit of heparin, azithromycin, and NSAIDs decreases, while there is no significant heterogeneity shown with the effectiveness of hydroxychloroquine, zinc, or glucocorticoids.

When glucocorticoids were used in combination with azithromycin, there was a decrease in benefit observed. With patients with a history of alcoholism, NSAIDs showed an increase in benefit. Diminished benefits were seen with zinc replacement in patients with congestive heart failure, which would be expected if any protective effects of zinc were due to mediation of thromboses because platelet aggregation inhibitors and anticoagulants are routinely prescribed to patients with this condition.

Implications

The authors in this paper proposed a method to substantiate the varying effectiveness of different medical treatments by training adaptive models, exploring the benefits of personalized treatments, and sharing statistical power between many treatments. This method was applied to mortality risk models of COVID-19 patients, and it revealed two pathways of mortality: inflammation and thrombosis.

This news article was a review of a preliminary scientific report that had not undergone peer-review at the time of publication. Since its initial publication, the scientific report has now been peer reviewed and accepted for publication in a Scientific Journal. Links to the preliminary and peer-reviewed reports are available in the Sources section at the bottom of this article. View Sources