Inaccurate and misleading meta-analysis of E-cigarettes and population-based diseases

12 May 2025

• Brad Rodu,

• Nantaporn Plurphanswat &

• Jordan Rodu 

https://link.springer.com/article/10.1007/s11739-025-03956-w?utm_source=rct_congratemailt&utm_medium=email&utm_campaign=oa_20250512&utm_content=10.1007%2Fs11739-025-03956-w

Introduction

Since electronic cigarettes (e-cigarettes) entered the U.S. market in 2007 [1], the health consequences have been a subject of controversy among public health researchers. Despite the fact that e-cigarettes do not produce as many toxic chemicals as combustible cigarettes [2], many researchers have used observational studies to claim that e-cigarette use increases the risk of smoking-related diseases, especially cardiovascular and respiratory diseases.

Glantz et al. [3] recently published a random-effects meta-analysis entitled “population-based disease odds for e-cigarettes and dual use versus cigarettes.” The authors “identified 124 odds ratios (94 cross-sectional and 30 longitudinal) from 107 studies,” and they concluded that “Direct epidemiological evidence based on actual use of e-cigarettes in the general population suggests that, at least for cardiovascular disease, stroke, and metabolic dysfunction, the odds of disease between current e-cigarette and cigarette use were similar.”

At first glance, the meta-analyses appear to be robust and convincing, because their pooled odds ratios (ORs) are often statistically significant. However, after publication, several issues were raised in letters on the journal website [4, 5]. These include the small number and short exposure durations of e-cigarette users in the source studies, lack of dose–response assessment [4], and two serious methodologic flaws involving assessing bias and certainty of evidence [5].

After a careful review, we document that Glantz et al. inexplicably commingled outcomes in cardiovascular disease (CVD) and chronic obstructive pulmonary disease (COPD) categories. Next, we demonstrate that the quality of source studies selected by Glantz et al. failed to meet a basic criterion, described by Egger et al. [6] as “Garbage in—garbage out?: The quality of component trials is of crucial importance: if the ‘raw material’ is flawed, the findings of reviews of this material may also be compromised. Clearly, the trials included in systematic reviews and meta-analyses should ideally be of high methodological quality…” Specifically, we show that (a) cross-sectional studies have been proven entirely deficient in crucial temporal information about exposure and disease diagnosis and, (b) existing longitudinal studies from the FDA’s Population Assessment of Tobacco and Health (PATH) survey are inadequately adjusted for changes in exposure during follow-up. Finally, we discuss other important analytic errors.

We will show that Glantz et al., failed to meet these quality control standards, which is crucial in meta-analyses [6, 7]. Consequently, their findings are inaccurate and unreliable.

Methods

In order to provide clear examples of the low-quality raw materials used by Glantz et al., we assessed the studies they selected for CVD, stroke and chronic obstructive pulmonary disease (COPD) among e-cigarette users vs. nonusers, as listed in Figure S2 and Table S2 of their Appendix [3]. We examined all of the source studies for these categories, particularly focusing on specific disease outcomes, data source(s), and relative risks (RRs) or odds ratios (ORs). Our results are displayed as descriptive tables.