Are vaping studies destined to fail? Research on e-cigarette health effects often includes former smokers—skewing results and misleading the public. Dr. Arielle Selya explores why flawed study designs, academic incentives, and NIH funding pressures shape the science on vaping and tobacco harm reduction.
Featuring:
DR. ARIELLE SELYA
Behavioural Scientist, PinneyAssociates
Consultant, CoEHAR
@ArielleSelya
Transcription:
00:10
Brent Stafford: Hi, I'm Brent Stafford and welcome to another edition of RegWatch on GFN.TV. For years, public health officials have pointed to research linking vaping with lung disease and many other harms. But what if the problem isn't vaping, but the research itself? Many of these studies are built on a fundamental flaw. They include former smokers, people who may still be suffering from the long-term damage of smoking. So are researchers stacking the deck by creating research destined to fail? Joining us today to help answer that question is Dr. Arielle Selya, a behavioral scientist at Penny Associates, a U.S.-based firm that provides consulting services on tobacco harm reduction to dual labs. She is also a consultant for the Center of Excellence for the Acceleration of Harm Reduction, COHAR, at the University of Catania. Dr. Selya, thanks for coming back on the show.
01:07
Arielle Selya: Great to be back.
01:09
Brent Stafford: So tell us, how big of a problem is flawed study design when it comes to research on vaping and health?
01:17
Arielle Selya: It's a very pervasive problem. Week after week, I closely keep up with the forthcoming literature on PubMed on vaping. And there's so many studies that have the same recurring flaws week after week. So today we'll be talking about observational studies of vaping and health outcomes, especially respiratory health. And some of the problems are inherent to the data. With any observational study, you can't get around the confounding problem that people who vape are different from people who don't. So there might be other reasons for health differences. But there are some pretty simple changes that you can make to the analysis to overcome a large amount of the bias at least. And that's just not being done often.
01:59
Brent Stafford: Explain what confounding means for the viewer.
02:03
Arielle Selya: Sure, so confounding is basically comparing apples and oranges. So if you have people who vape or people who smoke and you compare them to complete non-users, they're going to be different on all sorts of other characteristics. They might have other differences in lifestyle factors. They might have different pre-existing health conditions. They might have different environmental exposures. So all of these things are confounding factors because if you find a difference in health outcomes, you don't know if it's due to the vaping or the smoking or any of these other factors that also tend to separate across these apples and orange groups.
02:40
Brent Stafford: So Dr. Selya, your latest work takes a deep dive into studies that link vaping to respiratory harm. You have two new companion papers that just came out in February. The first is a systematic review of 10 studies titled, Respiratory Effects of Electronic Cigarette Use in Individuals Who Never Smoked. And the second is a narrative review titled Association Between Electronic Cigarette Use and Respiratory Outcomes Among People With No Established Smoking History. So let's start with the first study of the studies. Tell us about that.
03:17
Arielle Selya: Yeah, so as you mentioned, this is a pair of companion papers that emerge from the same project. The systematic review, that's kind of the end of the story, so I'll start with that and then work backwards. That is a review of all available literature that's already been published on respiratory symptoms associated with e-cigarette use among people who never smoked. So we looked at all the studies on this, narrowed them down to the studies that were more likely to be higher quality. So looking at prospective or longitudinal data, as well as requiring that the studies have a separate analysis among people who never smoked. And this is to separate it out from people who formerly smoked who might have lingering health effects. Essentially what we found are that we did not find evidence that vaping is associated with severe or clinically relevant respiratory conditions, but we did find a possible signal for mild coughing and wheezing symptoms.
04:15
Brent Stafford: And when you looked at the studies like in the hundreds, I guess, for inclusion in the review, you were only able to include 10. Why is that?
04:24
Arielle Selya: That's right. So our initial pass of the literature identified about 350 studies, and we had to exclude them down to 10. And the reason for that is one big factor for exclusion was using prospective data or longitudinal data. Most of the studies out there are cross-sectional, so they're kind of a snapshot in time that ask about respiratory outcomes as well as vaping and smoking behavior, but this doesn't tell you about the ordering. It doesn't tell you which occurred first. So those studies are less valuable in looking at this evidence. The other main criteria was that most studies defines who's currently using e-cigarettes and who's currently not using e-cigarettes as the two comparison groups. The problem with this is that people who are currently using e-cigarettes might have never smoked a cigarette in their life or they might have smoked for 20 years and then quit two months ago. And all these people are lumped into the same group. So when you find a difference between the e-cigarette use and non-use groups, you don't know if it's due to prior smoking. So there were very few studies actually that separately reported results among people who never smoked.
05:34
Brent Stafford: So obviously, I guess if to lay person, that would mean that you can really trust the findings of these studies.
05:44
Arielle Selya: Yeah, the findings are definitely biased because they take health effects that might be due to previous smoking behavior and they attribute them to e-cigarette use.
05:57
Brent Stafford: Can you give us an example, maybe an extreme example, if you could, about the way in which the structure of these studies can actually distort the conclusions?
06:09
Arielle Selya: Sure. So one thing that is about the temporal ordering of smoking and vaping. So somebody that has COPD and switched from smoking to vaping, possibly driven because of the COPD symptoms and they wanted to quit smoking, that person is counted the same as somebody who never smoked. and maybe even just vaped once in the prior weekend for the first time ever, and those people are combined into the same category. So there really is a distortion of the results by smoking status that's not usually being accounted for. Some studies do account for it in a better way, but most just look at, do you use e-cigarettes now or do you not use e-cigarettes and use those as the two main comparison groups?
06:52
Brent Stafford: Now, and we see this a lot, don't we, when it comes to youth smoking or most importantly, youth vaping, that there is inclusion in the group of some people that really shouldn't be. Is that correct?
07:05
Arielle Selya: That's correct. Some of these criteria for the group definitions are very broad. So in the case of youth vaping, it's typically measured as any use in the past 30 days. And so somebody that tried any cigarette for the first time because a friend offered on a weekend is in the same group as somebody that's vaping daily. But those would represent two very different risk behavior groups.
07:29
Brent Stafford: Now, isn't that a conscious decision by the researchers or by the government reporting agencies on that to kind of grow the number?
07:39
Arielle Selya: I can't say what the intention is or what the rationale behind those are. And in some cases, it's kind of become convention. So typically the government reports on ever use, which means at all in your lifetime or past 30 day use. And even with past 30 day use, a lot of that could be transient and experimental and not necessarily represent problematic use patterns. But for better or worse, this is kind of the standard in the literature now is to measure past 30 day use.
08:10
Brent Stafford: Well, by what measure would it be valuable to record if a person has ever used an e-cigarette?
08:20
Arielle Selya: Yeah, it's not clear. It seems like ever use would have pretty negligible health effects in and of itself if it doesn't progress. I think the motivation, especially when studying youth, is that public health efforts want to catch people that are experimenting and stop them before they develop more established smoking habits. I think that's the motivation, but again, it's not directly relevant to health really in and of itself.
08:47
Brent Stafford: In the second paper of the companion papers, which was the deep dive into the narrative, maybe you first explain if you could what exactly is a deep dive into the narrative.
08:57
Arielle Selya: So this was kind of a narrative review and I said with the systematic review that I started at the end of the story. This is kind of towards the beginning of the story when as we were combing through this literature. We were really reading everything reading the full text reading the supplement to see the different analyses that each paper presented and there was each paper presented quite a different approach to looking at this. Some of them were. higher quality than others. So we really felt like a narrative review into the methodological issues would be helpful to identify some of the common flaws that could be ideally prevented in forthcoming research. So some of those, for example, are what were the outcomes? Were they clinically meaningful or not? And because there were so few studies, there were only 10 that made it into our systematic review, we really kept a wide net of different outcomes. So we included something clinically relevant like self-reported diagnoses of emphysema or COPD all the way down to symptoms, self-reported symptoms such as wheezing in the past year, I think was one of the more lenient ones. And of course, wheezing in the past year could be due to asthma, it could be due to a cough or flu, it could be due to any number of reasons. So there's a distinction even in the 10 studies that we looked at. as to is the outcome kind of severe or clinically relevant versus is it mild and maybe transient and maybe due to something entirely different from vaping.
10:30
Brent Stafford: So based on what you had found and you of course just mentioned, the coughing and wheezing as a potential mild symptom from vaping, is it something that vapors should be concerned about?
10:41
Arielle Selya: It's a possible concern, but it really depends on where a person is coming from and what they would be doing otherwise. So if somebody is using vaping to keep off of cigarettes, then that would be the reduced harm approach. However, it's good to be aware of any possible effects.
11:00
Brent Stafford: Well, it's certainly not the making you bleed out of your eyes and all the other things that we always hear in, you know, the scare stories around vaping.
11:12
Arielle Selya: That's right. There's really a lot of sensationalist and alarmist language in the media.
11:18
Brent Stafford: So how could these studies be better structured to avoid these problems?
11:24
Arielle Selya: There's a couple of simple analysis approaches that could mitigate some of the flaws in the data. So I mentioned confounding earlier, that's going to be inherent to almost any study because you're looking observationally at who smokes or vapes and who doesn't. So the confounding issue can't be entirely solved, but you can do a better job generally in the analysis stage of number one, removing cases where the respiratory condition happened before the person ever started vaping, because it's often attributed to vaping, but if vaping came after the respiratory condition, then you can't make that conclusion. Another major thing that could mitigate some of the bias here is adjusting for detailed smoking history. Most of the studies that I've seen control for smoking status, which means that they separate people into groups of, do you currently smoke? Do you formerly smoke? Do you never smoke? And even so, sometimes the last two groups are combined. But when you're looking at people who formerly smoked, it's really important to measure the range of smoking behavior that they've had over their lifetime. So adjusting for pack years, for example, some studies out there show that that can explain some of what initially appears to be an association with e-cigarettes is actually due to pack years or history of smoking. Another factor is time since you quit smoking, because the longer it's been since you quit smoking, the more your lingering health effects might decrease.
13:03
Brent Stafford: Dr. Salia, my question here is that if you've got these confounding issues with this research and you've got former smokers, you know, included in kind of skewing the results, does that not mean that the research is almost destined to fail, destined to show the kinds of harms that they seem to love to promote in the media?
13:30
Arielle Selya: It certainly is a bias to combine people who formerly smoked and people who never smoked. Again, I can't speak about the intentions of the researchers because I have to admit that I was a formerly naive academic that had at one point never heard about the tobacco harm reduction side of things. So I think in a lot of cases, researchers are reading the literature and kind of absorbing what they hear from the other academic literature as well as from the media and trying to do the best they can to publish and survive the academic pressures.
14:05
Brent Stafford: Right. But nevertheless, just on its face, this kind of research is destined to deliver these kinds of results.
14:12
Arielle Selya: Yeah, I think it's a pretty consistent bias that any, any analysis that combines former smoking and never smoking, uh, conflates the effects of e-cigarettes with the effects of smoking.
14:23
Brent Stafford: And I would imagine too, and let's, you know, uh, give, you know, credit to an issue when it's due. And that is it's not always easy to find, um, vapors that were never smokers.
14:37
Arielle Selya: Yes, that's a good point too. And I think that's one of the limitations in the literature and maybe why there isn't more research on people who never smoked, but they So in one of the studies we reviewed, for example, by Kenkel and colleagues, they used PATH data to analyze respiratory diagnoses in relation or following baseline e-cigarette use. And there were only 12 participants in the data set that vaped and did not smoke or never smoked. And zero of them had a respiratory condition at follow-up. So this is impossible to analyze. You can't put that into a model and come up with a reliable statistical estimate. So one of our recommendations at the end of our companion papers is that we need more dedicated research on never smokers.
15:25
Brent Stafford: And Dr. Selya, in one of those companion papers, you mentioned the reverse temporal sequence issue, where smokers with pre-existing lung damage switched to vaping. How often do researchers mistake this for vaping causing harm?
15:42
Arielle Selya: It happens quite a bit, I think, and part of the reason is because so many of these studies are analyses of cross-sectional data, which just provide a snapshot of one point in time. And some of these surveys are national surveys that are so large that there's not time for the participant to get into the details of when were you diagnosed with COPD, when were you diagnosed with chronic bronchitis, and when did you start vaping. So some of these happen because of a limitation inherent to the available data, but some of them, some other studies do have the available information to tease apart cases where the e-cigarette use happened first versus second.
16:23
Brent Stafford: There are some pretty big examples, egregious examples of this. Now I know Dr. Stanton Glantz, who's a notoriously well-known tobacco control researcher. He's had some papers withdrawn around Hart, and I think there was some around COPD. Explain for us what we know about those instances.
16:48
Arielle Selya: Yeah, so this is a good example of the reverse temporality issue. The retracted paper that he has is on heart attacks and e-cigarette use. And the data included cases where the heart attack happened first and then e-cigarette use. So that biases the results and you don't know the temporality there unless you go into the data and look for it. Sometimes it's not there, but sometimes it is. But in this case, there might be what's called a sick quitter effect, where if somebody is really facing health challenges from smoking, that motivates them to quit and go to e-cigarette. So it's kind of a reverse directionality that's driving the association.
17:28
Brent Stafford: And what about the COPD example?
17:31
Arielle Selya: So there's a COPD example in our systematic review. We reviewed a paper by Don Cankle and colleagues, which was actually a reanalysis of one of Stanton Glantz's papers, looking at COPD and other respiratory diagnoses. And in the original paper by Stanton Glantz, he combined former smokers and never smokers into one group. So the results look like there was an effective e-cigarette use on respiratory outcomes, but then when cankala colleagues broke it out into separate former and never smokers. Well, there wasn't really reliable data because this was the case where there were zero cases of the thing that you're looking for, but on the other hand, it's encouraging that it was low.
18:13
Brent Stafford: Well, it sure feels like stacking the deck though, doesn't it?
18:18
Arielle Selya: Yeah, for sure. Grouping, those are making that grouping together, conflates, smoking and vaping.
18:27
Brent Stafford: I noticed Dr. Selya that Dr. Ricardo Paloso is one of your co-authors on these papers. What has it been like working with him on the Kohar project in Veritas?
18:40
Arielle Selya: It's been great working with Dr. Palosa and the entire team. He's extremely prolific and he's really at the cutting edge to advance tobacco harm reduction research. So I'm really happy to be a part of that team.
18:53
Brent Stafford: And what makes the Veritas project different, say, than other research projects?
18:58
Arielle Selya: So the Veritas project involves contribution by consumers. And so in the study that we have, we focused on never smokers who vaped and looking at respiratory outcomes and kind of patterns of use. And there was a lot of input that was consumer driven, which was great to have. I think the field as a whole can really benefit from having consumer involvement in research.
19:23
Brent Stafford: No, I wonder, are there any ethical considerations around using people who have never smoked but, you know, are now vaping? I guess you would never hand a vape to somebody who wasn't already using it, or do you?
19:35
Arielle Selya: Right. No, exactly. You wouldn't do that because it's unethical. So that's why we rely on observational data and So you're not putting anybody in danger or at risk of anything that they're not already doing. But what you do is you go out and say, okay, for people that have already chosen to do this, let's give them a survey and see what their respiratory symptoms are.
20:00
Brent Stafford: So what does the Veritas research say so far?
20:04
Arielle Selya: So far, it's actually very consistent with the results from our systematic review in that we're not seeing any signal of severe or moderate respiratory symptoms. We are seeing maybe a slight increase in mild coughing and wheezing symptoms.
20:22
Brent Stafford: Dr. Selya, last time we had you on the show, you talked about how universities chase NIH grants, and that's the National Institute of Health here in the United States, because of the high indirect costs they bring in. Can you break down how these overhead fees influence the kind of research that gets prioritized and why this is happening?
20:45
Arielle Selya: Sure. So let me first explain what indirect costs are. So let's say a professor at a university gets awarded a $1 million grant from NIH to perform research at their university. The amount that the NIH actually pays to the university is the $1 million plus a certain percentage, which varies across institutions, which is meant to cover operating costs and overhead costs. So Harvard, for example, has an indirect rate of about 69%, which means that for every $1 million NIH funds, the university itself gets closer to $1.7 million. So I think that there's pros and cons to the high level of indirect. On the benefit side, there's a lot of institutional resources that researchers need and rely on to use their research. So lab space, especially if you have animals or cells, computing equipment, IRB oversight, and it also covers some of the costs of applying for grants itself. So in some cases like the entire grant seeking industry is kind of a its own self-perpetuating entity kind of because you need a team of administrators at universities to help apply for these grants and help administer the funds and the indirects can kind of offset some of those those direct costs from applying to grants. Where I think the disadvantage come in is that the sheer discrepancy between the high level of indirects from NIH versus other funding agencies, that makes NIH funds much more highly prized by universities. So I, as a junior faculty member in academia, I was told that anything that's not an NIH grant, quote, doesn't count towards my promotion and tenure requirements. It's only NIH dollars that count because of the high level of indirects. So I think as a result, the high indirects have kind of created like a systematic problem where the majority of US researchers are beholden to a single funding agency and what they said is their funding priorities.
22:48
Brent Stafford: Now, the real problem there, at least up until now, is that the NIH seems to only be funding one side of the science.
22:58
Arielle Selya: That was my experience when I was a faculty member and it might have changed in the past several years. So like on the optimistic side, for example, I know that FDA is currently funding messaging on relative risk to help overcome misperceptions that might keep people on the worst products. But it was the case in academia when I was there that there was not really a clear place for tobacco harm reduction, especially in research on younger populations.
23:26
Brent Stafford: But as you know, the Trump administration has put a cap on indirect fees for NIH grants and there's a lot of pushback. If you just check your X account, you'll notice.
23:37
Arielle Selya: So I went on record with your show last year well before the indirect issue became a politicized issue. I recognized the benefit of indirects for researchers that are lucky enough to get an NIH grant, but a year ago I talked to you about how I think that's also contributed to some of the systemic issues that trickle down into nicotine and tobacco research. So primarily it's one funding agency setting the funding priorities. And this affects the very level of what topics are considered scientifically credible or worth pursuing because junior faculty members now are under a lot of pressure to get external funding, especially NIH. So they will modify their research activities to match those of the funding agencies.
24:25
Brent Stafford: It'll be interesting to see how all this shakes out. Let's turn to something a bit more hopeful. Am I being optimistic, thinking that there is some momentum being built by researchers towards tobacco harm reduction?
24:40
Arielle Selya: I'm seeing optimism too in the literature. So there's a couple of recent studies within the last year or so that came out of labs that have historically been focused on the harms of vaping for younger populations. And there's two pretty notable studies now that recognize that e-cigarettes might have displaced smoking among 12th graders. going into adulthood who smoked at baseline. There was another study looking at flavored jewel actually, concluding that menthol flavored jewel versus tobacco flavor can have greater appeal for adult smokers and that it had a harm reduction conclusion, which was very encouraging. So I think it is becoming more acceptable to recognize the harm reduction angle.
25:28
Brent Stafford: Dr. Selya, as you know, the 12th edition of the Global Forum on nicotine. The annual conference on safer nicotine products takes place again this year in Warsaw, Poland from June 19 to 21, 2025. The conference theme is challenging perceptions. In your mind, what are some solutions to address the misunderstandings, misconceptions, and mischaracterizations that are holding back progress for safer nicotine products?
25:58
Arielle Selya: Yeah, that's a great question and it's an exciting topic for this year's GFN. I think there's a lot of different fronts for improvement and starting on the research side, for example, I think that drawing attention to some of the recurring flaws in the literature, what can help motivate future research to be improved. I'm not saying that all of the flaws can be overcome, because as I mentioned, there's some limitations that are inherent to the data, but researchers can make wise choices that are fairly simple to help overcome at least some of that bias and make for stronger research going forward. I also think academics need to be better at communicating their findings and this especially comes across when speaking to the media. The limitations of the underlying studies don't always come through and the media on their side needs to be better at kind of acknowledging those limitations and toning down the alarmist language. I think there's also a lot of self-reflection that needs to happen on what are the harms of this kind of messaging and how they might be perpetuating misperceptions that can have actual harms on people.
27:09
Brent Stafford: And final question, Dr. Selya, you've been selected to deliver the Michael Russell oration at this year's Global Forum on Nicotine Conference in Warsaw, Poland. Get your tickets now, everyone. So tell us, who is Michael Russell and why is this important?
27:25
Arielle Selya: Michael Russell was really a huge pioneer and safer nicotine. He has a famous quote that most of us have heard that people smoke for the nicotine, but die from the tar. So he was really essential for developing nicotine replacement therapy, getting that to market, and laying the perspective of tobacco harm reduction in general. So I'm really, really honored to be asked to give this year's oration. And I've got big shoes to fill, not only from Michael Russell, but also the less several oration speakers like Cliff Douglas and Roberto Sussman.
27:58
Brent Stafford: It is definitely a great honor.
28:01
Arielle Selya: Yeah, I'm very excited and I'll be focusing on some of the same issues that we discussed here. So flaws in the literature and kind of structural incentives and then what we can do to encourage better communication and correct perceptions.