Dr Arielle Selya presents a review of the common flaws in observational research on e-cigarette use and discusses the impact that these flaws have on study conclusions. Dr Selya offers suggestion on how future studies in the field could be improved. #GFN25ScienceLab
Transcription:
00:05 - 01:45
[Garrett McGovern]
Anyone who was at the speech last night of Ariellele, she'll need no introduction, but I'm going to go through her bio anyway. She's a behavioral science consultant specializing in tobacco harm reduction through a primary role as senior scientist at Pinney Associates Incorporated. She supports dual labs incorporated in the scientific and regulatory efforts through behavioral study protocols, statistical methodology, and scientific publishing strategy and execution. Dr. Selya is also a consultant at the Center of Excellence of the Acceleration of Harm Reduction on e-cigarette use, behavioural research and critical analysis of scientific literature. She received her PhD in neuroscience at Rutgers University and prior to becoming a consultant spent a decade in academia studying adolescent substance use and addiction. In addition to being a prolific contributor to scientific literature with over 70 publications to her name and rising, Dr. Sayah is also actively engaged in both post-publication critiques, and if you've ever read any of this stuff, we've been in communication myself, and she's the go-to person when science is published and a research piece that usually, I think, we'll say of dubious quality. But anyway, she's absolutely phenomenal at that job, as is Clive Bates, and commercial scientific findings to a broader audience. Now, I'm going to just go through all of our things so we won't have to do it again. I'm going to let Ariellele get moving on our first presentation, which will be a video, and then Ariellele will discuss all the findings. Over to you, Ariellele.
01:45 - 06:40
[Video]
Hi, I'm Ariellele Selya, and this talk is about how to avoid common pitfalls in observational e-cigarette health research. These are the disclosures of myself and my co-author, Ricardo Pelosa. There are over 100 articles on e-cigarette use and various health outcomes. The typical patterning of results as shown for this example of cardiovascular disease is that non-use is lowest risk followed by exclusive e-cigarette use followed by cigarette smoking with the highest risk usually found in dual use. However, most studies are heavily flawed, which prevents us from being able to conclude about direct health risks of e-cigarettes. Meta-analyses compound these flaws because they summarize several underlying studies with the same underlying flaw, so they also reflect that flaw but are considered or interpreted as higher quality evidence because they're meta-analyses. These are the three main categories of flaws that I'll cover in today's talk. Confounding, predominantly by smoking history, temporality issues, and poor definitions of exposure. a lot of these problems stem from a conceptual overly simplistic interpretation of product uses static over time however there are different trajectories that are important to consider as well as important homogeneous subgroups within each category in particular it's important to look at smoking history and smoking trajectories or changes in smoking behavior over time for both the e-cigarette exclusive group and the dual use group In the exclusive e-cigarette use group, while a small percentage of them never smoked, most of them formerly smoked and might have lingering health effects that are in the analysis being interpreted as effects of e-cigarette use rather than smoking. also dual use is interpreted usually as additive exposures from both smoking and vaping however when somebody starts e-cigarette use and becomes a dual user often they substantially reduce their cigarette consumption and that leads to important subgroups or sub-distinctions of dual use as well Most studies adjust for smoking status, meaning current, former, or never. However, this doesn't capture the full range of intensity and cumulative exposures that's important to understand disease risk. Alternatively, analyses can focus on people who never smoked, which avoids the issue of confounding by smoking history. There's also temporality issues in treating product use as a static snapshot. For example, in people who switch from smoking to e-cigarettes, depending on when the survey happened to be administered with respect to that trajectory, they could be categorized as e-cigarette only or as dual use. Even worse, there could be cases of reverse directionality where the health outcome occurred before e-cigarette use. And there could even be reverse causality, such as a sick quitter effect, where people who developed smoking-related illness were motivated to switch to e-cigarettes because of that illness. And in the analysis, that pre-existing illness is bundled into the e-cigarette use group. The third major flaw is poor definitions of e-cigarette exposure. Usually e-cigarette exposure is defined as any use in the past 30 days. This further complicates some of the subgroups because it could capture people that are transiently experimenting with e-cigarettes at the time of the survey. Or it could include all the way up to prolonged and heavy use, which at least for smoking is required for long-term health effects. Reinforcing the importance of these flaws, Gal Cohen and Steve Cook recently published a best practices paper for analyzing observational data on health outcomes. Although we probably can't completely prevent these flaws given the limitations of existing data, there are some easy steps that researchers can take to mitigate these to a large extent. So not just adjusting for smoking status, but adjusting for detailed smoking history, or alternatively, focus on people who never smoked. For temporality issues, it's important to account for where they were coming from, so history and prior behavior rather than what they were doing at the time of the survey. Also, we need better measures of e-cigarette exposure that include things like duration, heaviness, or frequency. So in this talk, we've highlighted some common flaws in articles looking at e-cigarette use and health outcomes. Meta-analyses often suffer from the same flaws as the underlying paper, but unfortunately, they exaggerate the reliability and confidence of these findings if they don't account for those flaws. Many of these harms are partly avoidable, so please do your part either as an author, peer reviewer, or consumer of the research to enforce better standards. Thank you, and happy to answer any questions.
06:45 - 06:55
[Garrett McGovern]
Absolutely wonderful, Arielle. Does anyone have any questions they'd like to ask Arielle? Any questions at all? Yes, Norbert.
06:58 - 07:12
[Norbert Schmidt]
Norbert Zillatron-Schmidt, Germany, consumer. I'm wondering, you suggest using never smokers who use e-cigarettes. Where do you find unicorns?
07:13 - 08:48
[Arielle Selya]
That's a great question. There are some never smokers who use e-cigarettes. There's not very many, which is part of the problem. But there were enough, I think there were maybe 10 papers that Ricardo and I systematically reviewed. And the results weren't always presented in a way that let us cleanly get an estimate of exclusive e-cigarette use. But the problem is that there's no right way to do this. We're never going to have perfectly clean data where you have somebody exclusively using e-cigarettes and no other confounders, and you can look at health outcomes. We're in an imperfect world, so we're limited to either the kind of observational studies that try to separate out the effect of former smoking, and those studies almost always do a very poor job at that because they control for are you a current smoker, are you a former smoker, are you a never smoker? But that doesn't capture the range of duration and intensity of smoking that could still explain some of the health effects. So that's one avenue, and that's flawed, and there's plenty of research looking at that, presenting flawed results, and then other research criticizing those flawed results. Another approach is to use never smokers. It is a bit cleaner, but as you mentioned, there's not that many of them, so it's harder to find these people. But if you can find them, it's more... cleanly separates out or eliminates the confounding effect.
08:48 - 08:54
[Garrett McGovern]
That's great, Ariellele. Any other comments to make? Yes, your good self there. Can you bring the microphone around?
08:56 - 09:24
[Pascal Michel]
Yeah, thanks for the presentation, Pascal Michel, Canada. I was wondering, Arielle, if you have any thought on, it seems to me that we have more and more epidemiological studies that are kind of weak on the method side. So why? Is this editorial? Is this teaching? What's going on there? And to your appreciation, do you see more of weak methods in the tobacco area than in other thematics? Thanks.
09:25 - 10:30
[Arielle Selya]
Yeah, so I could give a whole answer to that. And I'll refer anybody who didn't see to my talk last night for why I think there's such flaws. A brief rundown is I think there's incentives for researchers to align with the priorities of the funding agencies, which have typically been focused more on harms rather than benefits in this field. And I think some of it is not necessarily ill intentioned. I know people out there will disagree with me, but I think a lot of the researchers maybe have misperceptions and they're naively trying to do their best. But this is my attempt to try to call out some of the flaws in the literature and hopefully strengthen it. And as far as whether tobacco is worse than other fields, some people would say yes, I kind of don't think so. As a side interest, I've looked into nutrition research and I found that to be equally awful. And also kind of politicized and polarized. So I'm kind of pessimistic about that. I think there's any, this kind of infighting is common, unfortunately.
10:32 - 10:39
[Garrett McGovern]
Thank you. Anyone else got a question? Okay. Any further comments to make? You have a question?
10:43 - 11:00
[Sam Hampsher-Monk]
Just a feedback on that. Excellent question. Are there disciplinary contributions that are better or worse? I mean, you spoke about the topic and how it compares to nutrition, but are there some disciplines that do a better job at producing more robust evidence than others?
11:00 - 11:49
[Arielle Selya]
Yeah, that's a good question too. So occasionally I see, in general I think the quasi-experimental and economic literature is really strong in this field, because that does provide us with the closest that we're able to get to maybe estimating causal inferences as opposed to just kind of straight observational data. So I think that's the strongest evidence. And every now and then I see like a mathematical modeling paper or a simulation paper that comes in where authors that haven't published in this field before seem to have stumbled across it. And they almost always end up on the side of tobacco harm reduction because they're coming in with a fresh perspective, not really understanding the debate and the polarization in the literature, but that's always reassuring to see somebody with a methodologically rigorous mindset and what conclusions they come to.