Written by Sinclair Davidson

Recently The Economist published a report into a study that investigated vaping and taxation. The Economist reported the main conclusion of the study as being:

The study found that increasing ecigarette taxes reduces this, too. A $1 rise in ecigarette taxes brings    a 10-14% decline in the number of alcohol related traffic deaths per 100,000 among 16 to 20 year olds.

That seems to be a very impressive result. Yet, as always with public health related research, it should not be taken at face value. While The Economist itself does not provide any critique of the underlaying study, it does caution against the obvious policy conclusions that appear to follow from the study.

The study, ‘Can Anti-Vaping Policies Curb Drinking Externalities?: Evidence From E-Cigarette Taxation and Traffic Fatalities’ forms part of the Center For Health Economics And Policy Studies working paper series at San Diego State University and can be found at their website. It is also available at the NBER website and the SSRN website. At this time, the paper has not been published in an academic journal nor does it appear to have been subject to formal peer-review. It has very likely been workshopped and informally reviewed by the author’s colleagues and friends.

The paper itself has 5 co-authors. All of the authors are economists and identify as being labor economists and health economists. While the paper itself deploys the language of economics – the phrase ‘externality’ in the title and frequent mentions of ‘spillovers’ in the text – style of the paper is very much in the public health tradition. For example, there is no formal (or even informal) model to guide our interpretation of the empirical results. There are no hypotheses set out linking the empirical results to any model, definitions are vague and appear to subtly vary over the paper, summary statistics are not fully reported, the results of empirical estimations are not fully reported – for example, no goodness of fit statistics are reported at all – and, finally strong policy conclusions are reached that are not consistent with the evidence that has been produced. 

As with many of the papers we see in public health, there is a combination of the obvious, non-sequitur, and leaps of faith that combines with overly complex econometric technique that allows the authors to draw conclusions that are not fully supported by theory or data.

What is the purpose of this paper?

In the abstract, we are told:

This paper is the first to explore the spillover effects of e-cigarette taxes on teenage drinking and alcohol-related traffic fatalities.

Then in the introduction (pg.4), we are told:

This study is the first to study the effects of ENDS [electronic nicotine device systems] taxes on teenage and young adult drinking and alcohol-related traffic fatalities.

In the conclusion (pg. 28), we are told:

This study offers the first causal evidence on the impact of ENDS taxes on teen alcohol misuse and alcohol-related traffic fatalities.

So the authors claim to be investigating the relationship between taxation on vaping products and teen (or youth or young adult) consumption of alcohol and traffic fatalities. 

What does the paper claim to find?

From the introduction, we are told:

• ‘we confirm that ENDS taxation reduces teen ENDS use, a one-dollar increase in ENDS taxes reduces teen vaping by 5.4 percentage points (or approximately 24 percent), a substantial effect.’
• ‘we find that a one-dollar increase in ENDS taxes leads to a 1-to-2 percentage point reduction in the probability of teenage and young adult binge drinking.’
• ‘Our results indicate that a one-dollar increase in ENDS taxes results in a 0.4 to 0.6 decline in the number of alcohol-related traffic fatalities per 100,000 16-to-20-year-olds in a treated state-year.’

It is this latter result that The Economist reports on. This result is also the ‘externality’ that is found in the title of the paper. 

To be complete, what does the paper not find?

• ‘We find little evidence that alcohol use among those ages 21-and-older are affected by ENDS taxes.’
• ‘We find no evidence that ENDS taxes are related to teenage traffic fatalities that do not involve alcohol … .’

This latter point is very important – the story being told in the paper relates to the social cost of alcohol. It is alcohol in this story that contributes to traffic fatalities – not vaping, or even smoking, for that matter. Now it is true that some individuals may consume both alcohol and nicotine. Yet many consume neither, or just one of the two. The story being told in this paper is that government mandated efforts to reduce (even suppress) the incidence of vaping via taxation has the effect of also reducing alcohol consumption and, by consequence, traffic fatalities for individuals aged between 16 and 20 years old – but not for individuals over the age of 20. 

This result is so specific that it seems spurious. 

This result is also not replicated in the existing literature. Vaping is a somewhat recent innovation to the consumption of nicotine. Historically individuals have accessed nicotine via combustible cigarettes, cigars, pipes and the like. Governments have tended to tax combustible nicotine products and attempt to reduce (or suppress) consumption of these products. The authors of the paper do not report any result demonstrating an externality (or spillover) from tobacco taxation resulting in reduced alcohol consumption and consequently fewer road fatalities. 

By contrast, however, they do point to a study by Adams and Cotti (2008): 

… we observe an increase in fatal accidents involving alcohol following bans on smoking in bars that is not observed in places without bans. Although an increased accident risk might seem surprising at first, two strands of literature on consumer behavior suggest potential explanations — smokers driving longer distances to a bordering jurisdiction that allows smoking in bars and smokers driving longer distances within their jurisdiction to bars that still allow smoking, perhaps through non-compliance or outdoor seating.

It must be emphasised, the notion that the increased taxation of vaping will result in fewer traffic fatalities due to reduced alcohol influenced driving is a new, and unique, result in the policy literature.

Finally, it must be pointed out that the authors make a claim that they are performing a general equilibrium analysis. Three times they make the claim:

At page 4:

Understanding the general equilibrium effects of public health policies targeting ENDS use is necessary to document the full costs and benefits to society.

At page 28:

… in order to provide a more complete understanding of general equilibrium effects of public health policies targeting ENDS.

At page 31:

Given that ENDS taxation, and optimal ENDS policy more generally, is contentious and ongoing, considering general equilibrium effects is essential.

To be very clear – the authors simply do not provide a general equilibrium analysis of ENDS taxation. They perform a patial equilibrium analysis looking at the impact of taxation on vaping and then attempt to link that analysis to alcohol consumption and road fatalities. A general equilibrium analyses would have to, at least, incorporate substitution effects between vaping and combustible nicotine products and investigate the various (private and social) costs and benefits associated with policy choices. To be fair, the authors do indicate that increased taxes on vaping does result in increased consumption of combustible nicotine products but that insight is not incorporated into their empirical analysis. 

Is there a theoretical basis for the paper’s findings?

The authors, at page 4, offer this possible explanation:

If the adoption of ENDS taxes causes a sizable reduction in the number of ENDS users, such a policy shock could generate important changes in alcohol use, which may include drinking-related externalities with substantial social costs.

That statement is some general, and so vague, that it is difficult to dispute it. Yet we are never told what this statement could possibly mean. For example:

• It could mean that high levels of vaping taxation results in less vaping and less drinking.
• It could mean that high levels of vaping taxation results in the same amount of vaping, but less drinking. 
• It could mean that high levels of vaping taxation results in less vaping and more drinking.

The latter two possible meanings could be explained by a budget constraint – vaping and alcohol are consumed subject to a budget constraint and if one form of consumption becomes more expensive individuals substitute away from the more expensive activity to the less expensive activity. Or it could be that some individuals prefer, say, vaping to alcohol and when vaping becomes relatively more expensive they cut back on alcohol consumption to maintain their desired level of vaping.

The study simply does not explore these possibilities. We are informed that the results imply the very first possibility above. Vaping and alcohol consumption for 16 – 20 year olds are complements and the results show that increased taxation results in both less vaping and less alcohol consumption. 

Empirical Strategy

The paper combines data from 5 databases. Four of the five databases contain individual data as to the consumption of alcohol and nicotine for various groups and ages of respondents. The fifth database contains  traffic fatality data for the US by state and year. Being economists the authors estimate various sophisticated regression models and report robustness tests. While the paper is silent on the package used to estimate the regressions it is very likely to be Stata and a similar package and there is no doubt that the regressions have be correctly estimated.

There are problems, however, with the data that has been used in the regressions and in the specification of the equations. As is often the case, so inferences have to be made at either the 5% confidence level or even the 10% confidence level. In one instance the authors are reduced to telling us that the sign is in the correct direction.

A challenge with many public health research projects is that the data are collected from secondary sources and do not neatly match the purpose the researchers wish to apply it to. Furthermore control variables need to be applied – sometimes at higher levels of aggregation than the actual data. For example, in this study individual data as to alcohol consumption and vaping consumption is collected. While the paper suggests that this is done over the period 2003 – 2019, in fact vaping data are only collected after 2013. 

The questions relating to alcohol use are very broad. Any person who had at least one drink in the past 30 days is defined as being an alcohol user. Given that we are told that (some) surveys are distributed between January and June that means that anyone having had a drink over Christmas and New Year is not only a drinker but a multiple ‘offender’. As far as I can tell the regressions do not control for when the data was collected. 

They also include data bases that collects information about adult usage of alcohol and vaping. It is not clear why they do this, given that the study is about teenage drinking, vaping taxation, and fatalities. 

In the regression analysis they include control variables such as state based policy variables (at a high level of aggregation) and individual characteristics such as age, ethnicity, grade (surely highly correlated with age), sex, and in some specifications educational attainment. What they do not include are any indicators of a propensity for risky behaviour, part-time employment or some other source of income, whether or not they hold a drivers licence, or have access to a motor vehicle. In particular they do not control for whether the individual lives in a city or rural area (presumably having less access to various forms of public transport). Driving ages vary across the US by state and no attempt has been made to include this variable in the analysis. It is true that state based control variables are included in the analysis, but those variables are doing a lot work.

It is only the final dataset that directly addresses the research question that the authors claim to be investigating. 

Irrelevant Results

All of this data is used to demonstrate that higher levels of vaping taxation results in lower levels of vaping. Those results are shown in table 1. This is unsurprising. Demand curves slope down and this is how the world is meant to work.

In table 2 we see the impact that vaping taxation has on alcohol consumption. In the first panel we see there is no statistically significant relationship between ‘any alcohol consumption’ and the taxation of vaping. In the second and third panel we see that there is a statistically significant negative relationship between the number of drinks being consumed and, at least, one binge drinking incident and the taxation of vaping. This result could be consistent with a number of possible explanations, however, we cannot draw any serious conclusion from these results because the authors have not controlled for actual vaping in these results. The regression results in table 2 have a very serious omission – the lack of control for respondent vaping. 

The results in the final panel of table 2 relate to multiple binge drinking events. The author’s preferred specification is only statistically significant at the 10% level and is not robust to changes in the control variables used in the regression. 

Tables 3 and 4 contain robustness tests using a different regression analysis. Table 3 in particular shows clear negative relationships between alcohol consumption and vaping taxation. It, however, also suffers from the omitted variable bias that we saw in table 2.

In table 5, the authors investigate the overlap between those individuals who both vape and binge drink. While this group of individuals – and their propensity to get involved in traffic fatalities – is the very group that the authors claim to be investigating, very little shared about them. For example, we only discover on page 30 that 40% of teen vapers also binge drink. From the summary statistics we discover that 19.7% of teens (in the state-based sample) vape. That suggests that 7.9% of teens both vape and binge drink. While that may seem to be a high number, 19.9% of teens were classified as binge drinkers, so it would appear that 12% on teens binge drink, but do not vape. 

Table 5 is a lost opportunity. By including vaping in the dependent variable (a binary indicator) and not as a independent variable it reduces the ability for readers to form any firm views on the actualy dynamics in the data.

Tables 7 and 8 add other (adult) age groups into the mix. The results are age distributed – there are different effects for younger consumers than for older consumers. Given the stated research question, the results here are not interesting.

What is interesting are the results in table 6. Here the authors segment their data by sex, age, and ethnicity. A vaping tax reduces the number of drinks consumed by white males under the age of 17. At the 1% level of significance vaping taxes reduce binge drinking for 17 – 18 year olds, Hispanics and Other. Similarly at the 1% significance level a vaping tax reduces multiple instances of binge drinking for people of colour (Black, Hispanic, and Other). While public health academics may welcome results such as this, the fact is that the lack of consistency in the results undermines any confidence we can place in those results. It is very likely that random variation in the data is driving the random variations in the results. 

Getting to the Main Result

Table 9 contains results that address the research question that the authors claim to be answering. The results are not as promising as advertised. In this table the authors deploy data from the Fatality Analysis Reporting System (FARS). This data set contains state by state data on traffic fatalities. The authors extract the following information from the data set: ‘Total Traffic Fatalities, Traffic Fatalities with Driver BAC > 0, Traffic Fatalities with Driver BAC > 0.1, Traffic Fatalities with Driver BAC = 0 …’.

The authors claim that they have used the natural log of the ‘the age-specific traffic fatality rate (number of traffic fatalities per 100,000 population) in state s and year t’ as the dependent variable in a regression that includes vaping taxation and various state-based control variables. The authors do not explain why the have taken the natural log of the fatality rate. They also claim that some instances of a zero fatality rate as occurred and they have corrected for this by substituting the natural log of 1 (i.e. zero) in the regression. However, to my mind this suggests a data error in the analysis – it is not clear why any state in the US would have zero road fatalities in any of the age groups the authors claim to include in their analysis (16 – 20, 21 – 39, 40 and older). In the very instance the underlaying analysis is suspect.

There is a further problem with the dependent variable.

Consider how the authors describe their finding:

From the Abstract and again in the introduction:

… a 0.4 to 0.6 decline in the number of alcohol-related traffic fatalities per 100,000 16-to-20-year-olds in a treated state-year.

From page 15:

We focus on the period from 2003-2019 and generate a state-by-year panel of traffic fatalities for those ages 18-to-20, ages 21-to-39, and 40-and-older. Given our interest in traffic fatalities involving alcohol, we make use of information collected on Blood Alcohol Content (BAC) of the driver as well as the timing of the accident given that the alcohol-related fatalities frequently occur on nights and weekends.

At page 26, they describe the results in table 9 as follows:

• ‘Table 9 presents estimates of the effects of ENDS taxes on traffic fatalities among 16-to-20-year-olds, generated from equation (4).’
• First, we find that ENDS taxes are essentially unrelated to total traffic fatalities among 16-to-20-year-olds …
• ‘…our results show consistent evidence of an ENDS tax-induced decline in alcohol-involved traffic fatalities.’

At page 27:

• ‘… results imply an approximately 5-to-9 percent decline in alcohol involved traffic fatalities among 16-to-20-year-olds.’

It is very clear that they are describing fatalities amongst an age cohort (in this case 16 – 20). They are not describing the age of the driver, but rather the age of the people killed in the incident. 

By contrast also on page 15:

For traffic fatalities where the BAC of the driver is reported, the rate of traffic fatalities involving 18-to-20-year-old drivers with a BAC > 0 was 4.5 per 100,000 population. For those ages 21-to-39 and 40-and-older, the numbers are 5.9 and 2.5, respectively.

This is actually the variable that the authors should be using. Drivers aged 16 – 20 who have a BAC > 0. Yet, even here, they report the data for drivers aged 18 – 20. To be fair, this may be a typo. All the discussion and description – apart from this one instance – suggests that the authors have used fatality rate by age group as their dependent variable, not driver involved in a fatality aged 16 – 20. 

It is very likely that the authors have mis-specified their dependent variable of interest. The chain of causation that they want to demonstrate is that vaping taxation results in lower alcohol consumption amongst 16 – 20 year olds who then are less likely to cause traffic fatalities by drink driving. As it stands they are reporting results that demonstrate that vaping taxes lead to lower levels of alcohol consumption that result in fewer 16 – 20 year olds dying in traffic incidents where the driver of the vehicle is under the influence of alcohol but may not be aged 16 – 20. What makes this result even more problematic is that the authors demonstrate the effect they report only applies the individuals aged 16 – 20.  

Given this analysis it is very likely that the conclusions in this paper are based on a spurious regression.