Does It Really Make a Fracking Difference? The Conditional Effects of Question Wording on Support for Hydraulic Fracturing

Introduction

A persistent challenge to survey research involves the development of opinions toward complex policy issues. How do people, who often lack basic political information, form opinions on issues that are technical in nature? Our research considers this question in the context of hydraulic fracturing, an important contemporary energy policy issue.

To date, most research analyzing opinion toward hydraulic fracturing has been done using national level surveys. For example, work by Boudet et al. (2014) found that while the majority of Americans had heard nothing about fracking, those who were aware are split in their support or opposition to its use. Clarke et al. (2015) suggests that the word “fracking” elicited more negative top-of-mind and environmental associations than did the phrase “shale oil or gas development.” Finally, incorporating surveys from New York and from counties in Pennsylvania within the Marcellus Shale region with national level samples, Evensen et al. 2014 concluded that the word “fracking” carries a negative, or even lewd, connotation.

Departing from the use of national-level surveys, our research analyzes how the phrase “fracking or hydraulic fracturing” influences public perceptions toward awareness, safety, and the economic/environmental trade-offs of the drilling practice in differing statewide contexts. Utilizing surveys from Maryland and Louisiana, we consider how state of residence might alter the framing effects. Our findings suggest that the phrase “fracking or hydraulic fracturing” has little effect on awareness or perceptions of safety, yet it has a significant effect on whether the public believes state government should encourage new drilling; however, this effect is only present among Louisiana residents. Thus, results suggest that statewide context matters when explaining the differences in opinion toward hydraulic fracturing resulting from framing or question wording effects.

Question Wording Effects

The big impact of small changes in question wording on poll results is well established in the public opinion literature (Bishop, Oldendick, and Tuchfarber 1978; Borrelli and Lockerbie 2008; Lockerbie and Borrelli 1990; Schuman and Presser 1996; T. W. Smith 1987; E. R. Smith and Squire 1990). Within an environmental context, Schuldt et al. (2011) find that Democrats respond in the same way to climate change as global warming, but Republicans responded more favorably to climate change. Akerlof and Maibach (2011) find that public preferences for the terms climate change, global warming, or global climate change are related both to partisan affiliation and whether the respondent believes climate change is occurring.

Question wording effects also vary across a variety of individual level factors. For example, better educated respondents are less susceptible to question wording shifts (Narayan and Krosnick 1996; Schuman and Presser 1996) while less educated respondents are more susceptible to questions that provide additional contextual information (E. R. Smith and Squire 1990). Within the context of energy policy, Cacciatore et al. (2012) demonstrate that the public evaluates biofuels more positively than ethanol and this effect is particularly pronounced among Democrats.

Given the previous research, we expect the use of terms such as fracking or hydraulic fracturing to increase awareness and concerns about safety while decreasing support for drilling. The logic here is relatively straightforward. The terms hydraulic fracturing or fracking, compared to a generic description of the drilling practice, should be more familiar to respondents and elicit a stronger response given its established negative connotations. As noted above, opinion toward energy and environmental policy is highly partisan. Thus, we expect that opinions of Republicans, more so than their Democratic counterparts, to be more sensitive to question wording.

State context should also matter. Louisiana ranks 4th in total energy production while Maryland ranks 39th (U.S. Energy Information Administration, U.S. States 2012). These differences and their ancillary effects lead us to expect that Louisiana and Maryland residents hold fundamentally different views in regard to energy issues. Question wording should have a greater influence in Louisiana where the economy is strongly tied energy production.

Data and Methods

This research utilizes question experiments from statewide public opinion polls in Louisiana and Maryland to consider whether the inclusion if the phrase “hydraulic fracturing or fracking” influences public perceptions toward drilling practice. The data come from three separate statewide surveys in Louisiana (2012 and 2013) and Maryland (2013). See Table 1 for a summary of the surveys included in the analysis. All interviews were conducted by phone by trained interviewers at the Public Policy Research Lab at Louisiana State University and the Sarah T. Hughes Field Politics Center at Goucher College. All surveys were dual-frame with landline and cell phone respondents selected via random digit dialing. Final data are weighted to match the adult population demographics of their respective states.

Instrument Design and Experiment

Respondents were randomly assigned to one of two groups. One group received the battery of questions with the phrase “hydraulic fracturing or fracking” included while the other group received the battery of questions without “hydraulic fracturing or fracking.” All respondents received their questions in the same order – (1) awareness, (2) safety, and (3) state encouragement of drilling. Specific question wordings are provided below.

1. Awareness
   1. Fracking: How much have you heard or read about hydraulic fracturing or fracking – a process that uses the high-pressure injection of water, sand, and chemicals to remove natural gas from rocks deep in the earth’s surface – a lot, some, not much or nothing so far?
   2. Non-Fracking: How much have you heard or read about a process that uses the high-pressure injection of water, sand, and chemicals to remove natural gas from rocks deep in the earth’s surface – a lot, some, not much or nothing so far?
2. Safety
   1. Fracking: As far as you know, do you think hydraulic fracturing or fracking is a very safe method to extract natural gas from the ground, somewhat safe, not very safe, or not at all safe?
   2. Non-Fracking: As far as you know, do you think this process is a very safe method to extract natural gas from the ground, somewhat safe, not very safe, or not at all safe?
3. State encouragement
   1. Fracking: Some people say the state should encourage hydraulic fracturing or fracking because of the economic benefits, while others say the state should not encourage hydraulic fracturing or fracking because of potential environmental impact. Which comes closer to your view?
   2. Non-Fracking: Some people say the state should encourage drilling for natural gas by this process because of the economic benefits. Others say the state should not encourage drilling by this process because of the potential environmental impact. Which comes closer to your view?

Analysis

Table 2 presents a comparison of awareness, perceptions of safety, and whether the state should encourage by question wording. As Table 2 reveals, question wording matters very little in terms of self-reported awareness or perceptions of the safety. In Maryland, using the term “fracking” does appear to increase self-reported awareness but the difference is not statistically significant.

We do find significant differences for whether the state should encourage drilling, but only in Louisiana. Specifically, respondents in Louisiana were significantly less likely to say that the state should encourage drilling when the terms “hydraulic fracturing or fracking” were used.

So what explains this question wording effect, and especially its appearance in Louisiana but not Maryland? We argue the effect of question wording is contingent upon the larger economic and political context. First, overall support for drilling is generally lower in Maryland regardless of question wording. Considering the prominent role played by the oil and gas industry in Louisiana, greater statewide support for drilling is hardly surprising. Further, Maryland was considering a ban on fracking during the fielding period for the Goucher Poll. Second, according the Cook Partisan Voting Index, Louisiana is 12-points more Republican than the national average while Maryland is 10 points more Democratic. These partisan differences may help to explain not only baseline differences in support for drilling across states; they may also help to explain why the use of the term “fracking” has a larger impact in Louisiana than Maryland. Louisiana residents may be more likely than Maryland residents to adopt a standing decision to support additional drilling.

However, the use of the phrase “hydraulic fracturing or fracking,” primarily appears to give Louisiana respondents pause to reassess their support for drilling. We can see some evidence of this in the distribution of “don’t know” responses by question wording. When the phrase “hydraulic fracturing or fracking” is used, the percent of Louisiana respondents who said they “didn’t know” if the state should encourage drilling increased from 13 percent to 26 percent in 2012 and from 13 percent to 22 percent in 2013. We would expect this shift from support to uncertainty would be particular pronounced for those residents most inclined to support drilling absent additional information, i.e., Louisiana’s Republican partisans.

To test this hypothesis, Table 3 includes ordinal regressions predicting whether a respondent said the state should encourage drilling.^[1] Responses are coded 1 for respondents who say the state should encourage drilling, 0 for respondents said they did not know, and −1 for respondents who say the state should not encourage drilling. We include a dummy variable indicating whether the respondent received the question with the words “hydraulic fracturing or fracking,” a standard 7-point scale for partisan affiliation with strong Democrats coded as 1 and strong Republicans coded as 7, and interaction between these two terms. In addition, we include sex, race, education, and age as control variables^[2] as well as dummy variables to identify state and year. Finally, to provide a more stringent test of the hypothesis, we include a final model incorporating individual level awareness of fracking and perceptions of the safety as predictors. Awareness of fracking is coded as 4-point a scale ranging from 0 for respondents who reported have heard nothing so to 3 for respondents reporting have heard a lot. Perceptions of safety are coded on a 5-point scale ranging from 0 for respondents who thought fracking was not at all safe to 4 for respondents who thought fracking was very safe.

When the question includes the terms hydraulic fracturing or fracking, respondents were less likely to say the state should encourage drilling. When we include the interaction term in the model (column 2), we find that Republicans are less likely to say the state should encourage drilling only when the terms hydraulic fracturing or fracking are used in the question wording. Finally, this finding is relatively robust holding even after including measures for fracking awareness and safety.

In Figure 1, we illustrate the nature of the interaction. As the figure reveals, question wording has a more pronounced effect among Strong Republicans who are less likely to endorse hydraulic fracturing when the question wording specifically uses the term “fracking.”

Figure 1   Probability respondent said state should encourage drilling by question wording.

Conclusions

Survey researchers have long understood the importance of language to question wording and ultimately to our understanding of public opinion. In this paper, we add to this understanding by demonstrating the effect of the use of the terms “hydraulic fracturing or fracking” on public support for additional drilling. The use of language in this context matters, but its effect is contingent on local context and previously held political orientations. Specifically, question wording matters more in Louisiana where respondents are typically more inclined to support drilling. Notably, Louisiana residents do not express greater opposition when the question wording shifts to include fracking but instead grow more uncertain.