The Gallup Organization has been using a “generic ballot” question to estimate the number of seats the major political parties will win in their national pre-election polls since 1942, and many national polling organizations now routinely use this measure. Gallup’s wording is:
If the election for Congress were being held today, which party’s candidate would you vote for in your congressional district – The Democratic Party’s candidate or the Republican party’s candidate?” IF UNSURE: “As of today, do you lean more toward the Democratic Party’s candidate or the Republican Party’s candidate?
The Gallup generic ballot has historically been quite accurate in estimating congressional seats (Moore and Saad 1997), but critics argue that it is actually not that accurate of an early indicator (Erickson and Sigelman 1995); it underestimates support for incumbents and does not do a good job in predicting individual races (McGhee and Baldassare 2004); it overestimates Democrats (Blumenthal 2006); and that the relationship had weakened in recent elections largely (Franklin 2006).
Despite these criticisms, the generic ballot question is at the heart of many models used to predict the composition of Congress (e.g., Abramowitz 2010; Bafumi, Robert, and Wlezien 2010) and because of its perceived utility, it is regularly included in national polls and has become a staple of political punditry.
Generic Ballots in the States
In the past 20 years there has been an explosion of state polls. These have historically been conducted by in-state university researchers or local media, but increasingly, national media and out-of-state universities have gotten into state polling. State polls typically cover presidential, gubernatorial, senatorial, and occasionally congressional races, but beginning in 2010, some firms [notably Public Policy Polling (PPP)] have included generic ballot questions for state legislatures. This research tests whether the generic ballot question can accurately predict legislative outcomes in New Hampshire.
Why New Hampshire?
New Hampshire presents an excellent state to test the generic ballot in state elections as the New Hampshire House of Representatives is very large (the third largest legislative body in the English speaking world), and Representatives are largely unknown to voters — only 23 percent of New Hampshire likely voters would even venture a guess who any of their state representatives were (Smith and Hubbard 2000). On the other hand, the New Hampshire senate is considerably smaller, only 24 senators, but they are not much better known than representatives, only 30 percent volunteered that they could name their state Senator (Smith and Hubbard 2000). There is a considerable body of research on party heuristics that suggest voters in low salience elections would be most likely to rely on partisan cues (see Schaffner and Streb 2002) and the low salience of state legislative elections, such as New Hampshire, are an excellent place to test this.
Voting data for 2000 through 2010 come from the New Hampshire secretary of sate. Results for all House and Senate districts were summed and the percent Republican of the two-party vote was calculated. Generic ballot data come from the final pre-election poll of likely New Hampshire voters conducted by the University of New Hampshire Survey Center. The wording of the House generic ballot is:
Please think about the election for your representative to the New Hampshire House of Representatives. Do you plan to vote for the Democratic or Republican candidate for the New Hampshire House of Representatives … or do you plan to skip the election for New Hampshire House of Representatives this time?
and the wording of the generic Senate ballot is:
Please think about the election for your representative to the New Hampshire Senate. Do you plan to vote for the Democratic or Republican candidate for the New Hampshire Senate … or do you plan to skip the election for New Hampshire Senate this time?
In both cases, undecided voters were probed “Are you leaning toward voting for the Democratic candidate or for the Republican candidate?” These “leaners” are included as supporters of the party they lean to. The field period for each poll was the weekend before the election, typically Thursday to Sunday. Sample sizes range from 600 to 850 likely voters (sampling errors range from ±3.8 percent to ±3.4 percent).
A high percentage of respondents were still undecided about which party’s candidate they would vote for, even the weekend before the election and after having been probed. Undecided voters are handled in two ways – they are either ignored in the two-party vote calculation or are allocated to the party they are most likely to vote for using an algorithm developed by the University of New Hampshire Survey Center.
Looking first at the New Hampshire House, the generic ballot does a less than impressive job of predicting the final Republican vote percentage (Table 1), the average error is 4.3 percent, ranging from four to six percent, error drops slightly to 4.2 percent when undecided voters are allocated and is three percentage points higher than the 1.3 percent reported by Moore and Saad (1997). Similar to what others have found Democratic support is overestimated for each election (Franklin 2006; Moore and Saad 1997).
Turning to the New Hampshire Senate, the generic ballot does a somewhat better job. The mean error is 2.7 percent for unallocated poll results, improving to 2.3 percent when undecided respondents are allocated. Similar to house predictions, in all but one of these six elections, Democratic support was overestimated.
Generic ballots do not perform well in directly predicting the actual number of seats that a party will win, either at the national level or within a state. This is unsurprising as Republican and Democratic voters are not equally distributed over the country nor are they likely to be equally distributed within individual states. Political parties have long recognized this and have engaged in the practice of gerrymandering — drawing districts that advantage their party and disadvantage the opposing party.
To account for this, Gallup developed a simple regression model using the number of seats won in past elections as the dependent variable and the estimated two-party percentage from the generic ballot question as the independent variable (Moore and Saad 1997). This model greatly improves Gallup’s seat estimate (R2 = 0.91, standard error of estimate of 7.8 seats).
Similar models were developed for New Hampshire using results from 2000 to 2008 in order to predict the 2010 results. One model uses the unallocated undecided figures and the second uses the allocated figures. The actual number of Republican seats won is the dependent variable and the percent Republican from the generic ballot question (either unallocated or allocated) is the independent variable. The equations are: Predicted R Seats = –224.276 + 906.706* (unallocated generic House ballot) and Predicted GOP Seats = –309.034 + 1076.681* (allocated generic House ballot). Both the unallocated model (R2 = 0.99, standard error of estimate = 9.0 seats) and the allocated model (R2 = 0.96, standard error of estimate = 12.8 seats) have very good fits. They are also very good in predicting the results of the 2010 election, the unallocated model predicted Republicans would win 288 seats and the allocated model predicted 294 seats, very close to the 298 Republicans won (Table 2).
The decision of whether to allocate undecided voters does not seem have much practical importance as there is essentially no difference in these predictions (Figure 1).
Senate predictions are quite accurate as well (Figure 2). The equations for the Senate are: Predicted GOP Seats = –19.073 + 65.455* (unallocated generic Senate ballot) and Predicted GOP Seats = –24.646 + 76.829* (allocated generic Senate ballot). Again, both the unallocated model (R2 = 0.87, standard error of estimate = 1.5 seats) and the allocated model (R2 = 0.90, standard error of estimate = 1.5 seats) have very good fits. Both models predicted that Republicans would win 18 Senate seats in 2010 and they won 19. Over the six elections in this study, these estimates were off an average of 1.2 and 0.8 seats and did not favor either party.
The 2010 state legislative elections were extremely important in that these bodies (in most states) will determine U.S. congressional as well as state legislative districts, greatly impacting legislative control for the next 10 years. But political analysts had only a limited ability to predict the outcomes of these elections. Using generic ballots to predict the composition of Congress is a useful tool for political prognosticators and the experience of using generic ballots in New Hampshire shows that it can work at the state level as well.
There are some obvious limitations to this research. First, it was conducted in a state which may indeed be unique. But given that it works with both a very large House and a very small Senate, and with the unusual nature House districts, one would expect that it would work in other states as well. This remains an empirical question, but as it is a relatively low cost addition to polls (two questions), and that it can has been shown to be very accurate, one hopes that more researchers will adopt it as part of their election prediction tool kit. The biggest drawback to its adoption is that it needs to be used in a few elections to build regression models that greatly improve its accuracy.
A second drawback (Franklin 2006) is that the generic ballot nationally became less accurate in the 1990s due to the increasingly sophisticated ability of legislators to pack partisans into solid Republican and Democratic seats through redistricting. The redistricting following the 2010 Census will provide an opportunity to see just how great an impact this will have in New Hampshire. A new model may have to be developed in the wake of each round of redistricting.
A third limitation may be the likely voter model employed by researchers. A model that more stringently defines likely voters could perhaps reduce the percentage of respondents most likely to rely most heavily on party heuristics, the less informed voter who is also least likely to vote. The above analysis does include equal numbers of presidential elections, when turnout in New Hampshire typically exceeds 70 percent, and mid-term elections, when turnout hovers closer to 40 percent, suggesting that turnout models. But in presidential years, the average error in predicting House seats is 7.0 percent (unallocated model) and 8.7 percent (allocated model) compared with 5.7 percent (allocated) and 7.0 percent (allocated model) which is not what one would expect, given that a higher percentage of less-informed voters typically vote in presidential elections compared to mid-term elections.
New Hampshire may be unique, and pollsters in other states will have to calculate their own regression model in order to adjust their predictions, but by adding two simple questions to their state polls, pollsters can immediately provide a relatively accurate estimation of legislative outcomes, and with a few years of data, calculate state specific models that are highly accurate in predicting state legislative elections.
The Eagleton Institute at Rutgers University has already adopted this item.