This paper uses a pre-election poll conducted by the Centro de Investigaciones Sociológicas (CIS)[1] to demonstrate how election forecasts in Spain could be improved by using two modes of data collection. As a result of the differences in coverage and nonresponse across modes, the face-to-face and computer-assisted telephone interviewing (CATI) samples provide distributions of some of the population attributes that balance each other out resulting in a better performance of the combined sample than either sample considered separately. The full results of this research appear in Diaz de Rada Igúzquiza (2010).
The survey made simultaneous use of two data collection methods: face-to-face and CATI interviews with 1,600 interviews conducted using each method (Centro de Investigaciones Sociológicas 2005). A cluster multistage sample was selected from the primary units (municipalities) and secondary units (voting districts) using proportional random sampling. All interviews were carried out in the homes of the interviewees. For the face-to-face survey, the sample of target households within each voting district was drawn using random-route sampling.[2] Households for the telephone interviews were drawn at random from the municipal telephone directories.[3] Gender and age quotas were established for the final units, following the standard practice in Spain for the selection of survey respondents.
To aid comparison of the data collection methods, every effort was made to obtain the greatest possible similarity between the two groups: by using a similar sampling technique, the same municipalities as sample points, a single questionnaire, the same dates for the fieldwork, etc. in line with the recommendations made by Bowers and Ensley (2003, 1–2). The 3,205 interviews carried out show a sample error of ±1.8 percent for the joint surveys, taking 95.5 percent as a confidence level, and p=q. When comparing the key demographic variables in the survey with the target population, we find that the complete sample matches of the population distribution quite well in most of the cases.[4] However, we find two exceptions to this trend regarding economic activity and education (see Table 1). The survey results overestimate the unemployment rate and those with only elementary and high school education by more than 5 percentage points. This overestimation is worse when taking both modes separately. (The telephone sample was worse with estimation of unemployment rate and the face-to-face sample with education level). Differences between respondents interviewed face-to-face or by telephone are in line with previous research (Bowers and Ensley 2003; Green, Krosnick, and Holbrook 2001; Wessell, Rahn, and Rudolph 2000). We found telephone respondents were better educated, more likely to be employed, and religiously unaffiliated than respondents interviewed face-to-face (Diaz de Rada Igúzquiza 2010).
Respondents’ voting intentions were obtained by asking what party they would vote for if the general elections were to take place the following day.[5] Those who did not mention a specific political party were asked which party they agreed with the most, or felt closest to.[6] It is surprising to find such wide differences in voting intentions depending on the survey mode (Table 2). Two aspects stand out: the first is the “better response” provided by the interviewees in the telephone survey. In the telephone survey, 79 percent named a specific party compared to 68 percent in the face-to-face interview[7]. The second aspect that stands out is that the telephone survey obtained higher numbers of Bloque Nacionalista Galego (BNG)[8] and Partido dos Socialistas de Galicia (PSdG-PSOE)[9] voters.
Excluding respondents who failed to indicate a specific party, 43.7 percent of the interviewees named the Popular party (PP),[10] 33.5 percent the PSdeG/PSOE, and 21.2 percent the BNG. As can be seen from Table 2, these data produce a forecast very close to the actual election results when we take into account the complete sample and the observed deviations are not statistically significant.
There is noticeably less consistency in the results when the method of data collection is considered. Comparison of the results obtained by each data collection method with the overall results shows that the face-to-face interview overestimates the PPvote while underestimating the PSdeG/PSOE and BNG votes (Table 2). The deviations in the telephone survey work in the opposite direction: lowering the predicted PP vote and increasing the predicted number of PSdeG/PSOE and BNG votes.
When comparing each of the samples with the election results, the face-to-face sample provided a better estimation than the telephone survey. The face-to-face interview overestimates the PP vote by 2.7 percent and underestimates the PSdeG/PSOE vote by 1 percent. The total imbalance with regard to the election results is 5.3 percent. The telephone survey, however, results in underestimation of the PP vote by 4.6 percent, and overestimation of the PSdeG/PSOE and BNG votes by 2.8 and 3.2 percent, respectively, with a total absolute deviation from the election results of 12 percent. Finally, note that both surveys taken together provide a better forecast (5.2 percent deviation) than either one alone.
Thus, aggregateresponses from both survey modes provide a highly accurate forecast of the actual distribution of the vote. However, when the two are considered separately, the face-to-face mode provides a closer prediction than the telephone survey. These differences raise the question of which method is the best for this type of research; to which one might respond by asking: “Is it really necessary to opt for one or the other? Why not use both?” In our case of study, complementing the telephone sample with a face-to-face sample corrects the noncoverage associated to the use of landline[11] directories as the sampling frame and halves the total deviation of the election forecast. On the other hand, the telephone sample corrects the overrepresentation of people with lower education level and contributes to a slight increase in the precision of the election forecast.
This leads naturally to the proposal of a mixed mode data collection strategy that would combine the speed and economy of the telephone survey with the representativeness that can be obtained via the face-to-face interview and provide results closer to the reality under investigation. The results of such a research strategy are not only more representative of reality but also less costly,[12] since they draw on the strengths of both survey modes, while minimizing the drawbacks of using either on its own. A high coverage rate, greater ease of achieving cooperation, and a reduction in prediction error, which are associated with the face-to-face interview, are combinedwith the economy and speed of the telephone interview (De Leeuw 2005, 233–55; De Leeuw, Hox, and Dillman 2008, 299–135; Dillman, Smith, and Christian 2009, 300–330). Note, however, that the application of this strategy calls for caution in matching the questionnaire design to the data collection method and the target population in each case (Dillman 2008, 161–75). The same questionnaire cannot be used in all types of procedures, because the interaction that takes place between the interviewer and the respondent in the face-to-face situation is very different from what happens when the respondent is interviewed by telephone.
Assigned to theMinistry of the Presidency of the Spanish Government, the Centro de Investigaciones Sociológicas, Sociological Research Centre, is an independent entity set up to study Spanish society, mainly through public opinion polls. Its present name and organization dates back to 1977, although its predecessor, the Instituto de la Opinión Pública, was founded in 1963.
Random-route is a sampling procedure commonly used in Europe to ensure random selection of addresses for face-to-face surveys. It is an alternative to full on-site enumeration and random selection of households, a technique usually employed in area probability samples in the United States. In each of the selected sampling units, voting districts in our case, a starting address is selected at random. Departing from this address, the rest of households are chosen systematically along a prescribed route (i.e., the interviewer try to contact every other building in the case of addresses with multiple households and one in three, in the case of those with only one household). The route directions give every household in the cluster an equal chance of being selected for the survey (Hoffmeyer-Zlotnik and Krebs 1996; Hoffmeyer-Zlotnik 2003; Martínez Martín 2003).
Area codes in Spain are incorporated into the subscriber’s number and allow for a perfect matching between the boundaries of the sampling area and the boundaries of the random digit dialing (RDD) prefixes when sampling at the basic regional level (province) or above (region). The efficiency of RDD decreases dramatically for samples selected from smaller administrative areas as departments or municipalities.
Note that sex, age, and geographic distribution of sample units was controlled with quotas at the fieldwork stage.
Suppose that the elections to the Galicia Parliament took place tomorrow, which political party or coalition would you vote for?
(Answered by those who failed to name a party in the previous question). In any case, which of the following parties do you feel the most affinity with or which do you consider to be closest to your own way of thinking?
This was mainly due to the fact that there was a large number of people who stated – in the face-to-face interview – that “they still did not know” and who did not answer the question. This same tendency is detected in the analysis of the subsequent questions, which refer to the different political parties.
Nationalist party.
Socialist party, left-wing.
Popular party, right-wing.
In 2005, 12 percent of households were mobile-only and are excluded from the directories used as the sampling frame for the telephone survey.
It is cheaper to conduct a survey with 1,600 face-to-face interviews and 1,600 telephone interviews than it is to conduct one with 3,200 face-to-face interviews.