Introduction

Although it is likely that persuasion attempts by interviewers are an important tool to reduce refusal rates in CATI surveys (Dijkstra and Smit 2002), little is known about the dynamics in interviewer-respondent interaction and critical elements in persuading reluctant respondents.

For most people, survey participation is not an activity of utmost importance. Therefore, Groves, Cialdini, and Couper (1992) argue that in deciding whether or not to participate, most potential respondents take a heuristic approach, i.e., devoting only small amounts of time or cognitive energy. Groves, Cialdini, and Couper (1992) apply the six principles of compliance (Cialdini 1988) to the survey participation request. Accordingly, Dijkstra and Smit (2002) assume that most rejections reflect the heuristic approach and that interviewers who are more successful in persuading respondents make use of these compliance principles. Observing interviewer-respondent introductory CATI interactions in a nonexperimental study, they found persuading respondents by means of a more personal style (which emphasized the principle of liking) was more successful than persuading respondents by means of a more formal style (which emphasized the principle of authority). In addition, applying the principle of social validation was not successful. In the current experiment, we aimed to examine these findings and use behavior coding as a manipulation check. In any experiment involving interviewer behavior, a manipulation check is essential to distinguish ineffectiveness of the treatment from inadequate implementation of the treatment. For example, Dijkstra (1987) used behavior coding as a manipulation check in an experiment concerning manipulations of a personal and a formal style of interviewing. Other experiments involving interviewer behavior (e.g., Houtkoop-Steenstra and Bergh 2000; O’Neil, Groves, and Cannell 1980) did not implement such a manipulation check, while this could have been useful in explaining their findings.

Methods and Data

An experiment was designed to study the effects of two persuasion styles of interviewers on respondents’ participation behavior. The study was implemented in a CATI survey on health-related behaviors and attitudes. It was conducted in February and March 2004. The twelve participating interviewers received general instructions on how to behave during the introductory interaction (see Table 1). Specific instructions, as derived from Dijkstra and Smit (2002), were given to six interviewers trained in the personal style and six trained in the formal style.

Data Collection

A sample of 1,500 telephone numbers was drawn from a website with telephone listings of Dutch households. Candidate-interviewers were recruited through various university communication channels. The selection criteria were: some experience and affinity with interviewing, social skills, and availability for all sessions. The interviewers were all female students aged between 19 and 28, who were financially compensated.

The training consisted of three six-hour sessions, during which interviewing techniques were discussed and practiced. The interviewers interviewed for four evenings within a period of two weeks. During the fieldwork, the interviewers were monitored by means of digital recordings. If necessary, interviewers were instructed to improve their behavior.

Coding of the Data

In total, 1,131 contacts with unique phone numbers were analyzed (only the last call to each contact was stored). The sound files were transcribed and coded in Sequence Viewer by two graduate students. The coding scheme that was used was based on a scheme used in earlier studies (Dijkstra and Smit 2002). The coded transcripts were systematically analyzed for rare codes and common errors. The first researcher independently double-coded 109 interactions (i.e., entire calls). The overall Kappa value of coding for all variables (0.74) indicates substantial agreement. More details on the coding scheme and reliability analysis are available upon request from the first author.

Results

Main Effect of Persuasion Style

Although interviewers who were trained in the formal style have a slightly higher cooperation rate (58 percent) than interviewers trained in the personal style (56 percent), the two experimental groups do not differ significantly in participation rates (χ² [1, n=1,131] 0.498, _p=_0.48, ω=0.02). This result indicates that by looking exclusively at the manipulation both persuasion styles had similar effects on cooperation. However, since we do not know whether interviewers indeed behaved as instructed, we decided to analyze the interactions in more detail.

Immediate Compliance

Using interviewer-respondent interaction coding, actual interviewer behaviors were analyzed and compared to the instructions the two interviewer groups received. It appears that in most cases that end up in an interview (78 percent), respondents agree to be interviewed immediately after the interviewer’s request. In addition, in six cases, respondents hung up the phone during the request (i.e., blocking declinations, Schaeffer et al. 2013). The proportion of immediate acceptances appears to be slightly higher among interviewers in the formal style (81 percent) than among interviewers in the personal style (75 percent), (χ² (1, n=1131) 2.58, p=0.11, ω=0.05). However, among the cases where no immediate acceptance occurred (n=594), there is no difference in the proportion of cases ending up in an interview among formal interviewers (22 percent) and personal interviewers (24 percent), (χ² (1, n=594) 0.55, p=0.45, ω=0.03). So, different training protocols did not differentially increase participation.

Actual Use of Arguments

The use of arguments by interviewers was verified for those cases where there was no immediate acceptance and no blocking declination (n=588). Table 2 provides a manipulation check for implementation of personal and formal persuasion styles. There is no difference in the percentage of cases where interviewers provide any type of argument, although this percentage appears to be slightly higher in the personal style of interviewing. It is striking that the personal interviewers, against instructions, did mention study importance. However, this argument indeed was used more often by the formal interviewers than in the conversational group. Similarly, the social validation argument was used more often in the formal group, whereas personal interviewers more often used personal importance than formal interviewers.

Participation Per Argument

Since the manipulation check showed interviewers did use different arguments (but sometimes against instructions), we further explored which arguments are more successful in obtaining cooperation. In Table 3, the response rates are given depending on the type of arguments used by interviewers. Four panels are displayed which correspond to the persuasion styles and the compliance principles. In panel A, calls during which the interviewer uses any argument anywhere in the interaction, are compared with calls during which the interviewer gives no argument at all. Panel A shows that it is useful to give any argument after a respondent’s refusal move; the response rate increases to 28 percent when interviewers provide an argument. We also verified whether this effect is the same for formal and personal interviewers, which indeed was the case (personal interviewers: χ² (1, n=309) 13.1, p<0.01, ω=0.21, formal interviewers: χ² (1, n=279) 20.9, p<0.01, ω=0.27). Panels B, C, and D show specific arguments within the three principles of compliance. It is possible that interviewers used multiple different arguments within a call. Also, if interviewers used the same argument multiple times, this call still appears only once in the table. Panel B shows that it is useful to mention the importance of the study for the university. Moreover, although higher participation rates were not found for interviewers who were instructed to use this argument (i.e., in the formal group, χ² (1, n=279) 0.25, p= .615, ω=0.03)) higher participation rates were found for interviewers who were not instructed to use this argument (i.e., in the personal group, χ² (1, n=309) 8.80, p<0.01, ω=0.07).

A logistic regression model with participation as the dependent variable, and interviewer group (reference=personal), use of the authority argument and interaction was marginally significant χ² (3, n=588) 7.34, p=0.06, Nagelkerke R²=0.019, with significant between interviewer group and argument (b=1.347, standard error (SE) [b], p<0.01, Exp [b] (95 percent confidence interval)). The odds ratio of 3.95 shows that personal interviewers who use this argument have an odds ratio four times higher than formal interviewers in getting respondents to agree to be interviewed.

Panel C shows that mentioning personal relevance increased participation. Formal interviewers, who were not instructed to use this argument, did use this argument in three cases (which appeared to be two different interviewers), all of which were successful (although due to the low numbers this finding cannot be verified statistically). Finally, panel D shows that mentioning that most people enjoy the survey is only moderately successful. Personal interviewers never used this argument (as instructed), and for formal interviewers, this argument did not significantly increase participation.

Conclusion and Further Research

From our findings, we conclude that a failure of interviewers to apply their training could not account for the absence of a difference in participation between the two groups of interviewers. Although in the majority (78 percent) of the cases that eventually ended up in an interview (n=610), householders complied without any need for persuasion, interviewers managed to successfully persuade 21 percent of householders who initially refused (n=588). Participation among initially declining householders increases close to 30 percent when interviewers use an argument of any kind in persuasion. This effect was not different between the two differently trained groups of interviewers. Therefore, our first conclusion is that any argumentation is better than none. However, the causal direction of this effect is, of course, not clear. It is possible that interviewers’ expectations of success drive the results; interviewers might only make the effort to provide an argument when from the householder’s reaction they expect to be successful in persuading that particular person.

Looking at arguments actually used in the interaction, interesting findings arise. For the importance of the study as an argument, we found a significant interaction effect showing that interviewers trained in the personal style (not instructed to use this argument) had more success in gaining cooperation than interviewers trained in the formal style (instructed to use this argument). Likewise, two interviewers trained in the formal style of interviewing, who stressed (against the rules in their condition) their personal relevance, were equally successful in using that argument as interviewers in the personal style (who were specifically instructed to do so.) Therefore, we posit that interviewers using arguments as instructed develop too much of a unauthentic routine in expressing these arguments, whereas using arguments outside instructions are likely to be expressed in a more natural, spontaneous way and therefore more convincing. In addition, providing interviewers with too specific instructions on which argument they should use also moves them away from tailoring arguments to specific householders. Both tailoring (Groves, Cialdini, and Couper 1992) and spontaneous behavior (i.e., nonverbatim reading, see Houtkoop-Steenstra and Bergh 2000) have been demonstrated to be most effective in persuading reluctant householders.

In addition, this study shows that it is useful to include manipulation checks in experiments involving interviewer behavior. Researchers testing the effects of a manipulation of interviewer behavior should take into account more detailed checks on how these behaviors were actually implemented.

Erratum: This article was updated on March 28, 2018 to correct errors in Table 3 and accompanying text. These changes had no effect on the conclusion. The original version of the article may be found in the supplemental materials.