Providing incentives to potential respondents has been shown to be one of the more effective means of encouraging participation in surveys across modes of data collection. The increasing number of cell-phone only (CPO) households has sparked an interest in best practices for interviewing cell phone respondents. One aspect of interviewing cell phone respondents that has not been adequately addressed is the question of the need to provide some type of remuneration to those who complete the interview on a cell phone. In this research, we provide evidence that suggests that providing such remuneration does not have a significant impact on either response rates or results.

Although the literature on this topic is not extensive, there have been several studies of this topic (Dillman 2007; Singer 2002; Trussell and Lavrakas 2004). For example, Brick and his colleagues (2007) found in a national survey of cell phone households that a $10 incentive significantly improved participation over a $5 incentive. Similar studies by Diop and his colleagues (Diop et al. 2008; Diop, Kermer, and Guterbock 2008) found that a $10 incentive produced a higher rate of production compared to no incentive or to a $5 incentive. These earlier studies were based on unscreened dual-frame designs that included all cell phone users – that is, those both with and without landlines – while our research focused on those in cell phone only households, which may account for some of the difference in the findings.

In the early 2000s, most initial approaches to contact cell phone users presumed that some type of incentive or remuneration would be given. This was usually done as a reimbursement for possible charges incurred by the respondent, although in some cases they may have functioned as incentive payments. One of the conclusions from the Cell Phone Sampling Summit II was that “survey respondents reached on their cell phone should be properly reimbursed for their time on a research call” (Lavrakas and Shuttles 2005). As the cost structure of cell phone billing has changed and more users have moved to virtually “unlimited minutes” plans, does reimbursement of cell phone users remain a sine qua non?

To test the impact of incentives on response rates for cell phone users, we conducted experiments as part of two dual-frame telephone surveys of the adult population of South Carolina done in November and December 2009. When a number was reached, a series of screening questions determined whether the person was contacted on a cell phone, was 18 years of age or older, a resident of the state, and did not have a landline residential telephone.

The survey introduction requested potential respondents to participate in “a confidential study of public opinion” being conducted by the University. All procedures in the “incentive” and “non-incentive” conditions were the same, except at the end of the survey introduction those in the incentive condition were read the statement, “If you are eligible and complete this study, you will receive a $10 Wal-Mart gift card.” (A more extensive description of the procedures used in this study is available from the authors).

Table 1 provides the final call dispositions for 1500 numbers dialed in each of these conditions. As expected given the independent samples in each condition, the percentage of not-in-service and ineligible (business, changed number, household with a landline, respondent under 18, or not a resident of the state) are quite similar, with the largest differences (between conditions in Survey I) being in the percentage of not-in-service numbers (32.7 percent vs. 36.8 percent) and the percentage of those contacted who were not in CPO households (15.4 percent vs. 13.1 percent).

More importantly for our purposes, the percentage of completed interviews did not vary substantially across conditions, ranging from 5.9 percent in the Survey I incentive condition to 4.5 percent in this survey’s non-incentive condition. The standard response rates (AAPOR RR4) for these four conditions were:

The slightly higher response rate in the expected direction in Survey I is offset by the difference in the opposite direction in Survey II. In neither case was the difference in response rates between conditions large enough to make the case that providing an incentive produced a substantial difference.

The lack of difference between the two conditions is also reflected in the demographic composition of the two samples. Table 2 shows the background characteristics for each of the cell conditions, as well as those for the landline samples. As would be expected given the characteristics of those living in CPO households, the cell phone samples reached a much higher percentage of younger people, minorities, men, and those from lower income families than did the landline samples. When the characteristics of the respondents in the cell phone conditions are compared, none of the differences is significant at the 0.05 level. The only difference that approaches significance is that for age in Survey II, which had a higher percentage of those under 30 in the non-incentive condition and a higher percentage of 30–44 year olds in the incentive condition.

Similarly, there were very few differences in responses on the substantive questions in these surveys between the cell phone groups. Of the 29 closed-ended items in Survey I, significant differences between the groups were found on three, each involving participation in the arts; of the 24 closed-ended items in Survey II, no differences were significant to the 0.05 level, and three were significant at p<0.10.

Overall, the results of this study in terms of response rates, the demographic composition of respondents, and the substantive results provide little indication that providing an incentive to participants makes an appreciable difference. While there are other considerations involved in the decision as to whether to provide remuneration to such participants, this decision does not appear to have a significant impact upon response rates or the results.

Finally, we recognize some of the limitations of this study. First, it focused on cell phone only users, who may be more likely to have unlimited minute plans and be less concerned about remuneration. This factor may be key to understanding the differences between the results of this study and those of previous research that found significant effects for incentives, in that cell phone only users may be more likely to have extended or unlimited minutes plans and be less concerned about the cost of the call. Second, only one type of incentive was offered, and increasing the value or altering the type of incentive may produce different results. Third, the study is based on a limited number of cases. It is also a single state study and these results may not generalize to other geographic areas. Despite these limitations, we found the results convincing enough to change our standard survey practice for cell phone surveys from offering remuneration to participants to not doing so.