Loading [Contrib]/a11y/accessibility-menu.js
Skip to main content
Survey Practice
  • Menu
  • Articles
    • Articles
    • Editor Notes
    • In-Brief Notes
    • Interview the Expert
    • Recent Books, Papers, and Presentations
    • All
  • For Authors
  • Editorial Board
  • About
  • Issues
  • Blog
  • Subscribe
  • search

RSS Feed

Enter the URL below into your favorite RSS reader.

http://localhost:25298/feed
Articles
Vol. 4, Issue 1, 2011January 31, 2011 EDT

Incentives for Cell Phone Only Users: What Difference Do They Make?

Robert W Oldendick, Dennis N Lambries,
survey practice
https://doi.org/10.29115/SP-2011-0002
Survey Practice
Oldendick, Robert W, and Dennis N Lambries. 2011. “Incentives for Cell Phone Only Users: What Difference Do They Make?” Survey Practice 4 (1). https:/​/​doi.org/​10.29115/​SP-2011-0002.
Save article as...▾

View more stats

Abstract

Incentives for Cell Phone Only Users: What Difference Do They Make?

Providing incentives to potential respondents has been shown to be one of the more effective means of encouraging participation in surveys across modes of data collection. The increasing number of cell-phone only (CPO) households has sparked an interest in best practices for interviewing cell phone respondents. One aspect of interviewing cell phone respondents that has not been adequately addressed is the question of the need to provide some type of remuneration to those who complete the interview on a cell phone. In this research, we provide evidence that suggests that providing such remuneration does not have a significant impact on either response rates or results.

Although the literature on this topic is not extensive, there have been several studies of this topic (Dillman 2007; Singer 2002; Trussell and Lavrakas 2004). For example, Brick and his colleagues (2007) found in a national survey of cell phone households that a $10 incentive significantly improved participation over a $5 incentive. Similar studies by Diop and his colleagues (Diop et al. 2008; Diop, Kermer, and Guterbock 2008) found that a $10 incentive produced a higher rate of production compared to no incentive or to a $5 incentive. These earlier studies were based on unscreened dual-frame designs that included all cell phone users – that is, those both with and without landlines – while our research focused on those in cell phone only households, which may account for some of the difference in the findings.

In the early 2000s, most initial approaches to contact cell phone users presumed that some type of incentive or remuneration would be given. This was usually done as a reimbursement for possible charges incurred by the respondent, although in some cases they may have functioned as incentive payments. One of the conclusions from the Cell Phone Sampling Summit II was that “survey respondents reached on their cell phone should be properly reimbursed for their time on a research call” (Lavrakas and Shuttles 2005). As the cost structure of cell phone billing has changed and more users have moved to virtually “unlimited minutes” plans, does reimbursement of cell phone users remain a sine qua non?

To test the impact of incentives on response rates for cell phone users, we conducted experiments as part of two dual-frame telephone surveys of the adult population of South Carolina done in November and December 2009. When a number was reached, a series of screening questions determined whether the person was contacted on a cell phone, was 18 years of age or older, a resident of the state, and did not have a landline residential telephone.

The survey introduction requested potential respondents to participate in “a confidential study of public opinion” being conducted by the University. All procedures in the “incentive” and “non-incentive” conditions were the same, except at the end of the survey introduction those in the incentive condition were read the statement, “If you are eligible and complete this study, you will receive a $10 Wal-Mart gift card.” (A more extensive description of the procedures used in this study is available from the authors).

Table 1 provides the final call dispositions for 1500 numbers dialed in each of these conditions. As expected given the independent samples in each condition, the percentage of not-in-service and ineligible (business, changed number, household with a landline, respondent under 18, or not a resident of the state) are quite similar, with the largest differences (between conditions in Survey I) being in the percentage of not-in-service numbers (32.7 percent vs. 36.8 percent) and the percentage of those contacted who were not in CPO households (15.4 percent vs. 13.1 percent).

Table 1  Call Dispositions – Cell Phone Samples, Incentive and Non-Incentive Conditions.
Survey I
Survey II
Incentive Non-Incentive Incentive Non-Incentive
No Answer/Busy 39 (2.6) 35 (2.3) 39 (2.6) 38 (2.5)
Answering machine 240 (16.0) 245 (16.3) 262 (17.5) 268 (17.9)
Not in service 490 (32.7) 552 (36.8) 480 (32.0) 472 (31.5)
Business/Changed 44 (2.9) 44 (2.9) 23 (1.5) 37 (2.5)
Language/Ill 6 (0.4) 5 (0.3) 10 (0.7) 11 (0.7)
Has LL/Not cell 231 (15.4) 197 (13.1) 221 (14.7) 221 (14.7)
Under 18/Not res. 59 (3.9) 85 (5.7) 86 (5.7) 77 (5.1)
Unable to complete 123 (8.2) 107 (7.1) 107 (7.1) 136 (9.1)
Refusals 181 (12.1) 162 (10.8) 199 (13.3) 158 (10.5)
Completions 87 (5.9) 68 (4.5) 73 (4.9) 82 (5.5)

More importantly for our purposes, the percentage of completed interviews did not vary substantially across conditions, ranging from 5.9 percent in the Survey I incentive condition to 4.5 percent in this survey’s non-incentive condition. The standard response rates (AAPOR RR4) for these four conditions were:

Survey Standard response rates (AAPOR RR4) for four conditions
Survey I, Incentive Condition: 17.8
Survey I, Non-Incentive Condition: 16.2
Survey II, Incentive Condition: 15.0
Survey II, Non-Incentive Condition: 16.9

The slightly higher response rate in the expected direction in Survey I is offset by the difference in the opposite direction in Survey II. In neither case was the difference in response rates between conditions large enough to make the case that providing an incentive produced a substantial difference.

The lack of difference between the two conditions is also reflected in the demographic composition of the two samples. Table 2 shows the background characteristics for each of the cell conditions, as well as those for the landline samples. As would be expected given the characteristics of those living in CPO households, the cell phone samples reached a much higher percentage of younger people, minorities, men, and those from lower income families than did the landline samples. When the characteristics of the respondents in the cell phone conditions are compared, none of the differences is significant at the 0.05 level. The only difference that approaches significance is that for age in Survey II, which had a higher percentage of those under 30 in the non-incentive condition and a higher percentage of 30–44 year olds in the incentive condition.

Table 2  Demographic Comparisons - Cell Phone Conditions and Landline.
Survey I
Survey II
No Inc Incent Landline No Inc Incent Landline
AGE
 18–29 36.7 29.6 6.3 46.3 30.1 6.7
 30–44 30.0 34.6 18.8 18.8 35.6 18.1
 45–64 30.0 28.4 42.3 31.3 30.1 42.1
 65 or over 3.3 7.4 32.7 3.8 4.1 33.2
 (N) (60) (81) (640) (80) (73) (630)
 p* NS <0.10
RACE
 Black 23.2 34.1 25.2 40.7 31.5 23.5
 White 68.3 56.1 71.6 51.9 65.8 72.7
 Other 8.3 9.8 3.2 7.4 2.7 3.8
 (N) (60) (82) (647) (81) (73) (637)
 p* NS NS
SEX
 Male 52.9 52.9 41.2 52.4 49.3 43.8
 Female 47.1 47.1 59.8 47.6 50.7 56.2
 (N) (68) (87) (673) (82) (73) (657)
 p* NS NS
EDUC
 Less than high school 11.5 9.8 10.7 17.5 6.8 11.3
 High school graduate 27.9 30.5 26.3 37.5 30.1 28.7
 Some college 37.7 41.5 27.9 22.5 32.9 27.7
 College graduate 23.0 18.3 35.0 22.5 30.1 32.3
 (N) (61) (82) (642) (80) (73) (638)
 p* NS NS
FAMILY INCOME
 Less than $50,000 67.9 63.9 56.8 65.2 61.3 53.6
 $50,000 or more 32.1 36.1 43.2 34.8 38.7 46.4
 (N) (53) (72) (521) (69) (62) (503)
 p* NS NS

p*=chi-square test for differences between the incentive and non-incentive cell phone conditions.

Similarly, there were very few differences in responses on the substantive questions in these surveys between the cell phone groups. Of the 29 closed-ended items in Survey I, significant differences between the groups were found on three, each involving participation in the arts; of the 24 closed-ended items in Survey II, no differences were significant to the 0.05 level, and three were significant at p<0.10.

Overall, the results of this study in terms of response rates, the demographic composition of respondents, and the substantive results provide little indication that providing an incentive to participants makes an appreciable difference. While there are other considerations involved in the decision as to whether to provide remuneration to such participants, this decision does not appear to have a significant impact upon response rates or the results.

Finally, we recognize some of the limitations of this study. First, it focused on cell phone only users, who may be more likely to have unlimited minute plans and be less concerned about remuneration. This factor may be key to understanding the differences between the results of this study and those of previous research that found significant effects for incentives, in that cell phone only users may be more likely to have extended or unlimited minutes plans and be less concerned about the cost of the call. Second, only one type of incentive was offered, and increasing the value or altering the type of incentive may produce different results. Third, the study is based on a limited number of cases. It is also a single state study and these results may not generalize to other geographic areas. Despite these limitations, we found the results convincing enough to change our standard survey practice for cell phone surveys from offering remuneration to participants to not doing so.

References

Brick, J.M., P.D. Brick, S. Dipko, S. Presser, C. Tucker, and Y. Yuan. 2007. “Cell Phone Survey Feasibility in the U.S.: Sampling and Calling Cell Numbers versus Landline Numbers.” Public Opinion Quarterly 71 (1): 23–39.
Google Scholar
Dillman, D.A. 2007. Mail and Internet Surveys: The Tailored Design Method. New York: John Wiley and Sons.
Google Scholar
Diop, A., D. Kermer, and T.M. Guterbock. 2008. “Prince William County Citizen Satisfaction Survey: Report of Results 2007.” University of Virginia Center for Survey Research.
Diop, A., Y. Kim, J.L. Holmes, and T.M. Guterbock. 2008. “Prince William County Cell-Phone Pilot Study: A Supplement to the 2007 Citizen Satisfaction Study.” University of Virginia Center for Survey Research.
Lavrakas, P.J., and C.D. Shuttles. 2005. “Cell Phone Sampling Summit II: Proposed Statements on ‘Accounting for Cell Phones in Telephone Survey Research in the U.S.’”
Singer, E. 2002. “The Use of Incentives to Reduce Nonresponse in Household Surveys.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little. New York: John Wiley and Sons.
Google Scholar
Trussell, N., and P.J. Lavrakas. 2004. “The Influence of Incremental Increases in Token Cash Incentives on Mail Survey Response: Is There an Optimal Amount?” Public Opinion Quarterly 68 (3): 349–67.
Google Scholar

This website uses cookies

We use cookies to enhance your experience and support COUNTER Metrics for transparent reporting of readership statistics. Cookie data is not sold to third parties or used for marketing purposes.

Powered by Scholastica, the modern academic journal management system