Introduction

Address based sampling (ABS) is a viable alternative as a sampling methodology to random digit dialing, especially since it is a frame that allows researchers to address issues related to: 1) exclusion of cell phone only homes, 2) number portability issues, and 3) the decline in participation of younger hard-to-reach demographic (as cited in Link et al. 2009). The body of literature to date in this area has focused on how ABS can assist us in addressing the issue of achieving better representation particularly of hard-to-reach subgroups. But when an ABS frame is used to successfully contact these hard-to-reach subgroups like 18–34 year olds, African Americans and Hispanics, what is their mode preference when responding to the survey? Thus, the primary focus of this paper is to further assess mode preferences among those who are successful contacted and choose to respond with specific interest with 18–34 year olds, African Americans, and Hispanics.

Our examination of mode return preferences will compare and contrast data collected across four different measurement periods in February, May, July, and November of 2011 taken from an ABS-based frame and from a television audience survey. The primary purpose is to examine when respondents are given a choice in how to return a survey, what is their desired mode of response – to return the survey by mail, web, or phone? Responding via a mode that is different than the contact mode suggests a preference for that tool. The implication being that respondents may be more likely to complete future surveys if contacts are made via their response mode choice. For example, if a survey is initially sent via mail with a mail, web, or phone response option and a respondent chooses to complete the survey via web, perhaps he or she is more likely to complete a follow up survey administered online and sent to his or her e-mail address as opposed to a second mail survey. This is of particular interest if hard-to-reach demographics choose a response mode that is different than the contact mode, because it has implications for increasing representation among hard-to-reach demographics in follow up surveys.

Mixed-Mode Survey Approach

Researchers use multiple survey modes in order to reduce costs and maximize response rates (Groves et al. 2009). Mixed-mode survey designs balance the costs associated with administration and survey error (de Leeuw, E.D. and Hox 2008). Our mixed-mode survey uses one mode for recruitment with the possibility of a different mode for survey administration. Generally, in mixed-mode surveys, the least expensive mode is used to make contact while mail and more expensive modes that increase response are used as follow-up like telephone (Biemer and Lyberg 2003). Typically across the literature, response rates are the highest for telephone followed by mail then web. On average, web surveys experience an 11 percent lower response rate than other modes (Fan and Yan 2010).

A mixed-mode survey is a survey where multiple and different data collection modes are used to make contact with respondents or to complete the total questionnaire (de Leeuw, E.D. and Hox 2008). Each additional mode introduced comes with the possibility of introducing additional error (Biemer and Lyberg 2003). However, research indicates respondents tend to stick with the contact mode out of comfort or ease of access (Manfreda et al. 2008). Analysis will explore the mode choice of Nielsen respondents across time and among different demographic groups.

Methods

The surveys were conducted in February, May, July, and November 2011 as part of a larger TV survey recruitment process that Nielsen conducts several times a year with the ultimate goal of getting households to agree to keep a TV diary for 1 week. The surveys were based on ABS sampling of targeted households stratified throughout 189 of the largest cities areas in the United States For each survey month, there were a total of 767,011 addresses for February; 795,622 for May; 794,583 for July; and 987,372 for November; all records were limited to regular sample. Of these records, approximately 60 percent of the sample can be matched to a telephone number while the remainder of sampled addresses cannot be matched to a phone number. It is these unmatched records (40 percent) where there is no phone number available that is of interest to us, and consequently, a survey is sent to these homes with the goal of obtaining a phone number to include them in the phone recruitment stage which occurs later in the TV recruitment process.

In the process of trying to obtain a phone number, sampled address are mailed three pieces of mail: 1) pre-recruitment mailing, 2) pre-recruitment survey asking questions about their TV viewing behavior but more important asking for their phone number, and 3) a pre-recruitment reminder letter. Last, households are informed that they can complete the pre-recruitment survey and return it by mail, complete the survey online or by phone.

Results

Table 1 depicts the sample sizes for the unmatched portion of the sample where a phone number is not available, and consequently, all these households were mailed a survey in an attempt to obtain their phone number. Table 2 summarizes the response rate by mode of preference selected by households that decided to respond and return the survey – either responded by mail, web, or phone. Across the four survey periods, the mail mode was the most preferable method of response chosen by respondents followed by the web and then by phone.

Tables 3–5 show the demographic breakdown of those who responded by mail, web, or phone. Across all demographics that were examined, mail was the mode of preference among the younger cohort, nonethnic, Blacks, and Hispanics. Though the numbers are relatively small for web and phone compared to mail, these hard to reach subgroups are still choosing to use the web and phone as a response option. For example, 18–34 year old cohort, were more likely to respond using the web than by telephone (see Table 3). And when looking at race alone, Black respondents preferred to respond by phone whereas nonethnics were more likely to select the web option (see Table 4). Last Table 5 indicates that the Hispanic subgroup (though not as clear across phone, web, or mail as found for age and race), Hispanics were more likely to respond via phone than non-Hispanics.

Conclusions

Data suggest that the mail mode is the primary mode of survey return and alternative modes like web and phone are still modes chosen (though not the most preferable) when offered as an option to these harder-to-reach groups (i.e., younger demographics, African Americans, and Hispanics). Though the response rates for these alternative modes are fairly low, there still remains utility in offering these modes to insure that we are able to maximize responses from harder to reach subgroups (18–34-year-old, African Americans, and Hispanics) as we are able from these underrepresented subgroups (as noted in Tables 3 and 5).

In a mixed-mode environment, the next steps would be to consider contacting certain demographics by their mode preference in the initial contact. A further step would be to consider conducting a mode order test pushing different demographics to the mode preferences identified in this study. For example, the findings here would suggest that if there were multiple modes available to collect survey responses, the web option should be offered first to those between the ages of 18–34 years as well as nonethnics and that phone as a mode should be considered for Hispanics. Hypothetically, if we were to consider the total mail rate of 17.1 percent for say the February measurement compared to all those who would responded across all modes of 21.7 percent (mail, phone, web), the metrics would increase slightly to improve representation of African Americans, Hispanics, and 18–34 year olds (see Table 6). On average the return rate increases by 4–5 percent for any given reported measurement. Last, from a cost perspective, since all respondents are mailed a survey with a noncontingent incentive and a response is really dependent on a respondent’s personal choice of responding by mail, going to a website, or by phone the cost is very negligible or cost neutral since incentives have already been sent to respondent and infrastructure has already been created to gather data via website or by phone center. In the end, mail as an option appears to be the most effective mode of response when looking at the overall rate of return and should always be offered, but other modes should be considered in conjunction with mail and could easily vary depending on the population you want to participate.