Cell phones have produced a dramatic decline in household coverage in telephone surveys using RDD landline samples. According to the most recent National Center for Health Statistics (NCHS) estimates, more than one in every five American homes (20.2%) had only wireless telephones during the second half of 2008. This limitation for telephone surveys using traditional RDD samples is now widely recognized, and dual frame samples are increasingly used in surveys to address it.
There is also increasing concern that an additional segment of households who are currently covered by landline sampling frames cannot be reached by landline telephone surveys because of their reliance on cell phones. According to the most recent NCHS estimates, 24.4% of households with both wireless and landline telephones received all or almost all of their calls on wireless telephones. This “wireless mostly” population represented 15.4% of adults in U.S. households in July–December 2008. If these persons are not answering their landline telephones, then a third of adults in telephone households may now be unreachable by traditional RDD telephone surveys.
The size of the covered, but effectively unreachable population in RDD landline samples was directly tested in this national dual frame survey. The survey found that while 15.8% of adults in the weighted dual frame sample received all or almost all of their calls on the cell phones, only 3.8% reported that it was very unlikely or not at likely that the household landline would be answered if it rang when someone was at home. Moreover, the majority of these “cell phone mainly” adults who said it was very unlikely or not at all likely that their landline would be answered actually completed the survey on their landline telephone.
These findings support the hypothesis that a portion of telephone households covered by landline sample may not be reached by RDD telephone surveys. However, this covered but unreached segment is much smaller than the “cell phone mostly” estimates generated by NCHS. This data suggests that the cell phone only population remains the central problem for RDD telephone surveys, although the unrepresented cell phone mainly population will contribute to the effective coverage error in traditional RDD telephone surveys.
The National Health Interview Survey was modified in 2007 to add a new question for persons living in families with both landline and cellular telephones. Respondents in this national face-to-face household survey were asked if the telephone calls received by their family were “all or almost all received on cell phones, some received on cell phones and some on regular phones, or very few or none received on cell phones.” Persons living in households in which all or almost all calls were received on cell phones were identified as “wireless-mostly”. During the first NHIS measurement period in which the wireless-mostly question was used (July–December 2007), NCHS estimated that 22.3% of households with both cellular and landline phones received all or almost all of their calls on cellular phones. The wireless mostly households constituted 13.1% of households and 14.0% of adults in the United States. The authors noted “this report is the first to demonstrate that the number of adults living in wireless-mostly households in the U.S. is growing and is nearly equal to the number of adults living in wireless-only households” (Blumberg and Luke 2008).
The concept of “wireless mostly” introduced the potential problem of the “reach” of traditional telephone surveys in households with cell phones as a corollary of the diminished sample “coverage” in these households. Although households with both cell phones and landlines were covered by landline random digit dialing samples, did some portion of these households rely on their cell phones to the effective exclusion of their landline phones? The authors noted: “If the prevalence of wireless-mostly households continues to grow, and if adults living in wireless-mostly households rarely (if ever) answer their landline telephones, landline telephone surveys may experience increasing rates of nonresponse” (Blumberg and Luke 2008).
This new source of potential non-response to traditional telephone surveys is important for three reasons. First, it is large — perhaps affecting 14.0% of adults in the United States in 2007. Second, it is growing — from 12.6% of adults in the first six months of 2007 to 14.0% in the second six months. Third, the characteristics of the wireless mostly population vary significantly from other households. During the July–December 2007 survey period, the NCHS researchers found that adults living in poverty or near-poverty were more likely to be living in wireless-mostly households, than were those in metropolitan areas, those with college degrees, and certain other demographic subgroups.
The National Center for Health Statistics has continued to track the wireless mostly households in the National Health Interview Survey. Based on the July–December 2008 survey period, NCHS estimated that now 24.4% of households with both cellular and landline phones received all or almost all of their calls on cellular phones. The wireless mostly households increased to 14.5% of households and 15.4% of adults in the United States. During the July–December 2008 survey period, the NCHS researchers continued to find that adults living in poverty or near-poverty were more likely to be living in wireless mostly households, than were those in metropolitan areas, and those with college degrees. Adults living with children were more likely than adults living alone or with other adult relatives to be found in wireless mostly households.
Cell Phone Mainly
The wireless mostly construct of the NCHS researchers is implicitly a measure of propensity to respond to a telephone survey. However, while certain population subgroups, such as males, young adults, minorities, etc., may have a lower propensity to respond to telephone surveys, this is a tendency, not an absolute barrier. By contrast, the guiding concept behind wireless mostly is that these individuals will rarely if ever answer a landline telephone call. These individuals have a near zero probability of participating in a telephone survey based on an RDD landline sample.
While we believe that this is an important and useful concept for telephone research, the specific measure currently used in the NHIS survey may not be optimal for this task. The cell phone only question used by NCHS is: “Of all the telephone calls that you or your family receives, are: All or almost all calls received on cell phones; Some received on cell phones and some on regular phones; or Very few or none on cell phones?” Although this appears to be an appropriate measure of the “volume” of all calls received by channel (landline phone versus cell phone), it does not specifically address the “likelihood of someone answering” a call received on the landline phone. How often someone uses their cell phone is fundamentally different than whether they answer their landline phone when it rings.
In developing the 2007 New Jersey Health Care Opinion Poll, the researchers from the Rutgers Center for State Health Policy and Abt SRBI developed a more direct measure of the propensity of adults with cell phones to respond to landline calls. The respondents who had both landline and cell phones in their household were asked the likelihood that the landline in their household would be answered if it rang. “Thinking just about the LANDLINE home phone, NOT your cell phone, if that telephone rang, and someone was home, under normal circumstances how likely would it be answered. Would you say it is: extremely likely the landline would be answered, very likely, somewhat likely, somewhat unlikely, or not at all likely the landline would be answered?” This more direct measure of likelihood of answering the landline in dual channel households was called “cell phone mainly” by the authors. The study found that the direct question yielded a much lower rate of “cell phone mainly” respondents from dual channel households in the statewide telephone sample than the proportion of “wireless mostly” adults reported by the NCHS from dual channel households in their national survey (Cantor et al. 2008). This statewide survey of cell phone mostly and cell phone mainly households was repeated and expanded in the 2008–2009 New Jersey Health Care Opinion Poll. The cell phone mostly and cell phone mainly questions were asked of those with both cell phone and landline phones in the household in both a statewide landline RDD and statewide cell phone sample. This study found that 17% of respondents from the statewide RDD landline sample reported that they (and other adults in their household) received all or almost all of their calls on their cell phone (cell phone mostly), but only about a third as many said that it was somewhat unlikely (2.0%) or not at all likely (4.5%) that the landline phone in their household would be answered if it rang when someone was home.
These findings suggest that the current measure of wireless mostly used in the National Health Interview Survey may not meet the objective of defining the sub-population of dual users covered by landline sampling frames, who rarely if ever answer their landlines because of their reliance on their cell phones. However, these two New Jersey surveys are based on statewide samples, rather than a national sample. It would be more instructive to compare the findings from a national dual frame sample using both measures of the cell phone mostly and cell phone mainly populations to evaluate the two definitions.
The 2009 Traffic Safety Culture Index Survey conducted by Abt SRBI for the AAA Foundation for Traffic Safety provided an opportunity to test the cell phone mainly measure compared to the wireless mostly measure among a dual-frame national sample of households. The first frame consisted of a total of 1500 completed telephone interviews with adults aged 16 and older from a national, list-assisted random digit dialing (RDD) sample of households with landlines. The second frame consisted of 1001 completed interviews with adults aged 16 and older from a national sample of cell phone banks. The landline sample was stratified to oversample non-Metropolitan Statistical Area (rural) counties as classified by the U.S. Census. The age of eligibility for the survey was 16, rather than 18, because this is the usual age requirement for drivers’ licenses in most states and driving was the primary subject matter of this survey. The field period for this telephone survey was April 15, 2009 to May 12, 2009.
The landline sample was weighted to correct for the disproportionate sampling of rural areas, the number of landline numbers in the household, and the number of adults in the household. The cell phone sample was weighted only on number of cell phones used regularly for non-business purposes. The weighted landline and cell phone samples were then combined and weighted to NCHS estimates of their relative population sizes of four telephone use segments (landline only, dual user not cell phone mostly, cell phone mostly and cell phone only). Finally, the sample was weighted to Census estimates of the distribution of the population on the basis of geography, age and gender, race and ethnicity, education, and marital status.
Findings: Cell Phone Mainly
The dual users in the landline sample were asked the likelihood that their landline would be answered if it rang when someone was home. About one in twenty dual users said that it was very unlikely (2.8%) or not at all likely (1.9%) that their landline home phone would be answered if it rang when someone was home. This translates into approximately 3.5% of the landline sample who would be classified as cell phone mainly (Table 1).
In the landline sample, the survey finds only a limited correspondence between the proportion of calls received on cell phones among dual users and the likelihood of answering the landline phone if it rang. Only a little more than one in ten “cell phone mostly” respondents report that it was very unlikely (8.1%) or not at all likely (3.3%) that their home landline phone would be answered if it rang (Table 2). The proportion of dual users in the landline sample who would be very unlikely (0.7%) or not at all likely (0.9%) to answer their landline phone if it rang was less among those who reported some family calls on cell phone and some on landlines. But the relationship between the two questions about volume and propensity to respond is not consistent with the proportion who were very unlikely (1.4%) or not at all likely (2.1%) to answer the landline, increasing somewhat among those who reported receiving very few or none of their calls on the landline. Indeed, the Pearson correlation between cell phone mostly and cell phone mainly in the landline sample was only 0.191.
Similarly, the dual users in the cell phone sample were asked the likelihood that their landline home phone would be answered if it rang when someone was home. A little more than one in ten dual users in the cell phone sample said that it was very unlikely (5.7%) or not at all likely (4.8%) that their landline home phone would be answered if it rang when someone was home. This translates into approximately 5.6% of the cell phone sample who would be classified as cell phone mainly (Table 3).
In the cell phone sample, the survey finds a slightly stronger correspondence between the proportion of calls received on cell phones among dual users and the likelihood of answering the landline phone if it rang. But, only a little more than one in five “cell phone mostly” respondents in the cell phone sample report that it was very unlikely (10.6%) or not at all likely (10.8%) that their home landline phone would be answered if it rang (Table 4). The proportion who are very unlikely or not at all likely to answer their landline phone if it rang declines to 2.4% of those who report some of the calls received by their family are on the landline and some on the cell phone. But, among those who report very few or none of their calls are received on the cell phone, the proportion of who say it would be very unlikely (8.9%) or not at all likely (2.5%) their landline would be answered if it rang, increased somewhat. Hence, the Pearson correlation between cell phone mostly and cell phone mainly in the cell phone sample, likewise, was only 0.200.
As noted earlier, the dual frame sample was weighted to correct for the geographic oversampling of non-MSA areas in the landline sample, the relative size of the cell phone and landline populations, and select population characteristics. In the combined weighted sample, the cell phone mainly adults (16 and older) represented 3.8% of the sample. Since the weighted estimate for cell phone mostly in the sample (15.8%) was virtually identical to the NCHS estimates of the size of this population (15.4%), we believe that the survey estimate of the size of the cell phone mainly population (3.8%) should be a reasonably accurate estimator of the true size of this population (Table 5).
Not only is the proportion of respondents small (3.8%) who report that it would be very unlikely or not at all likely that a ringing landline in their household would be answered if someone was home, but that proportion may still over-represent adults in telephone households who would be covered but unreached by an RDD telephone survey. Nearly two thirds of these cell phone mainly respondents (2.3% of all respondents) were interviewed on their landline phone. Only 1.5% of adults interviewed in a national dual frame survey were cell phone mainly and interviewed on the cell phone (Table 6).
This survey finds that a segment of the population who have landline telephones and, hence, are covered by RDD samples for telephone surveys report that it is “very unlikely” or “not at all likely” that the landline phone in their household would be answered if it rang and someone was home. This “cell phone mainly” population segment is “covered” by RDD sampling frames but, according to respondents self-report, very unlikely to be “reached” by telephone surveys. According to the estimates of this dual frame survey, approximately 3.8% of adults in households with (either landline or wireless) telephones in the United States would be classified as cell phone mainly.
The pioneering work on “wireless mostly” households introduced the concept of a covered but not reached population as a further possible source of bias in traditional RDD surveys in the United States. Their initial measure of the potentially unreached population — wireless mostly — was based on the proportion of calls received by cell phone (all or almost all) in dual user households. This survey, however, finds that there is only a modest correlation between proportion of calls reported received on cellular phones compared to landlines and the reported likelihood that the household landline phone would be answered if it rang. Hence, we believe that the more direct measure of likelihood of the phone being answered should be a more accurate measure of the reach of telephone surveys within households covered by RDD samples. However, both “cell phone mainly” and “cell phone mostly” are self-reported measures of behavior rather than direct observational measures.
The size of the covered but unreached population for RDD telephone surveys using the cell phone mainly measure (3.8%) is only one fourth as large as the cell phone mostly measure (15.8%). Moreover, the majority of the cell phone mainly respondents in the dual frame survey were actually interviewed on their landline telephones. Out of the 3.8% of potentially covered but unreached adults in the sample, 2.3% were drawn from the landline sample, while 1.5% came from the cell phone sample. Hence, this survey suggests that the proportion of adults in the United States who have a landline home phone, but are not accessible by their household landline for telephone surveys, appears to be very small at this time.
While the covered but unreached population of “cell phone mainly” adults is not large enough to represent a tipping point in telephone RDD surveys, it does further exacerbate the coverage bias in exclusively landline surveys. We believe that this is one further reason for recommending dual frame surveys at least when the population characteristics of interest are expected to be under represented in the landline sampling frame. We also believe that the size of the cell phone mainly population should be monitored in future dual frame surveys because increases in the size of this population segment might lead to different sample design recommendations.