Background
Cell phone sample providers offer a flag that indicates whether each phone number belongs to a prepaid or non-prepaid cell phone. This is important because prepaid phones are owned by minorities and individuals with low incomes or low levels of education, who are typically harder to reach in telephone surveys. If this prepaid flag does correctly identify respondents with these demographics, it would be possible for researchers to oversample cell phone numbers flagged as prepaid in order to increase the number of these hard-to-reach individuals in our samples. It was necessary to test whether this flag correctly identifies respondents with these demographic characteristics as well as understand how oversampling based on this flag would work.
Literature Review
The published literature on the prepaid phone flag is scant. David Dutwin tested the prepaid phone flag and reported the results in his American Association for Public Opinion Research (AAPOR) webinar on cell phones (Dutwin 2014). He found that 12 percent of the cell phone sample was flagged as prepaid. He also found that respondents whose numbers were flagged as prepaid were more likely to rent their home, be non-white, have lower incomes and less formal education, and have only a cell phone with no landline. He found no differences in region, age, size of household or status as a parent. The Pew Research Center study expanded on the variables examined to include political and some attitudinal measures.
Prepaid phone flags are part of a larger offering by Targus of cell phone activity flags. These were examined by Dutwin and Malarek (2014) in the Survey Practice article “The use of recent activity flags to improve cellular telephone efficiency” and by Schalk et al. (2015) in the AAPOR presentation “Implications for weighted survey estimates when flagged-inactive cell phones are excluded or subsampled.”
Methods
The analysis in this report is based on telephone interviews conducted February 18–22, 2015, among a national sample of 1,504 adults, 18 years of age or older, living in all 50 U.S. states and the District of Columbia (There were 526 respondents interviewed on a landline telephone, and 978 were interviewed on a cell phone, including 559 who had no landline telephone). The Pew Research Center survey was conducted by interviewers at Princeton Data Source under the direction of Princeton Survey Research Associates International. A combination of landline and cell phone random digit dial samples were used; both samples were provided by Survey Sampling International. The Targus prepaid flag was appended to the entire cell phone sample. Interviews were conducted in English and Spanish. Respondents in the landline sample were selected by randomly asking for the youngest adult male or female who is now at home. Interviews in the cell sample were conducted with the person who answered the phone, if that person was an adult 18 years of age or older. For detailed information about the survey methodology, see http://www.pewresearch.org/methodology/u-s-survey-research/.
Findings
Incidence and Response Rates
In the survey, 12.4 percent of numbers in the total cell sample were flagged as prepaid cell phones while 15.4 percent of completed interviews were flagged as coming from prepaid cell phone numbers.
The response rate did not differ dramatically between the two groups of cell phone numbers. The response rate for the prepaid flagged cell numbers was 9.2 percent vs. 9.7 percent for the non-prepaid flagged cell numbers. It is interesting to note that the prepaid flagged numbers had a much higher e1[1], which can be considered a measure of the working residential rate. This makes sense if one assumes that the numbers either are working or were working recently in order to have been flagged as prepaid, although limited information from Targus leaves it is unclear how the prepaid flag is constructed to know for sure whether that is true. The higher working rate was augmented by a higher contact rate for the prepaid flagged numbers. However, these were counteracted by a slightly lower cooperation rate for individuals with prepaid flagged cell phone numbers.
Demographics
Demographically the respondents with prepaid flagged cell phone numbers are significantly less likely to be white and more likely to be Hispanic (37 percent self-identify as Hispanic compared to 11 percent of the non-prepaid cell sample). This in turn results in significantly more Spanish language interviews in the prepaid sample (27 percent vs. 4 percent for non-flagged cell phone numbers). The respondents with prepaid flagged cell phone numbers are also lower educated, lower income, self-identify as lower class and are more likely to report that the recession had a major effect on their finances. They are more likely to live in urban areas and more likely to only have a cell phone with no landline. Politically they are less likely to be registered to vote, less likely to be Republican or Republican leaning and more likely to be independent or non-leaning. Religiously they are less likely to be Protestant and more likely to be Catholic although there is no difference in their religious attendance.
Self-Reported Prepaid Phone Status
In an effort to assess the accuracy of the flag, we asked respondents what kind of cell phone plan they had: one with a contract for a set period of time or one without a contract, such as a prepaid phone, GoPhone or pay-as-you-go plan (see Appendix for exact question wording). Respondents are not expected to be entirely accurate with this since many cannot even accurately report what type of phone they have when asked in the past. Approximately eight in ten respondents’ self-reported prepaid phone status matched their cell phones’ prepaid flags. However, it is interesting to note that 14 percent of respondents with a prepaid flagged number reported having a contract phone and 19 percent of respondents with a number not flagged as prepaid reported having a prepaid phone.
One issue in testing the accuracy of the prepaid flag or the mismatch between the flag and self-reports is that it is unknown exactly how the prepaid flag is constructed. Inquiries made to the sample provider indicated that Targus provided extremely limited information regarding how this flag was constructed. The prepaid flag might only include TracFone type phones where one buys minutes and loads them on to their phone. It also might include phones sold by a wireless provider, such as the AT&T GoPhone, where individuals receive a bill each month but do not have a contract and can cancel at any time. This makes constructing a question to match the prepaid flag, and thus ascertaining the error either in self-reports or in the flag itself, almost impossible.
However, the utility of the flag may not depend on perfect precision. If researchers are just trying to target certain hard to reach demographic groups such as Hispanics, and this flag allows them to do that, it does not necessarily matter if all flagged phone numbers actually belong to prepaid phones.
Considerations in Oversampling Based on This Flag
Oversampling based on this flag is actually quite costly. It is a post-append flag, meaning one buys sample as usual, and then adds this flag. If researchers use it to oversample flagged numbers, then they must discard part of the non-flagged, but otherwise useful, sample. So they need to pay not only for the flag itself but also for a lot of additional sample they are not going to use (and the flag for that sample).
Additionally, it might not be the most efficient way to oversample certain hard to reach groups. Rate centers and other frame data may do a better job at targeting certain groups. That being said, the design effect (DEFF) from oversampling based on this flag using a modest oversample, e.g., 1.5 prepaid to 1 non-prepaid pieces of sample, may result in a general population sample that is almost self-weighting. The need for less post-stratification weighting will reduce the design effect and counteract the DEFF from the unequal selection probability in the oversampling.
Conclusion
The prepaid cell phone flag works very well to identify hard to reach demographic groups. The response rate for flagged numbers is also not substantially lower than the response rate for non-flagged numbers. However, using the flag for oversampling, given that it is a post-append procedure, is costly and inefficient. Given these difficulties, the Pew Research Center will most likely not use this flag in the sample design of general population studies; however it, may be of future interest for telephone studies of particular hard to reach populations.
Appendix
ASK ALL CELL PHONE SAMPLE:
PREPAID: Thinking about the cell phone you are using right now, what kind
of cell phone plan do you have? Is your plan based on a contract for
a set period of time or do you not have such a contract, such as a
prepaid phone, GoPhone or pay-as-you-go plan?
-
Have a contract
-
No contract, such as prepaid phone, GoPhone, pay-as-you-go plan
-
Employer pays for cell phone (VOL.)
-
Don’t know what kind of plan (VOL.)
-
Refused (VOL.)
According to the AAPOR standard definitions, e1 is the estimated frame eligibility of non-contacts.