Appending a Prepaid Phone Flag to the Cell Phone Sample

Kyley McGeeney Pew Research Center, 1615 L. St. N.W., Suite 800, Washington, DC 20036, USA

Abstract

Telephone sample providers offer a flag that can be appended to cell phone samples to identify which phone numbers belong to prepaid or “pay as you go” cell phones. These phones tend to be used by people with lower incomes, a group that is often unrepresented in surveys. The Pew Research Center tested whether this flag could be used to correctly identify prepaid cell phone numbers and therefore the hard to reach demographic groups that typically use such phones. While the flag succeeded in identifying these groups, using this flag to oversample them is probably too costly and inefficient for a general population study. The Pew Research Center would only consider using this flag in the future for studies of particularly hard to reach demographic groups but not for general population studies.

Background

Cell phone sample providers offer a flag that indicates whether each phone number belongs to a prepaid or non-prepaid cell phone. This is important because prepaid phones are owned by minorities and individuals with low incomes or low levels of education, who are typically harder to reach in telephone surveys. If this prepaid flag does correctly identify respondents with these demographics, it would be possible for researchers to oversample cell phone numbers flagged as prepaid in order to increase the number of these hard-to-reach individuals in our samples. It was necessary to test whether this flag correctly identifies respondents with these demographic characteristics as well as understand how oversampling based on this flag would work.

Literature Review

The published literature on the prepaid phone flag is scant. David Dutwin tested the prepaid phone flag and reported the results in his American Association for Public Opinion Research (AAPOR) webinar on cell phones (Dutwin 2014). He found that 12 percent of the cell phone sample was flagged as prepaid. He also found that respondents whose numbers were flagged as prepaid were more likely to rent their home, be non-white, have lower incomes and less formal education, and have only a cell phone with no landline. He found no differences in region, age, size of household or status as a parent. The Pew Research Center study expanded on the variables examined to include political and some attitudinal measures.

Prepaid phone flags are part of a larger offering by Targus of cell phone activity flags. These were examined by Dutwin and Malarek (2014) in the Survey Practice article “The use of recent activity flags to improve cellular telephone efficiency” and by Schalk et al. (2015) in the AAPOR presentation “Implications for weighted survey estimates when flagged-inactive cell phones are excluded or subsampled.”

Methods

The analysis in this report is based on telephone interviews conducted February 18–22, 2015, among a national sample of 1,504 adults, 18 years of age or older, living in all 50 U.S. states and the District of Columbia (There were 526 respondents interviewed on a landline telephone, and 978 were interviewed on a cell phone, including 559 who had no landline telephone). The Pew Research Center survey was conducted by interviewers at Princeton Data Source under the direction of Princeton Survey Research Associates International. A combination of landline and cell phone random digit dial samples were used; both samples were provided by Survey Sampling International. The Targus prepaid flag was appended to the entire cell phone sample. Interviews were conducted in English and Spanish. Respondents in the landline sample were selected by randomly asking for the youngest adult male or female who is now at home. Interviews in the cell sample were conducted with the person who answered the phone, if that person was an adult 18 years of age or older. For detailed information about the survey methodology, see http://www.pewresearch.org/methodology/u-s-survey-research/.

Group Unweighted sample size Plus or minus …
Total sample 1,504 2.9 percentage points
Cell sample 978 3.6 percentage points
Prepaid cell sample 151 9.1 percentage points
Non-prepaid cell sample 827 3.9 percentage points

Findings

Incidence and Response Rates

In the survey, 12.4 percent of numbers in the total cell sample were flagged as prepaid cell phones while 15.4 percent of completed interviews were flagged as coming from prepaid cell phone numbers.

The response rate did not differ dramatically between the two groups of cell phone numbers. The response rate for the prepaid flagged cell numbers was 9.2 percent vs. 9.7 percent for the non-prepaid flagged cell numbers. It is interesting to note that the prepaid flagged numbers had a much higher e11, which can be considered a measure of the working residential rate. This makes sense if one assumes that the numbers either are working or were working recently in order to have been flagged as prepaid, although limited information from Targus leaves it is unclear how the prepaid flag is constructed to know for sure whether that is true. The higher working rate was augmented by a higher contact rate for the prepaid flagged numbers. However, these were counteracted by a slightly lower cooperation rate for individuals with prepaid flagged cell phone numbers.

Incidence rates
Total cell sample Cell completed interviews
% %
Prepaid cell 12.4 15.4
Non-prepaid cell 87.6 84.6
100 100
Unweighted sample size 23,120 978
Data are unweighted. Survey conducted February 18–22, 2015.
PEW RESEARCH CENTER
Response rates
Total cell sample Prepaid cell Non-prepaid cell
% % %
Contact 61.4 67.2 60.4
Cooperation 15.6 13.8 16.0
e1 60.2 78.5 57.5
Response rate (RR3) 9.6 9.2 9.7
Unweighted sample size 23,120 2,871 20,249
Survey conducted February 18–22, 2015.
PEW RESEARCH CENTER

Demographics

Demographically the respondents with prepaid flagged cell phone numbers are significantly less likely to be white and more likely to be Hispanic (37 percent self-identify as Hispanic compared to 11 percent of the non-prepaid cell sample). This in turn results in significantly more Spanish language interviews in the prepaid sample (27 percent vs. 4 percent for non-flagged cell phone numbers). The respondents with prepaid flagged cell phone numbers are also lower educated, lower income, self-identify as lower class and are more likely to report that the recession had a major effect on their finances. They are more likely to live in urban areas and more likely to only have a cell phone with no landline. Politically they are less likely to be registered to vote, less likely to be Republican or Republican leaning and more likely to be independent or non-leaning. Religiously they are less likely to be Protestant and more likely to be Catholic although there is no difference in their religious attendance.

Demographic characteristics by sample type
Total cell sample Total cell sample Prepaid cell Non-prepaid cell
Weighted Unweighted
% % % %
Male 56 62 59 62
Female 44 38 41 38
18–29 27 22 19 23
30–49 38 32 36 32
50–64 24 27 33 26
65 or more 11 17 13 18
White, non-Hispanic 61 67 42* 71
Black, non-Hispanic 12 10 15 9
Hispanic 16 15 37* 11
Other 9 7 4 7
Interviewed in Spanish 8 8 27* 4
High school grad or less 36 30 48* 27
Some coll/Assoc. degree 34 28 27 29
College grad or higher 29 41 25* 44
<$30K 33 28 53* 24
$30K–74,999 34 33 32 34
$75K or higher 28 31 7* 36
Upper, upper-middle class 12 14 8* 15
Middle class 46 49 40* 51
Lower-middle class 30 27 28 27
Lower class 11 9 22 6
Sample size 978 978 151 827
*Prepaid group statistically significantly different than non-prepaid group.
Survey conducted February 18–22, 2015.
PEW RESEARCH CENTER
Recession had major effect, finances not recovered 30 31 44* 29
Recession had major effect, finances recovered 28 29 26 29
Recession did not have major effect 41 39 28* 41
1 Person household 17 19 23 19
2 People 27 31 28 31
3 People 21 19 15 19
4 People 20 17 16 17
5 or more people 15 13 18 12
Registered to vote 66 71 52* 74
Republican/Lean rep 41 45 30* 47
Democrat/Lean dem 49 46 53 44
Independent-no lean 10 10 17* 8
Protestant 41 42 30* 44
Catholic 22 22 32* 20
Other 7 7 9 7
Unaffiliated 29 28 27 28
Attend services weekly or more 31 33 32 33
Attend less than weekly 68 67 66 67
Northeast 15 13 10 14
Midwest 21 22 23 22
South 41 41 40 41
West 24 23 26 23
Rural 14 17 15 17
Suburban 50 49 42 50
Urban 36 34 43* 33
Dual landline/cellphone 33 43 32* 45
Cellphone only 67 57 68* 55
Sample size 978 978 151 827
*Prepaid group statistically significantly different than non-prepaid group.
Survey conducted February 18–22, 2015.
PEW RESEARCH CENTER

Self-Reported Prepaid Phone Status

Self-reported prepaid phone status
Total cell sample (WTD) Total cell sample (UNWTD) Prepaid cell (UNWTD) Non-prepaid cell (UNWTD)
% % % %
Self-reported contract cell 65 66 14* 76
Self-reported prepaid cell 30 29 81* 19
Unweighted sample size 978 978 151 827
*Prepaid group statistically significantly different than non-prepaid group.
Survey conducted February 18–22, 2015.
PEW RESEARCH CENTER

In an effort to assess the accuracy of the flag, we asked respondents what kind of cell phone plan they had: one with a contract for a set period of time or one without a contract, such as a prepaid phone, GoPhone or pay-as-you-go plan (see Appendix for exact question wording). Respondents are not expected to be entirely accurate with this since many cannot even accurately report what type of phone they have when asked in the past. Approximately eight in ten respondents’ self-reported prepaid phone status matched their cell phones’ prepaid flags. However, it is interesting to note that 14 percent of respondents with a prepaid flagged number reported having a contract phone and 19 percent of respondents with a number not flagged as prepaid reported having a prepaid phone.

One issue in testing the accuracy of the prepaid flag or the mismatch between the flag and self-reports is that it is unknown exactly how the prepaid flag is constructed. Inquiries made to the sample provider indicated that Targus provided extremely limited information regarding how this flag was constructed. The prepaid flag might only include TracFone type phones where one buys minutes and loads them on to their phone. It also might include phones sold by a wireless provider, such as the AT&T GoPhone, where individuals receive a bill each month but do not have a contract and can cancel at any time. This makes constructing a question to match the prepaid flag, and thus ascertaining the error either in self-reports or in the flag itself, almost impossible.

However, the utility of the flag may not depend on perfect precision. If researchers are just trying to target certain hard to reach demographic groups such as Hispanics, and this flag allows them to do that, it does not necessarily matter if all flagged phone numbers actually belong to prepaid phones.

Considerations in Oversampling Based on This Flag

Oversampling based on this flag is actually quite costly. It is a post-append flag, meaning one buys sample as usual, and then adds this flag. If researchers use it to oversample flagged numbers, then they must discard part of the non-flagged, but otherwise useful, sample. So they need to pay not only for the flag itself but also for a lot of additional sample they are not going to use (and the flag for that sample).

Additionally, it might not be the most efficient way to oversample certain hard to reach groups. Rate centers and other frame data may do a better job at targeting certain groups. That being said, the design effect (DEFF) from oversampling based on this flag using a modest oversample, e.g., 1.5 prepaid to 1 non-prepaid pieces of sample, may result in a general population sample that is almost self-weighting. The need for less post-stratification weighting will reduce the design effect and counteract the DEFF from the unequal selection probability in the oversampling.

Conclusion

The prepaid cell phone flag works very well to identify hard to reach demographic groups. The response rate for flagged numbers is also not substantially lower than the response rate for non-flagged numbers. However, using the flag for oversampling, given that it is a post-append procedure, is costly and inefficient. Given these difficulties, the Pew Research Center will most likely not use this flag in the sample design of general population studies; however it, may be of future interest for telephone studies of particular hard to reach populations.

References

Dutwin 2014
Dutwin, D. 2014. Cellular telephone methodology: present and future [Webinar]. In: AAPOR Webinar Series. Available at: http://www.aapor.org/AAPORKentico/Education-Resources/Online-Education/Webinar-Details.aspx?webinar=WEB0114.
Dutwin and Malarek 2014
Dutwin, D. and D. Malarek. 2014. The use of recent activity flags to improve cellular telephone efficiency. Survey Practice 7(1).
Schalk et al. 2015
Schalk, M., K. McGeeney, C. Kennedy, C. Turakhia and D. Williams. 2015. Implications for weighted survey estimates when flagged-inactive cell phones are excluded or subsampled. Presented at the Annual Conference of the American Association for Public Opinion Research, Hollywood, FL.

Appendix

ASK ALL CELL PHONE SAMPLE:

PREPAID: Thinking about the cell phone you are using right now, what kind of cell phone plan do you have? Is your plan based on a contract for a set period of time or do you not have such a contract, such as a prepaid phone, GoPhone or pay-as-you-go plan?

1 Have a contract

2 No contract, such as prepaid phone, GoPhone, pay-as-you-go plan

3 Employer pays for cell phone (VOL.)

8 Don’t know what kind of plan (VOL.)

9 Refused (VOL.)

Footnotes
1 According to the AAPOR standard definitions, e1 is the estimated frame eligibility of non-contacts.


About Survey Practice Our Global Partners Disclaimer
The Survey Practice content may not be distributed, used, adapted, reproduced, translated or copied for any commercial purpose in any form without prior permission of the publisher. Any use of this e-journal in whole or in part, must include the customary bibliographic citation and its URL.