Introduction and Background
Landline telephone samples are very accurate in terms of geographic location (Lavrakas et al. 2007). This level of accuracy is based on known geographic relationships with the sampled phone number. It allows for a proportional distribution of the sample across any geographic jurisdiction. Cell phone sample assignments are based on limited knowledge of geographic locations. The area code of the telephone number may not accurately indicate the location of the potential respondent (Chowdhury et al. 2018; Kafka, Chattopadhyay, and Chan 2015). Rate centers may provide additional information on locations of cell phone respondents (Christian, Dimock, and Keeter 2009; Pew Research Center 2015). Since the population which moves from county-to-county or state-to-state tends to be younger, more likely to be minority, male and of disproportionately lower income (Mateyka 2015), the loss of geographic information in telephone samples that shift from landline based telephone numbers to cell phone numbers may result in bias.
A rate center delineates the local call boundaries set by service providers for cell phone billing purposes. These boundaries are updated quarterly and can be mapped to Census blocks. Each Census block is assigned to a single rate center. The rate center is able to identify, to some degree, the geographic locations of cell phone numbers (TeleCom Routing Adminstration 2018), based on the identification of the cellular 1K block assignments.
In 2016, there were approximately 504,159,000 working cellular phone numbers in the United States. Generally, cell phone users are known to be younger, disproportionately minority, more likely to live in urban areas, and have higher levels of education (Chowdhury et al. 2018; Tarnai, Schultz, and Moore 2009). Of the 3,142 U.S. counties, 3,097 have rate centers (98%). Approximately 2,804 counties have rate centers with at least 1 active cellular 1K block. The counties with no rate centers (45) nor any active 1K cellular blocks (293) are assigned to neighboring counties.
The Behavioral Risk Factor Surveillance System (BRFSS) conducts a state-based health related survey to over 400,000 respondents annually in the United States and participating territories. In 2016, the sample of residents in the states and Washington, DC totaled 477,665 of which 230,013 (48.2%) were cell phone respondents (Centers for Disease Control and Prevention 2016). The BRFSS sample includes a county assignment for each potential respondent phone number. Since samples are drawn by state, respondents are asked to identify the correct state of residence during the screening process. County of residence is asked later in the survey. If a respondent lives in a state other than the one of the sample, interviews are completed, and data are transferred to the correct state of residence. Using this information, from the sample and the responses, levels of accuracy can be determined for state and counties. For those respondents who complete the entire survey, demographic comparisons and health outcomes can also be measured.
Using the cell phone respondents from the 2016 BRFSS, three levels of accuracy were produced: correct state/county, correct state/incorrect county, and incorrect state/county. The accuracy groups were then compared by phone ownership (dual user or cell phone only), demographic characteristics, and health outcomes. Finally, accuracy groups were compared at various geographic levels. Rates of accuracy were mapped by state and by county to illustrate accuracy relationships among adjacent counties.
Results and Discussion
Table 1 illustrates the responses to demographic and health outcome questions as well as the single question on phone use. Persons most likely to have incorrect state and county matches were more likely to be younger, male, with some college (not achieving a college degree), and less likely to be married or self-identify as “white.” Interesting, while those who have incorrect county assignments were less likely to smoke, they were more likely to use e-cigarettes, and more likely to report binge drinking. In terms of chronic disease, incorrect matches were less likely to report that they had been diagnosed with chronic illness. Persons most likely to have incorrect matches were disproportionately cell phone only (rather than dual phone users). All of the differences by county accuracy reported in Table 1 are statistically significant.
As Table 2 illustrates, there were differences by region and metropolitan status as defined by the Census (U.S. Census Bureau 2018) in the ability to predict cell phone respondents’ county of residence. Western states had higher levels of accuracy, as did micropolitan areas. There were also differences by individual states (see Table 3). Two jurisdictions (Hawaii, Washington DC) did not complete interviews with out-of-state residents, so county accuracy was not calculated. Alaska collected interviews for out-of-state cell phone respondents but did not collect county information for those respondents.
Overall, the county-level accuracy rate was 58%. Nine states had low (under 50%) accuracy, 34 states had accuracy rates between 51%–70%, and 5 states had high accuracy rates (over 70%). For some states, the accuracy was uniform across the state. For example, of the 16 counties in Maine, only 4 had rates under 70%. In New Jersey, rates were more uniformly low, with only 5 of 21 counties having high (over 70%) accuracy. Of the 63 Florida counties, 26 had high rates of accuracy (over 70%), and 19 had rates under 50%, making it more difficult to assess the accuracy across the state as a whole. Accuracy at the state level was at 93% (correct state but incorrect county) for all cell phone samples.
The Figure 1 illustrates the distribution of respondents’ county by the total number of respondents who were sampled for King County, WA. A total of 1,818 respondents who completed the interview were sampled as King County. Of those, 1,581 actually resided in the county (87%). The Figure 1 shows the number of respondents from the King County sample who reported other counties (a few respondents reported other states). As Figure 1 illustrates, most of the incorrect county assignments were for residents of neighboring counties. Counties with larger number of responses were shaded in darker tones.
In Figure 2, Ramsey County, MN, is highlighted. There were a total of 760 completed interviews from persons identifying themselves as residents of Ramsey County. Ramsey is one of the 45 counties without a rate center. As the figure illustrates, 651 (86%) of Ramsey County respondents were from the rate center in neighboring Hennepin County. The 18 who are indicated by the sample assignment as being from Ramsey are from landline-to-cellphone ported numbers, not from rate center assignments. Only a few Ramsey County residents were originally sampled from other counties within Minnesota.
Most counties have rate centers which can be used to assist in the sampling of cell phone samples at the county level. County-level cell phone sample accuracy rates are overall at slightly over 50% but vary widely by state, region, and metropolitan status. Even within states, there are counties which have accurate rates of cell phone sampling, and others that have low accuracy rates. In most instances, respondents with incorrect county assignment live in neighboring counties. In counties without rate centers, most respondents are found to be sampled from more urban neighboring counties.
This research shows that it is possible to create rates of cell phone sampling accuracy by counties. Generally, the demographic profile and health characteristics of persons with incorrect county assignment was supportive of literature on cell phone only respondents in other studies. Persons who were most likely to have incorrect assignments were younger, less likely to report chronic health conditions and more likely to engage in some health risk behaviors, such as e-cigarette use and binge drinking. They were also more likely to engage in physical activity outside of the workplace and report higher levels of overall good health. Overall, we found that cell phone samples are accurate at the county level 58% of the time and 93% of the time at the state level. However, accuracy rates vary by state, region, and metropolitan status as well as by demographic characteristics and survey items. Researchers should exercise caution when interpreting data from cell phone samples which do not include respondents who have moved into the geographic areas of interest.
As will all research efforts, this project had a number of limitations. The District of Columbia and the State of Alaska did not collect information on county for out-of-jurisdiction respondents, so data for those respondents was not available. Only respondents who completed the survey were able to provide the county of residents, and in some counties, the total number of completes was very low. The metro status accuracy rate was highest for micro status, followed by metro and non-metro; however, the metro status as measured by the BRFSS is not precise as Census designations of metro status and is based on self-reported information on locations of residence.
The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Carol Pierannunzi, PhD
National Center for Chronic Disease Prevention and Health Promotion
Centers for Disease Control and Prevention
4770 Buford Highway NE MS S107-6
Atlanta, Georgia 30341