1. Introduction
The survey research literature has demonstrated how interviewer characteristics can affect cooperation during interviewer-administered surveys (Davis et al. 2010; Lavrakas 2008; West and Blom 2016). These studies suggest that characteristics such as gender, Race and Hispanic origin, and age may be predictive of a sample respondent’s decision to participate in surveys. At least one other study suggests that what the interviewer is wearing can also impact how respondents behave (Benstead 2014). In this article, we explore the notion of face coverings worn by enumerators and whether this may influence a household’s decision to participate in the 2020 U.S. Census nonresponse follow-up operation.
Given the COVID-19 pandemic and adjusted timelines, in July 2020, the 2020 U.S. Census was winding down the self-response phase and preparing for personal-visit nonresponse follow-up, slated to begin in August. During this phase, census enumerators go door-to-door to interview households that have not self-responded. Census management sought information on public attitudes regarding census enumerators’ use of face coverings when contacting households. As the pandemic continued, the use of face coverings in public became a polarizing topic, with mandates to do so that varied from state to state and place to place (Flood 2020; Kim, Andrew, and Froio 2020; Stanley-Becker 2020; Wilson 2020).
2. Methodology
To understand the ramifications of enumerators wearing masks, a question on the topic was added to the 2020 Census Attitudes Daily Tracking Survey. This survey collected data on awareness, attitudes, and intent to participate in the U.S. 2020 Census and consisted of a probability random-digit-dial phone survey and a nonprobability online survey. The analysis in this article draws from the nonprobability, online survey responses collected over four weeks (June 12 – July 8). Studies suggest that whether a question is posed positively (e.g., 'Gay marriage should be allowed) or negatively ('Gay marriage should be banned) can systematically affect answers (Bregje et al. 2016; Hippler and Schwarz 1986). Consequently, we included a questionnaire design experiment.
The online survey leveraged respondents from two independent web panels, Dynata and ThinkNow, a Hispanic-focused panel. Dynata recruited respondents via direct email in partnership with globally recognized brands, while ThinkNow recruited panel participants in Spanish from targeted Google Ad campaigns and other Spanish-language websites. Quotas were set for the number of respondents originating from each panel. Hispanic English and Spanish speaking respondents, which make up 15% of the quota, originated from ThinkNow and all other respondents originated from the Dynata panel. Responses from both panels were combined, processed, and weighted together. Sample quotas ensured a balanced sample across the following key demographic characteristics: Age × Gender, Region, Race and Hispanic Origin, Age × Non-Hispanic White/Not Non-Hispanic White, and Education. Since the online survey is a nonprobability opt-in survey, a response rate cannot be calculated. However, the American Association for Public Opinion Research (AAPOR) participation rate was 34.4% (American Association for Public Opinion Research 2016).
The results discussed in this article are limited to findings from univariate and bivariate estimates. Bivariate tabulations compared each of the experimental questions across key independent variables of interest, including: age,[1] Race and Hispanic origin,[2] and region.[3] The tables also included the results of statistical hypothesis tests, which were performed pairwise on all combinations of pairs within a given independent variable. Hypothesis tests show differences using a two-proportion t-test at an α=0.05 significance level, adjusting for multiple comparisons within a single independent variable using the Bonferroni method.[4] Variance estimations use the Taylor Linearization estimator.
Persons indicating they had not yet self-responded to the 2020 Census at the time of the tracking survey were asked the face covering questions (approximately 17–24% indicated they had not yet self-responded, depending upon date of interview). Experimental question wording was administered using a randomized split panel, whereby half of the sample got the “wearing a face mask” positive version and half got the “NOT wearing a face mask” negative wording (see below).
Version 1 - Positive: "If the census taker came to your door and was wearing a face mask, would you be more likely to respond, less likely to respond, or would it not make a difference?
Version 2 – Negative: “If the census taker came to your door and was NOT wearing a face mask, would you be more likely to respond, less likely to respond, or would it not make a difference?”
3.1 Results of the split panel
Table 1 presents results from the split panels that received either the positive “presence of a mask” or negative “absence of a mask” version. The percentage of respondents who reacted negatively to enumerators wearing masks (less likely to respond to Version 1 and more likely to respond to Version 2) was significantly different between the two versions. When presented with the scenario of enumerators wearing a face covering, approximately 22% indicated they were less likely to respond compared with just under 12% who indicated they were more likely to respond if the enumerator was not wearing a face covering. However, once combined, those who reacted favorably to face coverings or said it would not make a difference were large majorities, regardless of the version asked (around 77% for Version 1 and 88 % for Version 2).
3.2 Results by demographics and region
For simplicity, we present data from the “presence of a mask” version only when examining age, Race and Hispanic origin, and region. Likelihood to respond across these characteristics was similar for both question versions. For all age groups, the majority reacted favorably to face-coverings or said it would not make a difference (see Table 2). Compared to 45–64-year-olds, significantly more 18–34-year-olds said they were less likely to respond if a census taker was wearing a mask. For the “not wearing a mask” question, +65-year-olds were significantly more likely to say they would be less likely to respond if the enumerator was not wearing a mask compared to 35–44-year-olds (74% vs. 49%, data not shown).
Over one-third of each Race and Hispanic origin group reacted favorably to enumerators wearing masks (see Table 3). Non-Hispanic Black respondents were significantly less likely to say it does not make a difference (compared to Non-Hispanic Whites). When asked as “not wearing a mask,” significantly more Non-Hispanic Asians said they were less likely to respond if the enumerator was not wearing a mask (compared to Non-Hispanic Whites; 72 % vs. 52%, data not shown).
Finally, Table 4 indicated a large majority of respondents who were from all four regions of the country were more likely to respond or say it would not make a difference if the enumerator had a face covering. Depending upon the region, around 20–25% indicated they would be less likely to respond if the census enumerator wore a mask, with no significant differences between them.
4. Conclusions
Survey organizations that conduct personal-visit interviews are experiencing major disruptions to “business as usual” during the COVID-19 pandemic. This article reports findings from an online tracking survey to inform about public opinion regarding enumerators’ use of face coverings. We embedded a split-panel questionnaire wording experiment as a randomized controlled trial, but several limitations are noted. First, the online responses were generated from a nonprobability opt-in sample. As a result, while the web sample enacts quotas and weights to reflect the general population, unknown biases may undermine the true “representativeness” of the results. Second, although respondents were randomly assigned to one of the two questions, there were a few underlying differences in the sample composition for each question. For example, significantly more males aged 25–34 saw the positively worded question. Otherwise, there were no significant differences between the samples (for the demographics we examine here). Finally, the data were collected over a four-week period as mandates to wear masks were changing rapidly across the U.S. with some areas adding the requirement over the course of data collection.
Results of the split-panel questionnaire wording experiment suggest that likelihood to respond during 2020 Census nonresponse follow-up would be higher (or not influenced) if census enumerators make their door-to-door contacts wearing face coverings. This was true across age groups, Race and Hispanic origin, and region of the country. A few within-subgroup differences were found, for example, compared to Non-Hispanic Whites, Non-Hispanic Blacks expressed less ambivalence toward face coverings. We also found significantly fewer respondents reacted favorably to the absence of face coverings when the question was posed in the negative direction (enumerators not wearing masks). This is in line with negativity bias literature suggesting that respondents have more difficulty endorsing topics when presented in a negative direction. Regardless, a sizeable minority overall (around 20–25%) still indicated lower likelihood of cooperating with enumerators wearing face coverings. Survey practitioners should consider these findings as organizations look for ways to move forward with personal-visit interviewing in the COVID-19 world.
Contact information
nancy.a.bates@census.gov
301.763.5248
U.S. Census Bureau, Associate Directorate for Research and Methodology Room 5K140, Washington DC 20233
Based on their response to the age question, each respondent was assigned to one of the following four age range categories: 18–34, 35–44, 45–64, or 65+.
Based on their response to race and Hispanic origin questions, each respondent was assigned to one of five race and Hispanic origin categories: Hispanic of any race or non-race, Non-Hispanic (NH) White only, NH Black/African American only, NH Asian only, or NH Other (races such as NH Native Hawaiian and Pacific Islander, NH American Indian and Alaska Native only, other races, and multiple races). NH Other was excluded from this analysis due to small sample sizes.
Based on their response to the zip code question, each respondent was assigned to one of the following four geographic regions: Northeast, Midwest, South, or West.
Because multiple comparisons induce an increased likelihood of committing Type I errors, a Bonferroni correction was applied to each alpha value to ensure an accurate assessment of significance. To operationalize the Bonferroni adjustment, a p-value, for a given hypothesis test is adjusted by taking the product m where m is all combinations of pairs within a given independent variable for a given table. The null hypothesis is rejected using a Bonferroni adjustment if p-value ≤ α/m. It was operationalized by using: m*p-value ≤ 0.05.