Background
In the following section, we present the trends in mobile ownership, Internet access, device use, and Internet navigation among Hispanics. Throughout the paper, we use the term Hispanic and Latino interchangeably. This review should provide the background for the importance of studying Web survey design among Spanish-dominant Hispanics in the United States.
Mobile Ownership and Internet Access on Phones
Mobile phone usage and ownership is widespread in the Hispanic community in the United States. According to a report by the Pew Research Center, about 71 percent of Hispanic adults own a smartphone (Smith 2015). This percentage is higher than the 64 percent of the general population who report owning a smartphone. Despite most Hispanics being owners of mobile devices, they are still limited on their access to Internet on mobile devices. Brown, López, and Lopez (2016) defined Internet users as “someone who says they use the internet or email at least occasionally and/or says they access the internet on a cellphone, tablet or other mobile handheld device at least occasionally.” The Pew report finds that Spanish-dominant Hispanics are less likely to use Internet in their mobile devices (71 percent) compared to English dominant Hispanics (86 percent) (Brown, López, and Lopez 2016). Ideally, access to Internet should correlate with the opportunity to participate and complete Web surveys. However, there is a “participation paradox” that Couper, Antoun, and Mavletova (2017) identify, which means that an increase of access to mobile devices does not necessarily translate into Web survey completion in mobile devices or smartphones. This paradox is of special interest for the study of response rates among Spanish-dominant Hispanics. If Spanish-dominant Hispanics are less likely to use mobile Internet, then the participation paradox can magnify when conducting Web surveys among this population.
Use of Additional Devices for Accessing Internet
Couper, Antoun, and Mavletova (2017) explain that “[o]wning an Internet-enabled smartphone is necessary to reach a respondent but not sufficient.” This statement applies to Hispanics as well. Despite most Hispanics being owners of mobile devices, they are still limited on their use of Internet because many do not have access to other devices such as PCs or tablets or do not have Internet service at home (Brown, López, and Lopez 2016; Smith 2015). Hispanics face structural barriers when using their mobile devices to access Internet. Almost half of Latino mobile users (49 percent) indicate they usually reach the maximum amount of data included on their data plans which leads to higher suspension of their Internet service (Smith 2015). In summary, Hispanics have high percentages of mobile ownership and access the Internet mainly through their mobile devices.
Internet Use
While Hispanics are more likely to access the Internet in a mobile device, their Internet use is often limited to social media purposes (Brown, López, and Lopez 2016). For example, in 2015, the Pew report found that among Latino Internet users, Facebook is widely used (73 percent compared with 71 percent for all Internet users) followed by Instagram (34 percent vs. 26 percent among the general population of internet users) and Twitter (25 percent compared to 23 percent of the general population of internet users) (Krogstad 2015). Thus, the Web survey coverage of Hispanic mobile device owners may not be high because they primarily use the Internet for social media purposes. Even when a survey reaches a Spanish-dominant Hispanic who owns a smartphone, one still needs to examine whether this person feels comfortable responding to surveys at all regardless of mode.
In summary, Hispanics are likely to use Internet on their phones, in particular Spanish-dominant Latinos. Hispanics may not have other devices they can use to access the Internet and mainly use their phones for social media. Therefore, there is still an open question on whether they know or are familiar with how to navigate their phones when completing a survey within an Internet browser. Studying the behavior of Spanish-dominant Hispanic mobile Web users advances our understanding of the specific challenges to participating in Web surveys. We examine this issue by conducting cognitive interviews and usability tests among Spanish-dominant Hispanic participants. Through this research, we expect to provide insights into challenges faced by inexperienced web users who are completing a survey task on a mobile device. We acknowledge that it is not common that Web surveys specifically encourage responses via mobile devices, yet this study can contribute to the conversation of emerging research related to the use of multi-devices, mobile-only surveys, and the use of app-based surveys (e.g., Buskirk and Andrus 2012; Sugie 2016; Wells, Bailey, and Link 2013).
Methods
A purposive sample of 24 respondents participated in a mobile Web survey in July 2015. It was part of a pretest of an electronic census form during the research and testing phase planned to inform design decisions for the 2020 Census. According to Etikan, Musa, and Alkassim (2016), a purposive sample is a nonrandom technique in which a “researcher decides what needs to be known and sets out to find people who can and are willing to provide the information by virtue of knowledge or experience. It is typically used in qualitative research to identify and select the information-rich cases for the most proper utilization of available resources.” The interviews were conducted in Illinois and New Jersey with Hispanics who owned smartphones or tablets, reported accessing the Internet with those devices, and reported feeling somewhat or very comfortable using the Internet. Eligible participants also needed to be Spanish dominant also referred to as monolingual or participants with a limited English proficiency as measured in the American Community Survey. That is, when asked how well they spoke English, they reported speaking English not well or not at well. During the screening process, we used the American Community Survey’s English questions: “How well do you speak English?” The response options are very well, well, not well and not at all. Participants were selected to include people with a variety of demographic characteristics such as country of origin, age, gender, and household composition as shown on Table 1. They were observed and video recorded responding to a self-administered Web census form and probed about the experience as well as about their interpretation of certain terms in the questions.
Findings and Lessons Learned
Table 2 shows a summary of the lessons we learned when conducting cognitive interviews among Spanish-dominant Latinos who reported feeling comfortable using Internet and owned a mobile device. The findings focus on the use of Internet applications, use of Internet and browsing, and familiarity with mobile Web surveys. In Table 2, we describe each of these findings.
Entering URLs
During the cognitive interview, study participants were asked to enter a URL to access the census form on their mobile devices. When participants tried to input the URL they were provided, they encountered challenges due to lack of familiarity with typing website addresses on their mobile phones, the length of the URL, and the difficulty of typing the URL that included several periods and inverse slashes (https://mobile.respond.qa.census.gov/censustest). For example, on one occasion, a participant simply looked at the URL in the materials with suspicion as if finding it hard to believe that she needed to enter the full long URL. She thought the URL was too long, tried to type it, but mistyped it (no “s” on “https” at the first attempt, and then no slashes the second time), leading her to shorten it and type www.census.gov. It is also important to mention that the URL included English only terms, which could be a challenge for non-English speakers to type in the screen of their mobile devices.
Another participant opened a browser but did not know where to enter the URL or how to use the search or address bar. This was a typical action since most of the participants tended to use a limited number of Internet applications (often limited to social media applications such as Facebook). This high focus only on social media may have influenced participant’s behavior when given a URL to access the census form. For example, we observed people entering the assigned URL on the Google search engine application but not knowing what page to select after that. While participants for this study were familiar with social media applications, their familiarity with the Internet decreased once, they were asked to input a Web address or to navigate the Internet to look for a specific website. There is a combination of lack of familiarity but also a usability challenge because the design of the test survey used a very long and nonintuitive URL to access the census form. In addition to this, the fact that the URL was not clickable prevented people to access to the survey. This happened because it is not common to have people typing long URLs in their smartphones.
Logging Off from a Website
In case participants might leave the application before completing the census form, they were asked to write down a personal identification number (PIN) that would allow them to sign in again. The same screen asked the participant to select one of four security questions and enter an answer for the selected question. The four questions were as follows and available in Spanish: (1) What color was your first car?, (2) What is the name of your first pet?, (3) In what city were you born?, and (4) and What is your paternal grandfather’s first name? Some participants keyed the PIN in the field for the answer to the security question. Others, who understood they were supposed to select a security question and enter their answer, nonetheless took some time to review their options and select one. A few of the questions that are standard online were actually culturally problematic for participants. For example, one participant was concerned that the questions were too easy and someone else would know her answers. Another participant thought the four possible security questions were actually part of the census form. Knowing how to log off from a survey is important because there are applications that time out and respondents should be able to return to the survey and complete it.
Questionnaire Design and Visual Cue Conventions
In the usability interviews, we also learned about the participants’ low level of familiarity with common conventions used on the Web, including not realizing that underlined text in blue can be a hyperlink. Several participants were also unfamiliar with scrolling down to see the entire contents of a survey question screen. This was problematic because they could not find the “next” button, which was located further down in the screen. Respondents might not see all response options and might choose a response that did not correspond to what they were looking for. Another usability finding is about the interpretation of check marks. Check marks were used in the design of the survey as a visual cue to indicate that all questions had been completed for a household. These visual cues went unnoticed, and the meaning of the check marks were not interpreted as the survey designers intended. Several participants did not realize they had finished answering for the person with a check mark and did not know how to interpret the visual cue.
Discussion and Conclusions
In general, cognitive interviews and usability testing offers two lessons on how to improve the administration of mobile Web surveys for Spanish-dominant Hispanics. First, our findings suggested that owning a device, such as a smartphone, does not correlate with familiarity with surveys among Spanish dominant Hispanics. Instead of interpreting this as bad news for mobile Web surveys, we argue that high rates of mobile device ownership present an opportunity for expanding the use of mobile Web surveys among this population. We believe that by learning about the issues and challenges one can plan to minimize them. For example, our data shows that our participants had trouble entering URLs. Thus, future mobile Web surveys should consider alternative ways to access mobile surveys, for example by sending links to access the survey, or generating quick response (QR) codes with the link to the survey to minimize the need for people to type URLs. Other possible alternatives can include using links that do not require the use of http or https followed by colon and slashes at the beginning of the address and using shorter vanity URLs which are more common in social media applications. Mobile surveys should also minimize adding hyperlinks and complex unnecessary steps such as the use of a PIN.
A second lesson from the cognitive interviews conducted among Spanish-dominant Hispanics is related to the likelihood for nonresponse. How can one encourage a participant to complete a mobile Web survey that, for example, is going to require prior experience filling out forms using the mobile device, or downloading a free app? A comprehensive view that includes structural factors (e.g., limited data plans) as well as elements related to the usability of mobile Web surveys (e.g., questionnaire and visual cue conventions) will allow us to better understand how we can reach and encourage respondents who may experience a language barrier when responding to mobile Web surveys. Given that Spanish dominant Hispanics are likely to use social media on their mobile devices, it would be good to consider testing whether survey access and completion rates can be improved if the instrument is available, for example, through a social media post. Further research is necessary to understand how web surveys in general are reaching out to English-dominant and Spanish dominant Hispanics through social media applications.
Disclaimer
Any views expressed are those of the author(s) and not necessarily those of the U.S. Census Bureau.
Acknowledgement
The authors would like to thank colleagues and reviewers who provided their thoughtful feedback to this piece. We also acknowledge Research Support Services and RTI International for conducting the cognitive interviews and usability testing for this study.