In a recent study, Nielsen (2011) reported that 38% of mobile phone users in the U.S. owned a smartphone and that nearly 85 million people in the U.S. 13 years and older own such a device. Morgan Stanley (2010) recently projected that U.S. internet traffic among mobile devices (including netbooks, tablets and smartphones) will exceed that of desktop devices by 2014. The continued rise in smartphone adoption along with the expected upward trends in internet usage on such devices creates a viable mode for survey data collection that now needs to be formally investigated. The smartphone offers a multimode device accessible via voice, text or internet and can make use of synchronous multimedia messaging.
To date relatively few research studies pertaining to smartphone surveys and their implementation have appeared in the literature. Buskirk et al. (2011) reported the preliminary results of a mode effect experiment comparing completion times, open ended response length and web address input errors for an online survey completed on mobile and desktop browsers. Other research exploring various design, layout and question/response types are also slowly emerging in the survey research literature. For example, Peytchev and Hill (2010) reported the results for a series of experiments comparing various aspects of questionnaire design and layout, including horizontal scrolling, number of questions per screen, direction of response options, impact of embedded images and the use of open-ended options among others using a Samsung Blackjack Smartphone.
Couper (2010) presented experiments for online survey research applied to mobile browsers regarding types and sizes of data entry fields, monitor/web browser size and spacing and alignment of response options, among others. Callegaro (2010) provided a more technical and detailed account of browser sizes and resolution for mobile devices along with suggestions for using paradata to track the type of device associated with online survey completion. Callegaro (2010) also provided four specific ways for survey researchers to deal with the ever increasing reality that online surveys are being completed using mobile devices including: (a) making no changes to the online survey (i.e., requiring Smartphone users to adapt to regular online surveys); (b) blocking mobile phone respondents (e.g., issuing splash pages that redirect mobile users to complete the survey via desk/laptop computers); (c) developing a mobile survey version optimized to a specific type of mobile device(s) (e.g., iPhone and Android); (d) creating an online version of the survey accessible by any device. The last two recommendations coincide with Couper (2010), who suggested developing different versions of online surveys for mobile devices rather than having online mobile users adapt to current online standards.
Clearly, best practices and current research related to online survey development can serve as a basis for such design and implementation. However, differences in processing and form factors along with variability among smartphone platforms (Dawson 2010; Koch 2010) suggest regular online surveys taken via mobile browsers may not be optimal (Callegaro 2010) and that current best practices must be adapted, expanded and modified for surveys on mobile devices.
In this paper we provide a detailed overview of three specific approaches for implementing online surveys to be completed via mobile devices. The advantages and disadvantages of each approach are discussed within the framework of online surveys augmented with the most recent recommendations for online mobile survey layout and questionnaire design. In the next section we discuss the current smartphone landscape in order to provide context for the mobile survey implementation approaches discussed in section three. We close the paper with recommendations for survey practitioners endeavoring to expand their online surveys into the mobile landscape.
Overview of Current Smartphone Landscape
Currently in the U.S. smartphones utilize one of at least 9 different operating systems. The top three (i.e., iPhone, Android and RIM Blackberry) can be found in over 85% of active smartphones. Using GSM Arena (www.gsmarena.com), which offers a comprehensive review of current smartphones by brand, we queried size, resolution and input methods for currently available iPhone, Android or Blackberry devices in the U.S. Table 1 provides summary measures of screen size and resolution for these devices. Clearly, the results indicate considerable variability in screen size and input method across current devices. A resolution of 480 pixels wide by 800 pixels high was the most prevalent for the smartphones found in our query (38 of the 139 (27%)). Callegaro (2010) gives more specific details on screen resolution and provides implications for questionnaire layout for mobile surveys.
In addition to form factor variability, data input methods also vary for current smartphones. Nearly 65% of the 139 smartphones in our query had capacitive touch screens (i.e., finger touch capable, like the iPhone), 62% had only virtual keyboards for data entry and nearly 75% had an accelerometer (i.e., the internal component that assists in changing screen layouts from “portrait” to “landscape”). We note that while these form factors will certainly change in the future, it is vital to understand the least common denominators of these factors at the survey design stage. Understanding differences in operating systems and the types of scripting allowed (e.g., adobe flash, java scripting, etc.) is also important for determining whether certain question types can be displayed or implemented in the mobile format.
Specific Approaches for Conducting Online Surveys via Smartphones
Current advances in smartphones provide researchers multiple options for implementing and conducting surveys via these mobile devices. In this section we discuss three main approaches including: online surveys taken via the mobile browser, app-administered surveys, and a hybrid approach combining the online mobile browser and app-based approaches. We provide advantages and disadvantages for each of these approaches and illustrate both the online mobile browser approach and the hybrid approach. Throughout this section online surveys will refer to those originally developed and deployed using desktops/laptops. Mobile Browser Surveys (MBS) will refer to those surveys developed and intended for smartphones.
Approach 1: Online Surveys Taken via Mobile Browsers
Passive Version
Smartphone users can browse the internet via native mobile browsers that are running on the operating system of the device. If a mobile device accesses a website without a mobile version, the user will still be able to navigate the page. However, differences in screen size will usually require the user to perform gestures or scrolling in order to browse the content in its entirety. Therefore, if a smartphone accesses a non-mobile version of an online survey, then it is likely that the respondent will see only a portion of the content/question. The smartphone user would likely have to scroll horizontally and/or vertically as well as to zoom in to select a response or press the next button. Nevertheless, allowing smartphone users to access such online surveys requires neither content modification nor development of a mobile version of the survey. We call such online surveys completed via smartphone Passive-Mobile Browser Surveys (P-MBS). An example of a P-MBS survey accessed via an iPhone is displayed in Figure 1. Table 2 provides detailed advantages and disadvantages of the P-MBS approach for conducting smartphone surveys.
Active Version
Just as normal online websites can have mobile versions modified and adapted to account for differences in browser size, so it is with online surveys. Both Peytchev and Hill (2010) and Dawson (2010) note that unformatted images included on a mobile webpage will account for a disproportionate amount of the overall display. Resolution factors and image display sizes become crucial elements to consider within the mobile framework. Other issues relevant to mobile survey development include selecting question types that minimize or eliminate the need for horizontal scrolling (Callegaro 2010; Peytchev and Hill 2010), minimizing the use of flash content (Dawson 2010) and formatting response options to be read and selected without zooming. Many online survey packages such as Qualtrics and Snap have options for publishing a mobile (e.g. iPhone, Blackberry) version. This type of survey is analogous to a website’s mobile version. When accessed via smartphone, horizontal navigation is often not required (or at least minimized), fonts and images appear proportional and the need for zooming or other similar gestures reduces. We call online surveys reformatted and deployed as mobile versions Active-Mobile Browser Surveys (A-MBS). Figure 2 displays an example of an A-MBS survey accessed via an iPhone (a) and Android Phone (b). Table 3 lists advantages and disadvantages of the A-MBS approach.
Approach 2: Surveys Administered via Smartphone Apps
One of the defining features of smartphones is the availability of executable applications (or APPS) stored on the devices. There are hundreds of thousands of apps available in total across the various smartphone operating systems. If you want to check the weather on your smartphone, “there’s an app for that.” If you want to conduct a survey, there is an app for that, too. Surveys administered via such smartphone apps use what we call the App Based smartphone Survey approach (ABSS). This approach relies on a smartphone operating system specific app installed on the respondent’s smartphone that pushes survey content to the end user and uploads collected data whenever an internet connection is available. Techneos’ Survey On-Demand Application (SODA) (http://www.techneos.com/resources/product_specs) provides one example of an app-based survey tool used to develop ABSS’s across major smartphone operating systems. While apps developed for survey conduct seem the natural choice for mobile surveys, development costs and that respondents have to download, install and use the app for survey data collection are among some aspects that may limit its ubiquity. Table 4 highlights advantages and disadvantages of the ABSS approach.
Approach 3: Surveys Administered via Mobile Browsers that Appear App-Like
While the ABSS approach is promising it is unlikely to be viable for single use surveys. Moreover, the A-MBS approach may not provide as much customization as a survey researcher may desire. The final approach we discuss here represents a hybrid of the A-MBS approach and the ABSS approach. Specifically, surveys deployed using the app-like mobile browser survey approach (A-LMBS) are completed using the mobile web, but they rely on active browser refreshment using a combination of server side scripting and JavaScript to create “Active” Mobile web pages. The web pages appear app-like with no web address bar present after the survey commences and only new portions of the screen must be loaded on each subsequent page. For example, if the survey had a series of four multiple-choice questions with the same drop down answer choices, then the first question and the answer choices would appear on the first screen in the series. On subsequent screens, only the new question wording would have to be loaded onto the page – the multiple choice options would persist on the page. This feature of the A-LMBS approach allows the resulting survey to process more quickly beyond the initial page and gives the user an app-like experience rather than a mobile browser experience. Please refer to the Appendix for additional technical information about developing surveys using the A-LMBS approach. Figure 3 provides smartphone images for a demonstration survey deployed using the A-LMBS followed by a discussion of the advantages and disadvantages in Table 5.
Discussion
Just as there are many types of smartphones currently available in the U.S., there are various choices for how one might deploy a survey on these mobile devices. While a one-size-fits all solution for conducting such surveys on smartphones would be optimal, current heterogeneity in operating systems, form factors and applications across smartphones suggest this goal is not on the immediate horizon. For current smartphone survey deployment, we suggest researchers select an implementation approach similar to one of the three we have presented here. For some surveys, there may be no obvious optimal approach, but one may clearly be inappropriate. For example, it is unlikely that a short one-time survey would be optimally implemented using the ABSS approach. Similarly, surveys that contain a fair number of graphic images would not be best suited for the P-MBS approach. In any case, survey researchers should make every effort to use an implementation method that is at least as smart as the smartphone!
Acknowledgments
Special thanks to Shavani Singh for her assistance with data collection for the GSM Arena query..
Appendix: Technical Details regarding the Development and Deployment of the A-LMBS Approach:
The AL-MBS approach generally utilizes three technologies: a web server (Apache), a Python template engine (Django), and a JavaScript (JS) framework (Sencha Touch). The Apache web server handles the request for a webpage when the participant enters the assigned survey URL. When Apache receives such a request it uses the Web Server Gateway Interface (WSGI) to communicate with Python Django. Python determines appropriate survey content (e.g., welcome page, question 1, etc) and generates the actual HTML, CSS and JavaScript code, which is then passed to the JavaScript framework (Sencha Touch) via the Apache server. Sencha Touch then renders app-like survey content onto the respondent’s browser. Survey responses are submitted and subsequent page content is rendered within this context via Asynchronous JavaScript and XML (AJAX). This process is illustrated in Figure 4.
