Loading [Contrib]/a11y/accessibility-menu.js
Skip to main content
Survey Practice
  • Menu
  • Articles
    • Articles
    • Editor Notes
    • In-Brief Notes
    • Interview the Expert
    • Recent Books, Papers, and Presentations
    • All
  • For Authors
  • Editorial Board
  • About
  • Issues
  • Blog
  • Subscribe
  • search

RSS Feed

Enter the URL below into your favorite RSS reader.

http://localhost:5959/feed
Articles
Vol. 5, Issue 1, 2012January 31, 2012 EDT

Smart Surveys for Smart Phones: Exploring Various Approaches for Conducting Online Mobile Surveys via Smartphones

Trent D. Buskirk, Charles Andres,
survey practice
https://doi.org/10.29115/SP-2012-0001
Survey Practice
Buskirk, Trent D., and Charles Andres. 2012. “Smart Surveys for Smart Phones: Exploring Various Approaches for Conducting Online Mobile Surveys via Smartphones.” Survey Practice 5 (1). https:/​/​doi.org/​10.29115/​SP-2012-0001.
Save article as...▾
Download all (4)
  • Figure 1  National Survey of Student Engagement as seen from an iPhone (www.nssesurvey.org).
    Download
  • Figure 2  The A-MBS approach illustrated below uses a Demonstration Survey built via Qualtrics-2011 and published as an (a) iPhone Mobile Version ( https://mobilehealth.slu.edu/ex1) and (b) Android Mobile Version.
    Download
  • Figure 3  The Demonstration Survey deployed using the A-LMBS approach ( https://mobilehealth.slu.edu/ex2) and viewed via an (a) iPhone and (b) Android Phone.
    Download
  • Figure 4  Background processes used to deploy smartphone surveys using the A-LMBS approach.
    Download

Sorry, something went wrong. Please try again.

If this problem reoccurs, please contact Scholastica Support

Error message:

undefined

View more stats

Abstract

Smart Surveys for Smart Phones: Exploring Various Approaches for Conducting Online Mobile Surveys via Smartphones*

In a recent study, Nielsen (2011) reported that 38% of mobile phone users in the U.S. owned a smartphone and that nearly 85 million people in the U.S. 13 years and older own such a device. Morgan Stanley (2010) recently projected that U.S. internet traffic among mobile devices (including netbooks, tablets and smartphones) will exceed that of desktop devices by 2014. The continued rise in smartphone adoption along with the expected upward trends in internet usage on such devices creates a viable mode for survey data collection that now needs to be formally investigated. The smartphone offers a multimode device accessible via voice, text or internet and can make use of synchronous multimedia messaging.

To date relatively few research studies pertaining to smartphone surveys and their implementation have appeared in the literature. Buskirk et al. (2011) reported the preliminary results of a mode effect experiment comparing completion times, open ended response length and web address input errors for an online survey completed on mobile and desktop browsers. Other research exploring various design, layout and question/response types are also slowly emerging in the survey research literature. For example, Peytchev and Hill (2010) reported the results for a series of experiments comparing various aspects of questionnaire design and layout, including horizontal scrolling, number of questions per screen, direction of response options, impact of embedded images and the use of open-ended options among others using a Samsung Blackjack Smartphone.

Couper (2010) presented experiments for online survey research applied to mobile browsers regarding types and sizes of data entry fields, monitor/web browser size and spacing and alignment of response options, among others. Callegaro (2010) provided a more technical and detailed account of browser sizes and resolution for mobile devices along with suggestions for using paradata to track the type of device associated with online survey completion. Callegaro (2010) also provided four specific ways for survey researchers to deal with the ever increasing reality that online surveys are being completed using mobile devices including: (a) making no changes to the online survey (i.e., requiring Smartphone users to adapt to regular online surveys); (b) blocking mobile phone respondents (e.g., issuing splash pages that redirect mobile users to complete the survey via desk/laptop computers); (c) developing a mobile survey version optimized to a specific type of mobile device(s) (e.g., iPhone and Android); (d) creating an online version of the survey accessible by any device. The last two recommendations coincide with Couper (2010), who suggested developing different versions of online surveys for mobile devices rather than having online mobile users adapt to current online standards.

Clearly, best practices and current research related to online survey development can serve as a basis for such design and implementation. However, differences in processing and form factors along with variability among smartphone platforms (Dawson 2010; Koch 2010) suggest regular online surveys taken via mobile browsers may not be optimal (Callegaro 2010) and that current best practices must be adapted, expanded and modified for surveys on mobile devices.

In this paper we provide a detailed overview of three specific approaches for implementing online surveys to be completed via mobile devices. The advantages and disadvantages of each approach are discussed within the framework of online surveys augmented with the most recent recommendations for online mobile survey layout and questionnaire design. In the next section we discuss the current smartphone landscape in order to provide context for the mobile survey implementation approaches discussed in section three. We close the paper with recommendations for survey practitioners endeavoring to expand their online surveys into the mobile landscape.

Overview of Current Smartphone Landscape

Currently in the U.S. smartphones utilize one of at least 9 different operating systems. The top three (i.e., iPhone, Android and RIM Blackberry) can be found in over 85% of active smartphones. Using GSM Arena (www.gsmarena.com), which offers a comprehensive review of current smartphones by brand, we queried size, resolution and input methods for currently available iPhone, Android or Blackberry devices in the U.S. Table 1 provides summary measures of screen size and resolution for these devices. Clearly, the results indicate considerable variability in screen size and input method across current devices. A resolution of 480 pixels wide by 800 pixels high was the most prevalent for the smartphones found in our query (38 of the 139 (27%)). Callegaro (2010) gives more specific details on screen resolution and provides implications for questionnaire layout for mobile surveys.

Table 1  Summary results for form factor variables of a sample of 139 smartphones currently available in the U.S. (running one of the top three operating systems).
Variable Minimum Maximum Mean Median Standard Deviation
Screen Size (Inches) 2.25 5.00 3.35 3.20 0.58
Display Width (Pixels) 240.00 640.00 414.39 480.00 108.99
Display Height (Pixels) 240.00 960.00 610.43 640.00 220.89
Pixels Per Inch (PPI) 114.29 329.65 219.81 227.55 52.02

In addition to form factor variability, data input methods also vary for current smartphones. Nearly 65% of the 139 smartphones in our query had capacitive touch screens (i.e., finger touch capable, like the iPhone), 62% had only virtual keyboards for data entry and nearly 75% had an accelerometer (i.e., the internal component that assists in changing screen layouts from “portrait” to “landscape”). We note that while these form factors will certainly change in the future, it is vital to understand the least common denominators of these factors at the survey design stage. Understanding differences in operating systems and the types of scripting allowed (e.g., adobe flash, java scripting, etc.) is also important for determining whether certain question types can be displayed or implemented in the mobile format.

Specific Approaches for Conducting Online Surveys via Smartphones

Current advances in smartphones provide researchers multiple options for implementing and conducting surveys via these mobile devices. In this section we discuss three main approaches including: online surveys taken via the mobile browser, app-administered surveys, and a hybrid approach combining the online mobile browser and app-based approaches. We provide advantages and disadvantages for each of these approaches and illustrate both the online mobile browser approach and the hybrid approach. Throughout this section online surveys will refer to those originally developed and deployed using desktops/laptops. Mobile Browser Surveys (MBS) will refer to those surveys developed and intended for smartphones.

Approach 1: Online Surveys Taken via Mobile Browsers

Passive Version

Smartphone users can browse the internet via native mobile browsers that are running on the operating system of the device. If a mobile device accesses a website without a mobile version, the user will still be able to navigate the page. However, differences in screen size will usually require the user to perform gestures or scrolling in order to browse the content in its entirety. Therefore, if a smartphone accesses a non-mobile version of an online survey, then it is likely that the respondent will see only a portion of the content/question. The smartphone user would likely have to scroll horizontally and/or vertically as well as to zoom in to select a response or press the next button. Nevertheless, allowing smartphone users to access such online surveys requires neither content modification nor development of a mobile version of the survey. We call such online surveys completed via smartphone Passive-Mobile Browser Surveys (P-MBS). An example of a P-MBS survey accessed via an iPhone is displayed in Figure 1. Table 2 provides detailed advantages and disadvantages of the P-MBS approach for conducting smartphone surveys.

Figure 1  National Survey of Student Engagement as seen from an iPhone (www.nssesurvey.org).
Table 2  The advantages and disadvantages of using the P-MBS approach for conducting smartphone surveys.
Advantages:
  • Normal deployment of online [desktop/laptop] surveys – no additional formatting required
  • Programming costs are comparable to those of online surveys
  • Possible expansion of intended audience to include mobile browser users
Disadvantages:
  • Surveys may not be viewable without additional effort of pinching, scrolling or zooming
  • Question completion may require zooming first in order to select desired response
  • Various question types (e.g. multiple choice, lists) will not generally take advantage of the smartphone’s native features (i.e. spin wheels)
  • Higher survey drop-offs/longer completion times may result due to increased navigation burden required to answer questions
  • Persistent internet connection needed to complete survey
  • Loading times may be increased for pages with images/graphics or large grids
  • Volume of data transferred is likely to be larger than necessary (potential threat to smartphone data-plan quotas)
  • A server must generate each subsequent survey page and data exchanges occur with each new page
  • Longer surveys may be difficult to process due to the form factor and the internet lag

Active Version

Just as normal online websites can have mobile versions modified and adapted to account for differences in browser size, so it is with online surveys. Both Peytchev and Hill (2010) and Dawson (2010) note that unformatted images included on a mobile webpage will account for a disproportionate amount of the overall display. Resolution factors and image display sizes become crucial elements to consider within the mobile framework. Other issues relevant to mobile survey development include selecting question types that minimize or eliminate the need for horizontal scrolling (Callegaro 2010; Peytchev and Hill 2010), minimizing the use of flash content (Dawson 2010) and formatting response options to be read and selected without zooming. Many online survey packages such as Qualtrics and Snap have options for publishing a mobile (e.g. iPhone, Blackberry) version. This type of survey is analogous to a website’s mobile version. When accessed via smartphone, horizontal navigation is often not required (or at least minimized), fonts and images appear proportional and the need for zooming or other similar gestures reduces. We call online surveys reformatted and deployed as mobile versions Active-Mobile Browser Surveys (A-MBS). Figure 2 displays an example of an A-MBS survey accessed via an iPhone (a) and Android Phone (b). Table 3 lists advantages and disadvantages of the A-MBS approach.

Figure 2  The A-MBS approach illustrated below uses a Demonstration Survey built via Qualtrics-2011 and published as an (a) iPhone Mobile Version ( https://mobilehealth.slu.edu/ex1) and (b) Android Mobile Version.
Table 3  The advantages and disadvantages of using the A-MBS approach for conducting smartphone surveys.
Advantages:
  • Graphic images can be sized and displayed in proper proportion to mobile web browser size resulting in a reduction in the total amount of data exchanged and faster page loads
  • Next/Back buttons and other navigation tools can be explicitly visible on every page
  • Horizontal scrolling can be more easily limited
  • Number of questions appearing on a page can be explicitly controlled
  • Question layout is optimized (to the extent possible) for particular mobile browsers and can take advantage of native features such as scroll wheels
  • Streamlined survey completion via mobile browser
Disadvantages:
  • Persistent internet connection needed to complete survey
  • A server must generate each subsequent survey page (in its entirety) and data exchange occurs with each new page
  • Longer surveys may be difficult to process due to the form factor and the internet lag
  • Some data capture regarding GPS and other features may require additional permissions set by the user
  • Cost of development may be slightly higher in order to develop and deploy a mobile browser specific version of the survey
  • Can’t control the orientation of the screen used to complete the questionnaire
  • Scrolling may be unavoidable if images or grids are used
  • Requires beta testing on various platforms/mobile browsers
  • Different smartphone operating systems may interpret the mark-up languages and survey formatting slightly differently than intended and these variations are often beyond the control of the survey designer. For example, Figure 2(b) illustrates the need to scroll horizontally for answer input using the Android Browser while Figure 2(a) shows no such scrolling required on an iPhone.

Approach 2: Surveys Administered via Smartphone Apps

One of the defining features of smartphones is the availability of executable applications (or APPS) stored on the devices. There are hundreds of thousands of apps available in total across the various smartphone operating systems. If you want to check the weather on your smartphone, “there’s an app for that.” If you want to conduct a survey, there is an app for that, too. Surveys administered via such smartphone apps use what we call the App Based smartphone Survey approach (ABSS). This approach relies on a smartphone operating system specific app installed on the respondent’s smartphone that pushes survey content to the end user and uploads collected data whenever an internet connection is available. Techneos’ Survey On-Demand Application (SODA) (http://www.techneos.com/resources/product_specs) provides one example of an app-based survey tool used to develop ABSS’s across major smartphone operating systems. While apps developed for survey conduct seem the natural choice for mobile surveys, development costs and that respondents have to download, install and use the app for survey data collection are among some aspects that may limit its ubiquity. Table 4 highlights advantages and disadvantages of the ABSS approach.

Table 4  The advantages and disadvantages of using the ABSS approach for conducting smartphone surveys.
Advantages:
  • Question layout is self contained in the app without the need for scrolling (horizontally) or zooming
  • Can capture various types of data including GPS, Pictures and Video from within the App itself
  • Orientation of the survey questions (presentation and completion) can be controlled
  • Data collection can occur without the need for a persistent internet connection, which may reduce break-off related item nonresponse
  • Page refresh rates/overall survey load times are not dependent upon the strength of the internet connection
  • Specific survey content can be optimized to known operating system to reduce overall amount of data exchanged
  • Can reliably deploy video/flash/image/audio content
  • Automates data submission and can provide survey alerts/invitations
  • Content pacing does not rely on strength of internet connection
Disadvantages:
  • Requires development/deployment of potentially multiple app versions (i.e. one for each smartphone operating system)
  • Requires the respondent to download/install an app on their smartphone
  • Some data capture regarding GPS and other features may require additional permissions to be set by the user
  • Cost of development/data capture/storage may be high for single use surveys
  • Newer approach to data collection- users may have to receive some guidance on how the app works
  • Number of available APPS on all marketplaces continues to rise- may have harder time with salience or may have to compete with “app clutter”
  • Some question types may not be readily available in App toolkits- may require additional programming to implement non-standard question types

Approach 3: Surveys Administered via Mobile Browsers that Appear App-Like

While the ABSS approach is promising it is unlikely to be viable for single use surveys. Moreover, the A-MBS approach may not provide as much customization as a survey researcher may desire. The final approach we discuss here represents a hybrid of the A-MBS approach and the ABSS approach. Specifically, surveys deployed using the app-like mobile browser survey approach (A-LMBS) are completed using the mobile web, but they rely on active browser refreshment using a combination of server side scripting and JavaScript to create “Active” Mobile web pages. The web pages appear app-like with no web address bar present after the survey commences and only new portions of the screen must be loaded on each subsequent page. For example, if the survey had a series of four multiple-choice questions with the same drop down answer choices, then the first question and the answer choices would appear on the first screen in the series. On subsequent screens, only the new question wording would have to be loaded onto the page – the multiple choice options would persist on the page. This feature of the A-LMBS approach allows the resulting survey to process more quickly beyond the initial page and gives the user an app-like experience rather than a mobile browser experience. Please refer to the Appendix for additional technical information about developing surveys using the A-LMBS approach. Figure 3 provides smartphone images for a demonstration survey deployed using the A-LMBS followed by a discussion of the advantages and disadvantages in Table 5.

Figure 3  The Demonstration Survey deployed using the A-LMBS approach ( https://mobilehealth.slu.edu/ex2) and viewed via an (a) iPhone and (b) Android Phone.
Table 5  The advantages and disadvantages of using the A-LMBS approach for conducting smartphone surveys.
Advantages:
  • Can capture various types of data including GPS, Pictures and Video using JavaScript to activate these smartphone features
  • Portions of the online layout can be “preloaded” and do not need to be reloaded with each new screen (e.g. back/next buttons)
  • Placement of navigation bar can be controlled and can be made “persistent” (i.e. will not require reloading the entire ribbon on each page)
  • Data collection can occur without the need for a persistent internet connection, but will require additional JavaScript programming
  • Page refresh rates/overall survey load times will generally be shorter compared to Active Web Browser approach
  • No internet web address bar is present after the survey begins- the survey is “encased” in what appears to be an App potentially increasing available screen size
  • Can take advantage of the smartphone’s native features including loading wheel, spinners and slider bars
  • Survey presentation is more uniform across smartphone operating systems
  • Graphic images can be optimized and scaled for mobile browsers
  • Can control questionnaire layout and suggested page orientation
Disadvantages:
  • Requires additional JavaScript programming to make survey web pages “active” rather than dynamic
  • Limited number of pre-programmed survey question/response formats available
  • Approach will not be optimal for non-touchscreen smartphones.
  • Initial (i.e. first page) amount of data exchanged on the introduction/welcome page will generally be higher compared to the A-MBS approach.
  • Cannot explicitly control the orientation of the survey (i.e. landscape or portrait)
  • If JavaScript is disabled on a respondent’s smartphone, then some functionality will not work (e.g. older Blackberry devices)
  • Some data capture regarding GPS and other features may require additional permissions to be set by the user
  • Approach is not currently prepackaged or part of survey software.

Discussion

Just as there are many types of smartphones currently available in the U.S., there are various choices for how one might deploy a survey on these mobile devices. While a one-size-fits all solution for conducting such surveys on smartphones would be optimal, current heterogeneity in operating systems, form factors and applications across smartphones suggest this goal is not on the immediate horizon. For current smartphone survey deployment, we suggest researchers select an implementation approach similar to one of the three we have presented here. For some surveys, there may be no obvious optimal approach, but one may clearly be inappropriate. For example, it is unlikely that a short one-time survey would be optimally implemented using the ABSS approach. Similarly, surveys that contain a fair number of graphic images would not be best suited for the P-MBS approach. In any case, survey researchers should make every effort to use an implementation method that is at least as smart as the smartphone!

Acknowledgments

Special thanks to Shavani Singh for her assistance with data collection for the GSM Arena query..

Appendix: Technical Details regarding the Development and Deployment of the A-LMBS Approach:

The AL-MBS approach generally utilizes three technologies: a web server (Apache), a Python template engine (Django), and a JavaScript (JS) framework (Sencha Touch). The Apache web server handles the request for a webpage when the participant enters the assigned survey URL. When Apache receives such a request it uses the Web Server Gateway Interface (WSGI) to communicate with Python Django. Python determines appropriate survey content (e.g., welcome page, question 1, etc) and generates the actual HTML, CSS and JavaScript code, which is then passed to the JavaScript framework (Sencha Touch) via the Apache server. Sencha Touch then renders app-like survey content onto the respondent’s browser. Survey responses are submitted and subsequent page content is rendered within this context via Asynchronous JavaScript and XML (AJAX). This process is illustrated in Figure 4.

Figure 4  Background processes used to deploy smartphone surveys using the A-LMBS approach.

References

Buskirk, T.D., M. Gaynor, C. Andrus, and C. Gorrell. 2011. An App a Day Could Keep the Doctor Away: Comparing Mode Effects for a iPhone Survey Related to Health App Use. Phoenix, AZ: American Association of Public Opinion Research.
Google Scholar
Callegaro, M. 2010. “Do You Know Which Device Your Respondent Has Used to Take Your Online Survey?” Survey Practice. http:/​/​www.surveypractice.org/​.
Google Scholar
Couper, M.P. 2010. “Visual Design in Online Surveys: Learnings for the Mobile World.” In The Mobile Research Conference 2010. London. http:/​/​www.mobileresearchconference.com/​uploads/​files/​MRC2010_Couper_Keynote.pdf.
Google Scholar
Koch, P.-P. 2010. “Smartphone Browser Landscape, a List Apart Article” no. 320. http:/​/​www.alistapart.com/​articles/​smartphone-browser-landscape/​.
Google Scholar
Morgan Stanley. 2010. “Internet Trends.” http:/​/​www.morganstanley.com/​institutional/​techresearch/​pdfs/​Internet_Trends_041210.pdf.
Nielsen, W. 2011. “In US, Smartphones Now Majority of New Cellphone Purchases.” June 11, 2011. http:/​/​blog.nielsen.com/​nielsenwire/​online\_mobile/​in-us-smartphones-now-majority-of-new-cellphone-purchases/​].
Peytchev, A., and C.A. Hill. 2010. “Experiments in Mobile Web Survey Design: Similarities to Other Modes and Unique Considerations.” Social Science Computer Review 28:319–35.
Google Scholar

This website uses cookies

We use cookies to enhance your experience and support COUNTER Metrics for transparent reporting of readership statistics. Cookie data is not sold to third parties or used for marketing purposes.

Powered by Scholastica, the modern academic journal management system