How to Do Mail Surveys in the Digital Age: A Practical Guide

Introduction

Finding information about what to expect in terms of costs, time commitments, and logistics can be a major challenge in planning a mail survey. While there is a substantial body of literature on survey design and the effects of choices on response rates (e.g., Edwards et al. 2002; Fanning 2005; Jans et al. 2015; Mitchell and Brown 1997; Roberts et al. 1993; Ziegenfuss et al. 2014), much less literature focuses on how mail surveys are implemented. This piece aims to help fill that gap by presenting information about logistics, time requirements, and costs associated with the post-design and pre-analysis phase of a mail survey, based on one self- and one professionally administered implementation of the same survey.

Despite the rise of Internet-based surveying, mail surveys remain valuable and offer some advantages not achievable by other methods. For example, populations of interest may be too small or too specific for web survey coverage (Qualtrics, personal communication, 2016). Estimated costs per response of between $20 and $40 are similar to those of computer-assisted telephone interviewing (Qualtrics, personal communication, 2016), but mailed surveys allow for the possibility of anonymity, are better suited to certain types of questions (like rankings or open-ended questions) (e.g., Reetoo et al. 2005), and allow respondents more flexibility given that they are self-administered. Thus, information about mail survey logistics can particularly benefit people who might be deciding between mail and other survey modes.

Methods

The mail survey described here was delivered as a twelve-page questionnaire booklet (5.5 by 8.5 inches), folded cover sheet (8.5 by 11 inches unfolded), and self-addressed stamped return envelope (8.9 by 5.8 inches) inside an outer envelope (9.5 by 6.5 inches). Manual addressing and stamping was not necessary for the outer envelope, as the return address, delivery address, and Every Door Direct Mail (EDDM) indicia (which substitutes for postage) were printed on the envelope directly. Return and delivery addresses were also directly printed on the smaller, enclosed return envelope, but a single first-class stamp was placed on each of these inner envelopes manually.

This descriptive paper presents costs for the envelopes, cover sheets, booklets, and stamps, all ordered and received in the United States in late 2015/early 2016. In addition, this paper describes the time required to fold cover sheets, apply stamps to return envelopes, stuff outer envelopes, and prepare and ship mailings to distribution post offices. Time was tracked for each preparation step using a stopwatch, with subintervals tracked for every 100 units completed for the purpose of measuring learning. Additional time spent on activities like filing paperwork for post office delivery, shipping surveys to distribution post offices, and purchasing supplies is approximate.

An equivalent hourly wage for the survey preparation tasks is derived by comparing the time spent preparing a U.S. survey to charges by an Australian survey company for preparation of a nearly identical mailing in Australia in October 2015. The self-administered survey described involved preparation and mailing of approximately 10,000 questionnaire mailpieces and 20,000 pre- and post-mailer postcards.

Results and Discussion

Weight-Based Logistics

The physicality of a mail survey is difficult to understate. Most notably, mail surveys are quite heavy: this project’s 10,000 mailings weighed approximately 900 pounds, using 70 lb paper (104 g/m²) for the cover letters and surveys and 24 lb paper for the envelopes (90.3 g/m²).^[1] This weight has three major logistical implications. First, shipping costs can be high, though attention to discount codes and opportunities to pick up rather than ship materials can reduce the burden. For example, this project used a discount code worth $440 to ship 20,000 envelopes for free. Printing the questionnaires and cover letters locally and picking them up directly from the print shop allowed for an additional shipping savings of $380. Notably, shipping the locally-printed questionnaires and cover letters about 10 miles would have cost almost as much as shipping the envelopes 3,000 miles, despite similar weights for the two orders. These distance-insensitive (and sometimes distance-independent) costs are due to the use of flat rate weight- or volume-based shipping (United States Postal Service 2016), which makes local pick-up compelling where possible.

The second implication of a mail survey’s weight is that moving and storing materials requires strength and space. This survey’s volume was approximately 40 cubic feet, packed into 69 individual boxes. Boxes needed to be moved frequently over the roughly two months during which mailpieces were prepared:

• for loading and unloading from delivery vehicles,
• for access to individual boxes,
• for temporary storage of empty boxes intended for reuse for shipping prepared mailpieces, and
• in response to alternate needs for the space where boxes were stored.

Through these moves, boxes need to be kept upright as much as possible to avoid bending contents packed with a particular orientation, and careful organization is required to ensure access to unprepared, mid-preparation, and completed mailpiece material.

Considering when, where, and how material will be received is worth some thought. This survey was prepared mostly in the region’s wet season, which caused some logistical issues associated with keeping paper goods dry. Transportation more generally required considerable coordination. For example, the full order of 10,000 questionnaires did not fit into a typical personal vehicle and was therefore partitioned into three separate printing runs. Envelopes arrived via United Parcel Service, which required coordination between the author and the delivery driver to unload 40 boxes (380 lbs) onto the shipping driveway, after which the boxes needed to be moved to a safe (and dry) storage location immediately. Since each of the 40 boxes had a tracking bar code that needed to be scanned, unloading took about a half hour even though the driver could hand boxes directly to the author to be placed onto the driveway without leaving the truck. This relatively long idling period could cause problems not only for the driver but for other traffic in the area where delivery is taking place, so planning for delivery is important.

The third implication of the mail survey’s weight is that postage requirements are a step function based on weight and can thus be optimized. This mail survey used unusually-sized envelopes to enclose returned questionnaires in part due to weight limits. Despite use of relatively lightweight paper for the questionnaire, returned pieces (12-page 8.5 by 5.5-inch survey booklets in a 5.75 by 8.875-inch envelope, plus a stamp and the weight of printed ink) weigh about 0.95 ounces, just below the transition from one to two-ounce pricing. A marginally heavier unit (for example, one using slightly larger envelopes) would require $0.71 in postage rather than $0.49 (as of early 2016), an overall project cost increase of $2,200. This postage savings justified the $310 (40 percent) higher cost of the nonstandard envelopes.

Material and Time Buffers

Mailers should plan for delays and accidents when constructing a timeline and making orders. For those with specific numerical requirements, ordering a small material overrun is advisable in case of misprints, damage, or sample needs. As received, about 0.5 percent of the materials described here were damaged in some way, and accidents mostly related to water damage of envelope adhesive destroyed about 20 additional units. In addition, those planning to drop ship mailpieces to post offices for local delivery should be aware that post offices require that a sample mailpiece be included for inspection (e.g., to verify weight and compliance with size standards) (United States Postal Service 2012). For this survey, sample mailpiece requirements created demand for another 30 units.

Planning for delays is also advised. For example, this project involved a large order of an unusually sized envelope. Order fulfilment forced restocking by the supplier, delaying receipt of the units by about ten days. Similarly, large orders of stamps, particularly specific designs, can require mail orders that take between five and seven mail days to arrive.

Calendar days, mail days, and business days are different, which might be a consideration when materials are ordered close to the mailing date and mailings are deliberately spaced (e.g., for a one-week follow up). Also, some variation in delivery time is likely. In the most extreme example experienced for this survey, four of eight boxes of mailers shipped simultaneously to the same post office on a Friday afternoon arrived by Saturday morning, while the rest did not arrive until Monday end-of-day, resulting in a three calendar day difference in mailer delivery in one community.

Costs

Materials for 10,000 surveys, including a pre-mailer postcard, mailpiece as described above, and post-mailer postcard, cost approximately $18,000, or $1.83 per address. This figure includes $260 in taxes, $2,300 in discounts, and the use of the United States Postal Service EDDM service for delivery of the 30,000 individual mailpieces, which reduces postage costs. Costs and discounts are detailed in Table 1.

Given a response rate of between 5 and 10 percent, depending on the region, these printing and materials costs imply a cost per response of $20 to $40. The effect of the use of EDDM rather than conventional first class mail on response rate will be presented in a future paper. (For consistency, Table 1 presents postage costs as though all mailpieces were delivered using EDDM postage. In reality, some were delivered using traditional first-class postage as an experimental condition.)

Time

Packing and preparing 10,000 surveys for mailing required approximately 240 hours of labor by people with no previous envelope stuffing experience. For all tasks, the first attempt was by far the slowest, but learning proceeded rapidly. Notably, the learning effect seems to be mainly due to development of better processes rather than increased skill, as faster packing was teachable. For this project, an assistant packer was able to pack nearly as quickly as the primary packer on the first attempt after being taught the mature process.

For cover letter folding, speed increased by 25 percent over the first thousand units, from 13 minutes to 10 minutes per hundred, where the speed stabilized. For applying stamps, both process and skill improvements were relevant: speed improved from 26 to 16 minutes per hundred over the first thousand units, and then further improved to about 11 minutes per hundred by the 3000th unit, where it stabilized. For envelope stuffing, average speed was relatively consistent, but the likelihood of anomalously fast intervals increased over time: that is, average stuffing time remained stable at 20 minutes per hundred units over the entire process, but unusual speeds as low as 14 minutes per hundred became possible starting around the 3000th unit.

For all tasks, context matters. Doing nothing but survey preparation or listening to upbeat music resulted in the fastest speeds. More engaging secondary tasks like listening to audiobooks or watching television reduced speed by about 25 percent relative to no stimulus or music-only stimulus but increased endurance (the number of units that could be comfortably completed in a single sitting) by about an order of magnitude. This result suggests that trading off speed for mental engagement is likely worthwhile for medium to large surveys, but it might not be so for small surveys. Insights on process development, ambient conditions, and endurance are likely to be specific to individuals, so the overall recommendation is that survey packers time themselves under various conditions to discover their own patterns and apply that knowledge.

Resource Allocation

For this study, preparing surveys by hand results in a savings of $46 per hour versus hiring a company for the task, based on preparation costs alone. Including time and costs associated with project management, like contacting post offices, raises the equivalent wage to $85 per hour. Further including the printing discounts achievable through self-management increases the equivalent wage to $142 per hour. While the self-prepared survey was undertaken in the United States and the professionally prepared survey was undertaken in Australia, similar techniques and cost structures apply in both countries, confirmed by local inquiries in the United States, suggesting the comparison is valid.

Crucially, the estimate of time value derived from this study is dramatically higher than that implied by per-piece estimates for various tasks at print shops for reasons that likely apply beyond this case study. Mail survey practitioners should be cautious of making decisions about packing personally versus paying for packing based on per-piece estimates rather than an all-in quote. For example, print shops quoted $0.03 to $0.05 per sheet to fold cover letters (e.g., Ace-E 2016; Staples 2016), implying a savings of $15–25/hour for the 20 hours it took to fold sheets by hand. However, accessing folding services at those rates required the use of specific printers, with prices three to six times the actual price paid. Similarly, a quote for envelope stuffing of $0.09 per three-insert mailer implies an equivalent hourly wage of $5.90 for stuffing only (Ace-E 2016). However, printing costs at those organizations were similarly high. Further, including the cost of shipping and returning externally printed materials increases the equivalent hourly wage estimate to $21. Further, per-piece estimates do not account for special requests like applying a specific first-class stamp rather than metered or indicia postage in hopes of increasing response rates (e.g., Armstrong and Lusk 1987; Harrison et al. 2002).

Recommendations

This case study details the results of a mail survey preparation experiment where some units were prepared and shipped by a professional survey company, while others were prepared and shipped manually by the survey designer. Cost- and time-tracking suggests that preparing the survey personally was worth an equivalent hourly wage of about $46 for preparation or $85 for preparation plus project management. Including the savings from aggressive pursuit of printing discounts raises the equivalent hourly wage for preparation plus project management to $142. Overall costs were $5.26 per unit for the professionally administered project vs. $1.83 plus 1.4 labor minutes per unit for the self-administered project. This evaluation assumes no cost for survey delays and material storage, two factors that justified the use of the more costly professional administration in that subsample for this case study.

For those interested in preparing a mail survey where the limiting factor is cost rather than time and space, this case study offers three major recommendations. First, as a rule of thumb, expect to spend 1–2 minutes per unit to pack a survey. Second, make decisions about packing-related time and cost tradeoffs based not simply on task-based unit costs like paper folding, but on all-in costs, including shipping, restrictions on printers, and special requests like first class rather than metered postage. Third, for those with some time flexibility, be aware that print shops often offer discounts to first-time customers, customers who sign up for a newsletter, or customers purchasing during a holiday season. Such discounts were observed at multiple printers, allowing for comparison shopping and well-timed purchases resulting in substantial savings. Where money is a more important limitation than time, mail survey practitioners willing to self-administer can dramatically reduce financial costs at the expense of labor time and logistical constraints.

Acknowledgments

The author gratefully acknowledges the work of Noelle Herring, a project research assistant. This work was supported by Stanford University’s Emmett Interdisciplinary Program in Environment and Resources, School of Earth, Energy and Environmental Sciences, and Vice Provost for Graduate Education and by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE-114747. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.