Special Issue on Non-probability Samples

This issue covers only one topic – nonprobability sampling. Andy Peytchev selected the articles and edited the issue. Some articles have formulas and the content of many articles is too complex for display using the software we use to publish SP, so we are experimenting with PDFs.

The articles span a broad spectrum, including the evaluation of bias in a nonprobability sample, the review of assumptions in a nonprobability sampling method that provide the potential for bias, the conditions under which a nonprobability sampling design can lead to valid conclusions in comparative research, case studies on the use of nonprobability methods and samples to facilitate a probability-based study, and a proposed method to combine probability and nonprobability samples under certain conditions.

Gerty Lensvelt-Mulders and colleagues use a probability-based web survey with telephone follow-up and propensity score matching in order to evaluate bias in a nonprobability web panel survey. This design and analytic approach allow them to attempt to separate bias due to self-selection from bias due to undercoverage in the panel survey.

Although not nearly as much in the survey literature, Respondent Driven Sampling has received considerable attention as a nonprobability sampling method that claims to produce representative estimates. In her article, Sunghee Lee dissects the sources of error in RDS from a total survey error perspective.

Murray Straus presents an evaluation of the validity of cross-national comparisons using nonprobability sampling, when the study design is held similar across countries. Based on data from 29 countries, he uses construct validity to evidence the usefulness of nonprobability sampling for comparative research under such conditions.

Marcus Berzofsky and his colleagues present an ongoing large-scale establishment survey in which a probability sample design is aided by nonprobability quota sampling to increase efficiency, within a single model-aided sampling design. They also present an evaluation of the tradeoff between bias and efficiency (reduction of cost and respondent burden hours).

Barry Johnson and Kevin Moore present an example where the Survey of Consumer Finances, a probability sample, is augmented with estate tax return data, a nonprobability sample, using a multiplier technique. This allows the reporting of detailed estimates for relatively small subpopulations. Time trends and simulations are used to support the data augmentation.

A somewhat similar but more general approach to augmentation of a probability sample with data from a nonprobability sample is proposed by Michael Elliott. A detailed description is provided for the estimation and scaling of weights that can then be used on the combined data from both samples. A simulation study shows that when model assumptions are sufficiently satisfied, bias and MSE of estimates can be reduced.

Articles in the Special Issue on Nonprobability Sampling

  • Separating Selection Bias and Non-coverage in Internet Panels using Propensity Matching
  • Understanding Respondent Driven Sampling from a Total Survey Error Perspective
  • Validity of Cross-National Research Using Unrepresentative Convenience Samples
  • Combining Probability and Non-Probability Sampling Methods: Model-Aided Sampling and the O*NET Data Collection Program
  • Using the Tax Data to Estimate Wealth for Key Segments of the U.S. Population
  • Combining Data from Probability and Non-Probability Samples Using Pseudo-Weights

The Editors