Background

The decline in survey response rates raises significant concerns for survey research (de Leeuw, Hox, and Luiten 2018; Williams and Brick 2018). Traditionally, information on non-participation has been gathered through paradata and short non-response surveys distributed to non-respondents after refusal. The classification of ‘non-respondents’ may vary per study, but generally these are people who rarely participate in surveys. To gain a deeper understanding of non-participation, we pursued a research approach involving semi-structured interviews with non-respondents. We developed an interview guide drawing from key survey theories (Dillman 2020; Keusch 2015) and the practical experiences of Statistics Netherlands (SN), to explore non-respondents’ experiences and reasons for not participating in surveys.

Recruitment

Between June and August 2023, 29 experienced SN interviewers visited 2763 addresses of individuals who had not responded to a request by mail and reminders (two additional letters) of one of six different SN surveys: the Labor Force Survey (n=1303), The Netherlands Depicted Survey (n=626), the Health Survey (n=525), the Social Cohesion Survey (n=158), the Lifestyle Monitor (n=102), and the Time-Use Survey (n=49). Each survey operates with a complex sequential mixed-mode design, where each individual was initially offered computer-assisted web interviewing, except for the Time-Use Survey which starts with direct home observation. Individuals who did not respond were asked once more to participate in a SN survey. The interview could be conducted at the doorstep immediately, or there was the option to schedule an appointment for a computer-assisted personal interview. If the individual declined, the interviewers offered the option to take part in a qualitative interview study conducted by the University of Groningen (UG). It was emphasized that this was not a survey but a conversation about survey experiences. If the non-respondent agreed to participate in the UG study, they were asked to fill out and sign a form (in compliance with GDPR) to facilitate further contact for the UG interview.

Evaluation

Despite the diligent efforts of interviewers, none were able to successfully convince non-respondents to engage in a conversation about surveys as part of the UG interview study. Of the people contacted, 1085 successfully participated in a SN survey upon request, while 1678 declined or provided another explanation for non-participation, with 781 specifically declining personally. Refer to Table 1 for a breakdown of the reasons for non-response and other explanations for non-participation, with the main reason for non-response recorded by the SN interviewers for each individual (i.e., only one reason was selected).

An evaluation of the recruitment phase in an expert meeting with SN interviewers, revealed that most individuals declined participation in the UG interview study for similar reasons to their refusal for the SN survey. The SN interviewers specifically mentioned that in a few cases they were very close in persuading non-respondents. However, once they asked for the personal information to be added to the form, people again refused. Adding a signature gave the process added weight and raised concerns about privacy. Therefore, we recommend to develop an electronic way to establish consent. Once the consent request becomes electronic and integrated into the registration of a prior refusal, the recruitment process could run more naturally with potentially fewer privacy concerns.

In the expert meeting, the interviewers proposed three strategies for future steps:

  1. Explore a collaborative approach where a UG researcher joins the SN interviewer during recruitment to emphasize the importance of the UG study and encourage participation.

  2. Consider offering an incentive, which was not possible in the current study because respondents did not receive compensation for participating in the SN surveys and therefore it was decided to not offer an incentive during recruitment for the UG study as well.

  3. Develop a tailored fact sheet for the UG study to enhance the relevance and saliency of the research and its topic to potential respondents. This idea was based on previous positive experiences with SN’s own customized fact sheets to persuade potential respondents.

Table 1.Reasons for non-response
Personal Declines (n=781)
No time/Not interested 539
Privacy/Anonymity concerns 48
Long-term illness 45
Participated too many times in surveys already 37
Conciously not responded after reading SN request and reminders 31
Survey topic 29
Refusal through other person 28
Aggressive decline 12
Too old / too young to participate 11
Other 1
 
Other Explanations (n=897)
Language barrier 276
Individual has moved 155
Unable to reach anyone 129
No availability within fieldwork period 116
Scheduled appointment not honored 45
Vacant building / under construction 34
Moved abroad 27
Address not attended to 25
Incorrect address data 19
Interview ended prematurely 16
Institution (e.g., nursing home) 12
Unsafe situation 9
Person temporarily admitted to institution (e.g., hospital) 7
Deceased 3
Other 24
 
Total 1678

Funding

This project was funded by the SSH Open Competition XS grant of the Dutch Research Council.

Lead author contact information

Marieke Haan, PhD
marieke.haan@rug.nl
Sociology department,
Faculty of Behavioral and Social Sciences,
University of Groningen
Netherlands.