Context of this Research
There are a number of reasons to drop (or to add!) a survey mode in a panel survey (Lynn 2013). In the case of the Swiss Household Panel (SHP) 2013 refreshment sample (SHP III), the aim of the first wave was to reach as many as possible sample households and convince them to complete the biographical paper and pencil questionnaire. The Swiss Federal Statistical Office (SFSO) drew the SHP III at random from the Swiss population register, matched the names and addresses with its own telephone register, and delivered the registered numbers to the firm which conducts the survey. Note that the matching algorithm does not match all numbers and the SFSO does not provide unregistered numbers. To complete the household grid questionnaire, households with delivered telephone numbers received a telephone call, while the rest received a face-to-face visit (“computer-assisted personal interviewing [CAPI]-households”). In addition to the household composition, the grid questionnaire asks for (not matched) landline and mobile telephone numbers, in particular from the CAPI-households. From the second wave (2014), the SHP III sample conducts the annual questionnaire using computer-assisted telephone interviewing (CATI). The main reason to drop the face-to-face mode was high costs.
This context gives rise to a number of questions, such as:
- How many of the CAPI-households provided a landline telephone number (which could not be matched) or a mobile phone number?
- To what degree does registered landline match and telephone availability (and disclosure) as well as socio-demographic characteristics have an effect on participation status in the second wave?
Data and Descriptive Statistics
For the analysis, we use the 4,001 households from the SHP III who answered the grid questionnaire in 2013. As depicted in Table 1, 79.7 percent of these households had a matched landline in 2013. The 79.7 percent are larger than the proportion of the households sampled in 2013 that were matched by the SFSO (67.7 percent; see Lipps 2016). This shows that the group of matched households is more likely to be tracked, and/or easier to be contacted, and/or participate more (Sala and Lillini 2014). The complete picture of landline availability from the participating households in 2013 is depicted in Table 1. We list the CAPI households separately and include mobile telephone availability.
More than half of the CAPI households reported a landline number, which the SFSO could not match or did not deliver. The proportion of (at least one) reported mobile from CAPI households does not differ from the overall report of mobiles.
Next, we analyze if households distinguished by telephone availability exhibit a different retention behavior in 2014 (see Table 2).
The sample with a matched landline participated significantly more than the sample with a self-reported landline. The latter, in turn, are more likely to participate than those with only a mobile phone available.
Multivariate Model
In this section, we analyze the final participation status in 2014, using a multinomial model. The categories of the dependent variable are “participation” (reference category; 79.9 percent of the sample households), “not tracked” (4.3 percent), “no contact” (3.4 percent), and “refused” (12.5 percent).
The main independent variable is the survey mode (telephone vs. face-to-face) and telephone disclosure (self-reported landline and/or mobile) in 2013. In addition, we use characteristics of the households and the household reference persons from 2013 which have been shown to have effects on participation behavior (Groves and Couper 1998; Lipps 2016; Stoop 2005). Finally, we use call data information from 2013:
- CAPI vs. CATI (reference category)
- (self-reported) landline in the household that was not matched in 2013
- Reported availability of at least one mobile phone in the household
- Presence of at least one child in the household, and age of youngest child if children present: no child, 0–2 years, 3–6 years, 7–17 years
- Reference person is a foreigner from another than a neighboring country (and thus not sharing one of the survey languages)[1]
- Reference person is a foreigner with a short to midterm residence permit (up to B-permit)
- Reference person is a foreigner with a permanent residence permit (C-permit)
- Reference person is single and never married
- Single-person household
- Reference person is retired
- Living in a big city (more than 100,000 inhabitants)
- A deprivation index (number of market goods – max. 7 – not present because the household is too poor to purchase them)
- Number of calls in 2013
- Number of contacts in 2013
We tested interactions between the survey mode in 2013 and the other independent variables but did not find any significant effects. This is evidence of a uniform effect from the survey mode on attrition for all sample members.
In Table 3, we list the predicted marginal effects. We give a reading example for the row CAPI 2013, and the column “No contact”: the probability of not being contacted in 2014 is ceteris paribus 16.9 percent points higher if the sample household was surveyed via CAPI instead of CATI in 2013. In the interpretation, we take both significance and effect sizes into account.
As for effects from the mode in 2013 and telephone availability, we find that the CAPI households exhibit an increased likelihood to drop out for all reasons (significantly so only for no contact). These households are likely to be different from the others in respects related to participation (integration, income, socioeconomic conditions, etc.). Those with a self-reported landline or a mobile drop out to a smaller extent due to noncontact or noncooperation. While a mobile phone certainly improves contractibility, we suspect that both groups show trust, are easier to be contacted and cooperate more.
Regarding socio-demographic variables, we find an increased likelihood of an unsuccessful tracking for households with children and in particular those with a small child, or those with a short-term permit as well as those living alone. The likelihood of making contact is not particularly different across the socio-demographic groups, including the deprivation index. Foreigners from a non-neighbor country and to some extent retired people cooperate less.
Concerning fieldwork related variables, those with a higher number of calls are less likely to be tracked, to be contacted, and especially to cooperate. Noncooperation is also high for those with many contacts.
Fieldwork Recommendations
Based on these findings, we give some recommendations to lift the retention rate: Households that switched the survey mode from face-to-face to telephone, those with small children, foreigners with a short-term residence permit, and one-person households need special care to be tracked. These household types are known to move more often than others (Couper and Ofstedal 2009). More tracking effort is likely to be effective and in particular gathering more reliable contact information in the first wave to be able to find these household in the next wave. Again mode switchers, foreigners with a nationality from a country not sharing one of the survey languages and to some extent retired people may be motivated to stay in the panel with an extra incentive. Foreigners could also be approached by bilingual interviewers (Kappelhof 2015). Mode switchers need most care to be contacted. For them and for those with many calls, optimizing calling times (Lipps 2012) could be a good idea.
Foreigners from a neighboring country show no differences to native Swiss.