Introduction

A growing body of literature (e.g., Biemer and Lyberg 2003; Groves et al. 2009) examines the influence of different CATI interviewers’ characteristics (e.g., sex, age, beliefs) on the quality of survey data. However, little research examines the interviewers’ opinions of their work and the difficulties they encounter. In this paper, we present the results of the study conducted among CATI interviewers.

Methodology

The study was conducted between November 2009 and August 2010. Twelve major Polish commercial survey organizations participated in this project: 4P research mix; ASM Centrum Badan i Analiz Rynku; ARC Rynek i Opinia; Expert-Monitor (at present: Kantar Media Intelligence); GfK Polonia; IMAS International; IPSOS; Grupa IQS; MillwardBrown SMG/KRC; PBS DGA; Pentor Research International (at present: TNS Polska); and TNS OBOP (at present: TNS Polska). Each of these companies has CATI facilities, and each carries out telephone interviews on a regular basis. The criterion by which organizations were selected to participate was possession of a certificate in the CATI category issued by the Interviewing Quality Control Programme.[1] In 2009, a total of 16 firms were certified; however, six of these did not participate in the study – either they refused to participate or because their studio had been closed down.

A total of 846 interviewers took part in this study. Response rates fluctuated between 20 and 100 percent. (The former was the number of interviewers who were at work at least once within one month before the research was conducted.) Two methods of data collection were used. Across 10 agencies, interviewers were asked to complete paper questionnaires. In two other companies, we used an electronic questionnaire (CASI or CAWI).

Of course, the results are not representative of the entire population of CATI interviewers in Poland. Apart from certified agencies, there are firms performing telephone surveys which do not hold certificates. Moreover, at present, more and more research firms in Poland outsource fieldwork duties. This outsourcing occurs frequently across the CATI survey environment, likely due to the high costs associated with setting up and maintaining a CATI studio (Kelly et al. 2008).

In the questionnaire, we administered a Likert-type scale that presented 11 different interviewer duties, and participants were asked to what extent they found each item stressful. The choice of possible answers given was “very stressful,” “fairly stressful,” “not very stressful,” “not stressful at all,” “unable to say,” and “I’ve never come across such a situation.”

Findings

Exploratory factor analysis was employed to examine the structure of the data. Based on screen plot, we extracted two primary factors which render a telephone interview to be perceived as “stressful.” Use of Varimax rotation method showed that the first factor is associated with situations which may result in the respondent breaking off the interview. The second factor is concerned with activities which may cause cognitive difficulties to the interviewer. The detailed results are presented in Table 1.

Table 1  Stressful interviewer duties – factor analysis.
Factor – interviewer activity Mean (scale of 1: not stressful
at all to 4: very stressful)
Factor 1: Activities which may cause a break-off 2.64
 Conducting an interview with a respondent, who picks
 on many questions, complains about their pointlessness,
 incorrectness, etc. (0.807)		 2.85
 Introducing an interview with an aggressive respondent –
 who is shouting, using threats, etc. (0.791)		 2.85
 Conducting a long interview, lasting over 30 minutes (0.586) 2.62
 Refusal conversion – persuading a respondent, who refused
 earlier during a previous call (soft refusal), to participate in
 the research (0.710)		 2.26
Factor 2: Activities which may cause cognitive difficulties to the interviewer 2.03
 Conducting an interview with a hard of hearing respondent
 (0.773)		 2.54
 Conducting an interview with a respondent, who has a serious
 speech impediment – stutter, unclear speech, etc. (0.809)		 2.26
 Writing down a respondent’s very long answer to an open-
 ended question (0.627)		 1.82
 Conducting many different projects within a single shift (0.475) 1.53

Determinant = 0.027; K-M-O = 0.854; Varimax rotation; options “unable to say” and “I’ve never come across such a situation” were excluded from the analysis; numbers in brackets are the factor loadings; three additional items are not presented in the table, as they were not connected to any determined factor. (The excluded items are “Probing a respondent whose answers stray off the subject, are ambiguous, curt, etc.;” “Introducing an interview with a respondent, who claims that his phone number is on an opt-out telephone list and the research center has no right to call him (RDD sample);” “Introducing an interview with a respondent, who thinks that the real aim of the conversation is not to conduct an interview, but to persuade him to buy something instead.”)

The results indicate that activities which may result in terminating the interview are more stressful for the interviewers. Being able to finish the already started conversation is probably considered by the interviewers to be one of the indicators of the quality of their job. They might be aware of the fact that a break-off generates nonresponse error (see Peytchev 2009), as well as puts the research company to some expense, as the terminated interview must be repeated with another respondent. Usually, the reward structure which is in effect in survey centers reflects this relationship – in general, the higher the interviewer’s ratio of completed interviews, the better the interviewers are assessed. The risk of breaking off the conversation is associated with the CATI respondent’s personality traits (tendency toward aggression, complaints about the research tools, reluctance to participate in the research) or the length of interview. Activities that are perceived as somewhat less stressful are those which may result in cognitive difficulties experienced by the interviewers. These difficulties are primarily related to communication disruptions between interviewer and respondent, as well as a lack of coordination between the hearing and writing processes.

In order to analyze the results in more detail, we examined the correlation between participants’ stress levels and other variables that included their demographic profile and the items that pertained to the duties associated with their CATI interviewer position – see Tables 2–5 below.

Table 2  Stress level and interviewer’s sex.
Factor Interviewer’s sex
Woman Man
Factor 1 – mean 2.73 2.47
Factor 2 – mean 2.04 2.01
Table 3  Stress level and interviewer’s age.
Factor Interviewer’s age (years of age)
Up to 22 23–24 25–28 29 and more
Factor 1 – mean 2.56 2.53 2.70 2.78
Factor 2 – mean 2.06 1.96 1.97 2.12
Table 4  Stress level and interviewer’s work experience.
Factor Interviewer’s work experience in CATI studio (months)
Up to 4 5–11 12–23 24 and more
Factor 1 – mean 2.51 2.65 2.68 2.70
Factor 2 – mean 2.05 2.00 2.01 2.02
Table 5  Stress level and interviewer’s workload level.
Factor Interviewer’s workload level
Low Medium High
Factor 1 – mean 2.71 2.64 2.58
Factor 2 – mean 2.02 2.05 1.90

As can be seen, participants’ stress levels were more closely associated with CATI interview break-offs than interviewers’ cognitive difficulties. Specifically, women, older persons, interviewers with longer work experience, and interviewers with higher workload levels appeared to have rated situations as more stressful than the other participants. It is possible that older interviewers and those who have longer work experience regard their job in a more serious light. For instance, this job may cover their costs of living and may be used to provide for their families. As a consequence, such people may be more sensitive to situations in which they are fearful of being regarded as failures or poor performers. Lower stress levels among interviewers with higher workload levels, in turn, can be explained in terms of continuous interview skill improvement, and becoming less sensitive to potentially stressful conditions over time (see Zajonc 1968). However, lower susceptibility to stress can be also a personality trait, and more intense working habits may be a consequence of this lower susceptibility.

Activities which may result in cognitive difficulties experienced by the interviewer are also, albeit to a lesser extent, associated with analyzed background variables, with age and workload level acting as the primary indicators. Interestingly, the youngest and the oldest participants (i.e. up to 22 years of age and over 29 years of age) were more likely to regard their jobs as stressful when compared to participants between 23 to 28 years of age. In addition, participants with the highest workload level appeared to experience less stress when compared to people with fewer duties. Probably, the interviewers working less intensively are not very familiar with dealing with difficult situations and, as a result, they consider some situations to be more stressful (see Zajonc 1968). It is also possible that the same effect explains the fact that younger interviewers are relatively more stressed than others. In turn, older participants might experience higher stress level because of the more serious attitude they have towards their job (as explained in the previous paragraph).

Conclusion

This study demonstrates that there are two types of situations evoking stressful reactions among CATI interviewers. The first consists of interview activities surrounding the risk of terminating the interview, such as conducting long interviews, talking to an aggressive respondent, or encountering a respondent complaining about the questions asked. The second situation refers to the interviewers’ cognitive difficulties, such as in cases when respondents may be hard of hearing or have speech impediments.

CATI interviewers’ stress level seems to be an important factor in determining their overall performance while conducting interviews. Although this relationship has not been well examined in the telephone surveys environment, research findings from similar fields (call center customer service) indicate that employee stress has negative impact on job satisfaction, organizational commitment and performance (Babin and Boles 1996; Goolsby 1992). It has been argued that one of the main mechanisms that generates stress in a call center is the fact that the operators are often exposed to contradictory expectations (de Ruyter, Wetzels, and Feinberg 2001). For instance, CATI interviewers have to deal with talkative respondents, who digress and make irrelevant comments about the interview. On the one hand, interviewers feel they must obey the rule of politeness, while on the other, supervisors usually pay attention to the time that is dedicated to a single respondent – an interview that is too long becomes costly both in terms of money and time. As a consequence, the interviewers try to meet both requirements by balancing between being polite and being effective (Jablonski 2012).

Although being under stress is typical of CATI interviewer’s job, it seems it would be advisable that survey managers to enable operators to work in a more relaxing manner. They might try to reduce possible contradictions between certain expectations of interviewers. Moreover, they could also consider implementing call center relaxation programes and setting up recreation facilities to help staff take a rest during break times (see Mathiebe 2011).

Note

An extended version of this paper was presented at the AAPOR conference, Phoenix, AZ, May 2011.

  1. The Interviewing Quality Control Programme – modeled after British solutions to fieldwork issues – is the main Polish initiative promoting fieldwork quality standards in survey research. Research agencies which pass the audit can receive certificates in 8 categories: PAPI, CATI, CAPI, CAWI, qualitative research, self-administered studies, retail audit, and mystery shopping (Mazurkiewicz 2010).