Race-Matching: Interviewers’ Reactions to the Race-matching Process

Although there has been widespread attention among public opinion scholars to problems of data quality when dealing with racially sensitive items, there is little empirical evidence of how interviewers’ understand their work on survey projects that deal with racially sensitive questions and use race-matching protocols. For this exploratory study, we asked thirteen telephone interviewers involved in a racial attitudes survey to help us understand their experiences with that project and particularly with the race-matching process.

Public opinion scholars have established that the social environment matters, affording the characteristics of interviewers and perceptions of respondents weighting equivalent to the content itself (Anderson, Silver, and Abramson 1988; Davis 1997; Hatchett and Schuman 1975; Schuman and Converse 1971). Repeated studies have generated considerable knowledge about how interviewers affect data quality, but little is known about interviewers’ understandings of the process and its implications ^[1].

Project Background and Method

In 2003 the University of Arkansas at Little Rock, Institute of Government, Survey Research Center (SRC) initiated a telephone survey to collect, analyze, and disseminate community data relating to black/white relations. Most of the questions were based upon an instrument used by The Gallup Organization for a series on black/white relations in the U.S. (c1997–2007 The Gallup Organization ^[2].

Consistent with calls to mitigate the race-of-interviewer effects, the SRC developed an interviewer/respondent race-matching protocol (Cotter, Cohen, and Coulter 1982; Krysan 2002). Each interviewer was trained specifically for questionnaire content and the race-matching process.

We draw on qualitative semi-structured interviews lasting approximately 30 minutes. These thirteen interviewers represented thirty-three percent of the total workforce during the Year 2 survey. Nine respondents were black, nine were female; seven of the interviewers were employed for both the Year 1 and Year 2 surveys. Their ages range from 16 to 70. They completed 635 of the 1694, thirty-seven percent of all surveys in the Year 2 survey.

Findings

Senior staff at SRC conducted extensive training with interviewers to explain project content and the purpose of race-matching (RM). Some interviewers accepted the idea, others were ambivalent, and some were resistant. Acceptance of the practice indicated that interviewers understood RM as an attempt to increase respondent’s veracity.

While the interviewers may have heard the messages from training, they didn’t accept the premise necessarily. Interviewers were often confident in their skills and believed they should be able to interview anyone – a helpful trait to have among interviewers but one that led to confusion in this situation. Larry’s comments illustrate this tension.

“I was told that the purpose was that maybe some black people or some white people would feel more comfortable talking to someone of the same race, but I really didn’t see what was the difference.”

For some, RM emphasized race and accentuated difference. Making race salient to their work was uncomfortable.

“I didn’t agree with it at first . . .because in my eyes everybody is the same.” (Shirley)

Regardless of interviewers’ receptivity, training messages of how RM would promote comfort resonated. Almost all used the word “comfort” or some derivation.

“You feel more comfortable speaking with people of your race.” (Leslie)

While interviewers recognized RM might increase respondents’ comfort, they did not always connect it to themselves. Some acknowledged that RM increased their comfort.

“…It would be so uncomfortable for me personally to ask someone of a different race.” (Susan)

Our findings also suggest that interviewers may be affected by interviewing someone of a different race when the content is racially sensitive. Some interviewers noted they would have found it uncomfortable to interview respondents of a different race.

“If I was interviewing a white respondent, and they said something bad about my race, I would have been offended.” (Eleanor)

Some of these differences of opinion could happen when interviewers are race matched. For at least one interviewer, RM seemed paradoxical to the goals of ameliorating racial divisions.

“…I didn’t think it was a big deal that only black people should talk to black people and that only white people should talk to white people if they were willing to conduct a study, it really shouldn’t matter if you were black or white…” (Larry)

Regardless, most interviewers eventually embraced the idea because they believed RM increased honesty.

“I think that is the only way you were going to get positive, well, not positive but honest results regardless of what was said or how someone felt…” (Christine)

Nevertheless, interviewers’ perceptions of respondent honesty are complex. Some interviewers suggested that respondents were searching for socially desirable responses.

“They were trying to analyze the question and figure out what they think that we expect them to answer, and they were giving us those answers.” (Jean)

The commentary between questions may be more revealing than the answers themselves.

“…Always the white women would tell me ‘(whispers) we would move but we wouldn’t tell anybody that was why’ and I’m like that’s not an answer…Can you give me what I’ve asked you? And then they’d give me, ‘no, we wouldn’t move’…” (Shirley)

This finding reinforces the point that what goes on between questions is critical to data quality. Interviewers must be attentive to the total interaction. In so doing, they may feel frustrated by respondents’ contradictions.

Interviewers’ comments also suggest that respondents were frank -even if it was off-putting.

“I mean, I’m young, and there were a lot of older people, and they’d try to get me to agree with them. That’s what I really didn’t like about the answers, because you can express your opinion, but don’t say, “You know what I’m saying?” … “You’re one of us.” I didn’t like that. I may be black, but just because you feel that way doesn’t mean I feel that way.” (Christine)

Team leaders must help interviewers devise strategies to deal with respondents’ needs for affirmation.

Some interviewers suggested that when respondents recognized the match, they were increasingly honest.

“…When I would talk to black women, you could tell they were being distant, even through the phone, just by some of the answers they were giving, but then I think I would say a certain thing or a certain word a certain way, and, oh, she’s black, she’s a sister, and then they’d be, oh girl, let me tell you…and start talking to me like I’m their friend, and they’d be a little more honest, yes.”

Of course, respondents may not have recognized the match at the same points.

There were some unintended consequences for workers. Although there is contradictory evidence, the process may have altered the work environment. Some interviewers thought RM facilitated interaction among groups.

“…Since we had to occasionally stop and ask someone else to do the survey for us, that kind of made us interact more…like, no one hated each other going into it, no one hated each other coming out of it.” (Beth)

The team leaders shared this belief. Some interviewers disagreed and reported new levels of sensitivity.

”I believe some people did get upset about some things that were said. Like, we were sitting black person, white person, and I was, oh, God, don’t let this person say anything and then they’re going to hear it, and think I agree, and it’s not like that.” (Christine)

Interviewers expressed that they sometimes felt separated from each other because of RM.

“It was very divided…It was almost like we had the black people and the white people sitting on the other side of the room. Everything was, like, there’s this line; you can’t cross it. With the call records…With who I could talk to…And I knew that when I got on the phone with a black person, if I could tell by their voice that they were black, okay, well, hold on while I find someone of your race to answer these questions. That really separated caller from respondent, and it separated the respondents from each other, and it separated the workers from each other. It was very, very divided.” (Marie)

In spite of deliberate efforts to clarify the value and purpose of RM, interviewers felt some resistance to the process. While there were clearly differences in opinion, what emerged was a sense that respondents’ assumptions about the similarity of their positions frustrated interviewers. Their honesty and dishonesty posed challenges. Finally, a significant unanticipated consequence of race-matching was that the process highlighted racial difference among interviewers and may have created divisions.

Accordingly we offer the following tentative recommendations:

Inform respondents of the race match at the beginning of the survey in a standardized fashion.

Recognize interviewing is a social process; improving data quality depends on understanding respondents and interviewers.

Create debriefing opportunities for interviewers to share their experiences and concerns.

Offer mechanisms for interviewers to communicate knowledge they gain during the interview that may speak to the validity of respondents’ answers.

Integrate interactive workshops into training to develop greater appreciation of diversity and promote respect among interviewing staff.

Understanding how interviewers perceive their work and how they understand the data collection process is critical to improving data quality.