Short Overview of Research Methods
In 2007, the Pew Research Center conducted what is believed to be the first-ever national telephone survey of a probability sample of Muslim Americans, a rare, dispersed, and highly diverse population. The study examined the political and social values, religious beliefs and practices, and life experiences of Muslims living in the U.S. today. Conducting the study presented several difficult challenges.
First, since Muslims constitute a relatively small portion of the overall adult population in the U.S. – approximately 0.5% of typical English-only survey samples – simply contacting and identifying a large sample of Muslims required a considerable investment. We oversampled from counties with a high prevalence of immigrants from predominantly Muslim countries and supplemented our sampling frame by re-contacting and interviewing self-identified Muslims from previous Pew surveys. In addition, we sampled from a commercial database of individuals with Muslim names. Further details about the sampling design and methods can be found in the full report on the survey, located on the Pew Research Center website.
Second, since the Muslim population in the U.S. includes many immigrants, a sizeable portion of the target population is unable to complete a telephone survey in English. Extrapolating from U.S. Census data and our survey, we estimate that between 9% and 22% of adult Muslims in the U.S. speak one of three languages: Arabic, Urdu or Farsi. Accordingly, we employed interviewers who are fluent in these three languages, with the result that approximately 12% of all interviews (and 17% of all weighted cases) were ultimately conducted in these languages. One consequence of this is that Pew’s estimate of the number of Muslims living in the U.S. was somewhat larger than that derived from other survey-based methods, which have typically depended on English-only surveys.
A third challenge was the sensitivity of the population to being interviewed. We designed the study and tailored the questionnaire to meet this concern by beginning the interview with neutral questions about satisfaction with the community, personal happiness, and other innocuous topics. After considerable experimentation, we decided to reveal the purpose of the study to respondents early in the interview, immediately following the opening sequence of neutral questions. The logic for revealing the focus of the study – a practice not common in survey research – was that Muslim respondents would quickly realize that the study was focused on Muslims and Islam, and that there would be a greater chance of establishing a bond of trust by revealing the intent of the survey in its early stages. Indeed, pretesting of the questionnaire indicated that to allay fears about the purpose of the study, it was essential to describe the purpose of the study and have available clear explanations of the sampling method for the interviewers to use in response to questions. Cumulatively, these efforts aided in gaining respondents’ trust and maximizing cooperation rates.
In addition to the major challenges listed above, we also confronted many practical fieldwork challenges. These included the massive volume of calling necessary to locate eligible respondents in a portion of the RDD sample and potential cultural issues related to interviewer/respondent gender matching. The calling volume had serious implications for interviewer morale and readiness, since the vast majority of interviewer effort was devoted to unsuccessful screening interviews. We made an effort to give all interviewers a mix of RDD sample and targeted sample, which meant that they would more frequently have the opportunity to complete an interview with an eligible respondent. Similarly, we instituted a protocol of having interviewers first ask for respondents of their own gender, which we believe yielded a better cooperation rate.