Background and Introduction

Recruitment of minority research participants poses unique challenges as compared to recruiting among the general population. In addition to print and online advertising or flyer posting, recruiters have often used community organizations serving the target minority populations or key informants to implement a snowball sample recruitment using word-of-mouth (Appel et al. 1999; Gilliss et al. 2001; Harris et al. 2003; Hughes et al. 2004). While a considerable amount of research has been conducted to examine the strategies for recruiting racial and ethnic minority participants, they have concentrated in health research and special clinical population who speak English (Austin-Wells, McDougall, and Becker 2006; Clark et al. 2007; Gilliss et al. 2001; Horowitz et al. 2009; Jancey et al. 2006; Maxwell et al. 2005; Qualls 2002; Reed et al. 2003; Roosa et al. 2008; Sullivan-Bolyai et al. 2007; Yancey, Ortega, and Kumanyika 2006). The recruitment of non-English-speaking populations has not been studied as extensively.

Liu et al. (2013) and Park and Sha (2014) conducted systematic analysis of recruitment data of Asian research participants who speak little or no English. They examined the efficiency of four common methods for recruiting research participants: newspaper ads, physical flyers, online communication, and word-of-mouth. Newspaper advertisements were found to be most efficient in reaching a larger number of Asians while using the least amount of recruiters’ time. For recruiting non-English speakers, word-of-mouth by recruiters with strong ties to the ethnic community worked best.

Although we are not aware of studies that demonstrate to what extent Asians have been underrepresented in research studies, literature shows that Asians who have limited English language proficiency are hard to recruit because of a general lack of experience participating in survey and social science research, sociocultural differences, and mistrust (Park and Sha 2014). Tailoring the recruitment methods according to the recruits’ demographics may help with recruiting these hard-to-reach Asians with limited English language proficiency. However, we are not aware of prior research that provides systematic analysis. This paper fills the gap by examining whether there are considerable differences in demographic characteristics among the Asian research participants recruited through the four commonly used recruitment methods.

Methodology

The recruitment data were collected for a cognitive interviewing study conducted by the U.S. Census Bureau that pretested the Chinese and Korean translations of the American Community Survey (ACS) Language Assistance Guide (LAG). Recruitment data were collected by administering a screening questionnaire, including the sources where a person heard about the study, the respondent’s demographic characteristics, immigration year, self-rated English language skills, and additional characteristics that serve the purpose of the study. The recruitment took place between December 2009 to December 2010 in Illinois, North Carolina, and the greater Washington DC area. Out of the 845 completed screener records, we were able to include 778 (250 Chinese and 528 Korean) in this analysis.

Recruits could be monolingual or bilingual, but only those who speak little or no English were eligible to be selected for interviewing in the study. Four common recruitment methods were used: (1) newspaper ads were published in ethnic in-language newspapers in the Chinese or Korean languages; (2) physical flyers were posted at locations frequented by potential respondents (i.e., churches, ethnic grocery stores, or restaurants); (3) online information about study was disseminated through e-mailing lists or online forums subscribed by a large number of Chinese and Korean speakers); and (4) word-of-mouth where the recruiters contacted community leaders or key informants for help or recruited respondents face to face. All forms of advertising shared the following message: 1) directed toward Chinese and Korean speakers, 2) contained information about the purpose of the study, 3) stated the length of the interview and the incentive, and 4) mentioned the survey sponsor.

To examine the differences in demographics recruited through the four methods, we first reviewed the frequency of key demographic variables (sex, age, educational attainment, entry year to the United States, English speaking fluency, English reading fluency, whether monolinguals or bilinguals) for all recruits and then by Chinese and Korean languages. We then ran Pearson chi-square statistics to determine whether the differences were statistically significant.

Results

As shown in Table 1, the four recruiting methods reached different subgroups in varying degrees. When compared to newspaper ads, physical flyers, and word of mouth, the online method reached relatively younger, highly educated, recent immigrants. Compared to all recruits, the online recruits were younger (37 percent of online recruits were less than 35 years old compared to 23 percent of the entire recruits), highly educated (70 percent of them were college graduates compared to 55 percent of the entire recruits), and were likely to be bilinguals (50 percent of them were bilinguals compared to 33 percent of the entire recruits).

On the contrary, older, lower educated immigrants were recruited more frequently by the word-of-mouth method. It also recruited the highest percentage of monolinguals who chose “not at all” when asked to rate their own English speaking (35 percent) and English reading ability (37 percent). This is compared to 19 percent for English speaking and 20 percent for English reading ability for all recruits regardless of recruitment methods.

In addition, respondents recruited via newspaper advertisement and flyers were similar in terms of self-rated English speaking and reading abilities and entry year to the United States.

When Chinese and Korean data were examined separately, we observed a similar pattern as shown in Table 1’s combined data. For instance, online method reached younger, highly educated, bilingual Chinese or Korean speakers. For both language groups, the word-of-mouth method recruited more elderly, lower educated, and monolingual Chinese and Korean speakers.

As shown in Table 2, among Korean speakers only, word-of-mouth was the most effective recruitment method for recruiting monolingual Koreans (85 percent) compared to newspaper advertisement (47 percent), physical flyer (57 percent), and online (37 percent). In comparison, word of mouth and physical flyers were equally effective in recruiting 89 percent of monolingual Chinese speakers, followed by newspaper ads (83 percent) and online (72 percent).

To examine whether the demographic differences are statistically significant, we conducted Pearson chi-square statistics. As shown in Table 3, all of the differences in key demographics across the four recruitment methods were statistically significant in the combined data at p=0.05 level. The same statistically significant results are observed in the Korean dataset. In the Chinese dataset, half of the differences reached the statistical significance at p=0.05: education, entry year to the United States and English reading ability. However, age, English speaking ability and monolingual vs. bilingual characteristics were not statistically significant.

Discussion

Based on frequency reviews, we found that the online method reached younger, highly educated, bilingual Asians more frequently as compared to the other methods. A higher percentage of older, lower educated immigrants were reached through word-of-mouth. Newspaper ads and physical flyers reached somewhat overlapping population groups when demographics like education and self-rated English language ability are considered. Therefore, researchers may consider using only newspaper ads or only flyers to recruit these demographics. Such a decision should be made with consideration of cost, time, and specific needs of the project.

Compared to newspaper ads, flyers, and online methods, Asians who were contacted through word-of-mouth reported the highest percentage of “not at all” for their English speaking and reading ability. As shown in Table 1, a much higher percentage of monolingual speakers (87 percent) were recruited by word-of-mouth compared to 67 percent of all recruits regardless of methods. Because only monolingual Chinese and Korean speakers were eligible to be selected for interviewing in this project, word-of-mouth most effectively achieved this recruitment goal.

When the data was analyzed separately by Chinese and Korean languages, we found that the word of mouth method was the most effective method to recruit monolingual Korean speakers. Both word-of-mouth and flyers were equally effective recruiting monolingual Chinese speakers. The word-of-mouth method also recruited more elderly and lower educated respondents for either language, which is consistent with the observation from the combined data.

Our statistical testing showed that differences in key demographics (age, education, entry year to the United States, English speaking ability, English reading ability, and monolingual vs. bilingual) across the four recruiting methods were statistically significant in the combined dataset and in the Korean dataset. In the Chinese data, the lack of statistical power shown in half of the demographics (age, English speaking ability and monolingual vs. bilingual characteristics) could have reflected smaller differences of the tested key demographics among the recruitment methods. However, we rather suspect that a smaller sample size in some recruitment methods in the Chinese data could have contributed to this (e.g., n=18 for the physical flyers method).

Limitations

There are several limitations to this study. First, the recruitment data are not statistically representative and consisted of volunteers who responded to the recruitment messages. Second, word-of-mouth encompassed several approaches, such as asking for referrals, contacting community leaders and organizations for help, or recruiting in-person at group settings. Use of each approach depends largely on recruiters but could also be influenced by the recruitment sites and by language. Additionally, the purpose of the recruitment activities was to find monolingual Chinese and Korean speakers with specific characteristics desired for pretesting translated questionnaires. Therefore, recruiters could have contacted people who appeared to fit the selection criteria. Furthermore, the recruitment data and analysis were limited to Asian research participants. Thus, our findings can only be used to inform recruitment of non-English speaking Asian demographics, not other non-English speakers in the U.S. However, because recruitment methods in general are similar (print or online ads, flyers or word-of-mouth), these findings are helpful to inform the recruitment of other hard to reach populations. Future research should replicate the research methods presented in this paper and analyze recruitment of additional non-English speakers and contribute to the recruitment literature that this paper has helped to establish.

Disclaimer

This paper is released to inform interested parties of research and to encourage discussion of work in progress. Any views expressed on (statistical, methodological, technical, or operational) issues are those of the author and not necessarily those of the Census Bureau.