Loading [Contrib]/a11y/accessibility-menu.js
Survey Practice
  • Menu
  • Articles
    • Articles
    • Editor Notes
    • In-Brief Notes
    • Interview the Expert
    • Recent Books, Papers, and Presentations
    • All
  • For Authors
  • Editorial Board
  • About
  • Issues
  • search

    Sorry, something went wrong. Please try your search again.
    ×

    • Articles
    • Blog posts

RSS Feed

Enter the URL below into your favorite RSS reader.

https://www.surveypractice.org/feed
×
Articles
Vol. 8, Issue 4, 2015August 31, 2015 EDT

Do Different Recruitment Methods Reach Different Asian Demographics?

Hyunjoo Park, Mandy M Sha,
asian respondent recruitment cognitive interview
• https://doi.org/10.29115/SP-2015-0019
Survey Practice
Park, Hyunjoo, and Mandy M Sha. 2015. “Do Different Recruitment Methods Reach Different Asian Demographics?” Survey Practice 8 (4). https://doi.org/10.29115/SP-2015-0019.
Save article as...▾
  • PDF
  • XML
  • Citation (BibTeX)
Data Sets/Files (3)
Download all (3)
  • Table 1  Frequencies of respondent characteristics: combined data.
    Download
  • Table 2  Frequencies of respondent characteristics: Korean vs. Chinese speakers.
    Download
  • Table 3  Chi-square test of demographic variables across recruitment methods.
    Download

Sorry, something went wrong. Please try again.

If this problem reoccurs, please contact Scholastica Support

Error message:

undefined
×

View more stats

Abstract

In systematic analyses of recruitment data, Liu, Sha and Park (2013) and Park and Sha (2014) examined the efficiency of four common recruitment methods to recruit Asian research participants who speak little or no English. Continuing this line of research, this paper examines whether there are considerable differences in demographic characteristics among the Asian research participants recruited through these four recruitment methods: newspaper ads, physical flyers, online communication, and word of mouth. We examined 845 recruitment records completed with Chinese and Korean speakers for a cognitive interviewing study of the American Community Survey (ACS) Language Assistance Guide, conducted by the U.S. Census Bureau. The findings of this paper will be of interest to researchers of cross-cultural studies who would like to recruit specific Asian demographics.

Background and Introduction

Recruitment of minority research participants poses unique challenges as compared to recruiting among the general population. In addition to print and online advertising or flyer posting, recruiters have often used community organizations serving the target minority populations or key informants to implement a snowball sample recruitment using word-of-mouth (Appel et al. 1999; Gilliss et al. 2001; Harris et al. 2003; Hughes et al. 2004). While a considerable amount of research has been conducted to examine the strategies for recruiting racial and ethnic minority participants, they have concentrated in health research and special clinical population who speak English (Austin-Wells, McDougall, and Becker 2006; Clark et al. 2007; Gilliss et al. 2001; Horowitz et al. 2009; Jancey et al. 2006; Maxwell et al. 2005; Qualls 2002; Reed et al. 2003; Roosa et al. 2008; Sullivan-Bolyai et al. 2007; Yancey, Ortega, and Kumanyika 2006). The recruitment of non-English-speaking populations has not been studied as extensively.

Liu et al. (2013) and Park and Sha (2014) conducted systematic analysis of recruitment data of Asian research participants who speak little or no English. They examined the efficiency of four common methods for recruiting research participants: newspaper ads, physical flyers, online communication, and word-of-mouth. Newspaper advertisements were found to be most efficient in reaching a larger number of Asians while using the least amount of recruiters’ time. For recruiting non-English speakers, word-of-mouth by recruiters with strong ties to the ethnic community worked best.

Although we are not aware of studies that demonstrate to what extent Asians have been underrepresented in research studies, literature shows that Asians who have limited English language proficiency are hard to recruit because of a general lack of experience participating in survey and social science research, sociocultural differences, and mistrust (Park and Sha 2014). Tailoring the recruitment methods according to the recruits’ demographics may help with recruiting these hard-to-reach Asians with limited English language proficiency. However, we are not aware of prior research that provides systematic analysis. This paper fills the gap by examining whether there are considerable differences in demographic characteristics among the Asian research participants recruited through the four commonly used recruitment methods.

Methodology

The recruitment data were collected for a cognitive interviewing study conducted by the U.S. Census Bureau that pretested the Chinese and Korean translations of the American Community Survey (ACS) Language Assistance Guide (LAG). Recruitment data were collected by administering a screening questionnaire, including the sources where a person heard about the study, the respondent’s demographic characteristics, immigration year, self-rated English language skills, and additional characteristics that serve the purpose of the study. The recruitment took place between December 2009 to December 2010 in Illinois, North Carolina, and the greater Washington DC area. Out of the 845 completed screener records, we were able to include 778 (250 Chinese and 528 Korean) in this analysis.

Recruits could be monolingual or bilingual, but only those who speak little or no English were eligible to be selected for interviewing in the study. Four common recruitment methods were used: (1) newspaper ads were published in ethnic in-language newspapers in the Chinese or Korean languages; (2) physical flyers were posted at locations frequented by potential respondents (i.e., churches, ethnic grocery stores, or restaurants); (3) online information about study was disseminated through e-mailing lists or online forums subscribed by a large number of Chinese and Korean speakers); and (4) word-of-mouth where the recruiters contacted community leaders or key informants for help or recruited respondents face to face. All forms of advertising shared the following message: 1) directed toward Chinese and Korean speakers, 2) contained information about the purpose of the study, 3) stated the length of the interview and the incentive, and 4) mentioned the survey sponsor.

To examine the differences in demographics recruited through the four methods, we first reviewed the frequency of key demographic variables (sex, age, educational attainment, entry year to the United States, English speaking fluency, English reading fluency, whether monolinguals or bilinguals) for all recruits and then by Chinese and Korean languages. We then ran Pearson chi-square statistics to determine whether the differences were statistically significant.

Results

As shown in Table 1, the four recruiting methods reached different subgroups in varying degrees. When compared to newspaper ads, physical flyers, and word of mouth, the online method reached relatively younger, highly educated, recent immigrants. Compared to all recruits, the online recruits were younger (37 percent of online recruits were less than 35 years old compared to 23 percent of the entire recruits), highly educated (70 percent of them were college graduates compared to 55 percent of the entire recruits), and were likely to be bilinguals (50 percent of them were bilinguals compared to 33 percent of the entire recruits).

Table 1
Key demographics Categories Newspaper
(n=223)
Physical flyer
(n=181)
Online
(n=101)
Word-of-mouth
(n=273)
Total
(n=778)
Education Less than high school 5% 7% 6% 27% 13%
High school 28% 30% 24% 40% 32%
College or above 67% 64% 70% 34% 55%
Age 18–34 23% 29% 37% 15% 23%
35–44 22% 17% 22% 15% 18%
45–54 32% 21% 28% 21% 25%
55–64 13% 15% 10% 25% 17%
65+ 11% 18% 4% 24% 16%
Entry year to the United States Before 1980s 6% 9% 1% 10% 7%
1980s 15% 19% 10% 18% 16%
1990s 26% 25% 22% 21% 23%
2000s 52% 45% 67% 51% 52%
After 2010 1% 2% 0% 1% 1%
Self-rated English speaking ability Very well 6% 3% 7% 1% 4%
Well 29% 27% 33% 12% 23%
Not well 53% 61% 48% 51% 54%
Not at all 11% 9% 13% 35% 19%
Self-rated English reading ability Very well 9% 4% 12% 2% 6%
Well 36% 36% 39% 11% 28%
Not well 42% 50% 39% 51% 46%
Not at all 12% 10% 11% 37% 20%
Language proficiency Monolingual 54% 60% 50% 87% 67%
Bilingual 46% 40% 50% 13% 33%

On the contrary, older, lower educated immigrants were recruited more frequently by the word-of-mouth method. It also recruited the highest percentage of monolinguals who chose “not at all” when asked to rate their own English speaking (35 percent) and English reading ability (37 percent). This is compared to 19 percent for English speaking and 20 percent for English reading ability for all recruits regardless of recruitment methods.

In addition, respondents recruited via newspaper advertisement and flyers were similar in terms of self-rated English speaking and reading abilities and entry year to the United States.

When Chinese and Korean data were examined separately, we observed a similar pattern as shown in Table 1’s combined data. For instance, online method reached younger, highly educated, bilingual Chinese or Korean speakers. For both language groups, the word-of-mouth method recruited more elderly, lower educated, and monolingual Chinese and Korean speakers.

As shown in Table 2, among Korean speakers only, word-of-mouth was the most effective recruitment method for recruiting monolingual Koreans (85 percent) compared to newspaper advertisement (47 percent), physical flyer (57 percent), and online (37 percent). In comparison, word of mouth and physical flyers were equally effective in recruiting 89 percent of monolingual Chinese speakers, followed by newspaper ads (83 percent) and online (72 percent).

Table 2
Language group English proficiency News-paper Physical flyer Online Word-of-mouth Total
Korean Monolingual 83 (47%) 93 (57%) 24 (37%) 105 (85%) 305 (58%)
Bilingual 94 (53%) 70 (43%) 41 (63%) 18 (15%) 223 (42%)
Chinese Monolingual 38 (83%) 16 (89%) 26 (72%) 133 (89%) 213 (85%)
Bilingual 8 (17%) 2 (11%) 10 (28%) 17 (11%) 37 (15%)

To examine whether the demographic differences are statistically significant, we conducted Pearson chi-square statistics. As shown in Table 3, all of the differences in key demographics across the four recruitment methods were statistically significant in the combined data at p=0.05 level. The same statistically significant results are observed in the Korean dataset. In the Chinese dataset, half of the differences reached the statistical significance at p=0.05: education, entry year to the United States and English reading ability. However, age, English speaking ability and monolingual vs. bilingual characteristics were not statistically significant.

Table 3
Key demographics Combined data
(n=778)
Korean data
(n=528)
Chinese data
(n=250)
Age 71.2 (0.000) 74.8 (0.000) 18.8 (0.224)
Education 101.5 (0.000) 33.1 (0.000) 46.0 (0.000)
Entry year to the United States 22.6 (0.031) 31.7 (0.000) 22.7 (0.030)
English speaking ability 87.2 (0.000) 84.1 (0.000) 11.4 (0.248)
English reading ability 122.6 (0.000) 84.9 (0.000) 17.5 (0.042)
Monolingual vs. bilingual 83.8 (0.000) 58.6 (0.000) 6.7 (0.083)

Discussion

Based on frequency reviews, we found that the online method reached younger, highly educated, bilingual Asians more frequently as compared to the other methods. A higher percentage of older, lower educated immigrants were reached through word-of-mouth. Newspaper ads and physical flyers reached somewhat overlapping population groups when demographics like education and self-rated English language ability are considered. Therefore, researchers may consider using only newspaper ads or only flyers to recruit these demographics. Such a decision should be made with consideration of cost, time, and specific needs of the project.

Compared to newspaper ads, flyers, and online methods, Asians who were contacted through word-of-mouth reported the highest percentage of “not at all” for their English speaking and reading ability. As shown in Table 1, a much higher percentage of monolingual speakers (87 percent) were recruited by word-of-mouth compared to 67 percent of all recruits regardless of methods. Because only monolingual Chinese and Korean speakers were eligible to be selected for interviewing in this project, word-of-mouth most effectively achieved this recruitment goal.

When the data was analyzed separately by Chinese and Korean languages, we found that the word of mouth method was the most effective method to recruit monolingual Korean speakers. Both word-of-mouth and flyers were equally effective recruiting monolingual Chinese speakers. The word-of-mouth method also recruited more elderly and lower educated respondents for either language, which is consistent with the observation from the combined data.

Our statistical testing showed that differences in key demographics (age, education, entry year to the United States, English speaking ability, English reading ability, and monolingual vs. bilingual) across the four recruiting methods were statistically significant in the combined dataset and in the Korean dataset. In the Chinese data, the lack of statistical power shown in half of the demographics (age, English speaking ability and monolingual vs. bilingual characteristics) could have reflected smaller differences of the tested key demographics among the recruitment methods. However, we rather suspect that a smaller sample size in some recruitment methods in the Chinese data could have contributed to this (e.g., n=18 for the physical flyers method).

Limitations

There are several limitations to this study. First, the recruitment data are not statistically representative and consisted of volunteers who responded to the recruitment messages. Second, word-of-mouth encompassed several approaches, such as asking for referrals, contacting community leaders and organizations for help, or recruiting in-person at group settings. Use of each approach depends largely on recruiters but could also be influenced by the recruitment sites and by language. Additionally, the purpose of the recruitment activities was to find monolingual Chinese and Korean speakers with specific characteristics desired for pretesting translated questionnaires. Therefore, recruiters could have contacted people who appeared to fit the selection criteria. Furthermore, the recruitment data and analysis were limited to Asian research participants. Thus, our findings can only be used to inform recruitment of non-English speaking Asian demographics, not other non-English speakers in the U.S. However, because recruitment methods in general are similar (print or online ads, flyers or word-of-mouth), these findings are helpful to inform the recruitment of other hard to reach populations. Future research should replicate the research methods presented in this paper and analyze recruitment of additional non-English speakers and contribute to the recruitment literature that this paper has helped to establish.

Disclaimer

This paper is released to inform interested parties of research and to encourage discussion of work in progress. Any views expressed on (statistical, methodological, technical, or operational) issues are those of the author and not necessarily those of the Census Bureau.

References

Appel, L.J., W.M. Vollmer, E. Obarznek, K.M. Aicher, P.R. Conlin, B.M. Kennedy, J.B. Chaleston, and J.B. Reams. 1999. “Collaborative Research Group Recruitment and Baseline Characteristics of Participants in the Dietary Approaches to Stop Hypertension Trial.” Journal of the American Dietetic Association 99: S69–75. https://doi.org/10.1016/S0002-8223(99)00419-8.
Google Scholar
Austin-Wells, V., G.J. McDougall, and H. Becker. 2006. “Recruiting and Retaining an Ethnically Diverse Sample of Older Adults in a Longitudinal Intervention Study.” Educational Gerontology 32: 159–70.
Google Scholar
Clark, M., C. Neighbors, M. Wasserman, G. Armstrong, M. Drnach, S.L. Howie, and T.L. Hawthorne. 2007. “Strategies and Cost of Recruitment of Middle-Aged and Older Unmarried Women in a Cancer Screening Study.” Cancer Epidemiology, Biomarkers & Prevention 16: 2605–14.
Google Scholar
Gilliss, C.L., K.A. Lee, Y. Gutierrez, D. Taylor, Y. Beyene, J. Neuhaus, and N. Murrell. 2001. “Recruitment and Retention of Healthy Minority Women into Community-Based Longitudinal Research.” Journal of Womens Health & Gender-Based Medicine 101: 77–85.
Google Scholar
Harris, K.J., J.S. Ahluwalia, J.D. Catley, K.S. Okuyemi, M.S. Mayo, and K. Resnicow. 2003. “Successful Recruitment of Minorities into Clinical Trials: The Kick It at Swope Project.” Nicotine Tobacco Research 5: 575–84.
Google Scholar
Horowitz, C., B. Brenner, S. Lachapelle, D. Amara, and G. Arniella. 2009. “Effective Recruitment of Minority Populations through Community-Led Strategies.” American Journal of Preventive Medicine 376: S195–200.
Google Scholar
Hughes, C., S. Peterson, A. Ramirez, K. Gallion, P.G. McDonald, C.S. Skinner, and D. Bowen. 2004. “Minority Recruitment in Hereditary Breast Cancer Research.” Cancer Epidemiology Biomarkers Prevention 13: 1146–55.
Google Scholar
Jancey, J., P. Howat, A. Lee, A. Clarke, T. Shilton, J. Fisher, and H. Iredell. 2006. “Effective Recruitment and Retention of Older Adults in Physical Activity Research: PALS Study.” American Journal of Health Behavior 30: 626–35.
Google Scholar
Liu, L., M. Sha, and H. Park. 2013. “Exploring the Efficiency and Utility of Methods to Recruit Non-English Speaking Qualitative Research Participants.” Survey Practice 63: 1–8. http://www.surveypractice.org/index.php/SurveyPractice/article/view/56/.
Google Scholar
Maxwell, A.E., R. Bastani, P. Vida, and S. Warda. 2005. “Strategies to Recruit and Retain Older Filipino-American Immigrants for a Cancer Screening Study.” Journal of Community Health 303: 167–79.
Google Scholar
Park, H., and M. Sha. 2014. “Evaluating the Efficiency of Methods to Recruit Asian Research Participants.” Journal of Official Statistics 302: 335–54.
Google Scholar
Qualls, C.D. 2002. “Recruitment of African American Adults as Research Participants for Language in Aging Study: Example of a Principled, Creative, and Cultured-Based Approach.” Journal of Allied Health 314: 241–46.
Google Scholar
Reed, P.S., K.L. Foley, J. Hatch, and E.J. Mutran. 2003. “Recruitment of Older African Americans for Survey Research: A Process Evaluation of Community and Church-Based Strategy in the Durham Elders Project.” The Gerontologist 43: 52–61.
Google Scholar
Roosa, M., F. Liu, M. Torres, and N. Gonzales. 2008. “Sampling and Recruitment in Studies of Cultural Influences on Adjustment: A Case Study with Mexican Americans.” Journal of Family Psychology 222: 293–302.
Google Scholar
Sullivan-Bolyai, S., C. Bova, J.A. Deatrick, J.K. Knafl, M. Grey, K. Leung, and A. Trudeau. 2007. “Barriers and Strategies for Recruiting Study Participants in Clinical Settings.” Western Journal of Nursing Research 29: 486–500.
Google Scholar
Yancey, A.K., A.N. Ortega, and S.K. Kumanyika. 2006. “Effective Recruitment and Retention of Minority Research Participants.” Annual Review of Public Health 27: 1–28.
Google Scholar

Powered by Scholastica, the modern academic journal management system