Asians―Are They The Same? Findings from Cognitive Interviews

Hyunjoo Park; Virginia Wake Yelei

Asian populations in the U.S have grown at faster rates than the U.S. population as a whole. Monolingual Asian-language speakers are rarely included in U.S. surveys, so our understanding of surveys administered in Asian languages is limited. This paper presents findings from cognitive interviews in three Asian languages and reviews similarities and differences in respondents’ interpretations of translated survey messages used in the American Community Survey (ACS) materials.

METHODOLOGY

We analyzed cognitive interview data of monolingual Asian respondents’ understanding of ACS materials (introductory letter, thank you letter, Q&A brochure, and informational brochure), translated into Chinese, Korean, and Vietnamese by the Census Bureau. Twenty-four monolingual respondents for each of these three languages (72 respondents in total) were recruited by diverse recruiting methods: newspaper advertisement, direct contact with community leaders, and word of mouth. These respondents were recruited with diverse backgrounds consistent with previous ACS demographic statistics.

The protocol of the cognitive interviews was developed in English to standardize interviews and then was translated into each language through a modified committee approach (Schoua-Glusberg 1992). We examined the respondents’ answers to the retrospective probing questions to assess their understanding of the key messages (see Appendix A for more details). Our analysis was conducted on interview summary reports prepared by interviewers, who were professional researchers.

FINDINGS

Our analysis focused on respondents’ reactions to the supporting materials’ key messages, which included the survey participation request, ACS purpose, confidentiality assurances, and mandatory nature of the ACS.

Survey Participation Request

Most respondents failed to notice the survey participation request message, which appears in the first paragraph of the letter. More than two-thirds of the respondents appeared to overlook this message. This oversight was caused by both a different communication style in Asian culture and the respondents’ lack of knowledge about surveys.

The survey participation request message in the supporting materials was presented in Western letter-writing style at the beginning of the text. However, an Asian requesting message should be structured with the main message located at the end of a letter after background information is provided (Pan et al. 2006; Scollon and Wong-Scollon 1991). Consequently, Asian readers tend to look for the crucial points later. This mismatch of the main message organization led to the failure of respondents to interpret the message of survey participation as a request. In addition, the participation request was written without explaining the ACS, presupposing that readers understand the concept of surveys. Since most respondents did not have prior knowledge of surveys, they did not understand what was being asked in the letter and thus missed the participation message.

What is the ACS?

Many respondents confused the ACS with the Decennial Census, which collects only basic demographic characteristics of the population.

Korean respondents were more familiar with the concept of surveys, and even older respondents (aged 65+) said surveys were popular in Korea. However, no specific examples of ACS questions were included in the materials, which caused confusion.

Chinese and Vietnamese respondents had more difficulty, due to both their unfamiliarity with the concept of surveys and problematic translations. The Chinese term “美国社区调查” for “American Community Survey” is often mistaken as “美国社会调查” (American Social Investigation); there is only a one-word difference between the two translations. The current translated term “调查” has several meanings ― investigation, survey, or research. Since most Chinese respondents were familiar with “social investigation” but had little experience with “surveys,” they opted for a familiar concept, which led to the misinterpretation that ACS is asking for open-ended personal feedback.

The Vietnamese responses demonstrated a similar pattern by misinterpreting the ACS as a social welfare program. The Vietnamese translated term “Khảo sát cộng đồng Mỹ” for “American Community Survey” also contains a high-level vocabulary, which is difficult for many Vietnamese people with a lower level of education to understand. The current translation “ăn phòng Kiểm tra Dân số Hoa Kỳ” (the U.S. Census Bureau) is back translated to “U.S. Office of Examining Population” in Vietnamese, which carries a connotation of surveillance.

Survey Purpose

Most respondents understood that the survey data would be used to help communities, but did not understand how the data could help. The fact that respondents could not articulate the ACS’s purpose is rooted in poor translations. Another factor was respondents’ lack of social experience in their home countries, where individuals’ responses to surveys are not part of the government’s policy-making process.

The term “tribal governments” presented a problem across all groups. A few respondents thought it should be “local or county government.” Although the concept of tribes exists in China and Vietnam, such tribes are not considered in national decision-making processes in these cultures. This indicates that respondents did not have social experience concerning the functions of tribes in governmental decisions and were not familiar with the role of American Indians in U.S. history.

Mandatory Nature

Participation in the ACS is mandatory. However, half of our Chinese and Vietnamese respondents did not understand what this meant in terms of taking action. They guessed it might mean people should answer the survey honestly. A couple of Vietnamese respondents were confused and thought the confidentiality statement of a mandatory nature. They interpreted it to mean that the information they provided to the ACS would be confidential under the law. This is because of poor translation, which did not specify what is required by law.

The Korean respondents understood the meaning of mandatory nature better than the Chinese and Vietnamese respondents, but showed somewhat different reactions. They felt uncomfortable with the direct expression “You are required by U.S. law to respond to this survey.” In Korean culture, where politeness and interpersonal relationships are emphasized, this type of direct request is rarely used; respondents thought the translated mandatory participation message was rude.

Confidentiality

It was well understood by all groups that respondents’ answers are protected. However, the emphasis on law caused different reactions: fear of the law and distrust of the law. One-third of the respondents thought there was too much emphasis on this issue. They thought the information they provided would not be sensitive, so mentioning specific laws made them suspect hidden risks of participation. Additionally, some respondents distrusted the law: while they understood the literal meaning of the text, they did not believe it.

Distrust of law and government may also relate to the different relations between law and government in American and Asian cultures. Western conceptions of rule of law tend to emphasize the ways in which law limits the powers of the government and increases individual freedom; Asian conceptions tend to associate law as enhancing the power of the government (Tamahana 2004). This explains why emphasis on the law could cause fear and distrust rather than comfort or encouragement for Asian respondents.

CONCLUSION

We found that Asian populations are not exactly the same in their reactions to key ACS messages. Respondents exhibited similar reactions in the areas where they share some cultural and literacy traditions; however, each language group has its unique reactions.

Most of our respondents missed the survey participation request and did not understand the mandatory participation message. Many respondents did not believe their data would be protected, while others did not feel the necessity of protecting the data.

The Korean respondents understood ACS as a survey relatively well. This is because the concept of surveys is more common in Korea; the translation of the survey into Chinese and Vietnamese carried additional meaning such as “social investigation” or “surveillance.” The Korean respondents were uncomfortable with the direct mandatory participation statements because Korea’s cultural and social conventions emphasize explicit use of politeness.

Readers should be cautious about generalizing the findings of this research because our sample is not statistically representative. We identified the sources for differences between the groups based on our analysis of the interpretations of the projects’ language expert panel. Additional research needs to be conducted to expand our understanding.

Asian populations have typically been treated as a single entity. However, this one-for-all type of treatment may not be an effective way to deal with the unique issues of each language group. We also found that monolingual respondents’ exposure to U.S. society and mainstream cultures is limited and they tend to interpret social phenomena using cultural baggage from their home countries. Therefore, it is important to consider their different socio-cultural backgrounds when we include them in survey research. Our findings are consistent with previous studies (Park and Pan 2007; Pan and Landreth 2009), which also expand our understanding of how language, social practice, and culture impact Asian respondents’ interpretations of surveys and survey materials. From this study, we concluded that three major factors should be considered in survey development for Asian respondents: a) language difference that may compromise the survey messages to be communicated through direct translations; b) little or no social experience with surveys that impacts respondents’ understanding; and c) varied cultural norms of communication in different cultures that shape respondents’ interpretations.

Acknowledgements

Disclaimer: This report is released to inform interested parties of ongoing research and to encourage discussion of work in process. The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau. The authors wish to thank Yuling Pan, Terry DeMaio, and Stephanie Sheffield for reviewing this paper.

Suggested Citation

Park, Hyunjoo, Virginia Wake Yelei. 2010. “Asians—Are They the Same? Findings from Cognitive Interviews” Survey Practice, February: www.surveypractice.org.

Appendix A. Key Messages and Examples of Probing Questions

Key Message (Texts on Material)

Survey Participation Request The U.S. Census Bureau is conducting the American Community Survey. A Census Bureau representative will contact you to help you complete the survey. I would appreciate your help, because the success of this survey depends on you.
What is ACS? The American Community Survey is a survey conducted by the U.S. Census Bureau. It provides information each year about the social, economic, and housing characteristics of the United States. Previously, this information was available only when the U.S. Census Bureau conducted a population census every 10 years. Only a small sample of addresses is randomly selected to participate in the American Community Survey and they represent other addresses in the community.
Survey Purpose The ACS produces critical, up-to-date information that is used to meet the needs of communities across the United States. For example, results from this survey are used to decide where new schools, hospitals, and fire stations are needed. Survey data are used by federal, state, local, and tribal governments to make decisions and to develop programs that will provide health care, education, and transportation services that affect you and your community. This survey information helps communities plan for emergency situations that might affect you and your neighbors.
Mandatory Nature You are required by U.S. law to respond to this survey (Title 13, United States Code, Sections 141, 193, and 221).
Privacy/Confidentiality Your answers are confidential by law (Title 13, United States Code, Section 9). This law requires that every Census Bureau employee—including the Director and every Census Bureau representative—take an oath and be subjected to a jail term, a fine, or both if he or she discloses ANY information that could identify you or your household.