The United States is a multi-racial and multi-cultural society. Social scientists conducting surveys face one problem: they are dealing with people from different cultural backgrounds. Similar to the challenge of international studies, we have to make sure that the survey measures are comparable across subpopulations with different cultures. To ensure comparability, we should consider two important factors: (1) equivalence of presenting the measures (whether the presentation of the stimuli is equivalent and comparable across different cultures); (2) and equivalence of interpreting/responding to the measures (whether respondents would interpret and respond to the stimuli in the same way).
Research has found differences in responding to survey questions between different racial/ethnic groups. Bachman and O’Malley (1984) found Blacks were more likely than Whites to select extreme response categories, particularly the positive end of “agree-disagree” scales. Hui and Triandis (1989) found Hispanics had a stronger tendency of selecting extreme responses on 5-point scales. Marin et al. (1992) also found Hispanics tended to choose the extreme responses and agree with a given item more often than Whites did, and the level of acculturation among Hispanics affected the level of extreme and acquiescent responses, with the more acculturated Hispanics choosing these types of responses less often.
Studies that include Asians and Asian Americans suggest that they are more likely to select the midpoints and avoid extreme responses on Likert scales. Lee et al. (2002) studied a group of Chinese, Japanese, and Americans recruited at ethnic or general supermarkets, and found Chinese and Japanese selected midpoints more often on items that involved admitting to a positive emotion. Chen et al. (1995) compared response styles between East Asian and North American students, and found students from the two collectivist cultures (Chinese and Japanese) demonstrated a greater preference for midpoints and less preference for extreme values than those from the individualist cultures, especially the U.S. students. In another study of college students, Grandy (1996) found Asian American students tended to endorse middle options and avoid extreme responses on a 5-point Likert scale more than Whites did.
Social scientists do not agree on why Asians or Asian Americans would prefer middle options and avoid extreme responses, but most believe it is culturally-related. East Asian cultures emphasize the need to fit in with others and avoid conflict in society. Hoy (1993) had referred to this “aversion to the spotlight” as “cultural shyness.” Johnson et al. (2005) discussed two cultural orientations: individualism and collectivism, suggesting that a middling response style should better fit the cultural norms and imperatives of persons living in collectivist cultures, since collectivism is associated with a greater emphasis on interpersonal harmony and a lesser emphasis on individual opinions (Triandis et al. 2001). Therefore, we would assume that people living or growing up in East Asian countries and those who share similar cultures tend to behave modestly and politely and maintain harmony in social relationships by avoiding expressing extreme opinions.
Methods
The analyses were based upon data collected from the 2006 Multi-Institutional Study of Leadership. Over 170,000 students in 53 institutions across the U.S. were selected, and 55,623 students responded to the survey, including 49,283 completes and 6,340 partials.
In this on-line survey, students were asked to indicate their agreement or disagreement with 68 statements on the Social Responsible Leadership Scales (SRLS), which were designed to measure leadership outcomes across eight critical values (5–11 items per each value). The 68 items, displayed in grids of 11–12 questions per screen, were randomly re-ordered so that items of same topics would not group all-together on the same screen. For each of the 68 SRLS items, students were asked to report on a fully labeled 5-point Likert scale, ranging from “Strongly Disagree” (1) to “Strongly Agree” (5) with “Neutral” (3) as the midpoint.
A series of new variables were generated to capture respondents’ reporting preferences to the 68 SRLS items, indicating whether they selected the extreme responses, the middle points, or other response options. The percentages of times a student selected middle, extreme or other response options were then calculated among the SRLS items they actually answered, as a summary of their overall response preferences.
The key independent variables include respondents’ self-reported race/ethnicity, which was re-coded into five categories – Asian/Asian American, White, Black, Hispanic and others, and respondents’ citizenship status – whether they were born in the U.S. or in foreign countries. Two other demographics at the individual level, gender and class standing, were used as basic control variables.
We included two cultural-related institutional factors – the region of the institution where students study and the diversity of the institution’s cultural environment, measured by the percentage of Asian/Asian American students and the percentage of foreign students in the total student population.
Hypothesis and Analytic Plan
The first step of the analysis is to determine whether Asian/Asian American students would select the middle points and avoid extreme responses more often on the Likert scale. The second goal is to offer some explanations to the differences (if we found any) between U.S. born Asian American students and those born in foreign countries.
In addition, we will summarize the logistic regression result for each of the 68 SRLS items. Institutional characteristics will also be included in the analytic model to explore whether these factors would influence response behaviors.
Results
Table 1 summarizes the weighted proportions and unweighted counts of the independent variables. The percentage of Asian/Asian American students in the total student population is about 5.9% on average for 53 institutions, ranging from 0.1% to 33.8%. The percentage of foreign students is about 4.9% on average, ranging from 0.2% to 14.3%.
We found Asian/Asian American students consistently reported lower values to the 68 SRLS items compared to other racial/ethnic groups (Table 2). This is attributed to the fact that Asian/Asian American students chose the middle option substantially more often and extreme options less often than the other groups (Figure 1).
Our second analytic goal is to explore whether the transition from Eastern to Western cultures would affect their response styles. We found that U.S. born Asian American students selected middle options significantly less often and the extreme responses more often than foreign born Asian students (Figure 2).
The overall response preferences for Asian/Asian American students were correlated with the density of the Asian/Asian American student population in the institution. The muiltivariate model indicated that Asian/Asian American students selected middle options more often and extreme options less often as the proportion of Asian/Asian American students among the student population increased.
We also conducted logistic regressions on each of the 68 SRLS items to examine students’ response preference on each single measure (Table 2). Among the 68 items, the differences in their response preferences between Asian and White students were significant for 53 items (78%); the differences between Asian and Black students were significant for 65 items (96%); the differences between Asian and Hispanic students were significant for 64 items (94%); and the differences between Asian and other racial/ethnic groups were significant for 66 items (97%). The consistent pattern suggests that Asian/Asian American students favor middle options and avoid extreme responses more often than any other racial/ethnic groups, no matter what the question was asked about.
Similarly, logistic regressions on 68 SRLS measures showed some evidence of the different response preferences between U.S. born Asian American students and foreign born Asian students; however, the evidence was not consistent across all the items. Although 87% of the items suggested the same direction as our hypothesis that U.S. born Asian American students were less likely to select the mid-points on the scale, only 47% were significant (Table 2).
There was also evidence that the response style of Asian/Asian American students was correlated with the diversity of the institution’s cultural environment. For 78% of the items, the odds of selecting middle options for Asian/Asian American students increased as the percentage of Asian students in the total student population increased.
Conclusion
We found respondents with different race/ethnicity had different response preferences when answering questions on Likert scales. Although our study focused on undergraduate students, the finding was consistent with previous research on the general population, which suggests that Asians or Asian Americans prefer middle options while Blacks and Hispanics prefer extreme responses. There was also evidence that U.S. born Asian American students differ in their response preferences from foreign born Asian students due to their cultural similarity to the westerners; however, the evidence was not consistent across all the Likert scale items.
We also found that response preferences of Asian/Asian American students were correlated with the cultural environment of the institution. The larger percentage of Asian/Asian American students in the institution, the more likely they would select the middle options and the less likely they would select the extreme responses. This finding supported our hypothesis that the distinctive response style of Asians/Asian Americans is culturally related. One possible explanation could be that Asian students would have more opportunities to socialize with other Asian students or be exposed to eastern cultures within institutions that have a larger Asian student population, thus their behaviors would be more like the easterners than the westerners.
Several limitations of the study should be acknowledged. Since the sample selection was implemented by each institution, we do not know the sampling rate applied by the institution to draw the simple random sample. All the analyses were conducted by ignoring the different sampling rate in each institution. However, nonresponse weights were applied to account for potential response errors, and possible clustering effects (homogeneity within each institution) were also taken into consideration in the analysis. Although we had statistically significant findings that supported our hypotheses, we were cautious to make any generalization to a larger population due to the lack of information about the sampling procedures.
The implications of the findings are more important than identifying the differences. It is crucial for survey researchers to be aware of this distinct response style when their studies are involved with asking Asians/Asian Americans and other racial/ethnic groups to report on Likert scales. False conclusion can easily be drawn in surveys like the Leadership study, where the consistent lower scores Asian/Asian American students obtained on the SRLS items was not a good evidence of their weak leadership capacities. Therefore, we suggest survey practitioners use Likert scales with more cautions.
More efforts should be made to minimize the differences in the response preferences between different racial/ethnic groups and make the items measured by the Likert scales more equivalent across people with different cultural backgrounds. We also suggest that survey researchers, especially those who are responsible for designing and testing questionnaires, further examine (1) whether other types of questions would better capture Asians/Asian Americans’ opinions than the Likert scale (2) and whether scales with more (or fewer) points or even points (no midpoint) would better capture their opinions than the 5-point scale.