Examining the Association between Cultural Environments and Survey Nonresponse

Survey response rates have decreased over time, particularly in the developed nations of North America and Western Europe (Battaglia et al. 2008; Curtin, Presser, and Singer 2005; de Leeuw, E. and W. de Heer 2002). Since nonresponse can affect the quality of survey data, it has become an overarching research topic among survey methodologists (cf. Groves et al. 2002). Groves and colleagues (2009) provide a framework for survey participation, suggesting that nonresponse stems from a variety of sources, including social environmental conditions, characteristics of respondents (or households), interviewers, respondent-interviewer interactions, and survey design features.

Although these sources of nonresponse are diverse, much of the empirical research has emphasized individual predispositions (e.g., interviewer or respondent) and/or study design features as nonresponse predictors, disregarding higher-level social environmental factors that also might be associated with survey participation. Ironically, one of the most robust predictors of survey nonresponse that has been identified is spatial in nature: urbanicity (Demaio 1980; Fitzgerald and Fuller 1982; Goyder, Lock, and McNair 1992; Groves and Couper 1998; Kim et al. 2007; Smith 1983). In contrast, another social environmental variable, culture, has not to our knowledge been directly investigated as a potential predictor of survey nonresponse. The purpose of this paper is to report an empirical assessment of the associations between several cultural dimensions and survey nonresponse.

Dimensions of Culture

Culture has been conceived in many different ways, and literally hundreds of definitions now exist (Borofsky et al. 2001; Kroeber and Kluckhohm 1966). Triandis (2007, pp. 63–64) has observed that virtually all conceptualizations have in common the belief that culture is a product of “adaptive interactions between humans and environments,” that it consists of shared elements, such as language, norms, values, beliefs, expectations, and life experiences, and that these elements are passed down from generation to generation.

These conceptualizations ironically stand in contrast to the approach commonly used to operationalize culture in empirical research, one that emphasizes objective biological traits (e.g., race or ethnicity) or residential status (e.g., nation or region), rather than subjective cultural elements. Considerable progress has been made in recent decades towards “unpackaging” these crude measures of culture by identifying various values and orientations that may prove useful for interpreting similarities and differences in social behavior across broad cultural groups. Some of the more prominent frameworks have been outlined by Hofstede (1980, 2001), Schwartz (2004), Inglehart (1997), Triandis (1996), and Trompenaars and Hampden-Turner (1998).

Perhaps the most widely recognized of these was first proposed by Hofstede (1980), who initially identified four dimensions that were believed to vary systematically across national cultures. These dimensions were labeled individualism-collectivism, power distance, uncertainty avoidance, and masculine vs. feminine cultures. Briefly, individualism was posited to represent a self-enhancement orientation, whereas collectivism represented a group-enhancement perspective. According to Triandis (1995), persons from individualist cultures behave as independent actors who emphasize their personal interests. Persons within collectivist cultures, in contrast, have more interdependent social relationships and emphasize membership and duties within well-defined social groups to a much greater degree than do those within individualist cultures. Within collectivist societies, much sharper distinctions also are made between members of one’s own in-group vs. members of out-groups; these distinctions are believed to influence behavior to a much greater degree than within individualist societies (Gudykunst 1997).

Hofstede’s dimension of power distance addresses the degree to which the unequal distribution of power within a society is accepted and considered legitimate by that society’s less powerful members (G. Hofstede and Bond 1984). This dimension is largely concerned with the nature of relationships between persons of different social status. High power distance cultures are believed to be more authoritarian and to emphasize conformity. In contrast, low power distance cultures are viewed as less authoritarian and as emphasizing independence.

Masculine vs. feminine orientations identify the degree of nurturance within a society. Assertiveness, competition, and material success are encouraged and highly valued within masculine societies. In contrast, a much greater emphasis is placed on cooperation and quality of life considerations, and competition and personal success are less important, in feminine societies (G. Hofstede 1998). Sharp occupational stratification by gender commonly is found in masculine cultures, whereas there is less differentiation of occupational opportunities within feminine cultures.

Uncertainty avoidance addresses the degree to which uncertainty and ambiguity can be tolerated. Consensus is highly valued within cultures high in uncertainty avoidance. These cultures generally have more formal rules, laws, and regulations, and there is less tolerance of deviance (Gudykunst and Kim 1997). In the second edition of Culture’s Consequences, Hofstede (2001) added another cultural value to his framework, which he labeled long vs. short-term orientation. Long-term orientations, common in China and other Eastern countries, value persistence, respect for traditions, long-term commitments, and thrift (Chinese Culture Connection 1987). In contrast, short-term orientations, which are more commonly found in Western societies, do not focus as strongly on these values and thus are able to change and adapt more rapidly.

Culture and Survey Nonresponse

Although considerable work has explored racial/ethnic correlates of nonresponse within the U.S. and other nations and cross-national comparisons of nonresponse behaviors between countries also have been reported (cf., Johnson et al. 2002), there is to our knowledge no available research that investigates the association between cultural orientations and survey nonresponse. In this paper, we examine cultural dimensions rather than making racial/ethnic comparisons. We examine three of Hofstede’s cultural dimensions as potential correlates of nonresponse: Individualism-Collectivism (IC), Power Distance (PD), and Masculinity-Femininity (MF). This empirical study explores whether these cultural contexts are associated with nonresponse in a telephone survey conducted statewide in Illinois.

Individualism-Collectivism

As discussed earlier, persons in collectivistic cultures clearly differentiate in-groups from out-groups, making a contribution only to in-groups. Those embedded in more individualistic social environments, however, tend to help and cooperate with strangers regardless of in-group/out-group distinctions. Previous empirical studies have found that IC is strongly correlated with helping behaviors. Kemmelmeire and his colleagues (2006) revealed that individualistic cultures were positively correlated with prosocial behaviors such as volunteering and helping strangers, and Conway et al. (2001) reported collectivism to be negatively associated with helping behaviors. Because it seems reasonable to assume that persons willing to help others and to volunteer also would be more likely to cooperate with survey requests, we hypothesize the following:

H₁: Persons embedded in individualistic cultures may be more likely than those in collectivistic cultures to participate in surveys.

Power Distance

In high power distance cultures, inequality exists between superiors and subordinates, who expect to be told what to do (Gudykunst 1997). Research also indicates that people in low power distance cultures are more cooperative than their counterparts (Negandhi and Prasad 1975). Hence, our second hypothesis states:

H₂: Persons embedded in low power distance cultures may be more likely than those in high power distance cultures to participate in surveys.

Masculinity-Femininity

As suggested above, this dimension focuses on the degree to which there is a division of social roles between men and women. People within masculine social environments tend to value things associated with power, such as performance and success; those within more feminine environments are more likely to have values reflective of nurturance, such as helping others and service to the community (Jung 2002). Because survey participation requires respondents to give their time, people living in feminine social environments can be expected to be more cooperative than those within masculine ones. We thus hypothesize that

H₃: Persons embedded within more feminine environments may be more likely to participate in surveys, compared to persons embedded within more masculine environments.

We note that each of these hypotheses is primarily concerned with survey nonresponse in general and with survey refusals in particular. We suspect, though, that the same cultural orientations that might be associated with willingness to participate in a survey might also encourage individuals to be more likely in general to assist in helping interviewers contact other household members who might have been randomly selected to participate in a survey interview.

Methods

Data

We used two data sources for analysis: an RDD telephone survey data set and aggregate data from the 2000 Census. The survey data were derived from a University of Illinois at Chicago Survey Research Laboratory RDD telephone survey administered to a random sample of Illinois residents in 2003 (Johnson et al. 2004). In total, 4,155 interviews were completed. The response rate was estimated to be 32.7%, based on AAPOR response rate formula RR3 (American Association for Public Opinion Research 2009). The target population for this survey were Illinois residents age 16 years or older in noninstitutionalized households. We obtained ZIP code level data from the 2000 Census then matched the survey responses with these data for each of the 20,774 telephone numbers included in our sample.

Measures

Three binary nonresponse indicators were examined: nonresponse, noncontact, and refusal to participate. For details on variable construction, see Johnson et al. (2006).

ZIP code level indices were constructed using 2000 Census data to represent each of the three cultural dimensions hypothesized to be associated with nonresponse: collectivism, power distance, and femininity. For each, three Census items were identified that were conceptually linked to the underlying cultural construct. The items associated with each were standardized and summed to form single indicators of each construct. Acceptable variables for constructing indices of Hofstede’s other two cultural dimensions were not available in these data.

As shown in Table 1, collectivism was measured using the following three items: percentage of multigenerational households, ratio of persons carpooling to work to those driving alone, and percentage of minorities (see Vandello and Cohen 1999). The standardized alpha for the three items was 0.60. Higher scores represent more collectivist orientations.

The power distance orientation was measured with three items: the GINI index ^[1], percentage of adults with a high school degree or more, and percentage of persons employed in professional, technological, and managerial occupations (see Hofstede 1980). The standardized alpha for this index was 0.80. Higher scores represent greater levels of power distance.

The feminine cultural dimension also was measured using three items: percentage of females in the labor force, percentage of females with professional or doctoral degrees, and percentage of females in professional, technological, and managerial occupations (standardized alpha=0.83). Higher values represented more feminine cultural orientations.

In addition, two ZIP code level covariates were included in these analyses: urbanicity and community economic status. Urbanicity was measured as population density, which is calculated as the total population divided by square miles within each ZIP code. As the distribution of this value was highly skewed (mean: 246, standard deviation: 1680), we used the log transformation. Community economic status was measured as ZIP code level 1999 median household income, which was also log transformed. These covariates were included for different reasons. It was necessary to control for urbanicity, given its documented association with survey nonresponse in prior research (Demaio 1980; Fitzgerald and Fuller 1982; Goyder, Lock, and McNair 1992; Groves and Couper 1998; Kim et al. 2007; Smith 1983). Community economic status was included as a covariate because affluence is known to be associated with several cultural dimensions and may serve as a confounder if not controlled (G. H. Hofstede 1980).

Analysis

We employed multilevel logistic regression models to test the hypotheses. The models adjusted for clustering of outcomes measured at the person level (level 1) within each Zip code (level 2). These models take the following basic equation:

$$\begin{equation}\text{Level 1: log}\bigg[\frac{P}{1-P}\bigg]=\beta_{0}\end{equation}$$

$$\text{Level 2: }\beta_0 = \gamma_{00}+\gamma_{01}\text{(Collectivism)}+\gamma_{02}\text{(Power Distance)}+\\ \gamma_{03}\text{(Femininity)}+\gamma_{04}\text{(Urbanicity)}+\gamma_{05}\text{(Economic status)}+\mu_0$$

Findings

We first examined a general nonresponse model (see model 1 in Table 2) in which the three cultural indices, along with the urbanicity covariate, were included as independent variables. In this model, several variables were associated with survey nonresponse. The signs of the regression coefficients for collectivism, power distance and femininity were each consistent with our hypotheses. Hence, the model provided preliminary confirmation for H1, that the likelihood of nonresponse would be greater among persons embedded within more collectivistic (or less individualistic) social environments. Consistent with H2, the likelihood of nonresponse was also greater among persons living within more power distant cultural environments. In contrast, those embedded in more feminine cultural environments were more likely to participate in a telephone survey, compared to those in less feminine environments, providing preliminary confirmation for H3. However, this relationship was only marginally significant. The impact of urbanicity on nonresponse also was highly significant (p<0.001), consistent with previous findings suggesting that persons living in urban areas are less likely than their counterparts living in more rural environments to participate in telephone surveys.

The second equation in Table 2 examined the association between respondent refusals and the ZIP code level measures. Two of the cultural dimensions, collectivist and feminine orientations, were not significantly associated with telephone survey refusal. Power-distance, though, did remain a significant predictor (p<0.05) in this model: persons embedded within higher power distance cultures were more likely to refuse an invitation to participate in the telephone survey. Urbanicity was only marginally associated with survey refusal.

For noncontact, the third model presented in Table 2, all three cultural dimensions were found to be significant predictors. Noncontact was found to be greater in collectivist and more power distant cultural environments and lower in more feminine environments. Urbanicity was found to be strongly associated with noncontact in this model, confirming that it is more difficult for telephone interviewers to contact potential respondents residing in urban areas.

Because many cultural values are believed to be associated with socioeconomic status (G. H. Hofstede 2001), we next introduced an additional control for community economic status into these models (see Table 3). Compared to the earlier models in Table 2, the effects of most cultural dimensions were considerably attenuated once median household income was controlled. Supportive of H1, persons in collectivist environments continued to be less likely to be respondents in the telephone survey (first model in Table 3). H3 also received partial support, as persons embedded in feminine environments were borderline significantly more likely to be respondents. Urbanicity remained strongly associated with nonresponse.

The one measure associated with refusals (second model in Table 3) was urbanicity, and this relationship was only borderline significant. As with nonresponse, both collectivism and urbanity were positively and significantly associated with noncontact (third model in Table 3). Although it clearly attenuated the effects of other variables in these models, community economic status was itself not independently associated with any of the nonresponse measures examined in Table 3.

Discussion

It is generally understood that survey nonresponse is influenced to some degree by the social environments in which people are embedded (Groves and Couper 1998). However, few studies have attempted to document a relationship between culture at a spatial level and survey nonresponse. We empirically tested whether the cultural environments within which individuals are embedded might be associated with three measures of nonresponse in a statewide telephone survey. Our findings offer evidence consistent with a model that suggests that survey nonresponse and noncontact may be greater among persons living within more collectivistic geographic areas—that is, areas in which there is likely to be greater recognition of and differentiation between in-groups and out-groups and in which there is a greater likelihood that perceived representatives of each of these groups will be treated differently. Findings also provide some evidence that survey nonresponse may be lower among persons living within geographic areas that are more likely to emphasize community service, nurturance, and providing help to others (i.e., those with more feminine orientations).

It is also noteworthy that the associations between cultural measures and survey nonresponse were considerably stronger in models that did not control for community economic status (Table 2). Hofstede (1980, 2001) previously has discussed the strong associations between cultural dimensions, individualism/collectivism, and power distance in particular, and spatial indicators of wealth and affluence. Failure to account for these relationships clearly would have led to far stronger conclusions regarding the nature of the culture-nonresponse relationship than are warranted.

Another important finding was the general lack of association between these cultural dimensions and refusal to participate in the telephone survey. We note that the three hypotheses proposed earlier were each based on cultural variability in willingness to help or cooperate. Although many would assume, as have we, that general helpfulness and refusal to cooperate approximate the opposite ends of a continuum, it may be that these are in fact less-than-perfectly -aligned concepts. If so, the cultural dimensions associated with each could well vary and might account for the null findings observed here. Nonetheless, the null findings reported here do not support our hypotheses.

Also, given that several alternative frameworks for conceptualizing culture are available (Inglehart 1997; Schwartz 2004; Triandis 1996; Trompenaars and Hampden-Turner 1998), it is important to recognize that there is no consensus on the most appropriate approach to measuring and assessing cultural dimensions. One critique, among many, of Hofstede’s model has been the concern that his work overgeneralizes cultural dimensions at the national level and ignores what in many cases is likely to be considerable heterogeneity at the subnational level (McSweeney 2002). This paper has demonstrated the usefulness of his model at a much smaller level of aggregation—ZIP codes within a U.S. state—for predicting an important individual level behavior—survey nonresponse. In doing so, it contributes to prior evidence regarding the heterogeneity of Hofstede’s dimensions within nations and their usefulness for research purposes (Vandello and Cohen 1999).

At the same time, the cultural indices employed in this paper are, in our opinion, far from adequate. An important challenge for this analysis was to identify and access useful indicators at an appropriate level of aggregation. The advantage of developing cultural indices at higher levels is the greater availability of useful indicators from multiple sources. Vandello and Cohen (1999), for example, were able to develop a comprehensive index of individualism/collectivism at the state level, at which numerous relevant indicators, from a variety of data sources, were available. Other levels of aggregation (e.g., counties) might be more useful but in some cases could be too socially heterogeneous. Cook County, Illinois, may be such an example in which the internal range of cultural, economic, and demographic variability is so great that it cannot be adequately represented by summary county-level indicators. Ultimately, we feel that the Census-tract level might prove to be a more optimal one at which to address the research questions of interest here. We examined the ZIP code level of aggregation in this paper, though, because it was the lowest level at which both Census data and telephone area code/exchanges could be linked. The question of appropriate levels of aggregation at which to construct the most meaningful measures of cultural dimensions remains unanswered. Solutions to these problems will be necessary if we are to make further progress toward evaluating the role that cultural environments play in the survey nonresponse process.

Acknowledgements

The authors wish to thank Lisa Kelly-Wilson for her careful and cheerful editing of this paper.

Preparation of this paper was supported by Grant No. 1R01HD053636-01A1 from the National Institutes of Health and Grant No. 0648539 from the National Science Foundation.