Effects of Unfolding Brackets on the Quality of Income Data in a Telephone Survey

In most sample surveys, questions on income have high levels of nonresponse. One method of reducing nonresponse to income questions is to ask follow up questions of initial nonrespondents by using a series of unfolding bracket questions. In this approach, if the respondent does not answer an initial question on income, a follow up question is asked in which the respondent is asked if the amount in question, such as household income, is above or below a rounded figure, such as $30,000. The respondent can then be asked a series of similarly structured questions so that a range for the amount in question can be determined. This method has been found to be effective in reducing uncertainty about measures of income and wealth on other surveys (Juster and Smith 1997; Pleis, Dahlhamer, and Meyer 2006).

It is not known whether the unfolding bracket approach can be used for an initial question on income rather than as a follow-up for nonrespondents to the initial question. In this analysis, I compare the distributions of responses to the income questions used in the 1994 and 1995 Behavioral Risk Factor Surveillance System (BRFSS) surveys. The BRFSS is a state-based system of surveys designed to provide estimates on health behaviors for adults 18 and older in the United States. The survey uses a repeated, cross-sectional design and has been carried out as a series of monthly telephone surveys since 1984 by state level health departments with methodological and technical assistance from the Centers for Disease Control (CDC).

Prior to 1995, the BRFSS asked a single income question using categorical responses – “Which of the following categories best describes your annual household income from all sources?” In 1995, survey designers replaced the single income question with a series of income questions consisting of unfolding brackets. In the revised income questions, the respondent is first asked “Is your annual household income from all sources less than $25,000?” with response categories of “yes” and “no”. If the response to this question is “yes”, the next question asks if income is above or below $20,000. If the response to the first income question was “no”, the next question asks if income is above $35,000 and so forth. The analysis here examines if there were 1) changes in the percentages of those who did not respond to the income questions and 2) changes in the responses among those who chose to answer the income questions.

Findings

Table 1 presents weighted estimates and confidence intervals for proportions of the sample that did not provide income data from the 1993, 1994, 1995 and 1996 BRFSS public use data files ^[1]. The change in the percentage of cases with missing data on income between 1995 and 1996 does not appear to be particularly large, especially when compared with changes that took place between 1993 and 1994 and between 1995 and 1996. The difference in the percentages with missing income between 1994 and 1995 is not statistically significant (Log-likelihood χ2=3.432, p=0.064). Thus, on the surface, it appears that moving from a single income question in 1994 to an income question with unfolding brackets in 1995 did not reduce the nonresponse to this item.

However, if we distinguish between two types of income nonresponse, “Don’t Know” and “Refused” responses, we see different patterns between 1994 and 1995 with the percentage of “Don’t Know” responses increasing between the two years and the percentage of “Refused” responses decreasing. Yan et al. (2006) show that income nonresponse rates on the Survey of Consumer Attitudes increased throughout the 1990s. It may be that the change in the income item increased the percentage of “Don’t Know” responses but there may also have been a trend of increasing “Don’t Know” responses to income questions. The questionnaire change may also have reduced the percentages of respondents refusing the income item between 1994 and 1995. If there had been a general trend of more respondents refusing to answer income questions, the differences between 1994 and 1995 on percentages refusing may be understated.

Table 2 shows weighted estimates and 95 percent confidence intervals of family income from the 1993–1996 BRFSS surveys. Except for the category of $15,000 to less than $20,000, all differences in percentages between 1994 and 1995 are statistically significant at the 0.01 level. The difference in the category “Less than $10,000” is particularly striking with the percentage reporting in this category dropping from 16.9 percent in 1994 to 7.5 percent in 1995. This suggests that the use of unfolding brackets may have countered tendencies by respondents to surveys prior to 1995 to respond with the first category read.

There were no major changes in the survey’s methodology between 1994 and 1995 that could reasonably account for these differences. The items preceding the income questions in both years were identical, which would rule out changes in question context as an explanation. In both years, the same 50 states were covered except that Rhode Island was not covered in 1994 and the District of Columbia was not covered in 1995. The median state level response rates were 69.9 percent in 1994 and 68.5 percent in 1995. In addition, as shown in Table 3, trends in the distribution of household income from the 1993–1996 Current Population Survey (CPS) do not reveal similar dramatic changes between 1994 and 1995.

Finally, I compared crosstabulations of income with employment status (employed; unemployed; not in the labor force) separately for the 1994 and 1995 surveys. If income is associated with employment, we might expect lower percentages of those being employed (and higher percentages of unemployment and not being in the labor force) among the lower income categories and that this pattern should be more pronounced in 1995 than in 1994 if the income data in 1995 are measured with less error than in 1994. In general, I found this to be the case. For example, in the 1994 survey, 40.7 percent of those in the “Less than $10,000” income category were employed while 46.7 percent of those in the category “$10,000 to less than $15,000” were employed – a difference of 6 percentage points. In the 1995 survey, with the unfolding bracket sequence, the difference is almost 12 percentage points (26.9 percent in the Less than $10,000” category vs. 38.6 percent in the “$10,000 to less than $15,000” category”.

Conclusion

Overall, the introduction of the unfolding bracket technique for the initial income question in the 1995 BRFSS may have reduced nonresponse to the income question, particularly for refusals but this is difficult to determine in the face of a potential trend of increasing nonresponse to income items. In addition, the change to the unfolding bracket income questions appears to have affected estimates of the distribution of income between 1994 and 1995. In particular, the percentage reporting an income of less than $10,000 in 1995 is much lower in 1994, possibly due to offsetting respondent tendencies to respond to the first category mentioned on a telephone survey.

While the use of unfolding brackets can reduce measurement error by reducing the chances that respondents will immediately select the first category they hear, the method can introduce measurement errors from two other sources. First, the entry point, or initial category chosen for an unfolding bracket question sequence can serve as an anchoring point for the respondent such that a higher anchoring point may produce a higher frequency of responses in the higher categories, producing an anchoring or entry-point bias. Second, since the unfolding bracket items are asked with “yes/no” response categories, bias due to acquiescence or “yea-saying” may result. In research on responses to questions about consumption, Van Soest and Hurd (2004) found evidence of large biases due to acquiescence but little evidence of anchoring bias after accounting for bias due to acquiescence. For future research, it may be possible to use the BRFSS data to see if BRFSS estimates on income are affected by anchoring biases and “yea-saying”.