National child health surveys often use “most knowledgeable adult” (MKA) as the selection rule for identifying an adult proxy reporter for a child’s health information (CDC 2011; Warren and Cunningham 2003). By definition, MKAs should provide the most accurate data about a child, and data about MKAs can be individually attached to that child. However, MKA selection requires additional callbacks to reach the household’s single qualified adult, potentially increasing both survey costs and survey error (in the form of unit nonresponse and nonresponse bias). For these and other reasons, including the increased sharing of child-raising responsibilities between spouses (Bianchi, Robinson, and Milkie 2007), alternative selection rules for child health surveys may result in lower survey costs and error.
We used an alternative proxy selection rule for a child health survey and then compared the responses of two types of proxy reporters: those who said they were the adult most knowledgeable about the selected child versus those who said they were sufficiently knowledgeable to answer questions about the child’s health. We found little to no difference in response quality, but nearly doubled the number of eligible respondents in multi-adult households. Because of this, and the lack of literature on the value of MKA selection, we recommend further testing of selection rules for child health surveys to challenge the orthodoxy of “most knowledgeable adult”.
New York City Department of Health and Mental Hygiene’s 2009 Child Health Survey was a citywide, landline telephone survey with interviewing in both English and Spanish of 3,002 households with one or more children under 13 years of age (Lundy De La Cruz, Goldmann, and Driver 2011). Approximately one-third of the interviews (946) were conducted after a randomly-selected adult in the household completed a 25-minute survey on the adult’s health conditions and behaviors. The remaining interviews were obtained from households independently contacted for the Child Health Survey. Once a child was randomly selected (if multiple children lived in the household) as the focus of the survey, the interview was conducted with a non-random adult who answered “yes” to the question, “Do you know (name or description of the selected child) well enough to answer questions about (his/her) health, (his/her) doctor visits, what kinds of food (he/she) eats, and (his/her) general activities?” Near the end of the survey, respondents compared their level of knowledge of the focus child to that of other adults living in the household. Respondents who said no other adult in the household knew as much about the focus child were termed a “most knowledgeable adult” (MKA) and all other respondents were designated a “sufficiently knowledgeable adult” (SKA). Full question wording is available in the Appendix.
Analysis was limited to households with more than one adult (i.e., those that would be affected by a proxy selection rule). We compared the responses of those who would have definitely been interviewed using the “most knowledgeable” selection rule (i.e., a MKA) to those from the adult who not have been selected as the MKA (i.e., a SKA). We excluded those where relative knowledge levels were unknown. Independent validation of responses was not available. Instead, we compared MKA and SKA respondents on the average number of reported chronic conditions and/or developmental delays, the average levels of item nonresponse, and means and variance across continuous measures.
Chi-square tests were used to compare the demographics of MKA vs. SKA respondents. Independent sample t-tests were used to compare the mean percent of reported problems among a maximum of 26 chronic conditions and developmental concerns and the mean percent of “don’t know”, “not sure”, or “refused” responses over a maximum of 106 survey questions. The number of questions asked was dependent on the child’s age and not all items were asked for every child. MKA and SKA responses to 16 continuous variables were also compared using t-tests for difference of means and Levene’s test for homogeneity of variance. All statistical tests were two-tailed, with the level of significance set at α = 0.05.
When calculating the total number of SKAs in multi-adult households, we made two conservative assumptions. First, when the respondent was the most knowledgeable adult in a two-adult household, we assumed the other adult was not sufficiently knowledgeable. When the respondent was the most knowledgeable adult in a household with three or more adults, we assumed that no more than one other adult was sufficiently knowledgeable.
Most interviewed households (2,499 of 3,002, or 83 percent) had more than one adult. Among multi-adult households, only 22 percent (537 of 2,474) of adult proxy reporters said they knew more about the selected child than any other adult in the household (i.e., were the MKA). The 25 remaining respondents in multi-adult households who did not provide information on relative knowledge were excluded from all further analyses.
Compared to SKAs, MKA respondents were significantly more likely to be female, under 40 years of age, live in households with a total income below 200 percent of the Federal Poverty Level, and rent rather than own their residence (Table 1). There were no significant differences between SKAs and MKAs for either race/ethnicity or nativity (US-born, foreign-born).
MKAs reported significantly more chronic conditions and/or developmental delays than did SKAs, but this difference was small (an average of 7.3 percent vs. 4.7 percent of asked conditions). Item nonresponse – answers of “Don’t know”, “Not sure”, or “Refused” – was low (0.8 percent on average for both MKAs and SKAs) and did not differ significantly. For 14 of the 16 continuous measures, means were statistically equivalent and variance statistically homogenous.
In multi-adult households, almost twice as many adults were eligible to provide proxy reports using an alternative selection rule: 4,833 adults vs. 2,499 using MKA (Figure 1). Overall, including both single- and multi-adult households, SKA selection increased the number of potentially eligible adults by 78 percent.
There are two substantial limitations to our study. The first is the lack of random assignment of households to using MKA versus the alternative selection rule. This prevented a direct comparison of survey costs and of response quality. Second, we lacked access to independent measures of the child’s health that would allow direct comparison of response quality between MKAs and SKAs.
The commonly-used selection rule of MKA limits the number of adults eligible to provide proxy information in a child health survey to one per household. In our study of an alternative selection rule, tested with a large, population-based sample, only 22 percent of respondents in multi-adult households said they were the most knowledgeable about the selected child. However, we found little to no difference between MKAs and other respondents in reporting chronic conditions and developmental delays of a randomly-selected child in the household, in levels of item nonresponse, and in response variance.
SKA selection substantially increased the pool of eligible proxy respondents, which is likely to reduce both survey cost and unit nonresponse. Callbacks needed to reach the single MKA may be unnecessary if another eligible proxy respondent is present. Additionally, refusal by the MKA does not necessarily prevent successful data collection from a different eligible proxy respondent, reducing potential bias from unit nonresponse. SKA selection is also well-suited for follow-up studies since the randomly-selected interviewed adult in an eligible household is more likely to be either the MKA or a SKA than he/she is to be the lone MKA.
While there has been a great deal of research on proxy knowledge, few, if any, studies have compared responses of differently-selected proxies. Given the small apparent impact on data quality and the potential gains in efficiency and response rates, future testing of alternative selection rule for proxy reporters in child health surveys is warranted, preferably with random assignment.
Appendix – Question Wording from NYC Department of Health and Mental Hygiene’s 2009 Child Community Health Survey
A. Question confirming that an adult is sufficiently knowledgeable to be the proxy respondent for a randomly selected child in the household
Q1. Do you know (CHILD) well enough to answer questions about (his/her) health, (his/her) doctor visits, what kinds of food (he/she) eats, and (his/her) general activities?
B. Questions comparing the proxy respondent’s level of knowledge about the randomly selected child to that of other adults in the household
Q1. How many OTHER adults, 18 and older, live in your household? _____
If one other one adult in HH (Q1 = 1)
Q2. Compared to you, when answering questions about (CHILD)’s health, would you say the other adult in the household is as equally knowledgeable as you, more knowledgeable than you, or less knowledgeable than you?
1 EQUALLY KNOWLEDGEABLE
2 MORE KNOWLEDGEABLE
3 LESS KNOWLEDGEABLE –> Respondent is the most knowledgeable adult (MKA)
7 DON’T KNOW
If two or more other adults in HH (Q1 = 2 or more adults)
Q3. Compared to you, when answering questions about (CHILD)’s health, would you say that any of the other adults in the household are equally or more knowledgeable than you?
7 DON’T KNOW
ASK IF Q3 = 1 (There are other adults in the household equally or more knowledgeable)
Q4. How many of these other adults are equally or more knowledgeable than you about (CHILD)’s health?
____ (RANGE 1 – TOTAL NUMBER OF OTHER ADULTS IN HH: Q1)
77 DON’T KNOW
ASK IF Q3 =2 (There are no other adults in the household equally or more knowledgeable)
Q5. Just to confirm, are you saying that there are no other adults in the household that are equally or more knowledgeable than you about (CHILD)’s health?
1 YES –> Respondent is the most knowledgeable adult (MKA)
7 DON’T KNOW