Medical and social sciences routinely include participant race as a demographic predictor or correlate of measured outcomes. For example, racial differences have been reported for access to health services (Boscarino et al. 2005), as well as a variety of health outcomes (Adams and Boscarino 2005). The validity of these findings rests on the reliable reporting of participant race. However, emerging research has suggested that racial designations were not always reliable. Moreover, reliability of race reporting appeared more salient for some racial categories than for others.
This study used data from a national survey of children involved with the child welfare system to assess agreement on child’s race using child self-report, caregiver report, and administrative records, as represented by child welfare caseworker report. Level of agreement across three reporter types has never been examined. Our aim was to provide descriptive data on agreement among these three sources of child race information. The findings have implications for both survey methodology and social science research involving measurements of children’s race.
Data for this paper came from the National Survey of Child and Adolescent Well-being (NSCAW) sponsored by the Administration for Children and Families. NSCAW was the first nationally representative longitudinal study of children and families involved in the child welfare system. It was also the first to collect data directly from children and caregivers having contact with that system. We analyzed data from the Child Protective Service sample (n=5,501). This sample was drawn from children who had contact with the child welfare system within a 15-month period beginning October 1999.
NSCAW data were appropriate for examining reliability of race reporting across respondent types for several reasons. First, child’s race was included as a standard demographic variable and was asked of caregivers, caseworkers, and children at least six years old. Second, NSCAW race questions were patterned after the 2000 Census which for the first time allowed respondents to select multiple races. Caregivers and caseworkers were shown a card with the race categories listed and asked to pick one or more categories from the card. Caseworkers were encouraged to refer to the child’s administrative records during the interview. However, children ages six years and up were asked the open-ended question, “What race are you?”. The interviewer coded the race(s) corresponding to the child’s response.
Many children were unable to identify their own race. Almost one-third of children’s responses were recorded as “Don’t know” by the interviewer. Less than one percent of caregivers and caseworkers were unable to respond. As expected, the younger the child, the more likely the coded response to race was “Don’t know” (see Exhibit 1).
Exhibit 2 shows the distribution of child’s race as reported by caregivers, caseworkers and children. To be consistent across respondent types, we excluded “Don’t know” and “Refused” responses to race from our analyses. Because only children age six and older were asked their race, we also excluded caregiver and caseworker race reports for children under age six.
As seen in Exhibit 2, the percentages were much lower for the Asian, Native Hawaiian/Pacific Islander, American Indian, Other, and Multiple Race categories. When comparing the percentages for each race across the three respondent types, the largest differences were for the American Indian and Multiple Race categories. While only one percent of caseworkers reported the child as at least part American Indian, five percent of children and six percent of caregivers reported the child as being American Indian. Similar differences were found in the Multiple Race category.
Replicating methods used by Gomez et al. (2005) and West, Geiger, Greene, et al. (2005), we used “sensitivity” to measure the level of respondent agreement on child’s race. Among members of one respondent type who said yes to a given race, sensitivity indicates the percent of the second respondent type who also said yes to that race. Sensitivity was calculated for the following pairs (Respondent type 1/Respondent type 2):
Exhibit 3 shows sensitivity values for the three pairwise comparisons between the child, caregiver, and caseworker reports of child’s race. As reported in the literature, the highest agreement was seen for the White and Black categories for all three comparisons while the least agreement was for the American Indian and multiple race categories. Additionally, the agreement percentages were consistently higher for the child and caregiver comparisons (middle bars) and lowest, with the exception of Native Hawaiian/Pacific Islander, when the caregiver was compared to the caseworker.
Since American Indian had such a low level of sensitivity, especially for caseworkers agreeing with caregivers, we examined the combinations of race responses for all three respondent types when American Indian was selected. As seen in Exhibit 4, almost all caseworkers who reported children as American Indian either reported them as only American Indian or American Indian and White. The children and caregivers, on the other hand, were more likely to report the child as being American Indian in combinations with other races.
Since far more caregivers reported the child as being all or part American Indian, we examined the caseworker reported race for children for whom the child and caregiver included American Indian as the child’s race. As Exhibit 5 shows, for both children and caregivers reporting the child was at least part American Indian, more than half of the caseworkers reported those children as being White only.
By examining responses to the child race questions in NSCAW, this study identified several findings that add to our understanding of race reporting in surveys from multiple respondent types. First, young children had difficulty reporting their own race. Second, caregivers and children were more likely than caseworkers to report that a child was multiracial. Finally, caregivers and children were more likely than caseworkers to report a child was of American Indian descent. Our findings that children and caregivers were better able than caseworkers to identify children as multiracial and American Indian were consistent with prior research showing that adult self report of race was more reliable than administrative records (Gomez et al. 2005; West, Geiger, Greene, et al. 2005).
Our findings had several implications for survey researchers. First, the age at which a child might be “qualified” to answer questions about race and ethnicity should be an important consideration when developing survey instruments. Second, instrument developers should consider asking children who are old enough and can read to choose from a list of race categories instead of responding to an open-ended question. Finally, the use of race data that come from administrative records must be carefully scrutinized. Our findings supported prior research that concluded minority races, especially American Indian and multi-racial, were under-reported in administrative data.
Areas of Further Study
Further research is needed to better understand the age at which children are able to identify the standard race categories and to examine consistency of child and caregiver reports of race over time. Explanatory factors contributing to inconsistency of race reports across respondent types should also be examined. The findings of this study serve as a reminder that race, when measured as a variable in social science, should be given careful consideration. Researchers must take care to ensure that instrumentation and methods for assessing race are valid and reliable. Above all, researchers must justify the measurement of race as an explanatory variable (Canadian Medical Association 2000; Stephen 2006).