Complementary or Contradictory? Testing Informed Consent Materials Using Surveys, Interviews, and Intercept Testing

Introduction

The U.S. Bureau of Labor Statistics’ (BLS) National Longitudinal Surveys are a set of surveys designed to gather information about respondents at multiple points in time on topics including labor market activities and significant life events. Respondents are interviewed starting in their youth through late adulthood. Two cohorts, originating in 1979 and 1997, are ongoing, with a new cohort launching in the coming years (U.S. Bureau of Labor Statistics 2025).

One important but understudied topic in survey methodological research is informed consent procedures, especially with youth respondents. Informed consent materials should clearly state the study purpose, length and duration of time required, risk/benefits, privacy protections, incentives, and whether participation is voluntary (U.S. Health and Human Services 2025). However, survey respondents tend not to read informed consent documents closely, lack comprehension of content, and do not retain the content of the forms well (O’Lonergan and Forster-Harwood 2011). With youth respondents, this can lead to noncompliance with study requirements and study attrition (Antal et al. 2017). Thus, it is crucial to ensure informed consent materials are clear and easy to understand for both caregivers and youth.

In past National Longitudinal Survey of Youth (NLSY) cohorts, interviewers walked respondents through the informed consent materials, and caregivers signed a form to give consent for their children (typically ages 11-17) to participate, while youth were provided with an assent form to agree to participate. BLS reviewed and assessed the informed consent materials to determine if there was a need to improve clarity or make other updates to ensure the forms were easy to understand and would facilitate continued participation in the NLSY throughout adulthood. Pretesting was conducted to assess multiple aspects of the informed consent process, including:

• Do caregivers and youth comprehend the information in the consent and assent forms as intended? For youth, are there differences in comprehension by age?

• Do caregivers and youth think they would be willing to participate in the NLSY? What concerns, if any, do they have? Do caregiver concerns vary by the age of the youth?

We used three methods to assess comprehension and willingness to participate:

• Unmoderated web survey of parents with youth ages 11-17

• Moderated family interviews with adults and youth ages 11-17

• Intercept testing (approaching participants in public settings) with youth ages 11-17

These methods were chosen to complement one another. For example, an unmoderated web survey could yield paradata on time spent reading the consent forms, and quantitative measures on comprehension and willingness to participate in the survey. Web surveys also facilitate a large, more geographically dispersed sample than would have been possible using only qualitative methods. Moderated, in-depth interviews with families (parents and children) provide richer detail, more context, and a fuller picture of the benefits and drawbacks of the consent and assent forms, specific comprehension issues with the forms, and any concerns about participating. Interviewers could ask follow-up questions for more detail on any confusion or concerns with the forms. Intercept testing with youth provides an opportunity to obtain quick and honest feedback from youth, while minimizing the risk of parental influence (Horm et al. 1996). By combining these three methods, we anticipated uncovering additional findings that would not have been possible had we used one of these methods alone.

Overview of Methods

For the unmoderated web survey, conducted in July 2024, 395 parents with youth in the age groups of interest (11-14 and 15-17)^[1] were recruited from a web panel (Hartman et al. 2023). Participants were asked to imagine that their youth were sampled for the NLSY and given a brief description of the survey. After reading the consent form, they were asked questions to assess comprehension and willingness to participate in the survey. Participants^[2] were paid $2.50 for completing the 15-minute survey.

Family interviews were held remotely (via Microsoft Teams). Participants were recruited through word-of-mouth and snowball sampling. A mix of parents of youth ages 11-14 and ages 15-17 were recruited^[3]. During the 60-minute session, participants read the appropriate consent or assent form individually, and marked passages they found helpful or unclear. During the debriefing, they discussed their overall understanding of the form’s content and whether they would participate in the study. Adults received a $50 gift card after completing the interview.

Finally, we conducted intercept testing with 31 youth^[4] at two schools and a public library. The study lasted under 10 minutes, and participants were recruited via word-of-mouth. Youth were presented with a brief description of the study, then asked to read the form, and highlight passages they found clear versus unclear. Afterward, they answered follow-up questions via a paper questionnaire to assess comprehension of the form and willingness to participate in the NLSY. Upon completion, youth received a non-monetary incentive (e.g., highlighter, keychain, candy) of their choice.

Results

Comprehension

Web Study

Participants spent a median of 90.1 seconds (range 3.2 to 1652.7) on the page with the informed consent materials, showing the variability in which people attend to informed consent language in a web survey context^[5]. Participants were asked a series of questions to assess their comprehension of the informed consent materials. Most (73.0%) correctly reported that the survey would take 60 to 90 minutes to complete, and 83.0% understood their child could skip any of the questions. Almost all (97.2%) indicated that their child would be able to withdraw from the study any time and would receive an incentive (96.2%).

In contrast, slightly over half (53.2%) correctly understood that the survey would occur multiple times, while 40.5% indicated that the survey would occur one-time only; 6.3% were not sure. Only 39.0% correctly reported that the caregiver, in addition to the child, would receive an incentive for participating; however, those who spent above the median time on the page with the consent form had better recall that youth and caregivers would receive incentives, respectively (X²(2,395)= 11.3, p = .004; X²(2,393)= 16.6, p < .001).

To assess participants’ understanding of privacy and confidentiality, we asked them to indicate which of the following groups could access NLSY data. Results are displayed in Table 1.

Just under two-thirds correctly understood that BLS would have access to the data collected. Notably, around 30% selected a fictitious agency, while over a quarter incorrectly thought that the Department of Education would have access to the data. Those who spent above the median time reading the form were more likely to correctly answer that BLS and the Department of Labor would have data access, (X²(1,392) = 4.9, p = .03; X²(1,392) = 9.3, p = .002, respectively).

Family Interviews

During the moderated interviews, parents typically read the form independently for 1-2 minutes. In contrast, we found that youth participants (especially the younger group) required interviewers to read the entire form aloud due to its complexity. Overall, parents and youth generally understood the purpose of the NLSY. However, they sometimes missed crucial information, such as the longitudinal nature of the survey: “Is this an ongoing study? Is it just one time?” Similarly, some youth participants also questioned how long the commitment to being part of the survey would last: “How long will we be doing the survey? Forever?”

Parents wondered about where the interview would take place: “Is it in my home, or another location?” Similarly, youth lacked understanding that the survey is interviewer-administered: “Can it be done virtually, on paper, or online?” Many preferred to complete the survey online. Some of this confusion stemmed from interchangeable use of the terms “interview” and “survey” in the forms. Several participants also expressed confusion about the acronyms and the length of the form: “Personally, I’m not a big reader. It would be good to summarize. It’s a lot of words.”

Most parents and youth understood that only BLS and other authorized agents would have access to their data and appreciated that the form explained how their data would be protected. However, some parents expressed concern about data usage and if their information would be sold to third parties.

Intercept Testing

Youth typically spent 7 minutes reading the form, including the time spent answering the follow-up questions. Most read the form twice (as they do in school); once when seeing the content for the first time, then again when highlighting the information they found helpful versus unclear. Younger and older youth understood the content in the assent form and the purpose of the study. They noted “how simplified everything was” and indicated it was “short [and] easy to read.” However, younger youth were not familiar with some terms (e.g., reproductive health), and some were unsure about the acronyms.

As seen in Table 2, about half (45.2%) understood the survey would occur multiple times; 6.5% indicated one time and 48.4% were not sure. Most younger youth (57.9%) were not sure, whereas older youth (66.7%) correctly answered multiple times.

Youth participants were unclear about the location of the study. Half of older youth (50.0%) thought the interview would be online, whereas 36.8% of younger youth indicated the interview would be at school and 26.3% thought it would be online. Only 19.4% of youth correctly answered that the interview would be at their home (see Table 3). When asked a follow-up question about their responses, two participants said they thought they could do the interview in a location they “wanted,” and could not recall the interview location.

Youth understood BLS and BLS agents would have access to their data, but one older youth expressed some confusion about who else might have access: “It is not very reassuring about who gets access to my information. It seems like a lot of people.”

Willingness to Participate

Web Study

Web participants rated how likely they would be to allow their child to participate in the NLSY on a 5-point scale from 1 (not at all) to 5 (extremely). Caregivers were more willing to allow their older (ages 15-17; M=3.6, SD=1.2) than younger (ages 11-14; M=3.3, SD=1.2) children to participate [F(1) = 11.9, p < .001]. We also asked participants whether they believed their child(ren) would be willing to participate in the NLSY, and found they believed their younger and older children would be equally willing (about 70%, see Table 4).

Family Interviews

All parents and youth indicated that they would enthusiastically participate in the NLSY: “I would say yes because it sounds like a fun study. I feel like adults don’t try to understand us sometimes.” Most thought the NLSY was interesting and important but felt compensation would be crucial for their continued participation: “I’d probably say yes I would be interested in taking this survey. Probably because it’s paying money.” Like the web survey, most parents believed their child would want to participate.

Intercept Testing

The majority (64.5%) of youth indicated they would be willing to participate in the NLSY. Compared to younger youth (47.4%), older youth (91.7%) answered “Yes” more often (see Table 5). Although younger youth (52.6%) were more unsure and answered “Maybe” they would participate, none of them selected “No” or indicated they would be unwilling to participate in the study.

Discussion

Our multimethodological study about informed consent indicated that these materials tend to be lengthy, complex, and difficult for respondents to understand. By assessing the forms with different methods, we found that each yielded some unique findings, while others overlapped or were complementary (see Table 6 for a summary).

We found that participants attended to the forms differently depending on the method used and context. Based on the wide range of time web participants spent on the informed consent page, they may have been multitasking and processed the information less carefully (Sendelbah et al. 2016), an important consideration for using unmoderated pretesting methods. Web participants were also unable to ask an interviewer for clarification. This may have led to reduced comprehension of the form and less willingness to allow their younger children to participate in the NLSY. Those participating in moderated family interviews were observed to read the materials more carefully and were able to ask questions about the contents. As a result, they showed better comprehension of the form and were more willing to participate in the NLSY. Youth in the intercept testing study read the form carefully due to the context (they read the form twice like they are instructed to do in school), and understood the forms relatively well, especially older youth, who were also more willing to participate in the NLSY. Thus, informed consent procedures may benefit from an interviewer being present to explain the study, ensure respondents understand the longitudinal nature of the survey, and answer respondents’ questions; this may be particularly true for research involving youth.

Importantly, our findings would be limited had we not used all three methods. The web survey collected paradata to assess time spent on the page with the informed consent materials, which varied widely. The family interviews revealed reasons for the confusion regarding survey mode, length, and incentives that would otherwise be unknown, though there was some risk of parental influence in the feedback from youth. These methods allowed for collection of qualitative and quantitative information from critical authority figures whose consent is necessary for future NLSY cohort interviews. The intercept testing collected rapid feedback directly from youth while minimizing the risk of bias associated with parental influence (Horm et al. 1996). These methods complemented one another to provide a fuller picture on where the informed consent procedures could be improved and streamlined.

There are several limitations to the research. First, all testing was hypothetical in nature, so it is unclear whether the research findings map onto actual response. Second, our voluntary participants may be more agreeable (and thus, generally more willing to participate in surveys) than the average respondent. Third, because each method used a different mode, there is no way to address potential mode effects that may have impacted the results. However, using different modes revealed different context effects that may impact substantive findings.

Together, using a multimethodological approach revealed more than using only one of these methods alone. Our research demonstrates the importance of using multiple methodologies in the context of pretesting materials for youth surveys, not just for questionnaires, but for informed consent processes as well. Table 7 summarizes practical recommendations for obtaining informed consent from youth participants in longitudinal surveys based on this research. We recommend that future research focus on making these materials more digestible, for example, using easy-to-understand visuals or simple videos to provide the informed consent information appropriately to each age group (e.g., Antal et al. 2017).

Corresponding author contact information

Name: Robin L. Kaplan
Email: Kaplan.Robin@bls.gov
Address: U.S. Bureau of Labor Statistics
Suitland Federal Center
4600 Silver Hill Road
Washington, DC 20212-0002