There is broad agreement among analysts that longitudinal studies provide a more accurate picture of human change than any combination of cross-sectional studies. Most social science theories predict or imply change in a person or a group of persons over time. Although cohort analysis and other techniques can provide some insights into human change, measurement of the same respondents over time provides the best indicator of human change. In educational research, longitudinal studies have been the gold standard for nearly 50 years.
To provide a more intensive examination of student achievement in science and mathematics (and their relationship to career choices), the National Science Foundation funded the Longitudinal Study of American Youth (LSAY) in 1985 and has continued that support over the last 20 years (NSF grants MDR-8550085, REC96-27669, RED-9909569, REC-0337487, DUE-0525357). This article reports on tracking procedures that were able to account for 94 percent of the original 5,945 students and obtain current addresses for 90 percent of the original sample (see Table 1).
Longitudinal Study of American Youth
The original LSAY participants were recruited as 7th or 10th graders in 1987 from a probability sample of public school districts in the United States. During the first seven years, each student was given mathematics and science achievement tests each fall and was asked to complete attitudinal questionnaires each fall and spring. Parents were interviewed by telephone, and information was collected from mathematics and science teachers and school principals. In 2005, the NSF provided funding for two new cycles of data collection in 2007 and 2008 to allow the construction of an empirical model of educational and career development. Once funded, the first task was to re-locate the original participants, now approximately 32 and 35 years of age.
LSAY Tracking Methods
From the beginning of data collection in 1987, the LSAY staff collected family contact information, which facilitated short-term tracking when students moved during the in-school years of the study and served as the foundation for subsequent tracking activities.
In April, 2006, LSAY staff began a new tracking activity using: (1) online database tracking, (2) newsletter mailing, (3) calls to parents and other relatives, (4) use of alternative online searching, and (5) questionnaire mailing. As a result of these tracking activities, we located contact information for approximately 90% of the original participants (see Table 1).
Online Tracking. The development of online databases has made tracking far easier than in the early years of the LSAY. During the fall of 2005, we examined several online databases and selected Intelius (www.intelius.com), using a premium search feature that verified addresses and phone numbers against utility bills. Customer service representatives will negotiate bulk rates – about 50 cents per search.
We used Intelius to search for all participants in Spring, 2006. We located 37 percent of the participants through Intelius, as well as addresses for parents of another 35 percent of the cases (see Table 2). Intelius is inexpensive and easy to use, but it was not useful for finding individuals with common names that had moved from their original city or who changed their names.
In a second cycle of online tracking, we acquired access to Accurint (www.accurint.com), a database operated by Lexus-Nexus. Accurint includes access to drivers’ licenses in most states and these records include the exact birthday of each person, which the LSAY collected in the initial cycle in 1987. Because Accurint links records by social security number (blinded from our view), we were able to do nationwide searches including women who had changed their last name. Using Accurint, we located addresses for 75 percent of the participants (see Table 2).
Fall 2006 Participant Newsletter. A fall of 2006 newsletter served as our first contact with the participants since 1994. The newsletter included information about the ways the LSAY data have been used and a few results from earlier data cycles.
United States Postal Service National Change of Address (NCOA). To reduce the number of undeliverable pieces of mail, the U.S. Post Office makes available through licensed firms a computerized data base of changes of address filed throughout the country. Prior to mailing the LSAY newsletter, we sent 5,690 participant addresses to Anchor Computer, a licensed NCOA vendor, (www.anchorcomputer.com), and we received forwarding addresses for 681 of the addresses. The cost of this service was about $75 and saved us far more than that in postage and labor costs.
Parent and other contact calls and letters. For respondents who could not be located through databases, we made follow-up calls to their parents and other relatives. Thirty-six percent of the relatives provided us with participant contact information and another 16 percent said they would forward information to the participant.
Alternative Search Methods. For cases not located otherwise, we used the online Social Security Death Index (www.ssdi.rootsweb.com). If we could not locate the person through the SSDI, we turned to prison records. Federal prisons (www.bop.gov/iloc2/LocateInmate.jsp) and prisons in 37 states have online databases. If a participant has a distinctive name, we search on Google.
Current Tracking Activities. Tracking is a continuous activity in any longitudinal study. LSAY participants move, get married or divorced, change names, and change jobs. We maintain a Web site (www.lsay.org) and encourage our participants to use it to report changes of address and other related information. Although we have e-mail addresses for 75 percent of our respondents, we continue to screen our mailing addresses through NCOA and use Accurint as needed.
The 2007 Questionnaire Cycle
In 2007, we initiated a new cycle of data collection to update our educational and occupational record for each individual. Early in 2007, each LSAY participant for whom we had an address received a letter asking them to participate in the new survey by going to an online site and completing a questionnaire or to return an enclosed postcard indicating that they would prefer a printed questionnaire or a telephone interview. Only two percent of participants asked for a telephone interview, 69 percent used the online questionnaire, and 29 percent completed a printed questionnaire. Each respondent was offered $20 for their cooperation in the study. After consulting with other studies of this kind, we increased the payment to $30.
Because the 2007 questionnaire was a report on educational and occupational activities since high school, we continued to write and call respondents throughout 2007. Follow-up e-mail messages were sent when an e-mail address was available. When e-mail was not available, follow-up mailings to non-respondents included a letter asking for cooperation and a printed questionnaire. When telephone numbers were available, follow-up telephone calls were made. By the spring of 2008, approximately 68 percent of the eligible sample completed the new questionnaire (see Table 3).
Discussion
The LSAY tracking activities and the 2007 cycle of data collection demonstrate that it is feasible to locate respondents in longitudinal studies even after a period of more than a decade without contact. We think that this should be encouraging to investigators thinking about launching a new longitudinal study or about the possibility of re-contacting respondents from previous studies. At no time in American history has it been more feasible to track samples of adults over a period of years.
For any longitudinal study, the fundamental requirement is to build an extensive tracking record in the early cycles and to maintain those tracking records through periodic contact. For new longitudinal studies, building a solid tracking information base should be a core objective. Building an initial tracking database requires staff, equipment, and persistence, but there is no substitute for the resulting tracking file.
It is important to encourage respondents to see their participation as contributing to something useful and important. The LSAY emphasized throughout the in-school years that the study was designed to help improve education and that it was only through their cooperation that educational leaders would know what was working and what was broken. When the LSAY data collection was resumed, we adopted a letterhead that reminded participants that they are a part of Generation X and that the LSAY is the only study that chronicles the history of their generation.
Despite the extraordinary tracking and data collection record, there are some unsolved problems. The recent Pew study of prison populations in the U.S. provides estimates on the percentage of young adults (by age groups) who are currently incarcerated (Pew Center on the States 2008). The estimated number. of young adults from our sample who we have identified as being in prison is significantly lower than the Pew estimates. We also note that the number of deceased LSAY participants is significantly lower than the number that would be estimated from reports from the Centers for Disease Control and Prevention (www.cdc.gov/nchs/deaths.htm).
If we were starting the process over … It is useful to look back at the last 20 years and think about what advice we would give an investigator who is planning to launch a new longitudinal study or is thinking about reviving an earlier study.
First, we would collect the full name – including middle initials – for all LSAY participants and their family members. We would also make certain that we had each participant’s legal name rather than a nickname or initials.
Second, we would keep an electronic record of all previous addresses for students, parents, and relatives. In the late 1980′s, computer storage was expensive and only the most recent address and phone number were retained in the electronic database, but today memory is cheap and all tracking records should be retained.
Third, we would have used Accurint from the beginning of the study and we would have used an NCOA vendor for all of our mailings. Neither of these services was available in 1987, but we recommend them for any current longitudinal study.
On balance, we think that the LSAY tracking and data collection experiences should be taken as encouraging. And we hope that younger investigators will think about 20-year studies when they think about how to address the issues and problems in their disciplines.