The Performance of Vignettes in Focus Groups and Cognitive Interviews in a Cross-Cultural Context

Mikelyn Meyers, Yazmín A. G. Trejo, Lucia Lykke

Abstract


Survey researchers conducting pretesting via cognitive interviews or focus groups often use vignettes to evaluate questions and answer categories in order to identify possible measurement problems, particularly for reporting situations that are relatively rare or for sensitive topics. Although research has compared the performance of vignettes in Spanish and Asian languages in cognitive interviews, there is little research that compares the performance of vignettes across pretesting methodologies in languages other than English. To address this gap, we investigated the performance of a vignette about a homeownership question that was administered in focus groups and cognitive interviews conducted in seven languages: English, Spanish, Chinese, Korean, Vietnamese, Russian, and Arabic. We coded the cognitive interviews and focus group summaries to quantify the responses and the types of comprehension problems respondents had, and we compared the vignette response data from cognitive interviews to the data from focus groups, by language. We find that administering the vignette in cognitive interviews was more effective than administering the vignette in focus groups for uncovering difficulties respondents had with the survey question, particularly for Spanish- and Arabic-speakers. We conclude that using vignettes in focus groups without cognitive interviews may not reveal problems with survey questions as effectively. Regardless of methodology, the vignette task was challenging for certain language groups, and further research is needed on the cross-cultural adaptation of vignettes.

Keywords


multilingual survey research, pretesting methods, vignettes, survey translation, cognitive interviews, focus groups

Full Text: PDF HTML

Comments on this article

View all comments


About Survey Practice Our Global Partners Disclaimer
The Survey Practice content may not be distributed, used, adapted, reproduced, translated or copied for any commercial purpose in any form without prior permission of the publisher. Any use of this e-journal in whole or in part, must include the customary bibliographic citation and its URL.