Development of a Practical Translation Approach for Survey Research Projects

Liana Manuel; Luis Miguel Contreras; Lisa Lee; Barbara M. Fernandez; Jennifer Vanicek; Meredith Gonsahn; Eduardo Salinas; Eileen Graf

doi:10.29115/SP-2024-0033

Introduction

One common data collection challenge in the United States is a language barrier, when an English language survey is inaccessible to respondents with limited English proficiency. Offering translation of survey materials in respondents’ preferred languages can enable research participation among populations that do not speak English (Lopez et al. 2008). Depending on the size of non-English speaking groups and how they differ from other respondents, offering translations can reduce nonresponse and nonresponse bias (Groves 2006). Enabling survey participation through translation can also increase the granularity of the data for non-English speaking groups and thereby enhance the utility of the data for identifying trends that may otherwise be missed due to the aggregation of data from smaller groups (Kauh, Read, and Scheitler 2021; FCSM 2020). However, a lack of budget and time may constrain researchers from translating their surveys. Further, researchers not familiar with the characteristics of a good translation may be tempted to take shortcuts (e.g., using a translated document from a translator without further review). Survey managers need guidance on managing the translation process and on understanding what constitutes a quality translation. Without such guidance, projects risk not having sufficient budget and time for translation, not following best practices, or not including translation in the original project scope.

Our interest in using translation in research projects surfaced during in-house consultations with researchers and survey managers on ways to increase engagement with surveys among underrepresented groups. We sought an approach to incorporate translation into project scope, especially for smaller groups of respondents who need translation and for projects with limited budgets and schedules.

In this article, we offer recommendations on how best to incorporate translations into research projects. Background is provided to define a high-quality translation and best practice approaches to translation, and we present an approach geared toward projects with limited budgets and time. We also present projects in which we used this approach. Finally, we include guidance on how to work with translation vendors and recommend the next steps.

Methods

We conducted an environmental scan that included a literature review and interviews with translation experts to inform the development of our guidance. The literature review included articles in academic journals and reports from federal agencies and research organizations (see Appendix 1 for details). We conducted one-hour in-depth interviews with translation experts from marketing, academic, governmental, and nongovernmental research institutions in the United States and Europe. We identified a group of 16 experts (14 external to our organization), using the literature review and referrals gathered during the interview process. From this group, we invited 13 experts, and we conducted seven video interviews—six external and one internal to our organization. Three invited experts declined the invitation, three did not respond, and three referrals remained uninvited. Interviews were conducted between July and August 2023 and were recorded and transcribed with the expert’s consent. We offered $50 compensation for their time.

The interviews focused on the expert’s research area and experience in translation; see Appendix 2. Questions concerned best practices in translation, design considerations when planning a project involving translation, how to ensure the accuracy of translations, and recommendations on how to incorporate translation into projects with smaller budgets and shorter timelines. Experts noted translation resources, including peer-reviewed literature, manuals, guides, and websites featuring frameworks and strategies to approach translation in research projects. After data collection, three independent researchers performed a coded thematic analysis of the interview data and summarized the main findings.

Findings from the environmental scan informed our development of a framework for translation geared toward projects with limited budget and time. We tested and refined the framework in two survey projects that required translation. The environmental scan and project input also informed the guidance we present for implementing the approach and practical recommendations.

What is a High-Quality Translation?

There are multiple components of a high-quality translation. A high-quality translation can be considered “reliable, complete, accurate, and culturally appropriate” (Pan and de la Puente 2005, 5). The translated text is reliable, complete, and accurate when it conveys the intended meaning of the original or source text without adding or excluding information and is grammatically correct (Pan and de la Puente 2005). The translated text is culturally appropriate and meets cultural and normative equivalence when it is applicable and considers and addresses societal or cultural differences between the source and target languages (Pan, Sha, and Park 2019; Pan and de la Puente 2005). For a translation, equivalence is key to convey the same intended message regardless of language and to prevent bias; high-quality translations reflect the meaning of the source text while using concepts and terms native speakers use, beyond accuracy based on linguistic rules. From a sociolinguistic perspective, the translation must also account for the social context in which language is used, with consideration of the social practices and cultural norms of the target population for whom the translation is intended (Pan, Sha, and Park 2019). For example, a translation may be structurally accurate in the target language, but if a concept presented in the translation is either expressed differently or is not a familiar concept in the culture of the target language, the translation will not be successful. While no single factor leads to a high-quality translation, successful translations are error-free, easy to read and understand, as close in meaning to the source/original version as possible, and balance faithfulness and fluency across translations.

Table 1 summarizes the characteristics of a high-quality translation.

Table 1.High-Quality Translation Checklist

Is the translation reliable (i.e., the translation conveys the intended meaning of the original/source text)?

Is the translation complete (i.e., the translation does not include or exclude information compared to the original/source text)?

Is the translation accurate and error free (i.e., the translation is gramatically correct, without typos or other errors)?

Is the translation easy to understand?

Does the translation adequately consider the target population's characteristics and subgroups within it?

Is the translation culturally appropriate?

Does the translation meet conceptual equivalence (i.e., the translation only includes concepts/ideas that are relevant to both source and target populations)?

Does the translation meet normative equivalence (i.e., the translation addresses any social and cultural differences between source and target populations)?

High-quality translations require professionally trained translators to ensure the translation is culturally appropriate. Criteria for selecting a translator are summarized later in the paper.

What are the Best Practices in Translation?

Our environmental scan finds a committee approach to be the recommended translation process, with the Translation, Review, Adjudication, Pretesting, and Documentation (TRAPD) method the preferred approach for high-resource projects (Mohler et al. 2016; Harkness, Villar, and Edwards 2010; Pan and de la Puente 2005).

The TRAPD approach features a large team, with groups of translators and reviewers, and parallel translations. At least two independent translations are developed; translators and reviewers compare the translations and decide on a final translation, and an adjudicator compares the translation with the source and approves the translation for pretesting (Harkness, Villar, and Edwards 2010). The TRAPD approach is resource- and time-intensive, with variations that offer less intensive options (AAPOR/WAPOR Task Force 2021). Ideally, all members of the committee will be familiar with the source and target languages. Additional reviewers—language experts who assist with quality—can compensate for reviewers or adjudicators less familiar with the target language (Mohler et al. 2016). One benefit of the committee approach is that including more individuals in the translation process gives the translation a wider range of team expertise, with the potential for more validation and cross-checking (Mohler et al. 2016).

For projects with moderate budgets and timelines, we identified two alternative approaches—a split translation with an interdisciplinary team or a single translation with an interdisciplinary team (AAPOR/WAPOR Task Force 2021). The approaches yield a single translation from one or several translators and may not include pretesting; for a review of pretesting methods, see Aizpurua (2020). Financial savings can be achieved by reducing the number of translators while using a diverse group of reviewers (AAPOR/WAPOR Task Force 2021). Projects with limited resources and timelines may consider a split translation with a single reviewer (AAPOR/WAPOR Task Force 2021). This approach achieves the greatest cost savings and places greater responsibility on the reviewer than other team approaches. All best practices require at least two translation experts (a translator and either a reviewer or language expert). Budget-friendly approaches with smaller teams assume that the cost savings outweigh the need for additional experts. See Table 2 for a summary of team approaches.

Table 2.Team Approaches Recommended in the Literature

Approach	Description	Benefits	Drawbacks
TRAPD (committee approach)	A team of two or three translators creates multiple parallel/full questionnaire translations.	This includes multiple teams of translators and interdisciplinary researchers who collaborate to review and adjudicate. The only approach that requires a pretest.	A generous budget, staffing model, and timeline are required.
	A second team with at least two members conducts iterative reviews.
	The translation is adjudicated by the second team or by a third team.
Split translation with an interdisciplinary review team (modified committee approach)	A team of two or three translators splits the questionnaire to create a single, full translation.	For projects with moderate budgets and timelines. This includes multiple translators to create one translation (which saves time and money), and an interdisciplinary team to review and adjudicate.	This is a smaller team than used in the TRAPD approach. Only one person translates each section (which will require coordination to ensure consistency), and the pretest is optional.
	A second team with at least two members iteratively reviews and adjudicates the translation.
Single translation with an interdisciplinary review team	One translator creates a single, full questionnaire translation.	For projects with moderate budgets and timelines. This includes an interdisciplinary team to review and adjudicate.	This is a smaller team than used in the TRAPD approach. Only one person translates the materials (which will take longer than the split translation approach), and the pretest is optional.
	A team with at least two members iteratively reviews and adjudicates the translation.
Split translation with a single reviewer	A team of two or three translators splits the questionnaire to create a single, full translation.	For projects with limited budgets and timelines. This includes multiple translators to create one translation and only one person to review and adjudicate (both save time and money).	This is a smaller team than used in the TRAPD approach. Only one person translates each section (which will require coordination to ensure consistency), only one person reviews and adjudicates, and there is no pretest.
	One person reviews and adjudicates the translation.

Another important aspect of translation is to achieve cultural and normative equivalence (a translation that conveys similar information across different contexts). If resources are available, decentering and advance translation techniques enhance the quality of the translation and are used when the source questionnaire is not yet final. Both techniques improve translatability and equivalence by allowing teams to adjust either source or target questionnaire rather than target only. Decentering avoids focusing translations on a single language or culture by not designating a source or target language to decrease overreliance on the source language (Chavez and Canino 2005). Advance translations start with a draft source questionnaire, revised in tandem with the translated questionnaires to omit terms or concepts that are ambiguous or challenging to translate in all language versions; the approach ensures that the text is relevant to all languages and cultures (Dorer 2023). This approach is helpful because challenges are often not clear until after the translation process begins (Behr and Shishido 2016).

In addition, the literature recommends that the translation team create and maintain a translation manual with challenging or frequently used words, phrases, or constructs that can help ensure transparency and improve translation quality and consistency. The manual should include existing high-quality translations for common words and concepts (translations that have been pretested and recommended by other literature), particularly when the survey and intended audience are similar to the proposed project (Sha and Immerwahr 2018).

Developing an Adapted Committee Approach to Translation

For projects with the most limited resources, we developed a new approach, based on our environmental scan and testing of a translation framework with two projects. Our approach offers multiple review stages while minimizing staffing resources. It is a five-stage process including 1) translation by a vendor, 2) first review, 3) adjudication, 4) second review, and 5) process documentation. The translation is outsourced to a professional translation service, and an internal team reviews and adjudicates the translation, with no pretesting.

Project Input: As we developed the adapted committee approach, we assessed feasibility and implementation using two web and phone survey research projects with budgets and timelines that could not accommodate the full committee approach. We found that the adapted committee approach required minimal additional time and costs and improved the quality of the translated questionnaires; see Appendix 3 for more information.

Project 1. A study of a multilingual state-wide adult population required translating a survey instrument into Spanish, Korean, and Traditional and Simplified Chinese. The original scope of work and budget only included resources and time for a vendor’s translation.

For the project, we added three steps to increase translation quality. First, before translation began, we provided the vendor with a summary of the project and study populations. Second, internal bilingual interviewing staff reviewed the translated questionnaires. They found errors related to grammar, formality, missing words or phrases, word choice, and sentence structure. Third, we provided the review to the translator, who made the recommended changes. The team documented the process.

One challenge was the reluctance of the translation vendor to facilitate a meeting between the reviewers and the original translator. The original translator reviewed feedback for all languages and accepted changes without further discussion. In the future, we plan to explore opportunities to improve adjudication with translation vendors and to find a broader pool of qualified reviewers in common target languages.

Project 2. A study of a rural Latino population required the client to arrange for a Spanish translation. The client conducted cognitive interviews and pilot testing to develop the Spanish-language materials, but we consider this an adapted committee approach because the original scope of work and budget only included a translation by the client.

We added four steps to increase translation quality. First, an internal team of three survey researchers (native Spanish speakers) reviewed the translation and recommended revisions, including suggestions for wording changes and for cultural and normative equivalence. Second, the client adjudicated the original version with the recommendations and developed a revised version. Third, internal bilingual interviewing staff reviewed the Spanish translation. They found and corrected minor grammatical and spelling errors. Fourth, after programming the questionnaire’s web and phone versions, the bilingual interviewing staff tested the Spanish-language version to confirm that programming was correct. The process will be documented in the project’s methodology report.

For both projects, we found that the adapted committee translation approach improved translation quality by identifying and correcting errors that would have been undetected in the original translation (by the vendor for Project 1 and the client for Project 2). The additional work was not in the original budget or timeline for either project; however, we identified project efficiencies that allowed us to incorporate the low-effort and low-cost tasks of our proposed approach.

Recommendation: Adapted Committee Translation Approach

Based on the environmental scan and project input, we developed an adapted committee approach for survey managers limited by a budget and schedule. The approach includes translation from a vendor without further review or a scope of work and budget that may not have included needed translation. Guidance on an adapted committee approach (a single translation with an internal review team) intends to help survey managers assemble a translation team, select and work with vendors, and train the vendor and other team members on their roles, while still providing a high-quality translation. Our recommended approach includes the following steps:

Translate: Work with a professional translation vendor who translates the source instrument to the target language. See Table 3 for considerations in choosing a translation vendor.
First Review: A reviewer fluent in the source and target language (either on the project team or a separate translator) reviews and provides feedback. If available, more reviewers can enhance the review with more linguistic, survey methods, or subject matter expertise.
Adjudicate: The original translator considers the reviewer’s comments and develops a final version, replying to the comments from Step 2 (Review).
Second Review: A reviewer (person who conducted first review or another reviewer) fluent in the language of the translation reviews the programmed electronic or hard copy versions of study materials. For web and phone surveys, the reviewer pays particular attention to automatic text fills (for example, he/she, they/them) and other logic that might function differently in non-English languages.
Document: The translator and research team document each step.

In the adapted approach, the survey manager must identify a translation vendor linguistically qualified to do the work and willing to collaborate with the team, including participation in a briefing on the project and the translation approach. Ideally, reviewers and adjudicators would be fluent in the source and target languages, and more than one reviewer would participate at each review step to bring diverse perspectives and minimize errors to help improve translation quality.

Guidance for Selecting a Translation Vendor

Selecting a translation vendor is a key step to successfully implement the adapted committee translation approach. Translation quality depends on the individual’s experience developing high-quality translations. Based on input provided in the expert interviews, we provide the following guidance on selecting a translation vendor. We recommend evaluating the experience and qualifications of the translation vendor and their translators, with attention to their familiarity and willingness to work within the selected translation approach. See Table 3 for considerations in choosing a translation vendor.

Table 3.Working with Translation Vendors

Questions	What To Look For
Vendor and Translator Qualifications
What are the qualifications of your translators who will work on my project? What topic areas are your firm’s specialties? What background and orientation will you provide the translators who work on this project?	Certification (for example, American Translators Association); years of experience; native-level fluency in the target language(s); experience living in/with communities where the target languages are spoken Familiarity with social science research and/or topical expertise The translation vendor will provide project orientation to the translator(s) and/or allow you to provide orientation
Translation Process
What is your translation approach? How will you ensure a high-quality translation? -Establish conceptual and/or cultural equivalence -Review and adjudicate differences between the translator and reviewer -Pretesting	The vendor is familiar with the committee approach or is willing to be briefed on the approach and use it Willingness to incorporate approaches for smaller budgets and shorter timelines (if needed) to preserve elements of high-quality translation practices The vendor does not use back translation for quality control purpose* The vendor has a process for review and adjudication of translation work
Collaboration
Can we meet with the translator(s) and/or provide background materials to provide orientation to the study? Can we review a draft before the full translation is delivered?	The vendor will take part in a meeting with the translator(s) and the research team and/or provide study background The vendor will provide a draft

* Back translation is no longer recommended as an approach to evaluate translation quality. It requires translating from the source to the target language, then back from the target to the source and comparing the results. The process emphasizes the original translation and perpetuates any errors in the original translation (Mohler et al. 2016; Sha and Gabel 2020).

Practical Recommendations

Translations for survey research projects increase study engagement for populations who do not speak the source language. Based on the environmental scan and our experience developing an adapted approach, we recommend the following steps at the start of research projects involving multilingual populations:

Plan in advance to maximize the budget and time available for translation.
Use the most robust team approach to translation that can be conducted, given your budget and schedule.
If using a vendor, select translators carefully, to ensure they deliver high-quality translations and will take part in the team process.
Provide the translation team—including the translation vendor, reviewers, and adjudicators—project and population background information, a translation manual, and instructions.
Document the translation process.

Based on findings from the literature review and expert interviews, we recommend providing the translation team with an overview of the study purpose and goals, population(s) of interest, languages, modes, and questionnaire content and topics. More detail includes the study population’s nationality(ies), dialect(s), literacy and education levels, income levels, age(s), and geographic location(s). In addition, consider providing official translations for key terms used in the survey, English language terms for terms commonly used by non-English-speaking individuals (for example, green card), and definitions of unique, sensitive, or complex terms in English (such as household, beneficiary, and gender identity). Further, include technical instructions or specifications, such as to check text fills (for example, he/her, you/your child).

Our guidance has focused on translation; however, proxy respondents may participate in a survey on behalf of a respondent with limited proficiency in a project’s source language, or a language assistant may offer live interpretation. In Appendix 4, we provide guidance for working with proxy respondents and language assistants.

Acknowledgments

Funding from NORC at the University of Chicago supported this work. The authors thank the experts who generously shared their expertise with us: Larisa Caicedo, MA; Juan Carlos Donoso, PhD; Ana Gonzalez-Barrera, MPP; Mark Hugo Lopez, PhD; Rafael Treviño, PhD; and two anonymous experts. We also thank the reviewers who provided valuable feedback and Mandy Sha for her support.

Lead author contact information

Manuel-liana@norc.org

NORC at the University of Chicago, 55 East Monroe St., Chicago, IL 60603