A Case of Developing Translation for in - AAPOR€¦ · Approach Translator Reviewer Adjudicator Pretest #1 Centralized, professional ... special opportunity. Translation Delivers

1

Copyright © 2012 The Nielsen Company. Confidential and proprietary.Copyright © 2012 The Nielsen Company. Confidential and proprietary.

A Case Study of Developing Translation Standards for Consumer Research in Emerging Markets

Jennie Lai NielsenMandy Sha RTI InternationalTeresa (Ye) Jin Nielsen

67th AAPOR Orlando FL, May 17th, 2012

2

Copyright © 2012 The Nielsen Company. Confidential and proprietary.

Introduction• Nielsen Consumer Panel & Retail Measurement

– Measure what retailers stock/sell and consumers buy– Consumer panels in 25 countries– Retailer panels in 90+ countries

• Growing research priorities in emerging markets– Africa, China, India– F2F recruitment & collection of households & retailers

• Challenges– Languages/Dialects

o 20+ official languages in Africa (> 2,000 languages)o 16+ Chinese dialects & 2 writing format in Chinao 20+ official languages in India

– Limited translation expertise and references available

Translation Standards Tailored for Local Challenges & Resource Needs

3


• TRAPD by Harkness et al.– Translation– Review– Adjudication– Pretesting– Documentation

• Census Bureau Guidelines by Pan et al.

– Translation Validation Form– Coding scheme to evaluate translation quality

• Translation Resources– Translator– Reviewer– Adjudicator

• Toolkit– Translation Validation Form– Translation Input Document– Vocabulary Bank– Translation‐by‐Number

Best Practices Resources & Toolkit

BackgroundTailored Translation Approaches

4


Methodology: An Experiment

Approach Translator Reviewer Adjudicator Pretest#1 Centralized,

professional translation service within Nielsen

Nielsen China-based data collection team manager

Nielsen U.S.-based methodologists

Convenience sample of respondents in China

#2 External translation firm with good past performance

Nielsen China-based data collection team manager



#3 Nielsen China-based data collection team member

Nielsen China-based survey specialist



Control Nielsen China-based data collection team member

Nielsen China-based survey specialist

N/A N/A

Documentation

5


• Qualitative Assessment– In‐depth interviews with end users

• Quantitative Scoring–Coding of translation issues for translators, reviewers & adjudicators

• Comparison of translation quality for experimental & control conditions

Evaluation Metrics

Measures of SuccessTranslation Quality

6

Copyright © 2012 The Nielsen Company. Confidential and proprietary.Title of

• National urban panel• Face‐to‐face & Online mode• Handheld Scanner• RM COI: recruitment material

development & translation

7


Developing Materials for TranslationTwo Chinese‐speaking

Researchersdeveloped English materials

Feedback fromData Collection Team

Comments from Statistical Teams

Data Collection Field Team in China

Data CollectionGlobal Governance Team in France

Finalize English Materials

- Operational design- Chinese content - Extensive experience

Collected data is linked with sampling and reporting docs.

8


Example of Recruitment Materials

9


Issues detected from English

Material Development

Panel样本框

Statistical Sampling科学的⽅法

Your OpinionCounts

你的看法⾄关重要

Community社区

Chinese cultural and linguistic considerations are adopted before source document is finalized

Multicultural membership

Be able to identify potential misunderstanding about the content and wordings of the material

Developers = Stakeholders

Advantages of Parallel Approach to

Materials Development

10


TRAPD: Translating from source to target language

Translation:Translator to translate to target language.Translation:Translator to translate to target language.

Review:Data Collection Team to review the Chinese translation.Review:Data Collection Team to review the Chinese translation.

Adjudication:Material Developers to review the Chinese translation. Adjudication:Material Developers to review the Chinese translation.

Pretesting:Material Developers to conduct material pretesting.Pretesting:Material Developers to conduct material pretesting.

Documentation:Material Developers quantify translation issues and create Vocabulary Bank document.

Documentation:Material Developers quantify translation issues and create Vocabulary Bank document.

11


Translation

TranslationCenter

1.Overview of study

2.Translation purpose and background

3. Input‐output document

Review

Data CollectionTeam

1.Review/Edit translation

2. Provide feedback or comments

Adjudication

Material Developers

1. Review/Edit translation

2. Group discussion and finalize translation

The reviews corrected errors, replaced high-level vocabulary that may be confusing, and simplified difficult words and sentences.

12


PretestingQualitative Interviews

o Developed interview protocol to test the Chinese version

o Conducted six one‐on‐one interviews

3 interviews: China data collection team

3 interviews: Chinese locals that resemble potential respondents

o Cognitive pretesting efforts revealed issues that came to light when the

communicative effect of the translation was discussed with respondents.

English Your home was specially selected.

Translation(Prior to Pretesting)

We in particular selected your family to participate in this project.

Translation(Post‐Pretesting)

The selection of your family’s participation is a special opportunity.

Translation Delivers the Intended Message

13


Documentation

Translator Reviewer

• Only stage where translation errors/ production errorswere found

• expected result

Reviewer Adjudicator

• Important improvements made

• Most modifications are linguistic rules & standardization

Adjudicator Pretesting

• Fewer issues observed• Most were cultural

norms

14


Lessons Learned• Biggest challenge is to identifying the

appropriate language experts in emerging market.

• Even with a parallel development process and multicultural development team, not all issues can be identified while materials were being developed.

• Challenge for quantitative coding is coders need to be trained to accurately identify and capture different issues.

Recommendations• Early identification and establishing a

long‐term relations with a translation vendor.

• If schedule and budget allow, use “advance translation” to detect possible translation problem in source material development stage.

• Use quantitative scoring as a way for managers who don’t speak the language to assess quality.

• TRAPD process can be applied to consumer research TRA is to correct language issues; Pretesting is to uncover issues

beyond words and sentences

15


Jennie LaiGlobal DirectorMobile Innovation Program

Email: [email protected]

Teresa JinSenior Research ScientistMobile Innovation Program

Email: [email protected]

Documents

A Case of Developing Translation for in - AAPOR€¦ · Approach Translator Reviewer Adjudicator Pretest #1 Centralized, professional ... special opportunity. Translation Delivers