Language Quality Management: Models, Measures, Methodologies

How do others do it?

Introductions

Gráinne MaycockVice President of Sales

Sajan, Inc.

• 18 years in localization industry

• Degree in applied languages

• Passion for language and technology

• Advisory board member Localization World

• Advisory board member Brand2Global

• Build global business solutions for:• Global translation management• Language quality management• TMS – CMS integration• Machine translation and post-editing• Global marketing content

Quality Expectations: We All Have Them

Language Quality Matters

• Translation quality is critical to your brand and company’s international success.

• Brand, message and product consistency need to be maintained across all languages as well as accurate terminology and local nuance and regulatory differences. Your style also needs to “translate” across markets.

Predictive Quality

Segment Level Revision Analysis

Scoring and Comparisons

Active Real-Time Metric Management“Early in Process”

Quality Matters

• But does one size fit all?• Is there one overarching definition?

o One industry standard model?o One way to approach resourcing?

• What role does technology play?• What are the organizational challenges to

optimizing language quality?• And the challenges?

Three Companies Tell Their Story

Language Quality Management for Microsoft Office

• Over 60 different applications and services.

• Moving to multiple platforms

• Continuous releases

• Over 100 languages supported

• Over 3,000 people involved

• Over 70 million User Interface resources managed in our TMS.

• Localisation as a service modelpartnering with our localisation vendors on:•Linguistic service•Translation service•Linguistic testing and support

How do we approach language quality?

Measures

Linguistic(monthly)

Define a reference! For each language, focusing on Style and terminology.

• Feedback received• Regular style update• Language reforms• Terminology update

Translation

(daily)

Check the translation! Translation errors against both the English source and approved terminology/style.

• Proofing editing• Self certification• LQA cross checks• Automated functional

testing

Visual(monthly)

Verify fit and finish.Check the translation in the context of the actual UI.

• Screenshot reviews• Scenario based reviews

User experience(Quarterly)

Collect user experience.Check that the translation matches the way a feature works.

• Scenario based reviews• Internal feedback• External feedback• Subject matter experts

Very classic approach to software quality, our challenge is to keep it agile!

Major quality challenges

Agile

•Can we break the 24 hours turn around barrier?

•Difficult to predict upcoming volume

•Continuous localisation + continuous release

MT + Post editing

•How efficient is MTPE for a User interface localisation?

Context and disambiguation

Addressing scale and agility: Context exampleProblem: Lack of context is the #1 cause of mistranslation for us.

Today: • Leaf environment supports already various types of context

• Key terminology• English screenshots (when available)• Translation suggestions

• Context query plug-in: • Translators should receive an answer within 4 days max• Leverage questions and answers across language teams

Future: • Improve the relevance of the context provided

• Provide terminology disambiguation• Reduce the context lag

• Anticipate the most common context queries• Fast track English screenshots in Leaf

Measuring success in that space

QTO: Quality of Translation Output% of resources changed 1-month after sign off.

This metric can be drilled further down by root cause of the change and should help us drive future investments in our translation environment.

Measuring success in that space

QTO: Quality of Translation Output% of resources changed 1-month after sign off.

This metric can be drilled further down by root cause of the change and should help us drive future investments in our translation environment.

13

Linguistic Quality for the entire Symantec Customer Experience

Linguistic Quality for the entire Symantec Customer Experience

Our Mandate

• Global Team of Language Quality Process Managers

• Terminology Mgt, Linguistic Review, Machine Translation, TM Management, Language Coordination

• ~ 30 Languages supported• ~ 1000 signoffs on products per year• ~ 25,000 signoffs on quality for product-related items per

year

• Main Goals:• Improve upstream quality• Provide Linguistic Standards and Manage Linguistic

Assets (Terminology Database and TM) • Quality Control

14

What Does Quality Mean To Us?

*True to Function*Unambiguous*Right for Purpose

*Fluent*On Message*Right for Audience

*Speed *Correct Product references

Terminology, Content Standards

Consistent Rating Scale

15

How?Process, Tools & Automation

Upstream Work and Standard Setting*Content Analysis*Terminology Harvesting, Translation, SEO*Translation Brief

Translation Enablement*Terminology Portal*Searchable Context UI Tool*Translator-Reviewer Network

Quality Control*Linguistic Review Tool*Consistent Rating for data collection*Translator-Reviewer Feedback Loops

Machine Translation:*Users enabled to machine-translate Support content*Small amount of human translation

16

Challenges

• Subject matter expertise, creativity• Updating context in agile development process• Agile cross-vendor collaboration on terminology and translation questions• Timing of Linguistic QA in agile process• Improving MT quality for Support

Outlook – What We Need Next

People:•Collaboration across our vendors on shared platforms•Partnership and expertise: vendors provide us on best practice and expertise

Technology:•Translators enabled in context environment•Environment enables fast-cycle translation by dedicated experts•Enabling user-based MT output improvement

17

Conor CluneProduct Development Manager

Localization Business Unit

Leading global innovator, manufacturer and marketer of tools, equipment, diagnostics, repair information and systems solutions for professional users performing critical tasks.•1920 - Founded•1931 - Entered the International arena•Today – Products and services can be found in more than 130 countries

Products and services include diagnostics tools (hardware and software), information and management systems and other solutions for automotive dealerships and repair centers. The content solutions is highly technical and specialized, which means it can be difficult for people outside the industry to understand what is being said.

Small team… 4 internal, 3 external and great business partnersVarious content types…•Vehicle Communications Software… 66 million word count a day processed•Component Test Meter… 10 million word count a year•System Software•Software delivery systems•Tech Publishing's… 1+ million word count a year

Tools Set… all developed in house and with business partners

Localization Business Unit

Scenario…Translator is processing large volume of strings (60,000 word count)

Quality is key to our business

Scenario…High fuzzy matches presented to the translator who has been working on the task for 6-7 hours.

Quality is key to our business

Quality is key to our business

Scenario…Herr Hertz (Mechanic) in the repair shop with Scanner tool is instructed to replace the Rear Right ABS system instead of the Rear Left…

Message: English: RR Wheel ABS Ctrl Status: Fault German: Status Rad ABS-Steuerung hinten links: Fehler

Potential Result…The customers ABS malfunctions on the Autobahn

• When a translator/linguist is reviewing thousands of strings over hours, mistakes will happen… snow blindness.

• How do you build in fail safes?– Education– Critical Term Glossary– Automation

Quality is key to our business

• Education– Work with the business partner to educate the

global translation team on the risks and impact– Identify processes for risk avoidance

• Critical Term Glossary– Build a list of know terms that could have negative

consequences– Match a source term with translated terms and its

synonyms

Quality is key to our business

• Automation– Develop tools for different stages of the process– Source: Identify critical misleading terms (full

form and abbreviated)– Translation: All content is checked against CTG

• Translators run automation• Loc Engineers run automation

– Internal:• Snap-on team review the automation reports from

Business Partner… sanity check and offer feedback• Further automation (in house) is run and issues feed

back to business partner

Quality is key to our business

Other Linguistic Issues•Automotive have terms that have a multiple meaning in a different context.•Education & Assistance

– Keep the translator focus on Automotive– Give the the translators access to source expects…

automated query system– Term mining– Language Champions– Crowd sourcing

Quality is key to our business

Common Threads

• Terminology and Style Management key• Context has and continues to be King

– Lack of context causes rework

• Speed and agility accelerating• Technology enablement is a must• Defining what will be measured and how upfront • Solid investment in client-Supply partnerships• Continuous investment in partnership

– Education, optimization, rapid continuous improvement

Panel Questions

• In the run up to this event, we reached out to a number of industry clients and vendors on this topic and asked what questions they would like to pose. The following slides contain a handful of those questions, which we will pose to the panel on your behalf. We will then open it up for any audience questions.

Question 1: Continuous Improvement

• How do you trend and measure improvement? And what do you do if your language partners are consistently missing the mark? Do you provide training? If so, who performs it and how?

Question 2: Content Types

• Do you set different measures for different content types and, if so, can you describe examples?

Question 3: Trends

• Are there any trends in speed, content type or tools that have helped or hindered your approach to measuring and delivering language quality?

Question 4: Challenges

• What has been the biggest challenge you have faced when adding a new language or a new product?

Question 5: Resources

• What skills do you believe resources need to have—both inside your organizations and when it comes to your language partners?

Question 6: Tools and Technology

• What tools, technology and automation processes do you employ to manage language quality (consistency, adherence to terminology, etc.)?

Question 7: Models

• What models or methodologies (LISA QA, TAUS DQF, internal standards, vendor-defined standards or other) do you use in your company to define and measure quality?

Question 8: Evolution

• Have you seen an evolution in the past 5+ years in models, tools and challenges?

Question 9: The Future

• What do you hope to see that will help you achieve language quality excellence moving forward?

Questions from the Audience