Localization testing with compare-locales and Silme

by Adrian Kalla

Me

● Computer Science studies in Wuerzburg/Germany

● Member of the Polish Mozilla-L10n-Community since 2002/2003

● Internship@MoCo: August-December 2008

Why do we need L10n-tests?

● Localizers:– to easily find possible problems

● Developers:– to make sure a localization does not break

everything

Silme & compare-locales

● Silme is a new localization library written in Python

● Compare-locales is a tool for testing the completeness and correctness of localizations

The world before Silme

● Mozilla Translator● Narro● Translate Toolkit● Litmus● compare-locales● and much more...

Mozilla Translator

● Probably the oldest tool for Mozilla L10n

● Still maintained● Causes some

problems with file layout

Narro

● Powerful localization tool

● Translation suggestions

● Language pack creation

● Web-only

Translate Toolkit

● Collection of many small tools● Main goal: convert every L10n file to PO to

work with every L10n-Tool that supports PO files

● Requires back and forth converting to and from PO

Litmus

● Huge collection of manual tests● Good for testing problems which cannot be

tested automatically● Requires much time for testing

compare-locales - History

● Started as a small Perl script for testing localizations for missing and obsolete files and entities, only

● More was needed → rewrite as a bigger Python application

– checking for other problems, like encoding or DTD parsing problems

– statistics (e.g.: “unmodified: 500 strings”)

● Beginning with August 2008: second rewrite, based on the Silme library

Other Tools...

● Verbatim (mozootle)● Probably every bigger localization team has

made an own set of scripts and tools – And every time they start, they start with the

question: ● “How do we access and parse the L10n-files...”

What are the common problems?

● focused on the front-end● usually dependent on one data format● not easy extensible● continuous wheel reinvention:

– missing layer between apps and data

Silme - Goals

● generic l10n operations simplified to maximum● format independency● platform independency● easy extensible● make localization over time easier

Silme - Target

● L10n tools developers● Localizers with beginner programming

knowledge● Application developers● Buildsystem admins

Silme – The Name

● Silme (seel-may) is a letter in the Tengwar alphabet (Tolkien)

Silme - Features

● strong diff support● extensible input/output (file, zip, sql, cvs, hg,

svn, ...)● extensible format support (dtd, prop., po,

l20n, ...)● modular (core, diff, formats, io)● multilocale

Silme - Structure

● silme.core● silme.diff● silme.io● silme.format● silme.fp

Silme - API - silme.core

● silme.core.Entity● silme.core.EntityList● silme.core.L10nObject● silme.core.L10nPackage

Silme - API - silme.core.Entity

# properties# propertiesofflineApps.manageUsage=Show settingsofflineApps.manageUsage=Show settings

# dtd# dtd<!ENTITY netError.search.button "Szukaj"><!ENTITY netError.search.button "Szukaj">

# gettext# gettextmsgid "YaST installation source"msgid "YaST installation source"msgstr "Źródło instalacji YaST"msgstr "Źródło instalacji YaST"

id valueofflineApps.manageUsage Show settings

netError.search.button Źródło instalacji YaST

YaST installation source Szukaj

id value

entity ->entity ->

Silme - API - silme.core.EntityList

itemHistory.label Browsing History

itemHistory.accesskey B

itemPasswords.label Saved Passwords

itemCookies.label Cookies

entityList = EntityList()entityList = EntityList()

entityList.id = “sanitize.dtd”entityList.id = “sanitize.dtd”

entity = Entity(‘itemHistory.label’)entity = Entity(‘itemHistory.label’)

entity.setValue(‘Browsing History’)entity.setValue(‘Browsing History’)

entityList.addEntity(entity)entityList.addEntity(entity)

Silme - API - silme.core.L10nObject

entity

\n\n

comment

\n

entity

\n

entity

entity

\ntemporary disabled\n

entity\ntemporary disabled\n

<!ENTITY itemCookies.label <!ENTITY itemCookies.label "Cookies">"Cookies">

< ! - - < ! - - temporary disabledtemporary disabled<!ENTITY itemDisabled <!ENTITY itemDisabled “Disabled”>“Disabled”>- - >- - ><!ENTITY itemCookies.accesskey "C"><!ENTITY itemCookies.accesskey "C"><!ENTITY itemCache.label <!ENTITY itemCache.label "Cache"><!ENTITY itemCache.accesskey "a">"Cache"><!ENTITY itemCache.accesskey "a">

Silme - API - EntityList, L10nObject

● Why both?– EntityLists:

● easy and fast● no file layout preservation

– L10nObject:● preserves file layout for 1:1 reconstruction● time consuming

Silme - API - silme.core.L10nPackage

ll 10nPackage = {10nPackage = {

i d : ‘ m o z a p p s ’ ,i d : ‘ m o z a p p s ’ ,

o b j e c t s : { } ,o b j e c t s : { } ,

packages : {packages : {

‘ ‘d o w n l o a d s ’ : . . .d o w n l o a d s ’ : . . .

‘ ‘h e l p ’ : {h e l p ’ : {

i d : ‘ h e l p ’ ,i d : ‘ h e l p ’ ,

p a c k a g e s : { } ,p a c k a g e s : { } ,

o b j e c t s : {o b j e c t s : {

‘ ‘help.dtd’ : L10nObject ,help.dtd’ : L10nObject ,

‘ ‘help.propert ies ’ : L10nObject }help.propert ies ’ : L10nObject }

}}

‘ ‘p r o f l e ’ : . . .p r o f l e ’ : . . .

}}

Silme - API - silme.diff

● silme.diff.(Entity|EntityList|L10nObject|L10nPackage)Diff

● silme.core.*.diff()● silme.core.*.applyDiff()

entityDiff = entity.diff(entity2)entityDiff = entity.diff(entity2)entityListDiff = entityList.diff(entityList2)entityListDiff = entityList.diff(entityList2)

entity2 = entity.applyDiff(entityDiff)entity2 = entity.applyDiff(entityDiff)entityListDiff2 = entityList.applyDiff(entityList2)entityListDiff2 = entityList.applyDiff(entityList2)

Silme - API - silme.io

● silme.io.manager● silme.io.file|svn|cvs|hg|jar|mysql● *.getL10nPackage()● *.writeL10nPackage()

Silme - API - silme.format

● silme.format.*.Parser● silme.format.*.Structure● silme.format.*.Serializer

Silme - API - silme.fp

● silme.fp.object● silme.fp.diff

Silme - Examples

1) Count the number of entities in a directory

2) Find all entities containing the word “karta”

Silme - Future

TODO:

1)PEP08

2)Documentation

3)Stable release

compare-locales “2.0”

● Main ideas:– use all the powers of Silme in a tool that would

provide at least the same functionality like the first Python version of compare-locales (for the beginning)

– use it to ensure that Silme works

compare-locales - log

● First attempt: make the existing compare-locales use Silme as the I/O API

– It worked (somehow), but with the use of many hacks

● Final attempt: write it from scratch

compare-locales - Requirements

● L10n.ini● specific output style● L10nPackage and L10nObject statistics● filtering packages and objects● collecting errors and warnings● INI-Files● adding new features

compare-locales – L10n.ini File

● Specifies where to look for L10n-files in Mozilla source

compare-locales – Output Style

● Tree-like with short paths● As requested by a community

member:– A second one with full paths

compare-locales – Statistics

● Provide statistics for entities, files and problems regarding them

compare-locales – Filtering

● Filter unneeded elements for testing

compare-locales – Errors

● There are many things, which should not be in a L10n-file. Find and report them:

– Encoding problems:

– Duplicated entities:

compare-locales – New features

● Some of the features on previous slides, like finding duplicate entities or checking for correct encoding, are already new features

● But there is more. For example:– Checking access keys for correctness

– Comparing not only Mozilla source files, but extensions too

– Spell checking of strings

– Your_feature_could_be_here...

compare-locales – Access Keys

● Basic access keys checks can be done automatically:

– access key present in the corresponding label

– access key with only one character

– are this policies being followed: https://developer.mozilla.org/en/XUL_Accesskey_FAQ_and_Policies

compare-locales – Extensions

● Compare extensions, too:

● Useful for extensions authors to check if missing entities will not break the whole application

● And for AMO: an extension could be checked while adding it

compare-locales – New features

● How complicated is it to add a feature?– Finding duplicate entities:

● 3 lines of code

– Finding broken access keys (basic version):● ~ 5 lines

– Adding support for comparing extensions:● Only a few hours of work

Limitations

● What is not possible with Silme and compare-locales:

– Testing how the localization works when using it with the application it was written for

● MozMill can do automated UI testing

– It will not make a coffee for you ;)

Links

● Silme:– http://diary.braniecki.net/tag/silme/ (Gandalfs blog posts about Silme)

– http://hg.mozilla.org/users/zbraniecki_mozilla.com/silme/ (HG repository)

● Compare-locales:– http://pypi.python.org/simple/compare-locales/ (Python version)

– http://hg.mozilla.org/users/akalla_aviary.pl/silme-patched/ (Silme version, HG repository)

– http://mxr.mozilla.org/seamonkey/source/toolkit/locales/compare-locales.pl (Old Perl version)

Q & A

Thank you