machine translation vs. common language: effects on idea exchange in cross-lingual groups (cscw...

!!

Machine Translation vs. Common Language: !Effects on Idea Exchange in Cross-Lingual Groups

Hao$Chuan)Wang )Na#onal'Tsing'Hua'University,'Taiwan'Susan)R.)Fussell )Cornell'University,'NY,'USA'Dan)Cosley ) )Cornell'University,'NY,'USA'')'ACM'CSCW'2013'.'San'Antonio,'TX,'USA'

'

!!

Wang, Fussell, Cosley

Globalizing Idea Exchange

Ideas)distributed)across)countries)and)cultures)•  Ex.)Different)language)edi?ons)of)Wikipedia)varying)in)content)of)the)same)topics.)

•  Valuable)source)for)intelligence)and)problem)solving.)

CMC:)Opportunity)for)global)idea)exchange)•  Need)to)cross)the)language)barrier)first.)

2)

[Wang)et)al.,)2011]'

[Hecht)et)al.,)2011]'

!!


Ways for Cross-Lingual Communication

3)

Common)solu?ons:))Common)language)(e.g.,)English))or)human)transla?on.))

•  Common)language:)Second)language)to)some)individuals.)Learning)required.)

•  Human)transla?on:)Expensive)and)inflexible.)

Machine(transla-on((MT)(as)a)non$expensive,)flexible)tool)for)cross$lingual)idea)exchange.()

!!


Problems in Machine Translation (MT)�

Concern)for)transla?on)quality)•  “computer'bug”'may)become)“computer'

insect.”'•  Transla?ons)of)complex)ideas)like)

“need'to'confirm'with'her'to'see'if'there's'enough'#me'to'make'it”'can)be)unpredictable.)

))People)oYen)communicate)beZer)over)a)common,)second)language)than)MT.)

4)

English

Chinese

[Yamashita)&)Ishida,)2006;)Yamashita)et)al.,)2009]'

!!


Can MT Still Be Useful?

MT$based)communica?on)has)both)cost)and)benefit)•  Cost:(Transla?on)errors)make)message)comprehension)and)repairing)difficult.(

•  Benefit:()Producing)messages)in)na?ve)language)is)easier)than)non$na?ve)language.)

)Current)study:)American$Chinese)idea)exchange)over)MT)or)

English)as)common)language))•  H1:'Chinese)produce)more)ideas)over)MT)than)English))(MT(benefit(>(MT(cost)(

•  H2:'Americans)produce)less)ideas)over)MT)than)English)(MT(has(no(benefit(but(only(extra(cost)(

5)

!!


Experimental Design

6)

MT-mediated American-Chinese Dyad

English-mediated American-Chinese Dyad

Randomly)assign)cross$lingual)groups)to)media?on)condi?on)

64)individuals)32)two$person)cross$lingual)groups)

!!


Experimental Design

8)

“Benefits and costs for having an extra

thumb”

“Benefits and costs for having an extra

eye”

MT-mediated American-Chinese Dyad

English-mediated American-Chinese Dyad

Randomly)assign)cross$lingual)groups)to)media?on)condi?on)

Measure)Produc?vity)and)Comprehensibility)

Brainstorm)two)trials)(15)min)each)))in)counterbalanced)order)

!!


Measurement: Productivity

Produc-vity)•  Manually)translated)Chinese)contribu?ons)to)English)for)consistency)of)coding'

•  Count)number'of'original'ideas'by)manually)coding)brainstorming)conversa?ons)

(

9)

A

Howaboutsleeping,withoneeyecontac5ngwith

pillow? idea 1‐someoriginality

B

Newopportuni5esforbusiness:glassesforthe3

eyedperson. idea 1‐someoriginality

B

Ithinksleepingonyoursidewouldbecomethe

norm. idea 0‐duplicate

A Arewes5lltalkingabouttheadvan? no_idea

B Newhatswouldneedtobemade. idea 1‐someoriginality

B sure. no_idea

A ok no_idea

B haircutswouldcertainlybemodified. idea 1‐someoriginality

Idea? Original?

!!


Results: Productivity

10)

0

2

4

6

8

10

12

14

American Chinese

Idea

Pro

duct

ivity

English MT

In)English)groups,)Americans)(na?ve)speaker))produced)more)ideas)than)Chinese)(non$na?ve)speaker).)

!!


Results: Productivity

11)

0

2

4

6

8

10

12

14

American Chinese

Idea

Pro

duct

ivity

English MT Chinese)produce)more)ideas)

using)MT)than)English.))p<.05)for)the)second)trial;))p=.1)overall.))(H1:'MT(benefit(>(MT(cost))

)MT)did)not)affect)Americans’)

produc?vity.)))(H2'not'supported:'no)MT)cost)on)produc?vity))

))

!!


Measurement: Comprehensibility

Perceived(Comprehensibility)•  Asked)par?cipants)aYer)each)brainstorming)trial))how)clear)were)the)messages)to)them.)

•  “I)could)understand)other)members’)ideas.”)“I)am)confident)that)other)group)members)understand)my)ideas.”))“I)didn’t)understand)what)other)group)members)said.”)(alpha=.89))

(

12)

!!


Results: Comprehensibility

13)

0

1

2

3

4

5

6

7

American Chinese

Perc

eive

d Com

preh

ensi

bilit

y

English MT Both)American)and)Chinese)par?cipants)rated))MT)less)comprehensible)than)English.)

)MT)harms)comprehensibility)

both)to)na?ve)and)non$na?ve)speakers.)

)Consistent)with)previous)

studies.)

!!


Discussion

14)

0

2

4

6

8

10

12

14

American Chinese

Idea

Pro

duct

ivity

English MT

0

1

2

3

4

5

6

7

American Chinese

Perc

eive

d Com

preh

ensi

bilit

y

English MT

MT)benefits)idea)produc?on)by)enabling)na?ve)language)use,)though)MT)harms)comprehensibility.)

))How)does)MT)benefit)Chinese)

par?cipants’)produc?on?)•  Was)it)because)Chinese)talking)

more?))•  Or)was)it)because)Chinese)being)

more)efficient)in)producing)ideas?)

!!


Message versus Idea Production Behaviors

15)

culture

Chinese

#New

Idea

s#C

hat T

urns

#Wor

ds

Time (seconds)

#New

Idea

s#C

hat T

urns

#Wor

ds

0 180 360 540 720 900Time (seconds)

#New

Idea

s#C

hat T

urns

#Wor

ds

Time (seconds)

English

MT

#New

Idea

s#C

hat T

urns

#Wor

ds

0

2

4

6

8

10

12

05

101520253035

Time (seconds)

Compare)number(of(new(ideas(and)number(of(chat(turns:)

)MT:)~40%)chat)turns)contain)

ideas)English:)~33%)turns)contain)

ideas))Chinese)produce)ideas)more)efficiently)over)MT)than)English.))

!!


Message versus Idea Production Behaviors

cultureAmerican Chinese

#New

Idea

s#C

hat T

urns

#Wor

ds

0

2

4

6

8

10

12

05

101520253035

Time (seconds)

#New

Idea

s#C

hat T

urns

#Wor

ds

00 180 360 540 720 900 0 180 360 540 720 900

Time (seconds)

Exploratory)analysis)of)American’s)produc?on)behaviors:)

)MT:)~44%)chat)turns)contain)

ideas)English:)~35%)turns)contain)

ideas)

Americans)talked)less)over)MT)than)English.)

#New

Idea

s#C

hat T

urns

#Wor

ds

Time (seconds)

English

MT

16)

!!


Style of Communication

MT)appears)to)influence)people’s)style)of)communica?on)•  Higher)propor?on)of)ideas)in)MT,)both)to)Chinese)and)Americans.)

•  Chinese)produced)more)ideas;)Americans)produced)less)turns.)

)Incoming)messages)s?ll)have)impact.))Future)work:)People’s)prac?ces)and)strategies)for)dealing)

with)poor)transla?on)in)different)language)communi?es.)

17)

!!


Design Implications

Asymmetrical(design:(MT)benefits)produc?on,)but)harms)comprehension.))))))))For)bilinguals,)combine)MT)and)second)language)ability:)•  Translate)outgoing)messages)to)enable)produc?on)in)na?ve)

language.)•  Leave)incoming)messages)untranslated.)People)use)second)

language)proficiency)for)understanding.)

)18)

translated

untranslated

“��” “How are you”

“I’m fine” “I’m fine”

!!


Use)highligh?ng)to)direct)people’s)aZen?on)to)keywords)in)transla?ons.)

)

19)

Or, Interface Design for Comprehension�

[Gao)et)al.,)CHI'2013])

!!


Acknowledgement Na?onal)Science)Founda?on)grants)#0845351)and)#1025425)

Undergraduate)assistants)Meaghan)Frank,))Johna)Seo)and)Monica)Chen)at)Cornell)University)

Anonymous)CSCW)reviewers)for)their)valuable)comments)and)sugges?ons)

machine translation vs. common language: effects on idea exchange in cross-lingual groups (cscw...

Documents

fussell cornelluniversity

ons idea

chuanwang na

crosslingual groups

ideab newhatswouldneedtobemade

nomt0costonpro americanchinese

englishor humantransla

taiwan susanr