UniversalSpeech Translation Advanced Research consortium(U-STAR)

Eiichiro Sumita, NICT.

4/11/2015 1© NICT

SPEECH TRANSLATION

4/11/2015 2© NICT

Client application of Speech Transaltion

Server of Speech Transaltion

Mechanism of Speech Translation

ASRConvert speech

into text

MTConvert Japanese text into English

駅はどこ

ですか

Where is the station

SSConvert text into speech

① Input speech is sent to server ② Server returns translated foreign speech

4/11/2015 © NICT 3

Speech Translation Server with speech and parallel corpus

4/11/2015 © NICT 4

HISTORY OF SPEECH TRANSLATION RESEARCH

• First working demo system

[1992]

• First smartphone application, VoiceTra

[2010]

Research was born in Japan [1986]

• Narita Airport

• au, docomo

• ATR-trek, FEAT

• Google

• Microsoft/Skype

Commercial systems

[2011~]

4/11/2015 © NICT 5

The world first experiment in 1992

4/11/2015 © NICT 6

Network-based Speech Translation System with Smartphone 2010.7, VoiceTra

7

Part1: how to usePart2: don’ts

4/11/2015 © NICT

S2S TOWARD 2020 TOKYO OLYMPICS

4/11/2015 © NICT 8

Global Communication Plan（GCP）

• 平成26年4月11日

– 総務大臣が記者会見にて発表

• Mission

世界の「言葉の壁」をなくす

• Vision

（１）グローバルで自由な交流の実現

（２）日本のプレゼンス向上

（３）東京オリンピック・パラリンピックでの「おもてなし」

• Action

– 関係する企業や関係省庁等と連携、協力しながら、まずは６年間のロードマップを共有して取り組む

• （プロジェクト１）病院、商業施設、観光地等における社会実証

• （プロジェクト２）多言語音声翻訳の対応領域、対応言語を拡大するための集中的な研究開発投資

• （プロジェクト３）2020年東京オリンピックにおける社会実装

9© NICT2014/11/07

http://www.soumu.go.jp/main_content/000285578.pdf

②おなかが痛い

①我的肚子疼

③所服用的药吃完了

④いつも飲んでいる薬がなくなってしまいました

②眺めのよい部屋に変えてもらえますか

①Anh có thể đổi cho tôi một phòng với khung cảnh.

①お客様の鞄は、品川駅で預かっています。

②I keep your bag at Shinagawa station.

中国語 ベトナム語

英語インドネシア語

①Apakah andamemiliki warna lain?

③お調べします。赤色があります。

④Mari saya periksa. Ada merah.

②色違いはありますか。

ショッピング

医療 ホテル

交通

We have doubled the reserachers!

2014/11/07 10© NICT

An extension of S2S:Medical Interpreter

Japanese staff English patient

4/11/2015 © NICT 11

企業の動き

2014/11/07 © NICT 12

Panasonic日経新聞（H26.7.23）

NTT docomoの機械翻訳の合弁会社設立プレスリリース（H26.9.29）

U-STAR for speech translation of MORE LANGUAGES

4/11/2015 13© NICT

Tourism domain demonstrates high-quality translation among many languages

14

0102030405060708090

100

tran

slat

ion

acc

ura

cy

4/11/2015 © NICT

Plug-in feature of Network-based Speech-to-Speech Translation (U-STAR)

Communication between Different Language Speakers

ASR Module

Thai

ASR Module

Japanese

S2ST Server S2ST Server

Japanese Speaker

S2ST Client

ThaiSpeaker

S2ST Client

MT Module

Japanese → Thai

S2ST Server

TTS Module

Thai

S2ST Server

MT Module

Thai → Japanese

S2ST Server

TTS Module

Japanse

S2ST Server

Network

For example, you can

easily add Myanmar

speech recognition

to many-to-many speech

translatorof U-STAR

4/11/2015 © NICT 15

Based on Standardized Communication Protocol

F.S2STreqs H.S2STarch

TitleFunctional Requirements for

Network-based S2ST

Architectural Requirements for Network-based S2ST

Recomm-Endation

(2010)

164/11/2015 © NICT

Members joined in 2014

30 Institutes from 25 countries/regions formed under the “MoU”

U-STAR is world-wide consortium

4/11/2015 © NICT 17

U-STAR covers 95% (orange areas) of the world’s official languages

4/11/2015 © NICT 18

4/11/2015 © NICT 19

LET’S TRY

4/11/2015 20© NICT

Please download the free app from AppStore or GooglePlay

4/11/2015 21© NICT