xisl: content-independent, device-independent mmi description language tsuneo nitta graduate school...
TRANSCRIPT
![Page 1: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/1.jpg)
XISL: Content-Independent, Device-Independent MMI Description Language
Tsuneo Nitta
Graduate School of Tech., Toyohashi Univ. of Tech.
![Page 2: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/2.jpg)
Issues on Interaction with Web
■ The coming of heterogeneous devices
and interaction manner
• access by : PC, Mobile phone, PDA, AutoPC, DTV, …
• interaction with: modality-rich terminals, mixed initiative, ...
![Page 3: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/3.jpg)
Ex. On-line Shopping Devices - order goods at Kiosk ..... touch panel
- confirm the order with a mobile phone ..... voice- change the order at home ..... DTV remote terminal
XML Document Server(Contents ; View ; Interaction)
Our Purpose: Seamless Services over Internet
![Page 4: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/4.jpg)
XML Document Server(Contents ; View ; Interaction)
How to do it?
<xhtml ........
<head>
VoiceXML
</head>
<body>
........
CSSCSS
<xhtml .......
Form
<salt: ......
![Page 5: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/5.jpg)
CSS
XISL: Extensible Interaction-Sheet Language
CSS
<xhtml .......
Form
<xisl>
<dialog>
<exchange>
........
contents-
independent
<xhtml .......
Form
VoiceXML
........
remodels VoiceXML to meet MMI requirements
![Page 6: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/6.jpg)
XISL: Device-Independent - 1/3
no modality-specific tag : input modality
<input type=“speech” event=“recog” target=“grammar.gxml”
match=“/grammar/goods” return=“goods_name”>
<param name=“mode”>data</param>
</input>
<input type=“dtmf” event=“push” target=“grammar.gxml”
match=“/grammar/goods_select”return=“goods_name”>
<param name=“mode”>digit</param>
</input>
...........
![Page 7: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/7.jpg)
XISL: Device-Independent - 2/3
Front-end Module
<input ….. >type: speech
event:recognize recognizegrammar rule:
introductionreturn: word1
<input ….. >
type: pointing
:
ASR
Pointing
Keyboard
Hello!
Hellowor
d1: H
ellogr
amm
ar
type: speech
event: recognize
target: ***.xml
match: /page
return:
word1:Hello
Dialog Manager
![Page 8: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/8.jpg)
XISL: Device-Independent - 3/3
no modality-specific tag : output modality
type event modewindow open, close, …speech play, stop, … TTS, rec_speech, …video play, stop, …audio play, stop, …agentmove, speak, …
![Page 9: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/9.jpg)
MMI Description Language: XISL
- is content ( & view ) -independent.
- has no modality-specific tags that enable XISL device-independent.- has MMI control tags and attributes.
→ • realizes seamless services over internet and reuse of XML documents. • meets modality-rich services and avoids patent issues.
![Page 10: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/10.jpg)
XISL enables combinatorial usage of modalities.
Sequential Parallel
adding a new modality
Alternative
![Page 11: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/11.jpg)
<?xml version=“1.0” encoding=“Shift-JIS”?><!DOCTYPE xisl SYSTEM “xisl.dtd”><xisl version=“1.0”> <head>.........</head> <body> <dialogue id = “WAP_phone”> <exchange> <operation comb = “alt”> <input type=“dtmf” event=“push” target=“/d_gram.gxml” match=“/d_gram/goods_select” return=“number”> <param name=“mode”>digit</param> </input> <input type=“speech” event=“recognize” target=“/gram.gxml” match=“/gram/goods_select” return=“goods_name”> <param name=“mode”>data</param> </input> < /operation >
XISL: example -1/2
![Page 12: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/12.jpg)
< action comb = “seq” > <output type=“speech” event=“play”/> <param name=“speaker”>nancy</param> <param name=“speech_text”> You selected No. One. < /output > < call dialogue_name=“2”/ > < /action > : : :
XISL: example -2/2
![Page 13: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/13.jpg)
Implementation -1/4
XISLXMLXSL
Inp
ut
Mod
alities
Ou
tpu
t M
odalitie
s
Document Server
Interpreter Interpreter
Dialogue
Manager
Dialogue
Manager
Input Integrator
Input Integrator
Response Controller
Response Controller
System Architecture
![Page 14: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/14.jpg)
Dialogue Control
<output>
I/O profile
XML editing
transition
<action>
Op. Control
GLRparser
Input Integration
<input>
user operation
<operation>
Inte
rpre
ter
Start Dialog Dia
log M
anager
Action
Execution
Implementation -2/4 D
ocu
men
t Server
Do
cum
ent S
erver
Fro
nt - en
d
Fro
nt - en
d
Do
cum
ent B
uffer
![Page 15: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/15.jpg)
Dialogue Control by Using GLR TBL
REDUCE-25REDUCE-14
SHIFT-53SHIFT-42
ACCEPT1SHIFT-2SHIFT-1SHIFT-30EXCH1BUYGOODScompletetimeoutgoods_spbuy_btn
GOTO TBLACTION TBL INState
0 SHIFT-3
3 SHIFT-5
5 REDUCE-2
Implementation -3/4
![Page 16: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/16.jpg)
①
②
③
Implementation -4/4
![Page 17: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/17.jpg)
XISL Tree View XISLの構造をツリー形式・・・
で表示Interaction Window
生成したXISLを表・・・示
Interaction Builder - binding MMI -1/4
![Page 18: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/18.jpg)
Interaction Builder - binding MMI -2/4
![Page 19: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/19.jpg)
<input type="touch" event="click" match="/OLS/goods_list/category/goods[@id='23']/object/picture[1]"/>
Interaction Builder - binding MMI -3/4
![Page 20: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/20.jpg)
<output type="speech" event="agent-speech"><![CDATA[<param name="agent-name">Merlin</param><param name="speech-text">このコートは着心地抜群です</param>]]>
Interaction Builder - binding MMI -4/4
![Page 21: XISL: Content-Independent, Device-Independent MMI Description Language Tsuneo Nitta Graduate School of Tech., Toyohashi Univ. of Tech](https://reader036.vdocuments.mx/reader036/viewer/2022062417/5518bdf6550346b31f8b5446/html5/thumbnails/21.jpg)
Future Work
- XHTML + XISL
- Implementation of 3GPP applications
- MMI Requirements: more complicated usage of MMI
- Rapid-prototyping: GUI-based developer support system (IPA project)
http://www.vox.tutkie.tut.ac.jp/XISL/XISL-E.html