EnglishMash  Usability  design  for  a  natural  mashup  

composi8on  environment  

Saeed  Aghaee  and  Cesare  Pautasso  

Faculty  of  Informa8cs  University  of  Lugano  

Switzerland  

ComposableWeb  2012,  Berlin,  Germany  

2  

Expressive  Power   Usability  

3  

UI   Mashup  Engine  

Expressive  Power  

Usability  

[A.  Morch,  1997]  4  

Mashup  Tools  Interac8on  Techniques  

•  Programming  by  Demonstra8on  •  Form-­‐based  Customiza8on  •  Visual  Programming  Languages  •  What-­‐you-­‐see-­‐is-­‐what-­‐you-­‐get  •  Spreadsheets-­‐based  •  Example  Modifica8on  •  Textual  Domain  Specific  Languages  

5  

6  

7  

8  

EnglishMash  

9  

EnglishMash:  End-­‐users  

10  

Programmers  

Non-­‐programmers  

•  Create  feature-­‐rich  components  (requested  by  non  programmers)  

•  Create  complex  mashups  (requested  by  non-­‐programmers)  

•  Reuse  exis8ng  mashups  

•  Modify  exis8ng  mashups  

•  Create  simple  mashups  

Pro.  non-­‐programmers  

•  Create  mashups  that  can  be  reused  by  non  programmers  

•  Help  non-­‐programmers  

EnglishMash:  Interac8on  Technique  

11  

Component  development  &  discovery  

Back-­‐end  development    &  modifica8on  

UI  Design  or  redesign  

(1)   (2)   (3)  

EnglishMash:  Interac8on  Technique  

WYSIWYG    +    Form-­‐based    +    ?                    +                                VL          

12  

Component  development  &  discovery  

Back-­‐end  development    &  modifica8on  

UI  Design  or  redesign  

(1)   (2)   (3)  

EnglishMash:  Interac8on  Technique  

WYSIWYG    +    Form-­‐based    +    Natural                    +              Language                  VL        Programming  

13  

Component  development  &  discovery  

Back-­‐end  development    &  modifica8on  

UI  Design  or  redesign  

(1)   (2)   (3)  

Natural  Language  is  ...  

•  The  main  Communica8on  means  between  people  •  Good  for  storytelling  •  Understandable    

But…  

•  Vague  and  ambiguous  •  Long  and  wordy  vs.  concise  formal  nota8ons  

14  

Natural  Language  Programming  

Structured   Informal  

15  

Natural  Language  Programming  

Structured   Informal  

16  

EnglishMash:  Architecture  

Scanner  

Formal  parser  +  NL  Parser  

Seman8c  Extrac8on  (Control  Flow)  

Seman8c  Extrac8on  (Data  Flow)  

Code  Emifer  (JOpera)  17  

EnglishMash:  Natural  Language  Mashup  Descrip8on  (NLMD)  

“Combine  CNN  News  and  BBC  News.  Show  the  result  on  the  table.”      

18  

EnglishMash:  Natural  Language  Mashup  Descrip8on  (NLMD)  

“when  the  map  is  clicked,  do  as  follows.  Search  tweets  around  the  loca=on,  and  combine  the  results  with  BBC  Top  News.  Finally,  show  the  result  on  the  table.”      

19  

EnglishMash:  Natural  Language  Component  Descrip8on  (NLCD)  

“when   ,  do  as  follows.  ,  and  

.  Finally .”      

20  

EnglishMash:  Programing    techniques  

21  

•  Loops:  •  While…  •  For  each  …  in  …  

•  Condi8ons:  •  If  ….  Otherwise  …  

•  Events:  •  When  …,…  

 

•  Blocks:  •  Do  as  follows  ….  

 

EnglishMash:  Programing    techniques  

22  

“  the  map  is  clicked,  do  as  follows.  Search  tweets  around  the  loca=on,  and  combine  the  results  with  BBC  Top  News.   ,  show  the  result  on  the  table.”      

EnglishMash:  Programing    techniques  

23  

“Search  tweets  for  mashup.  ”  

   

EnglishMash:  Natural  Language  Component  Descrip8on  (NLCD)  

“when   ,  do  as  follows.  ,  and  

.  Finally .”      

24  

EnglishMash:  Usability  Design  

UI  Design  

UI  Modeling  

Ac8vity  Modeling  

Use  case  Scenario  

25  

EnglishMash:  Use  case  Scenario  

NLMD author

Component selection

Develop mashup

Publish/deploy

Call for component development

Stack components

Design mashup user interface

Get immediate feedback

Receive writing aids

Develop mashup logic

Search components

26  

EnglishMash:  UI  Ac8vi8es  

publish mashups

search components

edit NLMDdesign mashup UI stack components

generate mashup UI

:publish

output : EnglishMash

Widget?Yes

27  

EnglishMash:  UI  Model  

resultskeywords

searchcharacteristics drag

stack

remove

feedbackwriting aids NLMD outputpublish

drop

EnglishMash

Search

OutputText field

Stack

28  

EnglishMash:  Main  Features  

29  

•  Immediate  feedback  •  Live  output  •  Error  

•  Auto-­‐comple8on  •  Data  Flow  sugges8on  •  NLCD  comple8on  •  NLCD  Sugges8ons  

•  Highlight  •  Data  Flow  •  Tex  to  Component  

EnglishMash:  User  Interface  

30  

EnglishMash:  Evalua8on  

31  

•  Auto-­‐comple8on  helps  them:  •  Learn  the  language  •  Effec8vely  use  the  language  to  implement  

mashups  •  Highlights  help  them:  •  not  to  get  lost  

But..  •  It’s  hard  to  choose  among  the  NLCD  sugges8ons  •  It’s  hard  to  debug  a  mashup  

EnglishMash:  Conclusion  

32  

•  Expressive  power  and  usability  trade-­‐off    •  NLP  is  understandable  by  end  users  •  NLP  problems  can  be  addressed  through:  •  Auto-­‐comple8on  •  Text-­‐based  visualiza8on  

EnglishMash:  Future  work  

33  

•  Smart  in-­‐line  component  search  •  Smart  component  and  composi8on  sugges8on  •  Enable  debugging  •  Collabora8ve  mashup  development  •  Full-­‐featured  UI  Design  toolbox  •  Mul8-­‐UI  mashups  •  Mobile  mashup  development  

EnglishMash  Usability  design  for  a  natural  mashup  

composi8on  environment  

Saeed  Aghaee  and  Cesare  Pautasso  

Faculty  of  Informa8cs  University  of  Lugano  

Switzerland    Email:  saeed.aghaee@usi.ch      Homepage:  hfp://www.saeed-­‐aghaee.info/  

ComposableWeb  2012,  Berlin,  Germany