purewidgets toolkit
DESCRIPTION
A presentation about the initial version of the PuReWidgets toolkit for interactive public display applications.TRANSCRIPT
![Page 1: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/1.jpg)
The PuReWidgets toolkit for interac6ve public display
applica6ons
Jorge C. S. Cardoso [email protected]
Ubicomp@UMinho
May 30, 2012
1
![Page 2: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/2.jpg)
Content
• Mo6va6on • PuReWidgets features • Architecture • Applica6on models • Programming library overview • HelloWorld
2
![Page 3: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/3.jpg)
Mo6va6on Complexity of incorpora6ng interac6ve features in public display applica6ons. Too many things to answer when designing the interac6on for a public display applica6on: • What controls can we use to offer interac6ve features to users?
– No standard interac6ve controls for public displays • What interac6on mechanisms can be used to provide those features?
– Various possible interac6on mechanisms (SMS, BT naming, OBEX, Email, IM, mobile app, desktop, QR codes, touch, etc.)
– Different loca6ons may have different interac6on resources • How will those features work?
– Desktop interac6on paradigm does not generally apply – Applica6ons may not always be visible
• How can the applica6on react to different users? – The interac6on environment is mul6-‐user – We need to iden6fy/differen6ate users
• How can users tell what they can do with the applica6on? – Where are the features presented?
3
![Page 4: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/4.jpg)
Mo6va6on We can solve all those ques6ons for ad hoc applica6ons, but
• This means wasted effort in developing applica6ons – Programmers must deal with issues extraneous to the main applica6on logic
• Will certainly lead to inconsistencies in the interac6on model – Each solu6on will behave slightly differently – Will confuse users
We need a programming toolkit for interac6ve public display applica6ons!
4
![Page 5: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/5.jpg)
Toolkit
• The toolkit should provide a consistent model for user interac6on and applica6on development – But s6ll allow some diversity in solu6ons
5
![Page 6: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/6.jpg)
PuReWidgets
• PuReWidgets tries to solve these problems. • To make sure it covers a wide range of applica6ons, we looked at various interac6ve public display systems to learn – General requirements – Types of controls – Types of input mechanisms
6
![Page 7: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/7.jpg)
PuReWidgets Widget-‐based toolkit.
• A widget represents an interac6ve feature. • Is represented by class in an object-‐oriented programming model. • Applica6ons instan6ate widgets, define a callback, and receive
interac6on event through the callback • Main features
– Mul6ple, extensible, controls – Independence of input mechanisms and modali6es – Automa6cally generated graphical interfaces – Asynchronous interac6on – Concurrent interac6on – Graphical affordances – Server & client libraries
7
![Page 8: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/8.jpg)
Features Mul6ple, extensible, controls. • Allows developers to create rich interac6ve applica6ons. Controls can be extended with custom
func6onality. • We studied various interac6ve display systems to define a set of basic interac6ve controls, common to the
majority of applica6ons:
• Ac6on bugon – Trigger an ac6on in the applica6on, e.g., play a video
• Op6on selec6on – Selects among a set of op6ons, e.g., to vote
• Text entry – Sends text to the applica6on, e.g., a comment, tag, search keyword
• Download – Receives a media file from the applica6on, e.g., download poster
• Upload – Uploads a media file to the applica6on, e.g., a photo to be displayed
• Check-‐in – Says “I’m here”
8
![Page 9: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/9.jpg)
Features
Independence of input mechanisms and modali6es. • Allows developers to focus on the core applica6on func6onality and not on the low-‐level interac6on details.
• Allows applica6ons to work in heterogeneous loca6ons, with different mechanism
• Interac6on can be accomplished with various mechanisms (SMS, Bluetooth naming, QR codes, and automa6cally generated graphical interfaces).
9
![Page 10: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/10.jpg)
Features
Automa6cally generated graphical interfaces. • Provide a richer interac6on for desktop and mobile devices.
• PuReWidgets generates web-‐based interfaces for each applica6on.
• It also generates QR codes for each applica6on's feature
10
![Page 11: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/11.jpg)
Features
Asynchronous interac6on. • Allows applica6ons to receive input events that were generated when the applica6on was not listening.
• PuReWidgets provides a persistent input queue for each applica6on
• Applica6ons can request past input at any 6me
11
![Page 12: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/12.jpg)
Features
Concurrent interac6on. • Allows various users to interact at the same 6me, while providing applica6ons with iden6ty informa6on that allows them to differen6ate users.
• Every input event carries a user id (if available) – Where possible, anonymous interac6ons are s6ll “iden6fied” (more on this later)
12
![Page 13: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/13.jpg)
Features Graphical affordances. • Allows users to immediately recognize interac6ve features and input feedback.
• Controls have an op6onal graphical representa6on that can be used on the public display interface. – Applica6ons may not show the features on the public display, but they will s6ll be available
• Input feedback is also (op6onally) displayed on the public display.
13
![Page 14: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/14.jpg)
Features
Server & client libraries. • Programmers can choose to use just the server, just the client, or both libraries together.
• Allows developers to choose the best applica6on model.
14
![Page 15: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/15.jpg)
PuReWidgets Architecture
15
![Page 16: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/16.jpg)
Interac6on services • PuReWidgets service – Keeps informa6on about every widget – Generates graphical interfaces for desktop, mobile, touch plaporms • These allow anonymous interac6ons, but they generate persistent cookie-‐based random ids.
– Generates QR codes for every widget – Generates unique textural reference codes for data-‐based interac6ons (SMS, BT naming, OBEX, etc.)
• IO infrastructure – (from Instant places) – Provides low-‐level data-‐based interac6on
16
![Page 17: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/17.jpg)
PuReWidgets library
• Provides an object-‐oriented library of widgets • Hides the communica6on details with the PuReWidgets service
• Provides other features, unrelated to interac6on, such as – Applica6on “place-‐aware” storage
• PuReWidgets provides a name-‐value server datastore for easy storage of place-‐specific seqngs/state
– Skeleton admin interface for place-‐owners • To configure place-‐specific seqngs
17
![Page 18: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/18.jpg)
Local applica6on model
18
![Page 19: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/19.jpg)
Local applica6on model
• Applica6on logic on the public display (player) – Subject to the schedule 6mings
• May not be able to react (or update data structures) immediately when user interacts – But will eventually (when it is displayed again)
• When displayed, the toolkit periodically asks the service for input events – And triggers the appropriate widget event
19
![Page 20: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/20.jpg)
Remote applica6on model
20
![Page 21: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/21.jpg)
Remote applica6on model
• Applica6on logic on the server • Can react (or update data structures) immediately to user input – Even if not on the public display, the reac6on can be visible on other channels
• The toolkit periodically asks the service for input events – And triggers the appropriate widget event
21
![Page 22: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/22.jpg)
PuReWidgets Implementa6on
• Google Appengine (server) • Google Web Toolkit – GWT (client)
• Overview of the library
22
![Page 23: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/23.jpg)
Using PuReWidgets
• Hello World applica6on (local model) • Video
23
![Page 24: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/24.jpg)
Ini6al Evalua6on/Development process
• Two applica6ons developed so far (ignore the “graphic design”, please) – Youtube player, media intensive, complex graphical components (several panels that change over 6me), local applica6on model
– Everybody votes, based on display owner created content, simpler graphical component, combines remote and local applica6on models
24
![Page 25: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/25.jpg)
Ini6al Evalua6on/Development process
• Con6nuous refinement – Develop interac6ve public display applica6ons – Gain more insight about the difficul6es – Refine the toolkit to address the iden6fied problems
– Refactor the applica6ons to include the toolkit changes
25
![Page 26: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/26.jpg)
Summary
• PuReWidgets facilitates applica6on development – Provides ready-‐to-‐use interac6ve controls – Integrates various input mechanisms via an I/O infrastructure
– Generates applica6on interfaces for desktop, mobile, and QR codes
– Provides client and server applica6on development models
26
![Page 27: PuReWidgets toolkit](https://reader033.vdocuments.mx/reader033/viewer/2022051612/54bbf5244a7959a2388b458c/html5/thumbnails/27.jpg)
The end
27