html5 haptics 1 html5 haptics standardization mohamed mansour [email protected] haptics...

HTML5 Haptics 1

HTML5 Haptics Standardization

Mohamed [email protected]

HTML5 Haptics Mohamed Mansour 2


Vision HTML5 Haptics

Plugins Google Chrome Extensions

Prototype Architecture Future

1 2 3

4 5

6 7






1 2 3

4 5

6 7


Vision

• Haptics Integration into HTML5• Rich JavaScript API to communicate to Haptics• Cross platform • Haptic device independent• Standardized in W3C






1 2 3

4 5

6 7


HTML51991 HTML

1994 HTML 2

1996 CSS 1 JavaScript

1997 HTML 4

1998 CSS 2

2000 XHTML

2002 Tableless

2005 AJAX

2009 HTML 5

INNOVATION


What is HTML5?


HTML5 New features

• Storage• Application Cache• Web Workers• Web Sockets• Notifications• Drag and Drop• Geolocation• Semantic Tags• Link Relations, and Microdata• ARIA

• New form fields• Audio and Video• Canvas and SVG• Font Text Support, • Opacity, Rounded Corners• Shadows, Gradients• Backgrounds, Transitions• Transformations, Animations• Device• A lot more!






1 2 3

4 5

6 7


Haptics

• Technology that interfaces with the user through the sense of touch [definition]

• The process of recognizing objects through touch [another definition]


Haptic Devices

• Omni

• Novint

• Gloves

• Wrist band

• Jacket

• Way more …


How they work?

• Robotic• Sensors• Feedback

• Ex:– You send forces to the device, and it magically

moves. – You request positional data from the device.






1 2 3

4 5

6 7


Plugins

• Browser plugins is the only way you can allow the user to communicate to native code.

• Current state of HTML is still limited … look at Flash, why did it exist?

• Can do almost anything!• But very very very Dangerous!


Plugins

• Active X– Microsoft Internet Explorer

• NPAPI (very low-level)– Google Chrome– Firefox– Opera– Safari


NPAPI

• Very long name …. Netscape Plugin Application Programming Interface.

• Very powerful!• It is just a single library (.dll, .so, .plugin) file

that gets loaded.• You just need at least 3 Library Exports:

– NP_GetEntryPoints– NP_Initialize– NP_Shutdown


NPAPI Architecture

<embed>

Scriptableto

NPObject*

Plugin

DLL EXPORTS

LOGIC

Browser APINPN_*

5

4

1

2

3






1 2 3

4 5

6 7


Google Chrome

• Multi Process Browser– Tab, Plugins, Renders, Extensions, Browser

• Very fast JavaScript and Rendering• GPU Accelerated Canvas and WebGL• Secure Sandbox, for Plugins• Very fast at implementing HTML5 features.• Open Source browser






1 2 3

4 5

6 7


Google Chrome Extensions

• Small software that can modify and enhance the functionality of Google Chrome.

• Using web technologies, HTML, JavaScript, and CSS.

• Easily expose NPAPI functionality.• Separate Process


Chrome Extension Basics

• Background page– Lengthy process, long lived, singleton.

• Browser Actions / Page Actions• Desktop Notifications• Override Pages• Browser Interaction (Bookmarks, Cookies,

History, Events, Management, Windows, Tabs)• Plugin Interaction.






1 2 3

4 5

6 7


Prototype

• HTML5 Device Tag <device>

• Interface

• Type


Device Tag

• Still in implementation (research phase)• It can change! Add a Haptics “type”Dreamy Ex:<p> To start, Select a haptic device: <device type=haptic onchange="update(this.data)"></p><script> function update(stream) { console.log(stream.data); } </script>


Prototype , Needed?

• Proof of concept needed to prove that Haptics needs love! And the future of computer human interactions!

• Rescue …


Prototype, Chrome Extension

• How do we do it right now!– HTML5 (Web Workers)– CSS3, Canvas (WebGL Context Graphics)– Chrome Extension

• Dedicated Thread (Background Page)• Haptics Controller (Browser Action)• Page Interactions (Page Action)• Accessibility (ARIA)

– NPAPI • Haptics Plugin communicates to Chrome Extension• C++ extreme (scene graph, triangles) collision detection algorithms






1 2 3

4 5

6 7


Architecture

• High Level

HTML5

C Plus Plus

Novint Device


Architecture Plugin

Plugin Module

Scripting

Bridge

Plugin Gate

Utils

Haptics Service

Haptics Device

Haptics Signal

NP_GetEntryPoints

NP_Initialize

NP_Shutdown


Glimpse of Plugin Code


Architecture, Extension

• Browser Action to act as the controller

• Background Page, to allow the plugin to live forever.

• JavaScript for Some Collision Detection, Haptic Loop, and Rendering Loop.


Framework

• O3D JavaScript API for creating rich and interactive 3D application in the broswer using Canvas, WebGL context.

• Haptics API– void startDevice();– void stopDevice();– void sendForce(double[3]);– double[3] position;– addEventListener(‘servo’, function(Haptic data));


Open Source’dRelease it to the public! That is how I learned how to code, give back to the community! You can fork it from GitHub from here:http://github.com/mohamedmansour/haptics-chrome-extension

http://github.com/mohamedmansour/haptics-chrome-extension


Working Prototype






1 2 3

4 5

6 7


Future

• Present this to the W3C Developer Group• Help Google Chrome complete <device> tag

implementation.• Implement the Prototype again using the

<device> tag.• Implement webpage interactions, like

annotated YouTube Videos to feel video touch.


Done! Thank you, questions?




1 2 3

4 5

6 7

html5 haptics 1 html5 haptics standardization mohamed mansour [email protected] haptics...

Documents