level: all about tim, interknowlogy & actus… interknowlogy () tim huckaby, founder/chairman (...
Post on 20-Dec-2015
225 views
TRANSCRIPT
Using Natural User Interface (NUI) Technologies to Improve User Experience (UX)
Tim HuckabyFounder / Chairman, InterKnowlogy & Actus SoftwareMicrosoft RD & MVP
Level: All
About Tim, InterKnowlogy & Actus…InterKnowlogy (www.InterKnowlogy.com)
Tim Huckaby, Founder/Chairman ([email protected]) Custom Application Development / Consulting / Software & Systems Engineering Firm headquartered in
Carlsbad, CA Design, Architect, Build and Deploy Enterprise Class Applications Industry Experts:
Most of the employees are published, MVPs, RD, Speakers, etc. Microsoft .NET Application development since 2000 Microsoft .NET Rich Client Pioneers / Industry Leaders Microsoft Touch Application Development Leaders (Surface / Windows 7 / Silverlight 3) Information Worker Solutions (VSTO & SharePoint) Integration / Messaging, B2B / B2C, Wireless / Mobility Leading Edge: Surface, WPF, Silverlight, Win7 touch, SharePoint, VSTO Cutting Edge Solutions on emerging Microsoft technologies Largest Client: Microsoft
Actus Software (www.Actus-Software.com)• Tim Huckaby, Founder/CEO ([email protected])
• Software Product Company, Coming soon…
A little bit about me and why I am here…
Touch Capable Computing: Using Natural User Interface (NUI) Technologies Improve The User Experience
Multi-Touch Software built in Microsoft .NET: WPF, Surface, Silverlight, & the phone…
Hardware & Software Platforms for the Natural User Interface (NUI)
NUI enabled applications used to improve the communication, education and user experience across the software continuum.
Presentation Agenda
The 3 Manifestations of NUI: Multi-Touch, Gestures, Neural
Demos, Demos, Demos
About the Computer I am Using Today…
Acer Aspire 1420PGive away “free” to all attendees at the 2009 PDCOne year old…Price in October-2009: $420Price in November-2009: $350
Multi-Touch ScatterView Photos Application
Tim HuckabyFounder, InterKnowlogy & Actus SoftwareMicrosoft RD & MVP
DEMO
The InterKnowlogy “Firebrick” Silverlight Multi-Touch ScatterView ControlWhat is it?
It’s a Silverlight Control for Intuitive & Comprehensive Multi-Touch Object Manipulation for your Web Applications.
Download Site: http://firebrick.interknowlogy.com/
Example Problem(s) it solves:Silverlight’s Touch API
Is rudimentary at best. Basically the API gives you “I know something has been touched; but, I don’t know what or where.”Is difficult; takes a ton of programming to implement at any significant level
The Silverlight Multi-Touch ScatterView ControlMakes building a Multi-Touch capable application “Brain-dead simple” in a matter of minutesContains all the “heavy lifting”: thousands of lines of calculus and trigonometryContains a sample application and detailed “How-to” documentation
Installation Notes: Free Developer and Low Cost ($99) production versions available
Usage Notes: Multi-Touch is supported in SL3 and above and currently only on Windows 7
Touch Capable Computing Devices are Not New
Past Future (Now)Present
1972: PLATO IV Touch Screen Terminal The Microsoft Surface Windows 7 Multi-touch
What is Relatively New is: Inexpensive Touch Capable, Non Proprietary
Hardware A Multi-Touch Capable Windows OS A Consistent Multi-Touch API for .NET across 3
Windows Hardware/Software Platforms Developer Productivity in Touch Capable Apps The Forecast of Touch Capability in Every New
Computing Device in the Short Term
Tim HuckabyFounder / Chairman, InterKnowlogyFounder / CEO, Actus SoftwareMicrosoft RD & MVP
Multi-Touch Distributed Computing (remote object manipulation and telestration)Microsoft Surface and Windows 7
VIDEO
History of Surface
2003The first prototype is presented to Bill Gates
2004A dedicated group is formed,
and more than 85 prototypes are developed
2007Microsoft Surface is unveiled to the world
2001The concept of an interactive table is born
2005The look and feel of Surface is finalized
2011?
Single Touch vs. Multi-Touch
You get single touch “for free” in Windows 7Almost everywhere in Windows 7, a touch = a mouse click
Multi-Touch is the capability to recognize 2 or more fingers on the screen at once.
The iPhone made the “pinch gesture” a cultural norm in usabilityVernacular on hardware capability: “4 touches”
TOUCHHARDWARE IMPLEMENTATIONS
CapacitiveInfraredResistive
Sample Hardware
CapacitiveHP Touchsmart tx2N-trig (2-10)
InfraredHP Touchsmart IQ816NewWindow(2)
CapacitiveDell LatitudeXT2N-Trig (2-10)
How to touch
Resistive Think Push Down and drag
How to touch
Capacitive Think Electric Impulse
Common hardware implementations
CapacitiveiPhone / iTouch – Capacitive multi-touch cell phone / PDA
HP TouchSmart TX2 – Dual touch capacitive laptop screen.
Dell Latitude XT – Multi-Touch tablet using N-Trig capacitive technology
MacBook Pro – Capacitive Multi-Touch pad with traditional click
N-Trig – Capacitive monitor overlay vendor (Windows 7 Drivers)
How to touch
Infrared Think laser pointer(s)
Common hardware implementations
InfraredSurface – Infrared diffused illumination multi-touch “table” device
Perceptive Pixel – Frustrated total internal reflection multi-touch panels
HP TouchSmart – Infrared dual touch bezel all in one computer
NextWindow – Infrared Multi-Touch bezel vendor (Windows 7 Drivers)
NUI GESTURES
• Tap• Flick/Fling/Scroll• Pinch/Spread• Drag
Tim HuckabyFounder / Chairman, InterKnowlogyFounder / CEO, Actus SoftwareMicrosoft RD & MVP
Surface Craps
VIDEO
MULTI-TOUCH SOFTWARE IMPLEMENTATIONS(ON THE MICROSOFT PLATFORM)
Windows Desktop: WPFSurface: WPFBrowser: SilverlightPhone: The Windows Mobile 7 phone
NUI: The Natural User Interface
Multi-Touch Scenarios
Touch Scenarios and Windows 7Developer Platform At the root is the touch developer platform that
exposes touch APIs for any application
UI Enhancements Focusing on core scenarios, many parts of the core UI
optimized for touch experiences
Gestures Multitouch gestures enable consistent panning and
zooming in most applications
Applications Powerful Multitouch-focused applications to ship
separately from Windows 7
Key Scenarios for Multitouch
Focused on Consumption
Navigating and consuming the Web
Reading and sorting email Viewing photos Playing casual games Consuming music and
video Navigating files and
arranging windows Using Microsoft Office
Applications
VitruView – Badge AuthenticationMicrosoft Surface
VitruView – 3D manipulationMicrosoft Surface
VitruView – Adding annotations with the Virtual Keyboard in 3D
Patient ManagementMicrosoft Surface
Bedside Machine ManagementMicrosoft Surface
Hospital ManagementMicrosoft Surface
Molecule ViewerMicrosoft Surface
Silverlight… “WPF Light”
Interactive experiences for devices, PC, & Mac platforms (and others…)Strict subset of Windows Presentation FoundationLightweight download (4.2mbs)Programming model uses XAML + Jscript + WPFAccess to WPF toolingRudimentary Touch Capability in Version 3.0
Lightweight WPF runtime
<XAML> <Button Width="100px"> OK <Button.Background> LightBlue </Button.Background> </Button></XAML>
XAML-based Tools
Eclipsys Maestro: Hospital Management
Eclipsys Maestro: Hospital Management
Tim HuckabyFounder / Chairman, InterKnowlogyFounder / CEO, Actus SoftwareMicrosoft RD & MVP
EMN8 & REZN8
CUSTOMER
Tim HuckabyFounder / Chairman, InterKnowlogyFounder / CEO, Actus SoftwareMicrosoft RD & MVP
demofrom our Marketing VP Madeline Pantalone: “It's okay for you to use application for demonstration as long as you are sure to preface it with the phrase: "This is test data only. Products you see may have been created for testing purposes only."
NUI: Games as a Tool for Learning
Learning made Fun / EntertainingProven as a learning tool beyond just ChildrenComputer based Games are a significant part of popular cultureEducational Video Game Curriculums
MIT, UC Irvine, Minnesota, Mich State, etc.
Tim HuckabyFounder / Chairman, InterKnowlogyFounder / CEO, Actus SoftwareMicrosoft RD & MVP
NASA: Mars Bound
DEMO
XBOX Kinect (Natal)
Motion Based Interface$149.99Availability: November 4th 40 million Xbox 360 owners RGB camera, three sensors
(one depth sensor, one motion sensor, and one multi-array microphone)
tracks full-body movement, facial expression, and voice with precision Navigate the entire Kinect Hub interface on your Xbox 360 without touching a conventional 360 controller.While you're watching a movie, you can verbally instruct the Xbox to stop, fast-forward, rewind, exit out, etc. by calling out easy commands like, "Xbox. Fast-forward." or "Xbox. Pause.“Kinect sensor bar scans and monitors 1 million points on the body
XBOX Kinect (Natal)
Video• Tim Huckaby• Founder / Chairman, InterKnowlogy• Founder / CEO, Actus Software• Microsoft RD & MVP
The Bad News:Not many applications at launchNot really any applications at launch; just games“little kid games…”No exposed API at launch, plus the legal restrictions of “hacking it”
No building your own appsNo integration with any of the Microsoft StackNo vertical solutions….for the immediate future
XBOX Kinect (Natal)
Neural Interfaces
Conscious and non-conscious Interfaces
• High fidelity EG acquisition device• No conductive gel, etc.• Non-invasive• Use Case Examples:
• Conscious: mind manipulation of objects• Non-Conscious: Ranking music
• First target market: gaming• $300!!!
Video• Tim Huckaby• Founder / Chairman, InterKnowlogy• Founder / CEO, Actus Software• Microsoft RD & MVP
C++ Header files and Import Libraries
…which means…
Wrap the SDK in managed C++ thereby exposing a the API to a C# application It ships with sample C# codeThat is very powerful…
Emotiv SDK
Multi-Touch CapableCommon gestures of touch
Gesture CapableMotion Based InterfaceTracking of full-body movement, facial expression, and voice with precision
Neural CapableConscious and non-conscience interfacesNon Invasive
NUIThe Natural User Interface
Summary We are at the Cusp of the NUI computing revolution There are still many “standards” yet to be adopted Multi-Touch computing will become a mainstream aspect of NUI and
consequently of software development Paradigm Shift from GUI to NUI (Natural User Interface)
Touch capable computing changes the way you design systems and differentiates your applications
Shifting from desktop applications to multi-touch based applications will be a cultural change like the mouse was
There are other components of NUI other than just touch… Neural & Gesture based interfaces
More info on InterKnowlogy:www.InterKnowlogy.com Silverlight.InterKnowlogy.com
More info on Actus Software:Coming soon….
About Tim Huckaby…Founder/Chairman, InterKnowlogyFounder/CEO, Actus SoftwareMicrosoft® Regional Director – Southern CaliforniaMicrosoft® .NET Partner Advisory Council Founder / MemberMicrosoft® MVP - .NET INETA Speaker – International .NET Users Group AssociationBoard Member: Association for Competitive TechnologyBoard Member: Dev ConnectionsAuthor / Speaker
Tim HuckabyContact me: Tim Huckaby
E-mail: [email protected]
Phone: Office: 760-444-8640Mobile: 619 990 9200
Blog: http://team.interknowlogy.com/blogs/timhuckabyFaceBook:http://www.facebook.com/Tim.Huckaby.CarlsbadLinkedIn:http://www.linkedin.com/in/timhuckaby
PowerPoint Guidelines
Font, size, and color for text have been formatted for you in the Slide MasterThis template uses Segoe Light a standard font included in Office 2007, Office 2010, Windows Vista and Windows 7Use the color palette shown belowSample FillSample FillSample Fill
Sample FillSample FillSample Fill
PowerPoint TemplateSubtitle color
Example of a slide with a subheadSet the slide title in “All Caps”Set subheads in “Sentence case”Generally set subhead to 36pt or smaller so it will fit on a single lineThe subhead color is defined for this template but must be selected; In PowerPoint 2007/2010, it is the fourth font color from the left
Hyperlink color: www.microsoft.com
Chart Example
Category 1Category 2
Category 3Category 4
0
1
2
3
4
5
6
Series 4
Series 3
Slide for Showing Developer’s Software Code
Use this layout to show software codeThe font is Consolas, a monospace fontThe slide doesn’t use bullets but levels can be indented using the “Increase List Level” icon on the Home menu
Demo Title
NameTitleGroup
demo
Video Title
video
Partner Title
NameTitleCompany
partner
Customer Title
NameTitleCompany
customer
Announcement Title
announcing
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to
be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS
PRESENTATION.