reed townsend program manager microsoft corporation anson tsao program manager microsoft...
Post on 01-Apr-2015
232 Views
Preview:
TRANSCRIPT
Developing Multi-touch Applications
Reed TownsendProgram ManagerMicrosoft Corporation
Anson TsaoProgram ManagerMicrosoft Corporation
PC03
Multi-touch Is Here!
Hardware Multi-touch capable machines in
market today, multi-touch appears in a broad set of form factors
Software Windows 7, PDC partner
demonstrations & Touch platform announcement, .Net 4.0
Consumers New scenarios, next wave of user
experience, high “WOW” factor
Multi-touch in Action
Reed TownsendProgram ManagerWindows
demo
Agenda
Introduction Touch scenarios, “Good, Better, Best” model,
and developer roadmap Platform details:
Native Win32 gesture and touch support Manipulations and Inertia WPF support
UX guidelines for touch applications Where to get started - SDK, drivers, etc Wrap up
Touch Scenarios And Windows 7
We’ve identified several key scenarios for multi-touch:
Navigating and consuming the web
Reading & sorting email
Viewing photos
Playing casual games
Consuming music and video
Navigating files & arranging windows
Using Office applications
All focused on consumption
That led to four areas of investment: Developer Platform: At the root is the
touch developer platform that exposes touch APIs for any application
UI Enhancements: Focusing on the core scenarios, many parts of the core UI have been optimized for touch experiences.
Gestures: Multi-touch gestures have been added to enable consistent panning and zooming in most applications.
Applications: A set of multi-touch focused applications that demonstrate the power of touch. These will ship separate from Win7.
A Structured Approach To Adding Touch Support To Your Application
Which application scenarios get the biggest benefit from touch? Content consumption, mobile, media
and entertainment
Decide which tier of investment is appropriate: Good – Free support, UX tweaks Better – Gesture support, touch-friendly UX Best – Touch-optimized experience
Surface Hardware
Windows 7
Surface Hardware
Windows VistaWindows 7
NativeWin32
Application
Touch Development Roadmap
WPF 3.5
Surface SDK1.0
Managed Wrapper and Interop
Managed Wrapper and Interop
WinForms Application
Windows 7 ReleaseNET 4.0 / Surface 2.0 Release
WPF 3.5 SP1
WPF 4.0
Surface SDK 2.0
WPF Application
Surface Application
Multi-Touch Controls
Multi-Touch API
Surface Multi-Touch
Controls & API
Multi-Touch API
Multi-Touch API and Controls
Touch Platform Overview
Good Better BestAPIs For Free!
•Panning/zoom gestures•Right click gesture
•Gesture notifications•Pan/zoom/rotate/etc
•Raw touch data•Manipulation and
Inertia processors
Native Win32
•Controls with standard scrollbars
•WM_GESTURE message •WM_TOUCH•COM based
Manipulation and Inertia Processors
WPF •WPF 4.0 pan support in ScrollViewer
•Gesture events• Inertia configuration
• Touch events•Manipulation and Inertia
Processors
WinForms •Controls with standard scrollbars
•WM_GESTURE message• P/Invoke
•Manipulation and Inertia Processors in Microsoft.Ink.DLL•Real-time Stylus or Ink
Collector
Windows 7 Gestures & WM_GESTURECOMMAND
Reed TownsendProgram ManagerWindows
demo
WM_GESTURECOMMAND
WM_GESTURECOMMAND Better: Good gesture support is key Window message conveying gesture notifications
and information to applications Why use it? Richer than legacy notifications,
smoother more responsive scrolling and zooming
Compound gestures Coalescing Message bubbling and capture Legacy handling & SFP
WM_GESTURECOMMAND
BOOL WINAPI GetGestureCommandInfo( __in UINT uMsg, __in WPARAM wParam, __in LPARAM lParam, __in LPARAM lExtraInfo, __inout PGESTUREINFO pGestureInfo
);
typedef struct _GESTURECOMMANDINFO {UINT cbSize;DWORD dwFlags;DWORD dwCommand;DWORD dwArguments;POINTS ptsLocation;
} GESTURECOMMANDINFO, *PGESTURECOMMANDINFO;
WM_GESTURECOMMAND
Interpreting gestures:dwCommand dwArguments ptsLocation
Pan Distance between contacts
Current center of gesture
Zoom Distance between contacts
Current center of gesture
Rotate Absolute angle on rotate start, delta on updates
Current center of gesture
Two-finger tap NA Current center of gesture
Right click gesture NA First finger down
WM_GESTUREFuture Windows 7 Build
Minor API surface changes: Naming: WM_GESTURE, GESTUREINFO,
GetGestureInfo(), etc Additional fields in GESTUREINFO
Added gesture configuration support: WM_GESTURENOTIFY GESTURECONFIG struct Set/GetGestureConfig()
Single-finger Panning Future Windows 7 Build
Single-finger panning (SFP) provides an even easier way to pan where it is natural
How to control SFP in your application: SetGestureConfig() for GC_PAN Specify GC_PAN_WITH_SINGLE_FINGER_VERTICALLY
and/or GC_PAN_WITH_SINGLE_FINGER_HORIZONTALLY
If you aren’t getting the SFP default: Consider standard scrollbars SBM_GETSCROLLINFO (remarks here are helpful) Implement WM_GESTURE support
WM_TOUCH
Reed TownsendProgram ManagerWindows
demo
WM_TOUCH
WM_TOUCH* family of messages Best: WM_TOUCH enables touch-optimized
experiences WM_TOUCHDOWN, WM_TOUCHMOVE,
WM_TOUCHUP Semantically similar to mouse messages Conveys raw touch data to Win32 apps Scenario examples:
Finger painting, custom gestures, feeding higher level controls, etc
Capture Coalescing
WM_TOUCH
BOOL WINAPI RegisterTouchWindow( __in HWND hwnd, __in ULONG ulFlags);
BOOL WINAPI GetTouchInputInfo( __in HANDLE hTouchInput, // input event handle __in UINT cInputs, // number of elts in the array __out_ecount(cInputs) PTOUCHINPUT pInputs, // array of touch inputs __in int cbSize); // sizeof(TOUCHINPUT)
typedef struct tagTOUCHINPUT { LONG x; LONG y; HANDLE hSource; DWORD dwID; DWORD dwFlags; DWORD dwMask; DWORD dwTime; ULONG_PTR dwExtraInfo; DWORD cxContact; DWORD cyContact;} TOUCHINPUT, *PTOUCHINPUT;typedef TOUCHINPUT const * PCTOUCHINPUT;
BOOL WINAPI CloseTouchInputHandle( __in HANDLE hTouchInput); // input event handle
WM_TOUCH Future Windows 7 Build
WM_TOUCH consolidated to a single message
Manipulations & Inertia
Reed TownsendProgram ManagerWindows
demo
Manipulations
Best: Manipulations are a great foundation for touch-optimized experiences
Examples of higher level APIs/controls Manipulations are:
2D affine transformations (translate, scale, rotate) Superset of supported gestures Supports multiple concurrent manipulations
Need a source of raw data: WM_TOUCH or RTS Similar to Surface APIs
Interfaces: IManipulationProcessor IManipulationEvents
Inertia
Provides basic physics Works hand in hand with Manipulations
Interfaces: IInertiaProcessor IManipulationEvents – Same event interface as
Manipulations
Multi-touch In .Net Framework
For WPF: 3.5 update near Windows 7 launch
Multi-touch, gesture APIs 4.0 release
Multi-touch specific new controls
For WinForms (Windows 7 launch): Interop to native Win32 APIs – see sample Use managed Manipulations/Inertia wrappers
Multi-Touch In WPF
UIElement and UIElement3D changes Gesture events (tracking) Touch system gesture events (single) Raw touch events Touch and touch digitizer devices
Multi-touch support in controls: ScrollViewer update to accept pan gestures Base controls updated to be multi-touch aware Multi-capture support New multi-touch specific controls
Compatible with Surface SDK 2.0
Multi-touch WPF Application
Anson TsaoSenior Program ManagerWPF
demo
Gestures In WPF
Gesture events on UIElement & UIElement3D TouchGestureStarted, TouchGestureChanged,
TouchGestureEnded NOT routed, to enable multi-capture scenarios
Gesture in WPF based on Manipulations Enable Inertia at TouchGestureStarted
Inertial movements as TouchGestureChanged Events
Touch system gestures
Gestures in WPF
Anson TsaoSenior Program ManagerWPF
demo
Touch Events In WPF
Touch Events on UIElement and UIElement3D TouchDown, TouchMove, TouchUp TouchEnter, TouchLeave Preview variants
Promotion: Touch -> Stylus -> Mouse Multi-Capture
CaptureTouchDevice(TouchDevice) ReleaseTouchDeviceCapture(TouchDevice) ReleaseAllTouchDeviceCaptures() TouchDeviceCollection TouchDevicesCaptured { get; } TouchDeviceCollection TouchDevicesCapturedWithin { get; } IsAnyTouchDeviceCaptured(); IsAnyTouchDeviceCapturedWithin();
Touch Events In WPF
Anson TsaoSenior Program ManagerWPF
demo
Tablet Platform Extensions
Real-time Stylus (RTS) RTS is an excellent raw touch data provider IRealTimeStylus3::MultiTouchEnabled()
Getting multi-touch data via InkCollector is a little trickier. E.g.:
ATOM m_atom = ::GlobalAddAtom(MICROSOFT_TABLETPENSERVICE_PROPERTY);m_dwProperty = TABLET_ENABLE_MULTITOUCHDATA;::SetProp(m_hwnd, (LPTSTR)m_atomPenService, (HANDLE)m_dwProperty);
UX Considerations
General guidelines: Big targets Whitespace Avoid hover See UX guidelines
Be aware of hardware: Form factor Avoid on-hover UI Edges, jitter, etc
Gesture guidelines: Use common gestures Gestures need to be
intuitive and natural Moving away from
shortcut style gestures to manipulations and direct manipulation
Now Is The Time!
ISVs are enthusiastic about multi-touch! We are working with several ISVs today on
multi-touch applications, including: AutoDesk Avoco Secure Corel Cyberlink Fuel Industries Identity Mine Sonic
Call To Action
Decide which application scenarios make sense for touch
Decide which tier of investment is appropriate
Build something amazing!
Getting Started
What do you need to start coding? Hardware:
Multi-touch beta drivers for in-market devices:HP TouchSmart 2 Dell Latitude XT
Software: Window 7 M3 build – On the drive Touch Platform SDK and samples – On the driveNot in the Windows Platform SDK you received earlier!
WPF Multi-touch .NET 4.0 Preview Releases in 2009 .NET 3.5 update to be announced
Additional Resources
Related PDC Sessions: PC46 – WPF Roadmap PC17 – Developing for Microsoft Surface
Hands on Lab: PCHOL18: Developing Multi-touch Applications in Windows 7
PDC Whitepaper – Windows 7: Reach Out and Touch It!Product Pavilion – In the Windows boothAsk the Experts & Presentation and Client track lounge
Contact us: wintouch@microsoft.com
Evals & Recordings
Please fill
out your
evaluation for
this session at:
This session will be available as a recording at:
www.microsoftpdc.com
Please use the microphones provided
Q&A
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market
conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Backup Slides
Now Is The Time!
Multi-touch offers clear differentiation with a high “WOW” factor for consumers
Multi-touch enhances on-the-go use and opens up new scenarios for business
90% of people in focus groups claimed they would pay a 20-30% premium for a PC with multi-touch
Seeing is believing: Consumer purchase intent doubles after hands-on experience
Now Is The Time!ISVs are investing in multi-touch
ISVs are enthusiastic about multi-touch! We are working with several ISVs today on
multi-touch applications, including: AutoDesk Avoco Secure Corel Cyberlink Fuel Industries Identity Mine Sonic
Multi-Touch Capable PCs Are Available Today
HP TouchSmartAll-in-one PC with 2-touch digitizer
from NextWindow
Dell Latitude XTLaptop with multi-touch and pen
digitizer from N-Trig
Multi-touch & Windows 7
Multi-touch is a major investment area and strong reason for customers to upgrade to a Windows 7 PC
MT scored very favorably in Concept Value testing
Consumer awareness and interest in touch is high Customers perceive touch as the next stage of computing Touch offers a sense of personal connection and freedom Touch would help users be more efficient and be fun to use
Media response to MT disclosure has been very positive:
>130 unique articles, 1M video views in 72hrs, continued monthly coverage
WM_GESTURE Future Windows 7 Buildtypedef struct tagGESTUREINFO { UINT cbSize; // size, in bytes, of this structure
// (including variable length Args field) DWORD dwFlags; // see GF_* flags DWORD dwID; // gesture ID, see GID_* defines HWND hwndTarget; // handle to window targeted by this gesture POINTS ptsLocation; // current location of this gesture DWORD dwInstanceID; // internally used DWORD dwSequenceID; // internally used ULONGLONG ullArguments; // arguments for gestures whose arguments fit // in 8 BYTES UINT cbExtraArgs; // size, in bytes, of extra arguments, if // any, that accompany this gesture} GESTUREINFO, *PGESTUREINFO;
BOOL WINAPI GetGestureInfo( __in HGESTUREINFO hGestureInfo, __out PGESTUREINFO pGestureInfo);
WM_GESTURENOTIFY Future Windows 7 Buildtypedef struct tagGESTURECONFIG { DWORD dwID; DWORD dwWant; DWORD dwBlock; } GESTURECONFIG, *PGESTURECONFIG;
BOOL WINAPI SetGestureConfig( __in HWND hwnd, __in DWORD dwReserved, __in UINT cIDs, __in_ecount(cIDs) PGESTURECONFIG pGestureConfig, __in UINT cbSize);
BOOL WINAPI GetGestureConfig( __in HWND hwnd, __in DWORD dwReserved, __in DWORD dwFlags, __in PUINT pcIDs, __inout_ecount(*pcIDs) PGESTURECONFIG pGestureConfig,__in UINT cbSize);
SFP & SMB_GETSCROLLINFO
To help ensure SFP can control the scrollbars:
SBM_GETSCROLLINFO, SBM_SETSCROLLINFO, SBM_SETPOS and SBM_GETPOS
They also only have to do this if their scrollbar is not a system scrollbar (WS_VSCROLL and WS_HSCROLL styles I believe). The APIs we actually use would be the following which internally send these messages to the application: GetScrollInfo, GetScrollPos, GetScrollbarInfo, SetScrollInfo (may not need to use this one), SetScrollPos.
See the remarks section of this topic to see what developers need to do to support these messages. http://msdn.microsoft.com/en-us/library/bb787583(VS.85).aspx
We typically detect these types of scrollbars today, it’s just we don’t control them since they don’t pay attention to the WM_VSCROLL and WM_HSCROLL message params since these messages only support 16 bit scroll values. So they don’t need to do anything else for us to recognize their scrollbars (like set WS_VSCROLL). This won’t help windowless controls though since we cannot target portions of a window with this. Those apps would have to implement WM_GC to get their support.
Tell Me More!
What is Multi-touch? Multi-touch is two or more independently
trackable contacts It is not gesture support grafted on top of a
touch or pointing device nor pen and touch together
Choose the right input for the task at hand Touch and multi-touch are good for quick input
tasks, content consumption vs. creation, transactional input
top related