html5 touch interfaces: sxsw extended version
DESCRIPTION
SXSW Version...TRANSCRIPT
Creating Responsive HTML5 Touch InterfacesStephen Woods
Sunday, March 11, 12
Stephen WoodsFront End Engineer
Flickr
Sunday, March 11, 12
Sunday, March 11, 12
On the desktop we worry about browsers-moz-transform:rotate(-270deg); -moz-transform-origin: bottom left;-webkit-transform: rotate(-270deg);-webkit-transform-origin: bottom left;-o-transform: rotate(-270deg);-o-transform-origin: bottom left;filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=1);
Sunday, March 11, 12
On mobile we worry about devices.
Sunday, March 11, 12
Wait!Did you say they all
run webkit?
Sunday, March 11, 12
Wait!Did you say they all
run webkit?
Sunday, March 11, 12
Wait!Did you say they all
run webkit?
XSunday, March 11, 12
On mobile we worry about devices.
Sunday, March 11, 12
Media Queries, Break points, liquid layouts
Screen Sizes
http://www.alistapart.com/articles/responsive-web-design/Sunday, March 11, 12
Sunday, March 11, 12
iPhone 3GS256mb RAM
Geekbench: 271
Sunday, March 11, 12
iPhone 3GS256mb RAM
Geekbench: 271
==
Sunday, March 11, 12
Modern mobile devices are crappy
computers with decent video cards.
Sunday, March 11, 12
Sunday, March 11, 12
Perceived Performance
Sunday, March 11, 12
On the desktop it’s easy...
Throw up a spinner.Sunday, March 11, 12
Touch interfaces are tactile.
Sunday, March 11, 12
Touch interfaces are tactile.
Feedback must be continuous.
Sunday, March 11, 12
When the interface stops moving
during a gesture it feels like it died
Sunday, March 11, 12
Respect Convention
Sunday, March 11, 12
Mobile has conventions too
Sunday, March 11, 12
Mobile has conventions too
Sunday, March 11, 12
TouchEvent
• touchstart - fires once
• touchmove - fires continuously
• touchend - fires once
Sunday, March 11, 12
The touches Array
• You only get one on Android
• You get up to 11 on iOS
• Each touch gives you position information, and sometimes scale
Sunday, March 11, 12
iOS Gesture Events
• gesturestart
• gesturechange
• gestureend
Sunday, March 11, 12
iOS Developer Library
http://bit.ly/iOS-guide
Sunday, March 11, 12
Making Gestures Work
• Prioritize user feedback
•Use hardware acceleration
•Manage your memory
Sunday, March 11, 12
Prioritize User-feedback
•Don’t do any loading during gestures
• Treat the DOM as write-only (do your own math)
•When at all possible, use css transitions
Sunday, March 11, 12
Write-Only DOM
•DOM touches are really expensive
• You know where everything is
•Use matrix transforms to queue up positions
Sunday, March 11, 12
Swipe Basics
distance = e.touches[0].pageX - startX;
'translate3d('+distance+'px,0px,0px)'
Sunday, March 11, 12
Snap back/snap forward
• Keep track of last position, use transitions with easing to snap back
• Pick a swipe distance threshold, use that to snap forward (ontouchend)
• If the user is gesturing, the element must be moving
Sunday, March 11, 12
A Word about scrolling
•Use native if at all possible:
•-webkit-overflow-scrolling: touch;
• If not, use a library to simulate momentum scroll (iScroll 4, Scrollability)
Sunday, March 11, 12
Avoid Event Abstraction
Sunday, March 11, 12
Pinch to Zoom(there will be math)
Image © Brian Lalor Used with permission
Sunday, March 11, 12
Why you can’t use native Pinch to
Zoom
Sunday, March 11, 12
First:Use Matrix TransformsMinimize DOM touches, make your transforms simpler in the long run
Sunday, March 11, 12
It’s Not That Hard!
transform:matrix(1, 0, 0, 1, 10, 10);
Translate
Scale
Sunday, March 11, 12
[ [1,0,0,0], [0,1,0,0], [0,0,1,0], [tx,ty,tz,1]]
With Hardware Acceleration... (matrix3d)
Sunday, March 11, 12
Transforms keep complex state
without DOM reads
Sunday, March 11, 12
What is happening?
•Determine Center of the touch points
•Determine the scale factor (touch.scale)
• Scale the element by the scale factor, with the center of the touch points as the scale center
Sunday, March 11, 12
The Naive Example
Sunday, March 11, 12
The Naive Example
Sunday, March 11, 12
The Naive Example
Sunday, March 11, 12
The Right Example
Sunday, March 11, 12
The Right Example
Sunday, March 11, 12
The Right Example
Sunday, March 11, 12
Breakdown
Sunday, March 11, 12
Breakdown
Sunday, March 11, 12
Breakdown
Sunday, March 11, 12
Breakdown
Sunday, March 11, 12
translateX = scalePointX * (newWidth - oldWidth)
newWidth;
Sunday, March 11, 12
Pro Tips
• Beware the virtual pixels
•Moving the transform-origin won’t really work
•Remember to snap back
Sunday, March 11, 12
Dealing with browsers
Sunday, March 11, 12
Dealing with browsers
Sunday, March 11, 12
Remember Progressive Enhancement?
Sunday, March 11, 12
• Feature Detect
• Add transitions, don’t depend on them
•Gesture interaction is an enhancement, clicks should still work
• Be able to disable features per user-agent, if necessary
Progressive Enhancement
Sunday, March 11, 12
The Tool Chain
Sunday, March 11, 12
The dumbest thing that works
Webkit Browser with UA Spoofing
Sunday, March 11, 12
Weinre
Remote webkit debugger.
http://phonegap.github.com/weinre/
Sunday, March 11, 12
Internet Sharing + Charles Proxy
•Watch HTTP traffic
• Add breakpoints in ajax requests
• Serve web pages to your phone from your computer
Sunday, March 11, 12
Adobe Shadow
•Wrapper for Wienre
•Neat
•Has some limitations
Sunday, March 11, 12
Pile of Devices
Sunday, March 11, 12
Pile of Devices• iPad 1
• iPhone 3G
• iPhone 4
• Samsung Galaxy S
•HTC Desire
•Galaxy Tab
•Motorola Xoom
•Kindle Fire
•HTC Titan
Sunday, March 11, 12
Device Simulators & Emulators:
Basically Useless.
Sunday, March 11, 12
The Flickr Touch Light Box
Sunday, March 11, 12
UntitledBy protohiro
Sunday, March 11, 12
UntitledBy protohiro
Sunday, March 11, 12
UntitledBy protohiro
Sunday, March 11, 12
UntitledBy protohiro
Sunday, March 11, 12
Swiping Process
• Event Listener on top level for touch events
•Only visible nodes move via translate3d
•Rebuild next/previous happens when movement stops.
Sunday, March 11, 12
Performance Tricks
• Aggressive Pruning
•Clean off css transforms/transitions
•Write-only DOM.
•Do as little as possible during swipes
Sunday, March 11, 12
Frustrating Limitations
•Retina screen is huge, device memory is small
•Hardware acceleration is a crash festival
• Fighting automatic optimization
http://bit.ly/apple-image-size-restrictionsSunday, March 11, 12
http://www.flickr.com/photos/wafer/5533140316/http://www.flickr.com/photos/latca/2265637876/http://www.flickr.com/photos/spine/1471217194/http://www.flickr.com/photos/williamhook/3656233025/http://www.flickr.com/photos/isriya/4656385586/http://www.flickr.com/photos/yandle/3076451873/http://www.flickr.com/photos/uberculture/6632437677/http://www.flickr.com/photos/blalor/4934146981/http://www.flickr.com/photos/torek/3280152297/http://www.flickr.com/photos/nilsrinaldi/5157809941/
Stephen Woods @ysaw
Image Credits (http://flic.kr/y/kQ5cLh)
http://www.slideshare.net/ysaw/creating-responsive-html5-touch-interfaces
Sunday, March 11, 12