google glass and the wearable revolution - nyccamp 2013

Google Glass & The Wearable Revolution

Frank Carey - @frankcareyAnthony Ferrara - @ ircmaxell

First Some ContextLet's step back and look how we got here, and how Google Glass fits in to the larger

picture of wearables.

It all started with the IPhone*..

June 29, 2007

Happy 6th Birthday!

*not exactly, but lets move on

Smartphone Mkt Share - '07 vs '13

Nokia - 50%RIM - 10%Apple - 5%HTC - 3%Samsung - 2%Other - 27%

Total Shipping: 36MM in 4Q (130MM / yr)

Android - 75%iOS - 17%Windows - 3%Blackberry - 3%Linux - 2%Symbian - 1%

Total Shipping: 216 MM 1Q (900MM / yr)

All the Phones! +1 Billion per year

Evolution of Hardware

First Gen IPhone Specs:Launched - June 2007- 6 years agoScreen - 320 x 480Memory - 128MBStorage - 4/8 GBProcessor - 412MHz, single coreData - 2G/Edge (up to 250k) And WiFiCamera - (1) 2 MP (no video)Required sync with iTunes & No App Store

Evolution of Hardware

Latest Specs (HTC One): Launched - March 2013 (4 months ago)

Screen - 1920 x 1080 (16x)Memory - 2 GB (16x)Storage - 32/64 GB (8x)Processor - 1.7 GHz - quad + GPU (16x)Data - 4G (up to 56Mbps ) (228x)Camera - (2) w/ 2 MP front, 13*MP rear (10x)AppStores - Almost 1M each, Cloud Sync

Evolution of Hardware

First Gen IPhone Sensors:3 Axis Accelerometer (screen orientation)Cellular Positioning (very rough)Proximity Sensor (hold to face)Ambient Light (auto brightness)Microphone, Speakers & Mic

Evolution of Hardware

New Latest Gen Sensors:Gyroscope (faster, relative direction)Digital Compass (Cardinal direction)GPS (integrated) (position < 1m)NFC/ RFID, USB-OTG & Bluetooth 4.02nd Mic (noise reduction)Front Camera (face detection, eye tracking)Stereoscopic - (3D images and video)

Evolution of Hardware

Old Sensors, new techniques (ML, AI):Sensor Integration via hardware / OS○ Location based on Wifi, Cellular, and GPS○ Orientation via Compass, Gyro, Accelerometer○ Activity - Running, Walking, Biking, etc.

Text & Speech (on device)○ Text to Speech - Give it some text, and have it read

aloud○ Speech to Text - Returns array of likely sentences.

Hacks (Square) - Universal Connectivity by sending digital over microphone jack.

Evolution of Hardware

NEXT Gen Sensors:Environment (already Android supported)○ Temperature○ Humidity○ Pressure (Altitude)○ Light

Air Quality (CO, Smoke, O², Allergens)Body - Heart Rate, Blood Pressure, BloodDepth Cameras - (Kinect / Leap Motion)(..????)

Positive Externalities of Phones

Phones have drastically lowered the costs of sensors, screens, batteries, and low power processors at the same time as improving quality.

The specific build of an iPhone/Samsung/etc is proprietary, but the components that make it up are mostly commodity parts. (They are shared, and the chips are able to be purchased individually and cheaply).

Resulting TrendsWhat are we doing with those Phone

Processors, Sensors, and OSes ?

Cheap & Low Power Processor Platforms

● Computers/Tablets - Chromebook, Surface RT

● Open Hardware for Hacking - RaspPi ($25), BeagleBone ($45), Arduino Due ($58)

● Gaming Consoles/SetTop - OUYA, Gamepop

● Robotics using these platforms + sensorsAll take advantage of standard "mobile" processors (ARM), and many use Open Source linux based OS like Android, Chrome OS, and Debian!

Cloud Sync, Storage, and APIs

Phones have limited storage and ease of creating documents. Since phones are constantly connected, often better to store and retrieve via internet.

No need to plug into computer and sync with iTunes, buy Apps, transfer music or photos/video.

Phone Apps rely heavily on REST APIs

Cloud Sync, Storage, and APIs

HTML5 and JS based Apps to rival native apps, w/ some OS like Firefox OS, Chome OS

Cloud Rendering - Gaikai/OnLive have cloud gaming to "any" device. (think VNC on steroids?) Amazon EC2 w/ GPUs - used (initially) for Darpa Virtual Robotics Challenge.

Complex / compute heavy stuff can be done serverside. AR, Recognition, Planning, AI

Screens and Sensors

Oculus Rift - ($399) Dev Kit Shipping nowLow latency VR / Head Mounted DisplayBasically uses Tablet screen + Accel + Gyros

Screens and Sensors

Pebble Watch - ($150) Shipping NowUses a port of FreeRTOS, Accelerometers, Magnetometer, Light Sensors & BT4.0

Screens and Sensors

Leap Motion - ($79) Ships July 27thShort distance, high-res 3D sensor for mid-air 'touch' controls. Size of a pack of gum. Tech coming to phones and laptops soon.

Uses 2 cameras and infrared LEDs

What do many of these have in common?

* Slight diversion but we'll mention this later.

Many are Kickstarter / Indiegogo !

It's now easier* than ever to create a hardware startup.1. Create a prototype using standard (mobile)

components, open source hardware, and a 3D printer.

2. Crowdsource your funding with pre-orders.3. Collect Millions, and pray you can deliver on

schedule and under budget.*also easy to hang yourself. http://money.cnn.com/interactive/technology/kickstarter-projects-shipping/

Recent Example Crowdfund Projects

Pebble (April 2012) - $10.2M raised

Oculus Rift (Aug 2012) - $2.4M raised

Leap Motion (May 2012) - ???* raised*Had $11M Series A funding first, but had a 1 year pre-order on their website.

OUYA (July 2012) - $8.5M raised

What else do many have in common?

What else do many have in common?

Many are either offer New User Interfaces and/or are Wearable Computers.

A Wearable Computer?

circa 1990's

Wearables● "Smart" Devices that attach to the body

● Sensors - Touch, Camera, Accelerometer, Gyros, Microphones, Temp, Barometer.

● Feedback - Video , Sound, Haptic, even simple led. "Constant Interaction".

● Connectivity - Wifi, Bluetooth (Low power), Cellular, Xbee

WearablesIdea is that, eventually:● Everything gets smaller,● It becomes cheaper● User interfaces evolve to be more natural● The handheld screen/device goes away.● We end up merging with our technology● Singularity happens● TBD

But, We're not there yet :("peacock... peacock..."

Current Hurdles for Wearables

Current Hurdles for Wearables

* Batteries Not IncludedProbably single biggest issue...

Current Hurdles for Wearables

BatteriesStill need power, but you're not plugged in all the time● ..So, you'll need batteries● ..So, that means you need to charge it

somehow● ..So, that means you need to worry about

remembering to recharge● ..So, that means you'll get mad when it dies.

Current Hurdles for Wearables

Batteries

Ways to deal with the problem...

Current Hurdles for Wearables

Batteries

Cut out the things that use a lot of power.

Current Hurdles for Wearables

Batteries

Cut out the things that use a lot of power.

Screens, Processors, Radios, Sensors...

...everything!

The iPad! ...wait, too far..

Fight the Power! - Screens

Screens waste a LOT of photons that don't reach your eyes. LED backlights means heat is less an issue.● No Screen!

○ Leverage another device's (phone) screen (Fitbit)○ Use only sensors for UI, voice commands, buttons,

touch, haptic feedback, etc.● Alternatives

○ low power display like e-paper (Pebble)○ simpler, like single LEDS (Nike Fuelband)○ smaller screen close to eye (Google glass)

● Use lasers directly to retina (OASIS)

Fight the Power! - Radios

Radios use power when listening, but lots more when sending. ● Burst / Only periodically turn on radios to

make sensor measurements or send/receive data from base.

● Use only lower power RF, (usually lower range, lower throughput protocols)

● 4G > 3G > 2G > Wifi > Bluetooth > 4.0 LE● Drop radios and use a wire instead.● GPS (worst) uses radios too!

Current Hurdles for Wearables

User Interface - What should the UI be?.. But we just got used to touchscreens!

● No Screen (no keyboard, no touchscreen)● Voice Commands?● 3D Sensors, like leap?● Haptic (Vibrate) ?● Direct interface to Visual Cortex?● How much info? How often? Social norms?● What to do with data? Security?

Current Wearable Examples

- Fitbit (2008)- Nike+ Fuelband (Jan 2012)- Pebble Watch (Jan 2013)- Google Glass (Mar 2013)- Apple iWatch (rumored 2013)- Samsung "Gear" watch (rumored 2013)- Microsoft, Amazon, Facebook???- Many startups / smaller players

Next-Gen Wearables / Sensors

Smart Pills- Health and Authentication“It creates a signal and your entire body becomes your authentication. My arms are like wires, my hands are like alligator clips.”-- Regina Duncan, Senior VP for Advanced Technology & Projects at Google

Next-Gen Wearables / Sensors

Flexible Sensor Temporary Tattoos (No Battery)

Next-Gen Wearables / Sensors

e-Fashion - Clothes that can light up, change shape, sense, and adapt.

Are wearables the NBT? Engage the hype machine!

Next BIG Thing?

"[Wearables are] ripe for us all getting excited about. I think there will be tons of companies playing in this…I think wearables is incredibly interesting. It could be a profound area." -- Tim Cook, Apple CEO

Next BIG Thing?

"Product strategists who want to stay ahead of the curve should take a cue from companies like Intuit and experiment with wearables now, especially if you’re in an industry that will be disrupted by wearables, including apparel, software, media, gaming, and commerce." -- Forrester April 2012 http://bit.ly/HNeXyS

OK, Finally on to Glass!

Glass XE Hardware, Who has 'em

XE = Explorer Edition

Announced at Google IO 2012 - attendees given pre-order opportunity. ($1600)

Glass "Pioneer" Hackathons in NY and SF Jan 2012 and Started Shipping in April 2013

#IfIHadGlass "contest" adds more. ~8K in Wild

What is Glass?

#1 - It's a beta prototype! that means it's not 100% ready yet!

#2 - It's a beta prototype! It's a pretty decent attempt to solve issues around wearables and figure out a new UIs.

What Glass Brings to the Table

1) A truly standalone wearable computer (phone optional)

2) Brand New set of User Interfaces - still in development.

3) Mirror API - A simple REST API that makes it VERY EASY to write simple Apps.

4) Mostly Hackable!

Glass not the first smart glasses.

Steve Mann's interesting work - MIT / Toronto

Previous Glass Prototypes

Jean Wang - Project Glass Staff Hardware Engineer

Glass Hardware Specs (Current!)

Screen - 640 x 360 prism - 25in TV at 8ftProcessor - OMAP 4430 Soc, dual-coreMemory - 1GBStorage - 16GBWeight - 50g (.11 lb)Camera - 5MP, 720p videoSound - Bone conduction outputConnectivity - Wifi and Bluetooth tether~ 2012 Top of line specs

Glass Hardware Sensors (Current!)

AccelerometersGyroscopeMagnetometerProximity (Blink?)Light sensorGPS (w/ phone tether only)(Mirror API doesn't directly use sensors)

My thoughts: Hardware

I think the hardware is very good. The ability to have a screen, a 2 axis touchpad, light enough, most sensors, and a battery that lasts (mostly) all day.

Battery is probably the biggest drawback, they should double the capacity to enable using features without worrying.

Still, a great platform to experiment!

My thoughts: User Interface / APIs

Mirror API is good, but not great. Still very limited, but also very simple to create apps. Removes many worries from the application developer. Mirror API still in development.

Alternative is Native App development. Really Opens up interesting possibilities. Involves rooting device for now so Apps distribution is almost impossible. SDK should eventually happen!

Glassware: Using the Mirror APIA REST / Cloud based API (not native)Docs: https://developers.google.com/glass

Keywords:● Timeline● Cards● Menu Items● Pinnable● Contacts● Subscriptions (events)

Glassware: Example Card

Glassware: Cards

Use HTML and CSS (familiar, stuff)

Have some standard formatting (CSS):● Single message● Side image

Glassware: Timeline

Everything is a timeline item!

Your app only has access to timeline items that are shared with it or created by it.

You add cards, by adding a timeline item.

You can update cards instead of posting new ones. Cards expire disappear after a few days

Glassware: Menu items

Subscription (events)

Get updated when the user shares something with you or does some other menu action to one of the cards you own.

Also location updates. (Not real time... every 30 sec?)

REST JSON Example{

"collection": "timeline",

"itemId": "3hidvm0xez6r8_dacdb3103b8b604_h8rpllg" ,

"operation": "UPDATE",

"userToken": "harold_penguin",

"verifyToken": "random_hash_to_verify_referer" ,

"userActions": [

{

"type": "<TYPE>",

"payload": "<PAYLOAD>"

}

]

}

Hacking Glass

Google made it pretty easy to root. Opens up a LOT more possibilities like VR, new UIs, etc.

"Voiding your Warranty" at Google IO 2013

http://www.youtube.com/watch?v=OPethpwuYEk

Final Thoughts

Exciting Space!

What's next?

Alternatives?

Questions?Frank Carey - @frankcarey

Anthony Ferrara - @ircmaxell