bridging the gap between humans and the physical world
DESCRIPTION
Bridging the Gap between Humans and the Physical World. A Step Toward Reducing Energy Consumption and Increasing Comfort. Dr. Fred Jiang / 姜小凡 In collaboration with Mike Liang, Jeff Hsu, Caiquan Liu, Jie Liu, and Feng Zhao Mobile and Sensing Systems Group Microsoft Research Asia - PowerPoint PPT PresentationTRANSCRIPT
Bridging the Gap between Humans and the Physical World
A Step Toward Reducing Energy Consumption and Increasing Comfort
Dr. Fred Jiang / 姜小凡In collaboration with Mike Liang, Jeff Hsu, Caiquan Liu, Jie Liu, and Feng Zhao
Mobile and Sensing Systems GroupMicrosoft Research Asia
National Taiwan University / 2011-4-1
MotivationWe interact with our environment in very limited ways
Simple time series data collectionActuation is limited because of lack of uniform interfacePhysical objects cannot proactively reach out to us
Result: waste in energy and suboptimal comfort
Example 1: environment (physical objects) aware of who is acting on it, collect usage / energy data in real-time (personal energy footprint)Example 2: alert me if something is not normalExample 3: objects interacting with objects
Related Work
Ambient intelligenceRoomwareSmart objectsLocalizationX10Beaver monitoringUbicomp literature
We Propose
Bi-direction interaction and information flow between humans and physical objectsFor humans to have real-time finer-grained knowledge and control over physical objectsEnvironment to be aware of humans, and make intelligent decisions, proactively and reactivelyBridging the gap between humans and their environmentOur vision of IoT (it’s not about simply putting a networking interface on everything)
Three Primary Components
Precise indoor localization primitiveReliable IPv6 based networkingUniform interface for abstracting services of physical objects
Virtual Overlay
Entrance to 4th floor
name: fridgestates: on : true temp : 5 door : opencommands: turn_on() turn_off()events: notifyDorOpn notifyPwrUsgchildren: []
Indoor Localization
Previous workWhat we needChallenges
Localization Options
Wifi radioBT radioInertial sensorOptical (Kinect)RFIDMagnetic (PKE)
Magneto-Induction
Slide courtesy of Andrew Markham (http://www.comlab.ox.ac.uk/people/andrew.markham/)
Magneto-Induction
Slide courtesy of Andrew Markham (http://www.comlab.ox.ac.uk/people/andrew.markham/)
Magneto-Induction
Slide courtesy of Andrew Markham (http://www.comlab.ox.ac.uk/people/andrew.markham/)
Challenges for Indoor
4m x 1m antenna -> 5m^3 trackingWe don’t have that much space!We use a 8cm x 1.5cm transmitter antennaAnd 3D coil IC (2cm x 1cm) as receiving antenna
Power supply issuesOutdoor vs indoorInterference issuesPhone magnetometer (digital compass)
Microsoft LivePulse
Mains Power
AD/DC/DC Converstion90-240VAC -> 12VDC -> 5VDC -> 3.3VDC
Mechanical Relay
Hall-effectSensor
MI MOSFET Driver
125kHz MI Transmitting Antenna
CC430 SoCEnergy Meter
IC
900MHzPCB
Antenna
Microsoft LivePulse• Precise indoor
localization
Range: ~ 3m omnidirectional
~10cm dropoff zone
Consistent over timeRobust against interference and multipathBlock by metal
• Wireless energy monitoring
Microsoft LiveLink
3D MI Receiving Antenna
Wakeup
Networking
RequirementsReliableGlobally addressableScalableLow power
Radio Consideration
• 802.15.4– 2.4GHz– 863 MHz to 928MHz– 100-150m range– US
• 802.15.4c (c for china)– 779-787 MHz– 250m range
• Core module choices– 60USD Epic core– 20USD SuRF core
• Energy
Networking Overview
One-hopIPv6 communication over 802.15.4c
JSON / RESTful services over Internet
Repository of virtual objects
Virtual Object Abstraction Layerapartment
bedroom office bathroom hallway
desk desktop room portable lamp computer lamp heater
coffee laptop piano pinballmachine computer keyboard machine
LCD computer monitor tower
• Hierarchical• Event based
Object Representation
StatesE.g. On/off
CommandsE.g. turn on / read power / set temp / count people
EventsE.g. door opened / light-off / over-threshold
ChildrenPtr to children objects
name: officestates: lightOn? : True doorOpen? : False roomOccupied? : True roomTemp? : 30C roomEnergy? : 200W commands: toggleLight numOccRoom lastTimeDoorOpened
events:
notifyDoorOpen notifyNewOccupant notifyHourlyEnergyUsechildren [ipv6]: coffeeMachine deskLamp laptopComputer desktopComputer pianoKeyboard roomLamp pinballMachine portableHeater
1. DISCOVERY_REQ
2. DISCOVERY_REPLY
to http://<ipv6OfOffice>
3. EVENT_SUBSCRIPTION [notifyNewOccupant]
4. NOTIFICATION
“let me know if someone enters room”
“someone just entered your room”
Open Source Standard
JSON is the object encodingSOAP / WSDL / Zigbee SE Profile – Nooo
REST/HTTP is the interchange schemeHTTP_GET / HTTP_GET / HTTP_RESPONSE
IPv6Interoperate with existing IP devices without understanding additional protocols
Applications
Personal energy footprintSocial networkingHealth care
Related EffortsSmart meters (via utilities)
Enables time-of-day billingGoogle PowerMeter
Visualization of whole-house usageDefining a API for energy data
Microsoft HohmComplimentary work (help learn signatures)House-level / NILM / can benefit from DB of appliance signatures
In the right direction, but not enoughOnly at the whole-house levelLack of actuation / control
Source: http://www.microsoft-hohm.com
Source: X. Jiang Dissertation
Individualized Energy Feedback and Control
• Real-time energy apportionment– Individualized energy
accounting / billing– Energy map / trail of occupants
• Alerts and abnormally detection– Via cellphone– Remote alerts
• Control and actuation– Automated control of devices– Scheduling
An Energy-Centric Ecosystem – Social Network of Energy
Social network based on real-time individual energy usagesWindows 7 GadgetWindows Mobile 7 AppFacebook appReward systems
Building managers have fine-grain control and view
mobile application
web application - synergy
Social Network of Energy
LocalizationLocation Status updates, kind of like Foursquare Checkins.• “Mike is now in the coffee room.”
ApportionmentNow that we have localization primitive, we can specifically attribute the use of public energy to individual users. Very useful.• “Attribute the last 3 minutes of coffee energy usage to Jeff.”Add this to personal monitoring of private appliances, and we now have a complete carbon footprint.
web application dashboard - map
Social Network of Energy
Map of User’s owned areas…and where energy usage is taking place the mostSocial Interaction
Compare Energy Usage of Specific Appliances, or aggregateGames• User VS. User [One-Month Challenge, percent
improvement?]• Team VS. Team
web application dashboard - energy
Social Network of Energy
Add Appliances [Home and Work]Energy Usage Trends
Day to Day Comparisons in a tableEnergy Usage Graphs [Compare different devices]
Email Summary of Energy Usage to User [each day, or week, or month]
web application – user profile
Social Networking “Wine Party” App
Frequently, it is very difficult to meet people you click with at social gatherings.Ex. You are at a company party. You are bored. You pull out your mobile phone. A map of the room you are in shows up, and there are markers on the map. Click on a marker, and you can see that person’s interests, details, hobbies, etc.See twitter feed, facebook profile, etc.When someone who match your interest pass by, your phone vibrates to alert you.
Health Care
Room is aware of status of elderly occupants 1. phone insert trigger / event2. object representing the room 3. room send a signal to alert
Conclusion• Bi-direction interaction
between humans and physical objects
• For humans to have real-time fine-grained knowledge and control over physical objects
• Environment to be aware of humans, and make intelligent decisions, proactively and reactively
• Precise indoor localization primitive
• Reliable IPv6 based networking
• Uniform virtual representation of physical objects
Thank you
Questions?
Feel free to contact me at [email protected] http://research.microsoft.com/people/fxjiang