smart u-things & internet of things (iot) · u-things physical things with aeb two fundamental...
TRANSCRIPT
Smart u-Things &
Internet of Things (IoT)
e-Thing & e-Activity in Cyber World
e-Things: digital things in Cyber World on cyber space
Computers/Networks/Internet
Cyber Space
Web, WbS, SmW, Grid, P2P, EaaS, Cloud
e-Thing
e-Activity
Cyber World Physical World
Physical Thing
Usual Activity
u-Things Physical Things with AEB Two Fundamental Technology Trends
Continuing miniaturization of devices (Moore’s law, new material, nanotech., …)
Available interconnections by ubiquitous/pervasive wired and wireless networks
u-Things: Physical things with some kind of Attachment, Embedment, Blending
(AEB) of computers, sensors, tags, networks, and/or other devices
0.15x0.15 mm, Hitachi/06
u-Things in Physical/Real World
Physical World
u-Thing
Usual Activity
Sensor/M/NEMS, Comps & Per. Nets
Physical Space
UC, ID, Context, Emb. Sys., etc.
u-Things: on the physical space in physical-digital form
e-Thing
e-Activity
Cyber World
Computers & Networks/Internet
Cyber Space
WbS, SmW, Grid, P2P, EaaS, Cloud
Characteristics in 3 Computing Waves/Eras
m-to-1MC
1-to-1PC
1-to-mUC
Main Role/Obj
Proc. Content
Calculations Information u-Services
Data Media
u-Things + Cloud Mainframes Personal ComputersComp Element
Contexts
Size/Form/User
Weiser’s3 Relations3 Waves
Basic Behavior Passive ActiveInteractive
Large/Stationary/Few Small/Portable/Many Invisible/Everywhere/All
Theory/Tech Comp Sci/Eng Info Sci/Eng (? Science: Cybermatics)Smartness/Intelligence
Central Goal Fast/Precise/Reliable Rich/Convenient/Secure Aware/Comfort/Safe
Essential Questions
Can machines think ? AI (Artificial Intelligence)
- By A. Turing in “Computing Machinery and Intelligence” in 1950
Can things think ? Intelligent real/virtual entity
- Consortium of Things That Think (TTT) in MIT Media Lab since 1995
- “When things start to think” by Neil Gershenfeld of MIT in 1999
Can smart u-things be everywhere ? UI (u-Intelligence)
- Ubiquitous Intelligence (UI) or Pervasive Intelligence ( ) from 2003, Ma-Yang, etc
- Residing in everyday objects, environments, systems, ourselves, plant, animal, …
- Real Things u-Things Smart/Intelligent u-Things Smart World
p
IBM’s Smart Planet, New Intelligence 2008
Hyperworld and Smart World Hyperworld (1995)
- “Hyperworld Modeling”, Keynote in VIS, Melbourne, Feb. 1996 (Kunii, Ma, Huang)- “Its basic characteristic is direct mapping between virtual and real worlds via
active devices including sensors, actuators, micro-machines, robots, etc.” - 1996- “A Study on a Hyperworld System of One-to-Many Interaction”, ICAI’97 (Weiser’s 1-to-m relationship)
- “Towards Direct Mapping between Information Worlds and Real Worlds”, LNCS1306, 1997
Smart World (2003)- Based on ubiquitous computers, sensors, actuators, networks, etc.- Pervaded with smart, life-like ubiquitous things/u-things- Characterized by u-services & u-intelligence in the real world
Smart e-Thing
e-Service
Cyber World Smart World
Smart u-Thing
u-Service
Hyperworld
Cyber-Physical-Social Spaces
Web-Intelligence
WIUbi-Intelligence
UI
Ubiquitous IntelligencePervasive smart real things able to sense, think, act, …
Information Explosion Intelligence Explosion!
- The Intelligence Revolution, Interview by Wise Media, ID People Magazine, Apr./05- Ubiquitous Intelligence Summit 2005, A Think Tank, Oulu, Finland, June 15-17, 2005- Journal of Ubiquitous Computing and Intelligence (Yang & Ma), American Sci. Pub./05- IEEE Task Force on Intelligent Ubiquitous Computing (Duman, Yang, Ma), since 2006 - Journal of Autonomic and Trusted Computing (Yang and Ma), ASP, USA, ASP, 2006
Smart Object (Smart u-Object, Smartifact)- A physical object with AEB devices and some smartness/intelligence- Handheld, card, label, sensor, artifact, appliance, goods, furniture, textile, robot, ……
Smart Environment (Smart u-Environment)- A physical spatial environment/site integrating smart u-objects &/ devices- Smart u-Services via these objects/devices and their commun./cooperation
Smart System (Smart u-System)- May be a physical system/network for management, monitoring, emergence, …- May be a platform or middleware for smart u-objects/environments/services
Ubicomp/Percomp/AmI/CPS/IoT Computing of Smart u-Thing
Smart u-Things
Plant talks, IEEE Spectrum, May/09
Smart Grid, IEEE Spectrum, July/09
Basic features of smart u-things- Sense & effect for perception and interaction- Networking & naming for interconnection and communication- Computation & storage for information processing and memory
Smart u-Things – MIT Lover’s Cups
Explore the idea of sharing feelings of drinking between two people in different places
Two cups are connected with each other, and glow when your lover is drinking.
Smart u-Things – Smart Slippers & Nike
Smart u-Things – GrowCaps, etc.
Smart u-Things – Smart Home
Videos About Smart Object and System
Smart Trashbox – 勝手に入るゴミ箱
Smart Systems – Toyota’s Smart Mobility Society
Sony SmartWatch2 , Samsung Galaxy Gear ,
Galaxy Gear Concept 2013
Smart Shoe – Nike smart trainers
Smart Pill Bottle , Smart Pill Box , Smart Pill
Smart Toy – Smartphone + Toy
Smart Space/Environment
Smart Space/Environment Computerized active one
- Physical environments but digitally enhanced/integrated
- Awareness of contexts: users, ambience, resources, etc. - Context-Aware responses or services with certain intelligence- Where are users? Supervisors & computing elements!!
Sensors
Actuators
Space
Environment
Object
Artifact
Plant
Body
(User)
Ubi/Pervasive
Information
- Acquisition
- Archive
- Analysis
- Awareness
- Application
- …
Multimedia
Information
Convenience
Comfort Service
(Context)
Loop
Smart Space/Environment
Status/ attend
Status/ absent
RFID
Reader
Room 1 Room 2
Room 3
Motion‘Organ’
Memory‘Organ’
Percep.‘Organ’
Brain
BathToilet
StudySleep
Entrance Kichen
Living
Dinning
Action‘Organ’
Think‘Organ’
Control‘Organ’
Smart Physical Objects Management
Things Mapping - toy- game- key- …
Thing Reminder- umbrella- book- pencil- …
Users’ IndoorLocation Sharing
ユーザの屋内位置情報の共有
ROOM
サーバー
ROOM
アクセスポイント1
データベース
位置推定行動推定
アクセスポイント2ログデータ
ログデータ
電波強度加速度データ
EV2
EV1
User Location-Aware Music Player
Music data sent to speaker(s) near a user’s location
Sound-Aware Smart Space
Status/ attend
Status/ absent
RFID
Reader
Room 1 Room 2
Room 3
Sound-aware Reminder- Ambient sound capture- Ambient analysis- Advice judgment- Advice presentation- …
Sound Disturbance
- home
- classroom- theater- …
Robots forMobile Sensing & Object Finding
R-Reader
R-Controller (PDA)
R-Machine
Ultrasonic Sensor
Motor
Camera
R-Reader
R-Controller (PDA)
R-Machine
Ultrasonic Sensor Motor
Lifelog – Personal Experience/DB
LabLog – Info Sensing & Management
Sun SPOTS
Wireless networks
& Sensors
Java program
SSG
SSGOn Robot
(Time)
24℃
(Raw) SSG (Door)
(Time)
(Raw)
25℃
SSG (Robot)
HomeLog – Kid’s Activity Summarization
ID Tag and Sensor for User2007/11/10
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
1 Sensor A(study)
Sensed Activity Period
4
3 Sensor F(study)Sensor G(study)
2 Sensor E(study)
5 Sensor H(study)
Sensed Activity Change Time
Sensor for User12 1 2 3 4 5 6 7 8 9 10 11 12 1 432 5 6 7 8 9 10 11
2007/12/11
部屋入口で認識した子どものRFIDタグ
机の椅子のタッチセンサー
教科書のRFIDタグ
机付近で認識した子どもののRFIDタグ
漫画本のRFIDタグ
ID Tag and Sensor for User2007/11/10
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
1 Hosei CIS
Kid’s Name12 1 2 3 4 5 6 7 8 9 10 11 12 1 432 5 6 7 8 9 10 11
2007/12/11
Sleep Game Study Sleep
Videos About Smart Things and Services
Green Wheel - Smart Mobility & Ubiquitous Computing- 10 scenarios on Smart Ubiquitous Services
Display Glass - Smart Ubiquitous Displays- Displays in wall, surface, mirror, veneer, etc.
Smart Home - Home Energy Management System (HEMS)(community) - Smart energy supply, use, control, etc.
Japan Smart City Pilot Projects
http://jscp.nepc.or.jp/
Japan Smart Community Alliance
Established on April 6, 2010 to promote Smart Community through government-private sector cooperation
As of Nov. 2012, 397companies and organizations (including universities)
Cooperation across companies and organizations
https://www.smart-japan.org/default.aspx
Smart City Project
On Sep. 17th, 2009, Smart City Planning Inc. was established as Joint Venture Company of leading companies for the management of Smart City Project.
As of Sep. 2012, 22leading companies joined the project.
http://www.smartcity-planning.co.jp/
Nikkei Smart City Consortium
http://expo.nikkeibp.co.jp/scw/2011/exhibitors/english/http://expo.nikkeibp.co.jp/scw/2012/exhibitors/english/
http://bizgate.nikkei.co.jp/smartcity/
Numbers of Smart City Projects
http://itpro.nikkeibp.co.jp/article/Keyword/20120906/421044/ As of September 12, 2012
China
JapanIndia
Africa
Far-East
NorthAmerica
SouthAmerica
PacificOcean
Europe
Toyota Smart Center in 20xx
Smart u-Things – Real World Challenges
Smart u-Things are emerging- Many scenarios: Weiser’s Sal, AmI’s Maria/Dimitrios/Carmen, Aura’s Jane/Fred, …- Lots of research and many various prototypes Research Booming comes- But rare practical ones Application Booming is to come real world complexity
Ideal Smart u-Things expected and related challenges Able to act adaptively and automatically according to
1. Surrounding Situations Challenge 1: Situation Approximation
2. Users’ Needs Challenge 2: Knowing Users’ Needs
3. Things’ Relations Challenge 3: Complex Things’ Relations
4. Common Knowledge Challenge 4: Knowledge Management/Growth
5. Own Goal, Role, etc. Challenge 5: Self Awareness
6. Error and Exception Challenge 6: Looped Self Adjustment
7. Safety & Satisfaction Challenge 7: UbiSafe
Above challenges from real world intrinsic characteristics- RW == physical + social + natural + … uncertain, unpredictable, changing, …- RW computing: complicated/abstruse phil., social, ethical & other implications- Understanding real world (RW) diversity and complexity Extremely Hard !!- Novel cyber dimensions newly added in physical-digital combined u-things
Challenge 1 – Situation Approximation
Context- Information characterizing the situation of an entity (Dey) or 5Ws (Abowd)
- Whole contexts are a collection of various values from sensors or other sources
C(t) = {Ci(Bi, L, th), i∈I(t), th∈[t-h, t]}
Situation (Activity)- Relatively compact, more semantic, directly used for judgment/decision
S(t) =φ[C(t)] = {Sj(Wj, t), j∈J(t)}Designing a Context-aware System to Detect Dangerous Situations in School Routes for Kids Outdoor Safety Care, LNCS, Proc. of EUC'05.
C(t), S(t) approximations to a real environment- Are the contexts C(t) sufficient and precise enough to characterize a real environment?
- How correctly be S(t) determined using available but incomplete & uncertain contexts?
- What are consequences of situation judgment errors to context-aware/situated u-things?
Nyquist Sampling Theorem & Shannon Entropy/R(D) Theory Theory for context/situation?
Keep incompletion, uncertainty and misjudgment as basic design assumptions !
Challenge 2 – Knowing Users’ Needs
Interactive Mechanism- A request-response dialog process betw. users (activator) & computer (passive)- 3 user limits: too small/many/complex computers to be visible/interact-able/manageable
Proactive and Autonomic Mechanisms- Take actions proactively by anticipating users’ needs (Tennenhouse, Intel, 2000)
- Manage themselves under human supervisions/needs (Paul Horn, IBM, 2001)"Proactive Control of Group Revision Assistance Management Using P2P Technology" (Takata & Ma, ISCIT04)
Journal of Autonomic and Trusted Computing (JoATC, EiC, Laurence T. Yang and Jianhua Ma), ASP, USA
1st Int’l Wksp on Trusted and Autonomic Ubiquitous & Embedded Systems (TAUES-05, Japan, Dec. 2005)
2nd International Workshop on Trusted and Autonomic Computing Systems (TACS-06, Austria, Apr. 2006)
3rd Int’l Conf. on Autonomic & Trusted Computing (ATC06, China) ATC07 (HK) ATC08 (NW) ATC09 (Australia)
Users’ Needs- Multi aspects and subtle affection by many factors
N(t) = {Nu,v[Pu,v(t), S(t), S(t-Δ)], u∈U(t), v∈V(t)}
Hardness of knowing users’ true needs- How much can be known correctly and promptly for users’ true needs in changing situations?
- “Know you and know your face, but don’t know your mind” - Chinese saw
Challenge 3 – Complex Things’ Relations
Complex Relations among u-Things- “Everything will be connected to everything else” - by R. Lucky, 1999
- “what are the consequences of any given action (including simply walking into a room)”
- by M. Weiser, 1999
Cyber dimensions, except spatial/temporal/other conventional dimensions Complex dynamic relations among users, u-things, and other things
R(t) = {Rq(Tq, Uq, t), Uq∈U(t), q∈Q(t)}
- How to define, find, describe and use complex relations necessary for u-things’ systems?Smart Hyperspace: a set of interconnected smart spaces with situational-spatial-temporal relations
(“Modeling Interface with a Multimedia Hyperworld”, HIS’96, “Smart Hyperspaces and Project Ubikids”, USW05)
users
non u-thingsu-things
Challenge 4 – Knowledge Mgmt/Growth
Why common knowledge needed?- Knowledge is the base of analysis, reasoning, anticipation and judgment (Ref AI)
- Smart u-things are in the physical environments and serve people’s daily life and work- Smart u-things need some common knowledge about physical world, human society, etc.
K(t) = {Km(Fm, th), m∈M(t), th∈[t0, t]}
Issues are- What knowledge is necessary for smart u-things? - What knowledge should be initially set? - What knowledge can be added later on?- What knowledge may be self-learned during uses?- How knowledge is used for rich and varied real situations?
Challenge - how to abstract, learn, use complex knowledge about human & world
How about DAI, swarm, softcomp, agent, semantics, etc.?
Challenge 5 – Self-Aware u-Things
Why smart u-things should be self-aware- Physical, not virtual, things with attached/embedded/blended (AEB) computers/etc.- AEB devices are function parts or components of the real physical things- AEB is for enhancing original functions of real things following common rules- Smart u-things should be aware their roles and function without against the rules
A(t) = {Az(Gz, t), z∈Z(t)}
Context-awareness vs self-awareness- Context-awareness: knowing others- Self-awareness: knowing oneself
“Knowing others is wisdom, knowing yourself is enlightenment” - Tao Tzu
- What this exactly means? How to equip self-awareness to u-things?
Extreme Challenging in this research“we seem so ill equipped to understand ourselves”, The Society of Mind, Minsky, 1986
Workshop on Self-Aware Computer Systems - Chaudhri & McCarthy, SRI/DARPA, 2004
Challenge 6 – Looped Self-Adjustment
Decisions cannot be correct always in 100% Imprecise decision & exception are common
Loop mechanism is indispensable !
Challenge - adaptively correct errors & make adjustments How is this related to automatic control theory?
D(t) =ψ[S(t), N(t), R(t), K(t), A(t), D(t-Δ), E(t)]
ψS(t)
N(t)
R(t)
D(t)
K(t) A(t)
actions
D(t-Δ)
E(t)χ
Error(t)
Exception(t)
Challenge 7 – Ubisafe (Ubiquitous Safety)
Unobtrusive AEB devices in real objects and spaces Physical unsafe factors
Diverse Users with different ages/backgrounds/demandsHuman unsafe factors
Desired States of ubicomp environment
Calm Technology by Weiser and Brown
UbiSafe Ubiquitous Safety & Satisfaction
Basic Question
How to build a computing environment in which all people can fully enjoy the u-services under complex situations without worrying or thinking the safety problem any more?
Ubiquitous Comp Ubiquitous Services Ubiquitous Dangers UbiSafe
Ubicomp: Physical-Cyber Loop & Consequence
Cyber EnvironmentsComputers, Components
Software, Service, …
UserObject
Physical Input
Physical Output
Trust Safety
measure, model, monitor, management, analysis, evolution, …
Consequence of misbehavior or
unexpected outputfrom trustor/trustee?
Sensors
Actuators
Automatic PhyCybPhy
Loop
User Protection- Tech. + Social
Safety Guarantee- Sec. + Rel. + …
LossUnsafeDanger
B-2 Spirit Crash
On 23 February 2008, a B-2 crashed back onto a runway shortly after takeoff from Andersen Air Force Base in Guam. The aircraft was completely destroyed, a total loss estimated at US$1.4 billion.
The findings of the subsequent investigation stated that the B-2 crashed after "heavy, lashing rains" caused water to enter skin-flush air-data sensors, which feed angle of attack and yaw data to the computerized flight-control system. The water distorted preflight readings in three of the plane's 24 sensors, causing flight-control system to send an erroneous correction to the B-2 on takeoff.Because of the faulty readings, the flight computers determined inaccurate airspeed readings and incorrectly indicated a downward angle for the aircraft, which contributed to an early rotation and an un-commanded 30-degree pitch up and left yaw, resulting in the stall.
A Boy Killed by an Auto-door
2004/3/27、Asahi Newspaper
2004/4/20、
日本経済新聞2004.3.26 : a boy died due to a rotation door
SensingRanges
From Prof. Z. Cheng
Elevator Accidents
朝日新聞2006年6月8日
朝日新聞・産経新聞2006/6/17
2006.06.03, a high school student dead when he took a bicycle into a
elevator, which started to move without completely closing the door.
If pushing the open button just after the door
has been closed, the elevator may start to go
up with the door openedFrom Prof. Z. Cheng
Novel Features and Unsafe Factors
Unobtrusive AEB in real objects and environments Physical characteristic oriented unsafe factors
- Limited computation, open/changing/worse working conditions
- Used consciously or unconsciously
Diverse users with different backgrounds/demandsAll people from baby to elder, normal to disabled, ……
Human characteristic oriented unsafe factors
- No comp knowledge, no expected usage, no awareness of
dangers, no ability of handling, ……
- Safe/unsafe is not absolute, relative, up to individual/situation
Life-like systems, smart u-things from small to large scales Passive Interactive Active Life-like
life-like system characteristic oriented unsafe factors
- Imprecise sensing data, insufficient context, complex relation, ……
Ubiquitous Comp/Dev./Net Ubiquitous Risks/Dangers
The Internet … a Network of Networks that consists of millions of
private, public, academic, business, and government networks, of
local to global scope. - From Wikipedia
The Internet Internet of Computers (IoC)Leonard Kleinrock Lawrence Roberts Steve CrockerJon Postel Vinton Cerf Robert Kahn
What are the Internet?
Originated from the ARPANET around 1970
Available from 1980, got popular from 1990.
Key components
- Hardware: Routers connecting networks
- Software: TCP/IP protocol suite, IPv4 IPv6
- Addressing: 2**32 (IPv4) 2**128 (IPv6)
- Naming: DNS symbolic names
A Global Net
Connecting All
Computers
physical net
router
The World Wide Web, abbreviated as WWW or the Web, is a system
of interlinked documents accessed via the Internet. - From Wikipedia
The Web was originated from Tim Berners-Lee around 1990.
The Web, like Email, is one of the services that runs on the Internet.
Key components
- Uniform Resource Locator (URL) & Uniform Resource Identifier (URI)
- HyperText Markup Language (HTML)
- Hypertext Transfer Protocol (HTTP)
- Web server and web browser (client)
What are the Web?
Tim Berners-Lee Ted Nelson Hypertext Mark Andreesen
Netscape
1st Web Server
1st Web
Graphical
Browser
The Web Internet of Documents (IoD)
How about Social Media/Web2.0?
SM/Web2.0 Internet of People (IoP)
How about Cloud Computing?
Software
Cloud Internet of Resources (IoR)
Thing - An object, an entity, an idea, or a quality perceived, known, or
thought to have its own existence, … (dictionary)
Object – A tangible/visible thing; a person or thing seen as a focus or
target for feelings, thought, etc.; a purpose/objective; ... (dictionary)
Everyday Things/Objects – used in human daily lives
Inner Things – mind, directly insensible things, ...
Physical things, digital things, real/virtual things, …
What are Things?
Various Things! Many IoX!
IoCIoD
IoPIoR
What Kinds of Things in IoT?
u-Things in Physical/Real World
Physical World
u-Thing
Usual Activity
Sensor/M/NEMS, Comps & Per. Nets
Physical Space
UC, ID, Context, Emb. Sys., Cloud, …
u-Things: on the physical space in physical-digital form
e-Thing
e-Activity
Cyber World
Computers & Networks/Internet
Cyber Space
WbS, SmW, Grid, P2P, XaaS, Cloud
IoT u-Things in Physical World
IoT Not Anything Special Objects
The term “Internet of things” was first coined by the former Auto-ID
Center, founded in 1999, based at the time at MIT. (Kevin Ashton
and David L. Brock) RFID-based EPC Only term!
(Sean Dodson, 2003) "IoT" can be expressed as the building of a
global infrastructure for RFID tags.
@ You could think of it as a wireless layer on top of the internet
where millions of things from razor blades to euro banknotes to car
tyres are constantly being tracked and accounted for.
@ A network … is for computers to identify "any object anywhere in
the world instantly".
@ Put a tag - a microchip with an antenna - on a can of Coke or a
car axle, and suddenly a computer can 'see' it.
The Origination of IoT
Formal Introduction of IoT“We are heading into a new era of ubiquity, where the users of the Internet will be counted in billions, and where humans may become the minority as generators and receivers of traffic. Changes brought about by the Internet will be dwarfed by those prompted by the networking of everyday objects” – UN report, 2005
The Internet of things, also known as the Internet of objects, refers to
the networked interconnection of everyday objects. It is described as a
self-configuring wireless network of sensors whose purpose would be
to interconnect all things.
From Wikipedia
(IoT is) a network of interconnected computers to a network of
interconnected objects, from books to cars, from electrical appliances
to food, and thus create an ‘Internet of things’. These objects will
sometimes have their own Internet Protocol addresses, be embedded
in complex systems and use sensors to obtain information from their
environment and/or use actuators to interact with it.
From Internet of Things — An action plan for Europe (2009)
“Internet of Things” to refer to the general idea of things, especially everyday objects, that are readable, recognizable, locatable, addressable, and/or controllable via the Internet—whether via RFID, wireless LAN, wide-area network, or other means.
From SRI Consulting Business Intelligence
Other Definitions of IoT
Examples of Things in IoT
Many physical things are connected, can talk 全てのモノが繋がる世界 対話(通信)できる
Ubiquitous NetworksObject-To-Object (O2O), Thing-To-Thing (T2T)
Ubiquitous Connection of All Things
Internet of Everything
Companies/Societies
Individuals
SEA-net
Internet
/WWW
Cyber space
Physical Space
Social Space
Cyber networks
Physical networks
Social networks
Big Data
Big Data
Big Data
IoP
IoD
IoT
IoC
IoT Platform and Applications
From CERP-IOT
Videos About IoT
IOT will change everything from Cisco (30 videos)
Internet of Things by Dr. John Barrett at TEDxCIT
Harnessing the Internet of Things by Steve Lucas
Freescale's IoT Applications
An Introduction of Internet of Things from IBM