prof. a. taleb-bendiab, talk: whole body interaction workshop07, date: 03/06/2014, slide: 1...
TRANSCRIPT
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction Workshop’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction Workshop’07, Date: 10/04/2310/04/23, Slide: , Slide: 11
Bio-Cybernetic Control for Bio-Cybernetic Control for WBIWBI: :
a bird’s eye viewa bird’s eye view
Prof. A. Taleb-BendiabProf. A. Taleb-BendiabSchool of ComputingSchool of Computing
Liverpool John Moores UniversityLiverpool John Moores Universityemail: email: [email protected]@livjm.ac.ukhttp://www.cms.livjm.ac.uk/talebhttp://www.cms.livjm.ac.uk/taleb
Bio-Cybernetic Control for Bio-Cybernetic Control for WBIWBI: :
a bird’s eye viewa bird’s eye view
Prof. A. Taleb-BendiabProf. A. Taleb-BendiabSchool of ComputingSchool of Computing
Liverpool John Moores UniversityLiverpool John Moores Universityemail: email: [email protected]@livjm.ac.ukhttp://www.cms.livjm.ac.uk/talebhttp://www.cms.livjm.ac.uk/taleb
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 22
OutlineOutline In the context of the Whole Body Interaction (WBI)In the context of the Whole Body Interaction (WBI)
DefinitionsDefinitions Biocybernetics, control, Augmented Reality, WBIBiocybernetics, control, Augmented Reality, WBI
Drivers for WBIDrivers for WBI Overview of emerging requirements for WBI, Overview of emerging requirements for WBI,
Technical challenges, requirements Technical challenges, requirements State-of-the-art relevant to bio-cybernetics control for State-of-the-art relevant to bio-cybernetics control for
WBI including: WBI including: Bio and socio-inspired computational modelsBio and socio-inspired computational models
ambient intelligence, ambient intelligence, Autonomic ComputingAutonomic Computing MiddlewareMiddleware body area sensor and actuator networks, body area sensor and actuator networks, cognitive systems cognitive systems
Underpinning technologies, Underpinning technologies, Conceptual frameworks for bio-cybernetics controlConceptual frameworks for bio-cybernetics control ConclusionsConclusions
Open questions research challenges relevant to WBIOpen questions research challenges relevant to WBI Q&AQ&A
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 33
Some Definitions #1Some Definitions #1 Biological Cybernetics (Bio-Cybernetics)Biological Cybernetics (Bio-Cybernetics)
… … is the application of cybernetics theory and is the application of cybernetics theory and systemicssystemics principles to further our understanding principles to further our understanding of systems biologyof systems biology
integrate different levels of information and information integrate different levels of information and information processing to understand biological systems processing to understand biological systems
function and development of cellular, multicellular systems, function and development of cellular, multicellular systems, organisms, complex living systems and ecosystems. organisms, complex living systems and ecosystems.
[[http://www.biological-cybernetics.de/wiki/index.php/Main_Pagehttp://www.biological-cybernetics.de/wiki/index.php/Main_Page]] [Source: Wikipedia[Source: Wikipedia]]
Cybernetics Cybernetics the study of control and communication in the the study of control and communication in the
animal and the machine including: animal and the machine including: organization, information flows, controlorganization, information flows, control
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 44
Some Definitions #2Some Definitions #2 Augmented RealityAugmented Reality: :
……. combination of real world and computer . combination of real world and computer generated datagenerated data
uses of live video imagery which is digitally uses of live video imagery which is digitally processed and "augmented" by processed and "augmented" by
the addition of computer generated graphics. the addition of computer generated graphics. Advanced research includes the use of motion tracking Advanced research includes the use of motion tracking
data, etc.., data, etc.., and the construction of controlled environments and the construction of controlled environments
containing any number of sensors and actuators.containing any number of sensors and actuators. [http://en.wikipedia.org/wiki/Augmented_reality][http://en.wikipedia.org/wiki/Augmented_reality]
Mixed reality:Mixed reality: refers to the merging of real and refers to the merging of real and virtual worldsvirtual worlds to produce new to produce new
environments and visualisations where physical and digital environments and visualisations where physical and digital objects co-exist and interact in real time.objects co-exist and interact in real time.
[http://en.wikipedia.org/wiki/Mixed_reality][http://en.wikipedia.org/wiki/Mixed_reality] Virtual Worlds:Virtual Worlds:
“… “… simulated environment intended for its users to inhabit simulated environment intended for its users to inhabit and interact via avatars …” and interact via avatars …”
[http://en.wikipedia.org/wiki/virtual_world][http://en.wikipedia.org/wiki/virtual_world]
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 55
Socially Augmented Intelligent PICTSocially Augmented Intelligent PICT
HospitalNetwork General
PractitionerNetwork
Home Network
GymnasiumNetwork
Mobile DistrictNurse Network
Real-Time Communications
Real-Time Communications
Real-Time Communications
Real-Time Communications
HospitalNetwork General
PractitionerNetwork
Home Network
GymnasiumNetwork
Mobile DistrictNurse Network
Real-Time Communications
Real-Time Communications
Real-Time Communications
Real-Time Communications
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 66
Convergence of Real and Virtual Convergence of Real and Virtual Worlds: Worlds: Second LifeSecond Life
Second LifeSecond Life Virtual WorldsVirtual Worlds hand-held and wearable hand-held and wearable
systems that act as systems that act as gateways between the gateways between the real and virtual worldsreal and virtual worlds
Eyewear, display badges Eyewear, display badges and speakers worn about and speakers worn about the neck will allow us to the neck will allow us to live more fully through live more fully through our avatarsour avatars
a wearable box that a wearable box that creates a 3D sound field creates a 3D sound field that allows the wearer to that allows the wearer to hear voices from the hear voices from the virtual world without virtual world without completely shutting out completely shutting out the real people around the real people around himhim
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 77
Vodafone in Second LifeVodafone in Second Life
More More recently recently Vodafone Vodafone now allows now allows you to you to interact with interact with your real your real phone via a phone via a Second Life Second Life avatar of the avatar of the phonephone
The real phone can be The real phone can be answered from within SLanswered from within SL
Text messages can be sent and Text messages can be sent and received between real-world received between real-world and SLand SL
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 88
Sensors that Control Second Sensors that Control Second Life ObjectsLife Objects
Sending external data into virtual worlds such as Sending external data into virtual worlds such as Second LifeSecond Life
Using Potentiometer the cube can be controlledUsing Potentiometer the cube can be controlled
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 99
Importance of WBIImportance of WBI WBIWBI
InterfaceInterface Transparency and virtualisationTransparency and virtualisation
Systems and Processes Systems and Processes InteractionInteraction Behaviour Behaviour Structure/organisationStructure/organisation
AffectiveAffective Sensing and Actuation Sensing and Actuation
Co-evolveCo-evolve Technical RequirementsTechnical Requirements
Protocols for WBIProtocols for WBI Standards and legacy systems Programming, Interaction and Control Models Usability Assurance
QoS Safety and Security, Etc.
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1010
Example: Bio-Cybernetics Example: Bio-Cybernetics ControlControl
RS232
TINI MicrocontrollerETS Service
802.11b
802.11b
Heart MonitoringService
RJ-45 Ethernet
RS232
RJ-45 Ethernet
BIOPAC MP150
RS232RS232
TINI MicrocontrollerETS Service
802.11b802.11b802.11b
802.11b802.11b802.11b
Heart MonitoringService
Heart MonitoringService
RJ-45 EthernetRJ-45 Ethernet
RS232RS232
RJ-45 EthernetRJ-45 Ethernet
BIOPAC MP150BIOPAC MP150
TIME
VOLTS
10
5
0 1 2 Sec
TIME
VOLTS
10
5
0 1 2 Sec
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1111
Virtual LabVirtual Lab
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1212
IBM Sensors Link Real and IBM Sensors Link Real and VirtualVirtual
Sensor data is visualised in real-timeSensor data is visualised in real-time Blue balls with white designs represent Blue balls with white designs represent
active Bluetooth devicesactive Bluetooth devices
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1313
The Story so Far The Story so Far #1#1 State-of-the-art in Situated AmI including;State-of-the-art in Situated AmI including;
autonomic software models and autonomic software models and architecture, standards architecture, standards
tools and techniques to support tools and techniques to support the design, modelling, analysis the design, modelling, analysis and evolution of autonomic software and evolution of autonomic software
Define models for their Define models for their programming, control programming, control interaction models with human and/or interaction models with human and/or
other non-AC systems (legacy). other non-AC systems (legacy). Delegation of authority and its Delegation of authority and its
adjustmentadjustment
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1414
The Story so Far The Story so Far #2#2 Has been informed by a set of design paradigms Has been informed by a set of design paradigms
Model-based vs self-organising systems design Model-based vs self-organising systems design modelsmodels
Top-down Top-down vsvs bottom-up bottom-up Applying and/or revisiting:Applying and/or revisiting:
cybernetic principlescybernetic principles control systems theory, regulation, reward control systems theory, regulation, reward
and sanctionsand sanctions Decision theory, DAI and CIDecision theory, DAI and CI
dynamic planning, deliberative models, MLdynamic planning, deliberative models, ML Middleware supportMiddleware support
self-awareness, reflection and deliberationself-awareness, reflection and deliberation etc.etc.
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1515
Research Challenges #1Research Challenges #1 Conceptual Design and Management modelsConceptual Design and Management models
“… “… how do we design, build, and evolve how do we design, build, and evolve such systems so that they can meet such systems so that they can meet given—and evolving—requirements ...” given—and evolving—requirements ...” Incremental deployment of SAmI Incremental deployment of SAmI
capabilities in legacy systems. capabilities in legacy systems. Hot-swapping, Dynamic AOP-based Hot-swapping, Dynamic AOP-based
evolution, Interoperationevolution, Interoperation Programming, control and Programming, control and
Interaction Models Interaction Models Usability and Interface Usability and Interface
Balancing and adjusting Balancing and adjusting governance and autonomygovernance and autonomy
Social and other requirementsSocial and other requirements Not covered hereNot covered here
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1616
Cybernetics: The Viable System Cybernetics: The Viable System Model Model
Beer’s VSM implements a Beer’s VSM implements a control & communication control & communication structure via hierarchies of structure via hierarchies of homeostats (feedback loops) homeostats (feedback loops) (1950) (1950)
Defines 6 major systems Defines 6 major systems ensure ‘viability’ of the systemensure ‘viability’ of the system ImplementationImplementation S1S1 MonitoringMonitoring S2S2 AuditAudit S3*S3* ControlControl S3S3 Intelligence Intelligence S4S4 PolicyPolicy S5S5
Offers an extensible, Offers an extensible, recursive, model-based recursive, model-based architecture, devolving architecture, devolving autonomy to sub-systemsautonomy to sub-systems
System 5Policy
System 4Intelligence
System 3Control
System 3*Audit
System One (S1)
5
4
3
2
2
2
2
3*
1
Org
aniz
atio
nal E
nviro
nmen
t
System 2Coordination
System 2Coordination
System One (S1)
5
3
43*
1
System 2Coordination
System One (S1)
5
3
43*
1
System 2Coordination
AutonomicSystems
CognitiveSystems
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1717
Situated AmI CapabilitiesSituated AmI Capabilities A software system is A software system is
autonomic, if it possesses autonomic, if it possesses the following capabilities:the following capabilities: Self-configuringSelf-configuring —— choosing a choosing a
suitable behaviour, based on suitable behaviour, based on user preferences, context, …user preferences, context, …
Self-tuningSelf-tuning —— choosing choosing behaviours that optimize behaviours that optimize certain qualities (performance, certain qualities (performance, year-end profits, …)year-end profits, …)
Self-repairingSelf-repairing —— shifting shifting execution to another behaviour, execution to another behaviour, given that the current one is given that the current one is failingfailing
Self-protectingSelf-protecting —— choosing a choosing a behaviour that minimizes risks behaviour that minimizes risks (attacks, viruses, …)(attacks, viruses, …)
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1818
Our Approaches Our Approaches #1#1 Related Work: Related Work:
IBMIBM blueprint blueprint (www-03.ibm.com/autonomic/pdfs/ACBP2_2004-10-(www-03.ibm.com/autonomic/pdfs/ACBP2_2004-10-04.pdf)04.pdf)
An autonomic manager contains a continuous control loop An autonomic manager contains a continuous control loop that monitors activities and takes actions to adjust the that monitors activities and takes actions to adjust the system to meet business objectivessystem to meet business objectives
Autonomic managers learn from past experience to build Autonomic managers learn from past experience to build action plansaction plans
Elements need to be instrumented consistently, based on Elements need to be instrumented consistently, based on open standardsopen standards
Our model – J ReferenceOur model – J ReferenceMonitor Execute
Analyze Plan
Knowledge
Managed Element
Sensors Effectors
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 1919
Projects in the Networked Projects in the Networked Appliances Lab.Appliances Lab.
Composition of an Internet of Things Composition of an Internet of Things Infinitely Adaptable GamingInfinitely Adaptable Gaming Bridging between Physical and Bridging between Physical and
Virtual (Demo)Virtual (Demo) Managing 3D Multimedia StreamsManaging 3D Multimedia Streams
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 2020
What It Looks LikeWhat It Looks Like
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 2121
Prototype ImplementationPrototype Implementation
Paul Fergus, David Llewellyn-Jones, Amjad Shaheed, Madjid Merabti, Abdennour El Rhalibi, Networked Appliances: Manipulation of 3D Multimedia Streams, Accepted for publication in the 5th IEEE International Consumer Communications and Networking Conference: Workshop on Networking Issues in Multimedia Entertainment, 2008, Las Vegas, USA
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 2222
Augmenting FramesAugmenting FramesNetwork Interface
Virtual Environment
Meta Data Frame Data
Augmented Frame Engine
Game Engine
Network Interface
Network
Television Scene Model JPEG Image
3D MetadataEngine
2D FrameEngine
Render 3D Model Render Image
Device
Augmented Frame Engine
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 2323
Modelling AutonomyModelling Autonomy Our modelOur model
Using SSC used to formalizes the behaviour of Using SSC used to formalizes the behaviour of dynamically changing systems FOL (McCarthy, 1963).. dynamically changing systems FOL (McCarthy, 1963)..
Support concurrent actions and timing constraints. Support concurrent actions and timing constraints. Each situation can be viewed as a history of previous Each situation can be viewed as a history of previous
actions. actions. Action, guards and time can be modelled at Action, guards and time can be modelled at
deliberation points in an autonomic setting.deliberation points in an autonomic setting. M. Randles, A. Taleb-Bendiab, Philip Miseldine, Andy Laws, "M. Randles, A. Taleb-Bendiab, Philip Miseldine, Andy Laws, "
Adjustable Deliberation of Self-Managing SystemsAdjustable Deliberation of Self-Managing Systems", ", ECBS 2005ECBS 2005: 449-456. [: 449-456. [pptppt]]
Evolving and Adjustable AutonomyEvolving and Adjustable Autonomy Software governance (Control) viaControl) via
Formal modelling of norms, policiesFormal modelling of norms, policies Enactment support – from spec. to code Enactment support – from spec. to code
using using Neptune language Neptune language
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 2424
Programming Autonomic SystemsProgramming Autonomic Systems Neptune Meta-Language Neptune Meta-Language
Integrated Development Integrated Development Environment:Environment:
Miseldine, P., Taleb-Bendiab A. A Programmatic Miseldine, P., Taleb-Bendiab A. A Programmatic Approach to Applying Sympathetic and Approach to Applying Sympathetic and Parasympathetic Autonomic Systems to Software Parasympathetic Autonomic Systems to Software Design. to appear in the 2005 International Design. to appear in the 2005 International Conference on Self-Organization and Adaptation Conference on Self-Organization and Adaptation of Multi-agent and Grid Systems (SOAS’2005). of Multi-agent and Grid Systems (SOAS’2005).
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 2525
Conclusions and Future Conclusions and Future WorkWork Understanding WBIUnderstanding WBI
InterfaceInterface Transparency and virtualisationTransparency and virtualisation
Systems and Processes Systems and Processes InteractionInteraction Behaviour Behaviour Structure/organisationStructure/organisation
AffectiveAffective Sensing and Actuation Sensing and Actuation
Co-evolveCo-evolve Address the challenges including:Address the challenges including:
Protocols for WBIProtocols for WBI Standards and legacy systems Programming, Interaction and Control Models Usability Assurance
QoS Safety and Security, Etc.
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 2626
AcknowledgementsAcknowledgements Acknowledgements
My thanks to All CMS Research Team
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 2727
That’s the end – so I’m off !
Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: Prof. A. Taleb-Bendiab, talk: Whole Body Interaction’07, Date: 10/04/2310/04/23, Slide: , Slide: 2828
Extra Slides … Extra Slides …