cognitive systems in the fp6 ist programme ist 2004, the hague, 16th november 2004

Cognitive Systems in the FP6 IST Programme IST 2004, The Hague, 16th November 2004


CoSyCognitive Systems for Cognitive Assistants

Henrik I ChristensenKungl Tekniska Högskolan

Jeremy Watts & Aaron SlomanUniversity of Birmingham

Hans Uszkoreit & Geert-Jan KruijffDFKI

Bernhard Nebel & Wolfram BurgardAlfred Ludvig University of Freiburg

Bernt SchieleTechnical University of Darmstadt

www.CognitiveSystems.org

J. Kevin O’ReganCNRS, University of Paris V

Ales LeonardisUniversity of Ljubljana


GOALAdvance the Science of Mind

TARGET DOMAINHuman-like autonomous physical (robotic) systems

e.g. machines to help house-bound people who can’t rely on others

FOCUSIntegration of capabilities- perception- action- representations - interaction- reflection ...

PRACTICAL VALIDATIONExplorer robot

Playmate robotPhilosopher agent


CORE IDEASelf-modifying architecture(reactive, deliberative, reflexive)

CORE IDEACombine many capabilities:

Deployed concurrentlyInteracting asynchronouslyDeveloping over time

CRITICAL ISSUESLife-long open-ended incremental learningBalance between innate and acquired knowledgeCommunication: language & representations

CRITICAL ISSUESMultiplicity of representations

Recognition of affordancesCoping with uncertainty

Introspection


CoSyCognitive Systems for Cognitive Assistants

www.CognitiveSystems.org

Develop the Fundamentals of Cognitive SystemsAsk and Answer the Big Questions!

Think: Integration


JASTJoint-Action Science and Technology

Harold BekkeringInstitute for Cognition & Information

Ivan ToniCentre for Cognitive Neuroimaging

Radboud University Nijmegen(KUN)

Jan-Peter de RuiterMax Planck Institute for PsycholinguisticsNijmegen

Astros ChatziastrosMax Planck Institute for Biological CyberneticsTubingen

Nikolaos UzunogluInstitute of Communication &

Computer Systems, Athens

Ellen Gurman BardUniversity of Edinburgh

Alois KnollTechnical University of Munich

Estela BichoUniversidade do Minho

http://212.205.85.69/jast/index.htm


GOALExtend cognitive systems to multi-agent scenarios

TARGET DOMAINRobot-robot and human-robot co-operative joint action

FOCUSExploitation of shared beliefs, intentions, and goals

PRACTICAL VALIDATION

Two autonomous robotic agents co-operating on a

complex construction task

Joint-Action

Science and TechnologyScience and Technology


CORE IDEADialogue supported perception, reasoning, & action: cross-modal associations

CORE IDEASelf-organizing systems(non-linear dynamical systems model of cognition)

Inspired by brain imaging data

CRITICAL ISSUESImitationReal-time adaptive planningVerbal and non-verbal communication

CRITICAL ISSUES

Detection and repair of performance errors (derived

from both self and partner)

Joint-Action

Science and TechnologyScience and Technology


JASTJoint-Action Science and Technology

Co-operating Cognitive Systems Dynamical Models

Think: Shared Understanding

http://212.205.85.69/jast/index.htm


RobotCubRobotic Open-Architecture for

Cognition, Understanding, and Behaviours

Giulio SandiniDIST, University of Genova

Paolo DarioScuola Superiore Sant’Anna, Pisa

Luciano FadigaUniversity of Ferrara

Darwin CaldwellUniversity of Salford

Kerstin DautenhahnUniversity of Hertfordshire

www.RobotCub.org

Emilio BizziEuropean Brain Research Institute

Rolf PfeifferUniversity of Zurich

Claes von HofstenUniversity of Uppsala

Jose Santos VictorInstituto Superior Tecnico, Lisbon

Auke BillardEcole Polytechnique Federale de Lausanne

Francesco BecchiTeleRobot S.r.l.




Rodney BrooksMIT

Yasuo KuniyoshiUniversity of Tokyo

Hideki KozimaNICT (CRL)

Gordon ChengATR

www.RobotCub.org

Juergen KonczakUniversity of Minnesota


GOAL54 degree of freedom cognitive humanoid robot

TARGET DOMAINOpen research platform: humanoid robot size & shape of a 2 year old child

FOCUSCo-development of cognitive skills through exploration and manipulation


Hand-eye coordination

Grasp and manipulate lightweight objects

Crawl on all fours and sit up


CORE IDEA

Cognition emerges through

exploratory learning and co-development

through embodied physical interaction

CORE IDEA

Initially deals with immediate events

Increasingly acquires a predictive capability

CRITICAL ISSUESExplorationManipulationImitationGestural Communication

CRITICAL ISSUES

Phylogenic configuration vs

Ontogenic development

tAnticipation / Planning / Deliberation / Prediction




Cognition through Co-developmentExploration, Manipulation, Gesture, & Imitation

Think: Open Cognitive Humanoid Systems

www.RobotCub.org


COSPALCognitive Systems using Perception-Action

Learning

Gösta GranlundLinköping University

Josef KittlerUniversity of Surrey

Gerald Sommer

Christian-Albrechts-University of Kiel

www.cospal.org

Vaclav HlavacCzech Technical University in Prague


GOALNew architecture that combinesSymbolic Reasoning & Artificial Neural Networks

TARGET DOMAINRobotic exploration, Manipulation, & Problem solving

FOCUSSystem architecture able to semi-autonomously & incrementally build up knowledge and solution strategies


Emulate capability of small children to solve

manipulation puzzles


CORE IDEA

Action precedes perception:

Associations of perception and action are triggered by the system’s own actions

CORE IDEA

Bi-directional interface between - Perception-Action module- Symbolic module

CRITICAL ISSUESRobustnessGeneralizabilityScalability

CRITICAL ISSUES

Initial bootstrapping

Incremental learning

Attention

Exploration and adaptation


COSPALCognitive Systems using Perception-Action Learning

Combination of Symbolic Reasoning & Neural Networks

Think: Action-dependent Perception

www.cospal.org


MACSMulti-Sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances

Erich RomeFraunhofer AIS

Lucas PalettaJoanneum Research

Patrick DohertyLinköping University

Erol SahinMiddle Eastern Technical University, Ankara

www.macs-eu.org

George DorffnerÖFAI, Vienna


GOALNew architecture for mobilegoal-directed autonomous manipulation

TARGET DOMAINMobile manipulation robotics

FOCUSAffordance: ‘a resource or support that the environment offers an agent for action’, which the agent can directly perceive and employ(e.g. drivability of a track, graspability of an object)

PRACTICAL VALIDATIONMobile manipulator robot

using vision, 3-D laser scanner, 2 DoF arm & magnetic gripper


CORE IDEA

Use affordance to couple perception, action, and reasoning

CORE IDEA

Perception of affordances implies interpretation based on capabilities for physical action and current goals

Acquired by learning

CRITICAL ISSUESArchitecturePerceptionRepresentationLearning

CRITICAL ISSUES

Control based on action-oriented interpretation

(affordance-based control)


MACSMulti-Sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances

Affordance-based integration of perception, action, and reasoning

Think: Affordances

www.macs-eu.org


SPARKSpatio-temporal Patterns for

Action-oriented Perception in Roving Robots

Paolo ArenaUniversity of Catania

Ákos Zarándy ANALOGIC

Ángel Rodrígues VázquezANAFOCUS

www.spark.diees.unict.it

Barbara WebbUniversity of Edinburgh

Holk CruseUniversity of Bielefeld

Manuel VelardeIP-UCM, Universidad Computense, Madrid


GOALNew biologically-inspired architecture for action-oriented perception

TARGET DOMAINMobile robotics

FOCUSNon-linear dynamical systems theoryMulti-modal sensors


Robot guidance system


CORE IDEA

Perception is a spatio-temporal pattern formation process which directly influences the associated motor behaviour

CORE IDEA

Dynamically emergent patterns

Spatio-temporal array computer (SPARC)

CRITICAL ISSUESSynergeticsNon-linear dynamical systems theoryEmergent computationDynamic pattern formation

CRITICAL ISSUES

Integration of sensor stimuli

Concise iconic representation


SPARKSpatio-temporal Patterns for

Action-oriented Perception in Roving Robots

Perception-Action asEmergent Spatio-Temporal Patterns

Spatio-temporal Array Computer (SPARC)

Think: Non-Linear Dynamical Systems

www.spark.diees.unict.it


Mind RACESfrom Reactive to Anticipatory Cognitive Embodied Systems

Rino FalconeISTC-CNR

Christian BalkeniusLund University

Boicho KokinovNew Bulgarian University

Georg DorffnerOFAI

Ana Almeida e PaivaInstituto Superior Tecnico

www.mindraces.org

Fabio AdezioNoze s.r.l.

Joachim HoffmannUniversity of Wuerzburg

Juergen SchmidhuberScuola Universitaria Professionale

della Svizzera Italiana


GOALInvestigate different anticipatory cognitive mechanisms and architectures

TARGET DOMAINEmbodied systems

FOCUSAnticipation


Simulations

Robotic platforms


CORE IDEA

Anticipation is a key component of cognition

CORE IDEA

Comparative analysis of models of anticipation, and development of models

CRITICAL ISSUESExpectation-driven attention, monitoring, and controlGoal-directed behaviour, pro-active planning, and prediction by analogy

CRITICAL ISSUES‘Anticipatory emotions’

Performance metrics


Mind RACESfrom Reactive to Anticipatory Cognitive Embodied Systems

Comparative Analysis & Development of Anticipatory Systems

Think: Anticipation

www.mindraces.org


GNOSYSAn Abstraction Architecture for Cognitive Agents

John G. TaylorKing’s College London

Christos EmmanouilidisZenon S.A.

Panos Trahanias

Foundation for Research and Technology - Hellas

Hanspeter Mallot

University of Tuebingen

Pietro MorassoDIST, University of Genova

www.ics.forth.gr/gnosys


GOALComplete conceptual architecture for cognitive agents

TARGET DOMAINRobot navigation & surveillance

FOCUSComplete integration of the perception-knowledge acquisition-abstraction-reasoning- action generation cycle


Patrolling, incident detection, and tracking in unknown

outdoor environments


CORE IDEA

Abstraction creates concept hierarchies

Reasoning then makes inferences for virtual actions

CORE IDEA

Action selection based on greatest reward

Goals-oriented model to fuse user-derived and agent-derived tasks/goals

CRITICAL ISSUESAttention (complexity, priority, novelty, goals)Reinforcement learningActions modify current knowledge through perception

CRITICAL ISSUES

Inspired by human cognitive processes


GNOSYSAn Abstraction Architecture for Cognitive Agents

Conceptual Architecture

Think: Abstraction, Concepts, Reasoning

www.ics.forth.gr/gnosys


CoSy JAST RobotCub COSPAL MACS SPARK Mind GNOSYSRACES

Architecture

Robotic

Exploration

Robotic

Manipulation

Incremental

Learning

Attention

Affordances

Language/

Gesture

Multi-Agent

Based on limited information


CoSy JAST RobotCub COSPAL MACS SPARK Mind GNOSYSRACES

Self-Modifying

Architecture

Self-Organization

Symbolic

Representations

Neural

Networks

Dynamical

Systems

Based on limited information


CoSy

JAST

RobotCub

COSPAL

MACS

SPARK

Mind RACES

GNOSYS

Integration

Shared Understanding

Open Cognitive Humanoid System

Action-dependent Perception

Affordances

Emergent Spatio-temporal Patterns

Anticipation

Abstraction, Concepts, Reasoning

THINK …Project

cognitive systems in the fp6 ist programme ist 2004, the hague, 16th november 2004

Documents

fp6 ist programme ist

cosy cognitive systems

technology slide

integration slide

cognitive assistants

perception action

jast jointaction science

reflexive core idea