cognitive computer vision
DESCRIPTION
Cognitive Computer Vision. Kingsley Sage [email protected] and Hilary Buxton [email protected] Prepared under ECVision Specific Action 8-3 http://www.ecvision.org. Course outline. What is Cognitive Computer Vision (CCV) ? Generative models Graphical models - PowerPoint PPT PresentationTRANSCRIPT
Cognitive Computer Vision
Kingsley [email protected]
Hilary [email protected]
Prepared under ECVision Specific Action 8-3http://www.ecvision.org
Course outline
What is Cognitive Computer Vision (CCV) ? Generative models Graphical models Techniques for modelling cognitive aspects of CCV
– Bayesian inference– Markov Models
Research issues Coursework and case studies
So what is CCV ?
D. Vernon, Dagstuhl 2003 Monday 27th October 2003
Bernd Neumann, 2003 (ECVision Summer School on Cognitive Vision)
Cognitive Vision research requires multidisciplinary efforts and escape from traditional research community boundaries.
Computer Vision• object recognition, tracking• bottom-up image analysis• geometry and shape• hypothesize-and-test control• probabilistic methods
Knowledge Representation & Reasoning• KR languages• logic-based reasoning services • default theories• reasoning about actions & change• Description Logics• spatial and temporal calculi
Robotics• planning, goal-directed behaviour• manipulation• sensor integration• navigation• localization, mapping, SLAM• integrative architectures
Learning & Data Mining• concept learning• inductive generalization• clustering• knowledge discovery
Cognitive Science• psychophysical models• neural models• conceptual spaces• qualitative representations• naive physics
Uncertain Reasoning• Bayesian nets, belief nets• decision & estimation• causality• probabilistic learning
Natural Language• high-level concepts• qualitative descriptions• NL scene descriptions• communication
CognitiveVision
Cognitive Systems LaboratoryCSL
So what is CCV ?
In this course, we focus on using of ideas from cognitive science and psychology to do CCV
To show how we can build effective CCV systems that are more robust and more capable of solving non-trivial problems than those that do not embrace these ideas
Use statistical inference and machine learning as our tools for modelling cognitively inspired processes
We are not claiming “hard AI” in this course
Key Cognitive Elements
Objects, events, activities and behaviours– “What is it that we are observing?”
Attention and control– “How is it that we observe?”
Key Cognitive Elements
Visual learning and memory– Representation of objects and their behaviour– Recognition– Categorisation– These are “what” problems
Visual control and attention– Perception for tasks using models of expectation– Goals, task context– Resources, embodiment– These are “how” problems
Cognition– From perception to action
Key Cognitive Elements
Visual learning and memory - examples– Learning about objects and how their appearance
can change– Recognising activities by the interactions between
objects– Extracting invariant models from training data
Learning and “recognising” objects
(Murase and Nayar, 1996)
Learn and recognise activities
Coupled Hidden Markov Models (CHMM) techniques(Oliver, Rosario & Pentland, 1999)Activities with interactions via coupled states in a HMM
Learning invariant models
Variances for 3 clusters
Means for 3 clusters
Key Cognitive Elements
Visual control and attention– A framework for attentional control– Inferring likely behaviour using Bayes nets– Deictic markers– Attentional selection of objects
A Framework For Task Based Visual Control
Scene Interpretation
……
CONTROL POLICY(WITH STATE MEMORY)
FEATURECOMBINATION
d1 d2dNImage Data
Driven
Task Based Control
BBN Inference of likely vehicle tracks
Fixed camera gives direct set ofdependencies Image Grid PositionBBN has size/orient hidden nodes Leaf nodes ls1/2, lo1/2 observables
IGP
size orient
ls1 lo1 lo2 ls2
Gong and Buxton, 1993
Deictic Markers in inference of behaviour
Left: attention for overtake (overtaken & overtaking vehicle)
Right: attention for giveway (stopped & blocker vehicle plus
ground-plane conflict zone)
Howarth and Buxton,1996
Attentional selection using eye gaze
Attentional selection using predicted trajectory data
Attentional selection using predicted trajectory data
Attentional selection using predicted Space of Interest
Summary
Cognitive Computer Vision is a multi-disciplinary area of research
Here we use statistical inference and learning for robust models
Task based attentional control is key to prediction and cognitive systems design
Useful reference: “Visual surveillance in a dynamic and uncertain world” Buxton, H and Gong, S, Artificial Intelligence 78, pp 431-459, 1995
Next time …
Generative models– What are they?– Why are they so important to Cognitive Vision?