tasks, abilities, and term projects
DESCRIPTION
Tasks, Abilities, and Term Projects. Paul Cohen. But first...Names!. Emily Oliver Curious George ... Come on, people! . What we want. Define a handful of tasks, each of which requires multiple abilities. Each ability is used for more than one task - PowerPoint PPT PresentationTRANSCRIPT
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Tasks, Abilities, and Term Projects
Paul Cohen
COMPUTER SCIENCEUASCIENCE
Paul Cohen
But first...Names!
• Emily• Oliver• Curious George• ...• Come on, people!
COMPUTER SCIENCEUASCIENCE
Paul Cohen
What we want
Task 1 Task 2 Task 3 Task N
Ability 1 X X X
Ability 2 X X
Ability 3 X X X
...
Ability N X X X
Define a handful of tasks, each of which requires multiple abilities. Each ability is used for more than one taskEach task requires more that one abilityThe outputs of some abilities are the inputs of othersEach team works on one ability (or a small number) but exercises it in all tasks
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Tasks
• Meet and Greet• Turntaking games• Tidying up• Whazzat (Wubble World)
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Meet and Greet
• Amr Gaber’s idea:– Robot sits somewhere public, greets people, learns to
recognize them and their names, learns to recognize moods and attitudes, learns social conventions re chatting...
• Examples– What to say to someone you do/don’t recognize– What to say when two people are talking to each other– What to say when people are hanging around/walking fast
“Hi Paul, nice to see you again. How’s Carole? I remember you were in the lab last thursday with Cody. He is here a lot. Sometimes I think he sleeps here. ...”
“Hi Ian, I see you are talking with someone and don’t want to interrupt, but could you introduce me to your friend?”
COMPUTER SCIENCEUASCIENCE
Paul Cohen
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Meet and Greet Abilities
Meet&Greet Turntaking Games
Clean Up Wubble World
Face recognition x
See someone’s there x
Recall previous chats x
Generate smalltalk x
Detect affect x
Recognize activities x
Physical control eyes
Learn stuff about people
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Turntaking Games
Peekaboo !
Pattycake !
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Turntaking Games Abilities
Meet&Greet Turntaking Games
Clean Up Wubble World
Face recognition x
See someone’s there x x
Recall previous chatsgames
x x
Generate smalltalk game interaction
x x
Detect affect x x
Recognize activities x x
Physical control eyes arm
Learn stuff about people
repetitive activities
classes of things and
where they go
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Clean Up
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Clean Up Abilities
Meet&Greet Turntaking Games
Clean Up Wubble World
Face recognition x
See someone’s there x x
Recall previous chatsgames plans
x x x
Generate smalltalk game interaction actions
x x x
Detect affect x x
Recognize activities x x x
Physical control eyes arm arm, wheels
Learn stuff about people
repetitive activities classes of things and where they go
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Go to the red cone
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Work with Daniel Hewlett and Wesley KerrWubble and room design by Teresa Dey
Wubble World
COMPUTER SCIENCEUASCIENCE
Paul Cohen
COMPUTER SCIENCEUASCIENCE
Paul Cohen
How Wubbles Learn the Meanings of Words
"Go to the blue cylinder"
goto
the blue cylinder
1. parse
4. act or ask
3. score correspondences between retrieved representation and scene
2. retrieve a semantic representation of a word or phrase
5. given positive feedback, update semantic representation
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Wubble Abilities
Meet&Greet Turntaking Games
Clean Up Wubble World
Face recognition x
See someone’s there x x
Recall previous chatsgames plans word meanings
x x x x
Generate smalltalk game interaction actions
x x x x
Detect affect x x
Recognize activities x x x
Physical control eyes arm arm, wheels path planning
Learn stuff about people
repetitive activities classes of things and where they go
some word meanings
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Other Abilities
Meet&Greet Turntaking Games
Clean Up Wubble World
Face recognition x
See someone’s there x x
Recall previous chatsgames plans word meanings
x x x x
Generate smalltalk game interaction actions
x x x x
Detect affect x x
Recognize activities x x x
Physical control eyes arm arm, wheels path planning
Curiosity / motivation x x
Control x x x x
Conceptual Glue x x x x
Learn stuff about people
repetitive activities classes of things and where they go
some word meanings
COMPUTER SCIENCEUASCIENCE
Paul Cohen
Assignment for Next Week
• Break into groups for the four tasks (we’ll organize around abilities later)
• Storyboard a few minutes in the life of the robot doing the task• Make a refined list of abilities • Search for off-the-shelf software for these abilities in ROS and
online. Example 1, Example 2, Example 3, Example 4,• Pay attention to glue: concepts and control• Write and present a 15-minute dissertation on all of the above