.
Learning = Social (software) Robotics
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 2/25...
2/25
.
Social Robotics in Knowledge
Rebot
(careless) Input
Human Human
{structure}
(pinpoint) Select
Browse (or use otherwise)
Some Knowledge
(folksonomies, knowledge bases, databases, indexes, ontologies, etc.)
(metromaps )
07 myself+0 "On Context Management Using Metro Maps" SOCA, Matsue, Japan (2014)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 3/25...
3/25
.
Metromap: The Basic Concept
07 myself+0 "On Context Management Using Metro Maps" SOCA, Matsue, Japan (2014)
14 K.Nesbitt+0 "Getting to more abstract places using the metro map metaphor" 8th IV (2004)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 4/25...
4/25
.
A Practical Setting
Accident Something happened at Site A Causes Part A, Part B, Part C, … Human Factors… All Parts Part Z, Part Y, …, Human Manuals, … Rating
Blackswan scenario management platform
Storage, Database
Human judgment
Auto judgement
Report on site
07bmyself+0 "Black Swan Disaster Scenarios" PRMU研 (2014)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 5/25...
5/25
.
Definitions, Objectives, Terminology.Different Viewpoint..
.
classifier is not for finding hidden relations, but for clear separationbetween known and new.Learning Classifier...... a classifier that improves its inference over time based on human feedback
.Metromaps..
.... are used as the graphical interface between humans and robots
• MDC: Multi-Dimensional Classification• MC: Metromap Classifier
• folksonomy: BigData with very frivolous management of metadata
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 6/25...
6/25
.
Existing MDC Methods
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 7/25...
7/25
.
MDC Basics: Binary Relevance (BR)
• binary: YES or NO for each Y 11
• problem: no relation between classes Y -- this is where metromaps can behelpful
Training Tuples x1 x2 Y1 Y2 Y3
1 0.7 0.4 1 1 0 2 0.6 0.2 1 1 0 3 0.1 0.9 0 0 1 4 0.3 0.1 0 0 0
h1: X → Y1 h2: X → Y2 h3: X → Y3
11 J.Read+3 "Classifier chains for multi-label classification" Machine Learning, SpringerS (2011)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 8/25...
8/25
.
MDC Basics: PairWise Sets (PW)
• relations can be found by creating new classes for all unique pairs in Y 11
• problem: many classes = fuzzy results = low reliability
Training Tuples x1 x2 Y1 Y2 Y3
1 0.7 0.4 1 1 0 2 0.6 0.2 1 1 0 3 0.1 0.9 0 0 1 4 0.3 0.1 0 0 0
h1: X → Z1 h2: X → Z2
Z1 Z2 1 0 0 1 0 0 0 0
11 J.Read+3 "Classifier chains for multi-label classification" Machine Learning, SpringerS (2011)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 9/25...
9/25
.
MDC Basics: Label Combination (LC)
• basically, the extreme case of PW 11
• the same problem only worse -- there are too many classes!
Training Tuples x1 x2 Y1 Y2 Y3
1 0.7 0.4 1 1 0 2 0.6 0.2 1 1 0 3 0.1 0.9 0 0 1 4 0.3 0.1 0 0 0
h: X → Z
Z 1 0 0 0
11 J.Read+3 "Classifier chains for multi-label classification" Machine Learning, SpringerS (2011)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 10/25...
10/25
.
MDC Basics: Classifier Chains (CC)• classes are used in sequence 11
• merit: small number of classes -- only the necessary ones are used
• demerit: what is the correct order?
Training Tuples x1 x2 Y1 Y2 Y3
1 0.7 0.4 1 1 0 2 0.6 0.2 1 1 0 3 0.1 0.9 0 0 1
0.3 0.1 0 0 0
h1: X → Y1 h2: Y1 → Y2 h3: Y2 → Y3
h2 h1 h3
4
11 J.Read+3 "Classifier chains for multi-label classification" Machine Learning, SpringerS (2011)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 11/25...
11/25
.
MDC Basics: Graphical Methodology
• the graphicalmethodologybehind MDC 03
• all about jointprobability and howit is calculated usinggraph theory
03
D.Koller+1 "Probabilistic Graphical
Models: Principles and Techniques"
MIT Press (2009)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 12/25...
12/25
.
Metromap Classifier (MC)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 13/25...
13/25
.
Metromap Classifier (before)• problem: human load is too high!, ex: disaster scenarios 07b
Human judgment
Auto judgement
Folksonomy
07bmyself+0 "Black Swan Disaster Scenarios" PRMU研 (2014)
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 14/25...
14/25
.
Metromap Classifier (after)
Human judgment
Auto judgement
Folksonomy
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 15/25...
15/25
.
Metromap Classifier : Features
• human role1. build the metromap = relations between classes2. when robot fails, do the work manually3. do the human part (by design) of the work
• robot role1. classify incoming data into YES or NO for question: should human seethis?
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 16/25...
16/25
.
Experiment
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 17/25...
17/25
.
Experiment : Setup
• IEICE/ken is the source of data -- over 3000 presentations over 2-3 lastyears
• various combinations of title, keywords, abstract• usecase: which presentations should I look at closely?
◦ ... meaning the metromap reflects my personal research interests• Dumb Classifier (DC): one-dimensional yes or no
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 18/25...
18/25
.
Metromap Design: The Human
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 19/25...
19/25
.
Metromap Classifier: Logic• logic followed by the MC Robot
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 20/25...
20/25
.
Results: Title only
0 20 40 60 80 100 120Time sequence
0102030405060708090
Goo
d c
ount
Dumb ClassifierMetromap Classifier(smart) Hits on a timelinetitle
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 21/25...
21/25
.
Results: Title + Keywords
0 20 40 60 80 100Time sequence
0
10
20
30
40
50
60
70
80G
ood
cou
ntDumb ClassifierMetromap Classifier(smart) Hits on a timeline
title:keywords
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 22/25...
22/25
.
Results: Title + Keywords + Abstract
0 20 40 60 80 100 120Time sequence
0102030405060708090
Goo
d c
ount
Dumb ClassifierMetromap Classifier(smart) Hits on a timelinetitle:keywords:abstract
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 23/25...
23/25
.
Wrapup: Not Good Enough
• not perfect: about 30% of wrong decisions◦ FP: robot makes human look at bad stuff (false positive)◦ FN: robot passes on good stuff (false negative)
• future improvements: need a solid logic which avoids FP and FN cases
• note: current naive and MDCs are at most 40-60% reliable -- no help here!
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 24/25...
24/25
.
That’s all, thank you ...
M.Zhanikeev -- [email protected] -- Metromaps for Minimizing Human Interaction with Bayesian Classifiers -- http://bit.do/marat150129 -- 25/25...
25/25