from semantically annotated media to video documentaries stefano bocconi cwi amsterdam, the...
TRANSCRIPT
From Semantically Annotated Media To Video DocumentariesStefano Bocconi
CWI Amsterdam, The Netherlands
Talk Outline
• Motivation• Example• Scenarios• Technical details– Annotations – Editing Process
• Conclusions
Talk Outline
Philips, 31-05-2006
Case Study
Subject: the opinion of American people after 9-11
Shooting: from 27-10-2001 to 01-11-2001 in Stamford (CT), New York (NY), Boston (MA) and Cleveland (OH)
Editing: approximately 3 months
http://www.interviewwithamerica.com/
Philips, 31-05-2006
The problems
Only one final version: what do we show?
Who is pro and who is against? What are the numbers?
Biased versus objective trade-off: a choice must be made
Philips, 31-05-2006
Authoring Documentaries
Traditional video editing: there is only one final version, what is shown is the choice of the author/editor
Proposed video authoring: ● Annotate the video
material semantics● Show automatically
what the user asks to see, using presentation forms a film editor would use
Philips, 31-05-2006
Example
“I am never a fan of military action, in the big picture I don’t think it is ever a good thing, but I think there are circumstances in which I certainly can’t think of a more effective way to counter this sort of thing
Claim
Concession
Claim contradict
supportClaim
I am not a fanof military actions
War has never solved anything
Two billions dollar bombs on tents
I cannot think of a
more effective solutionweaken
Analysis of the Example
Scenarios
Vox Populi: Augmenting one interview with man-on-the-street opinion
Overview of the content of video footage:
● Example: trailers ● Browse the content by
opinion
Voices of Iraq
Philips, 31-05-2006
• What question was asked• Who answered (social categories)• Filmic (e.g. location/time/framing/gaze)
Possible query: show me all the answers to question “what do you think of the war in Afghanistan” given by black men
Descriptive Annotations
• Statements formally annotated:– <subject> <modifier> <predicate>– E.g. “war best solution”
• A thesaurus containing: – Terms (155)– Relations between terms: similar (72), opposite
(108), generalization (10), specialization (10)– E.g. war opposite diplomacy
• Annotations: 1 hour annotated, 15 interviews, 60 interview segments, 120 statements
Rhetorical Annotations
ClaimData
Qualifier
Warrant
Backing
Condition
Concession
57 Claims, 16 Data, 4 Concessions, 3 Warrants, 1 Condition
Toulmin Model
• Using the thesaurus, generate related statements and query the repository– E.g. from “war best solution” “diplomacy best
solution”, “war not solution”
• Create a graph of related statements– Nodes are the statements (video segments), edges
are either support or contradict
The Generation Process
• Partially tunable: examining the Segment graph gives feedback on the quality of the annotations and the thesaurus
• The annotator can revise the annotations in a guided manner
S1
S2
S3 S5
S4
S7
S6
S8
S9
S10
= support
= contradict
The Semantic Graph
• Video documentaries are not neutral account of reality: the selection and editing of the footage expresses a point of view
• Editing strategy: Balanced, Pro opinion X and Against opinion X
• Vox Populi uses:– Logos (the statements)– Ethos (based on user profile)– Film editing (framing, transitions, counterpoint editing)
Controlling Bias
• Vox Populi automatically presents video interviews augmented with supporting and/or contradicting material
• The user can determine the subject and the bias of the presentation
• Extensive annotations are needed
Conclusions
This presentation and a Demo available at:http://www.cwi.nl/~media/demo/IWA
Philips, 31-05-2006
Discussion
Philips, 31-05-2006
Issues
Advantages
Reuse the reasoning of the humans, no need for a very complex system like Cyc
Disadvantages
Extensive annotation is needed
Need for a knowledge base
Validation
User studies?
Mimicking the reality
• This presentation and a Demo available at:http://www.cwi.nl/~media/demo/IWA/
• This research was funded by the Dutch national ToKeN2000 I2RP and CHIME projects.
Philips, 31-05-2006
Using the Pathos
How and what emotions can we use to persuade?
Cognitive structure of emotions (Ortony)
The expressions for happiness, sadness, disgust, surprise, anger and fear are culture-, gender- and age independent (Ekman & Friesen)
Open Documentary
PRO
AGAINST
The annotations
• Descriptive– Question asked– Interviewee (social)– Filmic (e.g. location/time/framing/gaze)
• Rhetorical– Rhetorical Statement (mostly verbal, but visual also
possible)– Argumentation model: Toulmin model
Annotations
Philips, 31-05-2006
Rhetorical Tools
According to Aristotle, rhetoric is "the ability, in each particular case, to see the available means of persuasion”
Three main forms of rhetoric: ● Ethos: appeal based on the character of
the speaker.● Logos: appeal based on logic or reason.● Pathos: appeal based on emotion.
Open Documentary