CWI AmsterdamThe Netherlands

Supporting the Generation of Argument Structurewithin Video Sequences

Talk Outline

The motivation and vision of the work What is needed

Annotations Editing Process

Editor Support Conclusions

Existing Documentaries

Traditional video authoring: the footage is selected and edited for the final cut

there is only one final version, what is shown is the choice of the author / editor

Material can be very rich and controversial (e.g. Voices of Iraq)

New Paradigm Proposed video authoring:

Annotate the video material semantics Edit it automatically, selecting what the

user asks to see Use the Web as an interactive

distribution mean More than a sequence of matching

video fragments): Argumentation/rhetoric Narrative

Video material

Interview with America: video footage with interviews and background material about the opinion of American people after 9-11www.interviewwithamerica.com

Annotations: 1 hour annotated, 15 interviews, 60 interview segments, 120 statements

What do you think of the war in Afghanistan?

Example Explained

Claim

Concession

Claim

contradict

supportClaim

I am not a fanof military actions

War has never solved anything

Two billions dollar bombs on tents

I cannot think of amore effective solution

weaken

The annotations

Rhetorical Argumentation model: Toulmin model Rhetorical Statement (mostly verbal, but

visual also possible) Descriptive

Question asked Interviewee (social) Filmic (e.g. location/time/framing/gaze)

Encode statements Statement formally annotated:

<subject> <modifier> <predicate> E.g. “war best solution”

A thesaurus containing: Terms for each part (155 in total) Relations between terms: similar (72), opposite

(108), generalization (10), specialization (10) E.g. war opposite diplomacy

Relations in the thesaurus determine link type between statements

Automatic Linking Process STEP1: Using the thesaurus, generate

related statements, by replacing iteratively terms: E.g. from “war best solution” “diplomacy

best solution”, “war not solution” STEP2: Query the repository to see

whether the statement is present

Connect statements

Create a graph of related statements Nodes are the statements (video

segments), edges are either support or contradict

S1

S2

S3 S5

S4

S7

S6

S8

S9

S0

= support

= contradict

Author/Annotator support Capability of generating different

arguments depends on the quality of the Semantic Graph

Statements (and corresponding video segments) not connect are lost for generation: Our case: out of 118 statements 54 were not

connected Measure the performance of the automatic

linking process

Indices for statements

Measure how many statements are generated from a given one: depends on the quantity of the relations

in the thesaurus Measure how many generates

statements are present in the repository Depends on correctness of the relations

in the thesaurus

Index for relations

Measure how a particular relation in the thesaurus is performing: If a generated statement is present in

the repository, the relations used to generate it get one point on a hit score, otherwise one point on the miss score

The ratio hit/miss gives an idea of the semantic accuracy of the relation with respect to the repository

Current/future Work Automatic Relation suggestion:

Start with a fully connected thesaurus, keep only best relations

Suggest best relation to add to existing ones

Linking Process tuning Currently 3 iterations for performance,

but the process runs of-line: more iterations possible

Different repositories (VJ project)

Conclusions

New documentary production mechanism, multiple versions

Different authoring, author does not have full control anymore

Authoring support needed

Questions?

Thanks for your attention

Pointers & Acknowledgments

This presentation and Demo available at:http://www.cwi.nl/~media/demo/IWA/

This research was funded by the Dutch national ToKeN2000 I2RP and CHIME projects.

Author/Annotator support

Provide means to measure the performance of the creation of the Semantic Graph

Reengineer the Semantic Graph generation: Changing annotations Changing relations in the Thesaurus

What do you think of the war in Afghanistan?

I am not a fan of military actions

War has neversolvedanything

I cannot think of a more effective solution

Two billionsdollar bombson tents

Toulmin model

ClaimData

Qualifier

Warrant

Backing

Condition

Concession

57 Claims, 16 Data, 4 Concessions, 3 Warrants, 1 Condition