accessing news video libraries through dynamic information extraction, summarization, and...

Accessing News Video Libraries through Dynamic Information Extraction, Summarization, and Visualization

Mike ChristelCarnegie Mellon University, USA

June 28, 2001

Digital Video LibraryDigital Video Library

© Copyright 2001 Carnegie Mellon University

Informedia Project Information

Sponsors • NSF Digital Libraries Initiative

• NSF National Science, Mathematics, Engineering, and Technology Education Digital Library Program

Images in later slides ©1999-2001, Cable News Network. More information on project sponsors is at the Carnegie Mellon University Informedia Project web site:

http://www.informedia.cs.cmu.edu/


Informedia Video Surrogates

Purpose of Video Surrogate:

Represent contents of video clip through text, imagery, audio and/or video in a format that can be interpreted in significantly less time than the associated video clip.

Example Surrogates:• Thumbnail image

• Short text title

• Set of thumbnail images (aka filmstrip or storyboard)

• Abstract

• Transcript

• Storyboard plus text

• Video skim


Query-based Thumbnail Surrogates

Informedia Storyboard Interface


Informedia Storyboard Plus Text Surrogate


Informedia Video Playback Interface



The Need for Visualization Strategies

• As digital video library grew, so did result sets• Linear traversal of result pages sorted by

relevance/date/size/etc. not sufficient: needed overview capability

• Prior work offered many solutions:• Visualization By Example (VIBE) for matching entity

relationships

• Scatter plots for low dimensionality relationships, e.g., timelines

• Dynamic query sliders for direct manipulation of plots

• Colored maps for geographic relationships

(demonstration of Informedia system follows…)

Informedia Timeline Scatter Plot Interface


Informedia Map Interface


What’s Next…


Visualization Schemes with Image Overlays


Questions for Further Investigation

• How do you reduce 2,000,000 images to manageable set(s)?

• What are good strategies for dealing with uncertain data, e.g., errorful speech and image processing?

• How can the temporal dimension for motion video be better represented/summarized?

• How should evaluation for information visualizations be conducted?

• How should fast, scaleable interfaces supporting direct manipulation be delivered over the web?

http://www.informedia.cs.cmu.edu

accessing news video libraries through dynamic information extraction, summarization, and...

Documents