06/21/2004streettivo arjen p. de vries arjen@acm.org

06/21/2004 StreetTIVO

StreetTIVO

Arjen P. de Vriesarjen@acm.org

06/21/2004 StreetTIVO

N5: Semantic Access

• Main goal:– Improve search system engineering efficiency

• Approach:– Given

• Declarative specification of collection, background information, and context

• Declarative specification of retrieval model

– Generate a parameterised search system

06/21/2004 StreetTIVO

Collection +

Search Strategy

System

06/21/2004 StreetTIVO

Spiegle Prototype

Retrieval model specification (RAM) +

Collection schema (XML)+

Cirquid XML-IR (TIJAH) +

MonetDB/XQuery

06/21/2004 StreetTIVO

StreetTIVO

• Golden demo

“A demo not seen

anywhere else”

•Co-operation between projects:•N1: Multimedia Features Twente, UvA, Delft

•N3: Ambient Data Management CWI, Philips, Twente

•N5: Semantic Access CWI, Twente

06/21/2004 StreetTIVO

StreetTIVO

+ MultimediaN Features • Teletekst• ASR (automatic speech recognition) • Scene- and/or shot-segmentation • Detecting repeated objects • Tracking a library of famous people

06/21/2004 StreetTIVO

StreetTIVO

+ MultimediaN Features

+ Information retrieval• TIJAH XML-IR system (IR front-end

for MonetDB)• Integrated with MonetDB/XQuery (for

advanced DB structured queries)

06/21/2004 StreetTIVO

Retrieval System

• Reuse Waterland Demo– Demonstration system querying “De Digitale

Voorziening” at the Dutch broadcasters

MonetDB/XQuery

TIJAH 4 VIDEO

(IR engine)

NE

XI.c

gi

06/21/2004 StreetTIVO

06/21/2004 StreetTIVO

let $NEXI_QUERY:="//segment[about(.,metro)]"let $ranking := doc(fn:concat($NEXI_CGI,$NEXI_QUERY))let $vids := distinct-values($ranking//Result/@docID)

let $vranking :=element Vranking { for $v in $vids let $vshots := $ranking//Result[@docID=$v] let $vscore := for $f in $vshots/@score return $f cast as xs:double let $MAXvscore := max($vscore) let $MINvscore := min($vscore) order by $MAXvscore descending return element Vid { attribute id {$v}, attribute score {$MAXvscore}, for $s in $vshots[@score > $MINvscore] return $s }}

TIJAH’s NEXI query language ~ XPath+about

MonetDB/XQuery to present result fragments grouped by

video

06/21/2004 StreetTIVO

StreetTIVO

Idea: P2P Collaborative Multimedia Indexing

Work distribution Result exchange Fault tolerance

Challenge: feature detection takes long time

06/21/2004 StreetTIVO

StreetTIVO

CWI Seed farm

10 workstations

06/21/2004 StreetTIVO

StreetTIVO

Work distribution Result exchange Fault tolerance

AmbientDB

StreetTIVO aims for the world-wide

MythTV community (starting in NL).

Scalability and robustness!

CWI Seed farm

10 workstations

06/21/2004 StreetTIVO

StreetTIVO Architecture

StreetTIVOmodule

MultimediaFeature Detection

AmbientDBP2P

database

TIJAH 4 VIDEO

(IR engine)

XQuery/embedded

SO

AP

/X

Qu

ery

Extending the search functionality corresponds to installing a new

detector and updating the TIJAH 4 VIDEO XQuery queries

06/21/2004 StreetTIVO

Key Persons

Thijs Westerveld (CWI) Multimedia search

Matthijs Mourits (CWI) mythTV boxes

Jennie Zhang (CWI) P2P data management

Djoerd Hiemstra (CTIT) TIJAH

Roeland Ordelman (CTIT) ASR (Dutch)