xue xiangyang - fudan university

Research of Intelligent Media Computing

and its Application

Dr. Xiangyang XUE

Professor & Vice Dean, School of

Computer Science

FDU - Xiangyang XUE22009-5-21

Outline

Introduction of School of Computer Science

Research Topics in our Laboratory

RT1 Natural Language Processing

RT2 Multimodal Dialog

RT3 Image Classification

RT4 Cross-Media Search

Assistant Living Technologies in Future Home

APP1 IPTV ~ video search engine

APP2 Robot ~ multimodal dialog


School – Staff Members & Students

Students（full time）

Staff Members

Totally 160 staff members

19 full professors

39 associate professors

Undergraduates: 1100

Master students: 400

PhD Students: 170


School – Research Areas

Software Engineering

Compiler

CSCW

Theory: Complexity, Cryptography & Algorithms

Multimedia

Computer vision

Machine Learning

NLP & IR

Database

OS: virtualization


School – Research Labs

Laboratory of Intelligent Media Computing

Laboratory of Database Technology

Laboratory of System Software and Software

Engineering

Laboratory of Information Security

Laboratory of Embedded System

Shanghai Key Laboratory for

Intelligent Information Processing


Dr Xuanjing

HUANG

People in our Laboratory

Dr Wei Zhang

Text

Image

Video

Audio

Dr Hong LU

Dr Yuejie ZHANG

Dr Xipeng QIU

Dr Yuefei GUO Dr Yaqian ZHOU Dr Cheng JIN

Dr Mingmin

CHI

PhD Students：12

Master Students：40

Dr Wei LI

Dr Xiangyang XUE

Machine

Learning

Lab of

Intelligent

Media Computing


Projects & Publications in our Laboratory

Projects

Natural Science Foundation of China

Ministry of Science and Technology

Science and Technology Commission of Shanghai

Municipality

Industries

Publications

Journals: IEEE TMM, TKDE, TCAS-VT, TIP; Pattern

Recognition, Neural Networks, Information Processing and

Management etc

Conferences: ACM MM, ICCV, ICML, IJCAI ,ACL, etc


RT1 Natural Language Processing

Current Works

Question answering

Open domain

Trec 2008, Top 2

Sentiment analysis

Product review

Trec Blog, Top 5

Text

summarization

Information Extraction

DiscourseSyntaxWord

Chinese word segment

POS Tag

Chunking

Named Entity Recognition

Coreference

Discourse Segment

Parsing

Fundamental researches in NLP


RT2 Multimodal Dialogue System

Knowledge

Sources

Knowledge

Sources

Knowledge

Sources

Knowledge

Manager

Device

ManagerDevice API

Device

s

Device

sDevices

Dialogue

ManagerDialogue

Move Tree

Activity

Tree

Dialogue

Move

EngineMultimodal

Fusion

Response

Generator

NLU Gesture

ASR

NLG

TTS

Animation

Application：Robot


RT2 Multimodal Dialogue System

Knowledge

Sources

Knowledge

Sources

Knowledge

Sources

Knowledge

Manager

Device

ManagerDevice API

Device

s

Device

sDevices

Dialogue

ManagerDialogue

Move Tree

Activity

Tree

Dialogue

Move

EngineMultimodal

Fusion

Response

Generator

NLU Gesture

ASR

NLG

TTS

Animation

Application：Robot

Dialogue Move Engine

Domain adaptation

Semi-Supervised

Methods

NLU

Spoken Language

Unknown words

Ontology


RT3 Image Classification

Wedding

Party

Coworker

Family

Snake

Lion

Activity

People

Animal

http://images.google.cn/imgres?imgurl=http://www.thebestweddingreceptionever.com/images/WeddingParty.jpg&imgrefurl=http://www.thebestweddingreceptionever.com/&usg=__eTaozuHMjLdxiZxBheZ9OK6uCno=&h=309&w=580&sz=110&hl=zh-CN&start=11&um=1&tbnid=vKsmu8z2TogZbM:&tbnh=71&tbnw=134&prev=/images%3Fq%3Dwedding%26hl%3Dzh-CN%26sa%3DN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.wallcoo.com/photograph/sz_185_sweet_weddings_01/images/%255Bwallcoo_com%255D_0HR086_350A_wedding_dress.jpg&imgrefurl=http://www.wallcoo.com/photograph/sz_185_sweet_weddings_01/html/wallpaper16.html&usg=__6TaqNOJtJNhnupY-VegLNleVJTA=&h=525&w=700&sz=69&hl=zh-CN&start=49&um=1&tbnid=vVjWniirHOxegM:&tbnh=105&tbnw=140&prev=/images%3Fq%3Dwedding%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D36%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.engagementringquestions.com/images/jenna-bush-wedding-ceremony.jpg&imgrefurl=http://www.engagementringquestions.com/jenna-bush-wedding-ceremony-and-ring/&usg=__g0o0j7n3VuXzidS4gW3211aCFkY=&h=327&w=490&sz=68&hl=zh-CN&start=1&um=1&tbnid=LIO_188HsH3tLM:&tbnh=87&tbnw=130&prev=/images%3Fq%3Dwedding%2Bceremony%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://media-cdn.tripadvisor.com/media/photo-s/01/05/0a/ab/wedding-ceremony.jpg&imgrefurl=http://www.tripadvisor.com/Hotel_Review-g33203-d622837-Reviews-Shambhala_Ranch_A_Mendocino_County_Retreat_Center-Ukiah_California.html&usg=__22cYPY8WiHWpDok2ZxvKxcAs9xw=&h=412&w=550&sz=46&hl=zh-CN&start=13&um=1&tbnid=F165SeMl7ACgMM:&tbnh=100&tbnw=133&prev=/images%3Fq%3Dwedding%2Bceremony%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://blog.handbagsmaster.com/wp-content/uploads/2009/03/party1-buyvertureplicacom.jpg&imgrefurl=http://blog.sdgb.cn/lily1986/archive/2009/03/11/14671.aspx&usg=__gfM5fVvtgYUEEdV2xep8PQZiON4=&h=295&w=500&sz=52&hl=zh-CN&start=11&um=1&tbnid=wWVcqry85JvE6M:&tbnh=77&tbnw=130&prev=/images%3Fq%3Dparty%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://orlando.metromix.com/content_image/full/385407/560/370&imgrefurl=http://orlando.metromix.com/bars-and-clubs/photogallery/o-lens-slumber-party/385302/content&usg=__PVmihpilq9j6r1PQsJsr8Kl6ECo=&h=370&w=493&sz=36&hl=zh-CN&start=1&um=1&tbnid=uTSiTfASxw8tZM:&tbnh=98&tbnw=130&prev=/images%3Fq%3Dparty%2Bpub%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.jb193.com/attachments/2009/03/1_2009031018211011Ad22.jpg&imgrefurl=http://www.jb193.com/%3Fviewnews-2151-page-6.html&usg=__Dvvk_waD2BYGFCxDomOhKgRN8Fg=&h=400&w=600&sz=45&hl=zh-CN&start=101&um=1&tbnid=1ZsOAQWPtH9raM:&tbnh=90&tbnw=135&prev=/images%3Fq%3Dparty%2Bpub%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D90%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://farm4.static.flickr.com/3418/3388941447_7019a204e0.jpg&imgrefurl=http://threehairsysy.spaces.live.com/&usg=__Yibv8XHCYOnc3Ows7s7PWfXqBTk=&h=375&w=500&sz=97&hl=zh-CN&start=113&um=1&tbnid=XRDrjVd5fiaewM:&tbnh=98&tbnw=130&prev=/images%3Fq%3Dparty%2Bpub%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D108%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.wilshiregrand.com/images/photo_meeting.jpg&imgrefurl=http://www.wilshiregrand.com/meetings%26events/index.htm&usg=__2ZS9SWMcctT6-9Ndhu_6FPWHT0k=&h=311&w=466&sz=42&hl=zh-CN&start=1&um=1&tbnid=kmc9mRhPWU3v_M:&tbnh=85&tbnw=128&prev=/images%3Fq%3Dmeeting%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.itl.nist.gov/iad/photos/meeting-room.gif&imgrefurl=http://sbdcrn.blogspot.com/2006_05_01_archive.html&usg=__NyYkgxQiviwCqEDSuGVK5jEMwZk=&h=947&w=1440&sz=759&hl=zh-CN&start=5&um=1&tbnid=8ce-bYyWtXx3QM:&tbnh=99&tbnw=150&prev=/images%3Fq%3Dmeeting%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.now.cn/aboutus/images/meeting2.jpg&imgrefurl=http://www.now.cn/aboutus/accessing.php&usg=__aFPHpQSB0Gus5MrhpfO43Rg0hCw=&h=375&w=500&sz=109&hl=zh-CN&start=13&um=1&tbnid=C24rDoPUuvEYRM:&tbnh=98&tbnw=130&prev=/images%3Fq%3Dmeeting%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.oclc.org/asiapacific/zhcn/memberscouncil/images/200705/meeting.jpg&imgrefurl=http://www.oclc.org/asiapacific/zhcn/memberscouncil/meetings/2007/default.htm&usg=__IgiGAUkqeNPGAvpRXMBnhYarT8M=&h=399&w=600&sz=54&hl=zh-CN&start=40&um=1&tbnid=spyQbZIHaP_ZCM:&tbnh=90&tbnw=135&prev=/images%3Fq%3Dmeeting%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D36%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://chongeng.org/wordpress/wp-content/uploads/2008/06/family-at-mengkuang.JPG&imgrefurl=http://chongeng.org/wordpress/%3Fcat%3D31&usg=__8xFAfTzoqty0ezzs-9rm5wdH-7E=&h=1536&w=2048&sz=1130&hl=zh-CN&start=43&um=1&tbnid=VCP7puEjSL4HGM:&tbnh=113&tbnw=150&prev=/images%3Fq%3Dfamily%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D36%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://pos-psych.com/wp-content/uploads/2008/12/family01-jodie.jpg&imgrefurl=http://hk.pos-psych.com/news/guest-author/20081220328&usg=__gGXmJ_RcN5h1o0G7LqunEsgVrv4=&h=428&w=640&sz=35&hl=zh-CN&start=119&um=1&tbnid=hpbPmNwcNg-iHM:&tbnh=92&tbnw=137&prev=/images%3Fq%3Dfamily%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D108%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://k53.pbase.com/u41/adamslin/upload/27052014.family.jpg&imgrefurl=http://www.pbase.com/adamslin/image/27052014&usg=__6qIC9g8POvx7mLhIhU-KTitYswY=&h=474&w=725&sz=176&hl=zh-CN&start=185&um=1&tbnid=ZC-xF4X9Jax0aM:&tbnh=92&tbnw=140&prev=/images%3Fq%3Dfamily%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D180%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://i165.photobucket.com/albums/u60/amandamcfarland920/obama_family_essence_575.gif&imgrefurl=http://us.profile.myspace.cn/index.cfm%3Ffuseaction%3Duser.viewprofile%26friendid%3D179000201&usg=__T_AjEkzNim8UIfthNoPRs4wYILI=&h=385&w=575&sz=201&hl=zh-CN&start=221&um=1&tbnid=5F13scv90YmImM:&tbnh=90&tbnw=134&prev=/images%3Fq%3Dfamily%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D216%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.wallpaperbase.com/wallpapers/animals/snakes/snake_3.jpg&imgrefurl=http://tieba.baidu.com/f%3Fkz%3D440279980&usg=__bZuic-n6_7gy6LO1FMteDZczxxc=&h=1200&w=1600&sz=785&hl=zh-CN&start=2&um=1&tbnid=fkln0u_1X1GoPM:&tbnh=113&tbnw=150&prev=/images%3Fq%3Dsnake%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.floridasnakes.net/coral-snake.jpg&imgrefurl=http://www.floridasnakes.net/coralsnakes.htm&usg=__tinD_paL41mw8jnL-jnt8wotfmI=&h=336&w=448&sz=39&hl=zh-CN&start=3&um=1&tbnid=onbQ0aMnW3-L_M:&tbnh=95&tbnw=127&prev=/images%3Fq%3Dsnake%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.tapirback.com/tapirgal/gifts/friends/reptiles/cobra-snake-plastic-f451.jpg&imgrefurl=http://teachthemasses.wordpress.com/2009/05/04/snake-flu/&usg=__WEowTlqJSeTb56NAnDUy-IXQhTw=&h=324&w=400&sz=73&hl=zh-CN&start=5&um=1&tbnid=uUI49VOtbFK_SM:&tbnh=100&tbnw=124&prev=/images%3Fq%3Dsnake%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.proctormuseum.us/Reptiles/Modern%2520Reptiles/Modern%2520Snakes/TexasGarterSnake-P0785-72dpi-26Apr%2705.jpg&imgrefurl=http://www.proctormuseum.us/Reptiles/Modern%2520Reptiles/Modern%2520Snakes/ModernSnakes.htm&usg=__HnPaSoka20aol8O04DirRnhxiVE=&h=611&w=785&sz=242&hl=zh-CN&start=9&um=1&tbnid=6JGrDSiSIfdL-M:&tbnh=111&tbnw=143&prev=/images%3Fq%3Dsnake%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://bluepyramid.org/ia/lion.jpg&imgrefurl=http://toxicslugs.blogspot.com/2009/04/fun-facts-if-you-yelled-for-8-years-7.html&usg=__kxd83b1lZ0onMv4VgxhAEyUAebs=&h=341&w=434&sz=41&hl=zh-CN&start=2&um=1&tbnid=oXh020OHM96k-M:&tbnh=99&tbnw=126&prev=/images%3Fq%3Dlion%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://animals.nationalgeographic.com/staticfiles/NGS/Shared/StaticFiles/animals/images/primary/african-lion-closeup.jpg&imgrefurl=http://animals.nationalgeographic.com/animals/mammals/african-lion.html&usg=__J8bhfKq_nvVURoeNJo2f7lw7LZ0=&h=324&w=470&sz=30&hl=zh-CN&start=3&um=1&tbnid=y27wT1HmDvFlNM:&tbnh=89&tbnw=129&prev=/images%3Fq%3Dlion%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.imyspacegraphics.com/images/lion/lion22.jpg&imgrefurl=http://us.profile.myspace.cn/index.cfm%3Ffuseaction%3Duser.viewProfile%26friendid%3D118196294&usg=__rfwHs1DNUo9ccWzr_s4HmbE-C0U=&h=300&w=400&sz=27&hl=zh-CN&start=4&um=1&tbnid=dz9dF8c2_UcRWM:&tbnh=93&tbnw=124&prev=/images%3Fq%3Dlion%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://animals.nationalgeographic.com/staticfiles/NGS/Shared/StaticFiles/animals/images/primary/asian-lion.jpg&imgrefurl=http://animals.nationalgeographic.com/animals/printable/asian-lion.html&usg=__4Wv07WTJ7Bhov1I_ejYLijpwj0s=&h=324&w=470&sz=30&hl=zh-CN&start=12&um=1&tbnid=7v_6TCkl2Hjk7M:&tbnh=89&tbnw=129&prev=/images%3Fq%3Dlion%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.thebestweddingreceptionever.com/images/WeddingParty.jpg&imgrefurl=http://www.thebestweddingreceptionever.com/&usg=__eTaozuHMjLdxiZxBheZ9OK6uCno=&h=309&w=580&sz=110&hl=zh-CN&start=11&um=1&tbnid=vKsmu8z2TogZbM:&tbnh=71&tbnw=134&prev=/images%3Fq%3Dwedding%26hl%3Dzh-CN%26sa%3DN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.wallcoo.com/photograph/sz_185_sweet_weddings_01/images/%255Bwallcoo_com%255D_0HR086_350A_wedding_dress.jpg&imgrefurl=http://www.wallcoo.com/photograph/sz_185_sweet_weddings_01/html/wallpaper16.html&usg=__6TaqNOJtJNhnupY-VegLNleVJTA=&h=525&w=700&sz=69&hl=zh-CN&start=49&um=1&tbnid=vVjWniirHOxegM:&tbnh=105&tbnw=140&prev=/images%3Fq%3Dwedding%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D36%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.engagementringquestions.com/images/jenna-bush-wedding-ceremony.jpg&imgrefurl=http://www.engagementringquestions.com/jenna-bush-wedding-ceremony-and-ring/&usg=__g0o0j7n3VuXzidS4gW3211aCFkY=&h=327&w=490&sz=68&hl=zh-CN&start=1&um=1&tbnid=LIO_188HsH3tLM:&tbnh=87&tbnw=130&prev=/images%3Fq%3Dwedding%2Bceremony%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://media-cdn.tripadvisor.com/media/photo-s/01/05/0a/ab/wedding-ceremony.jpg&imgrefurl=http://www.tripadvisor.com/Hotel_Review-g33203-d622837-Reviews-Shambhala_Ranch_A_Mendocino_County_Retreat_Center-Ukiah_California.html&usg=__22cYPY8WiHWpDok2ZxvKxcAs9xw=&h=412&w=550&sz=46&hl=zh-CN&start=13&um=1&tbnid=F165SeMl7ACgMM:&tbnh=100&tbnw=133&prev=/images%3Fq%3Dwedding%2Bceremony%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://blog.handbagsmaster.com/wp-content/uploads/2009/03/party1-buyvertureplicacom.jpg&imgrefurl=http://blog.sdgb.cn/lily1986/archive/2009/03/11/14671.aspx&usg=__gfM5fVvtgYUEEdV2xep8PQZiON4=&h=295&w=500&sz=52&hl=zh-CN&start=11&um=1&tbnid=wWVcqry85JvE6M:&tbnh=77&tbnw=130&prev=/images%3Fq%3Dparty%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://orlando.metromix.com/content_image/full/385407/560/370&imgrefurl=http://orlando.metromix.com/bars-and-clubs/photogallery/o-lens-slumber-party/385302/content&usg=__PVmihpilq9j6r1PQsJsr8Kl6ECo=&h=370&w=493&sz=36&hl=zh-CN&start=1&um=1&tbnid=uTSiTfASxw8tZM:&tbnh=98&tbnw=130&prev=/images%3Fq%3Dparty%2Bpub%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.jb193.com/attachments/2009/03/1_2009031018211011Ad22.jpg&imgrefurl=http://www.jb193.com/%3Fviewnews-2151-page-6.html&usg=__Dvvk_waD2BYGFCxDomOhKgRN8Fg=&h=400&w=600&sz=45&hl=zh-CN&start=101&um=1&tbnid=1ZsOAQWPtH9raM:&tbnh=90&tbnw=135&prev=/images%3Fq%3Dparty%2Bpub%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D90%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://farm4.static.flickr.com/3418/3388941447_7019a204e0.jpg&imgrefurl=http://threehairsysy.spaces.live.com/&usg=__Yibv8XHCYOnc3Ows7s7PWfXqBTk=&h=375&w=500&sz=97&hl=zh-CN&start=113&um=1&tbnid=XRDrjVd5fiaewM:&tbnh=98&tbnw=130&prev=/images%3Fq%3Dparty%2Bpub%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D108%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.wilshiregrand.com/images/photo_meeting.jpg&imgrefurl=http://www.wilshiregrand.com/meetings%26events/index.htm&usg=__2ZS9SWMcctT6-9Ndhu_6FPWHT0k=&h=311&w=466&sz=42&hl=zh-CN&start=1&um=1&tbnid=kmc9mRhPWU3v_M:&tbnh=85&tbnw=128&prev=/images%3Fq%3Dmeeting%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.itl.nist.gov/iad/photos/meeting-room.gif&imgrefurl=http://sbdcrn.blogspot.com/2006_05_01_archive.html&usg=__NyYkgxQiviwCqEDSuGVK5jEMwZk=&h=947&w=1440&sz=759&hl=zh-CN&start=5&um=1&tbnid=8ce-bYyWtXx3QM:&tbnh=99&tbnw=150&prev=/images%3Fq%3Dmeeting%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.now.cn/aboutus/images/meeting2.jpg&imgrefurl=http://www.now.cn/aboutus/accessing.php&usg=__aFPHpQSB0Gus5MrhpfO43Rg0hCw=&h=375&w=500&sz=109&hl=zh-CN&start=13&um=1&tbnid=C24rDoPUuvEYRM:&tbnh=98&tbnw=130&prev=/images%3Fq%3Dmeeting%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.oclc.org/asiapacific/zhcn/memberscouncil/images/200705/meeting.jpg&imgrefurl=http://www.oclc.org/asiapacific/zhcn/memberscouncil/meetings/2007/default.htm&usg=__IgiGAUkqeNPGAvpRXMBnhYarT8M=&h=399&w=600&sz=54&hl=zh-CN&start=40&um=1&tbnid=spyQbZIHaP_ZCM:&tbnh=90&tbnw=135&prev=/images%3Fq%3Dmeeting%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D36%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://chongeng.org/wordpress/wp-content/uploads/2008/06/family-at-mengkuang.JPG&imgrefurl=http://chongeng.org/wordpress/%3Fcat%3D31&usg=__8xFAfTzoqty0ezzs-9rm5wdH-7E=&h=1536&w=2048&sz=1130&hl=zh-CN&start=43&um=1&tbnid=VCP7puEjSL4HGM:&tbnh=113&tbnw=150&prev=/images%3Fq%3Dfamily%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D36%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://pos-psych.com/wp-content/uploads/2008/12/family01-jodie.jpg&imgrefurl=http://hk.pos-psych.com/news/guest-author/20081220328&usg=__gGXmJ_RcN5h1o0G7LqunEsgVrv4=&h=428&w=640&sz=35&hl=zh-CN&start=119&um=1&tbnid=hpbPmNwcNg-iHM:&tbnh=92&tbnw=137&prev=/images%3Fq%3Dfamily%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D108%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://k53.pbase.com/u41/adamslin/upload/27052014.family.jpg&imgrefurl=http://www.pbase.com/adamslin/image/27052014&usg=__6qIC9g8POvx7mLhIhU-KTitYswY=&h=474&w=725&sz=176&hl=zh-CN&start=185&um=1&tbnid=ZC-xF4X9Jax0aM:&tbnh=92&tbnw=140&prev=/images%3Fq%3Dfamily%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D180%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://i165.photobucket.com/albums/u60/amandamcfarland920/obama_family_essence_575.gif&imgrefurl=http://us.profile.myspace.cn/index.cfm%3Ffuseaction%3Duser.viewprofile%26friendid%3D179000201&usg=__T_AjEkzNim8UIfthNoPRs4wYILI=&h=385&w=575&sz=201&hl=zh-CN&start=221&um=1&tbnid=5F13scv90YmImM:&tbnh=90&tbnw=134&prev=/images%3Fq%3Dfamily%26ndsp%3D18%26hl%3Dzh-CN%26sa%3DN%26start%3D216%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.wallpaperbase.com/wallpapers/animals/snakes/snake_3.jpg&imgrefurl=http://tieba.baidu.com/f%3Fkz%3D440279980&usg=__bZuic-n6_7gy6LO1FMteDZczxxc=&h=1200&w=1600&sz=785&hl=zh-CN&start=2&um=1&tbnid=fkln0u_1X1GoPM:&tbnh=113&tbnw=150&prev=/images%3Fq%3Dsnake%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.floridasnakes.net/coral-snake.jpg&imgrefurl=http://www.floridasnakes.net/coralsnakes.htm&usg=__tinD_paL41mw8jnL-jnt8wotfmI=&h=336&w=448&sz=39&hl=zh-CN&start=3&um=1&tbnid=onbQ0aMnW3-L_M:&tbnh=95&tbnw=127&prev=/images%3Fq%3Dsnake%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.tapirback.com/tapirgal/gifts/friends/reptiles/cobra-snake-plastic-f451.jpg&imgrefurl=http://teachthemasses.wordpress.com/2009/05/04/snake-flu/&usg=__WEowTlqJSeTb56NAnDUy-IXQhTw=&h=324&w=400&sz=73&hl=zh-CN&start=5&um=1&tbnid=uUI49VOtbFK_SM:&tbnh=100&tbnw=124&prev=/images%3Fq%3Dsnake%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.proctormuseum.us/Reptiles/Modern%2520Reptiles/Modern%2520Snakes/TexasGarterSnake-P0785-72dpi-26Apr%2705.jpg&imgrefurl=http://www.proctormuseum.us/Reptiles/Modern%2520Reptiles/Modern%2520Snakes/ModernSnakes.htm&usg=__HnPaSoka20aol8O04DirRnhxiVE=&h=611&w=785&sz=242&hl=zh-CN&start=9&um=1&tbnid=6JGrDSiSIfdL-M:&tbnh=111&tbnw=143&prev=/images%3Fq%3Dsnake%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://bluepyramid.org/ia/lion.jpg&imgrefurl=http://toxicslugs.blogspot.com/2009/04/fun-facts-if-you-yelled-for-8-years-7.html&usg=__kxd83b1lZ0onMv4VgxhAEyUAebs=&h=341&w=434&sz=41&hl=zh-CN&start=2&um=1&tbnid=oXh020OHM96k-M:&tbnh=99&tbnw=126&prev=/images%3Fq%3Dlion%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://animals.nationalgeographic.com/staticfiles/NGS/Shared/StaticFiles/animals/images/primary/african-lion-closeup.jpg&imgrefurl=http://animals.nationalgeographic.com/animals/mammals/african-lion.html&usg=__J8bhfKq_nvVURoeNJo2f7lw7LZ0=&h=324&w=470&sz=30&hl=zh-CN&start=3&um=1&tbnid=y27wT1HmDvFlNM:&tbnh=89&tbnw=129&prev=/images%3Fq%3Dlion%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://www.imyspacegraphics.com/images/lion/lion22.jpg&imgrefurl=http://us.profile.myspace.cn/index.cfm%3Ffuseaction%3Duser.viewProfile%26friendid%3D118196294&usg=__rfwHs1DNUo9ccWzr_s4HmbE-C0U=&h=300&w=400&sz=27&hl=zh-CN&start=4&um=1&tbnid=dz9dF8c2_UcRWM:&tbnh=93&tbnw=124&prev=/images%3Fq%3Dlion%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1

http://images.google.cn/imgres?imgurl=http://animals.nationalgeographic.com/staticfiles/NGS/Shared/StaticFiles/animals/images/primary/asian-lion.jpg&imgrefurl=http://animals.nationalgeographic.com/animals/printable/asian-lion.html&usg=__4Wv07WTJ7Bhov1I_ejYLijpwj0s=&h=324&w=470&sz=30&hl=zh-CN&start=12&um=1&tbnid=7v_6TCkl2Hjk7M:&tbnh=89&tbnw=129&prev=/images%3Fq%3Dlion%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1


Role: James Bond

RT4 Cross-Media Search for iTV

• Video Shot Boundary Detection

• Face Detection and Recognition

• Video OCR / ASR

• Correlation Analysis

镜头1 镜头2

镜头1 镜头2

·

镜头1 镜头2

镜头1 镜头2

·

Movie: Royal Casino

Application

http://images.google.cn/imgres?imgurl=http://www.coolnfreepix.com/James-Bond/james%2520bond1.jpg&imgrefurl=http://www.coolnfreepix.com/James-Bond.php&usg=__Dst4LrI_KBFjIcmFswgvUBLjVbc=&h=768&w=1024&sz=95&hl=zh-CN&start=3&um=1&tbnid=p8WxOtHyjW5LYM:&tbnh=113&tbnw=150&prev=/images%3Fq%3Djames%2Bbond%26hl%3Dzh-CN%26um%3D1%26newwindow%3D1


FUWA Robot

Shanghai Industry

Fair 2008

Intel Developer Forum

(IDF) 2008

Surveillance in Home

Interactive TV

Education and Gaming for Children

Multimodal HCI/Dialog


Assistant Living Technologies in Future Home

Thank You！

Q/A？

xue xiangyang - fudan university

Technology

dr wei li6fdu xiangyang

school of computer science

technology science

school research areasos

multimodal dialog2009

introduction of school

video search engine

nlp information extraction