Background Information & Suggestions for Joint Research Topics IVY Lab & MMLab

Wesley De Neve

ICU, Daejeon, South KoreaSeptember 2007

I am from...

• Belgium– located in Europe– 30,528 square km– 10.5 million people– weather has four seasons– three official languages

• Dutch– my native language

• French• German

http://en.wikipedia.org/wiki/Belgium

Belgium

I was born...

• on May the 31st, 1980• in Ghent

– 233,120 inhabitants

Ghent

http://en.wikipedia.org/wiki/Ghent

I am living...

• in Aalter– a city close to Ghent– 18,841 inhabitants

Background MMLab (1/2)

• Multimedia Lab (MMLab)– research group

– headed by professor Rik Van de Walle– part of Ghent University – IBBT

• Ghent University– about 25,000 students

• IBBT (Institute for BroadBand Technology)– fosters cooperation between different universities and

industrial partners in Belgium » Alcatel-Lucent, Scientific Atlanta (Cisco), Barco, ...

– people• 6 staff members• currently 25 researchers (Ph.D. students and “others”)• about 15 master students each year

Background MMLab (2/2)

• Multimedia Lab (MMLab)– teaching

• five courses in the field of multimedia

– main research topics• advanced video applications

– e.g., adaptation of scalable and non-scalable video content• mobile multimedia applications

– e.g., transparent transfer of multimedia sessions• standardization in the field of multimedia systems and

applications– MPEG, JVT, and W3C

http://multimedialab.elis.ugent.be/

My Education @ Ghent University

• Master in Computer Science (1998 - 2002)– Master’s Thesis

• “Practical study and analysis of the QuickTime framework”– programming on top of the QuickTime API using Visual C++

• Ph.D. in Computer Science Engineering (2002 - 2007)– Ph.D. Dissertation

• “Description-driven adaptation of media resources”– format-independent video content adaptation using XML

• other research topics– assessment of video quality– GPU-based video processing

• complete list of publications (published)– http://www.elis.ugent.be/en/node/648?G=*&A=De Neve,+W

Video Content Adaptation

• Investigation of format-independent techniques for the efficient adaptation of (scalable) video content– using an XML-based approach

• inspired by MPEG-21 BSDL and XFlavor• novel contribution: design of BFlavor (BSDL + XFlavor)

– results verified for H.264/AVC, SVC, VC-1, and so on

original bitstreamIBB

P

adapted bitstream MPEG-21 BSDLI

P

transformed XML

STX

XMLBFlavor

XML-based Content Adaptation: Principles

I picture

B picture

B picture

P picture

t

Bitstream Syntax Description (BSD) Compressed bitstream

<bitstreamxml:base="bitstream_30hz.mpg"><header>0 24</header><I_picture>24 2637</I_picture><B_picture>2661 746</B_picture><B_picture>3407 903</B_picture><P_picture>4310 1657</P_picture>

</bitstream>

adapted bit stream suited forplayback in a constrained

environment

Video Quality Assessment

• Analysis of rate-distortion performance of H.264/AVC– by means of an extensive number of experiments – compared to state-of-the-art video coding formats

• VC-1– Windows Media Video 9

• MPEG-4 Visual– XviD and DivX

• MC-EZBC– wavelet-based video codec developed by John Woods

– use of different objective quality metrics • PSNR and JND

GPU-based Video Processing

• Use of the GPU– for color space conversions

• e.g., YCoCg-R to RGB

– for video effects• e.g., blurring and sharpening

– for H.264/AVC decoding• using Direct3D and NVIDIA CUDA

– Compute Unified Device Architecture• focus on motion compensation

– results presented at WIAMIS 2007 and SPIE Optics + Photonics 2007

Other Experiences

• Participated in MPEG standardization– contributed to improving MPEG-21 Digital Item Adaptation

• in particular, MPEG-21 BSDL– proposed extensions to BSDL included in an amendment

to MPEG-21 Digital Item Adaptation• Workpackage management

– for projects at a national (i.e., Belgian) and European level• Supervised 15 master students

– mainly in the domain of video adaptation, video coding, and rich media applications

• rich media applications: based on MPEG-4 XMT or Microsoft Silverlight

Remaining Activities at MMLab

• Follow-up of research regarding– XML-driven video content adaptation– GPU-based video processing

• design of a novel video codec with following requirements– GPU-friendly– lossless– scalable– support for high dynamic range imaging

• Activities are supposed to be limited in effort

Personal Research Goals

• To work towards a number of co-publications• To increase (practical) knowledge w.r.t.

– scalable video coding (SVC)– programming for the GPU using NVIDIA CUDA– new coding formats such as HD Photo / JPEG XR– lossless video coding– high dynamic range imaging (HDRI)

Possible Research Topics (1/2)

• Topic 1– comparison of adaptivity provisions of

• Microsoft HD Photo / JPEG XR (http://blogs.msdn.com/billcrow/)• JPEG 2000• intra-only SVC• (MPEG-4 VTC)

– from an MPEG-21 DIA point of view• this is, using MPEG-21 BSDL• how can MPEG-21 DIA be useful for a service such as Flickr?

– adaptation in the compressed domain– rotation in the compressed domain

– allows for cooperation with MMLab– needs further thinking to find good research questions...

Possible Research Topics (2/2)• Topic 2

– GPU-based video processing using CUDA• new computing paradigm• http://developer.nvidia.com/object/cuda.html

– e.g., in the context of SVC• motion estimation has to be done at several spatial layers• computationally expensive

– allows for cooperation with Bart Pieters in MMLab– my preferred research topic...– needs further thinking to find good research questions...

• Other proposals or suggestions?– e.g., multimedia ontologies for semantic adaptation, ...?

Some Remarks

• Constraints– time constraints

• difficult to start a completely new research topic• probably, the best thing to do is to continue to build on my

previous research experience

– group constraints• preference for close cooperation with people in the group

– e.g., people working on video adaptation, SVC, GPU, ...?

• Proposed topics– allow for cooperation between IVY Lab and MMLab, and

hence, allow for co-publications