research & innovation 1 an industry perspective on vvg research oliver grau bbc research &...

1 Research & Innovation

An Industry Perspective on VVG Research

Oliver Grau

BBC Research & Innovation

VVG SUMMER SCHOOL '07


Overview

• Introduction– (Computer) Vision, Video & Graphics in media production

• Integration of (real) Video & Graphics• Production visualisation tools• Free-viewpoint video for Sport visualisation

• Summary


Introduction

• BBC is a public funded broadcaster producing different kinds of media. Mainly: Radio, TV and Online

• Video: One of the ‘Main businesses’• Graphics – used to:

– Add editorial value– Make programmes (visually) more exciting– Create new user’s experience

• Vision: Big toolbox to make things happen


Introduction

• Added editorial value


Introduction

• New user experience: 3D Virtual environment

CBBC Adventure Rock: www.bbc.co.uk/cbbc/adventurerock/


Introduction


Integration of video & graphics

• ‘Classical’ application: Weather forecast

• Studio with chroma-key facility


What makes an integrated ‘virtual scene’ looking ‘real’

Real Scene Virtual Scene

Depth perception

Camera perspective

Occlusions

Shadows

Reflections


Integration of video & graphics

• Virtual studio – inserting actor into virtual scene

Virtual background

Moving camera

Camera Tracking


3D reconstruction of dynamic scenes in the studio

• Virtual studios have limited features regarding optical interaction of real and virtual objects

• More realistic integration requires full optical interaction in 3D– Shadows + reflections– Occlusions– Free choice of camera angle


3D reconstruction of dynamic scenes in the studio

• 12 fixed, calibrated cameras with chroma-keying facility• Developed in the IST-ORIGAMI project


Overview

Block diagram

Volumetric visual hull

computation..

Input images,Camera parameters

Surface computationMarching cube

algorithm

Volumetric model Surface model

Texture mapping


Visual hull computation

• Visual hull computation from image silhouettes

Camera-1

Camera-2

Camera-3


Artefacts in Visual hull-based 3D reconstruction

• Fundamental feature: – only parts of the volume that are visible as background in at

least one of the silhouette images are taken out no concavities can be modelled

• Occlusion errors Phantom volumes Reduced by increasing number of cameras

• Approximation errors

• Sampling errors


Visual hull-based 3D reconstruction

• Sampling errors– Binary volumetric scene representation Quantisation noise

12 cameras6 cameras12 cameras


Artefacts in dynamic visual hulls

• “Moving edges” as effect of discrete voxel grid

C2

C1

object

reconstruction

(moving) edge


Improved visual hull computation: Super-sampling

– Sub-divide voxels by LSS levels

– Assign voxel a (pseudo-) continuous value

Camera-1


Improved visual hull computation

• Gaussian smoothing• (Optional) complexity reduction


Results 3D model using super-sampling (20000 triangles)

After Gaussian smoothing

Standard visual hull / marching cube


Results

• Video from ORIGAMI demo production


Results

• Non-photorealistic rendering


Re-lighting

Studio illumination map

3D scene model

irradiance

Specular component

Original image

Light-neutral image


Re-lighting

research & innovation 1 an industry perspective on vvg research oliver grau bbc research &...

Documents