digital visual effects yung-yu chuangcyy/courses/vfx/10...• digital photography: – simply mimics...
TRANSCRIPT
![Page 1: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/1.jpg)
Computational Photography
Digital Visual EffectsgYung-Yu Chuang
with slides by Fredo Durand Ramesh Raskar Sylvain Paris Soonmin Bae Amit Agrawalwith slides by Fredo Durand, Ramesh Raskar, Sylvain Paris, Soonmin Bae, Amit Agrawal, Ramesh Raskar
![Page 2: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/2.jpg)
Computational photography
wikipedia:C i l h h f b dl Computational photography refers broadly to computational imaging techniques that enhance
t d th biliti f di it l h t h or extend the capabilities of digital photography. The output of these techniques is an ordinary h t h b t th t ld t h b photograph, but one that could not have been
taken by a traditional camera.
![Page 3: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/3.jpg)
What is computational photography
• Convergence of image processing, computer vision computer graphics and photographyvision, computer graphics and photography
• Digital photography:– Simply mimics traditional sensors and recording by
digital technologyI l l i l i i– Involves only simple image processing
• Computational photography– More elaborate image manipulation, more
computation– New types of media (panorama, 3D, etc.)– Camera design that take computation into account
![Page 4: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/4.jpg)
Computational photography
• One of the most exciting fields.S i C i l Ph h d • Symposium on Computational Photography and Video, 2005
• Full-semester courses in MIT, CMU, Stanford, GaTech, University of Delaware
• A new book by Raskar and Tumblin in SIGGRAPH 2007.
• IEEE International Conference on computational Photography San Francisco 2009Photography, San Francisco, 2009.
![Page 5: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/5.jpg)
Siggraph 2006 Papers (16/86=18.6%)gg p p ( )Hybrid ImagesDrag-and-Drop PastingTwo-scale Tone Management for Photographic LookTwo scale Tone Management for Photographic LookInteractive Local Adjustment of Tonal ValuesImage-Based Material EditingFlash MattingNatural Video Matting using Camera ArraysRemoving Camera Shake From a Single PhotographCoded Exposure Photography: Motion Deblurring Photo Tourism: Exploring Photo Collections in 3DPhoto Tourism: Exploring Photo Collections in 3DAutoCollagePhotographing Long Scenes With Multi-Viewpoint PanoramasProjection Defocus Analysis for Scene Capture and Image DisplayM lti i R di l C t di t i I i f S C tMultiview Radial Catadioptric Imaging for Scene CaptureLight Field MicroscopyFast Separation of Direct and Global Components of a Scene Using High Frequency Illumination
![Page 6: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/6.jpg)
Siggraph 2007 Papers (23/108=21.3%)gg p p ( )Image Deblurring with Blurred/Noisy Image Pairs Photo Clip ArtScene Completion Using Millions of Photographsp g g pSoft Scissors: An Interactive Tool for Realtime High Quality MattingSeam Carving for Content-Aware Image ResizingDetail-Preserving Shape Deformation in Image EditingVeiling Glare in High Dynamic Range ImagingVeiling Glare in High Dynamic Range ImagingDo HDR Displays Support LDR content? A Psychophysical EvaluationLdr2hdr: On-the-fly Reverse Tone Mapping of Legacy Video and PhotographsRendering for an Interactive 360-Degree Light Field DisplayMultiscale Shape and Detail Enhancement from Multi light Image CollectionsMultiscale Shape and Detail Enhancement from Multi-light Image CollectionsPost-Production Facial Performance Relighting Using Reflectance TransferActive Refocusing of Images and VideosMulti-aperture PhotographyDappled Photography: Mask-Enhanced Cameras for Heterodyned Light Fields and Coded
Aperture RefocusingImage and Depth from a Conventional Camera with a Coded ApertureCapturing and Viewing Gigapixel ImagesEfficient Gradient-Domain Compositing Using QuadtreesImage Upsampling via Imposed Edges StatisticsJoint Bilateral UpsamplingFactored Time-Lapse VideopComputational Time-Lapse VideoReal-Time Edge-Aware Image Processing With the Bilateral Grid
![Page 7: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/7.jpg)
Siggraph 2009 Papers (17/78=21.8%)Gaussian KD-Trees for Fast High-Dimensional FilteringEdge-Avoiding Wavelets and their ApplicationsMulti-operator Media RetargetingPatchMatch: A Randomized Correspondence Algorithm for Structural Image EditingModeling Human Color Perception under Extended Luminance LevelsMoving Gradients: A Path-Based Method for Plausible Image InterpolationOptimizing Content-Preserving Projections for Wide-Angle ImagesContent-Preserving Warps for 3D Video StabilizationVisio-lization: Generating Novel Facial ImagesCoordinates for Instant Image CloningSkyFinder: Attribute-based Sky Image SearchPaint SelectionVideo SnapCut: Robust Video Object Cutout Using Localized ClassifiersInvertible Motion Blur in VideoDark Flash Photography 4D Frequency Analysis of Computational Cameras for Depth of Field ExtensionBokode: Imperceptible Visual Tags for Camera-based Interaction from a Distance p p g
![Page 8: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/8.jpg)
Scope
• We can’t yet set its precise definition. The following are scopes of what researchers are following are scopes of what researchers are exploring in this field.
R d i h i l i– Record a richer visual experience– Overcome long-standing limitations of conventional
camerascameras– Enable new classes of visual signal
Enable synthesis impossible photos– Enable synthesis impossible photos
![Page 9: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/9.jpg)
Scope
• Image formation
• Color and color perception perception
• Demosaicing • Demosaicing
![Page 10: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/10.jpg)
Scope• Panoramic imaging
• Image and video registration
• Spatial warping operations
![Page 11: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/11.jpg)
Scope• High Dynamic
Range Imaging g g g• Bilateral
filtering and HDR display
• Matting
![Page 12: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/12.jpg)
Scope
• Active flash methods • Lens technology
Flash
• Lens technology • Depth and defocus
No-flashNo flash
our ltresult
![Page 13: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/13.jpg)
Removing Photography Artifacts using Gradient Projection and Flash-Exposure SamplingProjection and Flash Exposure Sampling
![Page 14: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/14.jpg)
Continuous flash
Flash = 0.0 Flash = 1.0
Flash = 0.3 Flash = 0.7 Flash = 1.4
![Page 15: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/15.jpg)
Flash matting
![Page 16: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/16.jpg)
Depth Edge Detection and Stylized Rendering Using a Multi-Flash CameraRendering Using a Multi-Flash Camera
![Page 17: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/17.jpg)
Dark flash photography
![Page 18: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/18.jpg)
Motion-Based Motion Deblurring
![Page 19: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/19.jpg)
Removing Camera Shake from a Single PhotographSingle Photograph
![Page 20: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/20.jpg)
Motion Deblurring using Fluttered Shutter
![Page 21: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/21.jpg)
Scope• Future cameras • Plenoptic function and light fields p g
![Page 22: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/22.jpg)
Scope• Gradient image manipulation
![Page 23: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/23.jpg)
Scope
• Taking great pictures
Art Wolfe Ansel Adams
![Page 24: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/24.jpg)
Scope
• Non-parametric image synthesis image synthesis, inpainting, analogies analogies
![Page 25: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/25.jpg)
Scope
Motion analysis analysis
![Page 26: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/26.jpg)
Image Inpainting
![Page 27: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/27.jpg)
Object Removal by Exemplar Based InpaintingExemplar-Based Inpainting
![Page 28: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/28.jpg)
Image Completion with Structure PropagationStructure Propagation
![Page 29: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/29.jpg)
Lazy snapping
![Page 30: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/30.jpg)
Grab Cut - Interactive Foreground Extraction using Iterated Graph CutsExtraction using Iterated Graph Cuts
![Page 31: Digital Visual Effects Yung-Yu Chuangcyy/courses/vfx/10...• Digital photography: – Simply mimics traditional sensors and recording by digital technology – Il l il i iInvolves](https://reader034.vdocuments.mx/reader034/viewer/2022052105/60410d7f0a07786cf7536021/html5/thumbnails/31.jpg)
Tools
• Graph cuts, Graph cuts, – Segmentation and mosaicing
• Gradient domain operations, – Tone mapping, fusion and matting
• Bilateral filters, – Denoising, image enhancementg, g