and Computational Photography - ?· What is Computational Photography? Computational photography is…

Download and Computational Photography - ?· What is Computational Photography? Computational photography is…

Post on 25-Aug-2018

222 views

Category:

Documents

9 download

Embed Size (px)

TRANSCRIPT

<ul><li><p>Pixels,Numbers,andProgramsComputational PhotographyComputationalPhotography</p><p>StevenL.Tanimoto</p><p>Pixels,Numbers,andPrograms;S.Tanimoto 1ComputationalPhotography</p></li><li><p>Outline</p><p>Wh t i t ti l h t h ?Whatiscomputationalphotography?AutofocustechniquesHi h d i (HDR) h hHighdynamicrange(HDR)photographyCatadioptric camerasSeparatingimagesduetodirectandindirectlightingOtherdevelopments</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 2</p></li><li><p>WhatisComputationalPhotography?</p><p>Computational photography is an emerging technology forComputationalphotographyisanemergingtechnologyforacquiringimagesthroughacombinationofoptics,sensors,andcomputers.</p><p>Itincludesautomaticcameracontrol,computationofpixelvaluesusingmultiplesourcesofinformation,aswellasg p ,computingalternativerepresentationsofvisualinformationto2Dimages(e.g.,3Dmodels).</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 3</p></li><li><p>Autofocustechniques</p><p>Focusing a camera lens was once a tedious chore.Focusingacameralenswasonceatediouschore.Instant(film)camerasusedfixedfocus,limitingphotographytowelllitscenesandfast(highASA)p g p y ( g )films.Acousticrangefinding wasusedonsomecamerastog f gdeterminethedistancefromthecameratothefirstsurfaceinthemiddleofthefieldofview.Digitalcameraschangetheeconomicsofautofocus.Now,techniquesbasedonsharpnessmeasurements</p><p>ff tiPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography</p><p>areeffective.4</p></li><li><p>Autofocustechniques(cont.)</p><p>Brennersharpness:</p><p>Tenengrad sharpness:</p><p>HereGx andGy arethehorizontalandverticalgradientoperators</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography</p><p>x y g pusedintheSobel edgedetector.</p><p>5</p></li><li><p>Autofocustechniques(cont.)</p><p>x</p><p>Frame#1 Frame#10</p><p>Sharpnessasafunctionoffocussetting.Bestfocus</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 6</p><p>gisatthe10th frame.</p></li><li><p>HighDynamicRangePhotography</p><p>The dynamic range of a photo or camera is the rangeThedynamicrangeofaphotoorcameraistherangeofbrightnesslevelsthatitcanaccuratelycapture.Historically,dynamicrangewaslimitedby(a)filmy, y g y ( )technology,and(b)digitallightsensortechnology.</p><p>Thephotographerschallenge:takeapictureofascenethatcontainsbothbrightanddarkregions,withoutlosingthedetailsinoneoftheseregions.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 7</p></li><li><p>HDR(cont.)</p><p>The brightly lit area is capturedThebrightlylitareaiscapturedeffectivelyinthisphoto(takenwitha1/60sec.exposure).</p><p>Butthelowerhalfofthephotop(theindoorpartofthescene)istoodarkandlackingincontrast.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 8</p></li><li><p>HDR(cont.)</p><p>Here, the darkest areas areHere,thedarkestareasarecapturedeffectively(takenwitha15sec.exposure).</p><p>Buttheupperhalfofthephotopp p(theoutdoorpartofthescene)istoobrightandlackingincontrast.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 9</p></li><li><p>HDR(cont.)</p><p>Now, both areas have someNow,bothareashavesomevisiblestructure(takenwitha1/2sec.exposure).</p><p>Butneithershowsdetailswell.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 10</p></li><li><p>HDR(cont.)</p><p>HDR to the rescue!HDRtotherescue!Takingafullsequenceofexposures,webuildahighp , gdynamicrangeimageinsidethecomputer.Fromit,weapplytonemappingtogetanimagethathasgoodcontrastacrosstherangerange.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 11</p></li><li><p>HDR(cont.)</p><p>HDR to the rescue!HDRtotherescue!Takingafullsequenceofexposures,webuildahighp , gdynamicrangeimageinsidethecomputer.Fromit,weapplytonemappingtogetanimagethathasgoodcontrastacrosstherange Th id l i i i frange. Theidea:getluminosityinfoabouteachpixelfromeachimage,butweighttheevidenceaccordingtowhateachimageisgood at</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography</p><p>goodat.</p><p>12</p></li><li><p>Catadioptric Imaging</p><p>Using optical systems that combine lenses andUsingopticalsystemsthatcombinelensesandmirrors,anamorphicimagescanbecapturedthatcontainentirepanoramas.Computerscaninverttheanamorphicdistortion.</p><p>CourtesyofProf.ShreeNayar ofColumbiaUniversity.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 13</p></li><li><p>Catadioptric Imaging</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 14</p></li><li><p>Catadioptric Cameras</p><p>TheOneshot360(RemoteReality,Inc.).Threeothercatadioptric cameras.http://www.cs.columbia.edu/CAVE/projects/cat_cam_360/</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 15</p></li><li><p>SeparatingDirect&amp;IndirectLighting</p><p>Thecolorofanobject(specificallyatapointPontheobject)inascenedepends on two different physical effects:dependsontwodifferentphysicaleffects:a. Directreflectionoflightfromthelightsourceoffthesurfaceofthe</p><p>object(atpointP),andb Light coming from point P on the object that did not come directlyb. LightcomingfrompointPontheobjectthatdidnotcomedirectly</p><p>fromthelightsource:i.lightthathascomefromthelightsourcebutbouncedoffother</p><p>surfaces before getting to PsurfacesbeforegettingtoP.ii.lightthathasenteredthesurfacematerial(e.g.,thepaintlayer,</p><p>whichistypicallyadielectricmaterialandispartiallytranslucent)elsewhere from point P but emerges from point PelsewherefrompointPbutemergesfrompointP.</p><p>Weareoftenunawareof(ii).However,usingcomputationalphotography,we can separate the effects of (i) and (ii) The results are amazing</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography</p><p>wecanseparatetheeffectsof(i)and(ii).Theresultsareamazing.</p><p>16</p></li><li><p>Direct&amp;Indirect(cont.)</p><p>indirectlightsource surfacesin</p><p>thescene</p><p>P</p><p>camera</p><p>Rt</p><p>RadianceatPtowardsthecamera:Rt =Rd +RgRg istheglobalcomponentofradiance(basedonallindirectilluminationofP).Rt isthetotalradiance.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 17</p></li><li><p>Direct&amp;Indirect(cont.)</p><p>indirectlightsource</p><p>BarriertocastashadowonP</p><p>P</p><p>camera</p><p>Rg</p><p>RadianceatPtowardsthecamera:Rt =Rd +RgRg istheglobalcomponentofradiance(basedonallindirectilluminationofP).Rt isthetotalradiance.By measuring R and R then we can compute R as R R</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 18</p><p>BymeasuringRt andRg thenwecancomputeRd asRt Rg</p></li><li><p>ResultsofSeparation</p><p>Byusingstructuredlight(projectionofcheckerboards),itisnotnecessaryto have a separate shadow image for each pixel. In theory, only 2 imagestohaveaseparateshadowimageforeachpixel.Intheory,only2imagesareneeded,butinpracticeitsbesttohave10to20images,accordingtoKrishnanandNayar.</p><p>Rt Rd Rg</p><p>In this example most of the radiance from global illumination is due toInthisexample,mostoftheradiancefromglobalilluminationisduetointerreflectance amongtheeggs.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 19</p></li><li><p>ResultsofSeparation</p><p>AscenewithgrapesandcheeseMuch of the natural color (as seen in R ) comes from R and not RMuchofthenaturalcolor(asseeninRt )comesfromRg andnotRd .</p><p>Rt Rd Rg</p><p>Here,muchoftheradiancefromglobalilluminationisduetolightpassingthroughthetranslucentgrapesorcheese.Thecheese,shownwithdirectilluminationonly,issomewhatunappetizing.Thegrapesalsoappearblue.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography</p><p>Applications:imageenhancement,imageanalysis.20</p></li><li><p>OtherDevelopments</p><p>3Dcameras cameratakestwoormoreshotsofsamescene,then builds 3D model of scenethenbuilds3Dmodelofscene.Redeyereduction.Alreadystandardistheautomaticuseofapreflashtoconditionthepupilsofphotosubjectstoclosedown.Redeyecanalsobereducedusingfacedetectionmethodsandspecializedalgorithms.Light field cameras By capturing a sample of the complete lightLightfieldcameras.Bycapturingasampleofthecompletelightfieldattheviewpoint,operationssuchasfocusingonspecificobjectscanbeperformedlater.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 21</p></li><li><p>OtherDevelopments(cont.)</p><p>IncameraHDR.Automaticexposurebracketingisleadingtheway to built in HDR capturewaytobuiltinHDRcapture.Motionblurinversion.Thecameracanestimateitsmotionusingacombinationofaccelerometerdataandvideodata.Themotioninfocanthenbeusedtopartiallyinvertthemotionblur.Allsmilessnapshots.Usingfacedetection,smiledetectionalgorithms and inpainting techniques it is possibly to synthesizealgorithms,andinpainting techniques,itispossiblytosynthesizeanimageofagroupofpeopleinwhicheveryoneissmiling,eventhoughineachcomponentshot,therewassomeonenotsmiling.</p><p>Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 22</p></li></ul>

Recommended

View more >