raskar stanfordextremecompuimagingapr2016
TRANSCRIPT
A Proposed Strategy
Camera CultureRamesh Raskar
Ramesh RaskarAsso. ProfessorMIT Media LabExtreme Computational Imaging
Mohit, Shah, Angelino, Sinha
Provide oral health solutions via a tooth brush that takes achieves x-ray like screening4
Vision is important for day labor for his employment5
Huang, Wetzstein, Barsky, Raskar. SIGGRAPH 2014.Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computational Light Field Displays.
EyeSelfie, Swedish et al SIGGRAPH 2015
Siggraph 2015 Paper: EyeSelfie: Swedish, Roesch, , Raskar8
ReplacementExpansionPredictionPredictive Modeling of Future Health
Theia: API for Diabetic Retinopathy ScreeningOpen API and Documentationtheia.media.mit.eduDrag and Drop Web DemoSwedish et al
Visual Urban SensingNaik, Glaeser, Hidalgo, Raskar (email: [email protected])Crowdsourcing
Computer Vision+
3.6/10Sentiment Analysis for Street-level Images
Quantifying Urban ChangeApplications: Real estate, Data-driven urban policy and planning
DeclineImprovementstreetscore.media.mit.eduHow to use computer vision and crowdsourcing to evaluate the appearance of cities across the world at street-level resolution and global scale?
11
Smartphone spectroscopy for consumer applications
Rechargeable batteryArduino pro miniBluetooth moduleSpectrometer chipLEDFilter
Light switch
Power switch
Application in rapid testing of food quality: Chlorophyll fluorescence in fruitsAnshuman Das, Akshat Wahi, Ishan Kothari and Ramesh Raskar
Hand-held 3D Imager for biomedical applications
3D imaging of throat and tonsils for evaluation of obstructive sleep apneaAnshuman Das, Tulio Valdez, Ramesh Raskar et. Al., Biomed. Opt. Express7, 1445-1457 (2016)3D imaging of the tympanic membrane for accurately detecting ear infections
Anshuman Das, Julio Estrada, Ramesh Raskar et. Al., Proc. SPIE, 9303, 93031F (2015)
Health and Wellness Interfaces
Positions Open (tiny.cc/ccjobs)
Collaboration/Expt
Co-funding
16
http://cameraculture.infohttp://Fb.com/cameraculture MIT Media Lab
Raskar, Camera Culture, MIT Media Lab
Bit HackingPhoton HackingComputer VisionOpticsSensors
Signal ProcessingDisplaysMachine LearningComputational Light TransportComputational PhotographyIllumination
My work involves creative new ways to play with light by co-designing optical and digital processing.
My work lies at the INTERSECTION of processing of photons and processing of bits.
At MERL, I transformed the field of computational photography, with key papers and impact on productsAt Media Lab, I invented a new field computational light transport 18
DiffractionDynamic RangeDescattering
DefocusDepth
20
MilliMicroNanoPicoFemtoAttoConquer .. Time
If I turn this laser point ON and OFF in a trillionth of a second .. i.e. in a few femto-second ..
21
MilliMicroNanoPico
AttoFemto-graphyConquer .. Time NoiseSignal
I will create a packate of photons .. less than a milimeter long .. Bullet of light .. Traveling a million times faster than ord bullet22
Light in Slow Motion .. 10 Billion x Slow
Mitsubishi Electric Research LaboratoriesSpecial Effects in the Real WorldRaskar 2006
What if we send this bullet of photons to shatter into this bottleHow does light look in slow motionThe whole sequence is less than a nanosecond stretched to about 10 seconds i.e. 10 billion times slowed downIf an oridinary bullet, same distance at such a slow motion video, do you know how long you will have to sit here to watch the movie? A day, a week .. a whole year .. it will be a very boring movie ..
Clean_coke_mulSlow2v1.mov
23
Velten et al, Siggraph 2013
Mitsubishi Electric Research LaboratoriesSpecial Effects in the Real WorldRaskar 2006
Andreas Velten, Di Wu, Christopher Barsi, Everett Lawson
Mitsubishi Electric Research LaboratoriesSpecial Effects in the Real WorldRaskar 2006
25
Raw Data
so we have to use a stroboscopic sampling method to fire the bullet millions of times and do clever sync and record raw data, and use computational techniques to to weave together femto-video.
Worlds fastest videosSame photo about 10 million times to get enough photonsWorlds fastest slowest camera ..
26
Can you seearound corners ?
Superman can fly and heroes can becomes invisible .. But what about a new power for a future super-hero ..To see around corners .. 27
2nd BounceMulti-path Analysis1st Bounce3rd Bounce28
My idea is to use the multiple bounces of light i.e. echoes of light.
Echoes of sound -> Echoes of light
We all know about echoes of sound.But sounds travels slow and we can actually hear the echoesLight travels fast so we need specialized hardware to listen to these echoes.So we end up using light sources and cameras that run at a trillion frames per second (not a million and not a billion, but trillion)28
Collision avoidance, robot navigation, 29DARPA REVEAL Program by Dr.PredragMilojkovic
Motion beyond line of sightPandharkar, Velten, Bardagjy, Lawson, Bawendi, Raskar, CVPR 2011 30
, bronchoscopies, 31
And to see deep inside the body with endoscopes that look in narrow channels in lungs Or colons
Colonoscopy/cardioscopy32
Identify Materials
34
Naik, Zhao, Velten, Raskar, Bala, (SIGGRAPH Asia 2011)BRDF from a single viewpoint
Mitsubishi Electric Research LaboratoriesSpecial Effects in the Real WorldRaskar 2006
Single Photon Sensitive Imaging
Gariepy et al. Nature Comm (2015)
Millisecond
Microsecond
NanoSecond
PicoSecond
FemtoSecond
AttoSecond
RFlowCAT-scanPhone3DReading BookFrequencyTime ResolutionLook Around CornersAudioRFXrayLightTHzFLIM
37
Multi-Dimensional Light Transport5-D Transport
38
Nanophotography70 picosecond resolution
[Kadambi et al 2013]
40
Real-time Localization
ImagingReal Time Localization
Kadambi, Zhao, Shi, Raskar. "Occluded Imaging with Time of Flight Sensors." ACM ToG 2016 (Pres. at SIGGRAPH)
Wavelength vsShininess
Macroscopic Interferometry
Optical Interferogram
Electronic Interferogram
Kadambi, Schiel, Raskar CVPR 2016 (Oral)
Macroscopic InterferometrySNR Benefits
Works at long-range
Obeys theoretic boundsfrom optical coherence
Macroscopic Interferometry
Depth Sensor
Kadambi, Taamazyan, Shi, Raskar, ICCV 2015Polarized 3D
MitigateExploitCreate
Scattered Light
48
FLIMSatat et al. Nature Comm 2015Bhandari et al. 2015
49
Application: Fluorescence Lifetime ImagingToF Imaging : Relative Phase Ecodes Distance InfoFLIM : Relative Phase Ecodes Lifetime Info
Probing FunctionResponse
Time Domain FLIM
ToF ImagingFLIM Imaging
Frequency Domain FLIM
ExcitationFluorescence
Lifetime
Parameterizes Lifetime
Application: Fluorescence Lifetime ImagingBhandari, Barsi, Raskar, 2016Blind and reference-free fluorescence lifetime estimation via consumer time-of-flight sensors
Location + Fluoro Lifetime Behind Diffuser
Satat et al. Nature Comm (2015)Bhandari et al. 2015
Imaging Behind Thick Scattering Media
1.5cm Thick Diffuser
Hidden SceneReconstruction with ToFSatat et al. Imaging Through Thick Medium Using Time-Resolved Measurement COSI (2015)
Mohit, Shah, Angelino, Sinha
Provide oral health solutions via a tooth brush that takes achieves x-ray like screening54
Optical BrushHeshmat, Lee, Raskar,,Nature Scientific Reports, 2016
55
Heshmat, Lee, Raskar,,Nature Scientific Reports, 2016Optical Brush
Sanchez, Heshmat, Reza, Romberg, Raskar 2015
Ancient text witout damaging books58
Sanchez, Heshmat, Reza, Romberg, Raskar 2015
Bedri, Gupta, Mohit et alTime-of-Flight Microwave CameraCharvat, Temme, Feigin, RaskarNature SciRep 2015
Kadambi, Bhandari et al
How can we drive on foggy, crowded streetsAs if it is a sunny day on a clear road63
Millisecond
Microsecond
NanoSecond
PicoSecond
FemtoSecond
AttoSecond
RFlowCAT-scanPhone3DReading BookFrequencyTime ResolutionLook Around CornersAudioRFXrayLightTHzFLIM
64
Multi-Dimensional Light Transport5-D Transport
My Photo
24 roll film for one year
65
Mitsubishi Electric Research LaboratoriesSpecial Effects in the Real WorldRaskar 2006