Why ‘Media’ in MIT Media Lab

Why ‘Camera Culture’

Ramesh Raskar, Asso. Prof., Camera Culture, MIT Media Lab

Processor Network+

Processor

People+

Network+

Processor

Computing InternetComputing

SocialComputing

Processor Network+

Processor

People+

Network+

Processor

Senses , I/o+

People+

Network+

Processor

Computing InternetComputing

SocialComputing

SensorySocial

Computing

AugmentedSocial

Computing

Process

Human

NWSense

VisualSocial

Computing

Bits

Phot

ons

Co-Media Lab

Elec

tron

s

Neur

ons

Computing InternetComputing

SocialComputing

SensorySocial

Computing

AugmentedSocial

Computing

Cells

Mus

ical N

ote

Emerging Worlds

Emerging Tech

Predictable World

PredictableTech

M_L: last 30 years

M_LABx: next 30 years

Camera Culture: Motivating Questions• What will a camera/display look like in 20 years?

• How will the next billion cameras change the social culture?

• How can we augment the camera to support best ‘image search’?

• How will portable health diagnostics impact healthcare?

• Will we live mostly in virtual/augmented reality, telepresent?– TiVo for life?

• How will ultra-high-speed/resolution imaging change us?

• How can we improve ‘trust’ in imaging?

• Can we print anything .. Cars, food, .. babies .. – (what will make Amazon obsolete)

• What are the opportunities in pervasive recording?– e.g. GoogleEarth Live

• What will be in Photoshop2030?

• What is the future of movie-making, news reporting, sports viewing?

Bits

Phot

on H

acki

ng

Computer Vision

Optics

Sensors

Visual Social Computing

Computational Photography

Imaging Research: Codesign of Optical and Digital Processing

Signal Processing

Computational Light TransportDisplays

Machine Learning

HCI

Capture Analyze Share

Capture

Analyze

Share

ViSoCo

Camera CultureCreating new ways to capture and share visual information

MIT Media LabRamesh Raskar

http://cameraculture.infoFacebook.com/cameraculture

1.Light-Field CameraA new camera design exploiting the fundamental dictionary of light-fields for a single-capture capture of light-fields with full-resolution refocusing effects.

2. Color PrimariesA new camera design with switchable color filter arrays for optimal color fidelity and picture quality on scene geometry, color and illumination.

3. Flutter-ShutterA camera that codes the exposure time with a binary pseudo-sequence to de-convolve and remove motion blur in textured backgrounds and partial occluders.

4. Compressive CaptureWe analyze the gamut of visual signals from low-dimensional images to light-fields and propose non-adaptive projections for efficient sparsity exploiting reconstruction.

Computational Photography

1. Looking around cornersUsing short laser pulses and fast detector, we aim to build a device that can look around corners with no imaging device in the line of sight using time resolved transient imaging.

2. Reflectance RecoveryWe demonstrate a new technique that allows a camera to rapidly acquire reflectance properties of objects 'in the wild' from a single viewpoint, over relatively long distances and without encircling equipment.

3. Trillion Frames per Second ImagingA camera fast enough to capture light pulses moving through objects. We can use such a camera to understand reflectance, absorption and scattering properties of materials.

Femtosecond Imaging 3D Displays1. Tensor Display

A family of compressive light field displays comprising all architectures employing a stack of time-multiplexed, light-attenuating layers illuminated by uniform or directional backlighting

2. Layered 3DTomographic techniques for image synthesis on displays composed of compact volumes of light-attenuating material. Such volumetric attenuators recreate a 4D light field or high-contrast 2D image when illuminated by a uniform backlight.

3. Glasses-free 3D HDTVLight field displays with increased brightness and refresh rate by stacking a pair of modified LCD panels, exploiting rank and constraint of 3D displays

4. BIDI ScreenA thin, depth-sensing LCD for 3D interaction using light fields which supports both 2D multi-touch and unencumbered 3D gestures.

5. Living Windows 6D DisplayA completely passive display that responds to changes in viewpoint and changes in incident light conditions.

May 2012

1. Augmented Light FieldsExpands light field representations to describe phase and diffraction effects by using the Wigner Distribution Function

2. Hologram v Parallax BarrierDefines connections between parallax barrier displays and holographic displays by analyzing their operations and limitations in phase space

3. Ray–Based Diffraction ModelSimplified capture of diffraction model for computer graphics applications.

Post-Doctorial Researchers: Doug Lanman, Gordon Wetzstein, Alex Olwal, Christopher BarsiResearch Assistants: Matthew Hirsch, Otkrist Gupta, Nikhil Naik, Jason Boggess, Everett Lawson, Aydın Arpa, Kshitij Marwah Visiting Researchers & Students: Di Wu, Daryl Lim

1. Retinal ImagingWith simplified optics and cleaver illumination we visualize images of the retina in a standalone device easily operated by the end user.

2. NETRA/CATRALow-cost cell-phone attachments that measures eye-glass prescription and cataract information from the eye.

3. Cellphone MicroscopyA platform for computational microscopy and remote healthcare

4. High-speed TomographyA compact, fast CAT scan machine using no mechanical moving parts or synchronization.

5. Shield Fields3D reconstruction of objects from a single shot photo using spatial heterodyning.

6. Second SkinUsing 3D motion tracking with real-time vibrotactile feedback aids the correct of movement and position errors to improve motor learning.

Health & Wellness

1. BokodeLow-cost, passive optical design so that bar codes can be shrunk to fewer than 3mm and read by ordinary cameras several meters away.

2. Specklesense Set of motion-sensing configurations based on laser speckle sensing . The underlying principles allow interactions to be fast, precise, extremely compact, and low cost.

3. Sound AroundSoundaround is a multi-viewer interactive audio system, designed to be integrated into multi-view displays presenting localized audio/video channels with no need for glasses or headphones.

Human Computer Interaction Visual Social Computing

1. PhotocloudA near real-time system for interactively exploring a collectively captured moment without explicit 3D reconstruction.

2. Vision BlocksOn-demand, in-browser, customizable, computer-vision application-building platform for the masses. Without any prior programming experience, users can create and share computer vision applications.

3. LenschatLensChat allows users to share mutual photos with friends or borrow the perspective and abilities of many cameras.

Light Propagation Theory and Fourier Optics

Visit us online at

Cameraculture.infofb.com/cameraculture

Visual Social Computing

Xd

X++

X X+Y

X

X

neXt

How to come up with new ideas?

Great Research: Strive for Five

1. Before Five teamsBe first, often let others do details

2. Beyond Five yearsWhat no one is thinking about

3. Within Five layers of ‘Human’ ImpactRelevance

4. Beyond Five minutes of descriptionDeep, iterative, participatory

5. Fusing Five+ ExpertiseMulti-disciplinary, proactive

Ramesh Raskar, http://raskar.info

Ramesh Raskar, MIT Media Lab

Pick atleast 2 out of 3

FunCoolMedia Coverage

ImpactMoneySocial implications

ResearchNoveltyGeneralityScience

What distinguishes ML projects .. – Synthesize not just analyze– Use power of human intelligence

• Intelligence Amplification• Human in loop,

– Democratize, Power to the People– Be paranoid .. Are we relevant and what is next?

Topics for discussion (create your own group)• How to pursue 'ideas in the spirit of the media lab'? What is not in the spirit of ML? • How to make the best of ML resources? • What are the common problems in picking/initiating/pursuing/finishing great

projects? • Case studies of successful transitions of efforts into research/demos/products and

more• Some procedural topics: juggling classes vs research, Apprenticeship vs

independent research, group dynamics, media coverage

Ramesh Raskar, http://raskar.info

Be proactive not reactiveGeneralize today’s conceptsAvoid basing all on facebook/twitter/kinect etc or today’s hot techBut we are still slaves to available tech“Let’s do smart things with stupid technology today, rather than wait and do stupid things with smart technology tomorrow” - Bill Buxton. You can ofcourse do even smarter things with smart tech.

Be prepared but careful on what you doBe in optimist but be paranoid (vs pessimist + laidback)Defer judgment (don’t dismiss, believe or start instantly on any idea)Overnight success after months of workHave a list of 10-20 problems .. Don’t work on first one you think or that comes your wayTalk to a lot of people (you trust) to see if worth purusing as most ideas will be useless anyway

Don’t be religious, listen to othersI often find people too much in love with a tiny incremental idea if they came up with themselves Try and change what is difficult to some other situation which is easily done but is still importantDon’t work on same project for 2+ yearsFail fast

If u want to win .. Be willing change rules of the game, sometimes the game itselfRemember the 4Ps and their SEQUENCE

– Projects > Papers > Polished Demo/Prototype > Press– (Note ‘polished demo’ comes AFTER a paper or some external validation)– Don’t chase press before you have a serious project that is peer-reviewed or validated (novelty and

impact should be already understood)Ramesh Raskar, http://raskar.info