svr raskar
TRANSCRIPT
![Page 1: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/1.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Ramesh RaskarMitsubishi Electric Research Labs (MERL), Cambridge, MA,
USA
New Directions in Augmented RealityNew Directions in Augmented Reality
![Page 2: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/2.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Parent Organization: Parent Organization: MELCOMELCO
• Mitsubishi Electric (MELCO), including subsidiaries– $32B in annual sales– 125,000 people– 10 business sectors:
• Space Development and Satellite Communications• Communications and Information Processing• Visual Equipment and Systems• Electronic Devices• Home Electronics• Energy• Industrial Equipment• Public-Use Systems• Transportation
![Page 3: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/3.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
MERLMERL• Mitsubishi Electric Research Laboratories
– Dick Waters, President & CEO• Two labs, co-located in Cambridge, MA:
– MERL Technology Lab• Kent Wittenburg, Director• Advanced systems development• ~35 researchers
– MERL Research Lab• Joe Marks, Director• Basic research in applied computing• 23 researchers, interns, consultants• CV, HCI, Multimedia, CG, AI
![Page 4: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/4.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented RealityNew Directions in Augmented New Directions in Augmented RealityReality
• Spatially Augmented Reality: – Alternative approach
• Special effects in real world: – Next big graphics challenge
• Location aware RFID: – Converging technologies
www.raskar.com
![Page 5: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/5.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Non-trivial ARNon-trivial AR
• Painting with Light– [Bandyopadhyay, Raskar, Fuchs 2001]
![Page 6: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/6.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Spatially Augmented Reality Spatially Augmented Reality (SAR)(SAR)
HMD-VR Spatially Immersive-VR
AR using HMDSpatially Augmented
Reality
e.g. CAVE
Video or Optical see-through
SAR, Shaderlamps
![Page 7: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/7.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Classification of ARClassification of AR
![Page 8: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/8.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Classification of ARClassification of AR
Spatially Augmented Reality
![Page 9: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/9.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Spatially Augmented RealitySpatially Augmented Reality
Raskar, vanBaar, Beardsley, Willwacher, Rao, Forlines‘iLamps: Geometrically Aware and Self-Configurable Projectors’,
SIGGRAPH 2003
![Page 10: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/10.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
AR IssuesAR Issues
• Preprocessing:– Authoring
• Runtime:– Identification: Recognition of objects
• Using markers and visual tags– Registration: Finding relative pose of display
device• Dynamic estimate of translation and rotation• Render/Warp images
– Interaction:• Widgets, Gesture recognition, Visual feedback
![Page 11: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/11.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Advantages of ProjectorsAdvantages of Projectors
– Size of image
– Combination of images
– Shape of display surface
Image can be larger than device
Images can be superimposed and added
Displayed images may be non-planar
![Page 12: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/12.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
DisadvantagesDisadvantages
• Projector limitations– Limited depth of field– Shadows– Affected by display surface reflectance
• Challenges– Calibration required– Rendering involves complex relationships
![Page 13: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/13.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Advantages ofAdvantages ofSpatial Augmentation (SAR)Spatial Augmentation (SAR)
– Augmentation of objects not view– Wide area, High resolution
Comparison– Body-Worn Displays
•Better ergonomics•Reduced tracking requirements
– Hand-held Displays•Avoids ‘last foot’ problem
![Page 14: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/14.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Classification of ARClassification of AR
Spatially Augmented Reality
ShaderLamps
![Page 15: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/15.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Shader LampsShader LampsMotivationMotivation
View-dependent Appearance
![Page 16: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/16.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Shader LampsShader LampsImage based IlluminationImage based Illumination
– Basic Idea• Render images and project on objects• Multiple projectors• View and object dependent color
Raskar, Welch, Low, Bandyopadhyay, “Shader Lamps: Animating Real Objects with Image Based Illumination,” Eurographics Rendering Worksop (EGRW 2001)
![Page 17: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/17.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
![Page 18: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/18.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Changing AppearanceChanging Appearance
Projector Projector
Virtual light source
![Page 19: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/19.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
ExamplesExamples• Son et Lumiere
Projecting slide of augmented photo
![Page 20: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/20.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
ExamplesExamples• Disney’s Haunted Mansion
Pre-recorded video
Singing busts
Madame Leota
![Page 21: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/21.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
• Old– Large,
rigid installations
– A 2D image or video projection
– Single projector
– Texture
• New– Easy setup,
Non-trivial objects
– Real time 3D animation
– Multiple projectors
– BRDF
![Page 22: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/22.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
ChallengesChallenges• Complete illumination
– Image alignment– Special effects
• Changing appearance and lighting
– Complex geometry, self-occlusions– Merging multiple projectors
![Page 23: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/23.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
StepsSteps– Preprocessing
• Scan 3D object and create model• Approximately position projector(s)• Compute pose, P
– Find features– Find pixels that illuminate them
• Compute intensity correction– Run time
• Render images of 3D model• Intensity correction for object shape• Feathering for projector overlap
![Page 24: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/24.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
StepsSteps– Preprocessing
• Scan 3D object and create virtual model, G
Faro arm
![Page 25: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/25.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
StepsSteps– Preprocessing
• Scan 3D object and create model, G• Approximately position projector(s)• Find pose, P
![Page 26: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/26.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
MotivationMotivation• Projector - a 3D projection device
– Projector is a dual of a camera– Relates 3D space and image in framebuffer– A useful abstraction : geometric projection model
![Page 27: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/27.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Projector ModelProjector Model
• Pin hole model– Equations for perspective projection– Relationship between 3D and 2D– Intrinsic and Extrinsic Parameters
![Page 28: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/28.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Camera (and Projector) anatomy
Camera center
Image plane
Principal point
Principal axis
![Page 29: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/29.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
StepsSteps– Preprocessing
• Scan 3D object and create model• Approximately position projector(s)• Compute pose, P
– Find fiducials– Find pixels that illuminate them
• Compute intensity correction– Run time
• Render images of 3D model using matrix P• Intensity correction for object shape• Feathering for projector overlap
![Page 30: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/30.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Changing AppearanceChanging Appearance
Projector Projector
Virtual light source
![Page 31: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/31.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
StepsSteps– Preprocessing
• Scan 3D object and create model• Approximately position projector(s)• Compute pose, P
– Find fiducials– Find pixels that illuminate them
• Compute intensity correction– Run time
• Render images of 3D model using matrix P• Intensity correction for object shape• Feathering for projector overlap
![Page 32: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/32.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Radiance AdjustmentRadiance Adjustment
Virtual
L(x,) = F(x, , i) Li (x, i) di
I
d
Desiredradiance BRDF
Incidentradiance
![Page 33: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/33.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
k(x) cos(p )
d(x)2
Radiance AdjustmentRadiance Adjustment
Real
L(x,) = F(x, , i) Li (x, i) di
L’(x,) =
I
d
Ip (x, p)
Virtual
Resultantradiance
Pixelintensity
![Page 34: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/34.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
d ( x )2
k ( x ) cos( p )
Radiance AdjustmentRadiance Adjustment
Ip (x, p) =
I
d
, k ( x ) > 0L ( x, )
Virtual Real
Intensity correction
Desired radiance
Pixelintensity
Reflectance
![Page 35: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/35.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
– Rendering with• Light at c.o.p. : cos(p)
• Diffuse reflectance : k• Distance attenuation : 1/ d (x)2
p > 600 cut off
Intensity CorrectionIntensity Correction
Per-pixel factor
xL'
PIP
d
d ( x )2
k ( x ) cos( p )Ip (x, p) = L ( x, )
Rendered Image
![Page 36: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/36.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Feathering in OverlapFeathering in Overlap
A BProjectors
id
TraditionalSolution
A
BA+B
Projected Surface
Weights
![Page 37: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/37.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
FeatheringFeathering
A B’Projectors
id
TraditionalSolution
BA+BA
BA+B
Projected Surface
Weights
![Page 38: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/38.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
FeatheringFeathering
A B’Projectors
id
TraditionalSolution
A+B
A+B
Projected Surface
Weights
![Page 39: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/39.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
OcclusionsOcclusions
A BProjectors
hgd e f
A
A
B
BB
A+B
i
![Page 40: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/40.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Occlusion ProblemsOcclusion Problems
A BProjectors
hgd e f
A
AB
A+B
i
Depth discontinuity
![Page 41: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/41.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
New FeatheringNew Feathering
A BProjectors
hgd e f
A
A
AAB
B
B
A+B
B
A+B
Overlapbuffer 1 1 12 2 2
i
Depth discontinuity
Overlap 1 1
![Page 42: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/42.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
New FeatheringNew Feathering
A BProjectors
hgd e f
A
A
AA
B
A+B
B
A+B
Overlapbuffer 1 1 12 2 2
i
Depth discontinuity
1 1
![Page 43: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/43.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Virtual IlluminationVirtual IlluminationShadows, Shading and BlendingShadows, Shading and Blending
![Page 44: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/44.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
StepsSteps– Preprocessing
• Scan 3D object and create model, G• Approximately position projector(s)• Compute pose, P• Compute intensity correction,
– Run time• Render image [ I ]
using model G from pose P
![Page 45: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/45.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
StepsSteps– Preprocessing
• Scan 3D object and create model• Approximately position projector(s)• Compute pose• Compute intensity correction
– Run time• Render image [ I ]• Apply intensity correction for object shape [ ]*[ I ]• Apply feathering for projector overlap, [ ]*[ ] *[ I ]
![Page 46: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/46.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
StepsSteps
– Preprocessing• Scan 3D object and create model• Approximately position projector(s)• Compute pose• Compute intensity correction
– Run time• Render image [ I ]• Intensity correction, [ ] * [ I ]• Feathering, [ ] * [ ] * [ I ]
![Page 47: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/47.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
StepsSteps– Preprocessing
• Scan 3D object and create model, G• Approximately position projector(s)• Compute pose, P
– Find fiducials– Find pixels that illuminate them– Find projector pose
– Run time• Render 3D model G from P , [ I ]• Intensity correction for object shape [ ] * [ I ]• Feathering for projector overlap [ ] * [ ] * [ I ]
![Page 48: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/48.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Moving ObjectsMoving Objects
Moving Surface Moving Projector
Moving Viewer
![Page 49: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/49.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Apparent MotionApparent Motion
Ramesh Raskar, Remo Ziegler, Thomas Willwacher, “Cartoon Dioramas in Motion,” Proc. ACM Symposium on Nonphotorealistic Animation and Rendering (NPAR 2002)
![Page 50: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/50.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Virtual MotionVirtual Motion
![Page 51: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/51.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
ApplicationsApplicationsIndoors, under controlled lightingIndoors, under controlled lighting
– Architectural models• Augment walk-around scaled model of buildings• Project and ‘paint’ surface colors, textures• Lighting, sunlight, seasons• Internal structure, pipes, wiring
– Assembly line• Instructional text, images and procedures
– Entertainment• Live shows, exhibits, demonstrations
![Page 52: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/52.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Virtual Reflectance Virtual Illumination
InteractionVirtual Motion
ShaderLampsShaderLamps
www.ShaderLamps.com
![Page 53: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/53.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Projector-based AugmentationProjector-based Augmentation
www.ShaderLamps.com
Virtual Reflectance Virtual Illumination
InteractionVirtual Motion
![Page 54: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/54.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Desired Virtual Model
© Andrei State
Projected Guidance for Placement
![Page 55: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/55.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
![Page 56: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/56.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Projector-based ARProjector-based AR
Bimber, O., Fröhlich, B., Schmalstieg, D., and Encarnação, L.M. ‘The Virtual Showcase’. IEEE Computer Graphics & Applications, vol. 21, no.6, 2001.
![Page 57: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/57.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Steerable ProjectorSteerable Projector
Pinhanez et al 2003
![Page 58: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/58.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Projection TechniquesProjection Techniques• Projection Screen Geometries
– Planar
– Rectilinear
– Cylindrical
– Spherical
– Irregular
PlanarHomography
Quadric image transfer
Discretized Warping
![Page 59: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/59.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Planar Multi-Projector DisplayPlanar Multi-Projector Display1
1
1
1
2
2 2
2
3
3
3
3
4
44
4
10
Seconds
[Raskar, Jeroen van Baar]
![Page 60: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/60.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Planar projective transfer Planar projective transfer What is homography ?What is homography ?
– Two images of 3D points on a plane are related by a 3x3 matrix
Screen
Camera 1
M
i
jCamera 2
![Page 61: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/61.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
a1 a2 a3
b1 b2 b3
c1 c2 c3
Planar Homography (in 2D)Planar Homography (in 2D)
Two images of 3D points on a plane Related by a 3x3 matrix
Proj 1
A3 x 3
Proj 2
ij
jx
jy
1
j = A3 x 3 i~
=
ix
iy
1
k
jx = (a • i) / (c • i)
jy = (b • i) / (c • i)
![Page 62: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/62.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Planar projective transfer Planar projective transfer (homography)(homography)
– Two images of 3D points on a plane are related by a 3x3 matrix
Screen
Camera 1
M
i
jCamera 2
Defined by 4 or more corresponding pixels
![Page 63: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/63.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Keystone CorrectionKeystone Correction
1. Compute screen to image homography
2. Pre-warp input image
Screen
Projected image
R Raskar
![Page 64: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/64.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Automatic Keystone CorrectionAutomatic Keystone Correctionwith Camera and Tilt Sensorwith Camera and Tilt Sensor
1. Camera and tilt sensor to find projector pose
2. Compute screen to image homography
3. Pre-warp input image
Screen
Projected image
R Raskar
[Raskar and Beardsley01]
![Page 65: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/65.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Planar display surfacePlanar display surfaceUse homography (Use homography (AA3x33x3))
User
Single Projection Matrix !
V
M
i
jA
j = A ii = PT V~
~
PT
j = [A PT] V~
![Page 66: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/66.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Ad-hoc Planar ClusterAd-hoc Planar Cluster(Video)(Video)
Self-contained Units, No centralized controlNo markers or cameras in environmentBeyond the range of single camera
![Page 67: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/67.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Projection TechniquesProjection Techniques• Projection Screen Geometries
– Planar
– Rectilinear
– Cylindrical
– Spherical
– Irregular
PlanarHomography
Quadric image transfer
Discretized Warping
![Page 68: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/68.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Quardic curved shape DisplaysQuardic curved shape Displays
Planetarium
Sim/Viz Center
Raskar, vanBaar, Willwacher, Rao‘Quadric Transfer for Immersive Curved Displays’,
EuroGraphics 2004
![Page 69: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/69.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Curved projective transferCurved projective transfer Quadric classificationQuadric classificationProjectively equivalent to sphere:
Ruled quadrics:
hyperboloids of one sheet
hyperboloid of two sheets
paraboloidsphere ellipsoid
Degenerate ruled quadrics:
cone two planes
![Page 70: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/70.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Parametric Image TransferParametric Image Transfer
X
i
j
Planar Homography Quadric Transfer
X
i j
eiEiT iAj x 33 iAj x 33
![Page 71: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/71.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Overlap on Quadric Overlap on Quadric ScreensScreens
![Page 72: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/72.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
![Page 73: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/73.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
![Page 74: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/74.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Vertex Shader for Quadric Transfer in CgVertex Shader for Quadric Transfer in Cg• vertout main( appin IN, uniform float4x4 modelViewProj,
uniform float4 constColor, uniform float3x3 A, uniform float3x3 E, uniform float3 e) {
• vertout OUT;• float4 m1 = float4(IN.position.x, IN.position.y, IN.position.z, 1.0f );• float4 m, mi ; float3 m2,mp; float scale;• • m = mul( modelViewProj, m1);• m2.x = m.x/m.w; m2.y = m.y/m.w; m2.z = 1;• scale = mul(m2, mul(E,m2));• mp = mul(A,m2) + sqrt(scale)*e;• mi.x = m.w * (mp.x)/(mp.z);• mi.y = m.w * (mp.y)/(mp.z);• mi.zw = m.zw;• OUT.position = mi; • OUT.color0 = IN.color0; // Use the original per-vertex color specified• return OUT;• }
• (Code in Course Notes)
![Page 75: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/75.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Head Tracked Head Tracked Single Pass RenderingSingle Pass Rendering
![Page 76: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/76.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Projection TechniquesProjection Techniques• Projection Screen Geometries
– Planar
– Rectilinear
– Cylindrical
– Spherical
– Irregular
PlanarHomography
Quadric image transfer
Discretized Warping
![Page 77: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/77.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Non-planar Display VideoNon-planar Display Video
![Page 78: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/78.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
What image should be What image should be projected ?projected ?
Projector
V: Virtual 3D PointM:
Projected Point
jUser : T
Screen
?
![Page 79: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/79.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Step I : Step I : Calculate ‘desired’ imageCalculate ‘desired’ image
User
V: Virtual 3D Point
M
iDesired Image
Screen
Projector
![Page 80: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/80.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Step II : Step II : ‘Project’ the desired image from ‘Project’ the desired image from
TT
Screen
iDesired Image
User
M
Projector
![Page 81: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/81.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Step II : Step II : Render this scenario from Render this scenario from PP
Screen
V: Virtual 3D Point
i
j
Projector
Desired Image
User
Projected Image
M
![Page 82: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/82.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented RealityResult: Projecting a Result: Projecting a pre-warped image, so it pre-warped image, so it
looks correctlooks correct
Screen
V: Virtual 3D Point
i
j
Projector
Desired Image
User
Projected Image
M
![Page 83: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/83.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
AR with location-aware RFIDAR with location-aware RFID
![Page 84: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/84.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Ramesh Raskar, Paul Beardsley, Jeroen van Baar, Yao Wang, Paul Dietz, Johnny Lee, Darren Leigh, Thomas Willwacher
Mitsubishi Electric Research Labs (MERL), Cambridge, MA
R F I R F I GG Lamps Lamps : : Interacting with a Self-describing World Interacting with a Self-describing World
via Photosensing Wireless Tags and via Photosensing Wireless Tags and ProjectorsProjectors
![Page 85: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/85.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Radio Frequency Identification Tags Radio Frequency Identification Tags (RFID)(RFID)
microchip
AntennaNo batteries,
Small size,
Cost few cents
![Page 86: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/86.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Warehousing
Routing
Library Baggage
handling
Currency
Livestock tracking
![Page 87: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/87.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Micro Controller
Memory Computer
READER
Micro Controller
Memory
Conventional Passive RFID
![Page 88: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/88.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Tagged Books in a LibraryTagged Books in a Library Id : List of books in RF range
No Precise Location Data Are books in sorted order ?
Which book is upside down ?
![Page 89: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/89.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Prototype TagPrototype TagRF tag +
photosensor
![Page 90: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/90.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
READER
Micro Controller
RF Data
Memory
Conventional RFID
Computer
READER
Micro Controller
RF Data
Light
Memory
Photosensor
Computer
Conventional RF tag
Photo-sensing RF tag
![Page 91: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/91.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
READER
Projector Micro
Controller RF Data
Light
Memory
Photosensor
Computer
![Page 92: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/92.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Find tag location using
handheld Projector
Photosensing
Wireless Tags
Many geometric ops
R F I R F I
R F I DR F I D
Interactive stabilized projection
(Radio Frequency Id & (Radio Frequency Id & GeometryGeometry))GG
Siggraph 2004
![Page 93: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/93.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
AR with Photosensing RFID AR with Photosensing RFID and Handheld Projectorand Handheld Projector
![Page 94: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/94.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
RFID(Radio Frequency Identification)
RFIG(Radio Frequency Id and Geometry)
![Page 95: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/95.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Pattern
MSB
Pattern
MSB-1
Pattern
LSB
Projected Sequential Frames
•Handheld Projector beams binary coded stripes•Tags decode temporal code
![Page 96: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/96.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Pattern
MSB
Pattern
MSB-1
Pattern
LSB
Projected Sequential Frames
•Handheld Projector beams binary coded stripes•Tags decode temporal code
![Page 97: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/97.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Pattern
MSB
Pattern
MSB-1
Pattern
LSB
Projected Sequential Frames
•Handheld Projector beams binary coded stripes•Tags decode temporal code
![Page 98: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/98.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Pattern
MSB
Pattern
MSB-1
Pattern
LSB
Projected Sequential Frames
•Handheld Projector beams binary coded stripes•Tags decode temporal code
![Page 99: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/99.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Pattern
MSB
Pattern
MSB-1
Pattern
LSB
Projected Sequential Frames
•Handheld Projector beams binary coded stripes•Tags decode temporal code
![Page 100: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/100.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Pattern
MSB
Pattern
MSB-1
Pattern
LSB
For each taga. From light sequence, decode x and y
coordinateb. Transmit back to RF reader (Id, x, y)
0 1 1 0 0 X=12
![Page 101: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/101.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Visual feedback of 2D Visual feedback of 2D positionposition
a. Receive via RF {(x1,y1), (x2,y2), …} pixels
b. Illuminate those positions
![Page 102: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/102.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Visual feedback of 2D Visual feedback of 2D positionposition
a. Receive via RF {(x1,y1), (x2,y2), …} pixels
b. Illuminate those positions
![Page 103: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/103.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
3D from 2 Projector Views3D from 2 Projector Views(Structure from Motion)(Structure from Motion)
• Two+ unknown projector views• Correspondence is trivial• Applications
– Detect 3D deformations– Trajectory grouping
![Page 104: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/104.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Change Detection Change Detection without fixed camera, in any lighting conditionwithout fixed camera, in any lighting condition
Compare with new coordinates from a different
view
Record coordinates of tags from one view
Before After
![Page 105: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/105.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Finding OcclusionsFinding Occlusions
![Page 106: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/106.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Laser Guided RobotLaser Guided Robot
![Page 107: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/107.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Texture AdaptationTexture Adaptation
![Page 108: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/108.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Desktop-like InteractionDesktop-like Interaction
Selecting Selecting tagstags
![Page 109: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/109.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Support for handheld Support for handheld projectionprojection
![Page 110: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/110.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Mouse SimulationMouse Simulation
• Cursor follows handheld projector motion• Pre-warped image remains stable
![Page 111: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/111.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Image Quasi-StabilizationImage Quasi-Stabilization
Eliminate hand jitter using inertial Eliminate hand jitter using inertial sensors+camerasensors+camera
![Page 112: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/112.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Absolute StabilizationAbsolute Stabilization
Image stays registered with world Image stays registered with world featuresfeatures
![Page 113: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/113.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Image StabilizationImage Stabilization
![Page 114: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/114.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Interactive ProjectionInteractive Projection
![Page 115: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/115.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Adaptive ProjectionAdaptive Projection
‘‘Copy and Paste’Copy and Paste’Geometric and Photometric compensationGeometric and Photometric compensation
![Page 116: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/116.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Prototype Handheld Prototype Handheld ProjectorProjector
![Page 117: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/117.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Machine ARMachine AR
• AR for cameras and machines
• Face Dome [Debevec 2001]• 4D lighting [MPI, MERL]
![Page 118: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/118.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Robot ‘Laser’ GuidanceRobot ‘Laser’ GuidancePicking and Sorting Tagged ObjectsPicking and Sorting Tagged Objects
![Page 119: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/119.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
AcknowledgementsAcknowledgements• MERL
• Jeroen van Baar, Paul Beardsley, Remo Ziegler, Thomas Willwacher, Srinivas Rao, Cliff Forlines, Paul Dietz, Johnny Lee
• Office of the Future group at UNC Chapel Hill• Greg Welch, Kok-lim Low, Deepak B’padhyay• Henry Fuchs, Herman Towles• Wei-chao Chen
• Mitsubishi Electric, Japan• Yoshihiro Ashizaki, Masatoshi Kameyama,
Masato Ogata, Keiichi Shiotani• Images
• Oliver Bimber, Claudio Pinhanez• Ed Lantz (Spitz Inc.), Marc Pollefeys (UNC Chapel Hill)• Aditi Majumder, Rahul Sukthankar,, Chris Jaynes, Shree Nayar• Apologies
– (Not able to include work by others)• SVR, Absolut, Luciano Peres
![Page 120: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/120.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
•Projector-Camera WorkshopJune 2005
www.PROCAMS.org
•Web Page :
http://raskar.com/Projector/
•Projector mailing list [email protected]
“subscribe projector”
![Page 121: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/121.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented RealityNew Directions in Augmented New Directions in Augmented RealityReality
• Spatially Augmented Reality: – Alternative approach
• Special effects in real world: – Next big graphics challenge
• Location aware RFID: – Converging technologies
www.raskar.com
![Page 122: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/122.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Ramesh RaskarMitsubishi Electric Research Labs (MERL), Cambridge, MA
New Directions in Augmented RealityNew Directions in Augmented Reality
![Page 123: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/123.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Shader LampsShader LampsMotivationMotivation
View-dependent Appearance
![Page 124: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/124.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Shader LampsShader LampsImage based IlluminationImage based Illumination
– Basic Idea• Render images and project on objects• Multiple projectors• View and object dependent color
Raskar, Welch, Low, Bandyopadhyay, “Shader Lamps: Animating Real Objects with Image Based Illumination,” Eurographics Rendering Worksop (EGRW 2001)
![Page 125: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/125.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
![Page 126: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/126.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Virtual MotionVirtual Motion
![Page 127: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/127.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Virtual Reflectance Virtual Illumination
InteractionVirtual Motion
ShaderLampsShaderLamps
www.ShaderLamps.com
![Page 128: Svr Raskar](https://reader037.vdocuments.mx/reader037/viewer/2022102322/53ff352e8d7f72a50a8b4c96/html5/thumbnails/128.jpg)
Mitsubishi Electric Research Labs Raskar SVR04Augmented Reality
Prototype Handheld Prototype Handheld ProjectorProjector