Download - Mobile AR Lecture 8 - AR Panoramas
![Page 1: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/1.jpg)
Researcher: Zi Siang See
Lecture 8: Augmented Reality using High Fidelity
Spherical Panorama with HDRI
Research Presenta,on: Zi Siang See Human Interface Technology Laboratory New Zealand (HIT Lab NZ) Email (Industrial collabora,on): [email protected]
Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Taken from: SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interac,ve Applica,ons
Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality using High Fidelity Spherical Panorama with HDRI. SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interac,ve Applica,ons. hVp://dx.doi.org/10.1145/2818427.2818445 hVp://www.zisiangsee.com/research/zisiangsee_research_p009.pdf
Mobile-Based Augmented Reality Development
![Page 2: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/2.jpg)
Researcher: Zi Siang See
![Page 3: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/3.jpg)
Researcher: Zi Siang See
![Page 4: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/4.jpg)
Researcher: Zi Siang See
Research Presenta?on
1) Introduc?on, Related Work 2) Method and Apparatus 3) Discussion 4) Prac,cal Session
![Page 5: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/5.jpg)
Researcher: Zi Siang See
Figure: AR on Mobile
• Mobile Augmented Reality (AR) mixes a live real-‐world view with virtual interac,ve content on a mobile device.
• One of the key enablers for this is tracking technology, such as computer vision techniques for tracking off pre-‐defined markers or markerless images.
Introduc?on: Augmented Reality (AR) on Mobile Source: google search (keyword “augmented reality”)
![Page 6: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/6.jpg)
Researcher: Zi Siang See
Augmented Reality using High Fidelity Spherical Panorama with HDRI
High fidelity requirements – mobile AR environment recogni?on • Spherical Panorama • High Dynamic Range Imaging (HDRI) • Op,miza,on involves obtaining accurately reproduced spherical panorama with least acquisi,on ,me -‐ AR source tracking content has sufficient luminance by using HDRI having dynamic range extended from single acquired digital nega,ve for each angle in mul,row configura,on.
![Page 7: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/7.jpg)
Researcher: Zi Siang See
Figure: (a) AR using marker. (b) AR using panorama image, markerless experience.
• There have been previous studies on using pre-‐produced panorama images for AR tracking (Arth et al 2011; DiVerdi et al. 2008; Langlotz et al. 2014; Ventura and Höllerer 2013; Wagner et al. 2010).
• Most of related studies describe how the panorama images can be used for AR tracking.
• Therefore, there is a need for specifying the method for high fidelity produc,on of the source panorama images for mobile AR.
Introduc?on: AR using Panorama Image (a) (b)
![Page 8: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/8.jpg)
Researcher: Zi Siang See
Case Study: Corporate Project Requirements
![Page 9: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/9.jpg)
Researcher: Zi Siang See
Townships
Landscape
Augmented /Virtual Other Scope
Construc?on
Panorama Image Reproduc?on, Projects Heavy Requirement & Workload/Quan?ty
Industrial Case Study: Amount, Time, Quality
![Page 10: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/10.jpg)
Researcher: Zi Siang See
Figure: Panorama image reproduc,on process for AR /VR projects.
![Page 11: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/11.jpg)
Researcher: Zi Siang See
![Page 12: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/12.jpg)
Researcher: Zi Siang See
![Page 13: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/13.jpg)
Researcher: Zi Siang See
![Page 14: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/14.jpg)
Researcher: Zi Siang See
Introduc?on: Spherical Panorama Acquisi?on
![Page 15: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/15.jpg)
Researcher: Zi Siang See
Figure: reproduc,on op,ons available for fast panorama image acquisi,on
Introduc?on: Spherical Panorama Acquisi?on
LadyBug GoPro Rig Mobile (Handheld)
![Page 16: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/16.jpg)
Researcher: Zi Siang See
Figure: reproduc,on op,ons available for fast panorama image acquisi,on
Introduc?on: Spherical Panorama Acquisi?on
LadyBug GoPro Rig Mobile (Handheld)
Parallax error Shake /Blur
Resolu?on Limited Dynamic Range
Inconsistent ligh?ng distribu?on for mul?ple angles
Nadir difficulty
Ghos?ng effect (HDRI)
Inconsistent white balance for mul?ple angles
Time management
![Page 17: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/17.jpg)
Researcher: Zi Siang See
• Acquire wider panoramic visibility (Benosman and Kang, 2001) • Major difficulty for spherical panorama reproduc,on may occur, this can
include limited dynamic range reproduced and parallax error (Brown and Lowe, 2006; Felinto et al, 2012)
Figure: panoramic imagery with visual abnormality -‐ loca,on in Sannomiya Kobe, Japan.
Introduc?on: Spherical Panorama Acquisi?on Camera shake
Limited dynamic range Visual abnormality
Parallax error
![Page 18: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/18.jpg)
Researcher: Zi Siang See
Figure: combined luminance (EV) range being covered by HDRI outcome.
Darker images (luminance) Brighter images
(luminance)
Introduc?on: HDRI (for mul?ple angle images)
• High Dynamic Range Imaging (HDRI) in photography technique has a goal to increase the dynamic range resul,ng an improved luminance reproduced in shadow and highlight areas (Reinhard et al, 2010; Debevec and Malik, 1997; Fairchild, 2007), usually by combining mul,ple exposures consist of darker and brighter images.
![Page 19: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/19.jpg)
Researcher: Zi Siang See
• Obstacles and issues of HDRI that involves mul,ple exposures can include lengthy acquisi,on (for each angle), ghos,ng effect and misalignment issue due to inconsistent acquisi,ons (Reinhard et al, 2010)
Figure: HDRI with visual abnormali,es, ghos,ng.
Introduc?on: HDRI (for mul?ple angle images)
Mul,ple exposures for HDRI Single exposure (LDR) Visual abnormality
![Page 20: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/20.jpg)
Researcher: Zi Siang See
• Spherical panorama can be used for AR as source tracking image. • Various studies (Felinto et al 2012, Gawthrop 2007, Brown and Lowe
2006; Schulz, 2012) have shown that photographic combina,on for HDRI and spherical panorama can be possible, however condi,onally tolera,ng with obstacles and issues.
Spherical Panorama (Mul?row) HDRI requires mul?ple exposures
-‐4, -‐3, -‐2, -‐1, 0, +1, +2, +3, +4 (EV)
Difficulty: Spherical Panorama and HDRI for AR?
![Page 21: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/21.jpg)
Researcher: Zi Siang See
Figure: example of source tracking images or scenarios for AR
• For AR, Lieberknecht et al. [2009] has shown that it can be difficult to create source tracking images, or synthe,c images that reproduce the real effects of real-‐world phenomena such as ligh,ng, noise, mo,on blur, discre,za,on, blooming or limited color depths during real image acquisi,on.
• Image elements that influence real-‐,me tracking results include texture richness, the texture repeatability of the objects to be tracked, the camera mo,on and speed, the changes of the object scale, and varia,ons of the ligh,ng condi,ons over ,me.
Difficulty: Source Tracking Images for AR Source: google search (keyword “augmented reality”)
![Page 22: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/22.jpg)
Researcher: Zi Siang See
Research Presenta?on
1) Introduc,on, Related Work 2) Method and Apparatus 3) Discussion 4) Practical Session
![Page 23: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/23.jpg)
Researcher: Zi Siang See
.. no photographic process is complete un?l the final images can be proudly displayed (Phase One, 2015).
![Page 24: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/24.jpg)
Researcher: Zi Siang See
Method: Spherical Panorama
Addi,onal 3 nadir acquisi,on
Horizontal angles
Zenith
Nadir image acquired without including the base of the camera. Stable moun,ng is required for avoiding driring, shake and parallax error.
Nadir without obstacle, allow sufficient space.
![Page 25: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/25.jpg)
Researcher: Zi Siang See
Method: High Dynamic Range Image (HDRI)
Pixe
l Val
ue
Sequences of extendable dynamic range from single RAW (photographic recording instrument Nikon D3x)
• Extending Dynamic Range from single acquired Digital Nega,ve (RAW). • Consistent to the idea brought up by Debevec and Malik (1997) about
extending dynamic range can be possible from conven,onal single film processing -‐ mul,ple scanning method.
![Page 26: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/26.jpg)
Researcher: Zi Siang See
Method: High Dynamic Range Image (HDRI)
• -‐2 EV and +2 EV, resul,ng an addi,onal increment from 8.5EV to 12.5EV. • The finding in the study about na,ve EV of dynamic range reproduc,on
was observed to have similar approximated usable exposure values with the RAW produced by ISO100 according to the test by Rehm (2009).
Pixe
l Val
ue
Sequences of extendable dynamic range from single RAW (photographic recording instrument Nikon D3x)
![Page 27: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/27.jpg)
Researcher: Zi Siang See
Method: High Dynamic Range Image (HDRI)
Figure: an image from mul,ple angle before applying HDRI.
without
![Page 28: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/28.jpg)
Researcher: Zi Siang See
Figure: proposed mul,ple angles HDRI for high fidelity AR. (HDRI method that requires a single acquisi,on that extends the dynamic range from a digital nega,ve)
Method: High Dynamic Range Image (HDRI) with HDRI
![Page 29: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/29.jpg)
Researcher: Zi Siang See
Source Image for AR: High Fidelity Spherical Panorama with HDRI
mul?ple angles facilitated by HDRI required for producing the spherical panorama (least or near-‐zero visual abnormality)
![Page 30: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/30.jpg)
Researcher: Zi Siang See
Mul?ple angles HDRI of mul?row configura?on (from the proposed method) with reduced error or mistakes that requires rec?fica?ons, including near-‐perfect nadir angle reproduc?on.
Nadir Angle: Reproduc?on from the original loca?on-‐based scene, minimal post-‐produc?on correc?on required
cubic projec?on
Source Image for AR: High Fidelity Spherical Panorama with HDRI
![Page 31: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/31.jpg)
Researcher: Zi Siang See
mul?ple angles facilitated by HDRI required for producing the spherical panorama (least or zero visual abnormality)
Source Image for AR: High Fidelity Spherical Panorama with HDRI
![Page 32: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/32.jpg)
Researcher: Zi Siang See
HDRI ghos?ng effect using mul?ple exposures method.
HDRI reproduc?on from single acquisi?on, digital nega?ve (RAW) visual abnormality is eliminated or avoided (for mul?ple angles required for spherical panorama)
Source Image for AR: High Fidelity Spherical Panorama with HDRI
![Page 33: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/33.jpg)
Researcher: Zi Siang See
mul?ple angles facilitated by HDRI required for producing the spherical panorama (least or zero visual abnormality)
Source Image for AR: High Fidelity Spherical Panorama with HDRI
![Page 34: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/34.jpg)
Researcher: Zi Siang See
(a) LDR photo image (b) HDRI from single source acquisi,on (c) HDRI from mul,ple exposures
HDRI Panorama with reduced overall photographic produc?on processes and ?me.
Source Image for AR: High Fidelity Spherical Panorama with HDRI
![Page 35: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/35.jpg)
Researcher: Zi Siang See
Research Presenta?on
1) Introduc,on, Related Work 2) Method and Apparatus 3) Discussion 4) Practical Session
![Page 36: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/36.jpg)
Researcher: Zi Siang See
Augmented Reality using High Fidelity Spherical Panorama with HDRI (Authoring process for AR environment recogni?on)
Spherical Panorama facilitated with HDRI (Cubic projec,on)
Cropped /selec,ve cubic facade working with image-‐based AR
Figure: augmenta,on can be assigned on any cubic facades, speculated to work on most of the AR browsers /workflow.
![Page 37: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/37.jpg)
Researcher: Zi Siang See
AR for Mobile and Wearable
Figure: handheld experience of markerless AR.
![Page 38: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/38.jpg)
Researcher: Zi Siang See
AR for Mobile and Wearable
Figure: handheld experience of markerless AR (video).
![Page 39: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/39.jpg)
Researcher: Zi Siang See
Zeiss VR One
• Carl Zeiss announced VR One, a virtual reality headset for use with a smartphone. It is a viewer designed to work with phones between 4.7 and 5.2 inches (Zeiss, 2014).
• Options for HMD using mobile device as computing module are becoming available.
![Page 40: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/40.jpg)
Researcher: Zi Siang See
Samsung Gear VR
• Samsung has launched Gear VR in October 2014, co-developed with Oculus. Gear VR can house specific mobile device (Note 4) for providing virtual reality and augmented user experience (Samsung, 2014)
![Page 41: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/41.jpg)
Researcher: Zi Siang See
AR for Mobile and Wearable
Figure: stereo image of AR – adaptable for HMD such as Oculus or Google Cardboard.
![Page 42: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/42.jpg)
Researcher: Zi Siang See
Source Image for AR: High Fidelity Spherical Panorama with HDRI
Figure: HDRI spherical panorama sampled at Sannomiya Sta,on, Japan (01 Nov 2015).
![Page 43: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/43.jpg)
Researcher: Zi Siang See
AR for Mobile and Wearable
Figure: stereo image of AR – adaptable for HMD such as Oculus or Google Cardboard.
![Page 44: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/44.jpg)
Researcher: Zi Siang See
AR for Mobile and Wearable
Figure: stereo image of AR – adaptable for HMD such as Oculus or Google Cardboard.
![Page 45: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/45.jpg)
Researcher: Zi Siang See
Discussion
Poten?al obstacles and issues (before op?miza?on): • Parallax error (especially during hand-‐held) • Unstable image acquisi,on, driring mo,on (hand-‐held) • Nadir angle difficulty • Compromised geometrical registra,on in image • Low dynamic range image (LDRI) • High dynamic range requires mul,ple exposures • Long acquisi,on ,me if mul,ple exposures • High dynamic range ghos,ng with moving objects • Mul,ple exposures misalignment • Inconsistent ligh,ng distribu,on for mul,ple angles
Benefit of using our method and apparatus: • Free from parallax error • Stable image acquisi,on • Nadir angle with near-‐error-‐free authen,city • Accurate geometrical registra,on in image • High dynamic range image (HDRI) • High dynamic range from single RAW acquisi,on • High dynamic range with least acquisi,on ,me • High dynamic range with no ghos,ng • High dynamic range with perfect alignment • Consistent high dynamic range for mul,ple angles
![Page 46: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/46.jpg)
Researcher: Zi Siang See
Discussion
Poten?al obstacles and issues (before op?miza?on): • Parallax error (especially during hand-‐held) • Unstable image acquisi,on, driring mo,on (hand-‐held) • Nadir angle difficulty • Compromised geometrical registra,on in image • Low dynamic range image (LDRI) • High dynamic range requires mul,ple exposures • Long acquisi,on ,me if mul,ple exposures • High dynamic range ghos,ng with moving objects • Mul,ple exposures misalignment • Inconsistent ligh,ng distribu,on for mul,ple angles
Benefit of using our method and apparatus: • Free from parallax error • Stable image acquisi,on • Nadir angle with near-‐error-‐free authen,city • Accurate geometrical registra,on in image • High dynamic range image (HDRI) • High dynamic range from single RAW acquisi,on • High dynamic range with least acquisi,on ,me • High dynamic range with no ghos,ng • High dynamic range with perfect alignment • Consistent high dynamic range for mul,ple angles
![Page 47: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/47.jpg)
Researcher: Zi Siang See
Conclusion
• Markerless AR. • High fidelity spherical panorama with HDRI can provide a near-‐error-‐free
and dynamic range enhanced source of image-‐based AR tracking content. • Source tracking content is reproduced with liVle distor,on, producing a
result very similar to the original scene condi,on. • Without using extra sensors such as a GPS. • Allows an AR experience to be delivered on a mobile device with a lower
processing requirement -‐ adaptable in other wearable system or devices.
![Page 48: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/48.jpg)
Researcher: Zi Siang See
Implica?ons
• Android-‐based mobile with RAW capturing capability. • Approach adapta,on in VR/AR workflow for mobile applica,ons.
![Page 49: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/49.jpg)
Researcher: Zi Siang See
Future Study
• In the future we will conduct more extensive evalua,on studies to compare the tracking accuracy with the systems using our HDRI panorama images to other tradi,onal approaches.
• We will also explore other solu,ons suitable for HDRI panorama video and hybrid approaches that combine panorama image tracking and sensor input.
![Page 50: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/50.jpg)
Researcher: Zi Siang See
Thank you
(Next) Practical and Authoring
Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality using High Fidelity Spherical Panorama with HDRI. SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interac,ve Applica,ons. hVp://dx.doi.org/10.1145/2818427.2818445 hVp://www.zisiangsee.com/research/zisiangsee_research_p009.pdf
![Page 51: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/51.jpg)
Researcher: Zi Siang See
Research Presenta?on
1) Introduc,on, Related Work 2) Method and Apparatus 3) Discussion 4) Practical Session
![Page 52: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/52.jpg)
Researcher: Zi Siang See
Research Presenta?on
1) Introduc,on, Related Work 2) Method and Apparatus 3) Discussion 4) Practical Session
HDRI Panorama reproduc,on Interac,ve panorama output AR authoring Tes,ng and simula,on Real-‐world tes,ng
![Page 53: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/53.jpg)
Researcher: Zi Siang See
Research Presenta?on
1) Introduc,on, Related Work 2) Method and Apparatus 3) Discussion 4) Practical Session
HDRI Panorama reproduc,on Interac,ve panorama output AR authoring Tes,ng and simula,on Real-‐world tes,ng
![Page 54: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/54.jpg)
Researcher: Zi Siang See
![Page 55: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/55.jpg)
Researcher: Zi Siang See
Interac?ve Panorama Pano2vr (or other tools)
![Page 56: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/56.jpg)
Researcher: Zi Siang See
Source Image for AR: High Fidelity Spherical Panorama with HDRI
Figure: HDRI spherical panorama sampled at Sannomiya Sta,on, Japan (01 Nov 2015).
![Page 57: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/57.jpg)
Researcher: Zi Siang See
Augmented Reality using High Fidelity Spherical Panorama with HDRI (Authoring process for AR environment recogni?on)
Spherical Panorama facilitated with HDRI (Cubic projec,on)
Cropped /selec,ve cubic facade working with image-‐based AR
Figure: augmenta,on can be assigned on any cubic facades, speculated to work on most of the AR browsers /workflow.
![Page 58: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/58.jpg)
Researcher: Zi Siang See
Wikitude Studio hcp://studio.wikitude.com
![Page 59: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/59.jpg)
Researcher: Zi Siang See
Wikitude Studio hcp://studio.wikitude.com
![Page 60: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/60.jpg)
Researcher: Zi Siang See
Wikitude Studio hcp://studio.wikitude.com
![Page 61: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/61.jpg)
Researcher: Zi Siang See
Layar Creator hcp://www.layar.com
![Page 62: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/62.jpg)
Researcher: Zi Siang See
Layar Creator hcp://www.layar.com
![Page 63: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/63.jpg)
Researcher: Zi Siang See
Layar Creator hcp://www.layar.com
![Page 64: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/64.jpg)
Researcher: Zi Siang See
Load Panorama and Try AR for simula?on (also works on actual loca?on)
![Page 65: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/65.jpg)
Researcher: Zi Siang See
Thank you Research Discussion
collaboration /contact www.zisiangsee.com
Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality using High Fidelity Spherical Panorama with HDRI. SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interac,ve Applica,ons. hVp://dx.doi.org/10.1145/2818427.2818445 hVp://www.zisiangsee.com/research/zisiangsee_research_p009.pdf
![Page 66: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/66.jpg)
Researcher: Zi Siang See
Additional
![Page 67: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/67.jpg)
Researcher: Zi Siang See
Method: High Dynamic Range Image (HDRI)
Figure: early aVempt for HDRI computa,on from the source of single acquired digital nega,ve, RAW.
HDRI Computation Input: Source image in digital negative RAW format (e.g. NEF, DNG) Output: The HDRI is reproduced from a single acquired RAW image having its dynamic range extended from RAW
1. Three images, I, I+, I- are processed from the single source of RAW.
2. I+ and I- are respectively altered by +2EV and -2EV where dynamic range are extended. That is, the luminance lx of each pixel x is recalculated as (lx)new
= 2m (lx)old, where m is the change in EV.
3. A new image, I’+, is constructed out of I+, by preserving only the highlight regions of I+. Similarly, a new image I’- is constructed out of I-, by preserving only the non-highlight regions of I-.
4. I’+ is then overlaid on top of I’- with 50% opacity. The results of the two are then combined to form the final image I’.
![Page 68: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/68.jpg)
Researcher: Zi Siang See
mul?ple angles of mul?row configura?on, covering horizontal, zenith and nadir angles. (However having nadir angle difficulty, parallax error, low dynamic range)
mul?ple angle images
Introduc?on: Spherical Panorama Acquisi?on
![Page 69: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/69.jpg)
Researcher: Zi Siang See
1/8 sec, f8 1/15 sec, f8 1/30 sec, f8 1/60 sec, f8 1/125 sec, f8 1/250 sec, f8 1/500 sec, f8 1/1000 sec, f8 1/2000 sec, f8
Scenario 1 (high brightness)
HDRI Acquisi?on: Ligh?ng Scenarios
Mul?ple exposures approach Our HDRI approach (dynamic range extended from single acquired digital nega?ve, RAW)
![Page 70: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/70.jpg)
Researcher: Zi Siang See
1/8 sec, f8 1/15 sec, f8 1/30 sec, f8 1/60 sec, f8 1/125 sec, f8 1/250 sec, f8 1/500 sec, f8 1/1000 sec, f8 1/2000 sec, f8
Scenario 1 (high brightness)
HDRI Acquisi?on: Ligh?ng Scenarios
Mul?ple exposures approach Our HDRI approach (dynamic range extended from single acquired digital nega?ve, RAW)
Single Acquisi,on (RAW)
Process: Extending Dynamic Range
Extended +2EV Original
0EV Extended -‐2EV
HDRI from single source
HDRI that requires single acquisi,on HDRI that requires mul,ple exposures
-‐4, -‐3, -‐2, -‐1, 0, +1, +2, +3, +4 (EV)
![Page 71: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/71.jpg)
Researcher: Zi Siang See
1/2 sec, f8 1/4 sec, f8 1/8 sec, f8 1/15 sec, f8 1/30 sec, f8 1/60 sec, f8 1/125 sec, f8 1/250 sec, f8 1/500 sec, f8
Scenario 2 (moderate brightness)
Mul?ple exposures approach Our HDRI approach (dynamic range extended from single acquired digital nega?ve, RAW)
HDRI Acquisi?on: Ligh?ng Scenarios
![Page 72: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/72.jpg)
Researcher: Zi Siang See
4 sec, f8 2 sec, f8 1 sec, f8 1/2 sec, f8 1/4 sec, f8 1/8 sec, f8 1/15 sec, f8 1/30 sec, f8 1/60 sec, f8
Scenario 3 (low light)
Mul?ple exposures approach Our HDRI approach (dynamic range extended from single acquired digital nega?ve, RAW)
HDRI Acquisi?on: Ligh?ng Scenarios
![Page 73: Mobile AR Lecture 8 - AR Panoramas](https://reader033.vdocuments.mx/reader033/viewer/2022052606/58f04afa1a28ab49558b4591/html5/thumbnails/73.jpg)
Researcher: Zi Siang See
Additional (extended)