augmented reality and virtual reality: research advances in creative industry
TRANSCRIPT
Researcher: Zi Siang See
Augmented Reality and Virtual Reality:
Research Advances in Creative Industries
Zi Siang See
Research: [email protected]
Industrial collaboration: [email protected]
Research Workshop Seminar
Faculty of Arts, Sunway University
Research Workshop Seminar: Zi Siang See
Centre for Research Creation in Digital Media
22 June 2016 – University of Reading
Research &
collaboration
Researcher: Zi Siang See
Workshop
1) Introduction
2) Demonstration
3) Work-in-progress
4) Discussion
Research &
collaboration
Researcher: Zi Siang See
Professional Background:
Industrial involvements since 2003/04
Researcher: Zi Siang SeeVirtual Reality 360 (VR360) | Presenter: Zi Siang See
Research Interest
• Virtual Reality 360 (VR360)
• Augmented Reality
• Interactive Multimedia
• Human Computer Interaction
• Computational Photography
• Imography Research
Research and Specialization
Researcher: Zi Siang See
Researcher: Zi Siang See
Work: McDonald’s Malaysia
Creative: NagaDDB
Operation: Quality Ingredient Story
Creative Imaging: Reina Imaging
Researcher: Zi Siang SeeDigital Imaging (DI) image manipulation for advertising usage, high technical finishing | Presenter: Zi Siang See
Researcher: Zi Siang SeeDigital Imaging (D() for Advertising Visuals| Presenter: Zi Siang See
Researcher: Zi Siang SeeAdvertising visual | Presenter: Zi Siang See
Researcher: Zi Siang SeeDigital Imaging (DI) for Advertising Visuals| Presenter: Zi Siang See
Researcher: Zi Siang See
Researcher: Zi Siang See
Researcher: Zi Siang SeeObject Virtual Reality (Object VR) | Presenter: Zi Siang See
Researcher: Zi Siang See
Discover MINIEnter at your own risk of losing control
Researcher: Zi Siang See
www.vsmm2016.org
Researcher: Zi Siang See
“Creativity is not an option, it’s an absolute necessity”
Sir Ken Robinson 2014
Researcher: Zi Siang See
VISIONThe Centre seeks to be a cornerstone for transformative digital media research, scholarship and innovation to advance Sunway University onto the global new media-scape.
MISSIONTo facilitate synergy between the arts and sciences, cultivating an innovative media research-creation environment.
To create significant collaborations with key local and international centresand labs to catalyze new partnerships.
To motivate a new generation of students and researchers in the domain of creative digital media (CDM) merging core attributes of the arts, sciences, humanities, media, and technology.
Vision & Mission
Researcher: Zi Siang See
Centre for Research-Creation in Digital Media
Researcher: Zi Siang See
UC Berkeley
Collaboration
Researcher: Zi Siang See
The Project
Founding Projects
Researcher: Zi Siang See
Figure: VSMM2016 also features the system by Prof Sarah Kenderdine
Researcher: Zi Siang See
Introduction
Multimedia (Combination of Elements)
• Text
• Image
• Sound
• Video
• Animation
Industries
• Creative and advertising
• Corporate communications visuals
• Architectural imaging
• Cultural heritage preservation
• Scientific achieve
Researcher: Zi Siang See
Multimedia (Combination of Elements)
• Text
• Image
• Sound
• Video
• Animation
• Touch
• Scent
• Taste
• Sight
Introduction
Trend in Industries
• Mixed and Augmented Reality
• Human Computer Interaction
Industries
• Creative and advertising
• Corporate communications visuals
• Architectural imaging
• Cultural heritage preservation
• Scientific achieve
Researcher: Zi Siang See
Researcher: Zi Siang See
Researcher: Zi Siang See
Figure: a vision about 2000 imagined in 1910, through 24 illustrations by Villemard.
Researcher: Zi Siang See
Figure: a vision about 2000 imagined in 1910, through 24 illustrations by Villemard.
Researcher: Zi Siang See
Figure: Sensorama (Morton L Heiling, 1962)
Researcher: Zi Siang See
Major changes in computing platforms
Desk /Table Pocket Palm /Hand Head /Eye Ubiquitous
Researcher: Zi Siang See
Figure: augmented reality, key examples from search engine
Mixed, Augmented & Virtual Reality
Researcher: Zi Siang See
Virtual Reality
Figure: NASA, 1988 Virtual Environment System
Researcher: Zi Siang See
Development
Figure: Super cockpit project, Tom Furness.
Researcher: Zi Siang See
Near-eye-display and fully augmented interactive display experience
How far can we go?
Researcher: Zi Siang See
Augmented Reality
Figure: augmented reality experience via google glass project
Researcher: Zi Siang See
Augmented Reality
• Simulation of interface at www.glasssim.com
• Exploration in modern wearable interface (fashionable, UX practical)
Researcher: Zi Siang See
Google Glass
• Google Glass, developer version, 2013
• Can be configured on user’s customized or personal spectacles.
Researcher: Zi Siang See
Recon Jet
• Available 2014?
• Sporty design
• Cost “friendly”
• Android compatible
• Smartphone connectivity
Researcher: Zi Siang See
Figure: potential areas, google glass project
Researcher: Zi Siang See
Augmented Reality
• Augmented reality (AR) permits computer aided visualization that works with
human interaction (Billinghurst, 2012).
• Billinghurst, M. (2011). The Future of Augmented Reality in Our Everyday Life.
• Damian, I., Kistler, F., Obaid, M., Bueling, R., Billinghurst, M. and Andre, E.
(2013) Motion Capturing Empowered Interaction with a Virtual Agent in an
Augmented Reality Environment.
Figure: Human Interface Technology Laboratory (HITLab), University of Canterbury, NZ.
Researcher: Zi Siang See
Mixed Reality
• Human Pacman: a mobile wide-area entertainment system based on physical,
social, and ubiquitous computing. (Cheok et al., 2004)
• Digital Taste Interface project, Electronic taste stimulation (Ranasinghe et al.,
2011)
• Mediating intimacy in long-distance relationships using kiss messaging
(Saadatian et al., 2013)
Figure: Mixed Reality Lab, the future of internet.
Researcher: Zi Siang See
Introduction: Mixed & Augmented Reality
• Displays, tracers, and AR systems in general need to become more accurate,
lighter, cheaper, and less power consuming (Azuma, 2001).
• Augmented reality (AR) is related to the concept of virtual reality (VR); VR
attempts to create an artificial world that a person can experience and explore
interactively, predominantly through his or her sense of vision, but also via
audio, tactile, and other forms of feedback (Höllerer and Feiner, 2004)
Figure: wearable computing, Steve Mann.
Researcher: Zi Siang See
Possibilities
• Human Pacman: a mobbile wide-area entertainment system based on physical,
social, and ubquitos computing. (Cheok et al., 2004)
• Digital Taste Interface project, Digital taste and smell communication
(Ranasinghe et al., 2011)
• Digital Smell and Taste Communication, City University London.
Researcher: Zi Siang See
ExperimentAugmented reality medical laboratory
(example)
http://code.wikitude.com/medicalmurmurs
Hearable AR
Researcher: Zi Siang See
Augmented Reality: Various Approaches
• Various marker-based or markerless AR tracking approaches has
been discussed (Billinghurst, 2012; Cheok et al., 2004; Lieberknecht
et al., 2009)
• Image-based AR tracking (Lieberknecht et al., 2009)
• CAD /3D object feature tracking (Metaio, 2014)
• Face tracking (Metaio, 2014)
Researcher: Zi Siang See
Augmented Reality using High Fidelity Spherical Panorama with HDRI(Authoring process for AR environment recognition)
Spherical Panorama facilitated with HDRI(Cubic projection)
Cropped /selective cubic facadeworking with image-based AR
Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality using High Fidelity Spherical Panorama with HDRI. SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interactive Applications.http://dx.doi.org/10.1145/2818427.2818445http://www.zisiangsee.com/research/zisiangsee_research_p009.pdf
Researcher: Zi Siang See
AR for Mobile and Wearable
Figure: handheld experience of markerless AR.
Researcher: Zi Siang See
AR for Mobile and Wearable
Figure: handheld experience of markerless AR (video).
Researcher: Zi Siang See
AR for Mobile and Wearable
Figure: stereo image of AR – adaptable for HMD such as Oculus or Google Cardboard.
Researcher: Zi Siang See
Google Cardboard
• Carl Zeiss just announced VR One, a virtual reality headset for use with a
smartphone. It is a viewer designed to work with phones between 4.7 and 5.2
inches (Zeiss, 2014).
Researcher: Zi Siang See
Google Cardboard (v1)
• Carl Zeiss just announced VR One, a virtual reality headset for use with a
smartphone. It is a viewer designed to work with phones between 4.7 and 5.2
inches (Zeiss, 2014).
Programmable
NFC Tag
Low cost resin
/plastic lens
Mobile device as
computing module
Endless configurable size
and shapes (and materials)
Magnetometer
Researcher: Zi Siang See
Google Cardboard (v2)
more reading at http://www.gizmag.com/google-cardboard-2-review-initial/37777/
Presses the
screen
Figure: google cardboard v2 supports larger phone.
Researcher: Zi Siang See
What do I see?
Researcher: Zi Siang See
Samsung Gear VR
• Samsung has launched Gear VR in October 2014, co-developed with Oculus.
Gear VR can house specific mobile device (Note 4) for providing virtual reality
and augmented user experience (Samsung, 2014)
• Control interface included
Researcher: Zi Siang See
Figure: stereo image of AR – adaptable for HMD such as Oculus or Google Cardboard.
Samsung Gear VR
• VR /AR for social media
• UX prototyping
Researcher: Zi Siang See
Figure: able to produce 360 images or videos, for HMD and other interface usage.
Gear 360
Researcher: Zi Siang See
Oculus RiftFigure: Oculus, acquired by Facebook
Researcher: Zi Siang See
Android VRFigure: Android VR collaborating with other developers
Google has announced eight hardware partners that will make Daydream-ready phones, including Samsung, HTC, LG, Xiaomi, Huawei, ZTE, Asus, and Alcatel.
Researcher: Zi Siang See
MetaioFigure: Metaio, acquired by Apple
Metaio was one of the most technologically-ambitious AR organization, acquired by Apple in 2015
Researcher: Zi Siang See
Media Companies’ Interest
Corporate media investment into AR/VR statup - infographic.
Researcher: Zi Siang See
Researcher: Zi Siang See
Figure: stereo image of AR – adaptable for HMD such as Oculus or Google Cardboard.
User Experience
Researcher: Zi Siang See
Figure: European researchers used 3D-modeling and Instagram to recreate Mosul Museum destroyed in civil war.
Researcher: Zi Siang See
User Experience
Researcher: Zi Siang See
User Experience
Researcher: Zi Siang See
User Experience (Concept)
• Concept video produced by Infinity AR, 2014.
• Interface design, UX, daily applications.
Hyper-reality by Keiichi Matsuda
Researcher: Zi Siang See
Researcher: Zi Siang See
Mixed, Virtual, Augmented Reality
• Ikea AR
Researcher: Zi Siang See
Mixed, Virtual, Augmented Reality
• McDonald’s McMission AR
Researcher: Zi Siang See
Mixed, Virtual, Augmented Reality
• Penguins NAVI
Researcher: Zi Siang See
Mixed, Virtual, Augmented Reality
• Octagon 4D (Animal series)
Researcher: Zi Siang See
Mixed, Virtual, Augmented Reality
• Octagon 4D (Octaland series)
Researcher: Zi Siang See
Mixed, Virtual, Augmented Reality
• Concept video produced by Infinity AR, 2014.
• Interface design, UX, daily applications.
Researcher: Zi Siang See
Archiving our world in 3D
• LIDAR 3D Scanning
• Characteristics: trained users, costly, precision CAD content.
Researcher: Zi Siang See
Mixed, Virtual, Augmented Reality
• Quiver
• Physical involvement during creation process
• Visual prototyping /conceptualizing
Researcher: Zi Siang See
Gartner’s Hype Cycle
Researcher: Zi Siang See
Gartner’s Hype Cycle
Researcher: Zi Siang See
Definitions
Virtual Reality
• Virtual Reality (VR) which can be referred to as immersive
multimedia or computer-simulated life, replicates an environment
that simulates physical presence in places in the real world or
imagined worlds and lets the user interact in that world. Virtual
reality artificially creates sensory experiences, which can even
include sight, hearing, touch, and smell.
Augmented Reality
• Augmented Reality (AR) mixes a live real-world view with virtual
interactive content on a mobile or wearable device. One of the key
enablers for this is tracking technology, such as computer vision
techniques for tracking off pre-defined markers or markerless
images.
Researcher: Zi Siang See
Definitions
Virtual Reality
• Virtual Reality (VR) which can be referred to as immersive
multimedia or computer-simulated life, replicates an environment
that simulates physical presence in places in the real world or
imagined worlds and lets the user interact in that world. Virtual
reality artificially creates sensory experiences, which can even
include sight, hearing, touch, and smell.
Augmented Reality
• Augmented Reality (AR) mixes a live real-world view with virtual
interactive content on a mobile or wearable device. One of the key
enablers for this is tracking technology, such as computer vision
techniques for tracking off pre-defined markers or markerless
images.
Researcher: Zi Siang See
VR AR
Source and reinterpreted from http://www.augment.com/blog/which-headset-is-right-for-you/
Researcher: Zi Siang See
Research Possibility
Explore into
• Method and apparatus for augmented reality
• Method and apparatus for virtual reality
• Method and apparatus for mixed reality
• Ubiquitous computing
• Obstacles and issues
• Technology management
• Education technology
• Engineering and concept design
• Collaboration /Multi-disciplinary
• User interface (UI) design
• User experience (UX)
Researcher: Zi Siang See
Everything is achievable
through technology
Researcher: Zi Siang See
Research Collaboration & Publications
• Lee Xia Sheng, Chung Pui Yan and Zi Siang See (2016) Irregular Shaped Building Design Optimization with Building Information Modeling.Proceeding in 4th International Building Control Conference 2016, Universiti Malaya. [indexed in SCOPUS]
• Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality Using High Fidelity Spherical Panorama with HDRI. SIGGRAPH Asia2015 Symposium On Mobile Graphics And Interactive Applications, Kobe, Japan.
• Zi Siang See, Mark Billinghurst, Adrian David Cheok (2015) Augmented Reality Using High Fidelity Spherical Panorama with HDRI – Demonstration.SIGGRAPH Asia 2015 Symposium On Mobile Graphics And Interactive Applications, Kobe, Japan.
• Zi Siang See, Adrian David Cheok. (2015) Virtual Reality 360 Interactive Panorama Reproduction Obstacle and Issue. Virtual Reality, Volume 19, Issue2, pp 71-81, Springer. [indexed by ISI, SCOPUS]
• Lee Xia Sheng, Lee Ching Hua, Cheah Shyn Yi and Zi Siang See (2014) Architectural Transformation of a Night Market Stall. International Conferenceon Innovation and Technology for Sustainable Built Environment 2 2014, Universiti Teknologi MARA (UiTM) Perak.
• Zi Siang See, Khairul Hazrin Hashim, Harold Thwaites, Lee Xia Sheng, Ooi Wooi Har. (2012). Retrieving Extended High Dynamic Range From DigitalNegative Image - An Experiment On Architectural Photo Imaging. World Academy of Science, Engineering and Technology, Vol 6(8) August 2012,pp. 1833-1841. [indexed by EBSCO]
• Zi Siang See, Lee Xia Sheng. (2011). Non-bias Architectural Image Archive using High Dynamic Range Approach. 2nd International Conference onResearch and Innovation in Information Systems 2011 (ICRIIS’11) [CD Proceedings], Kuala Lumpur, Malaysia, 23-24 November. [indexed by SCOPUS,IEEE Xplorer]
• Zi Siang See. (2011). Non-bias Scientific Imaging Archive for Urban Morphology Research. Exchanging Views on How to Sustain Urban and RuralEnvironments in Aspects of Technology, Social and Ecology in Green Innovation Research Group (GIRG) [CD-ROM], Universiti Teknologi Malaysia(UTM), Malaysia.
Researcher: Zi Siang See
Thank you
Research Discussion
research collaboration
www.zisiangsee.com
Researcher: Zi Siang See
Facebook: Zi Siang See
research collaboration
www.zisiangsee.com