VR & AR research @ SINTEF

Costas Boletsis

Post-doc Research Fellow

Overview

• AR/VR general info

• Applications of AR/VR

• AR/VR work at SINTEF

• Questions/discussion

VR vs. AR

Present of VR

Oculus crowdfunds 2.4M, promises a $300 VR headset

2012 OCTOBER

2014 MARCH

Facebook acquires Oculus for reported $2B

Google releases Cardboard, a Do-It-Yourself VR headset

2014 JUNE

2013 MARCH

Oculus releases Oculus Rift Development Kit 1 headset

2016 MARCH

Oculus releases consumer Oculus Rift VR headset

HTC and Valve release Vive VR headset

2016 APRIL

• VR became accessible to everyone since mid-2013• Low cost of VR hardware• Better GPUs – better graphics – high fidelity• Robust networking capabilities – multiuser

Present of AR

After a long research tradition AR becomes popular with Layarapp for smartphones

2009 JUNE

Blippar acquires Layar

2014 JUNE

2013 FEBRUARY

Google releases Google Glass AR glasses

2014 SEPTEMBER

Meta releases Meta Spaceglasses Developer Kit 1

Apple acquires AR company Metaio

2015 MAY

2016 MARCH

Microsoft releases Hololens Development Edition AR glasses

Google and Lenovo release Tango AR phablet device

2016 JUNE

• AR seems "stuck" in 2009• Lots of attempts since then• AR performs well as enabling/supporting technology

Devices for VR & AR

• Head-Mounted Displays (VR)

• Mobile devices (AR & VR)

• Glasses (AR)

AR/VR Hardware vs. Content

AR/VR application fields

• Training / Education• simulators, training employees, experiential learning

AR/VR application fields

• Entertainment• video games, movies, 360 videos

AR/VR application fields

• Medical• treating phobias, visual navigation

AR/VR application fields

• Marketing• retail, real estate, VR shops, logistics

AR/VR application fields

• Engineering / Design• Virtual prototyping, AR/VR design tools

AR/VR application fields

• Data Visualisation• Big data visualization, visualization in place

Multi-user environments

• Significant functionality that can be applied to AR/VR systems

• 2 or more distant users being, experiencing, working together

• Creating new social digital environments, new affordances, new needs for communication

Research @ SINTEF

Past AR/VR projects & research

• Supporting first responder in-field communication and navigation using head-mounted displays

• Google Glass

• Enable in-field receiving of information from a command center, as well as in-field navigation and video streaming

• BRIDGE project: Bridging resources and agencies in large-scale emergency management

Past AR/VR projects & research

• Darwin project

• The project is focused on improving responses to expected and unexpected crises affecting critical societal structures during natural disasters (e.g. flooding, earthquakes) and man-made disasters (e.g cyber-attacks)

• Development of a Serious Game in VR for improving resource management and responses to expected and unexpected crises

Service Design VR project• Objective: optimize service prototyping, lead to more effective decision-making

• Method: we introduce Virtual Bodystorming, based on the Bodystorming service prototyping method

• Bodystorming: role-playing of service scenarios, can take place in low-fidelity environments or the actual service environmentLimitations: use of mock-ups & projections =

• low-fidelity & low user engagement -> low-quality feedback

• resource-demanding setup, physical presence = time & cost-demanding

Service Design VR project

→

• Virtual Bodystorming: simulate the service environment in VR & enable users role-play service scenarios

• VR in this project: use for fully-immersive, high-fidelity role-playing & experience prototyping

• Collaborative/Multi-user functionality: connect geographically distant participants, allow moderator to "shadow" users, allow users to collaborate and role-play

• Goals: create high-fidelity environments, stimulate user engagement, include distant users of various cultural backgrounds, get better feedback

• Target: services with human touchpoints, spatial elements

Service Design VR project• We developed a prototype of the system based on a mock case study about a mall

• Service designer: organizes the session

• 2 levels: Lobby and Service Level

• Conducted an expert review

• Future research: case studies to examine UX - usability

Service Design VR project• Developed with Unreal engine, next version will use a VR sandbox called ModBox:

• Creation of the service environment either from scratch or by importing a 3D scene

• Integrated multi-user capabilities, audio communication, physics, import of scenes & objects, well-documented SDK, community support

• Testing has begun, aiming for more cases…

Interaction-in-VR projects

• Since the revival of VR (mid-2013), HCI research in VR is addressing constructive research problems, putting aside conceptual and empirical

VR Locomotion

• VR locomotion (moving in VR environments) has been researched in the past, however the latest VR advances created new research needs

• We plan on examining current locomotion techniques (i.e. artificial locomotion, teleportation, room-scaled locomotion, motion-based locomotion) from a UX point of view, leading up to a framework and/or design suggestions

Interaction-in-VR projects

Text input in VR

• Examine text input & speech-to-text input (typing in VR)

• Typing in VR is still not optimized

• We will investigate current techniques in order to form empirical & conceptual knowledge

Points for discussion

• AR/VR applications in your field?

• AR/VR for businesses & decision making, altering internal processes

• 3D audio, AR audio

• AR/VR for marketing

Technology for a better society