vr juggler technical overview

VR Juggler Technical OverviewVR Juggler Technical Overview

A Virtual Platform for Virtual Reality A Virtual Platform for Virtual Reality Application DevelopmentApplication Development

Allen BierbaumAllen Bierbaum

Iowa State UniversityIowa State University

IEEE VR TutorialIEEE VR Tutorial

www.vrjuggler.org 2

Who am I?Who am I?

• Allen BierbaumAllen Bierbaum• WorkWork

– VR Juggler TeamVR Juggler Team– PhD StudentPhD Student

• Iowa State University – Virtual Reality Iowa State University – Virtual Reality Applications Center (VRAC)Applications Center (VRAC)

www.vrjuggler.org 3

ObjectivesObjectives

• Understand VR Juggler architectureUnderstand VR Juggler architecture• Knowledge of some technical detailsKnowledge of some technical details• Introduction to future VR Juggler Introduction to future VR Juggler

developmentsdevelopments

• Please: Ask questions!!! Please: Ask questions!!!

www.vrjuggler.org 4

What is VR Juggler?What is VR Juggler?

• Open Source VR software development Open Source VR software development environmentenvironment

• VR Virtual platform & Application VR Virtual platform & Application frameworkframework

• Developed at Iowa State UniversityDeveloped at Iowa State University– Project director: Dr. Carolina Cruz-NeiraProject director: Dr. Carolina Cruz-Neira– 5+ years of development5+ years of development– VR Juggler team (3 people VR Juggler team (3 people 10+ people at 10+ people at

VRAC)VRAC)

www.vrjuggler.org 5

VR Juggler backgroundVR Juggler background

MotivationMotivation• Standard VR development environment that is:Standard VR development environment that is:

– Flexible & extensibleFlexible & extensible– MaintainableMaintainable– ModularModular

– Cross-platform / Cross-VR SystemCross-platform / Cross-VR System– Freely available – OpenSourceFreely available – OpenSource– Allow community involvementAllow community involvement

– Extension and refinementExtension and refinement– Peer reviewPeer review

www.vrjuggler.org 6

VR Juggler featuresVR Juggler features• Virtual platformVirtual platform• Application objects and frameworkApplication objects and framework• Performance drivenPerformance driven

– Performance monitoringPerformance monitoring

• Flexible and extensibleFlexible and extensible• ModularModular• Open sourceOpen source• Cross-platform – Irix, Linux, Win32, others…Cross-platform – Irix, Linux, Win32, others…• Configuration systemConfiguration system

– Parameterization of VR system components Parameterization of VR system components – Run-time reconfigurableRun-time reconfigurable– GUI – vjControlGUI – vjControl

www.vrjuggler.org 7

Virtual platformVirtual platform

• Separates system dependent and system Separates system dependent and system independent componentsindependent components

www.vrjuggler.org 8

Virtual platform: BenefitsVirtual platform: Benefits

BenefitsBenefits• Architecture and OS independenceArchitecture and OS independence

– Handles cross platform issuesHandles cross platform issues– Virtual platform scalesVirtual platform scales

• Device abstractionDevice abstraction– Separates application from hardwareSeparates application from hardware– Proxies – handles to devicesProxies – handles to devices

• Defines an operating environment for VR appsDefines an operating environment for VR apps– The VP allocates resources and processing timeThe VP allocates resources and processing time

www.vrjuggler.org 9

System ParameterizationSystem Parameterization

• Juggler “config chunks” specify single unit of Juggler “config chunks” specify single unit of configurationconfiguration

• Parameterize all configurable parts of systemParameterize all configurable parts of system• Edited with Java base GUI (vjControl)Edited with Java base GUI (vjControl)

www.vrjuggler.org 10


Kernel subsystem: MotivationKernel subsystem: Motivation

Motivating needsMotivating needs• System must be extensibleSystem must be extensible

– Add new devices and graphics APIs Add new devices and graphics APIs

• System needs to be flexibleSystem needs to be flexible– Adapt to VR systems, multiple configurationsAdapt to VR systems, multiple configurations

• System needs to be dynamicSystem needs to be dynamic– Changes at run-time, added modularityChanges at run-time, added modularity

• System must evolve over timeSystem must evolve over time– App survivabilityApp survivability


Kernel subsystemKernel subsystem

Microkernel architectureMicrokernel architecture• Controls the entire systemControls the entire system• Manages communicationManages communication• Internal ManagersInternal Managers

– Core functionality beyond the scope of the Core functionality beyond the scope of the kernelkernel

• External ManagersExternal Managers– Provide an interface to the system that is Provide an interface to the system that is

specific to application typespecific to application type


Kernel subsystem: BenefitsKernel subsystem: Benefits

BenefitsBenefits• Modular systemModular system

– Managers can be added, removed, and Managers can be added, removed, and reconfiguredreconfigured

– Discourages monolithic architecturesDiscourages monolithic architectures

• MediatorMediator– Captures interactionCaptures interaction– SynchronizationSynchronization

• PortabilityPortability


Internal managersInternal managers

• Input managerInput manager• Display managerDisplay manager• Config managerConfig manager• Environment managerEnvironment manager


Input ManagerInput Manager

• Controls all devices in the systemControls all devices in the system• Updates the devicesUpdates the devices• Allows users to request devicesAllows users to request devices


DevicesDevices

• Base classes for all device typesBase classes for all device types• Extend system hereExtend system here


Device ProxiesDevice Proxies

• All Devices are accessed through proxiesAll Devices are accessed through proxies• Proxies hide the true device typeProxies hide the true device type• Allows code and configuration to be device independentAllows code and configuration to be device independent

0..1

0..1

0..*

0..*

vjPosProxy

vjPosProxy~vjPosProxyupdateDatagetUpdateTimesetTransformsetgetDatagetUnitgetPositionPtrgetTransformtransformDatagetChunkTypeconfiggetProxiedInputDevice

vjPosInterface

vjPosInterfaceoperator->operator*refresh

vjInput

vjPosition

vjPosition~vjPositionconfigstartSamplingstopSamplingsampleupdateDatagetDeviceNamegetPosDatagetPosUpdateTime

vjInputManagervjProxy

vjProxyconfiggetProxiedInputDevicegetChunkType


Device FactoryDevice Factory

• All device types are registeredAll device types are registered• Allows for dynamic discovery and loadingAllows for dynamic discovery and loading• Separates devices from the rest of the Separates devices from the rest of the

systemsystem


Display managerDisplay manager

• Contains all information about display Contains all information about display windowswindows

• Performs all viewing calculationsPerforms all viewing calculations


Other internal managersOther internal managers

Config managerConfig manager• Holds (re)configuration requestsHolds (re)configuration requests

Environment managerEnvironment manager• Communicates the state of the systemCommunicates the state of the system


External managersExternal managers

• Draw managerDraw manager• Sound managerSound manager


Draw managerDraw manager

• Provides external interface for applicationsProvides external interface for applications• Allows access to API specific functionalityAllows access to API specific functionality• Handles all details of the APIHandles all details of the API


Other External managersOther External managers

Sonix: Sound managerSonix: Sound manager• Generic interface for several sound libsGeneric interface for several sound libs• More in later talk…More in later talk…

Future…Future…• Area for extensionArea for extension


Application objectsApplication objects

– All VR Juggler applications are objectsAll VR Juggler applications are objects



– Override methods of a pre-defined interfaceOverride methods of a pre-defined interface– Derived class for each graphics APIDerived class for each graphics API



BenefitsBenefits• Single interface between app and systemSingle interface between app and system

– Low-coupling Low-coupling Robustness Robustness– Application survivabilityApplication survivability

• Allows for flexibility in kernelAllows for flexibility in kernel– Run-time changesRun-time changes

• Application managementApplication management– Capture application in single unitCapture application in single unit– Multiple applicationsMultiple applications


Run-time (re)configurationRun-time (re)configuration

Most current systemsMost current systems• Static configurationStatic configuration• No way to modify settings at run-timeNo way to modify settings at run-time• Configuring systemConfiguring system

– Test settings Test settings Change settings Change settings Restart application Restart application

Run-time ConfigurationRun-time Configuration• Configuration is done dynamically at runtimeConfiguration is done dynamically at runtime

Run-time ReconfigurationRun-time Reconfiguration• Maximum flexibilityMaximum flexibility• Allows settings to change during executionAllows settings to change during execution


Reconfiguration interfaceReconfiguration interface

• vjConfigChunkHandlervjConfigChunkHandler– API for handling configuration requestsAPI for handling configuration requests– Implemented by all Managers and some applicationsImplemented by all Managers and some applications


Config ManagerConfig Manager

• Pending configuration queuePending configuration queue• List of configuration requestsList of configuration requests• Stored in queue until kernel attempts reconfigurationStored in queue until kernel attempts reconfiguration


Reconfiguration processReconfiguration process


Reconfiguration benefitsReconfiguration benefitsSystem setup and configurationSystem setup and configuration

• Easier initial setup using interactive testingEasier initial setup using interactive testing– Reconfigure displaysReconfigure displays– Change physical devicesChange physical devices

Application adaptationApplication adaptation• Application switching at run-timeApplication switching at run-time• Configure applications parametersConfigure applications parameters

– Model loaded, navigation methodModel loaded, navigation method

RobustnessRobustness• Graceful recovery from device or app failuresGraceful recovery from device or app failures

Performance tuningPerformance tuning• Adjust configuration based upon performanceAdjust configuration based upon performance


Dependency managementDependency management

• Dependency checking between componentsDependency checking between components• Components needing checking register a checkerComponents needing checking register a checker


System interactionSystem interactionvjKernel

while(!quit)

vjInputManager

vjDisplayManager

vjGlApp

vjGlDrawManager

vjGlPipe

while(!quit)

draw():void

draw():void

wait()

triggerRender():void

<constructor>()

<constructor>()

sync():void

updateProjections():void

updateAllData():void

checkForReconfig():void

postFrame():void

intraFrame():void

draw():void

preFrame():void

start():void

initialize()

getDrawManager():vjDrawManager *

setup()

<constructor>()

setup()

<constructor>()

setApplication()

<constructor>()

start():int


System interaction (cont)System interaction (cont)

while(!quit)

vjGlDrawManager

vjGlPipe

while(!quit)

draw():void

draw():void

wait()

triggerRender():void

<constructor>()

<constructor>()

sync():void

updateProjections():void

updateAllData():void

checkForReconfig():void

postFrame():void

intraFrame():void

draw():void

preFrame():void

start():void

initialize()

getDrawManager():vjDrawManager *

setup()

<constructor>()

setup()

<constructor>()

setApplication()

<constructor>()

start():int

VR Juggler 1.1VR Juggler 1.1

VR Juggler development VR Juggler development directiondirection


MotivationMotivation

New abilities and research areasNew abilities and research areas• High-level toolsHigh-level tools• Technological advancesTechnological advances

Fix design problemsFix design problems• Yes, we actually made some mistakes…Yes, we actually made some mistakes…


VR Juggler 1.1 ModulesVR Juggler 1.1 Modules

Current ModulesCurrent Modules• Vapor – System abstractionVapor – System abstraction• Gadgeteer – Device managementGadgeteer – Device management• GMTL – High-performance mathGMTL – High-performance math• JCCL – ConfigurationJCCL – Configuration• VR Juggler – VR virtual platformVR Juggler – VR virtual platform• Sonix – Sound managementSonix – Sound management• Tweek – Common GUI frameworkTweek – Common GUI framework

This all used to be just VR Juggler…This all used to be just VR Juggler…• Now the Juggler suite of toolsNow the Juggler suite of tools• More on the way…More on the way…


New featuresNew features

VaporVapor• Serial port abstractionSerial port abstraction

GadgeteerGadgeteer• ClusteringClustering• Run-time device type extensionRun-time device type extension• Input from gfx windowsInput from gfx windows• Cross-platform device driversCross-platform device drivers


New features (cont)New features (cont)

VR JugglerVR Juggler• Revamped configuration GUIRevamped configuration GUI• Window viewportsWindow viewports• OpenSG & OpenSceneGraphOpenSG & OpenSceneGraph• Performer reconfigurabilityPerformer reconfigurability• Kernel shutdownKernel shutdown

TweekTweek• Palmtop interactionPalmtop interaction

SonixSonix• Sound managementSound management


High-level toolsHigh-level tools

• Many new high-level tools in the pipelineMany new high-level tools in the pipeline• All part of the Juggler suiteAll part of the Juggler suite

In developmentIn development• Collaborative networkingCollaborative networking• SonificationSonification• User interfacesUser interfaces• and more…and more…


AvailabilityAvailability

• Latest version always in CVSLatest version always in CVS• Developer releasesDeveloper releases

– VRJ 1.1 DR1 “on the way”VRJ 1.1 DR1 “on the way”

• New website with documentationNew website with documentation

vr juggler technical overview

Documents

vr systems

vr appsthe vp

graphics apis system

loadingseparates devices

extensibleadd new devices

true device typeallows

technical detailsintroduction

operating environment