an introduction to virtual reality iman mohammad rezazadeh school of biomedical engineering science...
TRANSCRIPT
An Introduction toAn Introduction to Virtual Reality Virtual Reality
Iman Mohammad Rezazadeh
School of Biomedical Engineering
Science and Research Branch
Islamic Azad University
Spring 2010
Virtual Environment (VE)Virtual Environment (VE)A computer generated world with
which the user can interact
Interaction can vary from looking around to interactively modifying the world.
What is virtual reality(VR)?
A computer generated,immersive(or wide field),multi-sensory informationprogram which tracks a user in realtime.
HistoryHistory
Year Person(s) Responsible Why it was important
1965 Ivan Sutherland The beginnings of VR
1977Dan Sandin, Richard Sayre and Thomas Defanti
Interaction through body movement
1982 Bonnie MacBird (Writer) The first computer- generated movie
1983 Myron Krueger First virtual environment
1984 William Gibson (His assistant) The term 'Cyberspace'
1987 Michael Piller (Writer) The Holodeck, idea of immersive VR
1992 Stephen King (Official Website) A look at the possible negative side of VR
1995 Silicon Graphics Virtual Reality Modeling Language
1999 Larry and Andy Wachowski Virtual Reality movie grosses $750M worldwide
TypesTypes
ImmersiveAugmented Text-basedDesktop (Window on a World)Video Mapping
Computer Graphics vs. VRComputer Graphics vs. VRCG is concerned with the modeling, lighting
and dynamics of environments and in general how to build a medium – so forth called Virtual World.
But, designing a VE is something beyond designing a CG.
When we become concern about how people experience the simulated environment, (re)act within it and interact with its components through the designed human-machine interface (HMI), we are talking about VE or VR
What is Immersive VR?What is Immersive VR?
A type of VR in which the user becomes immersed (deeply involved) in a virtual world. it is also a form of VR that uses computer related components.
Types of Non-immersive Types of Non-immersive VRVRText-based VR: when a reader
of a certain text form a mental model of this virtual world in their head from the description of people , places and things.
Augmented VR: the idea of taking what is real and adding to it in some way so that user obtains more information from their environment.
ApplicationsApplicationsArchitectureTrainingMedicineEngineering and DesignE-CommerceEntertainmentManufacturing
DevicesDevices
Head Mounted Displays (HMD):
was the first device providing its wearer with an immersive experience. Evans and Sutherland demonstrated a head-mounted stereo display already in 1965.
3D VIDEO Eyewear3D VIDEO Eyewear
winner of a 2005 & 2006 Consumer Electronics Show Design
Award.
VR Price: $499.00
Binocular Omni-Orientation Binocular Omni-Orientation Monitor (BOOM)Monitor (BOOM)
BOOM is a head coupled stereoscopic display device. Screens and optical system are housed in a box that is attached to a multi-link arm.
VR… VE…VR… VE…
14
AR in teleconferencingAR in teleconferencing
• person works at real desk
• remote collaborator represented by picture or video or “talking head”
• objects of discussion; e.g. a patient’s brain image, might also be fused into visual field
• HOW IS THIS ACHIEVED?
From University of Washington HIT Lab
Cave Automatic Virtual Cave Automatic Virtual Environment (CAVE)Environment (CAVE)
A room with projections on all walls, floor and ceiling
The users wear shutter glasses to get a 3D view of the world.
The users are able to move and control the environment with some kind of input mechanism◦ Camera◦ Device in hand
Bam and its Cultural Bam and its Cultural LandscapeLandscape After the earthquake of the 26th December
2003 and the immense destruction of the Bam heritage in Iran, the reconstruction of the ruins especially of the citadel of Bam has become an important task for experts.first phase of the project were to develop the 3Dimensional models of 7 case studies inside the citadel of Bam (Main gate,Bazaar,Tekkiyeh (religious plaza), Mosque, Sistani house, barrack, stable, four season reception room
Cont….Cont…. The team of drawing the 3D models
consisting of in Japan from Waseda University and the team of Raaz Ahang architectural firm in Iran from University of Tehran has made a tremendous effort to build the 3D models.
How about… learning to hang-glide?
I’m more of the down to earth type...
How aboutlearning todrive a car?
© www.ozedweb.com
Hey, that wasn’t there a moment ago...
Things are easywith VR.
Real Estate Agents
use it every day.
© www.ozedweb.com
...And Architects love it!
Wait till Wait till youyou
WalkWalkthrough!through!
© www.ozedweb.com
At last… something real.
Sorry… Sorry…
That’s VRThat’s VRtoo.too.
© www.ozedweb.com
Ahh...
It goes here.It goes here.
© www.ozedweb.com
Princeton Video WallPrinceton Video Wall
Augmented RealityAugmented Reality
A nice Little ARA nice Little AR
Share Virtual EnvironmentShare Virtual EnvironmentYou are a Avatar!!!! Wow…You are a Avatar!!!! Wow…
Classroom is a VEClassroom is a VE
VR CAVEVR CAVE
VR WorkbenchVR Workbench
Norman’s AffordancesNorman’s AffordancesAffordances:
◦Have perceived properties that may or may not exist
◦Have suggestions or clues about to how to use these properties
◦Can be dependent on the Experience Knowledge Culture of the actor
◦Can make an action easy or difficult
From McGrenere & Ho, Proc of Graphics Interfaces, 2000
Affordances of a Teapot?Affordances of a Teapot?
Slide adapted from Saul Greenberg
Real vs. Perceived Real vs. Perceived AffordancesAffordancesIn product design, where one deals
with real, physical objects, there can be both real and perceived affordances, and the two need not be the same.
Slide adapted from Saul Greenberg
Based on slide by Saul Greenberg
Transfer EffectsTransfer EffectsPeople transfer their
expectations from familiar objects to similar new ones◦positive transfer: previous
experience applies to new situation◦negative transfer: previous
experience conflicts with new situation
Based on slide by Saul Greenberg
Cultural AssociationsCultural AssociationsGroups of people learn idioms
◦red = danger, green = goBut these differ in different
places◦Light switches
America: down is off Britain: down is on
◦Faucets America: counter-clockwise is on Britain: counter-clockwise is off
VR and CognitionVR and Cognition
VR in Medicine VR in Medicine
I think I’m feeling ill.
Well, that’s not
a problem.
We’ll find acure with theVirtualpatient...
© www.ozedweb.com
…now I’m sure I am.
…in our virtual
hospital.
Why not?
© www.ozedweb.com
Virtual Body vs. CadaverVirtual Body vs. CadaverTraining on cadavers has several drawbacks:
◦ if trainee cuts a nerve or a blood vessel in a cadaver nothing will happen
◦ no action can be reversed on cadavers (what is cut is cut)
◦ dead tissue is harder, color is changed, arteries do not pulsate
Advantages of computer simulations:◦ procedures can be repeated many times with no
damage to virtual body◦ virtual body does not have to be dead - many
functions can be simulated for realistic visualizations
◦ organs can be made transparent and modeled
VR in EndoscopyVR in Endoscopy
Virtual Fighting and Virtual Fighting and Rehab.Rehab.
VR in EndoscopyVR in Endoscopy
VR in EndoscopyVR in Endoscopy
endoScope - A VR ToolendoScope - A VR Tool
EndoScope is a Motif/Inventor based model viewer that runs on SGI workstations
Developed by Biomedical Imaging Resource at Mayo Clinic
VR in ColonoscopyVR in Colonoscopy
VR in LaparoscopyVR in Laparoscopy
VR in LaparoscopyVR in Laparoscopy
MIS Training at EPFLMIS Training at EPFL
Group for surgical robotics and instrumentation, Swiss
Gastro-intestinal organ modeled and tissue properties simulated (pushing, pulling)
Force feedback generated for realistic simulation
Laparoscopic InterfacesLaparoscopic Interfaces
Immersion Corp. devices for minimally invasive surgery simulators
Offers tracking in 5 degrees of freedom (left-right, up-down, in-out, rotation around axis, open-close)
Version with and without force-feedback are available
Price range: up to $8,000
3-D Stereo Visualization3-D Stereo VisualizationUniv. of Illinois Chicago, School of
Biomedical and Health Information Sciences
ImmersaDesk system for temporal bone visualization
Fetus VisualizationFetus Visualization
Univ. of North Carolina at Chapel Hill
A VR system is developed for visualization of a fetus in a pregnant woman’s abdomen
The ultrasound image of the fetus is superimposed on the video image of the woman’s abdomen
The system can be used for pregnancy check-ups
Fraunhofer Institute Fraunhofer Institute SimulatorsSimulators
Fraunhofer Institute medical training simulators
The arthroscopy training simulator
Nasal endoscopy simulator
The trainee is able to practice techniques before facing a real patient
Eye Surgery SimulatorEye Surgery Simulator
Interactive Media Technology Center at Georgia Tech
Eye surgery simulator for:◦ education and training of medical students◦ training of surgeons to cope with emergencies
Simulator provides force feedback information for more realistic simulation of tissue cutting
Simulation of the tissue includes elasticity of the eye surface tissue before a cut is made
Craniofacial Surgery Craniofacial Surgery SimulationSimulation
Erlangen Institute, Germany In craniofacial surgery it is important to plan
and predict the outcome of surgical interventionThe face can be visualized after reconstructive
plastic surgery
Augmented Reality for Augmented Reality for SurgerySurgery
Julesz, Harward Medical SchoolAugmented reality visualization has three phases:
1. 3D laser scanning of the patient’s head surface
2. 3D registration of the scanned and imaged surface
3. Augmented reality display of tumor (green)
Augmented Reality in Augmented Reality in NeurosurgeryNeurosurgery
Harvard Medical School
Combined neurosurgery planning and augmented reality
Phobia ProjectsPhobia Projects
GA Tech GVU Center Virtual Reality Exposure Therapy project
Demonstrated effectiveness Advantages:
◦ cost effective◦ effective therapy◦ patient acceptance◦ suitable for network delivery
(telemedicine)
Spider Phobia Spider Phobia DesensitizationDesensitization
Univ. of Washington Human Interface Technology Lab
VE environment designed that contains virtual spiders (a large brown spider with fur and a small one were used
Patients are encouraged to pick up spiders with their virtual hands
Spiders are unexpectedly dropped of the ceiling, patient could pull the spider legs off
Acrophobia ProjectAcrophobia Project
Univ. of Michigan Balcony view from high floors are
generated The patient gradually watches the
environment from higher and higher viewpoints
A multi-session therapy cures the acrophobia patient
Medical Education at Medical Education at UCSDUCSD
UCSD Applied Technologies Lab project on VR-multimedia system for education of medical students (anatomy)
UCSD Virtual Anatomy WorldAnatomic structures are
linked to supporting multimedia contents to provide VR-MM anatomy lessons
Medical Education at Medical Education at Fraunhofer InstituteFraunhofer Institute
Fraunhofer InstituteVirtual anatomy
system for medical students
Virtual patientStudents are able to
understand complex interrelationships of anatomical structures
Remote Palpation Remote Palpation SimulatorSimulator Interactive Technology Media Center at
Georgia TechThe idea is to allow doctors to examine
patients at a remote location IMTC developed a haptic lens - a sensor that
measures 3D surface under a specific pressure
The device is pressed against an object and 3D surfaces, deformed under the pressure, are recorded in real-time
OK! OK ! No Military UseOK! OK ! No Military UseAnkle TherapyAnkle Therapy
Medical RoboticsMedical Robotics
Introduction(2)Introduction(2)CIS
Information flow in CIS
Motivation:◦Started with the weaknesses and
strengths of minimally invasive surgery (MIS)
◦Smaller incisions, shorter post-operative time, reduced infection, faster rehabilitation, lesser pain, better cosmetics, ...
◦Eye-hand coordination, difficulty in moving arms, degree of motion
ClassificationClassificationDepending on the degree of surgeon interaction during the procedure: ◦Supervisory-controlled;◦Telesurgical;◦Shared-control;
Application of Medical Application of Medical RoboticsRoboticsTelesurgery
◦Surgeon sits at a console
◦Has controls to move the robotic arms
◦Does not operate on the patient directly
◦Mainly used in minimally invasive surgeries
Application of Medical Application of Medical RoboticsRoboticsTelemedicine and Teleconsultation◦Telecommunciation channels to
communicate with other physicians/patients
◦Control an external camera which in turn controls an endoscopic camera – used to share images with a remote surgeon
Application of Medical Application of Medical RoboticsRoboticsRehabilitation◦Assistive
robots◦Wheelchair
with intelligent navigational control system
Application of Medical Application of Medical RoboticsRoboticsRemote surgery
◦Surgeon can be anywhere in the world
◦Remotely controls the robotic surgical system
◦Very useful for treating wounded people in battlefields
Application of Medical Application of Medical RoboticsRoboticsLaparoscopic Surgery
◦Performed in the abdominal cavity using MIS
◦Abdomen cavity is expanded using CO2
◦Uses Laparoscopic instrument Fiber optic channels to illuminate the inside of
abdominal cavity Lens optics to transmit image CCD camera at the outer end Image displayed on high resolution TV
Design of Robotic Design of Robotic TelesurgeryTelesurgeryThe Concept
Telesurgical system concept
Existing surgical systemsExisting surgical systemsDa Vinci Surgical® system by
Intuitive Surgical, Inc.
Existing surgical systemsExisting surgical systemsDa Vinci Surgical® system by
Intuitive Surgical, Inc.
VR in Brain Computer VR in Brain Computer InterfaceInterface
VR in BCIVR in BCI
Monitoring Driver’s Monitoring Driver’s AlternessAlterness
VR, BCI, ImaginationVR, BCI, Imagination
My Office in FutureMy Office in Future
The End of the Beginning
- Surgery Planning and SimulationSurgery Planning and Simulation- Rehabilitation and EmpoweringRehabilitation and Empowering- Mind Engineering Mind Engineering - Neurofeedback and BiofeedbackNeurofeedback and Biofeedback- Mixed RealityMixed Reality- Real VirtualityReal Virtuality- Pervasive computingPervasive computing- Ubiquitous computingUbiquitous computing
© www.ozedweb.com
Invitation for ContributionInvitation for Contribution
"Mixed Reality and Human-Robot Interaction”
Springer
What we are doing in IAUWhat we are doing in IAUVirtual Upper limb and Lower Limb
Rehab.
Mind Engineering and Neurofeedback(CARTEME)
Collaborative Interfacing
Affective Computing