AUGMENTED REALITY

AUGMENTED REALITY• Augmented reality is a live direct or

indirect view of a physical, real world environment whose elements are Augmented by computer generated sensory input such as sound, video, graphics or GPS data.

• This aims to create a sense of fully embodied interaction with spaces and people that are not immediately present in the same physical environment.

REQUIREMENTS AR displays and interface designs ;Require immersive interaction

with large scale 3D visualization.Have full sensory awareness of

the physical environment around them.

Are able to see and interact face to face with local and remote participants

Augmented Realityin

Military and Space

Why AR in military?o Situation awareness; keeping track

of the past present and future military operation

o Information Overload; the condition occurs when one is unable to process the multiple information presented into a coherent Situation Awareness (SA)

o Training; Embedding virtual training applications in existing live action training facilities can reduce modelling requirements and other infrastructure cost.

o Quick-reaction forces; an AR system can present exactly the right pieces of information-better and faster decision can be made and turned into correct actions.

AR MILITARY PROJECTS The super cockpit : application of AR

technology for fighter pilots developed at Wright-Patterson Air Force Base

beginning in the late 1960s [Furness(1969)]. Used where Visibility is limited, and airborne

tasks such as low-altitude navigation, target acquisition, and weapons delivery require pilots to reference landmarks on the terrain

sensors mounted on the aircraft can create visibility in areas that are occluded by the aircraft structure, or low light condition that prevent the pilot from seeing the real world

System was providing spatial awareness for the pilot to understand and incorporate into his course of action a variety of incoming data streams.

ASPEN MOVIE MAP; • Mechanism for pre-experiencing an unfamiliar

locale that allows the acquisition of spatial knowledge to take place in a meaningful, natural, and accurate way.

The Aspen Movie Map was based on the newly-available optical video disc technology of the 1970s to enable interactive computer control of the display of video frame.

In overview map mode, the user can trace the route taken or specify a route for the system to follow; these graphics will be merged with aerial overviews

Speed and direction were controlled through a scrollbar-like widget on the touch-screen display

C-130 Loadmaster Training

AC-130 is a military transport aircraft typically used in cargo and resupply operations by numerous countries

loadmaster is a cargo handler and rigging expert that delivers cargo by airdropping equipment and personnel out of the back of the C-130

The USAF Air Education and Training Command conducted a project to determine if AR was an effective tool for training C-130 loadmaster.

system was built from commercially-available hardware components and custom software.

Head-worn AR displays were mounted to standard-issue flight helmets for students of the training which provided a virtual situation to the trainees.

Augmented Reality In SPACE

Wearable Augmented Reality(WEAR)

WEAR is a wearable computing system that incorporates head mounted display over one eye to superimpose 3D graphics and data contents of its wearer's field of view

Controlled by voice Allows blending real world with simulator

elements

WEAR comprises an OTS (off-the-shelf) see-through Head Mounted Display (HMD) that has an OTS camera integrated on it and a headset for speech recognition and synthesis.

connected using VGA and USB to an onboard IBM A31p laptop (ESA dedicated payload )

WEAR in ISS Save crew time and increase efficiency Allowing hands-free operations and consultation

of manuals and technical information Avoid moving to different workstations to gather

required information Data that is relevant to procedures is presented

to the crew Just-In-Time (JIT) and Just-In-Place (JIP).

Wear includes onboard location and object identification to show astronauts precise information about what they are looking at and where a particular equipment is placed.

Provide step by step instructions to guide the astronauts through difficult, lengthy procedures and ensure hands free operation.

WEAR superimposes 3D graphics and data onto its wearer’s field of view, which enables them to have a pricise idea of their work.

• Incorporates 3D AR toolkit• Speech recognition• Object recognition and tracking• Consumer barcode reading technology

CHALLENGESChange from R&D to Flight hardware Constraints: –Time (to develop, available for testing & experimentation) –Logistics (access to enviroment, transportation,…) –Environment conditions (noise, light, ..) –Use of the COTS, in very specific application Technology: –Localization technology –Required performance –Calibration

NASA-MICROSOFTHOLOLENS

NASA and Microsoft team is developing software called OnSight, a technology that will enable scientists to work virtually on Mars using wearable technology called Microsoft HoloLens.

developed by NASA's Jet Propulsion Laboratory(JPL) in Pasadena, California

Provides scientists a means to plan along with the ‘Mars Curiosity rover’, conduct science operations on the Red Planet.

Provides our rover scientists the ability to walk around and explore Mars right from their offices

OnSight will use real rover data and extend the Mars Curiosity mission's existing planning tools by creating a 3-D simulation of the Martian environment.

Enable scientists to examine the rover's worksite from a first-person perspective, plan new activities and preview the results of their work firsthand.

Presently rover operations required scientists to examine Mars imagery on a computer screen, and make inferences about it. 3-D views, lack a natural sense of depth to understand exact spatial relationships.• OnSight uses holographic computing which blends a

view of the physical world with computer-generated imagery to create a hybrid of real and virtual

Scientists can stroll around the rocky surface or crouch down to examine rocky outcrops from different angles.

The tool provides access to scientists and engineers looking to interact with Mars in a more natural, human way. By looking at a target and using gestures they can select menu commands.

This tool for sure will enable researchers to understand the environment and workspace of robotic spacecraft - something that is challenging with the present tools.

JPL plans to begin testing OnSight in Curiosity mission operations later this year.

Future applications may include Mars 2020 rover mission operations, and other applications in support of NASA's journey to Mars.