augmented reality (ar) · 1. introduction augmented reality (ar) became a subject in computer...

Augmented Reality (AR) for Assembly ProcessesDesign and Experimental Evaluation

Stefan WiedenmaierOlaf OehmeLudger SchmidtHolger Luczak

Institute of Industrial Engineering and Ergonomics, Aachen University of Technology

2003

1. IntroductionAugmented reality (AR) became a subject in computer graphics forroughly 10 years ago

The first prototypes in the industrial environment were mainly developed in the aircraft industry.

AR Applications can be created for assembly, disassembly, inspection, fault finding, and training.

The challenge lies in making an AR assembly system user friendly, because it requires several stages of examination and redesign.

2. ARsemblyARsembly is a computer-aided assembly for manual workers who use AR technologies.

Grouped according to their degree of computer support

3. ARsembly Applications

Luczak et al. (2000) described the following AR services:

1. The customer can be supported by an electronic AR manual to execute his or her work. A similar but more sophisticated type of support could help service technicians who repair a machine at the customer’s location.

2. When customers need an expert’s advice, they can use the manufacturer’s telephone hotline. In doing so, they very often experience a problem in that the experts cannot see the worker’s field of view; they need a common visual basis of reference. To facilitate the hotline’s considerations, there should be some means of pointing at certain objects in the user’s field of view.

3. An autonomous AR system can be used by customers to solve theproblems by introducing an augmented field of view.

3 ARsembly Applications

AR applications are not actually useful for chain production, because they would interfere in small, optimized, and repetitiveassembly. Nevertheless, AR can be applied in the domain of training and quality control

4. Comparison of Display types

look-around displays seem to be most suitable for ARsembly.

5. Experimental Setup

From the point of view of usability, a scenario was based on the manufacture of a current industrial product that seemed to be most suitable.

The goal was to create user-friendly instruction for the assembly of the entire automobile door

This scenario was also interesting as a potential trainingscenario for assembly line production.

6. Assembly ScenarioAn AR scenario was Developed with three steps1. Mounting window regulator, 2. Wiring3. Fixing clips for the inner door panel

difficultyWR > difficultywiring > difficultyclips for the inner door panel.wiring,seemed to be particularly appropriate for augmentation because of its multiple branches and fixation points.

7. Assembly ApparatusA hybrid AR–HMD and touch panel display system was chosen for the AR-supportedassembly

This hybrid system makes wearing an HMD duringthe entire assembly unnecessary.

The touch panel display shows the easier assembly steps such as those in an electronic manual, and it is fixed to the frame for the door assembly.

7.1 Assembly apparatusThe assembler interacts with the system using his or her fingers.

During the AR-supported steps, the operator wears an HMD. A video camera is fixed on the HMD.

8. Method

The experiment was an investigation into the effect different kind of media support in the assembly scenario

1. paper instruction2. AR support3. expert tutorial

8.1 MethodThe participants were required to execute 22 steps

The assemblers were asked to think aloud during the execution ofthe assembly tasks. This way, the problems in interaction with the system were immediately noticeable.

The assembly procedure was videotaped.

After the assembly tasks were completed, the experimenter conducted a usability interview to evaluate each interface.

9. ResultsThe following null hypothesis (H0) was established

total assembly timepaper instruction = total assembly timeAR support = total assembly timetutorial.

10. ConclusionsDifficult tasks showed more application potential than easy tasksHowever, for intuitive and repetitive tasks, AR support did not lead to significantly lower assembly times than a paper manual.Occluded assembly places still pose a problem for ARsembly.The findings of the experiment described here offer evidence in favor of the use of clip-on displays.Even though the hybrid AR support was successful, using merely asimple Web-based manual on a touch-panel computer would not be enough.the hardware for AR systems is still too large and cumbersome and not really suitable for daily

Browsing the Real-World Wide Web: MaintainingAwareness of Virtual Information in an ARInformation Space

Rob KooperBlair MacIntyre

GVU Center and College of Computing, Georgia Institute of Technology (2004)

1. The dream“A future in which head worn displays are as light and unobtrusive as a pair of glasses and can therefore be worn continuously, allowing users to be immersed in an augmented information space as they move through their daily lives.”

2. Scenario“After taking a shower in the morning, Bill puts on his clothes, jacket (containing his wearable computer), and his glasses (a lightweight, see-through, head-worn display). The glasses connect to the wearable computer using a wireless connection and immediately display contextual information. A quick glance around the room shows Bill that one of the books at the bottom of his stack is marked as due today at the library…..

3. The Real World Wide Web

This work is based on a vision of an enhanced WWW in which each information object (i.e., “Web page,”containing two-dimensional [2D] and 3D visual and auditory information) may have contextual meta-data associated with it, such as a location, person, or activity. They refer to such a space as the Real-World Wide Web (RWWW) and interfaces to it as RWWW browsers.The authors believe that the RWWW will be around usall the time and will evolve naturrally from the WWW

4. Evolution

The authors believe that the WWW willevolve to the RWWWThat the RWWW will be around us all the time.The authors only see problems in showingkey information.

5. Constraints

Non interference – to not interfere the viewersworldMinimal interaction – the user should rarely ifever be required to interact without wanting to. Just like the Ambient room, to be succesful it must present continious awareness withoutdisturbing the focal task.

6. PrototypeFIGURE 1 When the user looks around the world, he or she sees small twinkling stars indicating the presence of information nodes.Ifthe user glances at a star (by pointing the center of his or her display screen, shown by a small white square, at the star), a thumbnail image of the Web page is displayed, along with the title of the page. The link closest to the center has a text description and small thumbnail (10 cm in world coordinates) displayed, connected by a leader line. A red circle is slowly drawn around the thumbnail (over the course of a few seconds). AEL = Augmented Environments Laboratory.

FIGURE 2 After a short gaze (the time taken to draw the red circle), a larger image of theWeb page appears in the upper left corner of the screen, and the circle turns yellow to indicate which node is gaze-selected and displayed in the upper left.

Figure 2(a) shows a non spatializednode associated with the room in which the user is located and is contained on a virtual billboard (see section 4.4). In (b), the node is located near the computer it describes. AEL = Augmented Environments Laboratory.

FIGURE 3 In Figure 3(a), a more detailed view of a node appears in the body space centered around the user’s waist (that can be viewed by looking down).

In (b), the user can toggle a thumbnail view on and off to enable rapid browsing of the information in a space. Although the thumbnail view significantly clutters the user’s view of the world, it allows him or her to more quickly see the content of the nodes and search for specific information. AEL = Augmented Environments Laboratory.

FIGURE 4 When an information node is pushed into the temporary storage around the user’s waist, the circle around its thumbnail turns green (as shown around the white thumbnail in Figure 4[a]), and the page in the body space is rotated to the right (as shown on the right edge of [b]). The user can now gaze-select a new page (the black page in [a] and [b]); the thumbnails, and the detail views in the body space, of the saved page and the new page are both available to the user. AEL = Augmented Environments Laboratory.

FIGURE 5 Two types of focus plus context. In Figure 5(a), nodes spatially near the selected node are displayed as thumbnails, with the size of the thumbnail being proportional to the Euclidean distance from the focal node.

In (b), a simple form of semantic distance is used whereby the distance function between nodes is based on the similarity of their URL prefixes. The nodes on the bottom row are the user’s to-do items, whereas the nodes in the top row are from different locations on the same site. AEL = Augmented Environments Laboratory.

7. Integration with a hand-held

FIGURE 6 The hand-held display can be used to (a) select the channel markers that are presented on the display, and which of them have their content displayed; (b) configure the system; and (c) display the detailed view of the currently selected information node. When the hand-held is used to display the information node instead of the body space, the built-in history and favorites mechanisms of the hand-held Web browser provide similar functionality as the temporary storage area of the body space.

FIGURE 7 Rob Kooper using the prototype system. The helmet holds an opaque head-worn display and camera (used to create a video-mixed AR system), the receiver for a tracking system (the black hardware on the back of the helmet), and four buttons (two on each side of the front, held on with black gaffers tape) connected to a Phidget IOKit board [15] (mounted on top of the tracker). A wireless HP 548 PocketPC was used as our hand-held display. The entire system is run from a laptop carried by the user (a Dell 8000, 1Ghz Pentium III with 512Mb of RAM and a GeForce2GO graphics accelerator); the wires leading away from the laptop in the left image connect to the tracking system we are using (which is not yet wireless) and to ethernet and power (batteries and wireless networking can also be used).

8. Discussion and Future work

Better tracking is needed.The environment needs to be modeled to place information. Outdoors is easier donethan indoors.Interaction, Maybe use eyetracking. Instead of the center of the display (Gazeselect).

Communication Behaviors in Colocated collaborative AR Interfaces

Mark BillinghurstDaniel BelcherArnab GuptaHuman Interface Technology Laboratory, University of WashingtonKiyoshi KiyokawaHuman Interface Technology Laboratory, University of WashingtonEmergency Communication Section, Communications Research Laboratory, Tokyo


present an analysis of communication behavior in face-to-face collaboration

using a multi-user augmented reality (AR) interface. Two experiments were conducted.

In the 1st experiment, collaboration with AR technology was compared with more traditional unmediated and screen-based collaboration.

In the 2nd experiment, the authors compared collaboration with 3 different AR displays.


The field of computer-supported collaborative work contains numerous examples of interfaces that facilitate mediated collaborationmuch of that work is focused on remote collaboration.In contrast, the authors are interested in developing interfaces for face-to-face collaboration;in particular, collaboration on tasks that involve viewing and manipulating spatial data.


difficult to manipulate three-dimensional (3D) data on a two-dimensional screen or with traditional input devicesScreen-based interfaces create a separation between the real and digital domainsReal objects and interactions within the real world play an important role in face-to-face collaboration


when people sit at a table, the space between them is used for sharing communication cues, such as gazes, gestures, and nonverbal behaviors.when users are collaborating in front of a screen, their attention is focused on the screen. collaborators may exhibit different communication behaviors when using a screen-based interface rather then sitting by a table.


The authors use augmented reality (AR) technology to overlay 3D virtual imagery directly onto the real world. In this way, the display space and communication space overlap.


The work performed by the authors use a variety of performance, process, and subjective measures.

Performance measures are those that measure a task outcome, such as the time required to complete a task.Process measures are conversational elements that occur during the collaboration, such as the number of words spoken.Subjective measures are the participant’s own subjective impressions of the collaboration.


In Experiment 1, the authors compared collaboration with an AR interface to a projection display and to unmediated face-to-face interaction. The three different conditions

FtF: unmediated face-to-face collaborationAR: augmented reality face-to-face collaborationProj: projection screen-based collaboration


The main differences among these conditions were in the viewpoints of the collaborators and the method for interacting with the shared objects


This involved placing nine real or virtual model buildingsFor each condition, the buildings needed to be placed to satisfy 10 rules, such as:

The CHURCH is next to the THEATER.The FIREHOUSE and the BANK are across the street.The CITY HALL is in the middle of town.

The rules were designed to be complementary, and each participant was given only 5 of them.


Participants were asked to arrange the same set of buildings under three conditions.

FtF: unmediated face-to-face collaborationAR: augmented reality face-to-face collaborationProj: projection screen-based collaboration


FtF: unmediated face-to-face collaboration


AR: augmented reality face-to-face collaborationThe participants were given a set of nine cards with Japanese Kanji characters marked on them.


A computer vision based tracking method was used to track the cards and overlay virtual buildings on them.


They also wore the DaeYang Cy-Visor head-mounted display with a small video camera attached


Proj: projection screen-based collaborationParticipants sat side by side at one end of a table, facing a large projection screenShown on the screen was a graphical interface that allowed the participants to place virtual models of buildings on a flat ground plane.


The participants in this experiment were:

14 pairs of college-age adults: 6 pairs of women and 8 pairs of men. Ages ranged from 21 years to 38 years. Several of these pairs were later dropped because of incomplete resultsThe final results were drawn from 12 pairs of participants.


SD = standard deviation

ResultsPerformance


ResultsProcess measures


Process measures


This figure shows the percentage breakdown for each type of gesture made out of all the gestures for each condition.


ResultsSubjective measures

participants filled out a survey regarding how they felt about the interface 1 = not very easy, 7= very easy.


Experiment 2compare face-to-face collaboration with three different AR displays


LCD: A 5-in (12.7 cm) diagonal LCD with a camera on the backHHD: A hand-held displayHMD: a head-mounted AR display


Engaged pairs of participants in the same urban design task used in Experiment 1.Participants completed the task in each of four conditions:

FtF – Two participants sat across a table with a street map marked on it.LCD – This condition was identical to the FtF condition, but the participants each held an LCD panel.HHD – identical to the LCD condition, but each participant used an HHD.HMD – identical to the LCD condition, but each participant wore an HMD.


Experimental measures were the same as those in experiment 1

Performance measures: the time to complete the building layout task.Communication process measures: turn taking, deictic speech, questions, gestures, and simultaneous speech.Subjective measures: postcondition surveys and interviews.


ResultsPerformance


ResultsCommunication process measures


ResultsCommunication process measures

gestures


ResultsSubjective measures

participants filled out a survey regarding how they felt about the interface 1 = not very easy, 7 = very easy.


ConclusionsEase of interaction and some task performance can be unchanged when using different AR displaysThe AR interface does affect the perception of communication cues.It is likely that results for other tasks will differ. In particular, conversational tasks, such as negotiation, will be even more susceptible to differences in the AR interface.

augmented reality (ar) · 1. introduction augmented reality (ar) became a subject in computer...

Documents