Make It So: What Interaction Designers can Learn from Science Fiction Interfaces Presentation Notes, Nathan Shedroff and Chris Noessel 9 March 2008, SXSW Conference This is the first presentation of only a portion of the material we've found in our analysis of Science Fiction films and television series. Weʼre also looking a industry future films (like Apple's Knowledge Navigator) as well as existing products and research projects. Our analysis includes properties (films and TV), themes (different issues in interface design), as well as the historical context of the work (such as the current technology of the time of the propertyʼs release). In addition, weʼre interviewing developers (including production designers from films) but this material isnʼt presented in this talk. For this presentation, weʼve focused on the major issues, part academic and theoretical, and part lessons (more practical) weʼve uncovered. How design influences SciFi and how SciFi influences design: We've chosen to focus on interface and interaction design (and not technology or engineering). Some visual design issues relate but, mostly, in this talk, weʼre not approaching issues of styling. Weʼve chosen the media of SciFi (TV and films) because a thorough analysis of interaction design in SciFi requires that the example be visual so interfaces are completely and concretely represented, include motion that describe the interaction, and (sometimes) has been seen by a wide audience. Scientifically determining “influence” in any context (whether from Design on SciFi or visa versa) is difficult, and much of what we illustrate is inference on the part of the authors.

Design influences SciFi: Design (reality) sets the paradigm that scifi builds upon In turn, Sci Fi influences design in one of four ways: Inspiration, Expectation, Social Context, and Proposed Paradigm. Science Fiction relies on the context set by the design and development of existing products. This allows SciFi to communicate effectively with its audience. For example, during the Industrial Age, when the first SciFi film was created, Le Voyage Dane La Lune, the metaphors used were industrial and had no interfaces. To open the door, you pushed the door open. While this principally reflected the whimsical and theatrical nature of the film, it also reflected a lack of “interfaces” in the world at the time.

By the time Fritz Lang directed Metropolis, we see an attempt to realistically present a vision of the future that builds upon the audience's awareness of the telegraph, tickertape, radio, and telephone technologies

As depicted in the scene, to control the screen, the user “checks messages” via a tickertape, “tunes” in the video like a radio, and talks via “telephone” to his employee. However, by the time Buck Rogers appears on screen, the audience is now familiar with electronic television, allowing SciFi interfaces to resemble the screens, dials, and interaction of this medium. Interestingly, the characters, must leave the room to talk to the ship through a radio interface, though, since “talking” through your television wasnʼt seen as a possibility.

By the 1980s, the personal computer is now common, and though the director of Jurassic Park must make some references to specific interactions (such as the mouse and file system), audiences are now able to understand and accept this medium, allowing the young girl to save the day.

Inspiration: Viewers like what they see and seek to replicate it in real world In the 1990s, when Douglas Caldwell, with the U.S. Army Topographic Engineers, saw a 3D map system in the film, X-Men, he realized it was a novel solution to an age-old challenge: how to represent any relevant topography in 3D in the battlefield.

His departmentʼs request for proposals for development of something similar yielded the Xenotran Mark II Dynamic Sand Table, including the improvement of a smooth surface and overhead projection.

Lesson: Sci Fi is a powerful cultural influence. It affects designer's ideas as well as those of our clients and audiences Expectation: Viewers see things in SciFi they begin to expect. Expectations are set when technologies are shown as desirable for their form or their function. Where's my jetpack? Function: Robots are a SciFi staple and have influenced the spending of millions, if not billions of dollars on human-like machines that, effectively, create an ethics-free slave class. This, despite the fact that robots with clear industrial use donʼt require any human reference points in representation, behavior, or movement.

GORT from The Day the Earth Stood Still

Delivery robot from I, Robot

Hondaʼs ASIMO Form: What audiences experience influences the desired forms and expectations of new devices (theyʼve seen before in SciFi). It also influences engineers and developers who are, often, SciFi fans themselves. Consider the juxtaposition of Star Trekʼs communicator with the Motorola StarTac exactly 30 years later:

Lesson: Users may already be predisposed to certain interface solutions solely based on what theyʼve seen in the media. These solutions may be more comfortable for them than other alternatives. Social Context: Reminders of limitations and constraints of who we are as people, who we relate to each other and how we relate to technology Anthropomorphism influences expectations of human behavior and sentience. This plays-out in several ways.

Microsoftʼs “Clippy,” “BOB,” and Ms. Dewey (msdewey.com) Most of the time, when designers flirt with anthropomorphized elements in the interface, it isnʼt successful. Either the technology or feature doesnʼt live-up to the expectations suggested by the representation or the “characterʼs” behavior is just annoying. The mechanisms that allow anthropomorphism to work are well defined and researched (see the work of Nass and Reeves,

and their book The Media Equation) and legitimate. However, simply adding a human (or animal or alien) character is a surface treatment that doesnʼt relate to the systemʼs behavior. There are examples of where this is done, effectively. For example, Appleʼs future scenario, Knowledge Navigator, uses an anthropomorphized character in the computer to provide a realistic level of function with a minimum of annoying behavior.

Lesson: Though it's been clearly shown that socially appropriate interfaces can aid learning, speed use, and make more comfortable interfaces (Nass and Reeves), when this is merely as a visual adornment, it often fails. Lesson: Wading into the social context is more tricky than it looks A fully human-like representation isnʼt necessary to see the anthropomorphistic effect. Sound, alone, can often suggest the expectations of human-ness or sentience. In the case of K.I.T.T., the talking car in the TV series Knight Rider, almost the entirety of this effect is accomplished through K.I.T.T.ʼs voice.

Likewise, audiences infer greater capabilities to the Enterpriseʼs computer system in the TV series Star Trek because of the quality of human-like representation in the voice.

In the Star Wars films, one of the most endearing characters in the franchise, R2-D2, is not able to speak at all and his behaviors and movements are limited. Instead, his sounds are enough for audiences to relate to and assume human-like sentience and feeling.

Lesson: Sound is enough to trigger anthropomorphism. Human-like sound and visual appearance isnʼt even necessary for anthropomorphism to take place. Behavior is often the mechanism for expectations of human-comparable function and behavior to exist. For example, Amazonʼs OneCLick™ is devoid of any animate references yet functions much like a favorite shop keeper or bar tender would (welcome you back, getting you your “usual,” trusting your choices, and handling the “details” of your transaction).

Amazonʼs OneClick™ Lesson: Anthropomorphism works through behavior as well as form. The degree of representation often affects the degree of expectations of human-like understanding and behavior in the system. The director of Until the End of the World could have

simply represented a computerʼs search function as a standard computer element. But, by giving it a character and animate representation, it inferred greater capability than a standard service.

Bounty Bear from Until the End of the World Likewise, in the film Matrix, system functions (programs) are represented as fully human characters, imparting greater impact, depth, and danger to the audience than a standard program representation would. Therefore, the hunt and destroy program, represented as Agent Smith, is felt as more dangerous and capable than one would expect form a program.

Likewise, the prediction program, The Oracle, seems more powerful and accurate represented as a person than we would expect from a collection of “code.”

Lesson: The more "human" the representation, the higher the expectations of behavior. When we cast technologies or processes in anthropomorphic ways, it raises expectations about the extent of their behavior. In SciFi this doesn't, necessarily, have to only be human (animals, aliens, etc.) but the effect is the same.

Proposed Paradigm: We can use SciFi as a kind of scenario to evaluate the interface/interaction and find potential solutions and lessons. SciFI often serves as a simple reminder of constraints and affordances (and their role) that we already understand. For example, in The Fifth Element, we find affordances taking the place of a manual for a device that was last used thousands of years ago.

Likewise, we know the frustration and confusion caused by interfaces without clear affordances or constraints, as in the humorous “interface play” Lifted.

Lesson: Designing constraints and affordances into the interface helps users quickly understand an interface and get by without the manual. Weʼve also found examples of bad solutions, such as when input devices should recognize the affective state of their users and adjust their behavior and reaction accordingly. In The Fifth Element, a choking character mashes at his technological desk's keyboard, triggering all sorts of inappropriate actions.

Instead, an example of the system ignoring identical inappropriate or ambiguous input can be found in 2001, where a childʼs bashing of the teleconferencing systemʼs buttons doesnʼt interfere with the call.

Lesson: Input devices should recognize the affective state of their users and adjust accordingly. This is especially important for some users (such as children and seniors), some situations (such as emergencies or critical/dangerous functions), and some contexts (like mobile or portable devices vs. desktop or ubiquitous, stationary ones). Weʼve found surprising cases where paradigms suggest solutions to interface problems. These examples allow us to see what solutions may work in improving situations—and even test them. In Star Wars, there is almost always a correlation between social hierarchy and the size of represented users in holography. For example, the social hierarchy of the “Empire” is nearly always represented during holographic conversations by scaling the projections to represent social status. In this example, though he is one of the highest leaders in the Empire, Darth Vader is still dwarfed by the projection of the Emperor, his superior. In other scenes, when Darth Vader addresses his subordinates, he is almost always projected bigger or higher than them.

In contrast, the Jedi councilʼs social hierarchy is one of equality. So, when they meet with mixed presence, even the older and more revered characters, are all portrayed at the same scale.

Lesson: Whenever there is a social context in the interface (which is often these days), whether it is a social network, video chat, community site, buddy list, etc.), social hierarchy is engaged (and might be appropriately represented). Our systems can make this hierarchy apparent when this is useful through scale.

Sometimes, the lessons are new and particular to the given technology. This is when Science Fiction most acts as a prototype for the designers to learn new things about it. In the case of Minority Report, the famous gestural interface is mostly presented as a powerful, deeply engaging tool for manipulating and investigating a huge amount of video and related data. The film bypasses the problems inherent in such systems such as fatigue, but does make a nod to another problem. When Anderton is introduced to Witwer during a scrubbing session, he turns to shake hands, he inadvertently wipes all of the content from his screen. This small bit of interface humor points out that users of future systems will need some explicit way to engage and disengage the system, enabling them to do other things with their bodies, such as attend to social or biological needs. Lesson: What works for audiences often works for users as well. Last Lesson: Watching Science Fiction can be beneficial to your career! Conclusions This is only the beginning. There's a lot more lessons we're learning and a lot more properties to explore. Look for a book on this material sometime next year and check back at the website for more updates: scifiinterfaces.com Send questions and inquiries to Nathan Shedroff (Nathan@nathan.com) and Chris Noessel (chrisnoessel@gmail.com).