Make It So: What Interaction Designers can Learn from Science

Fiction Interfaces Presentation Notes, Nathan Shedroff and Chris Noessel

4 September 2009, dConstruct 09 Conference, Brighton, UK

This is the first presentation of only a portion of the material we've found in our analysis of

Science Fiction films and television series. We!re also looking a industry future films (like Apple's

Knowledge Navigator) as well as existing products and research projects. Our analysis includes

properties (films and TV), themes (different issues in interface design), as well as the historical

context of the work (such as the current technology of the time of the property!s release). In

addition, we!re interviewing developers (including production designers from films) but this

material isn!t presented in this talk. For this presentation, we!ve focused on the major issues, part

academic and theoretical, and part lessons (more practical) we!ve uncovered.

How design influences SciFi and how SciFi influences design:

We've chosen to focus on interface and interaction design (and not technology or engineering).

Some visual design issues relate but, mostly, in this talk, we!re not approaching issues of styling.

We!ve chosen the media of SciFi (TV and films) because a thorough analysis of interaction

design in SciFi requires that the example be visual so interfaces are completely and concretely

represented, include motion that describe the interaction, and (sometimes) has been seen by a

wide audience.

Scientifically determining “influence” in any context (whether from Design on SciFi or visa versa)

is difficult, and much of what we illustrate is inference on the part of the authors.

Design influences SciFi: Design (reality) sets the paradigm that scifi builds upon

In turn, Sci Fi influences design in one of four ways: Inspiration, Expectation, Social Context, and

Proposed Paradigm.

Science Fiction relies on the context set by the design and development of existing products. This

allows SciFi to communicate effectively with its audience. For example, during the Industrial Age,

when the first SciFi film was created, Le Voyage Dane La Lune, the metaphors used were

industrial and had no interfaces. To open the door, you pushed the door open. While this

principally reflected the whimsical and theatrical nature of the film, it also reflected a lack of

“interfaces” in the world at the time.

By the time Fritz Lang directed Metropolis, we see an attempt to realistically present a vision of

the future that builds upon the audience's awareness of the telegraph, tickertape, radio, and

telephone technologies

As depicted in the scene, to control the screen, the user “checks messages” via a tickertape,

“tunes” in the video like a radio, and talks via “telephone” to his employee.

However, by the time Buck Rogers appears on screen, the audience is now familiar with

electronic television, allowing SciFi interfaces to resemble the screens, dials, and interaction of

this medium. Interestingly, the characters, must leave the room to talk to the ship through a radio

interface, though, since “talking” through your television wasn!t seen as a possibility.

By the 1980s, the personal computer is now common, and though the director of Jurassic Park

must make some references to specific interactions (such as the mouse and file system),

audiences are now able to understand and accept this medium, allowing the young girl to save

the day.

Inspiration: Viewers like what they see and seek to replicate it in real world

In the 1990s, when Douglas Caldwell, with the U.S. Army Topographic Engineers, saw a 3D map

system in the film, X-Men, he realized it was a novel solution to an age-old challenge: how to

represent any relevant topography in 3D in the battlefield.

His department!s request for proposals for development of something similar yielded the Xenotran

Mark II Dynamic Sand Table, including the improvement of a smooth surface and overhead

projection.

Lesson: Sci Fi is a powerful cultural influence. It affects designer's ideas as well as those of our

clients and audiences

Expectation: Viewers see things in SciFi they begin to expect. Expectations are set when

technologies are shown as desirable for their form or their function.

Where's my jetpack?

Function: Robots are a SciFi staple and have influenced the spending of millions, if not billions of

dollars on human-like machines that, effectively, create an ethics-free slave class. This, despite

the fact that robots with clear industrial use don!t require any human reference points in

representation, behavior, or movement.

GORT from The Day the Earth Stood Still

Delivery robot from I, Robot

Honda!s ASIMO

Form: What audiences experience influences the desired forms and expectations of new devices

(they!ve seen before in SciFi). It also influences engineers and developers who are, often, SciFi

fans themselves. Consider the juxtaposition of Star Trek!s communicator with the Motorola

StarTac exactly 30 years later:

Lesson: Users may already be predisposed to certain interface solutions solely based on what

they!ve seen in the media. These solutions may be more comfortable for them than other

alternatives.

Social Context: Reminders of limitations and constraints of who we are as people, who we

relate to each other and how we relate to technology

Anthropomorphism influences expectations of human behavior and sentience. This plays-out in

several ways.

Microsoft!s “Clippy,” “BOB,” and Ms. Dewey (msdewey.com)

Most of the time, when designers flirt with anthropomorphized elements in the interface, it isn!t

successful. Either the technology or feature doesn!t live-up to the expectations suggested by the

representation or the “character!s” behavior is just annoying. The mechanisms that allow

anthropomorphism to work are well defined and researched (see the work of Nass and Reeves,

and their book The Media Equation) and legitimate. However, simply adding a human (or animal

or alien) character is a surface treatment that doesn!t relate to the system!s behavior.

There are examples of where this is done, effectively. For example, Apple!s future scenario,

Knowledge Navigator, uses an anthropomorphized character in the computer to provide a

realistic level of function with a minimum of annoying behavior.

Lesson: Though it's been clearly shown that socially appropriate interfaces can aid learning,

speed use, and make more comfortable interfaces (Nass and Reeves), when this is merely as a

visual adornment, it often fails.

Lesson: Wading into the social context is more tricky than it looks

A fully human-like representation isn!t necessary to see the anthropomorphistic effect. Sound,

alone, can often suggest the expectations of human-ness or sentience. In the case of K.I.T.T., the

talking car in the TV series Knight Rider, almost the entirety of this effect is accomplished through

K.I.T.T.!s voice.

Likewise, audiences infer greater capabilities to the Enterprise!s computer system in the TV

series Star Trek because of the quality of human-like representation in the voice.

In the Star Wars films, one of the most endearing characters in the franchise, R2-D2, is not able

to speak at all and his behaviors and movements are limited. Instead, his sounds are enough for

audiences to relate to and assume human-like sentience and feeling.

Lesson: Sound is enough to trigger anthropomorphism.

Human-like sound and visual appearance isn!t even necessary for anthropomorphism to take

place. Behavior is often the mechanism for expectations of human-comparable function and

behavior to exist. For example, Amazon!s OneCLick™ is devoid of any animate references yet

functions much like a favorite shop keeper or bar tender would (welcome you back, getting you

your “usual,” trusting your choices, and handling the “details” of your transaction).

Amazon!s OneClick™

Lesson: Anthropomorphism works through behavior as well as form.

The degree of representation often affects the degree of expectations of human-like

understanding and behavior in the system. The director of Until the End of the World could have

simply represented a computer!s search function as a standard computer element. But, by giving

it a character and animate representation, it inferred greater capability than a standard service.

Bounty Bear from Until the End of the World

Likewise, in the film Matrix, system functions (programs) are represented as fully human

characters, imparting greater impact, depth, and danger to the audience than a standard program

representation would. Therefore, the hunt and destroy program, represented as Agent Smith, is

felt as more dangerous and capable than one would expect form a program.

Likewise, the prediction program, The Oracle, seems more powerful and accurate represented as

a person than we would expect from a collection of “code.”

Lesson: The more "human" the representation, the higher the expectations of behavior. When

we cast technologies or processes in anthropomorphic ways, it raises expectations about the

extent of their behavior. In SciFi this doesn't, necessarily, have to only be human (animals, aliens,

etc.) but the effect is the same.

Proposed Paradigm: We can use SciFi as a kind of scenario to evaluate the

interface/interaction and find potential solutions and lessons.

SciFI often serves as a simple reminder of constraints and affordances (and their role) that we

already understand. For example, in The Fifth Element, we find affordances taking the place of a

manual for a device that was last used thousands of years ago.

Likewise, we know the frustration and confusion caused by interfaces without clear affordances or

constraints, as in the humorous “interface play” Lifted.

Lesson: Designing constraints and affordances into the interface helps users quickly understand

an interface and get by without the manual.

We!ve also found examples of bad solutions, such as when input devices should recognize the

affective state of their users and adjust their behavior and reaction accordingly. In The Fifth

Element, a choking character mashes at his technological desk's keyboard, triggering all sorts of

inappropriate actions.

Instead, an example of the system ignoring identical inappropriate or ambiguous input can be

found in 2001, where a child!s bashing of the teleconferencing system!s buttons doesn!t interfere

with the call.

Lesson: Input devices should recognize the affective state of their users and adjust accordingly.

This is especially important for some users (such as children and seniors), some situations (such

as emergencies or critical/dangerous functions), and some contexts (like mobile or portable

devices vs. desktop or ubiquitous, stationary ones).

We!ve found surprising cases where paradigms suggest solutions to interface problems. These

examples allow us to see what solutions may work in improving situations—and even test them.

In Star Wars, there is almost always a correlation between social hierarchy and the size of

represented users in holography. For example, the social hierarchy of the “Empire” is nearly

always represented during holographic conversations by scaling the projections to represent

social status. In this example, though he is one of the highest leaders in the Empire, Darth Vader

is still dwarfed by the projection of the Emperor, his superior. In other scenes, when Darth Vader

addresses his subordinates, he is almost always projected bigger or higher than them.

In contrast, the Jedi council!s social hierarchy is one of equality. So, when they meet with mixed

presence, even the older and more revered characters, are all portrayed at the same scale.

Lesson: Whenever there is a social context in the interface (which is often these days), whether

it is a social network, video chat, community site, buddy list, etc.), social hierarchy is engaged

(and might be appropriately represented). Our systems can make this hierarchy apparent when

this is useful through scale.

""Sometimes, the lessons are new and particular to the given technology. This is when Science

Fiction most acts as a prototype for the designers to learn new things about it. In the case of

Minority Report, the famous gestural interface is mostly presented as a powerful, deeply

engaging tool for manipulating and investigating a huge amount of video and related data. The

film bypasses the problems inherent in such systems such as fatigue, but does make a nod to

another problem. When Anderton is introduced to Witwer during a scrubbing session, he turns to

shake hands, he inadvertently wipes all of the content from his screen. This small bit of interface

humor points out that users of future systems will need some explicit way to engage and

disengage the system, enabling them to do other things with their bodies, such as attend to social

or biological needs. "

Lesson: What works for audiences often works for users as well.

Last Lesson: Watching Science Fiction can be beneficial to your career!

Conclusions

This is only the beginning. There's a lot more lessons we're learning and a lot more properties to

explore.

Look for a book on this material sometime next year and check back at the website for more

updates: scifiinterfaces.com

Send questions and inquiries to Nathan Shedroff (Nathan@nathan.com) and Chris Noessel

(chrisnoessel@gmail.com).