heler martin multimedia taxonomy

7/24/2019 Heler Martin Multimedia Taxonomy

http://slidepdf.com/reader/full/heler-martin-multimedia-taxonomy 1/22

Using a Theoretical Multimedia TaxonomyFramework

RACHELLE S. HELLER, C. DIANNE MARTIN, NUZI HANEEF, and SONJA GIEVSKA-KRLIU

The George Washington University

Multimedia (MM) is a polysemous term, a term with many definitions, and in this case, manyroots. In this paper, multimedia is defined as the seamless integration of two or more media.Each ancestor brings another requirement, muddying the field and making it difficult to workthrough. A multimedia taxonomy based on a previous media taxonomy is proposed to helporganize the discipline. The taxonomy helps to classify the space called multimedia and to

draw attention to difficult issues. The paper outlines the forms contributing to multimedia—text, sound, graphics, and motion—and aligns them with probable formats—elaboration,representation, and abstraction—and sets them within a context—audience, discipline, inter-activity, quality, usefulness, and aesthetics. The contexts are more clearly defined in twoareas: interactivity and the information basis for a discipline. Examples are presenteddescribing the use of the taxonomy in the design and evaluation of student projects in acomputer science-based multimedia course.

Categories and Subject Descriptors: H.5.0 [Information Interfaces and Presentation]:General

General Terms: Design, Theory

Additional Key Words and Phrases: Computer uses in education, evaluation, multimedia

1. INTRODUCTION: WHAT IS MULTIMEDIA?

Multimedia (MM) is a polysemous term—a term with many definitions, andin this case, many roots. In this paper, multimedia is defined as theseamless integration of two or more media. If two or more media areattached to each other, but not in a seamless way, we refer to them asmultiple media. Depending on its roots, multimedia takes on differentcharacteristics. If the root is education, then the focus of the multimedia

discussion is the delivery of education, and the media are analyzed in termsof their effectiveness in delivering information. If the root is humancomputer interaction, the focus is the interface and the navigation afforded

Authors’ address: The George Washington University, Washington, DC 20052.Permission to make digital/hard copy of part or all of this work for personal or classroom useis granted without fee provided that the copies are not made or distributed for profit orcommercial advantage, the copyright notice, the title of the publication, and its date appear,and notice is given that copying is by permission of the ACM, Inc. To copy otherwise, to

republish, to post on servers, or to redistribute to lists, requires prior specific permissionand/or a fee. © 2001 ACM 1531-4278/01/0300 –0004 $5.00

ACM Journal of Educational Resources in Computing, Vol. 1, No. 1, Spring 2001, Article #4, 22 pages.



by the use of media. Similarly, if the root is computer graphics, then all of the issues inherent in computer graphics—issues such as frame rate,refresh, and lighting—are applied to multimedia. Each ancestor bringsanother requirement, introducing more complexity in the field and making

it difficult to work through. Every designer wants to build a high-qualitysystem, but actually doing so and evaluating it is a difficult task given thepolysemous context.

We propose a multimedia taxonomy to help organize the discipline on thebasis of a previous media taxonomy [Heller and Martin 1995]. The taxon-omy helps to classify the space called multimedia and helps to drawattention to difficult issues. While multimedia is an emerging field, the factthat we can provide a taxonomy indicates that it has reached some level of stability and maturity. Due to our outlook, we can see the directionsimposed on the field by its many roots. As we classify the field, we also

move forward to a deeper analysis.

2. THE TAXONOMY

Multimedia taxonomy can be visualized as a three-dimensional matrix(Figure 1). Media type pertains to the various media involved: text, sound,images, motion, and multimedia. Media expression, i.e., elaboration, repre-sentation, and abstraction, refers to the level of abstraction using themedia. Context, the third dimension, reveals the various roots of multime-dia: disciplines, interactivity, audience, aesthetics, quality and usefulness.

Media type is arranged as a series of individual media of increasing complexity (i.e., storage). The various media should be clear. Text is thepresentation of information using an alphabetical symbol system. Thisincludes prose in various languages as well as presentations in such formsas mathematics or other symbol systems. Sound includes spoken words aswell as generated tones forming music or other audible information. Imagesinclude photos and hand-drawn items, while motion can be motion picturesor animation. Multimedia, as defined above, is the combination of any twoor more of these.

The media expression categories require some explanation. These move

from concrete to abstract. Elaboration is where no information is editedout. So that if we are examining a textual presentation of a speech, theelaboration presentation would include the fully rendered speech. An imagewould be the complete image of something like a room with all the details.Elaboration presentations are the most concrete and require the least efforton the part of the user in decoding the symbol set. This does not mean thatthe level of information is simple, but that the medium is not hiding orsubsuming any information. Representational categories provide some ed-iting or ellipses of information. For example, in the textual category, thespeech might be represented as a series of bullets or outlines or evenpower-point slides, while the graphic image might be a blueprint of theroom. The following questions then arise: How does one classify a com-pressed image? Is it elaboration or representation? One answer is to

2 • R. S. Heller et al.

ACM Journal of Educational Resources in Computing, Vol. 1, No. 1, Spring 2001.



compare the question to one of lossy or lossless compression. Finally, theabstraction category is the abstract presentation of information, ofteniconic in form. This category is related to metaphor. For example, if thespeech is about acid rain, the text as abstraction might be written as falling rain in the style of the French poet, Apollinaire. Such use of text is notnew—illuminated manuscripts often used micrography to present informa-tion in text formed into patterns. Information about the room could bepresented in a icon showing merely sketches of the room.

The final dimension, context, is an attempt to respond to the fact thatinteractive multimedia applications do not exist in a vacuum. They are

expressions that are meant to be used by an audience within a topic area ordiscipline, and they are judged well done, or not, depending on criteria of usefulness, quality, and aesthetics. These measures reflect the polysemousnature of multimedia. Education has claims on multimedia, so purpose,interactivity, and learning become integral to it; and since art also claimsmultimedia, aesthetics is added; multimedia is also claimed by technicalfields like signal processing and computer science, so issues of quality areincluded. For example, the category “quality” refers to the technical aspectsof application delivery, and might be described by the attributes of clarityof presentation or fidelity of reproduction. Appropriate concerns within thiscategory might include clarity of characters on the screen as well as thetechnical synchronization of two media. Additionally, using the attributeswithin this category as part of an evaluation protocol will prompt an

(increasing

complexity)

(increasing abstraction)

MEDIA TYPE

MEDIA EXPRESSION

CONTEXT

Multimedia

Motion

Graphics

Sound

Text

E l a b o r a t i o n

R e p r e s e n t a t i o n

A b s t r a c t i o n

Audience

Discipline

Interactivity

Quality

Usefulness

Aesthetics

Fig. 1. Original multimedia taxonomy.

Theoretical Multimedia Taxonomy Framework • 3




evaluator to ask questions like: Is the motion of the lips synchronized to thespeech? Table I contains a sampling of questions that might be promptedby the quality dimension in the multimedia taxonomy.

3. USING THE TAXONOMY

The multimedia taxonomy can be used for design and evaluation of multi-media products and applications. The taxonomy aids brainstorming . Using it for design allows implementers, as they develop new materials, to

consider the available media forms and formats. Designers often considerissues of metaphor, interaction, cognitive accessibility, and the use of media. For example, in an application on a visit to the island of Goa (anisland community in India), the designer wanted to present most of theinformation using text. However, textual presentations are often boring,dense, and not attractive to the user, especially to someone browsing through a lot of material. Using the multimedia taxonomy, it becomes clearthat text does not have to consist of full sentences. It can be used as a diarywith printed notes and a series of post-its, underlines, and marginalia toindicate additional ideas. The diary is lively, and the information is given

in its entirety (Figure 2). In another application, in an attempt to presentthe history of one of the oldest churches in Detroit, the applicationcombines church bells, choir music, and a voice-over replete with echoesreminiscent of a thundering preacher. In an application on a visit to theMayan culture, using a graphical map as a navigational guide and agraphic of a Mayan leader to indicate the source of the information allowsthe developer to provide multiple avenues of information for a user.

In the Cite d’Arte presentation included in this paper, the studentcreator uses the metaphor of a magazine to present the place called Paris.He uses various forms of text to get his metaphor across, including fulltextual descriptions as well as representational text for tables of contentsand shorter descriptions of wine making. While his graphics are not alwaysof high artistic quality, his series on wine making uses three simple,

Table I. Sample Questions from the Quality Dimension

Media Form ElaborationMedia Expression

Representation Abstraction

Text Word choiceunclear/clear Math functions correctFont for bold and italicsreadable

Sound Speech capturedwithout stutters

Sound effects recognizable

Graphics Image details clear Quality of image (low to high)Size of image (too large/smallto just right)

Icons are clearIconic metaphor issharp

Motion Quality of motion(jerky to smooth)

Multimedia Media technicalsynchronization



http://final31.exe/

http://final31.exe/



representational drawings to show the process. While he could have used afull motion video sequence to portray this process, in keeping with themagazine metaphor, he chose the animation sequence. The student alsouses a variety of sound, speech, and music. The opening sequence includesa waltz-like score with animated text to introduce the CD. In all, thestudent uses various aspects of the taxonomy to provide a variety of mediapresentations and yet ensure the consistency of the message.

To use the taxonomy as an aid in evaluation, it is best to reviewevaluation as a stage setting for using the taxonomy. Experimental oppor-tunities, field evaluations, system testing, comparative testing, expertevaluations, and checklists are typical formative and summing-up evalua-tion experiences. Each can be enhanced by using the multimedia taxono-my’s form and format. Controlled experiments can be enhanced as they askquestions about particular media expressions, for example: Is this colorbetter (or worse) than that color? In the more open field evaluations, we canuse the taxonomy to help develop questions related to the purpose or theusers. Wu and Martin [1997] used the taxonomy in their field study of

media preferences. By observing the taxonomy categories, they were able todesign an experiment in which they asked shoppers in a mall how theypreferred to receive information. Reeves and Harmon [1993]) provide achecklist for multimedia evaluation that fits the taxonomy well, and using the taxonomy can be expanded or made more specific. In asking aboutaesthetics, they offer an analysis continuum from pleasing to displeasing asan evaluation check. The taxonomy suggests a series of additional catego-ries for aesthetic evaluations.

When using the taxonomy as an evaluation tool, the multimedia categoryis the richest. It allows us to reflect on the difficult issues of synchroniza-tion. For example: Do text and images synchronize temporally and spatiallyand well as cognitively? For two media to synchronize temporally, we wantto ask if they are timed to appear at the right moment. Text could appear

Fig. 2. Use of text based on taxonomy.





before, at the same time, or after an image (depending on the designer).Spatial synchronization involves asking about the physical layout of two ormore media: Is the text positioned correctly to the right (or left, or above orbelow) the image? Positioning could be related to style as required by an

outside source or by ease of use for a reader of left to right or right to leftlanguages. Cognitive synchronization can be thought of in terms of howwell we matchi the intellectual level of the material presentation with theaudience’s receptive ability.

Originally, the taxonomy had only three categories in media expres-sion.At the start of the work on evaluation, it was observed that questionsoften repeated themselves, regardless of the form of medium or expressionformat. For example, questions about text size are appropriate no matterthe test’s media expression. Based on this observation, the taxonomy wasexpanded to provide a category called general in the format. This category

is used for questions that are constant over the format space. The newlyexpanded taxonomy becomes a floor, not a ceiling, for a series of guidelinesthat can be used to generate a series of questions about an application.Contextual categories that are not applicable to the specific product being evaluated can be eliminated from the protocol. The questions generatedcould be framed either as a Likert scale question with five gradations oropen-ended questions requiring a free-form user response. For example, toevaluate use of color, one can generate a question for the form:

Not Pleasing Pleasing

Use of color 1................2............... 3............... 4.............. 5

The discipline aspect of the context is considered here. Selecting themedia and presentation techniques for the multimedia presentation dependto a large extent on the information to be communicated. Given theinformation content, it is the designer’s job to choose the most effective wayof conveying the important messages of the multimedia product. Contentmay be categorized, analyzed, and represented in different ways for differ-ent purposes. There are often many methods to present a certain kind of information, but there is no general guidance that can help designers toselect the media. In other words, better understanding of the information’scharacteristics, their relation to the characteristics of the media, and theidentification of users and tasks will help multimedia developers to makecritical decisions. For example, the presentation design will vary depending on whether we are dealing with physical entities in three-dimensionalspace or with abstract and logical entities in arbitrary, cognitive space.What would the designer’s choice be if the relations among the data in theset are temporal, spatial, or some particular kind of semantic relationship?

A research-based categorization of different types of data across manydisciplines is presented to help designers, developers, and evaluators of multimedia software. Such a categorization must identify the data charac-teristics that may impact the effectiveness with which a given mediumexpresses the information. The system should be as comprehensible as





possible, so that any number of other categories could be fit into thesystem.

Selecting the way to convey information does not only depend on the typeof information. In cases where more than one medium and more than one

presentation technique could be used, other factors will determine the finaldecision. The intent of the communication, user characteristics, and thetasks that the presentations are designed to support are important issuesas well. Many design decisions will be affected by factors like hardware andsoftware characteristics, conflicts in coordination imposed by multiplemedia, and time and space restrictions in the environment where theinteraction is to take place. Cross-factor and more comprehensive analysesare needed to find the answers for many design decision-making processes(e.g., media selection, presentation mode, and content selection) by taking all the relevant issues and their interdependencies into account.

4. CONTENT CHARACTERISTICS

It is a great challenge to identify the general information characteristicscritical for multimedia design. How can we define and characterize uniqueproperties of different types of information across many disciplines andapplications? Is there any existing classification that we can use formultimedia development? Should we have a holistic approach to thecontent or try to categorize smaller content units or even data entities? Theparallel between MM designers and painters given in Reiser and Reiser

[1995] could be the answer. Painters look at each detail separately and alsostep back to look at the composition as a whole. Designers, just likepainters, may find it difficult to not focus on details, and hence to forget thepurpose of those details in conveying the message as a whole. Thus, thedescription must identify general categories of data objects as well asrelationships among them that may affect the decision-making process.

A review of the classification systems already in existence may suggestsome of the information characteristics worth considering. Several researchareas address the problem of classifying information content. Perhaps themost systematic of these in terms of usefulness for multimedia develop-

ment are the systems in artificial intelligence (AI). Characterizing informa-tion is a prerequisite to defining the expressiveness of presentation tech-niques and automating the process of technique selection in intelligentmultimedia user interfaces [Feiner 1991; Feiner and McKeown 1993;Maybury 1992; Roth et al. 1993]. The type of information or the nature of the content was identified as one of the characteristics important for themedia-selection process. Various classifications of information types havebeen proposed [Andre et al. 1993; Roth et al. 1993; Arens et al. 1991], butthere is no general classification that could be used across many disciplinesand applications. Some of the differences among the content type categoriesin these systems are due to the characteristics of the discipline or domain-specific information the investigators were concerned with. The differencesin the classification systems were also affected by the different applications/





tasks for which the systems were devised. Many categories proposed in the AI literature seem to fit fairly well into the categories of our taxonomy,although few appeared to be examples of one category.

An exhaustive body of the literature on content can also be found in the

area of instructional design. Here the analysis of the content is orientedmore toward educational objectives. The content structure and organizationare discussed in terms of creating interactive and effective learning envi-ronments that can accommodate different learning styles and teaching approaches, and the impact on the content structure and organizationwhen applied to instructional design, in this case content structure, isdriven by the application as well as the discipline [Jonassen 1988; 1991;Mayer and Sims 1994]. Learning theory has given a lot of attention to thecognitive processes in the reception and organization of information [Ten-nyson 1977; Gregg and Farnham-Diaggory 1975]. The implication of this

research is relevant for future comprehensive analyses in multimediataxonomy when complex interdependencies of many categories and at-tributes (e.g., content type, perceptual problems, users’ cognitive capabili-ties and limitations, their previous knowledge of the content being taught,users’ goals, intent, and communication environment) will be analyzed.

5. INFORMATION TYPE TAXONOMY

Information type appears to be a necessary, or at least a useful, dimensionin categorizing content information. The proposed categorization should be

considered as a first step in the development of a content taxonomy.Information type taxonomy is presented on Table II. As with any aspect of a taxonomy, that of information type can be expected to expand with use.

The taxonomy appears to be general and applicable to different units of information—from single data objects, to sets with different relationalconnections, to classes, and even systems. It is built upon the foundation of language (e.g., objects (nouns, abstract or compound actions), verbs, etc.), acommonsense view of several application domains, and already existing classifications found in the literature. In an attempt to find a suitable wayto present the relationship between the information type and the media

taxonomy, a three-point scale (*, **, and ***) is chosen to rate theappropriateness of different media for presenting different types of infor-mation [Heller and Martin 1995; 1998].

The effectiveness with which each medium achieves its purpose varies.Each medium makes use of different skills and capabilities. Graphics mightbe preferred to text when presenting concrete information, which involvesphysical objects, persons or places with certain visual properties like shape,size, or color. Images convey information in ways different from, andsupplementary to, text.

It is a well-known fact that text is the most appropriate medium forpresenting abstract concepts and logic [Berinstein 1997; Roth et al. 1993].Still, there are few well-established symbols and icons that express ab-stract information and can be used for some abstract objects or terms.





Conceptual graphs are a good vehicle for presenting abstract concepts andlogic. Text should be used to convey quantitative and relational facts,although some graphic expressions like charts, tables, and graphs could beused for large numbers of quantitative or relational facts.

Spatial information shows where things reside in relation to one anotherin space or explain how we can get from one place to another. Locationinformation and composition of objects are best depicted by pictures and

Table II. Information Type Taxonomy

Media

Type

Information Type

Concrete

Physicalobjects,

persons,

places

Abstract

Abstractconcepts

Spatial

Location,spatial

relationships

between

objects

Temporal

Temporalrelationships

Quantitative

Quantifyinginformation,

relational

facts

Covariant

Semanticrelationship

between

different

pieces of

information

Text * ** *** *** **Examples Description

of an objectDefinitionof psychology

Timeline of events

Distancesbetweensites inmiles

Mathproblem andsolution

Sound * ** *** **

Examples Descriptionof an object

Speakergiving/telling definition of psychology

Speakerreporting

Speakerreporting productiongrowthfigures

Speakerreporting causes of anenvironmentaldisaster andits effects

Graphics ** *** ** ** **Examples Picture of

an objectMap of anarea; pictureof furnishedroom

Pictures of events,statesdisplayedfrom top to

bottom (orleft to right)

Bar chart of distancesbetweencities

Picture of aplace beforeand after anevent

Motion ** *** ** **Examples Video

sequence of an object,place orperson

Video clip of a room withzooming effectsdepicting each object

Videoshowing chronologicalorder of events

Animatedgraphs of productiongrowth of aproduct

Videosequence of an actionand itseffects

Multimedia *** *** *** *** *** ***Examples Combination

of graphics

and text

Combinationof graphics

and text orsound


and text, ormotion andsound

Anyappropriate

combination


and text

Anyappropriate

combination

Appropriateness: low * medium ** high ***





maps [Andre et al. 1993]. Actions and movements usually have a spatialattribute, too. Text or sound could be added for completeness.

Temporal specifications and temporal relations among states, events, oractions could be communicated by text. The sequential nature of text has

been exploited to tell stories and the history of events. Graphics can also beused for temporal information, but has limited applicability. Overlapping events and certain time specifications (e.g., mostly, periodically, or in the

future) are hard to express using only graphics [Andre et al. 1993]. Theorganizational scheme may reflect different functionalites such as taskcharacteristics, domain knowledge relationships, or logical ordering andconnections to accommodate user and designer goals. In addition, manyunits could be decomposed into more granular entities that do not necessar-ily belong to the same scheme. For example, information about the UnitedStates could be divided geographically (e.g., east coast, west coast) and

then presented in alphabetical order (e.g., their names) or numerical order(e.g., area size).

Different semantic relationships between data objects (e.g., cause/effect,logical ordering, and problem/solution) that must be presented as one pieceof information, called covariant information. The selection of the mediahere depends on the nature of the data we are dealing with and theirrelationships. Some pieces of information may fall into more than onecategory. For example, some of the covariant data types (e.g., cause/effect,and action/result) have spatial and/or temporal attributes. It became ap-parent that in trying to fit specific information types into the media

taxonomy, some information characteristics are described differently. Thus,this classification scheme contains some nonexclusive categories. The pre-sentation of artistic types of data (e.g., poems, songs, movies, dramas orballets, performance, sculpture, and painting) usually depend on the origi-nal art medium. They could be represented as a separate category or mayfall into one of the existing categories (e.g., painting is concrete; ballet isspatial; poem is abstract).

When the CD for Cite d’Arte was being created, the types of informationdescribed in the taxonomy provided guidance on presentation. The presen-tation includes concrete and abstract information about the painters’ lives

as well as their motifs. The references to the wine producing sections of France revolve around spatial and temporal information is addressed in thesequencing of wine production. There was an attempt, guided by thetaxonomy, to present covariate information in directing wine choices.

6. ORGANIZING CONTENT

People organize information in order to understand, to explain, or to controlpresentations. The content structure and organization influences the waypeople comprehend the information presented [Rosenfeld and Morville1998]. Structuring (i.e., segmentation or “chunking”) refers to partitioning the content into discrete units that are linked for the presentation. Thisinvolves a selection of the pattern (i.e. organizational scheme) according to


ACM Journal of Educati onal Resources in Computing, Vol. 1, No. 1, Spring 2001.

http://final31.exe/

http://final31.exe/



which the content information will be divided into smaller units. Theheterogeneous and ambiguous nature of multimedia makes it difficult toapply highly structured systems to the content. It is important to deter-mine the attributes of the data elements that will be used to partition oraggregate along some dimension.

The functional dependencies among the units will determine the appro-

priateness of different schemes. Consideration should be given to thearrangement of the content(s) and how the arrangement fits the designers’perspectives and overall plan for delivery. Several research studies showthat changes in the semantic content or variation in structure are reflectedin differences in recall [Last 1998; Mayer and Sims 1994]. The organiza-tional scheme may reflect different functionalites such as task characteris-tics, domain knowledge relationships, or logical ordering and connections toaccommodatea user and designer goals. In addition, many units could bedecomposed into more granular entities that do not necessarily belong tothe same scheme. For example, information about the United States couldbe divided geographically (e.g., East coast, West coast) and then presentedin alphabetical order (e.g., according to their names) or numerical order(e.g., by area size).

Table III. Content Structuring Pattern

Media Type

Content Organization

Alphabetical/

numerical

Geographical/

spatial

Chronological/

temporal

Logical

(e.g., causal)

Topical

(e.g., by subject)Text Text

informationabout states inUS inalphabeticalorder

US stateinformationdivided intoEast and Westcoast group

Chronology of events

Instructionsfor preparing a meal

Differentsection of alesson

Sound Speakerpresenting information inalphabeticalorder

II. (sound version)

Speakerpresenting events inchronologicalorder

Speakerpresenting astory

Sound clipscategorized byperformer orkind of music(e.g., country,

rock, jazz)Graphics Pictures of

animals innamealphabeticalorder

Image map of geographicarea withclickableplaces

Images of events on atimeline

Sequence of pictures oninstalling kitchenappliances

Images of animals dividedinto groupsbased on region

Motion Movie actorsin alphabeticalorder

Videosequence of ahouse interior,one room at atime

Videosequence of the samelocation atdifferent times

Videosequence of atornadofollowed by asequence ondamage

Movie clipsdividedaccording todirector

Multimedia Anycombination

Anycombination

Anycombination

Anycombination

Anycombination





Organizational patterns like task-driven or metaphor-driven organiza-tion [Rosenfeld and Morville 1998] are considered logical patterns of content. In this case there is always some kind of reason/logic underlying the relations among the functions, actions, and other parts of some task.

Metaphors are also used to organize logical patterns of content in anintellectually meaningful way.

Content organization is a set of modules and the possible paths thatusers may take in traversing the content. In other words, the designerdetermines which modules should be next and when the user should shiftto the next module. Domain knowledge is stored in the knowledge base thatis structured by a particular data model (e.g., entity-relationship model,object-oriented paradigm, and expert system). The data model defines theorganization of the information contained in the knowledge base. Theorganization defines the logical relationships among the content units in

the knowledge base.There are three possible types of content organization: sequential/linear;

hierarchical (i.e., tree structured); and Web structured (i.e., nonlinear,associatively structured information). Systems can be static or dynamic interms of content variability. Interactive expansion and modification of content organization is a feature of the generative organization. It is clearthat having adopted a certain interactivity style, we can choose an appro-priate organization type [Aleem 1997]. Hierarchical and Web organizationsare appropriate for a reactive type of interactivity, while Web and genera-

tive organization seems to be a natural choice for the productive interactiv-

ity style.

7. MULTIMEDIA TAXONOMY FOR EVALUATION

Multimedia taxonomy is best understood in its application. In evaluationstudies directed at graduate computer science classes, students typicallyidentify an expanded set of questions. In this course [Heller and Martin1998] students examine the media taxonomy at length and are thenassigned to teams to develop an evaluation protocol in one of the contextcategories for a specific application such as AdaMentor (website), a site for

teaching Ada online, or the Clinique promotional CD. For example, theaesthetics category produced the following concerns: In the general expres-sion of text, we might ask: Is the font size appealing? While in theelaboration expression of text, it is appropriate to ask about literaryexpression on the continuum from dry to poetic. In the representationalexpression of text, students questioned whether the format of the outline islogically well structured, and in considering the abstract expression withinthe aesthetic categories, they asked if the textual image as an icon ispleasing.

Similarly, when considering questions of the audience for text, studentsasked if the instructions were clear—in elaborative or representational textexpressions, they asked evaluative questions about the appropriatness of the feedback to the audience and the compatibility between the reading





level of the audience and the level of expression in the description. Theusefulness category prompted them to assess the value added by a media

type or expression. The extended multimedia taxonomy elicited questionsabout sound quality as well as sound impact.

Under the category of usefulness, students generated questions aboutwhether a medium like sound was functional by asking users to rate thesound on a Likert scale that ranged from annoying to helpful. They alsoprepared protocols with open-ended questions about what the user remem-bered about the sound in the application being evaluated. Thus, the newlyexpanded multimedia taxonomy became a floor, not a ceiling, for a series of guidelines that can be used to generate evaluation questions about amultimedia application. Parts of the taxonomy not relevant to the specific

product being evaluated were ignored by the students when developing theevaluation protocol and data-gathering instruments.

Table IV. Typical Aesthetic Evaluation Questions

Media Type

Media Expression

General Elaboration Representation Abstraction

Text Font size appealing Free of stereotypes(media overuse?)Overall impressionScreen designScreen spacing Use of color

Literary expression(dry to poetic)

Format (i.e. outline)well placedPosition of messagesappropriate

Metaphorsconsistent withinapplicationText image or icon

Sound Sound(annoying to helpful)Sound(noisy to tuneful) Volume (too soft/

loud to just right)What did you thinkof the sound?

Background sound(pleasing)

Sound effects (notpleasing to pleasing)Sound effects(clear to unclear)

Graphics Still imagesattractive?Use of colorpleasing?

Backgroundimages (too faint)

Icons pleasing?

Motion Video or animation(not pleasing to pleasing)

Multimedia Overall design(chaotic to organized)Overall design

(colorless to colorful)Overall design(dull to exciting)Overall design(dark to bright)Overall design(cold to warm)





7.1 Aesthetics

Table IV presents student-generated formative evaluation questions on theaesthetics, appearance (the artistic look or impression) of the presentation.Questions related to design, rather than function, are the focus here.

7.2 Audience

Audience guidelines are intended to direct the evaluator to issues of howthe IMM relates to the audience and how the audience might process themedia form in a specific media format. Before preparing an evaluation

protocol for a particular IMM, the evaluator should know who the intendedaudience is. For example, in text as elaboration (Table V), it is necessary toask whether the text is at the reading level appropriate for the intendedaudience. On the other hand, while text as representation does require theuser to read, there are fewer words and grammatical constructions in thiscategory; but the representation (e.g., outline) might not be familiar to theparticular audience.

7.3 Discipline

Guidelines here are intended to identify the content-specific material ineach of the media forms within a specific media format (Table VI). Asstudents prepared questions in this category, they used an expanded

Table V. Typical Audience Evaluation Questions

Type

Expression


Text Intended audienceis clearCognitive load(unmanageable tomanageable)Knowledge spacecompatibilityMedia overuse1

Pace controlled bythe userProgram of interestto intended user?

Appropriate levelof difficulty insentence structureand vocabulary

Feedback isaudience-appropriateInstructions areclear (no to yes)

SoundGraphics Metaphor

appropriate touser?

Motion Purpose of motionis clear

Multimedia

1In this category media overuse relates to whether or not the audience is being overstimu-lated by the use of media. For example, for field-dependent thinkers, the use of too muchmedia might be a distraction from the message.





version of the media taxonomy that included the work described in thecontent characteristics section described above.

For example, for concrete topics such as the description of Cliniquecosmetics, students queried whether the video presentation was clear andwell sequenced. Since the CD also included logical content on how to use aseries of cleansing products, students asked whether the textual checklistwas synchronized with the graphic presentation.

7.4 Interactivity

In this category, evaluators should address aspects of control, navigation,and linking. Aleem [1997] expanded the relationship between the interactivity

Table VI. Evaluations Related to Discipline

Type

Expression


Text Major points areeasily recalled Appropriate level of difficultyKnowledge space iscompatible withpurposeText is toomuch/littleMedia selectioninadequate

Information ismissing Easy to get tocentral pointInformation isfactually correctInformation content(irrelevant)Information(confusing)Information (boring)Information

(too little)Information lackscredibilityInformationpresentation obtuse

Metaphor intuitivefor purpose

Sound Sound conveys(too much/little)information

Sound effectsconvey information

Graphics Icons conveyinformation

MotionMultimedia Media integration

(uncoordinated to coordinated)Sound with stills(ineffective to effective)Sound with motion(ineffective to effective)Text with sound(ineffective to effective)Text with stills(ineffective to effective)Text with motion(ineffective to effective)





attribute and the media type and expression. Further, he subdivided the

attribute of interactivity into four categories: passive, reactive, proactive,and directive. With passive interactivity, the user has no control, insteadall control is embodied in the application (e.g., automated setting in aPowerPoint presentation). Reactive interactivity provides limited responsefor the user within a scripted sequence. Proactive interactivity allows theuser to play a major role in the design and construction of situations,typically by manipulating values. Multimedia that has interactivity at thedirective level allows the user to both respond and initiate actions withinthe application, as well as to tailor aspects of the environment such asselection of color choice, feedback choice, and so on. Table VII (excerpted

from Aleem [1997], and used with permission) identifies examples of mediaexpression within each of these categories for the media types of text andsound.

7.5 Quality

Table VIII refers to the technical, reproductive aspects of the IMM, andlists some of the questions prepared by the students in the project evalua-tion.

7.6 Usefulness

This category refers to the value of the material presented in the IMM aswell as its ease of use. Since interactivity covers navigational issues and

Table VII. Examples of Media Expression In the Interactive Category

Media type Passive Reactive Proactive Directive

Text No user control;

sequentialpresentation

Page turner

Linear spacing

Browsing,

hypertext, fixedanchors, pathswith choices foruser

Dialog-based,

creative writing,word processing

Graphics No user control;sequentialpresentation

Able to makepredefinedchanges ingraphics

User initiatedchanges tographics (size,shape, color,position)

User createdgraphics

Sound No user control;sequentialpresentation

Able to makepredefinedchanges, adjust volume

Changes tostations, tracks,fast-forward, loop

User createdsounds

Motion No user control;sequentialpresentation

Able to makepredefinedchanges such aspath or target of motion

User controlledstart, stop, pause,forward, reverse

User createdanimatedsequences

Multimedia No user control of any combination of types listed above

Any combinationof types

Any combinationof types

Any combination of types





usefulness covers the ease with which a user can operate the application,one might think there is some confusion between the two categories. Butthere is a distinction. In addition to asking whether the user can operatethe equipment necessary to make the application useful, one should askwhether the application is worthwhile to the user. Questions like: Can the

user operate the head tracker or the roller ball? and Is there a need forexternal devices to run the IMM? are appropriate in the ease of usecategory, and even more so are questions on the value of the application tothe user. Table IX includes some of the class questions. Students askedquestions about the usefulness of the application on the basis of the tasksaccomplished. For example, in the AdaMentor evaluation, students queriedwhether the representational instructions enabled the users to get to thespecific content area.

As can be judged from the foregoing, not all of the areas within thethree-dimensional multimedia taxonomy are complete. Some are being examined in detail, while others are still to be developed. Students in theSeminar on Multimedia Evaluation in the Department of Electrical Engi-neering and Computer Science at The George Washington University

Table VIII. Evaluation Questions Related to Quality

Type

Expression


Text Word choiceclear/unclear

Is it clear how toreach goalData is correct (noto yes)Speed appropriate/ inappropriateGrammar correctReading characterson screen (very hardto very easy)Spelling correct (noto yes)

Math functionscorrectIs it clear whichinput device to use

Position of icon andbuttons on screen(inappropriate toappropriate)

Sound Sound quality(unclear to clear)Sound speed(too fast/slow to just right)Sound volume(too loud/soft to just right)

Graphics Quality of image(unclear to clear)Quality of image(low to high)Quality of motion(jerky to smooth)

Size of image(too large/small to just right)

MotionMultimedia





[Heller and Martin 1995] have been successful in designing evaluationprotocols using the multimedia taxonomy as a guideline. Table X gives asample of the survey comments from the formative evaluation of AdaMen-tor, an online web site for the study of the Ada programming language.Multimedia taxonomy helped pinpoint such problems such as “too manyforms within the web site” and “the inability for a user to see the entirepage.” On the bases of the evaluation, students were able to suggest thatthe Ada Mentor developers reduce the navigational capabilities and clarifythe functionality of navigational icons.

Table IX. Evaluations Related to Usefulness

Type

Expression


Text Have you used the systembefore (No to Yes)Did you view the entiresystem?How many features haveyou used?(Did you get what youwanted?)Overall function(dysfunctional to functional)Ease of use(difficult to easy)

Easy to get to central pointin program? (No to Yes)Unity of action supportscentral theme (No to Yes)Program does not causedisorientation (No to Yes)Would you use the systemagain?How much did the presenceof a specific medium help inyour use of the system(very little to very much)

Specific medium information(not interesting to interesting)

Was the metaphorintuitive to theapplication?

Sound Specific medium information(not helpful to helpful)What specific medium didyou remember the most?Overall impression(confusing to clear)Overall impression(disorganized to organized)

GraphicsMotion

Multimedia Media integration(uncoordinated tocoordinated)





8. CONCLUSION

We believe that multimedia taxonomy is a reasonable organizing frame-

work. The extensibility of the taxonomy has already been demonstrated bythe fact that it supports the work by Aleem [1997] and by the extensionsdetailed in this paper. Admittedly, there is even more room for extending the multimedia taxonomy if new ways of thinking about multimediacontext are considered—e.g., motivation or stimulation. For example, it isworth considering whether the context dimension should be expanded—itmay be useful to add a category for a computing/presentation system. Sucha context dimension category might ask whether a product should bepresented on a web page or a kiosk, and how that decision will affect therest of the design.

Another extension is to treat each category in the context dimension as aminidimension itself and to identify values for it. The extensibility of thetaxonomy is demonstrated by the ability to take a category and form newdetailed subtaxonomies. For example, the discipline category, which in-cludes the content concept, could be enhanced to consider the following roles that MM may play in MM products:

Content: The information to be presented; for example, the body of a webpage.

Presentation mechanism: Those elements of the presentation whose pur-pose is to facilitate communication of the content (i.e., the user interface,control mechanisms, metacontent); for example, navigation buttons on aweb page and browser controls.

Table X. AdaMentor: Selected Responses Using the Taxonomy Protocol

Keyword Actual Comments of Concern Actual Comments of Praise

Text Hard to read

Could be biggerToo much technical language

Not too wordy

Navigation Hard to find exercisesButtons didn’t load in some casesNeed to move inherent iconsNeed better button descriptionTrouble getting back to where I was

Link design is efficient

Speed Took forever to loadSlow on T1 lineSlowness more trouble than its worth

Screen design Can’t see entire pageScreen size for exercises is too small

Clean setupEasy to look atNice graphic layout

Interactivity Not much better than a bookBook could have been read anywhereExercises are not interactive

Content Need better explanationsToo much technical languageInclude security information only when necessaryMake it more fun

Information on each topicwas sufficient





Merged content and mechanisms: Integrated or merged content andpresentation mechanisms or self-presenting material.

We could populate the space of this subtaxonomy using role and mediatype as its two dimensions, as shown in Table XI. For purposes of designand evaluation, it may be useful to differentiate between MM elementsacting as content versus those acting as presentation mechanisms.

Multimedia taxonomy is an attempt at the formalism needed to provideboth a theoretical and practical framework for the new and rapidly growing field of interactive multimedia. Currently, we are starting usability studiesusing the taxonomy. Specifically, we are interested in whether evaluatorscan use the taxonomy as a guideline to improve their existing evaluationprotocols or does the fragmentation of the taxonomy interfere with, ordistract from, the evaluation. In continuing efforts to establish qualityevaluation products, we are also starting to review the use of object-

Table XI. Examples of Roles for Each Media Type

Media type

Media Role

Content Presentation Mechanism

Merged content &

MechanismText Paragraphs in a book

Email message textNewspaper headline orarticle

Book page numbersLabels of email fields,e.g., “From”Continuation note at theend of a newspapercolumn, e.g., “Continuedon page A4 column 2”

Book index or table of contents;2

Sound Recorded weatherinformationSound track in CBTmodule

Telephone menuEarconsPresentation note in CBTmodule, e.g., “This is the

end of Lesson 1”.

“You have mail”announcement by a webportal application

Graphics Pie chartImage from a web-basedphoto library

Icons on navigationbuttonBrowser logoGUI elements such as thescroll bar and cursor ormouse pointer

Video gameImage map (graphic withembedded links)Hierarchical catalog of web-based library withlinks to catalogued items

Motion Motion video oranimation of planetarymovement in a CBTmodule on astronomy

Revolving Netscape iconshowing that download isin progress3

Talking head of instructorin a CBT module onastronomy

Multimedia CBT module with sound,

video, and animation

(Any of the examples in

other rows in thiscolumn)

Interactive lesson

Virtual realitypresentation

2Indexes and catalogs facilitate communication of primary content but also constituteinformation in their own right since they show the structure of the primary information.

3In terns of the Media Type dimension this is motion and in terms of Media Expression, thisis an abstraction. if, instead of this, we had a status message saying “Download is in progress”,it would be text in Media Type and elaboration in Media Expression. In bolth cases, in termsof Role, it is part of the presentation mechanism rather than content.





oriented design techniques and metaphors as a possible format for multi-media evaluation. As as a start, we are considering each medium (text,sound, stills and motion) as a specific object, and each object has its specificattributes (size, color, form declaration, representation, and form), and

behaviors (interactions). Objects can form clusters, and it is the clustersthat can have different relations and interdependencies. Investigations of cluster analysis as a methodology for multimedia evaluation is just begin-ning.

As an organizing principle, the multimedia taxonomy presented here canbe used to understand both design and content messages. Another futurestep is to review the taxonomy in light of various studies on the psycholog-ical and cognitive aspects of multimedia applications to determine whetherthis taxonomy can shed light on these areas. Questions such as how wecome to understand an image and how that understanding is different from

our understanding of text can be answered in part by using the taxonomy.We present a few examples of the impact of the multimedia taxonomy onthe design and implementation of evaluation protocols for multimediaproducts. More work in this area remains to be done.

REFERENCES

A LEEM, T. A. 1997. World Wide Web site. http://www.erols.com/aleem/mediatax.html orhttp://www. erols.com/aleem/interact.html,

A NDRE, E. E T A L. 1993. WIP: The automatic synthesis of multimodal presentation. In Intelligent Multimedia Interfaces, M. Maybury, Ed. MIT Press, Cambridge, MA, 75–93.

A RENS, Y., MILLER, L., AND SONDHEIMER, N. 1991. Presentation design using an integratedknowledge base. In Intelligent User Interfaces, J. W. Sullivan and S. W. Tyler, Eds. ACMPress, New York, NY, 241–258.

BERINSTEIN, P. 1997. Moving multimedia: Information value in images. Searcher, 40–49.FEINER, S. 1991. An architecture for knowledge-based graphical interfaces. In Intelligent

User Interfaces, J. W. Sullivan and S. W. Tyler, Eds. ACM Press, New York, NY, 259 –279.FEINER, S . K . AND MCK EOWN, K. R. 1993. Automating the generation of coordinated

multimedia explanations. In Intelligent Multimedia Interfaces, M. T. Maybury, Ed. Amer. Assn. for Artificial Intelligence, Menlo Park, CA, 117–138.

GREGG, L. W. AND F ARNHA M-DIAGGORY . 1975. Content and Structure in Learning.HELLER, R. S. AND M ARTIN, C. D. 1995. A media taxonomy. IEEE MultiMedia 2, 4, 36– 45.HELLER, R. S. AND M ARTIN, C. D. 1998. Multimedia taxonomy for design and evaluation. In

Handbook on Multimedia Computing, B. Fruht, Ed. CRC Press, Inc., Boca Raton, FL.JONASSEN, D. H. 1988. Instructional Design for Microcomputer Courseware. Lawrence

Erlbaum Associates Inc., Hillsdale, NJ.JONASSEN, D. H. 1991. Hypertext as instructional design. Educ. Technol. Res. Dev. 39, 1,

83–92.L AST, D. A. 1998. Effects of prior knowledge and goal strength. In Technology and Teacher

Education Annual. 476– 479.M AYBUR Y , M. T. 1992. Communicative acts for explanation generation. Int. J. Man-Mach.

Stud. 37 , 2 (Aug.), 135–172.M AYER, R. E. AND SIMS, V. K. 1994. To whom is a picture worth a thousand words? Extensions

of dual-coding theory of multimedia learning. J. Educ. Psychol. 86, 3, 389–401.REEVES, T. C. AND H ARMON, S. W. 1993. Systematic evaluation procedures for instructional

hypermedia/multimedia. In Proceedings of the Annual Meeting of the AERA.REISER, H. AND REISER, B. 1995. Aesthetic consideration unique to interactive multimedia.

IEEE Comput. Graph. Appl. 15.





ROSENFELD, L. AND MORVILLE, P. 1998. Information Architecture for the World Wide Web.O’Reilly & Associates, Inc., Sebastopol, CA.

ROTH, S. F. AND HEFLEY , W. E. 1993. Intelligent multimedia presentation systems: researchand principles. In Intelligent Multimedia Interfaces, M. T. Maybury, Ed. Amer. Assn. for Artificial Intelligence, Menlo Park, CA, 13–58.

TENNYSON, R. D. 1977. Content structure as a design strategy variable: I content acquisition.ED 137 314.

WU, M. H.-P. AND M ARTIN, C. D. 1997. An exploratory study of user media preferences in apublic setting. J. Educ. Multimedia Hypermedia 6, 1, 3–21.

Received: August 2000; revised: November 2000; accepted: December 2000



heler martin multimedia taxonomy

Documents