Cognitive Study Proposal

Kihyun RyooPaula Wellings

Peter WorthJuly 15, 2003

ED 333A

Learning Environment

Macromedia Flash is a tool for developing highly visual, interactive content and

applications. Since its first version in 1997, Flash has evolved from a simple animation tool into

a powerful, complex, and highly-integrated multimedia design environment. The software

consists of a number of conceptual models which enable users to create a wide variety of

artifacts. As Flash has adapted to meet the needs of interactive designers, programmers, and

animators, one side-effect of this evolution has been the accumulation of redundant or even

conflicting models for artifact creation. This complexity poses significant challenges to novice

users, not only because of the large collection of functions and procedures, but also because

users can create the same effect by using different methods.

Recognizing the power and utility of Flash despite these difficulties, many teachers in the

Sunshine Coast School District would like to learn to use the program. Many would like to

become expert facilitators with the ability to teach their students to use Flash. Others would like

to create multimedia artifacts for the classroom. Some would simply like a functional

understanding of how Flash works. The teachers come from a variety of backgrounds and levels

of technical literacy, but share a strong desire to become proficient in Flash. She commissioned

the Teach Flash study as a way to find an effective way for teachers to learn Flash.

After doing some preliminary research on available Flash training programs, the

Assistant Superintendent for Technology at Sunshine Coast, Julie Bunda, found that a number of

tutorials already exist. She noticed that some involve having the user deconstruct and/or recreate

a completed Flash artifact and others give an explanation of the available tools and how they are

used. Through experience and knowledge of studies in the area of learning software, she knew

that people find different instructional methods useful.

1

The number of ways in which people become proficient in software programs indicates

that the “Teach Flash” study of how best to instruct people in the use of Flash will be broad-

based. Studying just online tutorials or university classes would place unnecessary and

misleading constraints on the study. The learning environment in this case is not in any one

location, but in the program Flash itself, and in the interaction between learner and program. It is

in the construction of knowledge and the building and connecting of schema that we will find

learning, and this will be reflected in our methods. Thus, we need not study a particular Flash

class, but rather seek out expert models of Flash understanding. Obviously, issues of practicality

require that we limit our study somewhat. To that end, the subjects of our study will be advanced

or expert Flash users, and those teachers with a novice practice of Flash use.

Learning Problem

Existing Flash tutorials usually fall into two categories: one is to provide learners some

completed Flash artifacts and have learners deconstruct/reconstruct them; another one is

systematic introduction of Flash functions and design techniques. However, these tutorials are

designed based on certain expert mental models, which are not necessarily suitable for each

novice learner. Each expert mental model is related to the expert user’s background, including

previous experience with similar design/develop software in particular, and previous experience

with the world in general. Therefore, it is difficult for novice learners to learn Flash effectively.

To be more specific, novice learners can not easily construct their own mental model of Flash

tutorials and training programs.

2

Learning Goal

Learning is a process in which learners construct new ideas or concepts by combining

their current/prior knowledge and experiences with new concepts and structures through active

engagement in problem-based environments. From this cognitive perspective, we identify Flash

experts as those who “have obtained a level of knowledge and experience that enables them to

have meaningfully organized cognitive activity, principled and relevant representation of

problems, an efficient and informed strategy, ongoing and flexible self-monitoring, and

principled and coherent explanations of their work.”(Bransford et al., 2000) This more evolved

or integrated conceptual understanding is what, in this study, distinguishes an expert from a

novice.

Therefore, novice users of Flash face a number of challenges. The complexity of Flash

requires the formation of a new cognitive hierarchy to support the effective learning and use of

the tool by all levels of learners. It is thus difficult for the novice learner to easily access an

expert mental model within which to achieve their desired functions.

In order to solve this problem, it is important to understand how experts think about the

interface and to learn their mental models in order to facilitate Flash. Our learning goal is to

allow novice Flash learners to access particular expert mental models, which best correspond to

their knowledge and background, and construct their own mental model with efficiency and

effectiveness.

Design Principles

The Teach Flash study will examine the Flash learning process from a cognitive

perspective. Because the focus is on how individuals, regardless of situative or physical

3

environmental factors, construct their understanding of Flash, the study is grounded in three

principles of cognitive learning as outlined by Greeno, Collins, and Resnick (Greeno, et al.,

1996). Following principles reflect our assumption that we will be able to design an effective

means of teaching Flash to new users by understanding the conceptual models of expert users.

Cognitive design principle 1 (c1) asserts that learning occurs in an environment that

encourages people to construct an understanding through interesting problem-solving tasks. The

Flash environment provides continuous problem-solving and reasoning tasks for the novice, or

indeed the expert. Their ability to successfully use the program is contingent on their developing

a strategic knowledge of how and when to use the extensive array of tools and features to create

a desired effect, and the difficulty of gaining such strategic knowledge is a fundamental

component of the learning problem.

This principle of one’s understanding being constructed through interaction with an

environment will guide this study in that it will focus on how expert and novice Flash users

develop, use, and perceive their own strategic knowledge, and how newcomers to Flash have

developed problem-solving strategies for other working with other programs. The goal of this

facet of the study is to make explicit to new users the tacit procedural knowledge used by

experts, as discussed by Gott in her study of the LISP tutorial (Gott, 1989). This is complicated

by the fact that, with Flash’s evolution from a number of conceptual models, the sheer variety of

ways of achieving the same task means that there will be several sets of procedural knowledge.

While this may create some confusion, it also accommodates a greater variety of learning styles

and learner backgrounds.

Cognitive principle 2 (c2) addresses just this issue. Learners proceed through “sequences

of cognitive development” (Greeno, et al., 1996), in which new learning is acquired in the

4

context of what is already known. Thus, understanding and activating the learner’s prior

knowledge and frame of reference is essential in helping them to build connections to new

information and skills.

In the Teach Flash learning problem, this is a crucial issue. The teachers who will be

learning Flash have varied backgrounds and experiences in interacting with computer software.

Developing an effective instructional method will require a clear understanding of how those

backgrounds and experiences are linked to the creation of conceptual models of how Flash

works. The study phase will examine how proficient Flash users have created such models based

on their use, and how those models have changed over time. The tool and manipulation

paradigms within the Flash interface have been designed to relate to people’s previous

experience with other similar development environments, including previous version of Flash.

How these conceptual models can be utilized for different learners will be an area addressed in

this study.

Cognitive principle 3 (c3) addresses the practical use of Flash by the teachers. C3

emphasizes that learning can be demonstrated and applied when learners are able to generalize

information within a broader context. This principle works on two levels in this study. Within the

realm of acquiring proficiency in the use of Flash, it will be important to understand how users

see their individual actions and commands in the context of the wider Flash environment. While

this is related to c2 in that it requires examination of Flash users’ conceptual models of the

program, it is distinct in that the focus here is on the how the device model makes explicit the

user’s understanding of the overall concept of Flash, while c2 focused on how concept models of

other familiar programs could be utilized as prior knowledge in the teaching of a new program.

5

The other level on which the generalization principle will be a factor deals with how the

mastery of Flash, once attained, will be utilized. Our proposed solution will include the

assumption that, with limited resources of budget and time, teachers and the school districts

sponsoring their training will want to know how what they are learning will be implemented in

the classroom. The efficacy of the training will necessitate the explicit generalization not only of

how the individual functions relate to the overall program of Flash, but how those functions will

be useful to the students whom the teachers will ultimately be teaching.

Proposed Study

The “TeachFlash” study is based on the cognitive design principles explained above,

particularly that people create mental models representing their understanding, and that

understanding and making these conceptual frameworks explicit, and structuring instruction on

these frameworks represents an effective means of teaching new skills and concepts. To that end,

the study consists of six steps designed to identify experts and understand how their conceptual

frameworks of Flash differ from those of novice users. The first step uses a focused

questionnaire aimed at assembling an appropriate Flash-user group. Steps two and three involve

observing the Flash users creating artifacts and reflecting on their process. In step four we

analyze the data from steps two and three. Step five involves the expert reconstruction of a

novice’s artifact, while the final step is a comparison of the two construction models of the same

artifact.

6

Step One: Initial Questionnaire

The goal of the initial questionnaire is to identify potential study participants in the

Sunshine Coast School District. The questionnaire will be distributed through staff development

seminars as well as being available in professional development resource centers, both in schools

and at the district office. In addition to asking the questions that follow, the questionnaire will be

explicit with regards to opportunities and compensations associated with participating in our

study.

Initial Questionnaire

General Software QuestionsDo you currently use a computer? If yes, what sort of things do you do with your computer?If applicable:Do you use a computer to create things for your classroom?Do students use computers in your classroom?What software programs are you most comfortable using?What software programs have you used in your teaching?

Flash-Related QuestionsHave you heard of a software program called Flash, made by Macromedia?Have you used Macromedia Flash in your personal experience? If yes, what sort of things have you used the software for?Have you created anything for your class using Macromedia Flash? If yes, what sort of things have you created?Have you taught any classes where your students have used Macromedia Flash? If yes, what sort of activities did you engage in with your class?If you haven’t used Flash before, would you be interested in learning to use program?

Experience in Domains Related to FlashDo you have any experience in computer programming?Do you have any experience in visual design?Do you have any experience in film, video, or animation?

7

Based on the responses to this questionnaire, we are initially interested in following up

with teachers that have experience with Flash. We assume that this group will be relatively small

as Flash is a complex and expensive program and is not typically part of the business

productivity software installed on school computers. In addition, while a number of interesting

studies in the UK1 have suggested that Flash might be the new and improved LOGO, there has

not been a significant effort in the US to position Flash as a constructivist learning environment.

Our hope with this initial sorting questionnaire is to identify twelve study participants who have

experience in using Flash personally and with their students, as this population will likely have

the knowledge and problem solving skill in Flash that we are hoping to target, explore, and

leverage in future design solutions.

Step Two: Observation of Artifact Creation Process

In this step of our study, we are interested in observing how study participants with an

existing knowledge of Flash create artifacts. We are also interested in understanding how

participants support their use of the software with existing resources such as manuals, online

tutorials, and discussion boards. To support these objectives, we will be observing the artifact

creation process within the environment in which the participants usually create artifacts—be

that the home, computer lab, or other location. The participants will be asked to create an artifact

in Flash that will be used in some common context, such as a classroom lesson, a personal

website, or preparation for a classroom activity where students will use Flash.

1 http://www.tygh.co.uk/flashtalk/index.html

8

To capture this activity, we will set up a recording device that video captures the

participant while they are creating the artifact, as well as a recording device that captures on-

screen activities. A researcher will not be present during the actual artifact creation. This choice

is based on the desire to enable the participant to create the artifact on a normal time schedule

and also to decrease the impact of the researcher on the participant’s typical creation and

troubleshooting strategies. However, the recording software will document any recording

irregularities to ensure data accuracy.

Step Three: Reflections on Observation Videos

In this step, a researcher will review the video of the artifact creation process with the

participant in a session that is also video recorded. The researcher and the participant can see

both a video image of the participant working on their artifact and a synchronized view of the

computer monitor, and can hear any associated audio. Depending on the length of the recording,

the researcher may identify specific areas. The selection of these areas is based on identifying

moments where the participant appears to be engaging in activities that reveal elements of their

structuring of knowledge. The participants will be asked to describe what they were thinking

during the process of creating the artifact and to describe their process when they encountered

any challenges.

The observation video from Step Two acts as a prompt to the participant and facilitates

their reflection on their own cognitive processes. Of particular importance to the researchers will

be observing the participant’s methods of explanation and any accompanying gestures, which

may signify the manipulation of mental models or other representative schema. Additionally,

9

researchers will probe the participants to learn more about how they acquired their current level

of knowledge and ability.

Step Four: Categorization of Participant Activities Observed in Steps Two & Three

In this step, researchers will utilize a model premised on Baxter and Glaser’s model of

Cognitive Activity and Structure Knowledge to categorize the artifact creation practices of the

participants. While working from this general framework, specific categories of activities and

cognitive strategies will be articulated that relate to the activities encountered in the use of Flash.

TABLE 6.1 Cognitive Activity and Structure of Knowledge

  Structure of Knowledge

Identified Novices Identified Experts

Organized Cognitive Activity

Fragmented Meaningful

Problem Representation

Surface features and shallow understanding

Underlying principles and relevant concepts

Strategy Use Undirected trial-and-error problem solving

Efficient, informative, and goal oriented

Self-Monitoring Minimal and sporadic Ongoing and flexible

Explanation Single statement of fact of description of superficial factors

Principled and coherent

From How People Learn: Brain, Mind, Experience, and School (Expanded Edition) by John Bransford (Editor), Ann L. Brown (Editor), Rodney R. Cocking (Editor), 2000, National Research Council, Washington, D.C.

10

Based on this analysis, we hope to identify participants with activity and knowledge

structures that are representative of people in both columns of Glaser and Baxter’s table.

Although Flash users that we might locate on the “Identified Novice” of the above chart are able

to create an effective Flash artifact, their methods indicate not an intentional path, but an almost

accidental trial and error or “stumbling into the creation of a completed artifact. In contrast, the

teachers in the “Identified Expert” column of the above chart will have created Flash artifacts

using a clear, goal-oriented strategy. Those strategies may vary among the individual users, but

the fact that they have a coherent conceptual structure for a variety of activities teachers might

perform with Flash, and are able to implement that structure in the creation of an artifact, is of

interest to us here.”

Step Five: The Expert Reconstructs a Novice’s Artifact

In this step, we present an identified expert with a final product created by an identified

novice. Without access to any source files or materials, the expert is asked to construct a Flash

file that has a similar functionality and visual quality to that created by the novice. Similar to

steps Two and Three above, the expert is observed creating the artifact and then reflecting on the

creation process with the researcher. It is hoped that through this process we will see differences

in how the experts organize and create these Flash artifacts, and how they reflect on the process

of doing so.

Step Six: Comparison of expert and novice practices in creating artifacts with Flash

As a final step in this study, we will compare the data gathered from the novices and experts

to a gain a better understanding of how cognitive activities and structures of knowledge impact

11

the practice of creating artifacts with Flash. Comparisons that we think are valuable to make

include:

Comparison of Flash working documents

Flash is organized around the principle of having a working file that is then exported to a

final file upon completion of the artifact. As Flash enables artifacts to be created in a number

ways, the working file becomes an embodiment of how a participant understands Flash’s

functions and capabilities. Comparing these working files will enable us to better understand

the differences and similarities in participants’ conceptual understandings of the software.

Comparison of artifact creation methods and reflections

The way in which a participant creates an artifact and then reflects on their activities

enables us to understand how the participant structures their understanding of Flash. In

addition, we have the opportunity to gain an understanding of what types of knowledge in

building a Flash artifact are tacit to the participant and which are explicit. By comparing the

methods and reflection of the novice and expert participants we can further understand how

Flash is conceptually organized by both groups.

Based on these comparisons, we expect to find that the experts construct their Flash files in a

manner that is representative of their coherent cognitive understanding of the application, and

that novices are working through a process of learning and discovery while attempting to create

artifacts. Findings from these comparisons will enable us to present improved models for the

learning of Flash.

12

Next Steps:

Once we have determined the qualities and characteristics of both our identified novice

and expert users of Flash, we will be able to explore the impact that previous knowledge may

have on participants’ current levels of competence. Mapping our findings back to the original

questionnaire, it may be possible to find correlations between their use of other software and the

successful use of Flash. As Flash is most apt to relate to the previous experiences of designers,

programmers, and animators, (as opposed to teachers), it is also useful to consider the role that

previous experiences in these domains might play in how teachers use Flash.

Proposed Solution

To solve the learning problem mentioned earlier, we propose a prospective solution which

allows novice learners to construct their mental model based on their existing knowledge and

experience. This solution also would assist novice learners in accessing expert mental models,

which have a more complete and clear picture of the Flash functions and procedures. Our

proposed solution includes following steps:

1. Analyze expert mental models and organize them into different categories according to

the expert’s background.

2. Design and develop a universal and flexible training program of tutorials based on

different expert mental models.

3. Design a diagnostic tool for Flash learners which will identify the current knowledge and

previous experience with similar software, and recommend a useful sequence of tutorials.

13

4. Match the novice learner’s background to certain expert’s background, select the most

feasible learning program for the learner and assist him/her to construct a similar mental

model according to the expert mental model.

References

Bransford, John. (2000) How People Learn:Brain, Mind, Experience, and School (Expanded

Edition), 145. National Research Council, Washington, D.C.

Greeno, J. G., Collins, A. M., & Resnick, L.B. (1996) Cognition and Learning. Handbook of

Educational Psychology, 15-46. Macmillan Library Reference USA.

14