573-1466-1-sm

7/27/2019 573-1466-1-SM

1/38

Volume 17 Number 2 269

Adair-Hauck, Willingham-McLain, and Youngs

Evaluating the Integration ofTechnology and Second

Language Learning

Bonnie Adair-Hauck, Ph.D.University of Pittsburgh

Laurel Willingham-McLain, Ph.D.Duquesne University

Bonnie Earnest Youngs, Ph.D.Carnegie Mellon U niversity

ABSTRACT

This article reports the findings of a program evaluation project that as-

sessed the integration of technology-enhanced language learning (TELL)into a second semester, college-level French course. Thirty-three French II

students participated in this study. Students in the treatment group metwith the instructor three days per week and, for the fourth class, they

participated in TELL activities. The control group met with the instructorfour t imes per w eek. Both groups had the same instructor, textbook, andancillary materials. The article reports on student performance in French

for listening, speaking, reading, and writing skills, and on their culturalknowledge. Findings indicate that the students in the treatment group

performed equally well as the control group in listening and speaking andbetter on reading and writing achievement measures. The study also in-

cludes findings regarding student motivation, anxiety, and perceptions onmeeting the language learning goals students set for themselves. The re-sults ma y be interpreted that it is both feasible and desirable to integrate,

in principled ways, TELL a ctivities into the language learning curriculum.

KEYWORDS

Technology-Enhanc ed Lan guage Learning, Multimedia C urriculum, Inte-grat ion of Technology, P rogra m Evaluat ion, Co llege-Level French Teach-ing, Co mputer-Assisted Lan guage Learning, Co mputer-Mediated Writing

1999 CALI CO Journal

7/27/2019 573-1466-1-SM

2/38

270 CALICO Journal

Evaluating the Integration of Technology

INTRODUCTION

Background

Although the use of personal computers has proliferated into the work-place as well as in academe, there is a paucity of published research on

computer-assisted language learning (CALL). Conrad (1996) reviewedseveral professional journals (Modern Language Journal; Foreign L anguage

Annals; Unterrichtproxis; French Review; Hispania; and System) publishedfrom 1992-1995 and reports that only 1.4% of the articles published ad-dressed empirical CALL studies. The majority of CALL articles are prac-

tical in nature and show case a softw are program or suggest guidelines forthe implementation of software. The few empirical CALL studies pub-

lished have focused on topics such as feedback (Brandl, 1995; Nagata,1993; Nagata & Swisher, 1995; Robinson, 1989, 1991); pronunciation

and intonation training (Stenson et al. , 1992); interactive reading(Svenconis & Kerst, 1995); computer-facilitated student interaction andtalk (Beauvois, 1992; Chun, 1994; Cononelos and Olivia, 1993; Meunier,

1994); wr iting (Armstrong & Yetter-Vassot, 1 994; Barnett, 1989; G reenia,1992); and cognitive style (Liu & Reed, 1994; Raschio, 1990).

Previous studies on technology and language learning have comparedthe effectiveness of a new technology w ith more traditional modes of learn-

ing. In general, these studies have examined learning over a short timeperiod such as a few days. For example, Ra schio (1990) explored the cog-nitive style (field dependent or field independent) of 62 first semester

Spanish students. The goal of both the control and experimental groups

was to learn how to form Spanish direct and indirect object pronounsover a two day period; day three was reserved for testing. The results didnot reveal any statistically significant relationships between the level of

field dependence, mode of instruction (printed vs. CALL), and studentachievement. Although the study did unveil some interesting findings re-garding student a ttitudes, Ra schio suggests t hat the profession needs to

modify its traditional research framework to include process variables,not just outcomes in second language learning.

Besides a lack of r esearch-oriented CALL ar ticles, few published evalu-ations of second language (L2) programs have integrated CALL into t he

curriculum. Johnson (1985) conducted a study on computer-assisted learn-ing to promote L2 acquisition, but her report focused primarily on suchissues as equal access, software development, computers in composition,

typical practices, and model programs. There appears to be no publishedarticle on language program evaluation intended to assess language skill

development and the integration of technology into the curriculum. 1

At the beginning of this decade, G arrett (1991) articulated a number of

research q uestions regarding program d evelopment and t echnology, What

7/27/2019 573-1466-1-SM

3/38



kind of software, integrated how and into what kind of syllabus, at whatlevel of language learning, for what kind of language learners, is likely tobe effective for w hat specific purposes? Herron a nd Moo s (1993) reiter-

ated t he aforementioned concerns by stating that One of t he major con-cerns facing the foreign language teac hing and literat ure profession is howto integrat e new technological advances into instruction.

This article reports the findings of a longitudinal program evaluationproject that focused on L2 skill development (listening, speaking, read-

ing, w riting, and culture) and the integration of t echnology-enhanced lan-guage learning (TELL). These findings can serve to inform foreign lan-

guage departments as they make important decisions concerning the useof technology in their programs.

Innovations in Technology Application

In the 1980s, CALL softw are featur ed a behaviora l or stimulus-responseapproach t o language learning by stressing mast ery of grammat ical prop-

erties or discrete-point learning (Co nrad, 1996; Johnson, 1992; Pusack &Otto, 1997). Much of the software was grounded in a transmission ap-

proach to learning. For example, Johnson (1992) reviewed the ESL soft-w are for language minority children and found tha t in general these pro-

grams focused on grammatical forms, many of which were not that par-ticularly useful. Recently, tremendous progress has been made in tech-nology, and the profession has broadened its interest in the role of tech-

nology to encompass a multimedia approach by including computers with

CD -RO M drives, videodisc players, video players, and hypermedia capa -bilities. In this respect, the preferab le use of multimedia ha s evolved froman add on component (C ALL) to a fully integrated feature of the foreign

language curriculum (TELL).Pusack and Otto (1997) advocate that the strength of multimedia is

the synergy derived from presenting content using a variety of modalities

(listening, reading, w riting, and speaking) that are linked together in mean-

ingful ways[italics added] to provide an in-depth experience. Multime-

dia a lso has the advant age of providing students w ith longer stretches oforal and w ritten discourse embedded w ithin a rich visual cultural context

for communica tion (G arrett, 1991). Technological adva nces, such as theWorld Wide Web, co nnect our st udents to up-to -dat e authentic realia a ndto native speakers. Multimedia thus has t he capability to stretch our cur-

riculum beyond the traditional walls of the classroom and to integratemuch needed sociolinguistic authenticity into our programs (Meunier,

1994).Besides expanding its orbit of interest to include a multimedia approach,

the profession has also redefined the role of technology in L2 learning.

7/27/2019 573-1466-1-SM

4/38

272 CALICO Journal


Similar to the reoriented role of the teacher from central to supportive inthe language learning process, we have witnessed a metamorphosis in therole of the computer from transmitter of know ledge and skills t o a t ool

that supports and assists learners to complete tasks (DeVillar & Faltis,1991). The profession now realizes that a co mputer is a medium for learn-

ing and not a method for L2 instruction.

Although t here are numerous benefits of integra ting technology into thecurriculum, w e w ill addr ess t hree specific areas w ithin the context of our

program evaluation: (a) computer-mediated writing, (b) the integrationof culture, and (c) the sociocultural benefits of technology.

Computer-Mediated Writing

The research concerning computer-assisted composing and the devel-

opment of L2 writing has produced diverse results. While some studiesindicate that students spend more time and write more at the computer(H aw isher, 1989; Roblyer et al., 1988), some w riters tend to plan less and

give less att ention to their ideas or co ntent (H aa s, 1989; Theismeyer, 1989).Costa nzo (1994) stresses tha t newer w ord-processing softw are incorpo-

rates a process-oriented approach to writing as a recursive process of dis-covery, elaboration, and revision. Bernhardt et al. (1989), Phinney (1988)

and R odrigues (1985) have stressed t hat m any of the studies that reporton computer-mediated writing have been short-term studies. Phinney(1991) argues that at least one semester is needed for positive improve-

ment in students writing to occur. Phinney (1989) also notes the impor-

tant role of the teacher to serve as a guide in assisting learners on how touse computers in the writing process, Without specific instruction in us-ing the computer to facilitate the w riting process, from prewriting to r evi-

sion, the computer alone, appears to have little effect in changing thew riting behavior of naive writers.

Studies of student motivation and attitude toward writing have pro-

duced more positive results (Daiute 1984; Neu & Scarcella; 1991). Forexample, Phinney (1991) found that computer composing reduced block-

ing problems and improved the students overall att itude towa rd w ritingin English. Phinney noted, how ever, the import ance of t he teachers role in

providing writing act ivities during regular class time which stressed brain-storming, dra fting exercises, peer-commenting, and revising of d raft s. St ud-ies investigating com puter netw orks used in L1 and L2 classes underscore

that these netw orks encourage more w ritten discussion and t herefore moreinvolvement on the part of the learner (Bump, 1990; DiMatteo, 1990;

Kelm, 1992).

7/27/2019 573-1466-1-SM

5/38



Integrating Culture through Technology

Multimedia capabilities such as audio/videotapes, CD-ROMs, videodiscs,the World Wide Web, E-mail exchanges, and electronic conf erencing, en-

able the profession to incorporate much needed sociolinguistic authentic-ity into the L2 classroom (Meunier, 1994). Lafford and Lafford (1997)

explain that access to multimedia capabilities facilitates the learners un-derstanding of the various social and psychological forces at w ork today

in the tar get culture and provides a co ntext in w hich students can int er-pret the behavior of the tar get culture inhabita nts. These authentic mat e-rials can serve as the tools to frame meaningful, communicative class-

room-based discussions. Furthermore, the use of video exposes studentsto native speakers using appropriate interact ional communication strate-

gies including nonverbal or kinesthetic behaviors.

Language Learning as a Social Phenomenon

Since language is a social phenomenon, language learning occurs t hroughsocial interaction involving teachers and more capable peers (Tharp &

G allimore, 1988; Vygot sky, 1978; Wertsch, 1979). S imilarly, m ultimediais now seen as a tool to enhance communication a nd discovery-orientedlearning, much of which can be accomplished by working cooperatively in

small groups. In this way, multimedia is a tool w hich assists the learners asthey work in their zones of proximal development (Vygotsky, 1978). Co-

operation w ith a t eacher, more capable peers, or w ith multimedia a llows

students to perform tasks that they w ould be incapable of performing ontheir ow n. G enerally, these shared a ctivities need t o be f ollowed by indi-vidual practice on a similar task until learners can perform the task inde-pendently (Cummins, 1991). Multimedia can thus assist learners to move

from the in terpsychological to the int rapsychological l evel(Vygotsky, 1978).Johnson (1991) found that integrating computers into the classroom

can promote cooperative learning.

Rather than isolating and promoting asocial behavior, as manyhad f eared, there is a grow ing body of evidence that computer usecan promote new w ays of w orking together, productive peer teach-

ing, as w ell as high qua lity social and aca demic task-based inter-action, and that these kinds of interaction are related to higher

levels of interest, motivation and achievement.

The teacher, of course, plays a c ritica l role in selecting com puter softw arethat w ill influence the sociocultural dynamic of cooperative group w ork(Johnson, 1992; Mydla rski, 1987; Piper, 1986).

7/27/2019 573-1466-1-SM

6/38

274 CALICO Journal


Foreign Language Program Evaluation

Brow n (1989, 1995) defines language program evaluation as t he sys-tematic collection and a nalysis of information necessary t o improve a cur-

riculum, assess its effectiveness and efficiency, a nd determine par ticipantsattitudes within the context of a particular institution. Brown (1989)

continues by saying t hat format ive evaluation occurs during the develop-ment of a program; the purpose of gat hering information is to improve the

program. Summat ive evaluation occurs at the end of the program, a nd thepurpose of gathering information is to determine whether the programw as successful.

Various evaluat ion models exist as w ell. A part icipato ry model of evalu-at ion can be preferable to an out side expert model because the lat ter oft en

encourages adversarial roles between the expert and the teacher. Conse-quently, Alderson a nd S cott (1992) recommend a collaborative or partici-

patory model which includes a consultant who provides expertise andguidance a long with the teacher(s) of the program. A part icipatory evalu-ation model centers on insiders though benefitting from the advice of

outsiders. Ross (1992) concurs with t he notion of a collaborat ive or par-ticipatory model for it decreases the teachers anxiety of being w atched

and allows direct participation by the teacher as both practitioner andobserver.

In addition, B eretta (1986) summarizes the strengths of two framew orks,field research and lab orat ory research, a nd a rgues for m ore field researchin language program evaluation. Beretta not es that field research is long-

term, classroom-based inquiry into the effects of a program and t hat labo-

ratory research is short-term and only involves the testing of individualcomponents of a t heory in an environment in w hich extraneous variablesare art ificially held constant .

A further consideration is the timing of the evaluation: during or af terthe program or both and fo r how long? Brow n (1995) examined languageevaluatio n programs bet w een 1988 and 1994. Thirteen of the 16 langua ge

evaluations were longitudinal in nature and ranged from two months tothree and a half years. These evaluations were conducted during the pro-

grams. Evaluations can also occur at t he end of a program. For example,Snow and Brintons (1988) evaluation used a retrospective format after

students had completed a content-based program. Pitiyanuw at (1986) notesthat the profession needs to gather data from students after they havefinished a progra m and act ually have to use the language. Brow n (1995)

suggests that the best framew ork w ould include evaluation during the pro-gram, immediately after, and lat er in a f ollow-up phase.

With regard to data collection, Alderson and Scott (1992) point outthat quantitative data are both easier to gather, and more amenable to

analysis and summary. However, a number of researchers (Beretta, 1986;

7/27/2019 573-1466-1-SM

7/38



D onat o, Anto nek, & Tucker, 1996; Lynch, 1990; R oss, 1992) st ress theneed to provide multiple perspectives by triangulating da ta . As Ross (1992)argues, both qualitative and quantitative types of evaluation can be uti-

lized with a view to defining a program, and both may provide alterna-tive view s of the same classroom phenomena.

Finally, B row n (1995) defines the evaluation of product as any evalua-

tion w hich is focused on w hether the goals (product) of t he program w ereachieved, as contrasted to process evaluation which centers on what is

going on in the program (process) tha t helps to a rrive at tho se goals (prod-uct). Long (1984) argues for doing bot h w hen he stat es using process

and product evaluations in combination, one ca n then determine not onlyw hether a program really w orks, or w orks better, but if so, w hy, and if not,why not?

THE PROJECT

Project Rationale and History

We conducted a semester-long evaluation of the integration of TELL

into a second semester French class. We chose to examine the impact ofreplacing one class per week with multimedia activities for four reasons:

(a) t he paucity of empirical research on t he effects of TELL o ver time, (b)the mandate from our university to make learning remote in time andspace, (c) the diff iculty students ha d enrolling in four-day-a-week courses,

and (d) the success we had had w ith computerized gramma r and vocab u-

lary exercises in our self-paced courses. Informal observation indicatedthat even though the self-paced students had minimal contact w ith a pro -fessor a nd ot her students, man y of them excelled in their w riting. Some

students mentioned using Dasher2exercises as a w ay to prepare for w rit-ten portions of the course exams.

Endeavoring to b e thorough in our program evaluat ion, we drew on the

strengths of multiple evaluative procedures. Although our evaluation wasprimarily summative in that we sought to establish whether the TELL

curriculum was successful in meeting our goals, we also conducted for-mat ive evaluat ions in order to improve cont inually the TELL c omponents.

According to a part icipato ry model, we did f ield research in a classroomcontext throughout the entire semester. We collected both quantitativeand qualitative data in order to assess both the process and product of

language learning.

7/27/2019 573-1466-1-SM

8/38

276 CALICO Journal


THE

PILOT

STUDY

During our spring 1996 pilot study, students in one section of second-semester French at Ca rnegie Mellon University (C MU) par ticipat ed in the

following multimedia activities in lieu of t he fourth contact hour: comput-erized gra mmar and vocabulary exercises using Dasher, instructional vid-eos, and optiona l use of an on-line spell-checker and French-English glos-sary. A control group followed the traditional syllabus, meeting four times

per week. Qualitat ive and quantita tive data on student performance, anxi-ety, a nd mot ivation w ere gathered to determine w hether segments of lan-guage courses could be reasonably expected to become remote in space

and time and still ensure that students develop an appropriate level ofproficiency in all skill areas.

We found parit y on the measures of anxiety, mot ivation, listening, read -ing, and w riting. Although bot h groups showed significant improvement

on the posttest of cultural understanding, the treatment group showedsignificantly more improvement than the control group (p = .019). Fur-thermore, there was a strong trend tow ard significance (p = .066) of the

treatment group scoring better on t he semester-end speaking test. G iventhese encouraging results, we revised the course and the study design to

answer more research questions. Specifically, whereas Dasher exerciseshad been the primary focus of the pilot study, the study currently being

reported incorporated a rota tion of multimedia-based reading, gram mar,vocabulary, and listening exercises with systematic speaking and writingfollow-up activities.

Research Questions

The primary purpose of our study was to investigate the feasibility and

desirability of replacing the fourth contact hour with multimedia activi-ties outside the classroom. Our evaluat ion of t his program w as intended

to a nswer the following questions:

1. Would the TELL experimental group students perform as w ell asthe control group in listening, speaking, reading, writing, and cul-

tural understanding?2. How w ould TELL aff ect student foreign language motivation and

anxiety?

3. How w ould TELL aff ect students perception of meeting the lan-guage learning goals they set?4. In what w ays would TELL modify the roles of the teacher and

students?

5. More generally, w ould the TELL course be an effective curriculardesign for achieving second language development in this context?

7/27/2019 573-1466-1-SM

9/38



Design of the StudyD ESCRIPTIONO FSTUDE NTS

We began the study by administering a demographic and langua ge back-

ground q uestionnaire in order to determine w hether the t reatment group(n = 17) and co ntrol group (n = 16) were similar. There were no signifi-

cant diff erences betw een the groups in terms of sex, age, grad e point aver-age and year in college. There were 5 wo men and 12 men in the treatment

group, and 7 w omen and 9 men in the control group. The students ran gedin age from 19 to 33, but most were of traditional college age. Studentshad declared ma jors in five of the seven colleges at C MU, representing the

humanities, social sciences, natural sciences, technology, fine arts, andbusiness. Despite CMUs reputation for technology, 21 of these 33 stu-

dents were majoring in the humanities, social sciences, or fine arts. Everystudent w as familiar w ith computers; 21 owned a com puter, and only one

student in the treatment group reported not being comfortable using acomputer.

Although t he majority of the students in bot h groups were American by

birth and native speakers of English, many students reported knowingother languages. Out of the total number of students, only two from the

control group reported that most of their experience in French had beenoutside the classroom. Half of the students in each group had taken the

first semester French course with the researcher/instructor the previoussemester. This facilitated conversation among the students, since manyalready knew each other and were familiar with the teaching style and

general course requirements.

COURSEDESIGN

We conducted the research o ver the course of the spring 1997 semester

in a second semester French course, w hich used chapt ers 6-10 of Allons-

y(Bragger & Rice, 1996). The treatment group met three days a week

from 10:30 to 11:20, and, in lieu of the fourth class period, students chosewhen and where to do the multimedia components. The control group

met four days a w eek, from 12:30 to 1:20, and prepared the same multi-media components that w ere assigned to the treatment group, but in class,on paper, or by w atching the video as a gro up. Both groups w ere assigned

identical writing homework and follow-up speaking tasks. The controlgroup completed these tasks in class and for homework. The treatment

group prepared the assignments on their own time. For both groups, thespeaking tasks were evaluated in class, and the writing tasks were col-

lected and graded.The challenge w as to incorporat e systema tica lly the four language skills

and cultural understanding into the students wor k on a w eekly basis. We

developed a 12-w eek rot at ion of three TELL com ponents that w ere famil-

7/27/2019 573-1466-1-SM

10/38

278 CALICO Journal


iar and had been successful in past courses: the CMU French Reader,Dasher exercises, and the Allons-yvideo.3 In addition, we had alreadytested the materials sufficiently to know tha t a fter a brief orientat ion, stu-

dents w ere unlikely to encounter t echnical diff iculties. We also cho se thesecomponents because they are accessible to many instructors interested inimplementing them in a tra ditional classroom setting.

The CMU French Reader, a computerized reading program featuringauthentic journalistic texts, pre- and postreading questions, a glossary

(French-English, English-French), culture notes, grammar notes, and awriting pad was created by Bonnie Youngs, the course instructor, and

Christopher M. Jones (1996). This HyperCard-based program is housedon the Modern La nguages server and is accessible on any Ma cintosh com-puter on ca mpus. We used the Reader to introduce each Allons-ychapter

through a text relevant to the chapter theme (e.g., fo od, lodging).The second multimedia component in the cycle consisted of Dasher

exercises developed by C MU facult y. Dasher (Pusac k & Otto, 1992) is anauthoring program w hich permits an instructor t o w rite exercises of vari-

ous types (e.g., multiple-choice, fill in the blank, sentence completion,listening comprehension, written and visual-based text). For this project,

we assigned Dasher exercises that emphasized the grammar and voca bu-lary of each chapter.

The third TELL component was the Allons-yvideocassette (Heinle &

Heinle, 1996). The video presents brief scenarios for each chapter per-formed by native speakers in France and G uadeloupe speaking at natural

speeds. Students had previewing questions and listening comprehensionexercises to do while watching the video. We offered the video last in the

cycle because it provided a synthesis of the chapter content.For each of the 12 weeks, stud ents in the treatm ent group received a lab

assignment sheet outlining (a) their reading, gra mma r/vocab ulary, or lis-

tening comprehension task a nd (b) their w eekly w riting homework a ndfollow-up speaking task (to be prepared for the next class period). (See

the Lab Assignment Sheet in Appendix A.) The writing and speaking ac-tivities provided a way for students to apply and develop what they had

learned through their ow n personal expression in simulat ed authentic ta sks.The lab assignment sheet also cont ained a checklist of the to ols stud ents

used while doing the lab, for example, their textbook, paper dictionary,

electronic glossary, and spellchecker. The English-French/French-Englishelectronic glossary (Carnegie Mellon University, 1996) is integrated into

the CMU French Reader, but also can b e accessed as a stand-alone pro-gram. The Spellchecker is a foreign language spell checking program. Inaddition, f or each w eeks lab, w e asked students to indicate the amount oftime they had spent, to rate the lab on a scale of 1 poor to 5 great, toexplain their rating, and to describe in a few words how they felt while

doing t he assignment.

7/27/2019 573-1466-1-SM

11/38



RESEARCH

DESIGN

In order to test our primary hypothesis that students using the TELL com -

ponents in lieu of one class period per week would perform French tasksin all the skills as well as student s meeting four t imes a week, w e collecteda variety of qualitative and quantitative data early, throughout, and near

the end o f t he semester (see Table 1).

Table 1Overview of D ata Collected from Treatment and Cont rol G roups

Ea rly-sem est er D ur ing-semest er End -of -sem est erda ta data data

Foreign Language 24-item measure 24-item measure

Motivation a

Foreign Language 20-item measure 20-item measure

AnxietybG oal Setting/Atta inment & open-ended open-endedAffective Questionnairec questionna ire q uestionnaire

Cultural Know ledge Quiz QuizListening 5 achievement tests

Speaking Pa ired Test Pa ired TestReading 5 achievement testsWriting Writing on first lab 5 achievement tests Writing on last

and on first w eekly lab follow - lab and lastachievement test d up activit ies achievement test

Time on Task 12 w eekly labs(treatment grouponly)

Students Perspectives 12 w eekly labs Open-ended( tr ea tm en t g rou p q uest io nn air ee

only) Follow -up

interviews (se-lected students,

treatment grouponly)

aThe Foreign Language Motivat ion measure was ad apted from Tremblay and G ardner (1995;

personal communication, November 1995).bThe Foreign Language Classroom Anxiety Scale was adapted from Horwitz and Young(1991).cThe Go al Setting and Affective Questionnaire wa s adapted from Cham ot (personal com-munication, November 1995). We also collected information about learning styles using

Reids Perceptual Learning Style Preferences Questionnaire (1987), but these data werenot conclusive.dBoth groups had the same weekly homew ork wr iting assignments and test items. An inde-

pendent rater used an ada pted form of the ESL Composition Profile (Glisan, 1981) to rate

early and late semester writing samples. Note that the first and last writing test items arealso reported as part of the MANOVA analysis of the achievement tests from throughoutthe semester.eStudents perceptions of improvement in listening, speaking, reading, writing, cultural

knowledge an d self-confidence were ga thered on an end-of-semester open-ended q uestion-naire. (S ee end-of-semester q uestionnaire in Appendix D .)

7/27/2019 573-1466-1-SM

12/38

280 CALICO Journal


As noted earlier, we first established that there were no significant de-mographic differences betw een the tw o groups and could t hus comparethe language performance of the two groups using t-tests and MANOVA

w ithout having to adjust for initial differences between them. Recogniz-ing that t he instructor and st udents w ere there first and foremost for do-ing a second semester French course, and only secondarily for carrying

out our study, w e balanced the number of da ta collection instruments w iththe need f or t ime-on-ta sk learning French.

Instead of viewing the classroom context as a contaminated researchenvironment, w e chose to describe, in the ethnographic tradition, as many

of the variables as possible so that readers could themselves interpret ourdata and determine the relevance (or generalizability) to their own in-structional contexts. In order t o support each finding w ith evidence from

different sources, we collected scores on pre- and posttests, grades onclassroom chapter tests, self-reports from the students and instructor, in-

terview data from a sample of students, student evaluations of each labassignment, check-lists of materials used, and records of time-on-task.

Our research team consisted of t he instructor/researcher w ho w as largelyresponsible for developing the curriculum and the TELL components and

tw o researchers who designed the study and co nducted most of t he dataana lysis. Tw o student researchers ob served the classes, summa rized muchof the qualitative data, and coded the quantitative data. An independent

rat er analyzed the w riting samples.

Findings

EVALUATINGLEARNERMOTIVATIONAND ANXIETY

In addition to demographic background, we examined whether the

groups differed on measures of foreign language motivation and anxiety.Both measures consisted of sta tements answ ered using a Likert scale rang-

ing from 1 strongly agree to 5 strongly disagree. We adapted 18 itemsabout students motivation t o learn French from Tremblay a nd G ardners

questionnaire designed for students of French in Canada (1995; personalcommunication, November 1995). These items reflected various aspectsof their expanded construct of language learning motivation (e.g., d esire

to learn French, motivational intensity, persistence, and instrumental andintegrative orientations). We then added six technology related questions

such as The thought o f using various types of technology to learn Frenchsounds interesting, and I w ould rather spend my t ime on act ivities that

do not require the use of technology, to determine whether there weredifferences between the two groups in their motivation to use technologyas a means of learning French.

7/27/2019 573-1466-1-SM

13/38



To examine language-learning anxiety we chose items from H orw itz andYoungs (1991) Foreign Language Classroom Anxiety Scale and onceagain added questions about technology and language learning such as

Doing French exercises on the computer sounds like fun, and Thethought of using various types of educational technology to learn Frenchscares me. There proved to be no significant difference in motivation or

anxiety between the groups either on the entire set of items, or on thetechnology clusters alone, at the beginning or end of the semester. Fur-

thermore, there was no significant change over the semester in motivationor anxiety.

On the final questionnaire, one student explained how TELL helped tolower her level of anxiety, Last semester, when we did the video as awhole class, I didnt have a clue what was going on. But this semester, I

could do it b y myself, or with a f riend, and it became so much easier. Now,Im not afra id of t he listening sections for our tests. I used to be so scared

of t hose.

EVALUATINGLEARNERS ATTAINMENTO FGOALS

Very much linked to students motivation to learn French and their lan-guage learning perfor mance is the setting of specific goa ls (Tremblay &

G ardner, 1995).4 Therefore, using a simple open-ended q uestionnaire, w easked students to state their personal goals for this course: Why are youtaking elementary French II? Please state your projected goals for this

course. Be as specific as possible. Near the end of the semester, we asked,

This is the end of the semester in elementary French II. Do you believethat you have made progress tow ard your projected goals for the course?

The goals student s reported w ere similar f or bot h groups. The majority

(treatment group, 10 of 17 and control, 11 of 16) had linguistic goalssuch as gaining fluency, especially in spoken French. Only one or two ineach group stated t hat they w ere taking French because it f ulfilled a gen-

eral education requirement. (There is no foreign language requirement atCM U.) O ther goals included learning about French/Francophone culture,

traveling, using French in their careers, and living abroad.At the end of the semester, 14 of the 17 students in t he treatment group

reported that they had met their goals, tw o said they almost had, and onesaid he or she had not m et the projected goa ls. In the contro l group, 11 ofthe 15 students who filled out the final goals questionnaire stated they

had made clear progress, and four students had made some progress, butnot a s much as they had hoped for. Treatment group students reported, I

am more confident and not afraid to speak, Im on my way to beingfluent, I have sharpened my French skills, I am more confident in my

reading, wr iting, speaking skills, I have almost met my goals. Contr ol

7/27/2019 573-1466-1-SM

14/38

282 CALICO Journal


group students reported, Yes, because my vocabulary has increased, Imon my way to being fluent, I built a foundation for learning the lan-guage, I have made progress but w ould have liked to have made more,

Not as much as I would have liked. More students in the treatmentgroup were positive about having made satisfact ory progress tow ard t heirprojected goals.

On the final q uestionnaire, students reported t hat having the flexibilityto individualize the use of the multimedia activities played an important

role in their progress and development of certain language skills: One ofthe great t hings ab out t he multimedia act ivities is that you can spend as

much time on them as you w ant, and I liked t he flexibility of the multi-media act ivities, so that if you feel that you need more practice, you cando it a gain and again. The multimedia act ivities challenged students to

take greater responsibility in working tow ard t heir goals for learning French.

EVALUATINGKNOWLEDGEO FCULTURE

We designed a culture q uiz to diagnose knowledge of bot h general fa ctsabout t he Francophone world, and specific facts about t opics treated t hat

semester (e.g., university studies, shopping, lodging). (See the culture quizin Appendix B.) The treat ment group significant ly outperformed t he con-

trol group on both t he prequiz and postquiz, w ith average scores of 68.4%and 76.0% versus 57.0% and 61.9% respectively (see Table 2).

7/27/2019 573-1466-1-SM

15/38



Table 2Cultural K now ledge, Speaking, and Writinga

Ear ly-semester da ta End-o f-semester da ta

Mean SD Mean SD

Cultural KnowledgeTreatment (n = 17) 68.4 13.5 76.0 12.9

Control (n = 16) 57.0 11.0 61.9 13.3

p< .05 p< .005b

Speaking TestTreatment (n = 17) 80.8 7.8 86.1 6.2

Control (n = 16) 78.9 13.8 91.1 7.2n.s. p < .05b

Test Writ ingTreatment (n = 17) 17.41 2.06 18.94 2.03Control (n = 16) 17.31 2.36 18.38 2.09

n.s. n.s.

Homework Writing

Treatment (n = 14) 15.21 3.29 17.93 2.30Control (n = 12c) 18.25 2.74 16.92 4.10

p< .05 n.s.d

aWe used t-tests to examine differences in mean scores betw een the treatment a nd contr olgroups.bWhen change over time w as taken into consideration, there w as no significant differencebetween the treatment and control groups in cultural understanding and speaking.cWhereas all the students took the exams, several students did not t urn in their homew ork.dThere wa s a significant d ifference between the groups in the w ay they changed over t ime;the control groups homework wr iting scores decreased, and the treatm ent groups wr iting

scores increased.

How ever, t here was no significant diff erence between the tw o groups in

average gain made over the semester (computed as postquiz score minusprequiz score).

EVALUATINGSPEAKING SKILLS

Early and late in the semester, students took an audio-recorded, pairedspeaking test administered by the instructor and then rated blindly by

anot her member of our research team . The speaking test fo cused on func-tional use of language and meaning-making and consisted of three parts:

picture description, a role play, and past narration cued by a series ofpictures. (See the oral test in Appendix C.) The early and late tests were

similar in format , but the tasks were somew hat a ltered such that students

7/27/2019 573-1466-1-SM

16/38

284 CALICO Journal


had to use the same skills in new contexts. On the pretest, there were nosignificant differences between the average scores of the treatment andcontrol groups (80.8% vs. 78.9%). Although the control group performed

significantly higher on the posttest (treatment group 86.1% and controlgroup 91.1%), there was no significant difference between the groups ingain made over the semester, computed as posttest score minus pretest

score (see Table 2).

EVALUATINGLISTENING , READING , AND WRITING

Throughout the semester, students took five achievement tests, eachone testing listening, reading and writing, all at a paragraph length dis-

course level. These tests w ere designed and grad ed by t he instructor.5

Weused a Multivariate Analysis of Variance (MANOVA) to examine the dif-

ference between the groups performance on these tests in each skill. Inlistening, there was no significant difference between the two groups (seeTable 3).

7/27/2019 573-1466-1-SM

17/38



Table 3Listening, R eading, and Writing Testsa

Listening Reading Writingb

Mean SD Mean SD Mean SDTest I

Treatment 83.7 6.9 91.7 7.7 80.5 7.8

Control 74.3 15.0 78.4 11.1 88.6 5.1

Test II

Treatment 90.2 18.5 91.7 5.9 80.0 6.7Control 90.6 12.9 78.4 8.1 88.4 5.8

Test II ITreatment 59.6 11.3 99.3 3.0 79.9 8.0

Control 65.0 11.5 98.3 4.4 77.3 12.1Test IV

Treatment 88.5 17.5 99.8 1.0 73.5 10.0

Control 75.3 28.4 99.7 1.0 62.3 10.0

Test V

Treatment 80.9 16.1 84.6 16.2 82.6 7.3Control 72.4 17.3 86.2 15.4 79.4 8.3

Note:For all tests, Treatment G roup (n = 17) and Control G roup (n = 16).

aWe used MANOVA to examine differences between mean scores for the treatment andcontrol groups in listening, reading and writing. For listening, there was no significant

difference between the groups. The treatment group scored significantly better than thecontrol group on reading (Wilks Lamb da = .41; F(5,26) = 7.62, p < .001) and writing

(Wilks Lamb da = .24; F(5,26) = 16.18, p < .001).bThe writing on the first and last test was also analyzed by an independent rater using adifferent scale. See Table 2.

In reading, t he treatment group performed significantly bett er than t hecontrol group (Wilks Lambda = .41; F(5,26) = 7.62, p. < .001). S imilarly,

in writing, the treatment group outperformed the control group (WilksLambda = .24; F(5,26) = 16.18, p. < .001).

In order to furth er investigate the difference betw een the groups in w rit-ing, we had an independent rat er analyze the writing on the first and la stachievement tests and the first and last homework writing assignment

(follow-ups to labs 1 and 12 for the treatment group and the analogoushomework done by the control group).6 There was no significant differ-

ence between the group means on the first test, on the final t est, nor in theamount of improvement show n by each gro up over t ime.

In the writing homework, however, the control groups mean on thefirst assignment w as significantly higher than t hat of the treat ment group(treatment group mean = 15.21; control group mean = 18.25, p< .05).

7/27/2019 573-1466-1-SM

18/38

286 CALICO Journal


The pattern of change from the first to the final homework assignmentdiffered significantly for the two groups (p< .05). The treatment groupscored higher on the final assignment than on the first, and the control

groups scores decreased (experimental gro up mean = 17.93; control groupmean = 16.92). G iven that both groups performed equally well on thetests, clearly one group did not learn to write better than the other. In-

stead, it appears t hat at the end of the semester, tr eatment group studentsw ere more motivated than control group students to d o q uality wr iting on

their homework, even though it counted for less of a gra de than the tests.Indeed, the instructor had noted that the treatment group wrote longer

and more complex homework compositions than the control group.

TIMEON TASKDOING THE TELL COMPONENTS

For each of the twelve weeks of TELL labs, the treatment group filledout a lab assignment sheet. (See the Lab Assignment Sheet in AppendixA.) Table 4 port rays the systematic cycling of the Reader, Dasher exer-cises and Allons-yvideo for each of the five chapters.

7/27/2019 573-1466-1-SM

19/38



7/27/2019 573-1466-1-SM

20/38

288 CALICO Journal


In three instances, two of these components were combined into onew eek in order to f it the 15 labs into 12 w eeks.Because the treatment group students w ere working outside of class, w e

asked them to report t he amount of t ime they spent on the lab assignmentso that w e could compare their time on task w ith that of the control group,w ho used the majority of 50 minute class periods doing the in-class ac tivi-

ties described earlier.7 With the exception of the second chapter (Chapter7), on w hich the average amount of t ime spent w as 44 minutes, t he aver-

age amount of t ime spent on each of the three tasks per chapter did notvary more than 7 minutes from the overall mean of 34.6 minutes. The

control group spent about 40 minutes per week on the in-class activities,analogous to what the treatment group did during their lab time. Thecontrol groups actual time on task per week is therefore very similar to

the treatment groups averages.As the semester progressed, students were able to do increasingly com-

plex tasks in approximately the same amount of time, that is, 28-37 min-utes (see Table 5).

Table 5

Lab Assignment: Average Time on Task a nd RankingAverage t im e o n t a sk Average r anking of t askin minutes (range) on 1-5 sca le (range)

By taskReader 43 (97) 3.5 (3.2)

D asher 36 (55) 3.4 (3.4)Video 24 (41) 4.2 (2.0)

By chapterChapter 6 37 (62) 3.8 (3.3)Chapter 7 44 (123) 3.6 (3.0)

Chapter 8 34 (63) 3.5 (3.0)Chapter 9 27 (35) 3.8 (2.3)

Chapter 10 30 (38) 3.9 (2.7)

Overall 34 (64) 3.7 (2.9)

Note that the range of time spent decreased from over an hour in thefirst three chapters to a little over a half hour in the final two chapters.

The reason for this trend is unclear; students possibly learned to use thetechnology more efficiently or simply felt they had less time to spend on

the labs as the semester progressed. In addition, the dramatic decrease intime spent doing Dasher exercises between lab assignments 7 and 9 nodoubt r esulted from our d ecision to t urn off Dashers triple repetit ion ofitems answ ered wrong before giving students the correct answ er. This rep-etition had proved frustrating to students and thus seemed to hamper

rat her than enhance learning.

7/27/2019 573-1466-1-SM

21/38



STUDENTS

EVALUATIONS

O F

THE

TELLCOMPONENTS

Over the semester, the average student ranking of the lab assignmentsby chapter was fairly constant, from 3.5 to 3.9 on a Likert scale of 1

poor to 5 great (see Table 5). The range in students assessments oflab assignments, however, decreased slightly. The waning novelty of the

TELL components a s w ell as the general semester-end anxiety c ollege stu-dents experience was perhaps counterbalanced by their familiarity with

and enjoyment of the lab com ponents.Of the three component types, students preferred the video (4.2 rank-

ing) to the Reader and Dasher (3.5 and 3.4 respectively). This preferencew as evident in the w eekly lab reports, the semester-end questionna ire (seethe final questionnaire in Appendix D), a s well as the five follow -up inter-

view s we conducted w ith treatment group students. In add ition to spend-ing the least amount of time on the video, students reported that they

often viewed the tapes with a classmate. The students explained how theircollaboration w as beneficial, Yes, w e collaborat ed, because we are bothstrong in different areas. Working together increased how much we got

out of the lab assignments. Another student sta ted, With the videos, if Icouldnt understand a part, t hen I wa s able to a sk the other student if she

knew. Between the both of us, we could work it out together. This spon-taneous pair w ork increased their ability to understand these videos. Stu-

dents also found the videos to be the richest source of cultural informa-tion.

In spite of some students being annoyed by the precision required by

Dasher, seven students reported tha t t hey liked Dasher. On the final ques-

tionnaire, they said, for example, that Dasher gave an opportunity tofocus on the finer details of grammar, and that Dasher gave good prac-tice by example. A few students critiqued Dashers identical treat ment ofall errors (e.g., grammar, spelling, accents).

On the final questionnaire, students were most critical of the Reader.One or tw o students praised t he readings for giving them first-hand ex-

perience in French and for being informative, but others found themtedious. The most substantial cr itique made by a f ew students on the se-

mester-end questionnaire and in the follow-up interviews was that thecurrent design of the Reader did not need to be electronic. It workedequa lly well w ith pencil and pa per. If, ho w ever, the students chose to printout the Reader assignment, t hey were unable to a ccess the glossary andthe cultural a nd gramm atical notes intended to inform their reading and

help them develop their reading strategies. Worthy of note is the fact thatthe Reader w as the one multimedia component that had not been pilotedin second semester French, so it follows that changes in its design andimplementat ion w ould be required.8

We asked the treatment group students to indicate w hether they felt the

7/27/2019 573-1466-1-SM

22/38

290 CALICO Journal


multimedia activities were effective in helping them develop the four lan-guage skills. Similarly, w e asked the control group to assess the Tuesdayin-class activities.9 More students in the treatment group indicated they

thought they had developed their reading and writing skills than in thecontro l group (15 vs. 12, a nd 16 vs. 11, respectively). The treatment groupsperceived development of writing skills is corroborated by their higher

test w riting scores overall (see Table 3) and their improvement on home-w ork w riting (see Tab le 2).

In bot h groups, 15 students reported tha t t he lab or a nalogous in-classactivities had helped them develop their listening skills. In ranking the

helpfulness of language learning resources, the treatment students placedthe video higher than d id the control group, perhaps because they w ereable to listen to it as often as they liked in the lab, whereas the control

group viewed t he tape no more than tw ice in class and had no control overits use.

When asked w hether the multimedia ac tivities w ere effective in helpingthem develop their speaking skills, 4 of t he treatment group students said

yes; 10 of t he control group student s said the Tuesday in-class ac tivitieshad helped them do so. In their comments, treat ment group students noted

that although the three multimedia components did not require speaking,their speaking improved through vocabulary building and listening to thevideos; It didnt really help my speech or accent but it did help my vo-

cabulary, I dont know, I think it [speaking] ties in with listening a lot,and Videos were valuable for listening and speaking.

In addition, when asked whether the follow-up speaking activities pre-pared for Wednesdays class were effective ways to synthesize and apply

w hat they had learned in the lab, 9 of t he treatm ent group said yes, a nd5 somewhat. They commented, Yes, they reinforced the ideas of thelab well, and Yes, we were forced to use what we had just learned,

making it stick. So, w hereas t hey were not required to speak French dur-ing the three TELL components, st udents recognized that the components

did help prepare them for the simulated authentic speaking classroomactivities.

Finally, the treatment group gave overwhelming support for the TELL3-day-per-w eek French course as an opt ion. Fifteen of 17 students sa id itshould be of fered, only tw o of w hom suggested some modifications.

THE CHANGING ROLESO FTHETEACHERAND STUDENTS

G iven the results of this study and its supporting aca demic research

base, it would seem appropriate to institute some form of TELL-basedcurriculum for fo reign language courses. H ow ever, implementing a TELL-

based curriculum may b e a challenging task for teachers, a dministrat ors,

7/27/2019 573-1466-1-SM

23/38



and students. Instructors may expect that using multimedia w ill decreasetheir w orkload and that multimedia curricula a re more innovative, therebyensuring the attention of students. Although the research appears promis-

ing, some fallacies regarding TELL contribute to its largely uncontrolledand unguided use. Its informed implementation requires constant evalua-tion of using technology to learn foreign language.

Designing a TELL curriculum is a new task for most teachers but, liketraditional curricula, requires a sound general pedagogy. In addition, it

demands a sound instructiona l technology methodology. If an instruct ionalcontext combines a modified tra ditional classroom setting w ith lab time,

then the instructor must not only develop materials for learning a foreignlanguage in the classroom but also implement technology to enhance thelearning already gained in the classroom. This combination is difficult to

manage and does not save the teacher any time in materials developmentand implementation. It is a new methodology that d emands practice, plan-

ning, and versatility.Moreover, not meeting a class on a fourth da y may appear to b e a labor

saver. On the contrary, in addition to the work of teaching in the tradi-tional classroom setting, there is an extensive list of steps for the teacher

who uses a lab or computers. These steps include integrating technology-based mat erials w ith trad itional materials, mastering the implementationof t echnology-based mat erials, learning how to use the hardw are, tr ouble-

shooting both ha rdw are and softw are, orienting students to using the hard-ware and software, being available to students who have technological

anxieties and difficulties, adapting technology materials that fail into aspur of the moment traditionally oriented lesson plan, and constantly

assessing student learning and the classroom/lab environments to ensurecontinued student success.

Nonetheless, some educators misuse TELL and view it as a panacea for

current administrative issues. For example, at a large midwestern univer-sity, there have been significantly increasing enrollments permitted in be-

ginning level foreign language classes and a drastically decreased amountof conta ct time w ith the instructor (Musumeci, 1998). Research has clearly

shown that smaller class size is a critical fa ctor in language development(Wiggins, 1997).10 Therefore, at every level, caut ion is required w hen imple-menting TELL .

Like teachers, students a lso make personal adjustments in a TELL-ba sedcurriculum. In the early stages, they learn t o use new hardw are and soft -

w are and to reschedule their homework periods around availab le lab ornetw ork time. Although students may be intrigued b y the idea of miss-

ing class time in favor of lab time, they are now responsible for w ork-ing outside the traditional classroom setting and for keeping pace with therequirements of their new curriculum.

7/27/2019 573-1466-1-SM

24/38

292 CALICO Journal


Having adapted successfully to a TELL curriculum, however, studentsdo find interesting w ays to use their time awa y from t he classroom. In thisstudy, as noted above, students spontaneously engaged in collaborative

learning situations, especially when using the video. Although they mayhave had some difficulties adjusting to a TELL-based curriculum, our stu-dents proved t hat given the t ools and t he time needed, they w ill ta ke the

initiative to become active language learners.Program evaluation, both formative/summative and process-/product-

oriented, is a vital aspect of building a TELL curriculum. The question isnot should we use TELL? but how can we most effectively use it to

enhance student learning? We must weigh all aspects of this curriculartype before deciding on its implementat ion. Students must learn how touse it, and teachers must know their students and be aware that student

input focuses the instructor on the best use of technology. In order toachieve a balanced TELL curriculum, teachers must assess the materials

from the students perspective and understand how such a curriculumwill influence student learning, not only cognitively but affectively.

Discussion

Let us summarize the findings by a nswering our research questions.

1. Will the TELL experimental group students perform as well as thecontrol group in listening, speaking, reading, w riting, and cultural

understanding?

Treat ment group students performed as w ell as cont rol group students

in listening, a nd t hey outperformed them in reading and w riting overallon the achievement tests (see Tab le 3). In writ ing homewo rk, there wa s a

significant difference between the groups in the way they changed overtime; the control groups homework writing scores decreased, and thetreatment gro ups w riting scores increased (see Table 2). More treatment

students than control group students indicated that they felt the TELLcomponents (or analogous in-class activities) had helped them develop

their writing skills. Fewer treatment group students perceived the labs ashaving helped them in their speaking, and they performed somewha t more

poorly than the control group on the final speaking test. There was, how-ever, no st atistica lly significant difference betw een the groups w hen change

over time on the pre- and postspeaking test was taken into consideration.On both the pre- and postquiz of cultural knowledge, the experimentalgroup did better than the control group; they continued to score signifi-

cantly higher, even though both groups improved considerably on thismeasure.

7/27/2019 573-1466-1-SM

25/38



2. How does TELL affect student motivation and foreign languageanxiety?

Neither group changed significantly over the semester in their motiva-

tion t o learn French or in their foreign language an xiety. It is possible that15 weeks is insufficient time to effect a positive change in either of theselanguage learning fa ctors.

3. How does TELL affect students perception of meeting the goals

they set?

In the treatment group, 14 of 17, and in the control group 11 of 15

students reported having clearly made progress toward the goals (82% &73%, respectively).

4. In what w ays does TELL modify the roles of the teacher and stu-dents?

The model of three class periods plus a lab assignment does not de-crease the teachers work. Instead, the instructors energies are channeled

in different directions such as evaluating, choosing, designing, adaptingsoftw ar e, serving as consultant t o students, assuring that the overall courselearning objectives are being met, and that the course is an integrated

w hole. St udents also ta ke on a new role as they gain the freedom to w orkwhen and where they choose but also face the responsibility of doing con-

siderably more w ork outside of class.

5. Is the TELL course an effective curricular design for achieving sec-ond language development?

After carefully monitoring and assessing language skills in listening,

speaking, reading, w riting, and cultural know ledge, w e recognize that theTELL course was indeed effective in helping our students learn French.

Its strength in this particular instructional context appeared to be in w rit-ing, and its weakness in speaking. We are pleased that students were able

to apply skills from the three componentsthe Reader, Dasher, and thevideoto the w riting of paragra phs and longer discourse. A few studentsmentioned tha t t hey improved in their speaking skills thanks to increased

vocabulary and listening to the videos. We are confident t hat w ith a f ewmodif icat ions, such transfer t o speaking in semi-authentic contexts can be

increased. We need, how ever, t o help students make the connection more

clearly and to incorporate speaking opportunities directly into the multi-media components.

7/27/2019 573-1466-1-SM

26/38

294 CALICO Journal


CONCLUSION

The follow ing are the highlights of w hat w e learned in this study:

1. The answ er to our primary research question is that students in theTELL gro up performed as w ell as t hose in the cont rol group on every

skill and o n cultural understanding a ccording to o ur stat istical mea-sures. Students reported that they felt the multimedia tasks helped

them in writing, reading, and listening, but less so in speaking.2. It is indeed feasible and, fo r the majority of the TELL students, desir-

able to replace one day in four with TELL. Fifteen of 17 studentsfavored offering the three class period plus TELL f orma t a s an option,and , despite tr emendous semester-end pressures, student s ma intained

their interest in the TELL componentsactivities which they had tocomplete on their ow n time. In add ition, the instructor and research-

ers were satisfied with student performance as well as their attitudetow ard t he curriculum.

3. Treatment group w riting scores on the five achievements tests as awhole were significantly higher than those of the control group. The

independently scored early and late-semester writing show ed tha t tr eat-ment students improved in their homework writing and were moti-vated to w rite well even on the final homew ork assignment.

4. An unexpected consequence was that t he TELL components promotedpositive and spontaneously occurring collabor ation a mong students

outside the classroom.

Let us conclude by noting some limitations of this study, followed byrecommendations for continuing the study of TELL.

1. We are generally satisfied w ith the revisions we made to the programevaluation design aft er carrying out the pilot. It w ould, of course, be

advantageous to have larger numbers of students or to collect dataover even longer periods. Indeed, the Department of Modern Lan-

guages at CMU has extended the project to first and second semesterclasses.

2. The multimedia components that w e used could be enhanced, a pro-

cess constantly being done in the French program a t C MU.a. The TELL component needs to integrate even more authentic,

contextualized discourse. For example, there is room for consid-

erable improvement in the grammar/vocabulary exercises whichmight entail using softw are that is more flexible than Dasher. Ingeneral, interaction among students and between students andauthentic cultural documents in the target language could be in-

creased. 11

7/27/2019 573-1466-1-SM

27/38



b. A further revision to the TELL treatment would be to improvethe Reader; some students saw no need for the Reader, as it iscurrently designed, to be electronic. The Reader could, for ex-ample, include links to w eb sites, more linguistic support (the useof which would be required in follow-up assignments) and theremoval of access to t he glossary during the initial reading in or-

der to promote sight-reading strategies.c. As described above, the speaking skill needs to be integrated more

directly with the TELL component.

We envisage numerous directions that this research of multimedia en-hanced instruction could take. With respect to students and learning is-sues, the following questions need to be addressed: What is the role of

student s locus of cont rol in where and w hen they use technology? (To beoptimal in the college setting, it seems that students need access to all

TELL components in their rooms at any time of the day.) What is thevalue of collaborative work among students using TELL outside the class-

room? How could TELL be ad apted t o enhance learning by ta rgeting thevarious m ultiple intelligences? Ho w could TELL c omponents be individu-

alized for remediation of students w ith particularly w eak language skillsin language learning? And finally, can TELL be developed to track effec-tively the linguistic development and cultural understanding of students?

Clearly, the examination of the integration of technology into second lan-guage teaching and learning is fertile ground for future research.

7/27/2019 573-1466-1-SM

28/38

296 CALICO Journal


APPENDIX A

Lab Assignment Sheet

le franais 102A/Youngs/Printemps 97 Exercice de labo # 1Ltudiant(e) D ate:

Fill out this checklist for eac h assignment you do using multimedia or lan-

guage software.

Ta sk Wher e? H ow long ? Assessm ent S ca le: Check tools used1 (poor) to 5 (great)

Reader Fre G lossary (elec.)Chapt. 6 1 - 2 - 3 - 4 - 5 Paper D ictionary(server) Netscape

DasherModLang S erver

LLRC computersSpellchecker

G ramma r referenceAllons-ytextbookVerb bo ok

Ot her? (please list)

Additional informa tion (you may be brief):

1. Please comment on w hy you chose the assessment (1-5) above.

2. How did you feel during this assignment? (upset, nervous, enthusias-tic, interested, bor ed...)

3. Les Devoirs (essayez dutiliser le glossaire franais):

7/27/2019 573-1466-1-SM

29/38



a. A lcrit: Q uest-ce que vous fa ites pour vous prparer pour la rentre?

b. A loral: Pr parez un dialo gue entre vous et le vendeur ou la vendeuse

chez la librairie. Achetez tout ce quil vous fa ut pour la rentre.

APPENDIX B

Francophone Cultural K now ledge Quiz

Answer the questions in English, or give the French expression where

asked.1. Define Francophone.

2. Name five Francophone countries (not France).3. Name five cities in France.

4. Name denominations of French and Canadian currency.5. Name as many forms of public transportation in Paris as you can.6. What can you use, besides coins, to make a phone call in France, and

w here w ould you go t o get one?7. a . Name a French po lit ical f igure.

b . Name a French f ilm star.

c . Name a French scientist .

7/27/2019 573-1466-1-SM

30/38

298 CALICO Journal


8. a. What do you say if someone says merci to you?

b. How do you say youre feeling fine?

9. What are some things that French people typically have for breakfast?

10. What is a croq ue-monsieur?a . a dr ink b. an opera c. a sa nd w ich d. a t ype of building

11. Label the order in which the following items are typically served dur-ing a French dinner:

saladcheesecoffee

main dishdessert

12. What w ould you purchase at a charcuterie?13. Which best describes a tra ditional French university.

a. a complex of classroom buildings and sports facilitiesb. a sprawling suburban campus

c. an urban campus, no green areas14. Which field of study does lettres refer to?

a. education b. fine arts c. humanit ies d. sciences

15. Where do most French university students live?a. in rented rooms in town b. in university dorms

c. w ith their families16. How are floors of a building numbered in France?

17. What s a deux-pices?a. a coin b. an apartment c. a theater d. a vehicle

18. What d oes a green cross refer to?

a . a church b. a pha rma cy c. a polit ica l pa rt y19. What are the follow ing (city, region, country) and w here are they lo-

cated? What do you associate w ith each one?What? Where? Associat ion?

a . G ua deloupeb. Moroccoc. Louisia na

d. Normandye. Bordeaux

20. How wo uld you describe French people to someone from your coun-try? What stereotypes do you have of French people? Why?

7/27/2019 573-1466-1-SM

31/38



APPENDIX C

Or al Test G rading G rid

Picture description/comparisonVocabulary/content 5 4 3 2 1 0

Structures 5 4 3 2 1 0Accomplishes the task 5 4 3 2 1 0 /15

Role playVocabulary/content 5 4 3 2 1 0Structures 5 4 3 2 1 0Level of formality 3 2 1 0

(polite)Accomplishes the task 5 4 3 2 1 0 /18

Telling a story in the pastVocabulary/content 5 4 3 2 1 0Structures 5 4 3 2 1 0

Accomplishes the task 5 4 3 2 1 0 /15

OverallCommunication strategies 3 2 1 0

(asking for clarity,

getting point across)Pronuncia t ion/fluency 3 2 1 0 /6

(generally accurate pronunciation,not groping for w ords)

TOTAL /54

7/27/2019 573-1466-1-SM

32/38

300 CALICO Journal


APPENDIX D

Final Questionnaire (for t reatment group)

1. What did you like most about the multimedia activities (Da sher, reader,video)?

2. What did you like least about the multimedia activities (Da sher, reader,video)?

3. Were they effective in helping you develop the following skills inFrench? Yes or no? Please explain your answers.

ReadingWritingListening

Speaking4. Were they effective in helping you to increase your know ledge of

francophone culture? Why or why not ?5. At times w ere you frustrated with the multimedia activities? If so,

please tell w hich aspects frustrated you.6. D id the benefits of the multimedia activities outweigh your frustra-

tion? Please explain.7. D id the multimedia activities help your self-confidence to speak, read,

w rite and listen to French? Please explain.

8. D id you ever do parts of the lab in collaboration w ith another stu-dent? Yes or no (circle one)? I f yes, do you feel that w orking together

facilitated your learning of French?9. Were the speaking and w riting follow-up activities done on/for Wednes-

day effective ways to synthesize and apply what you had learned inthe lab? Please comment:Speaking activities:

Written work:10. Rank t hese language learning resources from 1 (most helpful) to 7

(least helpful). Feel free to comment.Allons-ytextbook

Allons-yvideoDasher exercisesFrench glossary (electronic)

paper dictionaryReader (electronic)

Spellchecker

7/27/2019 573-1466-1-SM

33/38



NOTES1 Nieves (1994) did a pilot study exploring the use of a multimedia program inSpanish entitled Exito. The project included the development of a first-semesterSpanish course using multimedia, classroom-based activities based on the multi-

media materials and assessment of the students language development. The re-search project highlighted that students could develop a beginning level of profi-ciency in Spanish by interacting with the multimedia program and by meetingwith a professor in small groups one period per week.

2 We used the Dasher authoring system to create vocabulary and grammar exer-cises (Pusack & O tto, 1992).

3 It is not our intent to promote any particular software, video, or t extbook mate-rials. Since we were working at Carnegie Mellon University, we chose materials

already adopted and t ested by the D epartment of Mod ern Languages.

4 Following Chamot, (personal communication, November 1995), we also col-

lected information about students affect at the beginning and end of the semes-ter. It is beyond the scope of th is article to report those findings.

5 We would ha ve preferred to administer externally rated pre- and posttests on a llskills but this would have taken too much class time and would thus have beenunfair to the students. Therefore, we chose to use data from classroom tests.

6 The writing w as ana lyzed using an ad apted form o f the ESL Composition Profile(G lisan, 1981).

7 G iven the w ide range of time students reported, w e wo nder if some students

may ha ve mistakenly included the time they spent preparing the fo llow-up writing

and speaking assignments in the early w eeks.

8 We chose to use the Reader because the coursewa re had been used successfully

and repeatedly in fourth semester French classes at Carnegie Mellon University.Ho w ever, neither the courseware nor the readings themselves had been piloted insecond semester c lasses.

9 Appendix D contains the final open-ended questionnaire administered to thetreatment group. Control group students filled out an a dapted q uestionnaire.

10 At Carnegie Mellon University, the Department of Modern Languages offers

instructors the option of capping classes at 18 students.

11 In 1998-1999, CMU does not use the Reader in the first year French curricu-lum but instead requires students to work with documents on the web. These

documents afford even greater authenticity and provide completely up-to-dateinformation (e.g., weather reports, news).

7/27/2019 573-1466-1-SM

34/38

302 CALICO Journal


REFERENCES

Alderson, J. C ., & Scott, M. (1992). Insiders, outsiders and participatory evalua-tion. In J. C . Alderson (Ed.), Evaluati ng second lan guage education(pp.

25-58). Cambridg e, UK : C ambridge University Press.

Armstrong, K . M., & Yetter-Vassot, C. (1994). Transfor ming teaching through tech-

nology. Foreign Language Annals, 27, 475-486.

Barnett, M. A. (1989). Writing as a process. French Review, 63(1), 34-44.

Beauvois, M. H. (1992). Computer-assisted classroom instruction in the foreign

language classroom: Conversation in slow motion. Foreign LanguageAnnals, 25, 455-463.

Beretta, A. (1986). A case for field-experimentation in program evaluation. Lan-

guage Learni ng, 36, 295-309.

Bernh ard t, S. A., Ed w ards, P., & Wojahn, P. (1989). Teaching c ollege compositionwith computers. Wri tten Communi cation, 6(1), 108-133.

Braegger, J. D ., & Rice, D . B. (1996). Al lons-y: L e Franai s par tapes(4 th ed.).Boston, MA: Heinle & Heinle.

Brand l, K. (1995). Strong and weak student preferences for error feedback optionsand responses. The M odern Language Journal , 79, 194-211.

Brow n, J. D . (1989). Language program evaluation: A synthesis of existing possi-bilities. In R. K. Johnson (Ed.), The second language curri culum(pp.222-241). Cambridge, UK : C ambridge University Press.

Brow n, J. D. (1995). Language program evaluation: D ecision, problems and solu-

tion. Annual Review of Applied Li nguistics, 15, 227-248.

Bump, J. (1990). Radical changes in class d iscussion using networked computers.

Computers and the Humani ties, 24, 49-65.Chun, D. M. (1994). Using computer networking to facilitate the acquisition of

interactive competence. System, 22, 17-31.

Con onelos, T., & Olivia , M. (1993). Using comput er netw orks to enhance fo reignlanguage/culture education. Foreign Language Annals, 26, 527-534.

Conrad , B . K . (1996). C ALL: non-English L2 instruction. Annual Review of Ap-

pli ed Li nguistics, 16, 158-181.

Costanz o, W. (1994). Reading, w riting and thinking in an a ge of electronic literacy.In C. L. Selfe & H. Susan (Eds.), Li teracy and computers(pp. 54-73).New York: The Modern Language Association of America.

Cummins, J. (1991). Introd uctory remarks. In R. D eVillar & C. Faltis (Eds.), Com-puters and cult ural di versit y: Restru ctur in g for school success(pp. vii-

ix). Albany, NY: State University of New York Press.

D aiute, C. (1984). Ca n the computer stimulate w riters inner dialogues? In W.

Wrench (Ed.), The computer in compositi on in structi on. Urbana, IL:NTCE.

D eVillar, R., & Faltis, C . (1991). Computers and cultu ral diversit y: Restructu rin g

for school success. Albany, NY: State University of New York Press.

7/27/2019 573-1466-1-SM

35/38



D iMatteo, A. (1990). Und er erasure: A theory of interactive w riting in real life.Computer and Compositi on, 7, 71-89.

D onato, R., Antonek, J., & Tucker, G . R. (1996). Monitoring and assessing a Japa-

nese FLES program: Ambiance and achievement. Language Learning,46(3), 497-528.

G arrett , N. (1991). Technology in the service of langua ge learning: Trends andissues. The M odern Language Journal , 75(1), 74-101.

G lisan, E. (1981). ESL Composition profile. In H. L. Jacobs, S. Zingraf , D. Wormuth,V. Hartfield, & J. Hughey (Eds.), Testin g ESL composit ion : A practicalapproach(pp. 323-324). Rowley, MA: Newbury House.

G reenia, G . D . (1992). Computers and teaching composition in a foreign language.Foreign Language Annals, 25, 33-47.

Hass, C. (1989). How the writing medium shapes the writing process: Effects of

word processing on planning. Research in t he Teaching of Engli sh, 23,181-207.

Ha wisher, G . E. (1989). Research and recommendations for computers and com-

position. In G . E. H aw ser, & C. L. S elfe (Eds.), Crit i cal perspecti ves oncomputers and compositi on in structi on(pp. 44-69). New York: Teach-ers College Press.

Herron, C., & Moos, M. (1993). Electronics media in the foreign language andliterature classroom: A fusion between science and the humanities. For-eign Language Annals, 26(4), 478-490.

Horwitz, E. K., & Young, D. J. (1991). Language anxiety: From theory and re-search to classroom impli cations. Englewood C liffs, NJ: Prentice Hall.

Johnson, D . M. (1985). Usin g computers to promo te the development of English as

a second lan guage: A report t o the Carnegie Corporat ion. Tuscon: Uni-

versity of Arizona. (ED RS:ED 278 211).

Johnson, D. M. (1991). Second language and content learning with computers:

Research in the role of social fact ors. In P. D unkel (Ed.), Computer-as-

sisted lan guage learni ng and testi ng: Research issues and practi ce(pp.61-83). New York, NY: Newbury House.

Johnson, D. M. (1992). Approaches in second language learni ng. White Plains,NY: L ongman P ublishing C ompany.

Kelm, O. R. (1992). The use of synchronous computer networks in second lan-

guage instruction: A preliminary report. Foreign Language Annals, 25

(5), 441454.

Lafford , P. A., & Lafford, B. A. (1997). Learning language and culture w ith internet

techno logies. In M. D . Bush, & R. M. Terry (Eds.), Technology-enhancedlanguage learni ng(pp. 215-262). Illinois: Nation al Textbook Com pany.

Liu, M., & Reed, W. M. (1994). The relationship between learning strategies and

learning styles in a hypermedia environment. Computers in H uman Be-havior, 10(4), 419-434.

Long, M. H. (1984). Process and product in ESL program evaluation. TESOL

Quarterly, 18, 409-425.

7/27/2019 573-1466-1-SM

36/38

304 CALICO Journal


Lynch, B. (1990). A context-adaptive model o f progra m evaluation. TESOL Quar-terly, 24, 23-42.

Meunier, L. E. (1994). Computer-assisted language instruction and cooperative

learning. Appli ed Language Learnin g, 5(2), 31-56.

Musumeci, D . (1998). Mallard, a web-based applicati on i n use at the Uni versit y ofI l l inois(technology demonstration). Pittsburgh, PA: U niversity of P itts-burgh.

Mydlarski, D . (1987). Cooperative computer-assisted language learning: Is it livingup to its promise? Journ al of Educati onal Techni ques and Technol ogies,20(1), 26-29.

Nagata, N. (1993). Intelligent computer feedback for second language instruction.The M odern Language Journal, 77, 330-339.

Nagata, N., & Swisher, V. M. (1995). A study of consciousness-raising by com-

puter: The effect of metalinguistic feedback on second language learn-ing. Foreign Language Annals, 28, 337-347.

Neu, J., & Scarcella, R. (1991). Word processing in the ESL writing classroom. In

P. D unkel (Ed.), Compu ter-assisted language learni ng and testi ng: Re-search issues and practi ces(pp. 169-187) New York: Newbury House.

Nieves, K. A. (1994). The development of a technology-based class in beginning

level Spanish: Experiences with Exito. (Doctora l dissertation, G eorgeMason University).

Phinney, M. (1988). Computers, composition and second language teaching. In M.

C. Pennington (Ed.), Teachi ng languages wi th computers: The state ofthe art. La Jolla, C A: Althesan.

Phinney, M. (1991). C omputer-assisted writing and writing apprehension in ESL

students. In P. D unkel (Ed.), Compu ter-assisted language learni ng andtesti ng: Research issues and practi ces(pp. 189-204). New York: New buryHouse.

Piper, A. (1986). Conversation and the computer: A study of the conversationalspinoff generated among learners of English as a foreign language work-ing in groups. System, 14, 187-198.

Pitiyanuwa t, S . (1986). On-the-job behavior of tra inees as a function of their reac-tion to th e training program and learning achievement. In A. Wangsoto rn,A. Maurice, K. Prapphal, & B. Kenny (Eds.), Trends in lan guage pro-gram evaluati on(pp. 142-154). Bangkok: Chulalongkorn University.

Pusa ck, J. P., & Ot to, S . (1997). Taking contr ol of multimed ia. In M. D . Bush & R.M. Terry (Eds.), Technology-enhanced language learni ng(pp.1-46).Lincolnwood, IL: National Textbook Company.

Raschio, R. (1990). The role of cognitive style in improving computer-assisted lan-guage learning. H ispania, 73(2), 535-541.

7/27/2019 573-1466-1-SM

37/38



Reid, J. M. (1987). The learning style preferences of ESL students. TESOL Quar-terly, 21, 87-111.

Robinson, G . L. (1989). The CLCC S C ALL study: Methods, error feedback, atti-

tudes, and achievement. In W. F. Smith (Ed.), Modern technology inforeign language educati on: Appli cation an d projects(pp. 119-134).Lincolnwood, IL: National Textbook Company.

Robinson, G . L. (1991). Effective feedback strategies in CALL: Learning theoryand empirical research. In P. D unkel (Ed.), Compu ter-assisted languagelearn ing and testing: Research issues and pr actice(pp. 155-167). New

York: Newbury House.

Roblyer, M., Castine, W., & King, F. (1988). Assessin g the impact of computer-based in structi on. New York: Haworth Press.

Rodrigues, D . (1985). Computers and ba sic writers. College Compositi on and Com-

muni cation, 36(3), 336-339.Ross, S. (1992). Program-defining evaluation in a decade of eclecticism. In J. C.

Alderson & A. Beretta (Eds.), Evaluati on i n second l anguage education

(pp. 167-195). Cambridge, UK: Camb ridge University Press.

Snow, M. A., & Brinton, D. M. (1988). Content-based instruction: Investigatingthe effectiveness of t he adjunct mod el. TESOL Quarterly, 22, 553574.

Stenson, N., D ow ning, B., S mith, J., & Smith, K . (1992). The effectiveness of com-puter-assisted pronunciation training. CALICO Journal, 9(4), 5-18.

Svenconis, D . J., & Kerst, S. (1995). Investigating the teaching of second language

vocabulary through semantic mapping in a hypertext environment.CALICO Journal , 12(2/3), 33-57.

Tharp, R., & Ga llimore, R . (1988). Rousing minds to li fe: Teachin g, learnin g, and

schoolin g in social context. Ca mbridge, UK: Cambridge U niversity Press.

Thiesmeyer, J. (1989). Should we do wha t w e can? I n G . E. H aw ser, & C. L. Selfe

(Eds.), Criti cal perspectives on comput ers and composition in struction.

New York: Teachers C ollege Press.

Tremblay, P. F., & G ardner, R. C . (1995). Expanding t he motiva tion con struct inlanguage learning. The M odern Language Journal , 79(6), 505-520.

Youngs, B. E., & Jones, C. M. (1996). CMU French Reader [Computer software].Pittsburgh, PA: Carnegie Mellon University.

Vygotsky, L. S. (1978). M ind in society: The development of hi gher psychologicalprocesses. Ca mbridge, MA: Harvard U niversity Press.

Wertsch, J. (1979). From social interaction to higher psychological processes: Aclarification and application of Vygotskys theory. Human Development,22, 3-22.

Wiggins, G . (1997). Educati ve assessment : D esigning assessments to i nform andimprove student performance. San Francisco, CA: Jossey-Bass.

7/27/2019 573-1466-1-SM

38/38

306 CALICO Journal


ACKNOWLEDGMENTS

We are grateful to G . Richa rd Tucker, Ba rba ra Freed, Christopher M. Jones,Keiko Koda and B rian MacWhinney for their feedback on t he pilot study

and their encouragement throughout this project. We are indebted to ElaineRubinst ein in the Of fice of Measurement and Eva luation of Teaching atthe University of Pittsburgh for conducting the statistical analysis, to

Michael West for his help in draft ing the culture quiz, and to Mark Sanfo rdfor rat ing the writing samples. We also w ant t o acknow ledge the work of

our tw o undergraduate research assistant s during the spring of 1997: EnaKhan and Emily Spencer.

AUTHORS ADDRESSES

Bonnie Adair-Hauck

University of PittsburghDepartment of Instruction and Learning

4M20 Forbes QuadPitt sburgh, Pa 15260Phone: 724/935-8275

Fax: 724/934-8832E-mail: [email protected]

Laurel Willingham-McLain, Associate Director

Center fo r Teaching Excellence

312 Administration BldgD uquesne University

Pittsburgh, PA 15282Phone: 412/396-1760

Fax: 412/396-6577E-mail: w [email protected]

Bonnie Earnest Youngs

Department of Modern LanguagesCarnegie Mellon UniversityBaker Ha ll 160

Forbes AvenuePit tsburgh, PA 15213-3890

Pho ne: 412/268-8050 (o ff ice); 412/621-2517 (home)Fax: 412/268-1328

E-mail: [email protected]

573-1466-1-sm

Documents