alternative assessment in the preparation of literacy educators: responses from students

This article was downloaded by: [Staffordshire University]On: 06 October 2014, At: 21:26Publisher: RoutledgeInforma Ltd Registered in England and Wales RegisteredNumber: 1072954 Registered office: Mortimer House, 37-41Mortimer Street, London W1T 3JH, UK

Reading PsychologyPublication details, includinginstructions for authors andsubscription information:http://www.tandfonline.com/loi/urpy20

ALTERNATIVEASSESSMENT IN THEPREPARATION OFLITERACY EDUCATORS:RESPONSES FROMSTUDENTSDiane D. Allen & Rona F. FlippoPublished online: 19 Jan 2011.

To cite this article: Diane D. Allen & Rona F. Flippo (2002)ALTERNATIVE ASSESSMENT IN THE PREPARATION OF LITERACYEDUCATORS: RESPONSES FROM STUDENTS, Reading Psychology, 23:1,15-26, DOI: 10.1080/027027102317345385

To link to this article: http://dx.doi.org/10.1080/027027102317345385

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy ofall the information (the “Content”) contained in the publicationson our platform. However, Taylor & Francis, our agents, and ourlicensors make no representations or warranties whatsoever asto the accuracy, completeness, or suitability for any purposeof the Content. Any opinions and views expressed in this

http://www.tandfonline.com/loi/urpy20

http://www.tandfonline.com/loi/urpy20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/027027102317345385

http://dx.doi.org/10.1080/027027102317345385

http://dx.doi.org/10.1080/027027102317345385

publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. Theaccuracy of the Content should not be relied upon and shouldbe independently verified with primary sources of information.Taylor and Francis shall not be liable for any losses, actions,claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arisingdirectly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and privatestudy purposes. Any substantial or systematic reproduction,redistribution, reselling, loan, sub-licensing, systematic supply,or distribution in any form to anyone is expressly forbidden.Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

15

Reading Psychology, 23:15–26, 2002Copyright © 2002 Taylor & Francis0270-2711/02 $12.00 + .00

Address correspondence to Diane D. Allen, Associate Professor of Reading, Universityof North Texas, Department of Teacher Education & Administration, P. O. Box 311337,Denton, TX 76203. E-mail: [email protected]

ALTERNATIVE ASSESSMENT IN THE PREPARATION OFLITERACY EDUCATORS: RESPONSES FROM STUDENTS

DIANE D. ALLEN

University of North Texas, Denton, Texas, USA

RONA F. FLIPPO

Fitchburg State College, Fitchburg, Massachusetts, USA

This study examined attitudes and concerns related to the use of alternativeassessments in literacy education courses. A Likert-style twenty-item questionnairewas developed and administered first as a pilot to determine item quality anddimensionality. Three subscales were identified: Self-Evaluation, PeerEvaluation, and Instructor Modeling. The questionnaire was then administeredto two sections of one literacy assessment course as pre- and post-courseassessments. The results were significant for Self-Evaluation and PeerEvaluation. Results for Instructor Modeling did not reach significance but weregenerally positive in support of modeling alternative assessment in educationclasses. Qualitative analysis of respondent comments supported the statisticalfindings.

Traditionally, test scores and grades for specific course work as-signments have been used to document students’ success, or lackof success in teacher education/literacy courses. However, as wemove our instruction to fit with a more transactive, constructiveview of learning, we are forced to look for assessment and evalua-tion techniques that support this view. The need for change at theuniversity level is also supported by public school implementationof alternative assessments (Cirincione & Michael, 1994; Valencia,1990). If new teachers are to effectively utilize these new methodsof assessment they must have more than just a surface knowledgeof these alternative assessment strategies. Crow’s study (1987) in-dicated that coursework itself had little effect on a new teacher’sbeliefs due to the effect of prior knowledge and experience; rather,preservice teachers needed new experiences to change belief sys-

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014

16 D. D. Allen and R. F. Flippo

tems. Providing preservice teachers experience in the use of alter-native assessments through assessment and evaluation of their owncoursework may allow them to construct a deeper understandingof the empowerment self-assessment brings.

Many of those who work with teacher education assessment havebeen experimenting with various schemes for group, peer, andself-evaluations as part of the course work for undergraduate andgraduate teacher education classes (Allen, 1996; Commeyras,Reinking, Heubach, & Pagnucco, 1993; Ford & Olhausen, 1991;Stahle & Mitchell, 1993; Wagner, Brock, & Agnew, 1994; Yore &Craig, 1994). These teacher educators have implemented alterna-tive assessment and evaluation methods because they believe thatteacher education students need alternative models for assessingtheir current or future students, and that one effective way of pro-viding models for preservice teachers is to actually use them incollege classrooms (Stahle & Mitchell, 1993). Teacher educationstudents will learn as much, if not more, of the assessment andevaluation concepts being taught, through personal utilization ofvarious group, peer, and self-evaluation schemes for their ownlearning. This leads to student empowerment and the acceptanceof responsibility by the students for their own learning.

While teacher educators are positive in their evaluation of alter-native assessments at the university level, few research studies haveanalyzed the attitudes and responses of preservice teachers to theuse of these types of assessments (Ford & Olhausen, 1991; Frazier,Palmer, Duchein, & Armato, 1993; Stokes, 1994; Yore & Craig,1994). The Yore and Craig (1994) study investigated preserviceteachers’ attitudes regarding portfolio assessment. Initially, a ma-jority of the students (95%) were anxious about the assessment;however, ninety-nine percent of these students later rated portfo-lios as very significant in their learning for the term. One study(Frazier et al., 1963) reported an improvement in student self-esteem which translated into a more positive attitude toward read-ing in general. Both the Frazier et al. study and an earlier study byFord and Olhausen (1991) indicated that students felt their knowl-edge of reading and appropriate assessments had increased. Stokes(1944) reported that students in a content-area reading courseresponded positively to a survey after developing portfolios forthat course.

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014

Alternative Assessment in Literacy Education 17

This research project expands on these studies by analyzing spe-cific areas of concern for students as they experience authenticassessments in the university classroom. Specifically, these ques-tions were investigated: 1) What impact does the use of alternativeassessments have on student empowerment and confidence to self-evaluate and to evaluate the work of peers? 2) What impact dostudents think alternative assessments will have on their learningin the course? 3) What impact do students think alternative assess-ments will have on overall assessment practices and grades at theiruniversity?

Method

Questionnaire

A questionnaire was developed for undergraduate and graduateteacher education students that addressed attitudes, understand-ings, and concerns related to alternative assessment measures (seeAppendix). Students responded on a Likert-type scale with re-sponses ranging from “strongly agree” to “strongly disagree.” Thestudents were also encouraged to add any additional comments toexplain their responses. An initial piloting of a twenty-item ques-tionnaire was conducted during the fall semester. Responses wereanalyzed to determine item quality and dimensionality. A revisedquestionnaire (20 items) was then administered at the beginningof the spring semester to a group of undergraduates enrolled in areading assessment course. The same questionnaire was also usedas an-end-of-course measure. Multivariate and univariate signifi-cance tests were utilized to determine the impact of course assess-ment activities on student attitudes about the use of alternativeassessments.

Participants

Undergraduate students (N = 67) enrolled in two sections of aliteracy course at a large southwestern university volunteered forparticipation in the main study. These classes utilized group, peer,and/or self-evaluation techniques. Participants were assessed toascertain their feelings, concerns, and attitudes regarding use of

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014


these evaluation methods. Participants were asked to indicate theextent to which they agreed or disagreed with twenty statements.Students were also encouraged to supply additional comments notcovered by the questionnaire items. Twenty-nine surveys were in-cluded in the final analysis. This low number was due to attritionin the classes and to incomplete surveys returned by some stu-dents.

Results

Quantitative Findings

Analysis of data collected from the pilot study identified threesubscales related to student attitudes and beliefs about the use ofalternative assessments in undergraduate teacher education pro-grams. All three factors had satisfactory internal consistency(Cronbach alpha) reliabilities.

Subscale 1 (Self-Evaluation) consisted of five items and had areliability coefficient of .83. Subscale 2 (Peer Evaluation) had threeitems; the reliability was .75. The last subscale (Instructor Model-ing) included two items and had a reliability of .78 (see Table 1).

For the main study, a multivariate within subjects analysis of vari-ance design was used to determine course alternative assessments.Results (multivariate F [3, 26] = 52.88, p < .001) indicated statisti-cally significant effects. This analysis was followed by univariatetests. To control the Type I error rate, the Bonferroni procedurewas used to set the comparisonwise level of significance at .05/3 =.0166. Results were statistically significant for the Self-Evaluationsubscale: F (1, 28) = 7.85, p < .0166; and for the Peer Evaluationsubscale: F (1, 28) = 15.50, p < .0166. Results for the InstructorModeling subscale were not statistically significant:F (1, 28) = 5.14, p < .0166 (Table 2).

Qualitative Findings

The survey provided opportunities for the students to make addi-tional comments regarding the questions and their attitudes and/or concerns about the alternative assessments they experienced.Their responses were read and categorized. Although not all stu-dents chose to make additional comments, those received provided

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014


additional support to the quantitative findings for each of thesubscales.

Students generally expressed positive comments related to self-evaluation. Several noted the importance of assessing their learn-ing over time; one student suggested that self-evaluation is moreproductive in an environment that is supportive of the individual

TABLE 1. Statement Numbers, Statements, and Component Loadings forSurvey Responses

Statement No. Statement Loading

Factor 1—Self Evaluation

9 Students’ evaluating their own learning .84further develops this empowerment

17 Self-evaluations are appropriate for use .67in graduate reading education courses

8 I believe in student empowerment, and that .66students should be given majorresponsibility for their own learning

4 Students will learn more by being involved .57in self-evaluation

15 Self-evaluations are appropriate for use .53in undergraduate reading educationcourses

Factor 2—Peer Evaluation

14 Group and peer evaluations are appropriate .72for use in undergraduate readingeducation courses

18 I think more of this professor for using .70group and peer evaluations for coursegrading purposes

3 Students in education programs will learn .68more by being involved in group andpeer evaluations

Factor 3—Instructor Modeling

5 Students in education programs need alternative .70models for assessing their current and/orfuture students

6 An effective way of providing these models is .66to actually have an opportunity toexperience their use in one or moreeducation courses

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014


needs of the learner. There were students, however, who expresseddiscomfort with the use of techniques for course grading purposes.One student stated, “I just need good grades and I feel more com-fortable with things that are set in stone.”

Peer evaluations were judged as more difficult for the partici-pants in this study. Respondents generally supported the use ofpeer evaluations but qualified their opinions by suggesting thatinstructors must be careful in the formation of groups and theassignment of partners. A sample student response illustrated thisconcern:

The concerns that I have consists (sic) of students not getting along in theclass. If students are put in groups that have conflict, it makes it difficultfor them to be honest in the work.

A second student response demonstrated how students thoughtpeer evaluations could be handled properly in a classroom.

Students must be taught how to do the evaluations before they assess oneanother. Criteria must be established for evaluating each activity andanother student’s contributions. They must also be taught to put asidetheir biases and grudges when evaluating their students. They need to betaught to put aside their personal problems. Perhaps in addition to self-evaluation and peer evaluations within groups who works (sic) together,there should be groups evaluating other students in other groups in orderto eliminate personal issues between peers working together.

Finally, all comments related to instructor modeling addressedthe importance of seeing and experiencing the actual assessmentsas a precursor to using the assessment tools with children. Forexample, one student stated, “By providing models for us theteacher has taught us what to look for.”

TABLE 2. Means, Standard Deviations, and F Values

Pretest Posttest F Value

Factor 1 M 10.41 9.03 7.85*SD 3.16 2.87

Factor 2 M 6.48 5.41 15.50*SD 1.70 1.80

Factor 3 M 3.38 2.97 5.14SD 1.12 1.15

*significant p < .0166

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014


Discussion

The first research question focused on students’ confidence toevaluate their own work and the work of their peers. The secondresearch question examined the impact of alternative assessmentson student learning. The last research question explored the im-pact of alternative assessments on university practice related toassessment.

The Self-Evaluation subscale (questions l and 2) included issuesand attitudes that participants held regarding the value of usingself-evaluation measures in their coursework. At the beginning ofthe course students were less positive regarding the value of usingthese measures. By the end of the semester, and after having expe-rienced various opportunities to self-evaluate their work, studentsindicated by their responses to the specific questionnaire itemsand by their additional comments that self-evaluation resulted ina sense of empowerment and control over their own learning andthat their learning of course material also increased as a result ofevaluating their work.

Items for the Peer Evaluation subscale (question 1) were relatedto the use of peer evaluation in literacy courses. Students indi-cated in their survey responses that group and peer evaluationswere acceptable for evaluating their work and that these types ofevaluations enhanced the learning in the class. Some individualcomments, however, urged caution and preparation of studentsprior to initiating peer evaluations.

Although the Modeling subscale results (question 3) were notstatistically significant, some students in this study indicated at theend of the study a more positive opinion regarding the need ofalternative assessments in higher education and that having theopportunity to experience these alternative assessments was aneffective way to learn about alternative assessments for elementarystudents.

The statistical and qualitative analyses support existing researchthat promotes the use of alternative assessments in teacher educa-tion courses (Craig & Leavell, 1995; McKinney, 1994; Ford &Ohlhausen, 1991; Stahle & Mitchell, 1993; Yore & Craig, 1964).These students were clear in their support of alternative assess-ments while acknowledging concerns about the possibility of mis-use. There was an indication of an awareness of the impact which

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014


alternative assessments made on their ownership of learning. Therewas clear evidence that the alternative assessments used in thiscourse gave students a sense of control over their own learningand provided them with examples for use in their own classrooms.

Clearly, this study is only the beginning of the search for under-standing of the impact of alternative assessment on student learn-ing. More research is needed to document the effectiveness ofalternative assessments in teacher education courses related to stu-dent learning. A second, and perhaps more important question toanswer is whether or not these preservice teachers will implementalternative assessments in their own classrooms as a result of theirtraining at the university. To make this determination, researcherswill need to observe these new teachers in action as they begintheir careers.

Appendix

1. I believe that students in education programs are able to fairlyand accurately evaluate each other’s work.

1 2 3 4 5strongly agree uncertain disagree strongly

agree disagree

2. I believe that students in education programs are able to fairlyand accurately evaluate their own work.


agree disagree

3. I believe that students in education programs will learn moreby being involved in group and peer evaluations.


agree disagree

4. I believe that students in education programs will learn moreby being involved in self-evaluations.


agree disagree

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014


5. I believe that students in education programs need alterna-tive models for assessing their current and/or future students.


agree disagree

6. I believe that an effective way of providing these models is toactually have an opportunity to experience their use in oneor more education courses.


agree disagree

7. I believe that students’ authentic use of assessment, instruc-tion, and evaluation procedures in a reading assessment coursecan lead to as much, if not more, understanding of the assess-ment and evaluation concepts than if they were taught themin a traditional reading diagnosis course.


agree disagree

8. I believe in student empowerment, and I believe that studentsshould be given the major responsibility for their own learning.


agree disagree

9. I believe that students’ evaluating their own learning furtherdevelops their empowerment.


agree disagree

10. I believe that use of peer, group, and self-evaluations in thiscourse will cause inflated grades.


agree disagree

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014


11. I believe that we have a problem with grade inflation on thiscampus.


agree disagree

12. I believe that the potential problem of inflated grades in thiscourse is significant enough to cause me to disapprove of theuse of peer, group, and self-evaluation.


agree disagree

13. I believe that students can be cheated of the professor’s ex-pertise if the professor does not do all the evaluations herself.


agree disagree

14. I believe that group and peer evaluations are appropriate foruse in undergraduate reading education courses.


agree disagree

15. I believe that self-evaluations are appropriate for use in un-dergraduate reading education courses.


agree disagree

16. I believe that group and peer evaluations are appropriate foruse in graduate reading education courses.


agree disagree

17. I believe that self-evaluations are appropriate for use in gradu-ate reading education courses.


agree disagree

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014


18. I think more of this professor for using group and peer evalu-ations for course grading purposes.


agree disagree

19. I think more of this professor for using self-evaluations forcourse grading evaluations.


agree disagree

20. Overall, I understand the rationale behind the professor’sdecision to use various peer, group, and self-evaluations forcourse grading purposes, and I approve of the idea.


agree disagree

21. I have the following concerns_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

References

Allen, D. D. (1996). Involving graduate students in personal literacy evaluationthrough the use of portfolios. In M. D. Collins & B. G. Moss (Eds.), Literacyassessment for today’s schools (pp. 75–81). Harrisonburg, VA: College ReadingAssociation.

Cirincione, K. M., & Michael, D. (1994). Literacy portfolios in third grade: Aschool–college collaboration. Reading Horizons, 34, 443–464.

Commeyras, M., Reinking, D., Heubach, K. M., & Pagnucco, J. (1993). Lookingwithin: A study of an undergraduate reading methods course. In D. J. Leu &C. K. Kinzer (Eds.), Examining central issues in literacy research, theory, and prac-tice. Forty-Second Yearbook of the National Reading Conference (pp. 297–304). Chicago: National Reading Conference.

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014


Craig, M. T., & Leavell, A. G. (1995). Preservice teachers’ perceptions of portfo-lio assessments in reading/language arts coursework. In W. M. Linek & E. G.Sturtevant (Eds.), Generations of literacy. Yearbook of the College Reading As-sociation (pp. 83–95). Pittsburgh, KS: College Reading Association.

Crow, N. (1987). Preservice teacher’s biography: A case study. Paper presented at theAnnual Meeting of the American Educational Research Association, Wash-ington, DC.

Ford, M. P., & Olhausen, M. M. (1991). Portfolio assessment in teacher educationcourses: Impact on students’ beliefs attitudes and habits. Paper presented at theannual meeting of the National Reading Conference, Palm Springs, CA.

Frazier, D. W., Palmer, P. S. Duchein, M. A., & Armato, C. (1993). Preserviceelementary teachers’ evolving perceptions of portfolio assessment. In D. J.Leu & C. K. Kinzer (Eds.), Examining central issues in literacy research, theory andpractice. Forty-second Yearbook of the National Reading Conference (pp. 305–314). Chicago: National Reading Conference.

McKinney, M. O. (1994). Institutional concerns in implementing portfolios in teachereducation. Paper presented at the Annual Meeting of the National ReadingConference, San Diego, CA.

Stahle, D. L., & Mitchell, J. (1993). Portfolio assessment in college methodscourses: Practicing what we preach. Journal of Reading, 36(7), 538–542.

Stokes, S. M. (1994). Teacher education methods courses: Modeling practice,but not perfection. The Reading Professor, 17, 43–50.

Valencia, S. (1990). A portfolio approach to classroom reading assessment: Thewhys, whats, and hows. The Reading Teacher, 43, 338–340.

Wagner, C. L., Brock, D. R., & Agnew, A. T. (1994). Developing literacy portfo-lios in teacher education courses. Journal of Reading, 37, 668–674.

Yore, L. D., & Craig, M. T. (1994). Alternative assessment modeled in preservice el-ementary school methods courses: Science and language arts. Paper presented atthe Annual Meeting of the National Science Teachers’ Association, Anaheim,CA.

Dow

nloa

ded

by [

Staf

ford

shir

e U

nive

rsity

] at

21:

26 0

6 O

ctob

er 2

014

alternative assessment in the preparation of literacy educators: responses from students

Documents