assessing the online learner · 2020. 4. 29. · such as simulations, role playing, or use of case...

ASSESSING THE ONLINE LEARNER

Rena Palloff and Keith Pratt

Be inspired. Be inspiring.

Welcome

Rena Palloff and Keith Pratt


Learning Outcomes for This Session

After reviewing this pre-recorded session, you should be prepared to:

• Align course design with assessment strategies.

• Describe characteristics of online assessment.

• Select appropriate assessment methods to determine student competency levels.

• Develop an online assessment activity and a rubric to evaluate student performance.

3


Start with Your Learning Outcomes

• What a student will be able to know or do at the end of a learning unit or course

• Measurable

• Observable action or behavior

4


Then Move to Learning Objectives

• Unit or activity based—contribute to the achievement of outcomes.

• Describe cognitive skills achieved (i.e., what the student will LEARN.

• Once you know what you want your students to be able to know, do, or learn, you can begin to design activities and assessments.

5


Cycle of Course Design

Purpose of Course

(Course Outcomes

& Competencies)

Learner

Characteristics

Learning Objectives Course Activities

Student Assessments


Designing Courses That Align with Assessment • Provide clear guidelines and objectives.

• Tasks and assignments should be relevant not only to the subject matter, but to students’ lives as well.

• Students should understand what is expected of them.

• Assessment should be in alignment with the course as a whole and is not to be seen as a separate and cumbersome task.

When the activities and assessment techniques are in alignment, instructors and students alike are more satisfied with the outcome of the learning process.

7


Assessment Basics

The following are some principles that should guide student assessment in an online course:

• Design learner-centered assessments that include self-reflection.

• Design and include grading rubrics for the assessment of contributions to the discussion as well as for assignments, projects, and the collaboration itself.

• Include collaborative assessments through public posting of papers along with comments from student to student.

8


Assessment Basics

The following are some principles that should guide student assessment in an online course:

• Encourage students to develop skills in providing feedback by providing guidelines to good feedback and by modeling what is expected.

• Use assessment techniques that fit the context and align with learning objectives.

• Design assessments that are clear, easy to understand, and that are likely to work in the online environment.

• Ask for and incorporate student input into how assessment should be conducted.

9


Assessment Basics

Angelo and Cross (1993) note that effective classroom assessment should be:

• Learner-centered

• Teacher-directed

• Mutually beneficial

• Formative

• Context-specific

• Ongoing

• Firmly rooted in good practice

10


Online

• Involvement in online discussion can improve reading/writing skills.

• Engagement in discussion can deepen knowledge acquisition and critical thinking skills.

• Technology can be used to assist in the assessment process.

Classroom

• Can improve reading/writing skills if assessment involves written assignments beyond tests/quizzes.

• Active learning techniques can deepen knowledge acquisition.

• Technology is not often used for assessment.

Differences Between Online and Classroom Assessment


Assessment in Online Courses

Six key qualities for assessment of online students (Morgan and O’Reilly, 1999):

• A clear rationale and consistent pedagogical approach

• Explicit values, aims, criteria, and standards

• Authentic and holistic tasks

12


Assessment in Online Courses

Six key qualities for assessment of online students (Morgan and O’Reilly, 1999):

• A facilitative degree of structure

• Sufficient and timely formative assessment

• Awareness of the learning context and perceptions

13


Ways to Measure Knowledge Acquisition • Use good questioning techniques to review text and

other reading material

• Rubrics

• Self-assessments

• Peer assessments

• Authentic assessments

14


Tests and Quizzes Online

Generally used as the first resource for online instructors because:

• Course management systems contain test and quiz functions.

• Instructors may have limited time or resources available to construct other forms of assessment (Rocco, 2007).

15


Why Tests and Quizzes May Not Be the Best Choice • May not align with course objectives or mode of

instruction.

• The need to write good questions—not all instructors do this well!

• Generally measure knowledge acquisition but not application.

• May actually encourage cheating.

16


Alignment of Objectives with Exam Questions Rocco (2007) provides the following alignment of test question types with desired course outcomes:

Objective

Level

Multiple Choice

True/False Matching Completion Short Answer

Short Essay

Long Essay

Knowledge X X X X X

Comprehension X X

Application X X

Analysis X X

Synthesis X X

Evaluation X X

17


Use Quizzes as a Form of Self-Assessment • Practice tests and quizzes align well with the use of

final exams online.

• Helps students understand what will be expected of them in terms of test questions and how to use the technology for the test.

• Practice tests and quizzes improve student performance in the course (Michigan State University, 2005).

18


Questioning Techniques

• Use open-ended questions that encourage students to talk to one another about what they’ve read.

• Use questions that encourage further research outside of assigned reading.

• Use questions that encourage application of material read, such as case studies, scenarios, etc.

• As the instructor, do not respond to every post. Strategically jump in to link or extend learner discussion.

• Do assess learner contributions to discussions!

19


Rubrics

• Should support self-assessment, self-monitoring, and instructor assessment

• Should be based on Bloom’s taxonomy, and criteria for each level of response assessed should be clear

20


Rubrics

• Define the performance levels for each gradable activity element.

• Provide students with a concrete way of evaluating their own performance as well as the performance of the members of their team in collaborative activity.

21


Types of Rubrics

• Holistic Rubrics—Measure overall course performance

• Analytic Rubrics—Break performance or product into its component parts

• Primary Trait Rubrics—Similar to analytic rubrics with the exception that they measure a common trait across all categories (i.e., participation in the online course)

22


Basic Discussion Rubric

• Summary Level Posts = C

• Analytic Level Posts = B

• Synthesis Level Posts (i.e., “You made me think!”) = A

B

l

o

o

m

s

T

a

x

o

n

o

m

y

23


Using Rubrics in Assessment

The following is an example of how to use a rubric for online discussion: Attached is a rubric for participation in online discussions. The way to use this rubric is to look at the “gradable” items listed down the left-hand side of the grid. Across the top are the performance levels, which roughly (and I do mean roughly!) equate to grades. To earn an A, for example, you would not need to fall in all of the boxes down the right-hand column—your performance in discussions might be extremely strong with a combination of items from the TWO columns on the right (proficient and distinguished) and maybe even one item that you feel shows basic performance. The idea is to look at the total picture of your discussion performance.

24


Using Rubrics in Assessment, cont.

So, at mid-term and again at the end of the term, what I want you to do is to send me an e-mail (I'll nudge you at mid-term) telling me where you think you fall on the rubric. For example, you might say that for including and applying course concepts and theories, you feel that you fall into the proficient category while you feel you are distinguished in applying real-world examples and applications of the theories. You'll go through each category and tell me where you think you fall on the grid. I don't generally give mid-term grades, so what I'll be doing is responding to your e-mail with my own assessment based on your self-assessment. I hope that's not too confusing! If there's a discrepancy in our perceptions, we'll discuss it. If we agree, then great! At the end of the term, I'm going to ask you to do this again. But, this time, I'm going to ask you to tell me what grade you think you've earned as well. Questions about this? Please ask!

25


Determining Performance Levels for Activities

• What evidence can students provide that would show they have accomplished what you want them to?

• What are the highest expectations for student performance?

• What is the worst performance you can imagine beyond not doing the assignment at all?

26


Determining Gradable Activities

• Activities should directly relate to course learning objectives.

• All course activities should relate to one another.

• Activities should be based on previous experience with like assignments.

• Activities should relate to the skill set of students entering the course or should help develop new skills.

27


Self-Assessments

• Depending on the level of the learners, criteria can be instructor-developed, peer-developed, or independently developed.

• Work best with some guiding questions.

• Can be used effectively for individual activities as well as overall performance in the course.

28


Peer Assessments

• After collaborative activity, can be used to debrief

• Can be in the form of a private e-mail to the instructor including both self-assessment and assessment of all other members of the group and team

• Can include suggested grades or simply be in narrative form

29


Authentic Assessments

• Generally take the form of application activities such as simulations, role playing, or use of case studies.

• Portfolios can illustrate ongoing knowledge acquisition and application.

• Effectively demonstrate not only acquisition of knowledge but ability to apply that knowledge in professional or other settings.

• Effectively assessed through the use of rubrics.

30


Using the Online Environment for Assessment Advantage

• Computer generated/scored tests and quizzes

• Internet-based research projects

• Peer review and assessment technologies

• Internet-based case studies

• Synchronous technologies to facilitate collaboration

31


Map Your Outcomes to Assessment Activities!

Learning

outcomes (include: knowledge, cognitive,

key and subject specific skills)—select core learning outcomes that

appear across units—it is not

necessary to list all learning

outcomes.

Learning-

teaching activity You will need this for the

learning-teaching-assessment

cycle part—just give idea of

different teaching and learning

used to accommodate a

learning outcome.

Type of

assessment E.g., essay, group work,

presentation, work-based,

extended project, etc.


NEXT – Develop an assessment activity and a rubric you can use to evaluate it!

[email protected]

www.WileyLearningInstitute.com

Thank you!

LIVE SESSION: TUESDAY, JUNE 26 12:00 – 2:30PM EST

assessing the online learner · 2020. 4. 29. · such as simulations, role playing, or use of case...

Documents