engineering education conference - spring 2009 increasing assessment effectiveness in a time of...

EngineeringEducation

Conference - Spring 2009

Increasing Assessment Increasing Assessment Effectiveness in a Time of Effectiveness in a Time of

Decreasing BudgetsDecreasing Budgets

Ken Stanton


PhD in Engineering PhD in Engineering Education at Virginia TechEducation at Virginia Tech

First cohort began Fall 2008

16 total students Coursework in

engineering, education, and engineering education

Teaching and research assistantships available

Graduate Recruitment Days every March and October– travel grants available!

www.enge.vt.edu


Introduction – The Introduction – The ProblemProblem

ABET EC2000 standards are a lot of work for engineering departmentsPurposes: assessment and accountabilityProblems: some faculty less familiar with

assessment, requiring more and more of programsSolutions: faculty development, assessment

experts

Economic downturn affects education budgetsProblem: difficult to implement the solutions above

Enrollment, and therefore class size, increasesTime and resources limited

Any solutions: must be efficient, use what we have, be simple and easy to implement


Introduction – Proposed Introduction – Proposed SolutionSolution

Address the need to increase assessment effectiveness from three angles1. Utilize lean assessment techniques2. Implement a process that maximizes

intrinsic motivation3. Take advantage of efficiency of assessment

methodology


Innovative Solution – Innovative Solution – Lean AssessmentLean Assessment

Goal of assessment

Teaching toward learning

objectives

Assignment created for

student

Student completes

assignment

Grade and feedback given to

student

Typical process for an assignment



Teaching toward learning

objectives

Assessment created for

student

Student completes

assessment

Feedback given to student, may

be graded

Collect and analyze evidence of learning

Use to improve teaching and assessments

Typical assessment process



Definitionsimple, efficient, and effective assessment

techniquesbased in the principle of lean manufacturing

productive and simple methodsstill attain useful data

MethodsSome existing

Angelo and Cross (1993)

Some modifiedSome novel



Technique: Time Requirement:

Purpose: Description: Notes:

Rubrics (for various activities)

MED;LOW if shared development

Categorize important assessment and grading criteria; set up structure to summarize performance; provide clear expectations to students

Important criteria are categorized and assigned point values in a table, then performance levels (e.g. novice, expert) described

Many resources online for pre-defined or custom rubrics; coordinating with other faculty can decrease time investment and increase consistency of course/material

One-minute problem summary

LOW Evaluate student understanding of assigned problems and underlying concepts

At the end of a problem, the student takes one minute to explain what major concepts were used and what was learned

Good to prompt students with leading question to answer in problem summary

Use to assess teaching and learning



Technique: Time Requirement:

Purpose: Description: Notes:

Individual project reports (in team project)

MED Assess and improve individual report-writing abilities

When team reports are required, have each student submit individual report at least one week preceding

Reviewing 3-5 times as many reports can be time consuming; a rubric should be used and specific aspects of reports focused on

Oral exams LOW and can reduce time

Assess oral abilities and allow for deeper probing of key topics; mimic business meeting

Students present and discuss project instead of written exam/project, one-on-one with instructor; oral questions provide richer evidence of knowledge and learning; grade is given before leaving meeting

A form of authentic assessment, abilities are tested in a more realistic manner that closely reflects real-world; time can be less than administering and grading exams/ projects


Innovative Solution – Innovative Solution – ImplementationImplementation

These assessment methods alone do not solve the problem

Consider motivation theory in implementation

Self-Determination Theory (SDT) by Deci & Ryan (2000) has 3 needs 1. Competence – confidence, self-efficacy2. Autonomy – choice that aligns with identity /

values3. Relatedness – connection to others identify with Satisfaction of these leads to internalization of

motivation – move from extrinsic to intrinsic



CompetenceLacking due to inexperience with assessmentLean assessment techniques – simple, effectiveFaculty helping each other

AutonomyChallenged by requirements being handed downHelped somewhat by flexibility of ABET criteriaUse assessment to address faculty’s teaching

concerns

RelatednessCan take opportunity to connect with students

through feedback



1. Review program educational objectives (PEOs), program learning outcomes, and course outcomes established for the program and course(s) of interest.

2. Interview faculty for input on assessment and what they would like to improve in their teaching and students’ performance. Review current assignments and assessments being used in their course(s) of interest.

3. Identify lean assessment techniques thatA. are good for measuring the outcomes from #1 B. meet goals that faculty name as important to themC. can complement/replace existing assignments and

activities in the course(s).



4. Briefly illustrate how the lean assessment techniques match the faculty member’s goals and can be used alongside or in place of existing course materials.

5. Assist with familiarization and use of the assessments, if necessary, and encourage sharing the expectations and motives with students.

Likely to be effective seeing as studies found faculty generally hold positive views of assessment


Innovative Solution – Innovative Solution – Assessment Assessment Methodology Methodology

Many times we develop courses by deciding material then assessmentsLess efficient and less productive

Wiggins & McTighe (2005) coined “Backward Design”Work from learning objectives to ways to

measure them (assessments) to how to teach itEnsures efforts are goal-oriented and

assessments can be used for accreditation

Using this process can increase efficiency of teaching and assessment efforts


Introduction – Proposed Introduction – Proposed SolutionSolution

Address the need to increase assessment effectiveness from three angles1. Utilize lean assessment techniques2. Implement a process that maximizes

intrinsic motivation**3. Take advantage of efficiency of assessment

methodology


Future workFuture work

Dissertation studyFrom a motivational standpoint, how do

engineering faculty members engage in assessment of student learning?

Also look at ways to integrate assessment into the culture of teaching and learning


THANK YOUTHANK YOU

QUESTIONS?QUESTIONS?

engineering education conference - spring 2009 increasing assessment effectiveness in a time of...

Documents