Diana Bajzek, Associate Director

Office of Technology for EducationCarnegie Mellon University

Gordon RuleProfessor

Department of Biological Sciences

Additional authors: Judy Brooks, William Jerome, Marsha Lovett, John Rinderle, Candace Thille

Assessment and LearningTwo Sides of the Same Coin.

Outline

• Content Delivery & Analysis of student responses.– OLI – Online Learning Initiative– DDL – Digital Dashboard for learning

• Simulator/Animation environment.– Development of authentic simulator environment– Embedding in OLI

• Coupling of simulator environment to assessment.– Detached– Proximal– Associated– Tightly integrated

• Reporting of student data by OLI/DDL.

Carnegie Mellon University

Office of Technology for Learning

• Digital Dashboard for Learning (DDL): Unique feedback

tool for instructors and students.

OLI: Research project creating exemplary online courses using the best information the learning sciences have to offer.

• Working in courses which are part of Open Learning Initiative -OLI

• Real courses, some fully online, some blended - all publicly available.

• These courses are designed by teams of learning sciences researchers, technologists, and domain experts.

• OLI courses provide rich interactive learning environments with comprehensive activity logging tools.

The Course Design Triangle

Tasks that provide feedback on students’ knowledge and skills

Contexts and activities that foster students’ active engagement in learning

Descriptions of what students should be able to do at the end of the course

The Course Design Triangle

Summative Assessments

The Course Design Triangle

The Course Design Triangle

The Course Design Triangle

Formative Assessments

• Identify difficult concepts

• Detect common misconceptions

• Reveal murkiest points

• Assess the depth of understanding of topics

Feedback Loops in Instruction/Assessment

Feedback Loops in Learning

Instructors and Students Would Like To:

Assessments

Instructional Improvement

Active Learning

Modification of Instruction by incorporating feedback.

Modification of Instruction by incorporating feedback.

Design of Animations/Simulations

1.Select concept.

2.Static image.

3.Movie clip.

4.Simulation

5.Student Interaction

Glucose Transport into Liver

Glucose

Cell Membrane

Hexose Kinase(G → G-P)

Glucose Transporter

Outside

Inside

Design of Animations/Simulations

Glucose Transport into Liver

Outside

Inside

Cast are given xi, vi.Follow Newton’s laws & instructor defined rules.

Simulator Environment:

Glucose: Can’t cross the membrane, unless it encounters its transporter.

Glucose Transporter: Only moves glucose across the membrane

Hexokinase: Enzyme that only binds glucose. Bound glucose converted to G-P

Membrane: Nothing goes through, in this example.

Hexose_kinase.html

Design of Animations/Simulations

Enzyme Kinetics

1. SimulatorStage

2. Student Input

3. Data Output

1.Select concept.

2.Static image.

3.Movie clip.

4.Simulation

5.Student Interaction

Providing Context for Simulations:

Disconnected Assessment:

Proximal Assessment

Associated Assessments

Coupled Assessments• Context given within

assessment.

• Student prompted to interact with the simulator to gain information to answer question.

• Student input passed to simulator, changing its state.

• Simulator can be used to drive data output.

• Question posed:• Hints available

• Individual feedback for answers.

• Student response & assessment activity can control simulator environment.

• Control of flow is possible as the student progresses through exercise.

Inhibitors_simulator.html

Coupled Assessments• Context given within

assessment.

• Student prompted to interact with the simulator to gain information to answer question.

• Student input passed to simulator, changing its state.

• Simulator can be used to drive data output.

• Question posed:• Hints available

• Individual feedback for answers.

• Student response & assessment activity can control simulator environment.

• Control of flow is possible as the student progresses through exercise.

tca_simulator.html

Assessment System• Familiar question types

• Sophisticated evaluation (scoring)

• Flexibly evaluated according to criteria specified by the instructor

• Multi-part questions

Feedback Reportfor the assessment asseen by an instructor

• Class Participation

• Best and Worst Questions

• Individual Question

Feedback Reportfor the assessment asseen by an instructor

• Class Participation

• Best and Worst Questions

• Individual Question

Feedback Reportfor the assessment asseen by an instructor

• Class Participation

• Best and Worst Questions

• Individual Question

• OLI project:

• http://www.cmu.edu/oli/

• Biology project:

• http://telstar.ote.cmu.edu/biology/

• Funding:

– The William and Flora Hewlett Foundation

– Howard Hughes Medical Institute