Research on Undergraduate Learning in STEM Disciplines

Karl A. SmithCivil Engineering

University of Minnesotaksmith@umn.edu

www.ce.umn.edu/~smith

National Research CouncilNational Science Resources Center

Math/Science Partnerships WorkshopDecember 5-7, 2004

BackdropNational Research Council Reports:1. How People Learn: Brain, Mind, Experience,

and School (1999).2. How People Learn: Bridging Research and

Practice (2000).3. Knowing What Students Know: The Science

and Design of Educational Assessment (2001).

4. The Knowledge Economy and Postsecondary Education (2002). Chapter 6 – Creating High-Quality Learning Environments: Guidelines from Research on How People Learn

Session Highlights

• Provide overview of some findings from reports related to teaching & learning.

• Do a activities with you to illustrate some of the points covered in the reports.

• Discuss implications for designing learning environments that are learner centered, knowledge centered, assessment centered, and community centered.

Designing Learning Environments Based on HPL

Learner-Centered Learning Environments

• Learners use their current knowledge to construct new knowledge. Effective instruction must take into account what learners bring to the classroom. Active engagement in learning supports the construction of knowledge.

Learner-Centered Learning Environments• Learners should be assisted in developing

metacognitive strategies.

"Metacognition refers to people's abilities to predict their performances on various tasks ... and to monitor their current levels of mastery and understanding" (HPL, p. 12)

Transfer can be improved by helping students

become more aware of themselves as learners who actively monitor their learning and performance strategies.

Learner-Centered Learning Environments

• Learners learn more efficiently and effectively when they are provided with feedback to help them monitor progress. Students need to be given opportunities to practice skilled problem solving and provided with both, feedback to monitor progress, and support to ensure progress.

Knowledge-Centered Learning Environments

• Students are not blank slates, so instruction should begin with students' current knowledge and skills.

• Instruction should help students organize

knowledge in ways that are efficient for recall and for application in solving problems.

• Instruction should focus on helping students gain

deep understanding of the major concepts and principles, rather than the acquisition of disconnected facts and skills.

Assessment-Centered Learning Environments

• Formative assessment (assessment done during the course of instruction to monitor students' progress and to help shape instruction) is pivotal for providing feedback to students so that they can revise and improve the quality of their thinking, and should be done continuously as a part of instruction.

• Formative assessment strategies should be

developed that make students' thinking visible to the instructor, to the learner, and to other classmates.

Assessment-Centered Learning Environments

• Summative assessments (assessment done at the end of instruction for such purposes as assigning grades or evaluating competence) should reflect the knowledge, concepts, principles, and problem solving & lab skills of the discipline that are considered crucial by experts.

• Students should learn how to assess their own

work and that of peers.

Community-Centered Learning Environments

• Learners are embedded in social contexts. To make effective use of their “prior knowledge,” they need to relate the origins of their learning to school-based concepts.

• It is important to help students see the relevance of

their school-based learning to non-school contexts and problem solving. (Students' "awake" time: 14% in school, 53% out of school.

• Communities of practice need to be encouraged. How? Internships, class participation, dorm floors

arranged by major, etc.

Summary Points

• There is an emerging science of learning• It has major implications for all aspects of

schooling -- curriculum, instruction, assessment, plus preservice and inservice teacher education

• It provides a basis for knowing when, how and why to use various instructional strategies

• It can guide the intelligent design and use of new curricular materials as well as information technologies

Lila M. Smith

Pedago-pathologies B Lee Shulman

Amnesia

Fantasia

Inertia

Shulman, Lee S. 1999. Taking learning seriously. Change, 31 (4), 11-17.

What do we do about these pathologies? – Lee Shulman Activity Reflection Collaboration PassionCombined with generative content and the creation of powerful learning communities

Shulman, Lee S. 1999. Taking learning seriously. Change, 31 (4), 11-17.

Lila M. Smith

Tracking Change - Seymour

"The greatest single challenge to SMET pedagogical reform remains the problem of whether and how large classes can be infused with more active and interactive learning methods."

Seymour, Elaine. 2001. Tracking the processes of change in US undergraduate education in science, mathematics, engineering, and technology. Science Education, 86, 79-105.

Formulate-Share-Listen-Create (Think-Pair-Share)

• Individually read the quote “To teach is to engage students in learning. . .”

• Underline/Highlight words and/or phrase that stand out for you

• Turn to the person next to you, introduce yourself

• Share words and/or phrases that stood out and discuss

To teach is to engage students in learning; thus teaching consists of getting students involved in the active construction of knowledge. . .The aim of teaching is not only to transmit information, but also to transform students from passive recipients of other people's knowledge into active constructors of their own and others' knowledge. . .Teaching is fundamentally about creating the pedagogical, social, and ethical conditions under which students agree to take charge of their own learning, individually and collectively

Education for judgment: The artistry of discussion leadership. Edited by C. Roland Christensen, David A. Garvin, and Ann Sweet. Cambridge, MA: Harvard Business School, 1991.

Strategies for Energizing Large

Classes: From Small Groups to

Learning Communities:

Jean MacGregor,James Cooper,

Karl Smith,Pamela Robinson

New Directions for Teaching and Learning,

No. 81, 2000.Jossey- Bass

Book Ends on a Class Session

Informal CL (Book Ends on a Class Session) with Concept Tests

Physics Peer InstructionEric Mazur - Harvard B http://galileo.harvard.edu

Peer Instruction – www.prenhall.comRichard Hake – http://www.physics.indiana.edu/~hake/

Chemistry Chemistry ConcepTests - UW Madison B

www.chem.wisc.edu/~conceptVideo: Making Lectures Interactive with ConcepTests

ModularChem Consortium B http://mc2.cchem.berkeley.edu/

STEMTECVideo: How Change Happens: Breaking the ATeach as You Were Taught@ Cycle B Films for the Humanities & Sciences B www.films.com

Thinking Together video: Derek Bok Center B www.fas.harvard.edu/~bok_cen/

Richard Hake (Interactive engagement vs traditional methods) http://www.physics.indiana.edu/~hake/

Traditional (lecture)

Interactive (active/cooperative)

<g> = Concept Inventory Gain/Total

The “Hake” Plot of FCI

Pretest (Percent)

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

20.00 30.00 40.00 50.00 60.00 70.00 80.00

ALS

SDI

WP

PI(HU)

ASU(nc)

ASU(c)

HU

WP*

UMn Traditional

XUMn Cooperative Groups

XUMn-CL+PS

Physics (Mechanics) Concepts:The Force Concept Inventory (FCI)

• A 30 item multiple choice test to probe student's understanding of basic concepts in mechanics.

• The choice of topics is based on careful thought about what the fundamental issues and concepts are in Newtonian dynamics.

• Uses common speech rather than cueing specific physics principles.

• The distractors (wrong answers) are based on students' common inferences.

FCI Question 17

An elevator is being lifted up an elevator shaft at a constant speed by a steel cable, as shown in the figure. All frictional effects are negligible. In this situation, forces on the elevator are such that:

(A) the upward force by the cable is greater than the downward force of gravity.

(B) the upward force by the cable is equal to the downward force of gravity.

(C) the upward force by the cable is smaller thanthe down ward force of gravity.

(D) the upward force by the cable is greater than the sum of the downward force of gravity and a downward force due to the air.

(E) None of the above. (The elevator goes up because the cable is shortened, not because an upward force is exerted on the elevator by the cable).

Pre64

18

2

11

5

Post36

60

0

2

1

Problem Based Cooperative Learning FormatTASK: Solve the problem(s) or Complete the project.

INDIVIDUAL: Estimate answer. Note strategy.

COOPERATIVE: One set of answers from the group, strive for agreement, make sure everyone is able to explain the strategies used to solve each problem.

EXPECTED CRITERIA FOR SUCCESS: Everyone must be able to explain the strategies used to solve each problem.

EVALUATION: Best answer within available resources or constraints.

INDIVIDUAL ACCOUNTABILITY: One member from your group may be randomly chosen to explain (a) the answer and (b) how to solve each problem.

EXPECTED BEHAVIORS: Active participating, checking, encouraging, and elaborating by all members.

INTERGROUP COOPERATION: Whenever it is helpful, check procedures, answers, and strategies with another group.

Technical Estimation ExerciseTASK:

INDIVIDUAL: Quick Estimate (10 seconds). Note strategy.

COOPERATIVE: Improved Estimate (~5 minutes). One set of answers from the group, strive for agreement, make sure everyone is able to explain the strategies used to arrive at the improved estimate.

EXPECTED CRITERIA FOR SUCCESS: Everyone must be able to explain the strategies used to arrive at your improved estimate.

EVALUATION: Best answer within available resources or constraints.

INDIVIDUAL ACCOUNTABILITY: One member from your group may be randomly chosen to explain (a) your estimate and (b) how you arrived at it.

EXPECTED BEHAVIORS: Active participating, checking, encouraging, and elaborating by all members.

INTERGROUP COOPERATION: Whenever it is helpful, check procedures, answers, and strategies with another group.

Model 1 (lower bound)

let L be the length of the room,let W be its width,let H be its height, and let D be the diameter of a ping pong ball.

Then the volume of the room is Vroom = L * W * H,

and the volume of a ball (treating it as a cube) is Vball = D3,

so number of balls = (Vroom) / (Vball) = (L * W * H) / (D3).

Model 2 (upper bound)

let L be the length of the room,let W be its width,let H be its height, and let D be the diameter of a ping pong ball.

Then the volume of the room is Vroom = L * W * H,

and the volume of a ball (treating it as a sphere) is Vball = 4/3 r3,

so number of balls = (Vroom) / (Vball) = (L * W * H) / (4/3 r3).

Model 1 (Vroom / D3ball) B Lower Bound

Model 2 (Vroom / (4/3 r3ball)) B Upper Bound

Upper Bound/Lower Bound = 6/ 2

How does this ratio compare with1.The estimation of the diameter of the ball?2.The estimation of the dimensions of the room?

Real World

Model World

Model

Vr/Vb

Calc

Problem-Based Learning

Problem posed

Identify what weneed to know

Learn it

Apply it

START

Subject-Based Learning

Told what weneed to know

Learn it

Given problem toillustrate how to use it

START

Normative Professional Curriculum:

1. Teach the relevant basic science,

2. Teach the relevant applied science, and

3. Allow for a practicum to connect the science to actual practice.

Problem-Based Learning (PBL)

Problem-based learning is the learning that results from the process of working toward the understanding or resolution of a problem. The problem is encountered first in the learning process B Barrows and Tamlyn, 1980

Core Features of PBL

•Learning is student-centered•Learning occurs in small student groups•Teachers are facilitators or guides•Problems are the organizing focus and stimulus for learning•Problems are the vehicle for the development of clinical problem-solving skills•New information is acquired through self-directed learning

Group ProcessingB Plus/Delta Format B

PlusThings That Group Did Well

DeltaThings Group Could Improve

Cooperative Learning is instruction that involves people working in teams to accomplish a common goal, under conditions that involve both positive interdependence (all members must cooperate to complete the task) and individual and group accountability (each member is accountable for the complete final outcome).

Key Concepts

Positive InterdependenceIndividual and Group AccountabilityFace-to-Face Promotive InteractionTeamwork SkillsGroup Processing

Cooperative Learning Research Support Johnson, D.W., Johnson, R.T., & Smith, K.A. 1998. Cooperative learning

returns to college: What evidence is there that it works? Change, 30 (4), 26-35.

• Over 300 Experimental Studies• First study conducted in 1924• High Generalizability• Multiple Outcomes

Outcomes

1. Achievement and retention2. Critical thinking and higher-level

reasoning3. Differentiated views of others4. Accurate understanding of others'

perspectives5. Liking for classmates and teacher6. Liking for subject areas7. Teamwork skills

Small-Group Learning: Meta-analysis

Springer, L., Stanne, M. E., & Donovan, S. 1999. Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-

analysis. Review of Educational Research, 69(1), 21-52.

Small-group (predominantly cooperative) learning in postsecondary science, mathematics, engineering, and technology (SMET). 383 reports from 1980 or later, 39 of which met the rigorous inclusion criteria for meta-analysis.

The main effect of small-group learning on achievement, persistence, and attitudes among undergraduates in SMET was significant and positive. Mean effect sizes for achievement, persistence, and attitudes were 0.51, 0.46, and 0.55, respectively.

Creating High-Quality Learning Environments:

Guidelines from Research on How People Learn

Understanding by Design Wiggins & McTighe

Backward Design

Stage 1.Identify Desired Results

Stage 2.Determine Acceptable Evidence

Stage 3.Plan Learning Experiences and Instruction

Wiggins, G. & McTighe, J. 1998. Understanding by design. ASCD.

Backward Design

Stage 1. Identify Desired Results

Filter 1. To what extent does the idea, topic, or process represent a big idea or having enduring value beyond the classroom?

Filter 2. To what extent does the idea, topic, or process reside at the heart of the discipline?

Filter 3. To what extent does the idea, topic, or process require uncoverage?

Filter 4. To what extent does the idea, topic, or process offer potential for engaging

students?

Backward Design

Stage 2. Determine Acceptable Evidence

Types of Assessment

Quiz and Test Items: Simple, content-focused test items

Academic Prompts: Open-ended questions or problems that require the student to think critically

Performance Tasks or Projects: Complex challenges that mirror the issues or problems faced by graduates, they are authentic

Backward Design

Stage 3. Plan Learning Experiences & Instruction• What enabling knowledge (facts, concepts, and

principles) and skills (procedures) will students need to perform effectively and achieve desired results?

• What activities will equip students with the needed knowledge and skills?

• What will need to be taught and coached, and how should it be taught, in light of performance goals?

• What materials and resources are best suited to accomplish these goals?

• Is the overall design coherent and effective?

It could well be that faculty members of the twenty-first century college or university will find it necessary to set aside their roles as teachers and instead become designers of learning experiences, processes, and environments James Duderstadt, 1999

We never educate directly, but indirectly by means of the environment. Whether we permit chance environments to do the work, or whether we design environments for the purpose makes a great difference.John Dewey, 1906

CAEE Vision for Engineering Education

 Center for the Advancement of Engineering EducationCindy Atman, Director

CAEE TeamUniversity of WashingtonColorado School of MinesHoward UniversityStanford UniversityUniversity of Minnesota

CAEE Affiliate OrganizationsCity College of New York (CCNY), Edmonds Community College, Highline Community College (HCC), National Action Council for Minorities in Engineering (NACME), North Carolina A&T (NCA&T), San Jose State University (SJSU), University of Texas, El Paso (UTEP), Women in Engineering Programs & Advocates Network (WEPAN) and Xavier University

CAEE - Elements for Success

• Scholarship on Learning Engineering Learn about the engineering student experience

• Scholarship on Engineering Teaching Help faculty improve student learning

• Scholarship on Engineering Education Institutes Cultivate future leaders in engineering education

Theory

Research Practice

CAEE Approach

Research that makes a

difference . . . in theory and practice

Center for the Integration ofCenter for the Integration ofResearch, Teaching, and LearningResearch, Teaching, and Learning

(CIRTL)(CIRTL)

NSF Center for Learning and TeachingNSF Center for Learning and Teaching

University of Wisconsin - MadisonUniversity of Wisconsin - MadisonMichigan State UniversityMichigan State University

Pennsylvania State UniversityPennsylvania State University

…develop a national STEM faculty ...

Research Universities

100 RUs => 80% Ph.D’s100 RUs => 80% Ph.D’s

FACULTY

Community CollegeLiberal Arts

HBCUMasters University

Comprehensive Univ.Research University

UNDERGRADS

Community CollegeLiberal Arts

HBCUMasters University

Comprehensive Univ.Research University

Teaching-as-ResearchTeaching-as-Research

• Engagement in teaching as engagement in STEM researchEngagement in teaching as engagement in STEM research

• Hypothesize, experiment, observe, analyze, improveHypothesize, experiment, observe, analyze, improve

• Aligns with skills and inclinations of graduates-Aligns with skills and inclinations of graduates- through-faculty, and fosters engagement in through-faculty, and fosters engagement in teaching reformteaching reform

• Leads to self-sustained improvement of STEM educationLeads to self-sustained improvement of STEM education

““The nation must develop STEM faculties who themselves The nation must develop STEM faculties who themselves continuously inquire into their students’ learning.”continuously inquire into their students’ learning.”

NATIONAL ACADEMY OF ENGINEERINGOF THE NATIONAL ACADEMIES

Center for the Advancement of Scholarship on Engineering Education

56

A Work-in-Progress: A Work-in-Progress: NAE Center for the NAE Center for the

AdvancementAdvancement of Scholarship of Scholarship on on

Engineering EducationEngineering Education

Norman L. Fortenberry, Sc.D.

Director, CASEE

http://www.nae.edu/CASEE

nfortenb@nae.edu

(202) 334-1926

November 8, 2003

NATIONAL ACADEMY OF ENGINEERINGOF THE NATIONAL ACADEMIES

Center for the Advancement of Scholarship on Engineering Education

57

CASEE MissionCASEE Mission

Enable engineering education to meet, in a significantly better way, the needs of employers, educators, students, and society at large.

Working collaboratively with key stakeholders, CASEE

Encourages rigorous research on all elements of the engineering education system, and

Seeks broad dissemination, adoption, and use of research findings.

CASEE ObjectivesCASEE Objectives

NATIONAL ACADEMY OF ENGINEERINGOF THE NATIONAL ACADEMIES

Center for the Advancement of Scholarship on Engineering Education

58

Research Thrust AreasResearch Thrust Areas1. Define the bodies-of-knowledge required for

engineering practice and use of engineering study for other careers.

2. Develop strategies that value diversity in the formulation and solution of engineering problems.

3. Develop cost-effective and time-efficient strategies and technologies for

• Improving student learning, and • Enhancing the instructional effectiveness of current and

future faculty.

4. Develop assessments of student learning and instructional effectiveness.

Conducting Rigorous Research in Engineering Education: Creating a

Community of Practice

NSF-CCLI-NDAmerican Society for Engineering Education

Karl Smith & Ruth StrevelerUniversity of Minnesota & Colorado School of Mines

Rigorous Research Workshop Initial Event for year-long project Presenters and evaluators representing

– American Society for Engineering Education (ASEE)

– American Educational Research Association (AERA)

– Professional and Organizational Development Network in Higher Education (POD)

Faculty funded by two NSF projects:– Conducting Rigorous Research in Engineering Education (NSF DUE-0341127)– Strengthening HBCU Engineering Education Research Capacity (NSF HRDF-

041194)• Council of HBCU Engineering Deans• Center for the Advancement of Scholarship in Engineering Education

(CASEE)• National Academy of Engineering (NAE)