1

Educational Change in Academic Institutions:

Learning the Lessons of Change at MIT

Daniel HastingsSeptember 2011

2

Agenda

• Why some educational innovation has occurred– Importance of enablers– Value of data– Role of the innovator

• Why some educational innovation has not occurred– Idea of perspectives– Strategic design of curricular change– Political issues– Cultural issues

• Aftermath at MIT

3

Since 2000, MIT has developed almost 80 educational innovations

Subjects

1.00,* 6.001*, 6.002x,* 8.02(TEAL),* 8.224, HST582J, 18.03, Mission 200X*, Terrascope (1.016), 2.001-2.006 & 18.02 Small Groups,* PE for ME (2.993/2.THU), 6.002X, Robotics: Science and Systems (6.188),* HST 100/527J

Miscellaneous 7.01 Concept Lab, Exploratory

Subjects, Interphase, Pass/No Record First Year, Summer Institute in Materials

Technologies

PIVoT, Cross Media Annotation System (XMAS),* MetaMedia,* PRS, iLabs,* Mathlets, Use of PDAs, XMAS

Programs The Undergraduate Exchange with

University of Cambridge,* Residence-Based Advising*

SpaceThe d’Arbeloff/TEAL classroom (26-152)

*Denotes multi-year or multi-semester assessment. Source: TLL

4

Enablers

• Money– $25 million from

Microsoft for iCampus– $11 million from

donors, Brit and Alex d’Arbeloff

• Culture that supports faculty innovation

• At least a few faculty innovators

• Dedicated organizations– Teaching & Learning Lab

• Provides data-driven assessment and expertise in STEM pedagogy

– Office of Educational Innovation & Technology

• Develops educational technology for faculty

– Office of Faculty Support• Provides support for

curricular changes

5 Unlocking Knowledge, Empowering Minds

MIT OpenCourseWare IS NOT:

MIT OpenCourseWare IS:

Unlocking Knowledge

What is OCW?

• An MIT education

• Intended to represent the

interactive classroom environment

• Degree-granting

• A Web-based publication of

virtually all MIT course content

• Open and available to world

• A permanent MIT activity

6 Unlocking Knowledge, Empowering Minds

Unlocking Knowledge

2,075 Courses• 2,075 Syllabi & reading lists

• 17,531 lecture notes

• 9,460 assignments

• 980 exams

• 705 projects

Many include:

• Audio/video (~100)

• Complete texts (~30)

• Simulations/animationshttp://ocw.mit.edu

7

Why it works – faculty view• Most popular course is 6.00 Introduction to Computer Science• Why it works

– 1) MIT likes to do leadership things– 2) The faculty came to believe that it was not giving away the

essence of an MIT education– 3) The OCW infrastructure meant that there is no additional

work required of the faculty in giving a course to OCW– 4) The faculty don’t have to answer questions from people

who read the OCW courses so there is not an ongoing issue– 5) Faculty still can publish books which belong to them

• Most common faculty use– Looking to see what colleagues are teaching

8

Some iCampus projects

• iLabs – students can use web browsers to design experiments and collect data from distant laboratory equipment

• iMOAT – the web is used to manage the process of large-scale assessment of student writing

• TEAL – two terms of introductory physics have been redesigned around inquiry, discussion, experimentation, and visualization

• XMAS – students can ‘quote’ video legally in their online discussions, presentations, and projects about films in courses such as Shakespeare

• xTutor is to be a tool kit for creating online courses; its strength is checking computer programming homework and providing feedback

9

Collection and use of data

• Collection and use of data is necessary (but not always sufficient) to show the efficacy of various educational innovations

• Maps well to the culture of MIT as a science- and engineering-oriented institution

10

TEAL increased learning gains

Sou

rce:

Dor

i, Y

. & B

elch

er, J

. (20

05)

“How

Doe

s T

echn

olog

y-E

nabl

ed

Act

ive

Lea

rnin

g A

ffec

t Und

ergr

adua

te S

tude

nts’

Und

erst

andi

ng o

f E

lect

rom

agne

tic

Con

cept

s?”

The

Jou

rnal

of t

he L

earn

ing

Scie

nces

, 14

(2):

243-

279.

Learning gains <g> = %Correct post-test - % Correct pre-test

100% - %Correct pre-test

0

0.1

0.2

0.3

0.4

0.5

0.6

Low Intermediate High

Le

arn

ing

ga

ins

<g

>.

Lecture Students 2002 TEAL Students 2003

11

Increases seen long term

n=52 control

n=120 experimental

Source: Dori, Y.J., E. Hult, L. Breslow, & J. W. Belcher (2005). “The Retention of Concepts from a Freshmen Electromagnetism Course by MIT Upperclass Students,” paper delivered at the NARST annual conference.

Scores

12

Learning Analytics – Assessing Learning• Learner Analytics

– e.g. Gates Next Generation Learning Challenge– Predict/Reduce drop, fail, withdraw rates in community

colleges, especially in gateway courses• Instead Assess the Learning Process

– Detailed student knowledge (then tutor weaknesses)– Reveal Student Habits– Measure Effectiveness of Instructional Elements

13Eductional Data Mining: Tutors give DATA!Tutor > 105 Bytes vs. Gradebook < 102

• Assessment of Detailed Skills of Student• Formative Assessment for the Teacher• Ultimately will guide individual tutoring

Fine Grain Assessment – Holy Grail

• What Habits help/hinder learning??• e.g. Homework copying reduces learning

Habits of Mind and Behavior

• Analysis of Student Use Data Improves Activity• Which Course Activity Correlates with Learning?• What type of activities give what types of learning?

Improve/Find Highly Instructional Activities

14Detailed Skill Profiles

Measured skill of each student in various topics – 100 is average

Red is below average, Green is above average

This is experimental skill-book in cybertutor.mit.edu (D. Pritchard’s online tutor)

15Improving Efficacy of Assessment Items

Revising the questions based on analysis of the student use data allows dramatic improvement in the performance of students. From 2001 to 2002 the percentage of students who requested the solution went down by ~ 50% (from 12% to ~6%). The number of wrong answers per question decreased by 40%

16

Innovations for which improved learning can be demonstrated did one or more of the following . . .

17

Employed pedagogies of engagement

18

Confidence in abilities increased in 6.01 with introduction of active learning

Programming

Ability to learn new program-ming language on own

2.67

3.16

Source: O’Leary, L. “6.01 Class Experience Survey Synopsis, Spring 2008,” 6/16/08.n = 153

19

Motivated students by grounding learning in real-world problems or projects

20Students’ confidence in crucial abilities strengthened in PBL freshmen subjects

Skill % who said skill strengthened by PBL

Break a problem into key components 87%

Identify tasks, resources, time, and people to solve a problem

79%

Select and combine the best ideas into a better approach

93%

Recognize constraints that limit a solution 91%

Greater confidence in ability to handle a difficult and ambiguous task*

84%

Able to use their own initiative* 87%

See connections between the PB class and what they learned in other subjects*

81%

* In comparison to regular subjects n=90

Source: Office of Faculty Support, “Project-Based Pilot Assessment Results,” 10/10/07.

21

Provided multiple opportunities for

‘practice at retrieval’

22

Performance improved in 16.100

Source: Darmafol, D. “Impact of Pedagogical Changes in an Undergraduate Aerodynamics Subject: A Summary,” 2004.

Pedagogical changes included: concept questions and mini lectures in class; look ahead (graded) homework; written take-home exams; oral exams; semester-long, team-based project

23

Educational technology as an enabler

• The merger of research and education tools– Faculty develop/want to develop research tools– Some of these tools can be used (appropriately

modified) in education

• Simulations and animations• Remote laboratories

24

24

• Early exposure to research tools and experience

This type of hands-on interaction with the molecule provides levels of insights that are not possible by viewing static images on a page on a computer screen

~ Prof. Graham Walker

• Used by 1000 MIT students students; 300 high school students

• StarBiogene, StarHydro, StarHPC

Research tools for learning

StarBiochem

25

• Real laboratory accessed through Internet

• From anywhere at any time

• Not a “virtual laboratory” (simulations)

The role of the innovator: The example of iLabs created by Jesus del Alamo

26iLabs at MIT

Microelectronics device characterization (EECS, deployed 1998)

Shake Table (Civil Eng., deployed 2004)

Dynamic signal analyzer (EECS, deployed 2004)

Polymer Crystallization (Chem. E., deployed 2003)

Heat exchanger (Chem. Eng., deployed 2001)

ELVIS (EECS, deployed 2006)

Spectrometer (Nuclear Eng., deployed 2008)

Force on a Dipole (Physics, deployed 2008)

27

Educational change that has not occurred at MIT

• Idea of perspectives

• Strategic design of curricula change

• Political issues

• Cultural issues

28

Perspectives (John Van Mannen@ Sloan School)

• They determine what data you see (hear, feel) in the organization– What questions you ask– Where your attention lies

• They determine how we interpret the data we see– They prioritize the data we receive and shape our

actions• No single perspective is adequate

– It is easy to get locked into a single perspective,but difficult then to deal with complexity

Perspectives are organized ideas (e.g., metaphors) that fundamentally shape our understanding of things and events.

29

Three perspectives

Strategic DesignOrganizations are

machinesAn organization is a mechanical

system crafted to achieve a defined goal. Parts must fit well

together and match the demands of the environment.

Action comes through planning. Cultural

Organizations are institutions

An organization is a symbolic system of meanings, artifacts, values, and

routines. Informal norms and traditions exert a strong influence on

behavior.Action comes through

habit.

PoliticalOrganizations are contestsAn organization is a social system

encompassing diverse, and sometimes contradictory, interests

and goals. Competition for resources is expected.

Action comes through power.

30

Let’s get specific:

Lessons learned

from the past decade

at MIT.

31

MIT’s Educational MissionIn 1998 the Task Force on Student Life and Learning, as part of two years of intensive study, consultation, and reflection, developed a formal statement of MIT's educational mission:

The Massachusetts Institute of Technology is devoted to the advancement of knowledge and education of students in areas that contribute to or prosper in an environment of science and technology. Its mission is to contribute to society through excellence in education, research, and public service, drawing on core strengths in science, engineering, architecture, humanities and social sciences, and management. This mission is accomplished by an educational program combining rigorous academic study and the excitement of research with the support and intellectual stimulation of a diverse campus community.

The message matters...

32

Senior leadership matters

• The presidentially appointed Task Force on Student Life & Learning was charged in 1996 with undertaking a comprehensive review of MIT's educational mission on the threshold of the 21st century. – President Vest asked: “What does MIT have to do to be the pre-eminent

university in Science and Technology in 2020?”

• The final report published in 1998 provided insight into the principles that define MIT and the attributes of an educated individual.

• In addition, the Task Force made recommendations regarding the General Institute Requirements (GIRs), advising, the first year, teaching, and undergraduate research.

33

The core curriculum remained untouched

34MIT Task Force on the Undergraduate Educational Commons

The Massachusetts Institute of Technology established a Presidential Task Force on the Undergraduate Educational Commons in order to undertake a fundamental, comprehensive review of the common educational experience of our undergraduates in the early years of the twenty-first century.

35

Task Force overview • The Task Force on the Undergraduate Educational

Commons was a committee of two dozen MIT faculty members and undergraduates.

• The Task Force comprehensively reviewed MIT’s ‘General Institute Requirements’ (GIRs), the rigorous foundation in natural science, mathematics, technology, humanities, arts, and social sciences that forms the core curriculum of every undergraduate’s MIT education.

• The Task Force Report affirmed the many ways in which this common curriculum has successfully prepared MIT’s graduates for a lifetime of learning and leadership, but it also recognized that changes in the wider cultural context required curricular change.

36

Strategic design

• After 2.5 years of work, a committee of 24 faculty presented a new core curriculum to the entire MIT faculty

• Tried to get input via– Departmental meetings– Town Hall meetings– Presentations to senior administrators and School

Councils– E-mail to faculty/students and a website– An independent student committee

37

Curriculum design changes • The portion of the General Institute Requirements that focuses

on science and technology should providescience and technology should provide greater flexibilitygreater flexibility in the choice of classes in the fundamental sciences while retaining the rigor that has been the historic hallmark of these classes.

• The Humanities, Arts, and Social Sciences Requirement should Humanities, Arts, and Social Sciences Requirement should bebe clarifiedclarified in order to provide a rigorous foundation in the study of human culture, expression, and social organization.

• MIT should make it clear that acquiring experience living and working abroad is an essential feature of an undergraduate education, and work to expand current international education international education programs that have proven successful in the MIT environment, and develop strategies to create other [global] opportunities.

[http://web.mit.edu/committees/edcommons/documents/task_force_report.html]

38Some recommendations were implemented: Online subject evaluations

• MIT did not have one uniform subject evaluation system. Some departments ran their own online subject evaluations, while most used paper forms provided and processed through the Office of Faculty Support.

• The ‘system’ was a patchwork of processes that did not provide the tools needed by faculty and academic administrators to compare or discern patterns in student responses over the semesters.

• Gradual scale-up & careful piloting has led to positive community response.

• Expected benefits of the online system include:– More thorough data collection and longitudinal analysis– Simplified administration– Better reporting capabilities

39

New GIRs: Full Model

40

Political perspective: School of Science• New Science Math and Engineering (SME) core called for more ‘flavors’ for science, and

for engineering to have a project-based experience, as well as a design class in the core

• Each 12-unit SME GIR element should be offered in a variety of flavors that would share core content ( 6 units).

• Disadvantaged in this would be the science departments particularly physics, math, and chemistry– Budgets are influenced by how many undergraduates departments teach; maybe

even future faculty slots– Math and chemistry support many graduate students as TAs

• These departments all opposed the changes• Dean of Science was in favor but could not carry the School

41

Political perspective: School of Engineering

• Engineering did not care enough to push hard

• Engineering could not or would not articulate a coherent core subject with broad support

• There was disagreement on the intellectual content of the project-based course

• Dean of Engineering was unable to get Engineering faculty buy-in

42

Political perspective: School of Humanities

• Humanities, Arts and Social Science requirement would be taught entirely in that School, so while there were some who were disadvantaged they could be accommodated

• Proposal was seen as providing more coherence, which was endorsed by many

• Advocates worked to create trust in advance, to speak within the community and build consensus, both bottom up and top down

• Dean of Humanities, Arts and Social Science was visibly behind the proposal

43

Cultural perspective

• The proposed SME changes raised deep questions– What is modern science (and how can you devalue my

science by throwing it out!)?– Who can teach this modern science?

• Can only physicists teach physics? (dept said yes)• Can only mathematicians teach math? (dept said yes)

– Is engineering really a fundamental subject like physics and math?

• Is there core content that is different from science? (some in SoE said yes, around design; many in SoS said no)

44

Cultural perspective

• Analytic rigor is a very deep value in MIT culture– The project-based course was heavily critiqued as

lacking rigor and unworthy of being in the core– An engineering GIR (in design, for example) would

require some science subject to be eliminated, which raised concerns about “watering down” the rigor of the undergraduate curriculum

• Some engineers proposed removing humanities or arts subjects but that School objected (power issue)

45

Aftermath

• Humanities changes passed faculty vote• Science Math and Engineering proposal did not pass

(required 60% vote)• Engineering still not part of core, but math has reached out

to work more with engineering• No energy for another try at MIT. Trying new ideas in

Singapore with SUTD and will bring back to MIT.• Several departments have created new degrees and

made changes in major programs• Emphasis on globalization is proceeding (but did not

require a faculty vote!)

46

Much curricula change has occurred

• Often at local level (individual faculty) or departments • Funds must exist to make this happen• Having dedicated organizations for educational

technology, pedagogy, and assessment is very helpful• Dissemination, scalability, and broader

coordination remain challenges• Wholesale change requires a three lens

perspective• Change can often occur more easily outside the

mainstream• MIT continues to be a dynamic exciting place to teach

and learn