oleh oleh hong kong mulyati

Oleh-oleh dari Hong Kong

KEGIATANTHE 2nd EAST ASIAN INTERNATIONAL CONFERENCE

ON TEACHER EDUCATION RESEARCH“Teacher Education for the Future: International Perspectives”

at The Hong Kong Institute of Education

HONG KONG, 15 – 17 DESEMBER 2010

1

INSIDE INDONESIA’S MATH CLASSROOMS

A VIDEO STUDY OF TEACHING PRACTICES

A joint Ministry of National Education (MONE) and World Bank Study

- funded under the Dutch Basic Education Support Program

2 Mae Chu Chang The World Bank

TEACHER EDUCATION FOR THE FUTURE IN INDONESIA

TEACHER EDUCATION FOR THE FUTURE IN INDONESIA

System: there are two departments (MONE, MORA)

Scope Over 3 million teachers 67% of teacher workforce require upgrading 100% required certification

Massive policy reform focused on teacher quality 4 year degree: minimum qualification Certification by 2015 In-service professional development Teacher incentives 4

INDONESIAN CONTEXTINDONESIAN CONTEXT

5

INCENTIVES: doubling of salary for certified teachers, remote area allowance, functional allowance contingent on certification.

PROFESSIONAL DEVELOPMENT INITIATIVES: university changes, teacher working group (KKG-MGMP), more distance learning programs.

Bachelors or more

Less than Bachelors

Indonesian teachers have ways to go before being qualified

(as defined by the new Law)

Indonesian teachers have ways to go before being qualified

(as defined by the new Law)

Source: "Towards Teacher Reform that Produces Professional, Dignified, and Prosperous Teachers", DIKTI PMPTK, MONE, 2010, pg 35

More than 75% of primary

school teachers do not have a

Bachelors degree.

More than 75% of primary

school teachers do not have a

Bachelors degree.

6

LOW LEVELS OF INDONESIAN STUDENTS’ COGNITIVE PERFORMANCE IN MATHEMATICS

(TIMSS 2007)

LOW LEVELS OF INDONESIAN STUDENTS’ COGNITIVE PERFORMANCE IN MATHEMATICS

(TIMSS 2007)

100806040200

20406080

100

Kore

a

Sin

gapore

Japan

Chin

ese

Taip

ei

Aust

ralia

Mala

ysi

a

Inte

rnati

onal M

edia

n

Thaila

nd

Indonesi

a

Syri

a

Moro

cco

Colo

mbia

Ghana

Advanced (>625)

High (550-624)

Intermediate (475-549)

Low (400-474)

Under Low (<400)

Source: 2007 TIMSS Results 77

Improving the Teaching System – Joint GoI and WB effortsImproving the Teaching System – Joint GoI and WB efforts

BERMUTU = Quality Better Education through Reformed

Management and Universal Teacher Upgrading

Project to develop a system to upgrade teachers through working groups, improve quality assurance and career path development for teachers.

Started in 2008, US$ 200 million Parallel studies and pilot in 2007 8

Key questions for the reform that require a video study:

9

What is currently happening in the classroom?What needs to change to increase teacher

effectiveness? What will be the impact of teacher reform on

pedagogical practices in Indonesia’s classrooms? What changes will occur?

How can we obtain rich qualitative and quantitative information?

How can this inform pre and in-service professional development?

How does teacher performance influence student achievement? And what specific classroom behavior impacts students?

How does teacher performance influence student achievement? And what specific classroom behavior impacts students?

10

Previous video studiesPrevious video studies

TIMSS Video Study 1995 (3 countries) Germany, Japan and the United States

TIMSS Video Study 1999 (7 countries) Australia, the Czech Republic, Hong Kong

SAR,Japan, the Netherlands, Switzerland, and the United States.

Based on the premise that the more educators and researchers can learn about teaching as it is actually practiced, the more effectively educators can identify factors that might enhance student learning opportunities and, by extension, student achievement.

Benefits and limitations of a cross-country comparisonBenefits and limitations of a cross-country comparison

Comparing teaching across cultures can introduce fresh perspective by widening the known possibilities and revealing alternative teaching practices.

A cross-country comparison does not necessarily give an answer of the best way of teaching Teaching and learning is culturally unique

BUT it can Reveal Indonesia’s own practices more clearly Show new alternatives Stimulate discussion about choices within

each country Deepen educators’ understanding of teaching 11

Alternatives to Video Study consideredAlternatives to Video Study considered

Interviews: Teachers may have forgotten what actually happened in classroom, teachers may not be honest, teachers may be mistaken about their performance, time consuming.

Questionnaires: Limitations of memory, honesty and perception.

Live Observations: Intrusive, time-consuming, no re-observation

Video Study: different observers can analyze same video, different aspects of classroom can be captured, permanent record, can rewind/pause/fast forward etc.

12

Video Study: OBJECTIVESVideo Study: OBJECTIVES

1. Characterize classroom teaching-learning behavior with reference both to curriculum intention and to classroom characteristics in other countries

2. Provide baseline data for comparison with data to be collected in 2011

3. Relate classroom teaching-learning behavior with student achievement in TIMSS 2007 and to determine effective teaching methods so as to inform ongoing teaching improvement programs

4. Produce an archive of classroom videos for use in research and teacher development in the future

5. Develop the capacity of relevant personnel in Indonesia.

13

Additional Uses of the Video StudyAdditional Uses of the Video Study

14

BERMUTU – videos integrated into teacher working group modules

Sample clips for ‘Center for Development and Empowerment of Teachers and Education Personnel’ to demonstrate examples of teaching practices

Use of videos as supplementary material Incorporating videos as illustrations for online

tutorialsResearch department:

Intends to replicate video study for PIRLS LiteracyPlans to do a science video study

Supervisors and PrincipalsFor use in evaluating teachers and providing feedback

CONCEPTUAL FRAMEWORKCONCEPTUAL FRAMEWORK

Teachers’ Classroom Instructional Practices (base-line)

Improved Instructional Practices

Teacher Law: sets minimum requirements

More qualified, higher quality teachers

Student Achievement (base-line: TIMSS 2007)

Improved Student Achievement (TIMSS 2011)

Video Study (Phase 1)




2005/6

Upgrading and certification

2007 2011

15

2 PHASED DESIGN

Phase 1

Linked to Trends in International Math and Science Study (TIMSS) examination

Design based on established video study methodology and analysis dimensions (Hiebert et al,1999)

Targeted participants: 100 8th grade mathematics classrooms

Analysis: regression ; cross-country comparison, discourse

Phase 2

Repeat of Phase 1 for comparison, trending

Intense observation, questionnaires , interviews and assessments focused on teacher beliefs and mathematical knowledge in teaching

Phenomenological Case Studies: reflective individual discussion

The video study is a means to monitor the classroom competence of teachers over a number of years.The video study is a means to monitor the classroom competence of teachers over a number of years.

16

Key Implementing Units InvolvedKey Implementing Units Involved

The video study is a collaborative effort between MoNE and the World Bank.

Departments involved within MoNE Directorate General for Quality Improvement of

Teachers and Education Personnel (PMPTK) Teacher Professional Development P rogram

(PPPG) Institute for Educational Quality Assurance (LPMP) Centre for Educational Assessment (Puspendik) Directorate of the Management for Junior

Secondary Education ICT Centre for Education (Pustekkom)

17

Some significant findingsFrederick K.S. Leung

The university of Hong Kong

Lesson Structure Content Instructional Practices Relationship between

instructional practices and student mathematics scores

19

Range = 39 – 100 minutesMedian = 66 minutes

Length of Lesson (in minutes)

4145 45 46 47 50 51

70

0

10

20

30

40

50

60

70

80

90

100

LESSON STRUCTURE

% of time used for learning mathematics

LESSON STRUCTURE

8995969697979898

84232211

31211111

0% 20% 40% 60% 80% 100%

IndonesiaAustralia

NetherlandsUnited States

Hong KongSwitzerland

Czech RepublicJapan

Mathematics Mathematics Organization Non-mathematics

Source: Indonesia results combined with Hiebert, J. et. al., (2003),

PracticeNew ContentReview

USE OF CLASS TIME

10

3958

24 2439 35

5348

3322

3960 34 40

23

4128 20

3716 27 25 25

0102030405060708090

100

Japa

n

Switz

erla

nd

Czech

Rep

ublic

Austra

lia

Hong

Kong

SAR

Unite

d St

ates

Nethe

rland

s

Indo

nesia

Percent of math class time by category

INSTRUCTIONAL PROCESSPercentage of time for public (full class) and private (group

or individual) interaction

44

52

55

57

66

68

79

80

55

48

45

43

34

32

22

20

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Netherlands

Australia

Switzerland

Indonesia

Japan

United States

Czech Republic

Hong Kong SAR

Public Interaction Private Interaction

% of Problems by Level of Complexity

Source: Indonesia results combined with Hiebert, J. et. al., (2003), page 71

LESSON CONTENT

17

69 6477

6369 67

57

45

22 2516

2922 27 4039

12 11 8 8 6 6 3

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

High Complexity

Medium Complexity

Low Complexity

6177 84

4157

69

41

247

4

5

1813

35

15 16 13

54

24 17 24

0102030405060708090

100

Make Connection

State Concept

Use Procedure

percentage of problems per math lesson of each problem statement type

Lesson Content

LESSON CONTENTAverage number of independent problems solved in the

lesson and average length of time per problem in minutes

3 35

7 78

10

13

7

15

43

42

54

02468

10121416

Average Number of Independent Problems Average Problem Length (minutes)

Percent of lessons with at least one problem where more than one solution is presented

10 16 23 24 25 30 37 42

90 84 77 76 75 70 63 58

0102030405060708090

100

Class has no problems with more than one solution

Class has at least one problem with more than one solution

LESSON CONTENT

LESSON CONTENTAverage percentage of problems per lesson that were

applications

Source: Indonesia results combined with Hiebert, J. et. al., (2003), page 91

16

34 35 40 4551 55

74

01020304050607080

28

% of time by instructional approach

Investigation3%

Practical work10%

Disussion15%

Problem solving

20%

Exposition52%

INSTRUCTIONAL PRACTICES

29

Average Teacher and Student Words(Calibrated to a lesson of 50 minutes)

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

Ind OCs Ind OCsAverage number of teacher words

Average number of students words

2,633

5,902

5,148

1,018

640197

Nu

mb

er

of

word

sOPPORTUNITIES TO TALK

30

Ratio of Teacher to Student Words

8

9

9

10

13

16

25

0 5 10 15 20 25 30

United States

Australia

Czech Republic

Switzerland

Netherlands

Hong Kong

Indonesia

Number of teacher words per one student word

INSTRUCTIONAL PRACTICES: OPPORTUNITIES TO TALK

State concept

Use procedure

Make connection

SET-UP OF PROBLEMS

Procedural complexity

PROBLEMS

% of problems with application

% of problems with proof

▲

▲

▲

New content

Assessment

Review

LESSON STRUCTURE

▲

Non-math

Math organization

Math

USE OF TIME

▼

▲

Practice ▼

Student presenting

Student and teacher

Lecture (teacher only)

FULL CLASS INTERACTION

▲

▲

▼

REGRESSION ANALYSIS: RELATIONSHIP BETWEEN TEACHING PRACTICES AND STUDENT MATHEMATICS SCORES

Student Math Score

TEACHER BACKGROUND

HOME

STUDENT

CLASS

SCHOOL

Control for variables

▲

% of class spent on problems

PROBLEM TIME

▲

Small group or individual work

Full class interaction

TYPE OF INTERACTION

Individual seatwork

Small groups

PRIVATE INTERACTION

Communication

Problem solving

Reasoning

LEARNING METHOD

▲

▼

Use textbook

Use math material

Use calculator

Use real world situations

Use projector

OBSERVED USE

▼

▲

▲

▼

= statistically significant positive relationship

= statistically significant negative relationship

Real world concepts

Math language and symbols

CONCEPTS USED

▼

▲

SUMMARY OF MAIN FINDINGS Lesson Structure In Indonesia, more time spent on non-math activities,

practice, non-problem work – less on review, assessment

Classes with good lesson plans and effective use of classroom time scored higher

Content Relatively few higher-order thinking problems Classes that used proofs scored higher Application problems used less often than other

countries Goal and summary statements were used by teachers

frequently

SUMMARY OF MAIN FINDINGS Instructional Practices Teachers tend to have a transmissionist orientation Teacher approach to maths: one correct approach to

the solution Students speak much less than teacher, use short

responses when answering questions Classes with higher student involvement tended to

score higher Student problem solving often involves simple

repetition of teacher examples Calculators were rarely used

Inside Indonesia’s Mathematics Classrooms

Paper 3:

Implementation Challenges and

Solutions34

Ratna Kesuma

Distribution of Research Sample

51 Districts: Red Aceh and Papua: Yellow

35

Partnership: MoNE and World Bank

MoNE perspectivesTeaching and learning insights linked to TIMSS scores (2007 and 2011).Professional development resource for quality improvement.

World Bank, Jakarta perspectivesMoNE capacity building in Video Study research methodology.Professional development tool: Teacher Working Groups and training institutions.

36

37

Video-Taping Process Dynamic

Mixing

Static

Product

ChallengeSchools: Honoured to be selected

“Let’s show our best: the dignity of the school is at stake”

“Let’s use our most skilled teachers and clever students – and, rehearse”

Let’s repair the classroom so it looks good on the video”

38

Challenges: classroom video-taping

39

Video can interfere with the classroom environment:

Stage managedEveryone on best behaviourTeachers: fear of the camera and being observedClassroom layout adjusted for cameras – not the norm

Challenges: MoNE Study Team

40

Video Study role in addition to normal workload

Lessons for Phase II (Video Study of TIMSS 2011)

Core Team: Lacked expertise in analysis and reporting. Experts in Maths only. University experts sourced for analysis and reporting.

Employ 4 full time personnel as Study Team.

Ensure schools are :-well-briefed -understand study focus: Realities in the Mathematics Classroom.

Support Team: Involved in data collection only. Unable to code data, analyse, report due to other work demands.

Examination Team: Provided and collected data. Limited due to other work demands.

Video Team: Insufficient resources for 100 schools. Employed external contractors.

Contract out the video team work.

Breaking down the data: Main coding tree

Structure

Math Organization

Math Work Non-Math Work

Review New Content Practice Assessment

Public Private

Group

Individual

Without

teacher

With teacherStudent

Teacher

Teach&Stud

Without

teacher

With teacher

Public Private

Group

Individual

Without

teacher

With teacherStudent

Teacher

Teach&Stud

Without

teacher

With teacher

Public Private

Group

Individual

Without

teacher

With teacherStudent

Teacher

Teach&Stud

Without

teacher

With teacher

Public Private

Group

Individual

Without

teacher

With teacherStudent

Teacher

Teach&Stud

Without

teacher

With teacher

A Branch of the Coding Tree

42

Individual private interaction - new content

problem non problem

exposition is sinp2 discussion is sinp2practical work iss2investigation iss2problem-solving iss2

mis-concept Q & A is2 mis-concept

open is2

close is2

true or false is2

rhetoric is2

item type is pm2

item solution is pm2

try and error is pm2

use graph is pm2

work backward is

pm2

from the simplest is

pm2

use path is pm2

solution type is pm2

non routine is pm2

routine is pm2

quality is pm2

open is pm2

close is pm2

context is pm2

real life is pm2

math language is

pm2

nature is pm2

concept is pm2

connection is pm2

proof is pm2

procedure is pm2

item type is inv2

item solution is inv2

try and error is inv2

use graph is inv2

work backward is

inv2


inv2

use path is inv2

solution type is inv2

non routine is inv2

routine is inv2

quality is inv2

open is inv2

close is inv2

context is inv2

real life is inv2

math language is

inv2

nature is inv2

concept is inv2

connection is inv2

proof is inv2

procedure is inv2

item type is pr2

item solution is pm2

try and error is pr2

use graph is pr2

work backward is

pr2


pr2

use path is pr2

solution type is pr2

non routine is pr2

routine is pr2

quality is pr2

open is pr2

close is pr2

context is pr2

real life is pr2

math language is

pr2

nature is pr2

concept is pr2

connection is pr2

proof is pr2

procedure is pr2

PHASE 2: DEEPER ANALYSIS

43

System and Policy

Influences

Cultural Influences

Pupil’s response

s

Teacher’s practices

Teacher’s beliefs

Teacher’s Mathematics Knowledge in

Teaching

What it is to be a mathematics -

capable pupil

How pupils learn to do

mathematics

How best to teach

mathematics

Mathematics subject

knowledge

Knowledge of teaching

approaches

Knowledge of pupils

StudentLearning

Source: framework based on model of Askew et al (1997)

Challenges & Solutions

Limitation Addition: Before and After Tests

TIMSS is a single test.•Cannot determine teacher “value add” to student progress over the course of the year.

•TIMSS differs in some areas from the national curriculum

Conduct:•Before test: A baseline mathematics test. •After test: A test at the end of the year .

Test: • Previous TIMSS exams (38%) + Nationally developed exams (62%). Benefits

• A “value added” measure and better understanding of the links between teacher background and teaching practices with student learning.• Can compare between the TIMSS results and the nationally-oriented results.44

Challenges & SolutionsLimitation Addition: Teacher

Assessment

TIMSS data on teachers:•Does not provide a sense of the teacher’s subject or pedagogic knowledge.

• Conduct a subject and pedagogy test for mathematics teachers at the beginning of the 2010 school year.

BenefitsCan provide insights into:•the role of teachers’ mathematical knowledge in teaching

on classroom instruction. •the importance of subject and pedagogic knowledge

on teaching practices and on student outcomes.45

Challenges & SolutionsLimitation Addition: Teacher Case Studies

• Video alone does not demonstrate why teachers choose to use various types of teaching techniques.

Aim: To understand factors that influence teaching practices (e.g. personal decisions, environment and support structure, various policies).

Target 10 teachers and conduct in-depth case studies: •Interviews; classroom & teacher working group observations; stimulated recall with teacher for insights about the video; teacher journals & lesson plans.Benefits:

•Qualitative analysis to provide deeper insights, ‘stories’ and to enable more complete evidence-based policy making.46

Challenges & SolutionsLimitation Addition: Further coding layers

along with qualitative analysis for video

• Initial coding was not intended to determine whether the teacher is conducting practices in an effective manner.

• Integrate new coding layers.• Deeper qualitative analysis of

videos focused on teaching quality, teacher enactment of mathematical knowledge, beliefs

Benefits:•Provide more insights on effectiveness and quality of classroom activities. •Explore the link of mathematical beliefs and mathematical knowledge in teaching with actual teaching practices

47

Phase 2 (Video Study of TIMSS 2011)

Repeat: BASE STUDY (2007)

48

Phase 2 (Video Study of TIMSS 2011) Deeper Analysis

49

Professional Development of Mathematics Teachers

Promotion of MONE’s policy of Active Learning, which is supported by findings that more active student participation is associated with higher test scores

Review training in pre-service, in-service, teacher groups in light of findings, which indicate need to:

Orient problem solving to encourage higher order thinking, use of applications and proofs, making connections, multiple solutions

Ensure effective and efficient use of class time, proper planning

Increase use of activities such as review and assessment, which currently receive little time

Encourage student engagement, participation and active learning

Effectively use resources (projector, textbooks, math materials) in learning

POLICY RECOMMENDATIONSPOLICY RECOMMENDATIONS

50

Professional Development of Mathematics Teachers

Incorporate video techniques into teacher group activities for reflective self and peer evaluation

Leverage videos for examples of good and innovative practices; develop a teaching practices archive

Educational System Support (MONE/MORA)Review current policies of combining two periods of

math, allow use of calculators in classSupervision by mathematics supervisors through on-

the-spot training using communicative supervision so that teachers and principals receive feedback useful for improving the quality of teaching.


51

Educational System Support (MONE/MORA)Monitoring by head teacher of classroom teachers

based on feedback applied by supervisorsExtending the video study:

Conducting similar video studies in other subjects such as science, literacy

Continuing the mathematics video study beyond 2011 to allow for long-term trending

Incorporating teacher assessment and pre- and post- testing of students to measure gains


52

Thank you very much for your attention!

53