seoul foreign schools all[1]

March 2011

All Schools Report

Seoul Foreign SchoolsSeoul, South Korea

600 Corporate Pointe, Suite 1180

Culver City, CA 90230

www.metiri.com

Table of Contents

1Dimensions 21 Introduction

Dimension 1: Forward Thinking, Shared Vision 2

Dimension 2: Systems Thinking 4

Dimension 3: 21st Century Skills and Learning Approaches 7

Dimension 4: 21st Century Learning Environments 11

Dimension 5: Teacher Proficiency with 21st Century Learning 16

Dimension 6: Access and Infrastructure 20

Dimension 7: Accountability and Results 25

Student Outcomes/Student Perspectives 28

Student Engagement 29

Classroom Structures to Engage Learners 36

All Schools Report

Introduction

Fast Facts

Project:

Location:

Timeframe:

Accomplishment:

Dimensions21 (D21)

Dimensions21 provides schools with insights

into the elements required to translate 21st

Century learning into action. The 7 dimensions

represent the divergent and innovative thinking it

takes to ground schools in emergent cognitive,

social, and neuroscience. Metiri Group

developed metrics that gauge a school or

district’s progress in establishing 21st Century

systems of learning. Each dimension is

calibrated on an 8-point scale:

D21 Scores: Year 1 (2011)

0 1 2 3 4 5 6 7 8

1: Vision

2: SystemsThinking

3: 21st CenturySkills

4: LearningEnvironment

5: TeacherProficiency

6: Infrastructure

7:Accountability

4.4

5.0

4.4

3.5

5.0

5.3

3.7

Province

31 0Total School N = Teachers & Administrators

The 1500 students attending Seoul Foreign

School represent approximately 52 nationalities,

making SFS easily the largest and most

well-established international school on the

Korean peninsula. Our students come mainly

from the expatriate business and diplomatic

community; all must have foreign (non-Korean)

passports to enroll at our school. The majority of

our students are from the United States,

followed by the United Kingdom, Canada,

Australia, and Japan. Our graduating class

consists of approximately 85 students each

year, virtually all of whom go on to 4 year

universities in North America and elsewhere

around the world.

1

Dimension 1: Forward-Thinking, Shared Vision

“The best vision is insight.”

-Malcolm S. Forbes

Vision matters. A forward-thinking, shared vision

serves as a unifying and energizing force of

change within a school system.

It sets the targets to which all curriculum,

instruction, assessment, scheduling, progress

reporting, resourcing, and community

communications are aligned.

The indicators and key questions within this

dimension include:

· A Forward-Thinking Vision for All

Learners. Is there a 21st Century vision for

all learners that defines what it means to be

educated in a knowledge-based, global

society?

· A Sound Base in the Learning Sciences. Does the vision represent current research

findings from the cognitive, social, and

neurosciences?

· Communication and Commitment. Have stakeholder groups had a voice in shaping

the vision? Has the jurisdiction

communicated with them about the vision?

Are they generally supportive of and

committed to the vision?

Aggregate Chart

Figure B

Dimension 1 Scores

The top chart at the right indicates the current

status of Dimension 1. In order to provide

context to the numeric scores, frequency reports

from several of the survey questions follow. In

some cases the results from the survey

questions will be in tables, in others charts, and

where appropriate, the responses will be cross

tabulated or provide comparisons between

respondent groups such as teachers and

administrators.

Dimension 1 Findings

Figure D1-2: Percentage of teachers reporting levels

of support for the Seoul Foreign School vision for 21st

Century learning.

Extremely supportive 58.1%Somewhat supportive 25.8%Undecided or don't know 16.1%

Total: 100.0%

0 1 2 3 4 5 6 7 8

Administrators

Teachers

Aggregate

4.4

4.4

Dimension 1

Figure D1-1: Summary of weighted scores from teacher

and administrator surveys.

2


The responses to this question reveal the extent to which teachers see each 21st Century Skill

embedded in the vision. Since most schools are focusing on a subset of these 21st Century Skills, there

may be value in comparing the teachers’ perceptions to the actual wording of the vision, or perhaps the

current emphasis on particular skills by the school or jurisdiction.

0 1 2 3 4 5 6 7 8

Social responsibility

Citizenship in a changing,global society

Individual integrity and ethics

Community connections

Cultural diversity

Digital literacy

Knowledge work andentrepreneurship

Lifelong learning

4.8

5.2

5.6

3.5

4.5

4.2

3.4

4.8

Figure D1-3: Teachers' ratings of the emphasis (on a scale of 1-8)

of each 21st Century Skill embodied in the school vision.

3

Dimension 2: Systems Thinking

"Systems thinking is a discipline for seeing

wholes, recognizing patterns and

interrelationships, and learning how to structure

those interrelationships in more effective,

efficient ways."

- Peter Senge, Schools That Learn

Fundamental to the application of systems

thinking in education is an openness to

innovation, reinvention, and formative, systemic

change driven by the vision. The indicators and

key questions within this dimension include:

Aggregate chart

Dimension 2 Scores

· Leadership. Are the school and/or

jurisdiction leaders leading a

high-performance education system that

enables each student the fullest opportunity

to achieve the vision?

· Curricula, Instruction, and Assessment.

Do the jurisdiction learning standards

integrate academic content with 21st

Century Skills? Are curricula, instruction,

and assessments aligned to provincial

standards?

· Professional Development. Do the school

and jurisdiction provide comprehensive

professional growth opportunities for

administrators, teachers, and other staff,

which build their capacity to advance the

vision?

· Culture of Learning and Innovation. Does

the school or jurisdiction encourage and

support school change that advances 21st

Century learning, and innovative,

educationally sound uses of technology?

· Policies Supportive of the Vision. Are

there established policies that formally

establish 21st Century learning and effective

technology use as a required design

element in all strategic planning, school

improvement, budgeting, human resources,

and accountability systems?

The top chart indicates the current status of

Dimension 2. Results from specific questions

follow.


Figure D2-2: Percentage of teachers reporting their

agreement with the statement: Teachers are provided

the resources and support to redesign classrooms into

21st Century learning environments.

Always 16.1%Sometimes 51.6%Rarely 19.4%Never 9.7%Don't know/Not sure 3.2%

Total: 100.0%

0 1 2 3 4 5 6 7 8

Administrators

Teachers

Aggregate

5.0

5.0

Dimension 2

Figure D2-1: Summary weighted scores from teacher

and administrator surveys (2009)

4

The opportunity for high-quality, relevant professional development is critical to any change process. The

following questions provide insights into teachers’ agreements with statements about the professional

development experiences currently made available to them through their school or jurisdiction.

The statements represent characteristics and attributes of high quality professional development

experiences. The reader should note where large percentages of teachers indicate that the statement is

"Never" or "Rarely" representative of their jurisdiction or school’s professional development, and set

targets to improve that situation.


Insert Figure G here

It is important that, as 21st Century Skills are integrated into lessons, it is accomplished systematically

and systemically, so all children are afforded such opportunities, not just those students who happen to

be assigned to classrooms of innovative teachers.

The data in the chart on the top of the next page represent strong indicators as to whether or not your

school or jurisdiction is taking the necessary policy actions that will result in systemic change and

integration of 21st Century Skills and effective technology uses to advance learning.

0% 20% 40% 60% 80% 100%

10%

16%

3%

16%

10%

13%

16%

29%

42%

39%

32%

55%

48%

42%

58%

35%

42%

45%

29%

29%

32%

3%

3%

13%

3%

3%

6%

3%

3%

6%

3%

6%

3%

Includes opportunities for teachers to see actual examples oftechnology applied to learning in classrooms similar to my own.

Includes opportunities for teachers to see actual examples of 21stCentury Skills applied to learning in classrooms similar to my own.

Allows teachers to practice skills acquired during professionaldevelopment in real or simulated classroom settings.

Prepares teachers to discuss specific research or theory upon whichthe training is based.

Prepares teachers to assess student work produced with technology.

Prepares teachers to assess student work related to 21st CenturySkills.

Includes time for teachers to work together, and to discuss and planfor using technology in the classroom.

Never Rarely Sometimes Always Don't Know/Not Sure

Figure D2-3: Percentage of teachers reporting on their level of agreement about these statements regarding the degree to

which professional development provided by their schools or jurisdiction:

5

Insert Figure H here


The final chart in this section reports the percentage of administrators in the province who report that

teachers are required to consider innovative approaches to teaching and learning in their classrooms

(see list of innovations in the chart below). These results are strong indicators of the degree to which the

jurisdiction is systemically integrating 21st Century learning and technology across the system.


Insert Figure I here

Figure D2-5: Percentage of administrators indicating their level of agreement with the statement, When

teachers implement jurisdiction curriculum, or design curricula, this school requires that they consider:

0% 20% 40% 60% 80% 100%

21st Century Skills have beenpurposefully incorporated into

learning standards.

21st Century Skills have beenpurposefully integrated into curricula

where appropriate.

Our assessments track studentprogress with 21st Century skills.

25%

14%

36%

58%

69%

44%

17%

17%

4% 16%

Rarely Sometimes Always Never

Figure D2-4: Percentage of teachers reporting on their level of agreement about these

statements (systemic integration of 21st Century Skills).

6

Dimension 3: 21st Century Skills and Learning Approaches

· Knowledge Age Literacies. Are students

acquiring and excelling at the skills needed

to be “literate” in the Knowledge Age?

· Inventive Thinking. Are students thinking

critically and creatively as they successfully

solve problems using high tech tools?

· Community Interaction. Are students

acquiring such skills?

· Generating Quality Results. Are students

learning to plan, manage, and achieve high

quality, impactful results?

· Authenticity and Engagement. Are

students being assigned rich, authentic work

that engages them and involves

construction of knowledge through

disciplined inquiry, resulting in products that

have value beyond the classroom?

The top chart indicates the current

status of Dimension 3. Results from specific

questions follow.

“For more than half a century, the United States

has led the world in scientific discovery and

innovation. It has been a beacon, drawing the

best scientists to its educational institutions,

industries, and laboratories from around the

globe. However, in today’s rapidly evolving

competitive world, the United States can no

longer take its supremacy for granted. Nations

from Europe to Eastern Asia are on a fast track

to pass the United States in scientific excellence

and technological innovation.”

--Taskforce on the Future of

American Innovation

Innovation is fueling the economy of the 21st

Century. Globalization has created new markets

and leveled the playing field citizens of all

nations around the world. As society changes,

the skills that citizens need to negotiate the

complexities of life also change. Innovative,

inventive thinking was once required for only a

limited few. Today, and tomorrow, it will be the

currency for success in virtually every field.

The elements within this dimension include:

Figure J

Aggregate Chart

Dimension 3 Scores


0% 20% 40% 60% 80% 100%

Teaching critical thinking skills

Building creative thinking skills

Teaching problem solving tostudents

7%

8%

7%

Teachers Administrators

Figure D3-2: One of the most critical 21st Century Skill area

is higher order thinking. This chart provides a 2008

reference point for thinking skills curricula.

The percentage of teachers and administrators answering

“yes” to the question: Does your school have a formal

curriculum for:

0 1 2 3 4 5 6 7 8

Administrators

Teachers

Aggregate

4.4

4.4

Dimension 3

Figure D3-1: Summary weighted scores from teacher and

administrator surveys (2009)

7

The following charts are purposely matched up. The left-hand column reports current status of initiatives

related to each skill as reported by administrators. The right-hand column charts both teachers’ reports of

relevance of each skill to their content, and teachers’ comfort level with each skill. It is important to

compare not only the level indicated by the administrators and the teachers, but any gaps that exist

between the teachers’ comfort level and relevance level for each skill.

Dimension 3 Findings Dimension 3 Findings

Figure L

Dig

ital

Lit

era

cy


Figure O Figure N

Inve

nti

ve T

hin

kin

g

0 1 2 3 4 5 6 7 8

Global Awareness

Scientific Literacy

Information Literacy

Multimodal Literacy

Figure D3-3: Administrators' mean score indicating the

degree to which a formal initiative on the 21st Century

skill is underway in their schools (scale 1-8):

0 1 2 3 4 5 6 7 8

Global Awareness

Scientific Literacy


Multimodal Literacy

5.1

2.5

5.3

4.1

6.5

2.8

6.4

5.6

Teacher comfort level Relevance to content

Figure D3-4: Teachers' mean score related to comfort

level with each skill and relevance to their content

area:

0 1 2 3 4 5 6 7 8

Creativity andInnovation

Critical Thinking

Flexibility &Adaptability

Self-direction

Systems Thinking




0 1 2 3 4 5 6 7 8

Creativity &Innovation

Critical Thinking

Flexibility &Adaptability

Self-direction

5.7

5.9

5.5

5.8

6.8

7.1

6.4

6.8




area:

8

The comparison of administrator and teacher perspectives on 21st Century Skills continues:


Figure Q Figure P

Effe

ctiv

e C

om

mu

nic

atio

ns


Figure S Figure R

Hig

h-Q

ual

ity

Pro

du

ctiv

ity

It is recommended that the reader identify any gaps that exist between the administrator and teacher

perspectives related to relevance of the 21st Century Skills or between the teacher comfort level and

relevance to content. Such gaps should be addressed through action plans at the jurisdiction and building

levels. Look for opportunities represented by teachers’ indicators for high relevance, but underdeveloped

comfort levels. In these situations the applicable professional development and teacher support can

quickly be leveraged into opportunities that positively impact students.

0 1 2 3 4 5 6 7 8

InteractiveCommunication

Teaming &Collaboration

Cross-cultural Skills

Personal & SocialResponsibility




0 1 2 3 4 5 6 7 8

InteractiveCommunication

Teaming &Collaboration

Cross-cultural Skills

Personal & SocialResponsibility

5.5

5.9

5.3

5.7

6.8

6.6

6.0

6.4




area:

0 1 2 3 4 5 6 7 8

Prioritizing, Planningand Managing for

Results

Effective Use ofReal-world Tools

Productivity Category




0 1 2 3 4 5 6 7 8

Prioritizing, Planning andManaging for Results

Effective Use of Real-worldTools

Productivity Category

4.7

3.9

4.4

6.3

6.1

5.8


Figure D3-10: Teachers' mean score related to

comfort level with each skill and relevance to their

content area:

9

Teachers assign a variety of types of work to students, depending on many factors some of

which are outside of their control. For each of the following categories of student work, teachers and

administrators were asked to estimate the percentage of work that fell into that category. The totals are

more than 100% because of duplication within single assignments.


Insert Figure T here

Two key factors in student engagement are opportunities for student choice and student creativity. The

following chart provides insight into such opportunities in your school or jurisdiction.


Insert Figure U here

0% 20% 40% 60% 80% 100%

Skill building exercises (worksheets,problem-sets, etc.)

Applying skills through essays, short-answerproblems or exercises

Written products that summarize content relatedto the curriculum (e.g. reports)

Written products that require the creation oforiginal content

Applying skills to problems that are complex andemulate work done in the real world

41%

33%

30%

31%

37%


Figure D3-11: Percentage of work assigned by teachers that falls in the

following categories, as reported by administrators and teachers:

0% 20% 40% 60% 80% 100%

11%

59%

38%

15%

Student-designed assignments

Teacher-designed assignments

Technology-based products defined by the teacher

Student-designed, technology-based products


Figure D3-12: Percentage of work in classrooms in your school that falls in

the following categories, as reported by administrators and teachers:

10

Dimension 4: 21st Century Learning Environments

The learning environment is where the vision

becomes a reality, where 21st Century learning

comes to life.

· Alignment with 21st Century Vision. Do

the jurisdiction content, instruction, and

assessment align to 21st Century learning

and academic content standards?

· Informed Practice. Are educators

establishing learning environments that are

structured as respectful classroom

communities where students can work

creatively and productively, places that

motivate, interest, and scaffold students to

think critically?

· Culture of Innovation, Engagement, and

Collaboration. Are professional learning

teams working together to design and

facilitate collaborative 21st Century learning

activities with students? Are activities

evidence-based? Are students producing

high-quality work that is valued by peers,

parents, and community?

· Resources Aligned to 21st Century

Learning. Do students have access to a

wide variety of multimodal resources? Are

these sources accessible inside and outside

the school environment?

· Digital Tools: Range of Use. Do students

have the opportunity to use a range of

technologies (e.g., productivity tools,

visualization tools, research and

communication tools, etc.) to support 21st

Century learning and academic

achievement?

· Assessment for Learning. Is assessment

systematically used to inform practice? Do

students set learning goals based on

standards? Are they actively engaged in

monitoring their own progress toward those

goals?

· Local and Global Connections. Are there

formal, technology-based structures that

engage stakeholders and learners in

meaningful exchanges, interactions, and

partnerships at the local and global levels?


status of Dimension 4. Results from specific

questions follow.

Figure V

Figure W


Dimension 4 Scores

0% 20% 40% 60% 80% 100%

Strongly Agree

Agree

Disagree

Strongly Disagree

Don't Know/Not Sure

Others

4%

8%

14%

6%

11%

56%

Overall

Figure D4-2: Percentage of teachers indicating

agreement with the statement: Teachers in the

Emerge program know what the school’s expectations

are for student attainment of 21st Century Skills.

0 1 2 3 4 5 6 7 8

Administrators

Teachers

Aggregate

3.5

3.5

Dimension 4

Figure D4-1: Summary weighted scores from teacher

and administrator surveys (2008)

11

The perceptions of teachers as to the applicability of the use of technology to their specific teaching

assignments provides insights into why some classrooms are integrating technology and others are not.


Insert Figure X here

If all students are to be afforded new opportunities in 21st Century learning and the innovative use of

technologies, all teachers must adopt such use systemically. Otherwise, student opportunity is

determined by which teacher’s classroom a student happens to be assigned. The chart below indicates

the extent to which teachers in your school are systemically implementing evidence-based practices.


Insert Figure Y here

0% 20% 40% 60% 80% 100%

25%

23%

39%

53%

13%

36%

50%

50%

55%

28%

33%

69%

36%

13%

25%

23%

22%

7%

13%

18%

11%

7%

6%

9%

38%

Reading

Language Arts

Mathematics

Science

Social Studies

Arts

Health/PE

Insignificant Role Moderate Role Significant Role No Role

Figure D4-3: Percentage of teachers indicating the role technology plays in building skills

or proficiencies in their students in the following content areas:(NOTE: These data reflect responses only from teachers who indicated the subject was applicable to their teaching

n: Reading (28), Language Arts (22), Mathematics (18), Science (15), Social Studies (16), Arts (11), and Health/PE (8).

0% 20% 40% 60% 80% 100%

Share little or no common understanding aboutevidence-based practices. Teachers decideindividually whether and how they will make

instructional decisions.

Share some common understanding aboutevidence-based practices; however, some

teachers implement these uses and others donot.

Share a common understanding aboutevidence-based practices, and there are clearexpectations that such practices will be used.

8%

32%

3%

Figure D4-4. Percentage of teachers that reported: In my school teachers in the

same grade or subject areas:

12

The type of instructional strategy used in classrooms can augment, or inhibit, 21st Century learning and

technology use. Highly qualified teachers use a variety of strategies. As a rule of thumb, the reader might

look for a balance of use across the instructional strategies in the chart below, while ensuring that the

strategies that engage learners (i.e., interactivity, inquiry, collaboration) and those that build skills and

meet individual student needs (e.g., differentiation) are all fairly high.


Insert Figure Z here

Students learn in a multiple of venues, many of which are outside the formal school day. Increasingly,

educators are recognizing the value in preparing students to be self-directed in these informal learning

spaces. This helps them ask deep questions and sustain curiosity, as they seek clarity, depth, accuracy,

comprehensiveness, and currency of the topics they are exploring. Informal learning represents

tremendous opportunity for extending students’ exploration and understanding of academic content

beyond the school day.

0 1 2 3 4 5 6 7 8

Direct Instruction (e.g., lecture, didactic questioning,demonstrations, guided reading, etc.)

Inquiry (e.g., guided inquiry, problem-based learning,learning from cases, etc.)

Mediating student thinking through questioningstrategies, thinking skills, and Habits of Mind

applications

Experiential learning (e.g., field trips, simulations,games, conducting experiments, etc.)

Collaborative teaming (e.g., students workingcollaboratively on an assigned project, etc.)

Independent study by individuals or teams (e.g., writingessays, producing videos, computer-assisted

instruction, virtual learning, journaling, researchprojects, etc.)

Interactive instruction (e.g., active learning strategies,debates, brainstorming, think/pair/share, jigsaw,

problem solving, conferencing, etc.)

Differentiation of instruction (i.e, multiple approaches tolearning a single topic)

5.2

5.1

4.5

3.9

5.7

5.5

4.6

4.3

3.7

6.6

5.9

6.3

6.6

6.0

5.8

6.3

Current UseImportance for 21stCentury Learning

Figure D4-5: Teachers’ ratings (scale 1-8) related to various instructional strategies.

13

The following chart indicates the importance teachers place on such informal learning and their

perceptions of students’ current use of such.


Insert Figure AA here

Technology use in schools is shifting to more collaborative and innovative uses of Internet resources,

Web 2.0 tools, and multimedia production. The chart below provides insights from administrators as to

current use and importance of such technology uses by students.


Insert Figure AB here

0 1 2 3 4 5 6 7 8

3.1

2.6

3.4

5.0

5.1

4.9

Informal learning at school (afterschool activities,

peer-interactions, etc.)

Informal learning beyond the school day(chat, text messaging, student web

browsing/searching, etc.)

Mentoring or coaching by an adult (e.g.,parent, teacher, expert, etc.)


Figure D4-6: Teachers’ ratings (scale 1-8) related to informal learning strategies.

0 1 2 3 4 5 6 7 8

Solve real-world problems

Produce print products

Produce multi-media, Web, digital audio,digital video, or presentation products

Conduct online research

Use drill and practice or tutorial software

Use the Internet to collaborate withstudents in your school, district, or local

community

Online communication with experts, peers,and others


Figure D4-7: Administrator ratings of current uses of technology by students in their schools,

and the administrators rating of the importance of such uses to 21st Century learning:

14

The Internet has literally linked the individual to social and professional networks 24/7. Those

communication avenues represent opportunities for schools to increase communication with parents,

community, and students. The following chart outlines the current status of your school’s uses of

technology to facilitate such outreach and interaction.


Insert Figure AC here

After a careful review of these data, the reader will want to consider the critical questions listed on the first

page of the section, identify any gaps in your schools performance, and set targets for improvement.

0% 20% 40% 60% 80% 100%

Parents’ involvement in my students’ schoolwork

Interactions with students’ parents

Students’ work on authentic projects in their localcommunity

Students’ work on authentic projects outside theirlocal community

23%

13%

71%

61%

48%

23%

13%

13%

29%

65%

16%

26%

Technology is not usedfor this purpose

Occasionally facilitatedby technology

Strongly facilitated bytechnology

Figure D4-8: Parental or community involvement facilitated by technology.

The percentage of teachers that reported:

15

Dimension 5: Teacher Proficiency

“On a daily basis, teachers confront complex decisions that rely on many different kinds of knowledge

and judgment and that can involve high-stakes outcomes for students’ futures.”

- John Bransford, Linda Darling-Hammond, & Pamela LePage

The transition to 21st Century requires systemic

action that builds the capacity of teachers and

administrators.

· Knowledge and Facility with 21st Century

Skills. Are teachers in this school familiar

with the concept of 21st Century Skills and

with the research underpinnings and

practical applications of these skills?

· Building 21st Century Skills. Do teachers

in this school have a variety of strategies for

building these skills?

· Designing Rigorous Authentic Curricula.

Are teachers skilled in designing rich

curricula that integrates content, 21st

Century Skills, and technology, which

provides a digital age learning context?

· Differentiated Instructional Strategies.

Are teachers in this school skilled in

engaging all students in learning through a

variety of teaching and organizational

strategies that are tailored to the needs of

individual students?

· Informed Use of Data and Research. Are

teachers in this school skilled at accessing,

organizing, and acting upon available data

to make important decisions about students

and learning?

· Assessment for Learning. Do teachers in

this school have a deep understanding of

the central role of assessment in the

learning process and leverage technology

resources to assess core content and 21st

Century Skills?

· Professional Practice and Productivity.

Are teachers skilled in the use of technology

to support their own professional practices

and do they depend on technology to

Dimension 5 Scores

Figure AE

Pie chart



status of the province in this dimension. In order to

provide context to the numeric scores, frequency

reports from several of the survey questions are

provided on the following pages.

Figure D5-2: The percentage of teachers indicating

their preparedness in assessing student products

created using technology.

Expert 17.2%Intermediate 58.6%Novice 24.1%

Total: 100.0%

0 1 2 3 4 5 6 7 8

Administrators

Teachers

Aggregate

5.0

5.0

Dimension 5

Figure D5-1: Summary of weighted scores from

teacher and administrator surveys (2009)

16

The charts on this page provide teachers’ perspectives on their preparedness to scaffold conceptual

learning, and similarly, their preparedness to do the same for authentic learning. Teachers typically report

higher levels of comfort with the conceptual learning, but, for 21st Century learning, the latter is critical.


Insert Figure AF here

stacked bar chart


Insert Figure AG here

stacked bar chart

If a significant percentage of your teachers indicate a lack of preparedness to scaffold conceptual learning,

your school or jurisdiction will want to provide professional development, modeling, or perhaps peer

coaching. Authentic learning is key to increasing student engagement and deep understanding of

academic concepts.

0% 20% 40% 60% 80% 100%

6%

19%

6%

48%

32%

58%

52%

26%

61%

23%

42%

19% 6%

Work that requires students to read andunderstand content related to your

subject area.

Work that requires students to applyskills from your content area to

hypothetical problems or situations.

Work that requires students todemonstrate understanding of the core

ideas within your content area.

Work that is multidisciplinary andconnects skills and concepts from

multiple content areas.

Somewhat comfortable Comfortable Extremely comfortable Not comfortable

Figure D5-3: Teacher preparedness to scaffold conceptual learning.

The percentage of teachers indicating their comfort level in incorporating the following

types of assignments into their teaching and their students’ learning.

0% 20% 40% 60% 80% 100%

Work that requires students to usethe language and methods

professionals would use when dealingwith the content you teach.

Work that involves students inapplying concepts to real-world

problems.

Work that has an audience outsidethe classroom.

13%

16%

26%

26%

39%

26%

48%

39%

32%

13%

6%

16%

Not comfortable Somewhat comfortable Comfortable Extremely comfortable

Figure D5-4: Teacher preparedness to scaffold conceptual learning.

The percentage of teachers indicating their comfort level in incorporating the following

types of assignments into their teaching and their students’ learning.

17

The top chart on this page provides insights into how prepared teachers believe they are in their use of

assessment data and research to inform their decisions. The bottom chart asks about their preparedness

to assess technology-based student products and to use technology for assessment purposes.


Insert Figure AH here

stacked bar chart


Insert Figure AI here

stacked bar chart

Many teachers are not yet experienced in assessing student products that are multimedia based (e.g.,

animation, movies, audio files, simulations, etc.). As technology is integrated into curricula it is paramount

that teachers learn to do so against established standards of content, process, design, and purpose. On

the flip side, teachers need to become accomplished users of the technology to collect, manage, and

interpret a continuous stream of data to inform their instructional decisions. Use these charts to analyze

your staff’s current preparedness, and to set targets for improvement.

0% 20% 40% 60% 80% 100%

Use assessment data to make decisionsabout students and learning

Use research to make decisions aboutstudents and learning

Create and implement performance-basedassessments

Providing opportunities for student toself-assess based on a rubric or standard

7%

3%

7%

17%

23%

7%

10%

59%

53%

48%

50%

17%

20%

38%

40%

Not yet prepared Novice Intermediate Expert

Figure D5-5: Teacher preparedness in assessment.

The percentage of teachers indicating their preparedness to inform

the decisions and practices using data and research.

0% 20% 40% 60% 80% 100%

Use technology for analyzing student data

Assess student products created by usingtechnology

13% 27%

24%

50%

59%

10%

17%

Not yet prepared Novice Intermediate Expert

Figure D5-6:Teacher preparedness in assessment.

The percentage of teachers indicating their preparedness to inform

the decisions and practices using data, research, and technology.

18

Finally, the following two charts are the administrators’ perspectives on teacher preparedness across a

broad array of teaching strategies critical to 21st Century learning.


Insert Figure AK here

stacked bar chart


Insert Figure AL here

stacked bar chart

This dimension on teacher proficiency is one of the most critical. Readers are encouraged to use these

data to chart a course that leads to increased teacher preparedness to teach and learn in 21st Century

classrooms.

Figure D5-7: Teacher preparedness: 21st Century learning.

Percentage of administrators reporting on levels

of teacher preparedness related to 21st Century learning

Figure D5-8: Teacher preparedness: 21st Century learning.

Percentage of administrators reporting on levels

of teacher preparedness related to 21st Century learning

19

Dimension 6: Access and Infrastructure

The level of access to technology tools and the

robustness and reliability of the technology

infrastructure serve as critical foundations for

21st Century learning. The elements of this

dimension include:

· Range of Technology Tools. Are a wide

range of technology tools, software, and

environments available to support all

aspects of teaching and learning?

· Robust Infrastructure. Are the network and

technical infrastructure of the school

sufficient to provide seamless access to all

in the school community?

· Longitudinal Data System. Are systems in

place to provide all educators in the system

with seamless access to the data that they

need to support their professional decision

making?

· Technical Support. Is there adequate

technical support to provide timely

assistance to all users within the system?

· Technology-Ready Facilities. Is the school

building well suited to 21st Century teaching

and learning?

· Digital Learning Environments. Where

appropriate, are digital and virtual access to

learning opportunities available to all in the

school community?

· Administrative Processes and

Operations. Is technology leveraged to

ensure well-informed and efficient

administration at all levels of the school and

jurisdiction?

· Service Orientation. Are all staff with

responsibilities for infrastructure, technology

deployment, and technical support oriented

toward providing high-quality service? Do

they acknowledge the primacy of the

The top chart indicates the current status of the

province in this dimension. In order to provide

context to the numeric scores, frequency reports

from several of the survey questions are provided

on the following pages.

Dimension 6 Scores

Figure ??

aggregate

Figure ??

Pie chart


0 1 2 3 4 5 6 7 8

Administrators

Teachers

Aggregate

5.3

5.3

Dimension 6



Figure D6-2: The percentage of teachers indicating the

degree to which technical support is provided with

little or no wait-time.

Excellent 6.5%Good 22.6%Adequate 38.7%Very poor 22.6%Non-existent 9.7%

Total: 100.0%

20

Schools want to be sure that purchased technologies are valued and used by teachers in the classroom.

The two charts on this page provide insights into how administrators', across the province, perceive the

usefulness of various technologies to the teaching responsibilities of their teachers, in comparison to

current availability.


Insert Figure AO here

stacked bar chart


Insert Figure AP here

stacked bar chart

Figure D6-3: Instructional technologies.

Comparision of percentage of administrators' ratings of availability of technology tools

to their perception of the usefulness of tools.

Available = Adequate to Meet Needs or Ubiquitous; Useful = Useful or Very Useful.

Figure D6-4: Technology peripherals.

Comparision of percentage of administrators' ratings of availability of technology tools

to their perception of the usefulness of tools.

Available = Adequate to Meet Needs or Ubiquitous; Useful = Useful or Very Useful.

21

Schools also want to balance their investments in high-speed networks with investments in computers

linked to that network. Imbalances in the direction of the network results in inefficiencies and untapped

potential, while imbalances in the direction of computers result in frustrations due to slow and inadequate

network capacity.


Insert Figure AU here

stacked bar chart


Insert Figure AQ here

stacked bar chart

0% 20% 40% 60% 80% 100%

The Internet connection is fast and reliable

Technologies are physically placed formaximum convenience and effectiveness

Access to networked files and resources isconvenient and easy to understand

100%

61%

84%

39%

16%

TrueNot True

Figure D6-5: Infrastructure.

Percentage of teachers who reported on how each statement applied to their school.

Figure D6-6: Adequacy of level of technology access.

Percentage of administrators who reported in each accessibility category for each item.

22

As those balances are achieved, the school will want to ensure accessibility to online learning structures

including blogs, wikis, communication systems, online courses, etc.


Insert Figure AR here

stacked bar chart


Insert Figure AS here

stacked bar chart

Just as critical as the high-speed network and adequacy of computer access, is the responsiveness of

technical support available to teachers and administrators.

Figure D6-7: Accessibility to online learning structures.


Figure D6-8: Technical support for technology.


23

Support also includes scaffolding and building capacity of teachers to use technology in their professional

practice. The chart below provides a snapshot of current perceptions of administrators on the extent to

which their teachers' professional use of technology is supported.


Insert Figure AT here

stacked bar chart

Figure D6-9: Support for teacher's professional use of technology.


24

Dimension 7: Accountability

The accountability dimension provides a look at

the alignment between goals, assessments, and

results.

The indicators include:

· Accountability System Aligned to Vision.

Has the accountability system been

redesigned to ensure that the vision is

achieved within a prescribed timeframe?

Have policies been rewritten to ensure that

planning, resource allocation, time

investment, curriculum redesign,

professional development, and other

elements of the system are orchestrated to

advance the vision?

· Clarity, Transparency, and

Consequences. Do educators, students,

parents, and community members

understand what the vision is, why it is

important, and what it means to their

respective roles in schools? Do they

understand what the vision will look like if

achieved, the assessments used to monitor

progress toward the vision, and the

consequences associated with failure to

achieve incremental milestones toward the

vision?

· Comprehensive, Prioritized Funding.

Have the school and jurisdiction analyzed

the full cost of implementing the vision over

time and have they committed sufficient

funds in the short and long term in order to

achieve the vision within established

timeframes?

· Decision Making Informed by Data and

Research/Results. Is the data analysis, in

combination with research, appropriately

informed, and does it contribute to the

continuous improvement of the system?

· Results. Are jurisdiction and school making

progress toward their goals?

Dimension 7 Scores

Figure AV

aggregate

Figure AW

Pie chart



status of your school or jurisdiction in this

dimension. In order to provide context to the

numeric scores, frequency reports from several

of the survey questions are provided on the

following pages.

0 1 2 3 4 5 6 7 8

Administrators

Teachers

Aggregate

3.7

3.7

Dimension 7



Figure D7-2: Percentage of teachers reporting the

level of their agreement with the statement: Students

in my classroom are clear about how they need to

demonstrate their skill level with the targeted 21st

Century Skills.

Agree 25.8%Disagree 67.7%Strongly disagree 6.5%

Total: 100.0%

25

The frequent use of data to inform instructional practices is critical in meeting all students’ needs. This

chart provides perspectives on the importance of various assessments in informing classroom practices.


Insert Figure AX here

bar chart


Insert Figure A3 here

bar chart

The chart below provides data on the use of such assessments to track students’ attainment of 21st

Century Skills.

0% 20% 40% 60% 80% 100%

District wide assessment

School wide assessment

Provincial assessment

End of course tests

Periodic quizzes and tests

Performance assessments (rubric based)

Technology literacy assessment

13%

25%

23%

28%

37%

11%

Figure D7-3: Assessments informing classroom practices.

The percentage of teachers reporting that the following types of assessments

were “important” or “key” factors in informing their instructional decisions.

0% 20% 40% 60% 80% 100%

Jurisdiction-wide assessment

School-wide assessment

End of course tests

Periodic quizzes and tests

Performance assessments (rubric based)

Technology literacy assessment

Classroom observations

Student self-assessment

Analysis of student products

Student peer reviews

We don’t assess 21st Century Skills

Figure D7-4: Assessments of 21st Century learning.

Percentage of administrators who indicated the following methods

were routinely used to assess student attainment of 21st Century Skills.

26

Student outcomes are ultimately the focus on 21st Century learning and effective technology uses. This

final chart provides the teachers’ perspective on the current level of student expertise with 21st Century

Skills.


Insert Figure A2 here

stacked bar chart

School and jurisdiction leaders are encouraged to identify those skills targeted in their short and

long-term goals and set targets to increase the percentage of students in the Intermediate and Expert

0% 20% 40% 60% 80% 100%

3%

6%

3%

3%

41%

37%

60%

42%

23%

32%

40%

59%

43%

55%

60%

30%

48%

68%

55%

57%

38%

47%

3%

10%

3%

6%

13%

3%

10%

Global Awareness


Critical Thinking

Self-direction

Interactive Communication

Teaming & Collaboration

Personal & Social Responsibility

Effective Use of Real-world Tools

Productivity

No skillNoviceIntermediateExpert

Figure D7-5: Percentage of teachers' rating of student level of expertise in 21st Century Skills.

27

Student Outcomes/Student Perspectivesby Emerge Research Team: Metiri Group and University of Calgary

Ultimately, the mission of schools is to ensure that students acquire the skills, knowledge, processes, and

dispositions that enable them to meet or exceed state and local learning standards. For increasing

numbers of schools those standards include academic and 21st Century learning goals, goals that will

prepare them to thrive in today’s global, high tech society and workplace.

Metiri's Student Outcomes/Student Perspectives report provides findings related to students, including:

Student Engagement

Student engagement represents actual current levels of engagement as reported by individual students.

Students in your school or jurisdiction completed Metiri Group’s Student Engagement Inventory online.

The data from those surveys were then analyzed and aggregated to report the percentage of students at

five different levels of engagement: Intrinsically Engaged, Tactically Engaged, Compliant, Withdrawn, or

Defiant.

Classroom Structures to Engage Students

This section looks at student perceptions of conditions in the school that have been linked by research to

increases in student engagement. The Metiri Group survey, Classroom Structures that Engage Students

(CSES), asked students in your school or jurisdiction to share their perceptions on characteristics of

learning environments that research indicates are directly related to the level of student engagement in

learning. These characteristics include choice, structure, diversity, intellectual safety, clarity, affiliation,

and authenticity. The student perceptions on these classroom structures were then clustered into three

domains (i.e., content, process, and product) for reporting purposes.

28

Student Engagement

Definition of Student Engagement

Degree to which students are actively

pursuing deep learning related to

established standards.

The measure of student engagement

comprises:

· Cognitive Engagement. A student’s

Investment in the effort required to

comprehend complex ideas and master

difficult skills.

· Behavioral Engagement. A student’s

participation in academic, social, and

extracurricular activities.

· Social/Emotional Engagement. A

student’s interdependence with classmates,

academics, teachers, and school.

Based on: Fredricks, J. A., Blumenfeld, P.

C., & Paris, A. H. (2004).

Are your students genuinely interested in the

topics they are studying? Are they highly

motivated and committed to learn? Do they

persevere when challenged with complex

topics? Are they achieving deep, authentic

learning? Are they self-directed? Answering

“yes” to these questions would suggest that your

students are engaged learners.

Unfortunately, many teachers are answering

“no” to these questions. Increasingly, schools

are finding that students feel alientated from

their schools, perceiving them as boring or

irrelevant to their lives. As a result, many

students do just enough to get by, while the

most disenfranchised simply drop out. Across

the U.S. the drop out rate is 30%, and in some

U.S. urban centers the drop out rate is reaching

50% or more. These students are not being

prepared to compete and excel in the 21st

Century.

Teachers’ interest in engagement is often driven

by the need to ameliorate low levels of academic

performance, inappropriate classroom

behaviors, and/or high numbers of drop outs. In

fact, engagement is influential in all students’

learning trajectories. Emergent research

identifies student engagement in learning as one

of the most powerful factors affecting

achievement of students at every ability level.

What exactly is student engagement? In his

book, Student Achievement in American

Secondary Schools, Fred Newmann states that

engaged learners make a “…psychological

investment in learning. They try hard to learn

what school offers. They take pride not simply in

earning the formal indicators of success

(grades), but in understanding the material and

incorporating or internalizing it in their lives” (p.

2).

For the purposes of this evaluation report,

student engagement in learning is the active

pursuit of deep learning to accomplish

established standards. Student engagement

comprises three scales: cognitive, behavioral,

and social/emotional engagement, as described

in the sidebar to the right.

Each scale is important in its own right, but the

three are also interdependent. While it is

obvious that the cognitive advances academic

achievement, emergent research also indicates

strong correlations between how emotionally

and socially engaged students are with teachers

and classmates, and how well they do

academically and whether they graduate.

Levels of Student Engagement

The Student Engagement Survey- Part B,

created by Metiri, includes a series of questions

aligned to the cognitive, behavioral, and

social/emotional elements of the definition. This

report, which is generated from your survey

data, provides your school/jurisdictionwith an

overall student engagement score as well as a

score on each of the scales (i.e., cognitive,

behavioral, and social/emotional).

Building off the research of recent engagement

theorists, a taxonomy of student engagement

29

levels was developed to distinguish different

types of engagement as listed below.

· Engaged

· Tactically Engaged

· Compliant

· Withdrawn

· Defiant

· Indeterminate

Using this taxonomy, one would expect an

engaged student to respond positively to “I like

anything I learn about in school.” Similarly, a

withdrawn student would rate a high level of

agreement with the statement “I do not go to

school activities after school. I like to leave

school as soon as I can.” Nine items were

written to reflect each of the five levels of

engagement for a total of 45 items.

Table SE-1 presents a few sample items by

engagement scale. The engagement level of

students was established by locating the level

with the highest mean across the 9 questions

within that category. Students whose responses

were mixed or were <2.5 in all categories were

classified as indeterminate. Table SE-2 shows

how the engagement levels differ in terms of

commitment and attention.

Table SE-1: Examples of statements written for the levels of student engagement

Engaged Tactical Compliant Withdrawn Defiant

After school, I go to school activities so I can be a good student.

I think that kids who do after school activities get better grades.

I always know if I did a good job on my schoolwork. When I don't do as well as I like, I think about what I can change for next t ime.

I keep a list of what I need to do to get a good grade.

Learning in school is important to me because my parents want me to get good grades.

I do just enough work in school to get by.

After I f inish schoolwork, I usually do not check over my work. I'm just glad to be done!

If I do my schoolwork, I do not care about the grades I get.

I do not go to school activit ies after school. I like to leave school as soon as I can.

I would never participate in an extracurricular activity in school.

Most of my schoolwork is junk. No one can make me do it.

High

Attention Low

Attentio n No

Atten tion Hig h

Commitment L ow

Co mmitment No

Commitmen t Diverted Attention

Engaged

Tactical

Compliant

Withdrawn

Defiant

Table SE-2: Attention and Commitment by engagement levels

30

The types of engagement are defined in more

detail below, followed by graphs of the

percentage of students in each level.

Characteristics: Engaged Learners

· Student sees the activity as personally

meaningful.

· The student's level of interest is

sufficiently high that he persists in the

face of difficulty.

· The student finds the task sufficiently

challenging that he believes he will

accomplish something of worth by doing

it.

· The student's emphasis is on optimum

performance and on "getting it right."

Characteristics: Tactically Engaged Learners

· The official reason for the work is not

the reason the student does the work-

-she substitutes her own goals for the

goals of the work.

· The substituted goals are instrumental-

-grades, class rank, college acceptance,

parental approval.

· The focus is on what it takes to get the

desired personal outcome rather than

on the nature of the task itself-

-satisfactions are extrinsic.

· If the task doesn’t promise to meet the

extrinsic goal, the student will abandon

it.

Characteristics: Compliant Students

· The work has no meaning to the student

and is not connected to what does have

meaning.

· There are no substitute goals for the

student.

· The student seeks to avoid either

confrontation or approbation.

· The emphasis is on minimums and exit

requirements: “What do I have to do to

get this over and get out?”

Characteristics: Withdrawn Students

· The student is disengaged from current

classroom activities and goals. The

student is thinking about other things or

is emotionally withdrawn from the

action.

· The student rejects both the official

goals and the official means of

achieving the goals.

· The student feels unable to do what is

being asked, or is uncertain about what

is being asked.

Characteristics: Defiant

· The student is disengaged from current

classroom activities and goals.

· The student is actively engaged in

another agenda.

· The student creates her own means

and her own goals.

· The student’s rebellion is usually seen

in acting out-and often in encouraging

others to rebel.

Figure SE-1: Overall Engagement

Baseline Data

The chart below provides data on engagement in

your schools.

0%

20%

40%

60%

80%

100%

42% 44%

12%

2%

46%

38%

8%4% 2% 3%

Engaged TacticallyEngaged

Compliant Withdrawn Defiant Indeterminate

Elementary Secondary

Elementary N=59; Secondary N=192

31

Figure SE-2: I really enjoy learning so I don’t consider it

“work,” even when it’s challenging.

Intrinsic Engagement

Figure SE-3: I focus on my schoolwork when studying and I

am constantly checking to make sure I understand the

material.

Tactical Engagement

Figure SE-4: Some of the topics I learn about in school

aren't that interesting but I still pay attention so I can get a

good grade.

Figure SE-5: Learning at school is really important to me

because it will help me to get into a good college and find a

good job when I finish school.

The following charts provide response

frequencies for individual questions. These

charts provide detailed information as to how

students responded to individual questions.

Note: If your jurisdiction used both the

elementary and secondary surveys, then the

original question asked of secondary students is

displayed. A similar question was asked of

elementary students, but was simplified to

decrease the reading level to Grade 3.

0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me

Neither likeor not like

me

A little likeme

A lot like me

1%4%

9%

29%

57%

10%

22%

68%


0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

14%10%

15%

34%

27%

9% 11%

19%

40%

20%



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

2% 3%

19%

44%

32%

2%

10%

18%

45%

25%



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

7%

29%

15%

31%

19%

9%

23% 26%

34%

8%



32

Figure SE-6: When I complete my assignments I tend not to

go back and check over my work. I'm just glad to be done!

Compliant Engagement

Figure SE-8: I am not interested in the topics I learn about in

school. Usually I think about other things, or daydream.

Withdrawn Engagement

Figure SE-7: I just do enough on each assignment to get the

grades that keep my teachers and parents off my back.

Figure SE-9: I don't pay any attention to the school rules,

and I hope no one pays any attention to me.

0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

34% 32%

15% 14%

5%

30% 32%

22%

10%6%



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

44%

25%

17%

10%

3%

31% 34%

18%13%

5%



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

37%

19%24%

12%8%

19%26% 25%

20%

10%



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

19%24% 25%

22%

10%10%

32%

18%

27%

13%



33

Figure SE-10: I do not complete my assignments and have

no intention of doing so.

Defiant Engagement

Figure SE-11: I don't pay much attention to anything we

study. It's not worth my time.

The set of data on student engagement was

also analyzed to determine the degree to

cognitive, behavioral, and social/emotional

engagement reported by students.

0

1

2

3

4

5

Behavioral Emotional Cognitive

3.9

4.3

3.93.8

4.2

3.7


Elementary N = 59; Secondary N = 192

Figure SE-12: Engagement types

Types of Engagement

Cognitive Engagement

Figure SE-13: I keep a list of exactly what is needed to get a

high grade and mentally check things off as I complete them.

Figure SE-14: I do not plan time for studying or completing

assignments.

Again, frequency responses from individual

questions are provided as additional insight into

each of these three areas.

0%

20%

40%

60%

80%

100%

49%

31%

10% 8%

2%

31% 29%

18%13% 10%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

20% 19% 20% 22%19%17%

27%22%

26%

8%



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

75%

17%

5% 3%

61%

20%

13%

3% 2%



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

76%

10% 10%

2% 2%

65%

15%12%

7%

1%



34

Figure SE-15: I spend enough time in this school just doing

the basics. I do not participate in extra-

curricular activities that are not required, unless my parents

really push me.

Figure SE-16: I don't really think about following rules and

acting positively. It's just the way I am.

Behavioral Engagement

Social/Emotional Engagement

Figure SE-17: It's important to me that the work I do

represents my best effort and helps me grow.

Figure SE-18: School isn't all important to me. I am not

interested in what we learn.

The Emerge Research Team recommends that

the schools review this data set and set targets

for improvement related to the percentage of

students in the engaged categories. The detail in

the charts above should provide insights as the

type of strategies that will be required to meet

those targets.

Bangert-Drowns, R. L. (2001). A taxonomy of student

engagement with educational software: An

exploration of literate thinking with electronic

text. Journal of Educational Computing

Research, 24(3), 213-234.

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H.

(2004). School engagement: Potential of the

concept, state of the evidence. Review of

Educational Research, 74(1), 59-109.

Schlechty, P. (2002). Working on the work. San

Francisco: Jossey-Bass.

Sources

0%

20%

40%

60%

80%

100%

81%

7% 5%2%

5%

70%

14%10%

5%1%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

2% 3%

18%

38% 39%

2%

10%

32%

56%

Secondary Elementary


0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

46%

32%

15%

3% 3%

40%

28%

14% 11%7%



0%

20%

40%

60%

80%

100%

Not at alllike me

Not reallylike me


me

A little likeme

A lot like me

36%

19%

36%

7%3%

30% 30%

21%

12%7%



35

Classroom Structures That Engage Students (CSES)

Some learning environments are more effective

than others in scaffolding deep learning in

students. Based on emergent research (see

reference list at end of section), The Emerge

Research Team identified those classroom

structures and surveyed students to determine

which of those structures were evident in the

classrooms they attended.

Students were asked to share their perspectives

on whether the classrooms they attend offer

choice, structure, diversity, collaborative

environment, and clarity. In addition, students

were asked whether the classroom structures

interface with content draws on their prior

knowledge, offers them opportunities to

collaborate with colleagues, and provides them

with avenues for fully demonstrating their

acquired knowledge and skills.

Emergent research on highly qualified teachers,

highly effective teaching, differentiation of

instruction, engagement, and response to

intervention all emphasize how the right

classroom structures can engage students in

deep learning (Marzano, 2007). The classroom

that introduces content in ways that trigger

student interest, challenge students to think, cue

prior knowledge, provide relevancy, afford

choice, offer clear standards, formalize

collaboration, and ensure intellectual safety,

lead to deep learning by children and

adolescents.

Background

Scales

Three major elements have emerged from the

research on the effectiveness of classroom

structures in advancing deep learning through

student engagement related to Content,

Process, and Product.

Content refers to student opportunities to

engage with academic subjects in ways that

result in deep understanding of concepts,

principles, and context. Process refers to the

learning activities through which the student is

able to make sense of, or master, the content.

The element of Product engagement refers to

the structures that enable students to rehearse,

apply, extend, and demonstrate what he/she

learned through an assignment.

The Engagement Survey- Part A, created by

Metiri, was administered to gauge student

perspectives on the existence of and quality of

classroom structures that could lead to their

engagement in learning. The survey was guided

by the work of the researchers in the reference

list. The survey asked students to rate 50

statements on classroom structures, i.e., 15

items on Product, 15 items on Process, and 20

items on Content, using a 5-point scale

anchored with 1=Completely False to

5=Completely True, with 3 as a neutral

mid-point.Figure CE-1: Overall classroom structures the engage

students (CSES) score (scale 1-5)

0

1

2

3

4

5


3.4 3.4


NOTE: The Overall Score on the Classroom

Structures That Engage Students (CSES)

reflects the average of the Product, Process,

and Content scales.

36

Figure CE-2: Product, Process, Content and Overall Engagement scores by level

0

1

2

3

4

5

Product Process Content Total ClassroomEngagement Score

3.9

3.2 3.13.4

3.8

3.2 3.33.4



Student Rating Cut Points Score Level Definition

1. Completely False

1.0 <= x <= 1.9 No classroom structures that engage students

2. A Little False 2.0<= x <= 2.4 Moderately low number/quality of classroom structures that engage students

3. Neither True nor False 2.5.<= x <= 3.4 Neutral

4. A Little True 3.5<= x <= 3.9 Moderately high number/quality of classroom structures that engage students

5. Completely True 4.0 <= x <= 5.0 High number/quality of structures that engage students

Table CE-1. The CSES Survey Cut Points and Definitions

Higher scores reflect higher evidence/quality of

classroom structures that engage students in

learning, whereas low scores reflect less

evidence/quality of such classroom structures.

Scores greater than or equal to 3.5 imply that

students perceive high to moderate evidence of

classroom structures to engage them in

learning. Scores lower than 3.5 indicate that

students reported either low or moderately low

evidence/quality of classroom structures that

engage them (See Table above.) The mean

scores by school level are presented above,

followed by a more detailed analysis of each

scale.

Prior to calculation of factor scale scores, the

student responses to reversed or negatively

worded items were adjusted appropriately.

Cronbach’s alpha internal consistency

reliabilities for the Product, Process, and

Content scales were 0.82, 0.58, and 0.83,

respectively. The alpha for combined scales was

0.903, indicating high internal consistency for

the Classroom Engagement Survey.

37

In order to provide more detailed insight into

these scores, frequencies are reported below for

sample questions from each of the scales.

Figure CE-4. The work that I do in my classes seems like the

same kind of work that I might do for a job one day.

Product Scale

Product refers to the projects that require the

student to rehearse, apply, and extend what he

or she has learned in a unit. Some examples of

how teachers promote product engagement

include giving students’ a choice of how to

express required learning (e.g., create a movie

or interview an expert) or encouraging students

to create their own product assignments as long

as the assignments contain specific elements.

Another method to increase product

engagement includes using rubrics that align

with and extend students’ varied skills levels.

Therefore, a high score on the Product scale

would indicate that students value class

assignments and perceive them to be

meaningful, they understand the standards by

which the product will be assessed, and there is

value to the product beyond the classroom.

A Product scale was computed as the mean of

15 items measured using the five-point

Completely false-Completely true Likert-type

response scale. Two of the items included in the

Product scale are presented below by school

level.

Sample Product Items

Figure CE-3. My teachers tell us in advance exactly how a

product, project, or other assignment will be graded.

Process Scale

Process refers to the activities in which the

student engages in order to make sense of or

master the content. A high score on this scale

would indicate that students perceive the

classroom as an environment in which they can

learn through intellectual risk taking without fear

of ridicule, they can work interactively and

interdependently with others, and they perceive

those activities and tasks to be meaningful, as

do persons of importance to them.

The following graphs present two of the

individual items in the Process scale by

academic level.

Sample Process Items

Figure CE-5. In my classes, my teacher likes us to have

other students read or view our work.

0%

20%

40%

60%

80%

100%

3%

17% 19%

52%

9%10%6%

18%

42%

25%

Completelyfalse

A little false Neither trueor false

A little true Completelytrue



0%

20%

40%

60%

80%

100%

19%16%

36%

25%

5%11%

22%28% 28%

11%

Completelyfalse





0%

20%

40%

60%

80%

100%

3% 5%9%

22%

61%

3% 6%10%

31%

51%

Completelyfalse





38

Figure CE-6. Often my teachers give assignments

that allow us to work together in groups.Using the Results Formatively

Schools interested in improving their CSES

scores should review their data, identify gaps

between their current scores and their goals,

identify areas for improvement, and then

develop and implement a strategic improvement

plan based on research.

Sources

Appleton, J. J., Christenson, S. L., Kim, D., &

Reschly, A. L. (2006). Measuring cognitive

and psychological engagement: Validation of

the student engagement instrument. Journal

of School Psychology, 44, 427-445.

Bangert-Drowns, R. L. (2001). A taxonomy of student

engagement with educational software: An

exploration of literate thinking with electronic

text. Journal of Educational Computing

Research, 24(3), 213-234.

Dweck, C. S. (2006). Mindset. New York: Random

House.

Dweck, C. S., & Leggett, E. L. (1988). A

social-cognitive approach to motivation and

personality. Psychological Review, 95(2),

256-273.

Fredricks, J., Blumenfeld, P., Friedel, J., & Paris, A.

(2003, March). School engagement. Paper

presented at the Indicators of Positive

Development Conference, Washington, DC.

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H.

(2004). School engagement: Potential of the

concept, state of the evidence. Review of

Educational Research, 74(1), 59-109.

Hoek, D., Terwel, J., & Eeden, P. (1997). Effects of

training in the use of social and cognitive

strategies: An intervention study in

secondary mathematics in co-operative

groups. Educational Research and

Evaluation, 3(4), 364-389.

Jimerson, S., Campos, E., & Greif, J. (2003). Toward

an understanding of definitions and

measures of school engagement and related

terms. The California School Psychologist, 8,

7-27.

Marzano, R. (2007). The art and science of teaching:

A comprehensive framework for effective

instruction. Alexandria, VA: ASCD.

Newmann, F. (1992). Student engagement and

achievement in American secondary

schools. New York: Teachers College Press.

Schlechty, P. (2002). Working on the work. San

Content Scale

Content refers to the material being presented,

or what the student needs to learn such as

concepts, principles, and skills. To score high on

the Content scale indicates that students find

the subject matter interesting and perceive it to

be relevant, important, and attainable. To

measure content engagement, students

responded to several statements. The following

charts present the results of two of the individual

items in the Content scale.

Figure CE-7. I get to be creative in my classes.

Sample Content Items

Figure CE-8. My teachers make uninteresting

subjects exciting to learn about.

0%

20%

40%

60%

80%

100%

Completelyfalse

A little false Neither true orfalse


14%8%

22%

33%

23%

9%14%

24%

31%

21%



0%

20%

40%

60%

80%

100%

Completelyfalse



6%

17%

28% 30%

19%

4%8% 11%

37% 39%



0%

20%

40%

60%

80%

100%

Completelyfalse



6%

23%28%

31%

11%9%14% 15%

33% 30%



39

seoul foreign schools all[1]

Education

21st century

21st century

stacked bar

21st century

21st century

scaffold conceptual

21st century

summary weighted