old saybrook systems analysis

Digital Learning for All, NOW!

Systems Reflection and Continuous Improvement Framework

Jonathan P. Costa, Sr.

© Corwin Press - 2011

A Systems Orientation

The principle of

85/15 © Corwin Press - 2011

Goals for Learning

Assessment & Data

Professional Support

Professional Evaluation

Curriculum & Communication

Teaching & Instruction

Resource Deployment

Leadership Focus


A Framework for Scoring


Level Description

0 - No Evidence The behavior or expected evidence is non-existent.

1 - Beginning

Some individual efforts or small groups at work, but no systemic evidence or process in place to support the behavior described in the indicator.

2 - Emergent

A system of some kind is in place, but its implementation is uneven and has yet to deliver meaningful changes in behavior or performance.

3 - Proficient

A system is in place and it is generally working. It is regularly creating evidence of meaningful changes in adult and student performance.

4 - Excellent

A system is in place and functioning effectively. There have been meaningful changes in student and adult performance and there is evidence that data is driving further improvements in the system.

Leadership & Focus

The degree to which leadership supports a

coherent movement toward a digital vision

of instruction.


1.1 Vision and Focus

A compelling vision for the attainment of 21st century skills for all students in a digital, one-to-one environment has been developed and implemented. This vision is focused on ensuring that all students have the most critical skills and attributes needed for life, learning, and work beyond school. This focused vision is grounded on the values of

meaningful engagement, collaboration, inquiry and higher-order thinking for all members of the learning community.

0 1 2 3 4

No statement of vision Complete and descriptive

Unarticulated beliefs Concise and compelling Disjointed or non-existent plans Interconnected and driving decisions Technical, mechanical – purchasing and deployment Focused on learning Dozens of skills to track Vital Few skills Exclusive Inclusive – Skills 21 for all

Rationale and Supporting Evidence: Areas of District Strength: Areas Where Growth is Needed:

If you rated yourself a 2 or lower:

◦ Immerse the staff in the literature related to digital learning and 21st century skills. Explore the realities of a digital future and what adequate preparation for this environment looks like.

◦ With that background knowledge, retreat for the purpose of vision building, transition to one-to-one planning, or some other process that will begin to forge a consensus on these critical issues.


1.2 Continuous Improvement

The district has a continuous improvement culture. New innovations and ideas are welcomed, expected, vetted, and implemented when appropriate. Creativity is valued so leadership, administrative, and technology systems are

flexible and adaptable, working well together by adjusting to new challenges and changing dynamics. Staff are focused on mission and outcomes rather than on position and history and welcome different approaches from any

source that might improve performance.

0 1 2 3 4

Stability Improvement Top down Any direction Cyclical review Continuous review Serendipitous improvements Systematic processes Past is future Future to be determined Who gets the credit? We are in this together

Rationale for Score: Areas of District Strength: Areas Where Growth is Needed:

If you rated 2 or lower:

◦ Identify and foster the elements of a continuous improvement culture. ◦ Use Force Field Analysis tools and CBAM study to gain understanding of change dynamics in your organization. ◦ Identify what you can do to encourage and systematically lead continuous improvement in any system and then be

explicit about modeling and celebrating those changes.


1.3 Systems Thinking and Alignment

The district takes a systems approach, focusing on how work is done, improving work processes and understanding the interconnected nature of the educational organization. The Board of Education has reviewed its policies and procedures across the district and made appropriate adjustments that shift the focus from print-era structures to

those that understand the impact of digital age learning.

0 1 2 3 4

Personalities Processes

Who get’s blame or credit? How the work get’s done

Just work harder Work smarter

Print structures Digital assumptions

Time as constant Learning as constant



◦ Create a systems map based on the eight systems identified in Digital Learning for All, NOW! Analyze how they work together to support a 21st century learning environment.

◦ Do a policy review to identify specific print-era assumptions. Propose adjustments that recognize the impact of the digital learning environment on the appropriate preparation of students for 21st century learning.


1.4 Culture and Modeling

The culture of your district and schools support and celebrate authentic use of digital tools for learning across the curriculum. These tools are used to advance both administrative and curricular applications. District, building, and instructional leaders have a shared vision of a 21st century school system, are conversant regarding the opportunity

and challenge of one-to-one learning, and are consistently working toward realizing its potential.

0 1 2 3 4

No systematic applications Systemic technology applications

Innovation derided Innovation celebrated

Haphazard examples Comprehensive use in all areas

No vision or inconsistent articulation Vision clear and consistent

Leader’s modeling absent Thoughtful and abundant modeling by leaders

Do As I say, Not as I do Come join me



◦ Look in the mirror – how have you modeled or not modeled these expectations?

◦ Initiate a Professional Learning Community process to explore the rationale for Digital Learning for All, NOW! that is described in Chapter One. Have administrative leaders work through the book’s reflection questions.

◦ Encourage and motivate leaders to demonstrate their ability to live the digital learning vision of the district.


Goals for Learning

The commitment you

have made to a small and focused number of

essential skills and learning outcomes.


2.1 Commitment to Rigor and Understanding

The culture of curriculum and supervision in your district supports a focus and value on deep learning, rigorous application, and long-term retention of key concepts. Mastery of a few is more important than coverage of many.

There is an explicit understanding that all content is not created equally and curriculum design represents this value.

0 1 2 3 4

Coverage Retention and understanding “Where are you?” “What are they learning?” Value on how much is taught Value on how much is learned No impact Drives decisions Pacing Guides Shared Evidence of Attainment



◦ Practice prioritization and research the impact of the Vital Few. ◦ Explore the foundations of Understanding by Design. ◦ Identify the CRITICAL learning outcomes that the district wants to support in each of the curricular areas and focus

district curriculum and instructional resources on the successful teaching and learning of just those outcomes.


2.2 Articulation of 21st Century Skills

The district has articulated and/or committed to a focused set of 21st century skills that define what students (and adults) should be able to do in a digital learning and work environment. Those standards have been articulated in such a way

that they are grounded in authentic contexts and are clearly everyone’s instructional responsibility.

0 1 2 3 4

No standards Clearly articulated evidence

No connection to classroom Standards drive decisions

More is more Focused and vital, True North 21

No review process Updated and reviewed frequently

Technology dependent Device neutral

Subject by subject Universal application

Rationale for Score:

Areas of District Strength: Areas Where Growth is Needed:


◦ Review the literature from NCREL, ISTE and the Partnership for 21st Century Skills. What does it mean to be prepared for life, learning and work in the 21st century?

◦ Explore what how different content topics are applied in authentic settings. What is the underlying skill set that enable the effective use of this content outside of the classroom?


Teaching & Instruction

The alignment of instructional practice

with the articulated goals for learning.


3.1 Purposeful Alignment

Teaching practices are focused and aligned with the goals for learning. From awareness to deep understanding, instructional design is purposefully directed to achieve the level of rigor, retention, and skill mastery called for in the curriculum and the teacher can articulate

how the strategies being employed are suited to obtain these outcomes.

0 1 2 3 4

Information delivery Progression of complexity

Lesson plan template Understanding by design

Activity as the focus Evidence as the focus

Point to point Connections between goals, activities and assessments

Focused on management Focused on outcomes

Time as constant Learning as constant




◦ Create discussion or work groups to explore the connections between goals for learning at various levels of complexity and the strategies most suited for helping students obtain that knowledge.

◦ Explore the Understanding by Design framework or any other schema that seeks to connect strategies with evidence of obtainment.


Preparing Students For a Knowledge Economy

Align Your Systems With Your Goals for Learning

Type of Assessment

Required

Subject Area Responsibilities

Everyone’s Responsibility

Content (Declarative)

Facts

Content Skills (Procedural)

Discrete Skills

21st Cent. Skills (Contextual)

Applied Understandings

Type of Knowledge

Desired

Type of Instruction Required

Lecture, video, films, assigned readings and

memory activities.

Classroom or textbook problems, experiments, discussions, practice and

repetition.

Complex projects, real time explorations, authentic and relevant

skill applications.

Amount of Time

Required

Discrete units, spiraled and predictable.

Ongoing, systemic and without a finite

or predictable end.

Discrete units, spiraled and predictable.

Recall & recognition based quizzes, tests,

and activities. Multiple choice, matching, etc.

(SAT/AP/Exams)

Checklists, analytic rubrics,

or other agreed upon skill standards

(AP/CMT/CAPT/Exams)

Holistic and, analytic rubrics,

or other agreed upon standards of rigor

(Portfolios, Exhibitions, Etc)

3.2 Higher-Order Engagement in Learning

Instructional practices support the movement toward meaningful, hands-on, student engagement in the learning process. This engagement is designed to prepare the learner for the appropriate accessing, processing, and

communication of knowledge and skills in authentic settings. Digital learning makes it possible for learners to engage in practices that encourage new ways of thinking, understanding, constructing and sharing knowledge.

0 1 2 3 4

Teacher centered Learner centered Demonstration of teacher competency Demonstrations of student competency High % of passive learning High % of active learning No authenticity Rigorous authenticity Application only Access, process, and communicate Factual recall only Higher order thinking The big test The big demonstration Memory challenge Intellectual challenge

Rationale for Score: Areas of District Strength: Areas Where Growth is Needed: If you rated 2 or lower:

◦ Create discussion or work groups to explore the benefits, processes required, and tools needed to create active learning environments.

◦ Build an effective professional development continuum to help professionals acquire the confidence and skills required to create active learning environments.


3.3 Rigor, Relevance and Self Direction The district is dedicated to rigorous standards of performance and allows students to take advantage of appropriate opportunities to take ownership of learning that meets these standards as well as student needs and interests both

within and outside of the prescribed curriculum of the district.

0 1 2 3 4

Classroom centered, fractured focus Life centered and coherent

Superficial Deep understanding

Textbook based Authentic applications

Classroom environments Varied environments

Extrinsic motivations Intrinsic motivations

Busy work Intense and purposeful

Teacher to student Student to life


Review assessments for appropriate levels of rigor and challenge.

Review the instructional planning template in Chapter 5. How can assignment RAFTs help build a framework to increase self-direction among learners in the existing curriculum?

Identify criteria for allowing students to engage in self-directed learning opportunities both within and outside of the formal curriculum.



Assessment & Data

How well the accountability systems

align with the stated goals for learning and

inform continuous improvement of

teaching and performance.


4.1 Shared & Reliable Assessments

The district has constructed shared and reliable standards of rigor for the mastery of 21st century skills. These standards are used by the entire community to assess student performances and are tied to observable and measurable

evidence. Practice and collaboration in their construction and refinement make it possible for faculty to apply these standards consistently in different contexts.

0 1 2 3 4

No shared standards Community wide sharing of standards

No connection to tasks Tasks are tied to standards

No observable or measurable indicators Observable and measurable indicators

Little consistency Reliable across the platform

Individual department interpretations Community interpretations




◦ Build a process where you can seek agreement on a vital few observable 21st century skills. What does success look like in these areas? Start building rubrics that test these ideas. Build and refine those rubrics based on that feedback.

◦ Build student performance tasks designed to create work aligned with your standards. Work in groups to score and compare assessments of quality. Use this feedback to refine your standards.


4.2 Deliverables and Evidence

Your assessment processes demand that specific products, performances, or standards of performance be met and measured in a logical continuum before credit or promotion is granted for the learner. This structure is systemic and

rigorous and demonstrates that mastery of content and skills are the determining factors in credit acquisition and clearly documents the growth of student performance over time.

0 1 2 3 4

Nothing defined Clearly marked progress

Seat time Demonstrated performance

Every teacher has a different standard Credit or promotion tied in to deliverables and benchmarks

One pathway One standard, many pathways

Snapshots in time Demonstrated growth over time




◦ Explore the current systems of assessment used in your district. Create a series of benchmarks of performance that can be tracked over time.

◦ Create and implement systems or practices that build a sense of shared accountability for student performance in your district.


4.3 Timely and Transparent Data

Your assessment processes enable the sharing of data regarding student performance in priority goal areas. Students and parents know how performance matches to standards, what areas of strength are, and where growth is needed. Generalized performance/growth data is available to all stakeholders and is transparently sortable by skill or teacher.

0 1 2 3 4

No system Electronic storage

Priorities not evident Value what you measure

Effort required Easy to access and interpret

Buried in the layers Dashboard

Generalized Skill and standard specific

Report cards Real time




◦ Are there ways your current data collection system can be focused on priority skill standards for 21st century success? How are you current data collection processes aligned or not aligned with those priorities?

◦ What is the data culture in your building? Explore Nancy Love’s work on creating a data-based improvement culture.


Curriculum & Communication

How curriculum resources support

instruction and inform constituents of the goals and plans for

learning.


5.1 Leading to Learn

Curriculum resources are focused on a manageable number of priority learning goals and have an identified set of performance tasks and/or teaching strategies specific to each. These strategies and resources are clearly

connected to the goal at hand and provide teachers with multiple pathways for leading students to acquire to goals involved for each segment.

0 1 2 3 4

Teacher by teacher Community resources Lists of goals Focused authentic tasks One learning style Variety of styles No sharing or support Sharing and support “Its mine” “Its ours” Curriculum use by inspection Curriculum use by choice



◦ Prioritize learning goals and focus curriculum work on strategies and processes. ◦ Given your Results and Assessments, decide what’s working and start compiling strategies that reinforce that

success. Be sure to include: 1. Purpose 2. Process 3. Materials

◦ Explore resources that will help to broaden your base of instructional options. ◦ Focus professional development on the creation of performance tasks and strategies that align with 21st century skill

development and acquisition.


5.2 Resource Rich

The curriculum in the district is wholly digital and is loaded with tools that support every level of the instructional program. Courses are broken into units and lessons with a variety of instructional resources (tasks, activities, materials),

assessments, and data collection options. There are enough resources available to teachers to make the curriculum a compelling destination capable of voluntarily shaping and driving instruction and assessment.

0 1 2 3 4

Manuals Online

List of goals only Resource rich

Inspection Used by choice

No connection to classroom practice Drives and supports classroom practice




◦ Explore the changing nature of curriculum and instructional resources in a digital environment.

◦ Explore electronic storage options for curriculum resources.

◦ Start by prioritizing goals and then collecting and posting instructional resources.


5.3 Transparent and Available

Curriculum resources in the district are openly shared and communicated to the parents and constituents. There is a culture of openness and an understanding that the more parents know about what is happening and what should be happening with their

students the more able they are to support it. Teachers communicate with parents and performance data is also transparent across the system.

0 1 2 3 4

Unknown in community Parent and public access

Hidden from sight Transparent

Jargon Accessible language

Us vs. them Parents as partners

Report cards Ongoing assessment data

Unapproachable Multiple communication pathways




◦ Explore the culture and beliefs around parent participation – review the data about how parent participation impacts student performance.

◦ Honestly discuss strategies for communicating and meaningfully engaging parents in an Information Age.


Professional Support

How the culture of professional learning will support the shift

to a truly digital learning environment.


6.1 Aligned and Purposeful

The formal professional learning program in your district is purposefully aligned with the key priority learning goals in the district. There is coherence and focus which clearly supports the attainment of 21st century skills for both students and adults in the learning community. There are a variety of options but always a central unifying

theme that demonstrates a systemic approach to professional learning.

0 1 2 3 4

Irregular/Non-existent Consistent opportunities

Haphazard Integrated with goals

Fill the time Purposeful and focused

Why are we doing this? Value and outcomes are clear

Take it or leave it Self-directed learning options

Few opportunities for learning Many opportunities for learning




◦ Examine the structures that drive the planning and selection of professional learning. What kind of systemic changes can be made to ensure the coherence and focus of what is offered so the choices are consistent with systems and Pareto Principle thinking?


6.2 Multiple Pathways

The professional learning culture expects and celebrates the importance and role of self-directed learning for the long-term health of the organization. Beyond the formal program, there is abundant evidence of professionals exploring, learning and sharing their progress with peers. This self-directed dynamic helps power continuous

improvement and the establishment of a true learning organization.

0 1 2 3 4

Three days a year Just in time

Haphazard Integrated with goals

Always the audience Internal resources

That’s their job Learning is everyone’s responsibility

Not in the contract Whatever it takes

Why don’t they… Why don’t I…




◦ Examine how individualized personal learning is either recognized or supported within the professional learning plan of the district.

◦ Determine how to best model, encourage, and reward the efforts of individuals to bring external expertise and resources into the district.


Professional Evaluation

How the evaluation process of professional

staff reflects the most critical elements of 21st

century learning success.


7.1 Engagement and Rigor

The focus of the professional evaluation tools and process are grounded on the value of meaningful student engagement in rigorous and relevant 21st century skill learning activities. Students working to employ these skills

is the ultimate goal of learning for life and work beyond school and should be the most important factor in determining evaluative professional feedback for both teachers and administrators in the system.

0 1 2 3 4

No underlying rationale Coherent focus

Everything under the sun Vital few

Unclear definitions & standards Clearly articulated standards

Surprise! Predictable and known

Diffused Laser




◦ Reexamine the value of the Pareto Principle and the law of the Vital Few.

◦ Collaboratively define the indicators of effective meaningful engagement in higher-order 21st century skills.

◦ Work to create measures and observable evidence claims that support the presence of these items in an instructional environment.


7.2 Student Performances

On the most fundamental level, student performance is impacted by the quality of instruction received and instruction is impacted by the quality of leadership and support received. A valid professional evaluation model must recognize these two realities and incorporate data and evidence of these factors as part of the process of

feedback, growth and accountability.

0 1 2 3 4 Adult action only Student performance included

Perception Data

Gotcha Growth

Teachers only Teachers and administrators

Paranoia Professional accountability




◦ Explore evaluation models that have successfully integrated student performance data as a consideration factor.

◦ Discuss and examine the connections between leadership, instruction and student performance. How are they linked and what evidence can we collect to demonstrate those connections?


Resource Deployment

The degree to which the district allocates it

resources to support the shift to a truly

digital learning environment.


8.1 Mission Alignment

There is evidence throughout the budget development and approval process that the district’s mission of 21st century skill preparation is driving the prioritization and focus of resource deployment throughout the district.

0 1 2 3 4 Historic patterns Current realities and future goals

Superficial Deep and impactful

Loudest voice Most important priority

Behind closed doors Out in the open

We have always… We need to start…




◦ Map each district cost center and analyze budget requests through a mission alignment lens.

◦ Explore capital and budget requests through an analysis criteria tied to digital learning and 21st century skill attainment.


8.2 Systems Thinking

In a systems thinking budget development process, there is explicit recognition of how resource deployment impacts interrelated systems. As such, cost center allocations are not made in isolation of one another but rather taken as whole. There is give and take recognizing that at times helping the whole is more important that getting

what you can for your own concern.

0 1 2 3 4 Historic patterns Current realities and future goals

Superficial Deep and impactful

Part by part Whole to part

What can I get? What do we need to do?

We have always… We need to start…




◦ Experience resource sharing scenarios such as Fishbanks which demonstrate how individualized budgets and fights over finite resources damage mission coherence.
