copyright © 2000. sri international. all rights reserved. session 6 the unfinished revolution - ii...

Copyright © 2000. SRI International. All rights reserved.

Session 6The Unfinished Revolution - II

Session 6The Unfinished Revolution - II

Networked Improvement Communities in Education

Roy Pea

Bootstrap Colloquium

Stanford University, Feb 10, 2000

Networked Improvement Communities in Education

Roy Pea

Bootstrap Colloquium

Stanford University, Feb 10, 2000


Center for Technology in Learning

Center for Technology in Learning

MISSION:

Improving learning and teaching through innovation and inquiry in computing and communications

MISSION:

Improving learning and teaching through innovation and inquiry in computing and communications

http://www.sri.com/policy/ctl/


How does CTL do this?How does CTL do this?• Advance theory and

research on effective learning and teaching.

• Innovative design, use, assessment of interactive learning environments.

• In short, learning sciences and engineering.

• Advance theory and research on effective learning and teaching.

• Innovative design, use, assessment of interactive learning environments.

• In short, learning sciences and engineering.


What’s the need?What’s the need?

• Education in the U.S. is decentralized and fragmented

• Education improvement efforts are isolated and incremental

• Technology can leverage the knowledge “in the system” in revolutionary ways

• Education in the U.S. is decentralized and fragmented

• Education improvement efforts are isolated and incremental

• Technology can leverage the knowledge “in the system” in revolutionary ways


A “powerful idea” for CTL...A “powerful idea” for CTL...

Networked Improvement Community (NIC)

• …is a coalition of organizations each engaged in a similar improvement process

• ... networks these organizations and crafts new mechanisms for improving their isolated improvement processes

• ...builds value and leverages knowledge for distributed communities

• Thanks to Douglas Engelbart for fruitful discussions about NICs

Networked Improvement Community (NIC)

• …is a coalition of organizations each engaged in a similar improvement process

• ... networks these organizations and crafts new mechanisms for improving their isolated improvement processes

• ...builds value and leverages knowledge for distributed communities

• Thanks to Douglas Engelbart for fruitful discussions about NICs


The Many NICs of CTLThe Many NICs of CTL

PROJECT TOPIC SCOPE

TAPPED IN On-line teacher professionaldevelopment institute

$1.7M (4 yrs),spurs Minerva spinoff

CILT Virtual center: K-14 learningtechnologies R&D

$6M (4 yrs)plus industry sponsors

PALS Performance assessmentsfor learning science

$1.2M (3 yrs)

OERL On-line library of educationproject evaluation resources

$1.75M (7 yrs)

ESCOT Interoperable mathematicseducational components

$2M (2 yrs)


Sponsors:

An Education Community-of-Practice Model for Scalable, Sustainable Teacher Professional Development

http://www.tappedin.org

Leader: Mark SchlagerContact: [email protected]

TM


• What? Web-based Virtual Conference Center

• Who? A growing community of over 7000 K-16 education professionals, 15 partner organizations and agencies, and scores of small groups who create on-line learning experiences from any computer, anytime.

• Multi-user virtual environment (MUVE)

• Synchronous and asynchronous communication

• Storing, sharing of Websites and documents (permanence)

• Community-wide activities and support services

TAPPED IN Overview TM


Private

Public

Pepperdine School of Education

Student Activities Center

TAPPED INBuilding

Gathering Place: TAPPED IN CampusTM


TM

Membership Growth 2/97 - 8/99

0

1000

2000

3000

4000

5000

6000

7000

Aug-97Oct-97 Dec-97 Feb-98 Apr-98 Jun-98

Aug-98Oct-98 Dec-98 Feb-99 Apr-99 Jun-99

Aug-99

Newsletter

started

After School Online

started

1998 Summer

Workshops

1999 Summer

Workshops

1997 1998 1999 Jan-007000+


University Schools of Education Pepperdine University (PT3) California State Univ., Sacramento University of Wisconsin (NSF) George Mason University

Local and State Education Agencies New Haven Unified School District, California (PT3, BTSA) Joint Venture: Silicon Valley 21st Century Education Initiative (TICG) Kentucky Department of Education Los Angeles County Office of Education TFL, TEAMS, LAAMP (NSF)

Teacher/Librarian Professional Development Organizations and Websites American Association of School Librarians Institute for Research on Learning MMAP program (NSF) Lawrence Hall of Science, University of California, Berkeley (NSF) Math Forum Website at Swarthmore College (NSF)

University Schools of Education Pepperdine University (PT3) California State Univ., Sacramento University of Wisconsin (NSF) George Mason University

Local and State Education Agencies New Haven Unified School District, California (PT3, BTSA) Joint Venture: Silicon Valley 21st Century Education Initiative (TICG) Kentucky Department of Education Los Angeles County Office of Education TFL, TEAMS, LAAMP (NSF)

Teacher/Librarian Professional Development Organizations and Websites American Association of School Librarians Institute for Research on Learning MMAP program (NSF) Lawrence Hall of Science, University of California, Berkeley (NSF) Math Forum Website at Swarthmore College (NSF)

TMSome TAPPED IN users


What can educators do on-line?

Educators engage in professionally relevant activities:• Planning and conducting projects with colleagues and students• Leading or joining topical discussions• Conducting and attending courses• Finding resources, experts, and new colleagues• Serving as resources for other educators• Trying out new ideas in a safe supportive environment• Using the Internet in new ways

Extend capacity to interact online through:

• Real-time and e-mail Help Desk

• After School Online discussion sessions

• Free offices for individuals and small groups

• Newsletter, events calendar, and MeetMe list

• How-to Guides (e.g., lead a discussion; conduct training)

• Consulting for TPD organizations

TM


What are we learning?

• Content is important, but learning occurs through discourse around content

• Membership diversity is essential for innovations to germinate and spread

– Preservice and new teachers learn about the profession and veterans become valued resources

– Members form groups that cross agency, program, project, stakeholder group, and geographical boundaries

• Both private and public places are needed to support the community

TM


No single organization can satisfy teachers’ changing TPD needs

• Yet sustainable on-line communities are hard & costly to establish

• TPD organizations must understand the teachers’ challenge begins where their efforts leave off

• Teachers need ownership of an on-line workplace that will be there after TPD courses, projects, workshops, or grants end

Teachers are ready for on-line learning; many leaders are not

• Leaders see their projects as unique and view going on-line as a costly risk, not a strategic cost- and time-saving tool

• Traditional TPD approaches do not transfer well to on-line learning

• TPD pedagogy, staffing, and budgeting must be rethought

Challenges? TM


Performance Assessment Links in Science


• Leader: Edys Quellmalz ([email protected])

http://www.ctl.sri.com/pals

• Focus: An interactive resource bank for improving standards-based science assessment

• Size: $1.2M for 3 years

• Funder: National Science Foundation

– Elementary, Secondary, Informal Education (EISE)

– Instructional Materials Development

• Leader: Edys Quellmalz ([email protected])

http://www.ctl.sri.com/pals

• Focus: An interactive resource bank for improving standards-based science assessment



– Elementary, Secondary, Informal Education (EISE)

– Instructional Materials Development


Project ParticipantsProject ParticipantsAssessment Consortia

• Council of Chief State School Officers (CCSSO)• California Systemic Initiatives Assessment Consortium

State Departments of Education• Connecticut Department of Education• Illinois Department of Education• Kentucky Department of Education

School Districts • Springfield, Illinois• Vancouver, Washington

Science Curriculum Projects

Teachers

Assessment Consortia• Council of Chief State School Officers (CCSSO)• California Systemic Initiatives Assessment Consortium

State Departments of Education• Connecticut Department of Education• Illinois Department of Education• Kentucky Department of Education

School Districts • Springfield, Illinois• Vancouver, Washington

Science Curriculum Projects

Teachers




Lessons Learned

• Start by addressing priorities voiced by key participant groups (state science assessment programs, professional development groups)

• Partner with representatives of participant groups

• Involve participants, contributors, and funders in prototype design, seed content, and user testing

Lessons Learned

• Start by addressing priorities voiced by key participant groups (state science assessment programs, professional development groups)

• Partner with representatives of participant groups

• Involve participants, contributors, and funders in prototype design, seed content, and user testing




Addressing Challenges• Obtain commitment of partners to content development

and review

• Engage partners in ongoing interactions and community events

• Design incentives for users to provide feedback

• Build on models for maintenance and growth

• Collect systematic evidence of quality, usability and impact

• Balance public service and commercial functions

Addressing Challenges• Obtain commitment of partners to content development

and review

• Engage partners in ongoing interactions and community events

• Design incentives for users to provide feedback

• Build on models for maintenance and growth

• Collect systematic evidence of quality, usability and impact

• Balance public service and commercial functions


Online Evaluation Resource LibraryOnline Evaluation Resource Library

• Leader: Edys S. Quellmalz ([email protected])

http://oerl.sri.com

• Focus: Improving Project Evaluation



– Research, Evaluation, and Communication

• Leader: Edys S. Quellmalz ([email protected])

http://oerl.sri.com

• Focus: Improving Project Evaluation



– Research, Evaluation, and Communication



Project Participants• NSF Current and Prospective Principal Investigators

and Evaluators

• NSF Program Officers

• Evaluation Professors, Students

• Evaluation Training Programs

• The Evaluation Community

Project Participants• NSF Current and Prospective Principal Investigators

and Evaluators

• NSF Program Officers

• Evaluation Professors, Students

• Evaluation Training Programs

• The Evaluation Community



Lessons Learned• Exemplary content may be hard to find

• Develop navigation supports for novice users

• Involve key stakeholders in design and reviews

• Seek multiple dissemination paths early on

Lessons Learned• Exemplary content may be hard to find

• Develop navigation supports for novice users

• Involve key stakeholders in design and reviews

• Seek multiple dissemination paths early on



Addressing Challenges• Enable participants to take multiple paths to access

information and resources; “no one size fits all”

• Engage participants in active learning experiences aimed at extending their knowledge base and strategies

• Support and provide incentives for content contributions

• Support collaboration and peer review

Addressing Challenges• Enable participants to take multiple paths to access

information and resources; “no one size fits all”

• Engage participants in active learning experiences aimed at extending their knowledge base and strategies

• Support and provide incentives for content contributions

• Support collaboration and peer review


Center for Innovative Learning Technologies (CILT)

Center for Innovative Learning Technologies (CILT)

• Leaders: Roy Pea & Barbara Means

SRI InternationalMarcia Linn

UC BerkeleyJohn Bransford

Vanderbilt UniversityBob Tinker

Concord Consortium

• Mission: To serve as a national resource for stimulating research on innovative technology-enabled solutions to critical problems in K-14 learning

• Funding sources:

– NSF ($6M total for 4 years)

– Industry partners (Intel: $100K/year)

• Leaders: Roy Pea & Barbara Means

SRI InternationalMarcia Linn

UC BerkeleyJohn Bransford

Vanderbilt UniversityBob Tinker

Concord Consortium

• Mission: To serve as a national resource for stimulating research on innovative technology-enabled solutions to critical problems in K-14 learning

• Funding sources:

– NSF ($6M total for 4 years)

– Industry partners (Intel: $100K/year)

http://cilt.org


CILT Activities and ParticipantsCILT Activities and Participants

• Annual workshop for harvesting knowledge and leveraging diverse efforts on “theme teams” of high-priority and breakthrough opportunity• Visualization and Modeling• Ubiquitous Computing• Community Tools• Assessments for Learning

• 400-500 Participants set priorities for partnership projects; CILT later “seed funds” promising efforts and promotes synergy efforts

• “CILT Knowledge Network” for People, Places, Projects, Bibliography, Collaborations, Syllabi

http://kn.cilt.org• Postdoctoral training program

• Annual workshop for harvesting knowledge and leveraging diverse efforts on “theme teams” of high-priority and breakthrough opportunity• Visualization and Modeling• Ubiquitous Computing• Community Tools• Assessments for Learning

• 400-500 Participants set priorities for partnership projects; CILT later “seed funds” promising efforts and promotes synergy efforts

• “CILT Knowledge Network” for People, Places, Projects, Bibliography, Collaborations, Syllabi

http://kn.cilt.org• Postdoctoral training program


CILT as a Multi-level NICCILT as a Multi-level NIC

SRI

Berkeley

Vanderbilt

Concord


SRI

Berkeley

Vanderbilt

ConcordResearchCommunity



SRI

Berkeley

Vanderbilt

ConcordResearchCommunity

IndustryPolicy

Practice



Lessons Learned: Getting Started

Lessons Learned: Getting Started

• NIC-ing takes a huge amount of coordination - plan for it!

– Tasks that are often viewed as “overhead” by partners become core NIC functions

• Work with all groups of stakeholders early

• Provide a concrete focus or venue for sharing and collaboration (e.g., improving learning technology R&D projects; workshops)

• NIC-ing takes a huge amount of coordination - plan for it!

– Tasks that are often viewed as “overhead” by partners become core NIC functions

• Work with all groups of stakeholders early

• Provide a concrete focus or venue for sharing and collaboration (e.g., improving learning technology R&D projects; workshops)


Indicators of CILT SuccessIndicators of CILT Success

• Informal results as important as formal ones

• Formal Industry Alliance partnerships are difficult, but ad-hoc industry/research interactions are fruitful

• Metrics include:

• CILT-sponsored collaborations that resulted in sustained outside funding

• Ideas we promoted that took hold in research, policy, education, product development communities

• Isolated individuals who got the connections/support they needed to succeed

• Informal results as important as formal ones

• Formal Industry Alliance partnerships are difficult, but ad-hoc industry/research interactions are fruitful

• Metrics include:

• CILT-sponsored collaborations that resulted in sustained outside funding

• Ideas we promoted that took hold in research, policy, education, product development communities

• Isolated individuals who got the connections/support they needed to succeed


Lessons Learned: CILT ChallengesLessons Learned: CILT Challenges

• Contracting: Universities and other institutions like to own, not share (public-domain)

• Financing: Under traditional models, the process of collaboration is rarely funded

• Time and attention: NICs bring together the best and brightest - and therefore “the busy”

• Design: It’s easier to share curricular resources when they were designed for adaptability from the start

• Contracting: Universities and other institutions like to own, not share (public-domain)

• Financing: Under traditional models, the process of collaboration is rarely funded

• Time and attention: NICs bring together the best and brightest - and therefore “the busy”

• Design: It’s easier to share curricular resources when they were designed for adaptability from the start


Leaders: Jeremy Roschelle, Chris DiGiano, Roy Pea (SRI); Jim Kaput (U. Mass., Dartmouth)

http://www.escot.orgFocus: Improving quality, interoperability and reuse of software components for teaching middle school mathematics

Size: $2 million for 2 years

Funder: National Science Foundation

NEED: Without a NIC, disparate math education efforts have their own isolated improvement processes, resulting in ‘application islands.’

CTL SW

Univ. ofCO SW

Key PressSW

SimCalcSW

ShowMeCurricula

MathForumResources

MathTeachers

• Redundant

• Non-cumulative

• Unconnected


ESCOT as a NIC

Evolving knowledge network and interoperable software library for continuous improvement by accumulating, integrating, sharing, and testing work.

CTLComponents

Univ. of COComponents

Key PressComponents

SimCalcComponents

ShowMeCurricula

MathForumResources

MathTeachers

ESCOT Testbed

• Cumulative

• Integrated

• Continual testing



Growing Partnerships and Results

Overall: 8 funded institutional partners, plus 25+ volunteer institutional partners; 23 software developers

Non-Profits: SRI International; U. Massachusetts; U. Colorado; Swarthmore College; Stanford University; UC-Berkeley; many middle schools

Small Companies: Key Curriculum Press, DesignWorlds

International: Centre for Constructive and Experimental Mathematics (Canada); Computer Technology Institute (Greece)

Results:

• ESCOT now has 37 components, inc. spreadsheet, web browser, grapher, scripting languages, simulation engines, geometry sketchpad...• ESCOT database of 5 major math textbooks indexes 3000 places to use technology components to enhance learning


Lessons Learned (startup)Lessons Learned (startup)

• Vet ideas for NIC in extensive planning process

• Pick motivated, complementary, communicative core partners

• Support extensive face-to-face contact for first 6 months

• Clear tasks / expectations for partner roles

• Create good multi-channel communications infrastructure (extranet site, e-mail lists, phone conferences, software configuration management)

• Vet ideas for NIC in extensive planning process

• Pick motivated, complementary, communicative core partners

• Support extensive face-to-face contact for first 6 months

• Clear tasks / expectations for partner roles

• Create good multi-channel communications infrastructure (extranet site, e-mail lists, phone conferences, software configuration management)


Lessons Learned (Ongoing)Lessons Learned (Ongoing)

• Structure IP cross licensing carefully; hard to both build trust and create the most value from synergy across developers

• Focus NIC around frequent integration task all participate in (Problem of the Week)

• Bi-weekly NIC newsletter serves to keep members motivated, yet hard to keep loosely coupled and busy partners on-task

• Challenge of finding $$ for project coordination

• Key metric: How easy is it for stakeholders to adopt techniques coming from the NIC?

• Structure IP cross licensing carefully; hard to both build trust and create the most value from synergy across developers

• Focus NIC around frequent integration task all participate in (Problem of the Week)

• Bi-weekly NIC newsletter serves to keep members motivated, yet hard to keep loosely coupled and busy partners on-task

• Challenge of finding $$ for project coordination

• Key metric: How easy is it for stakeholders to adopt techniques coming from the NIC?


A guiding principle for CTL A guiding principle for CTL

• Developing NICs powerfully amplifies your ability to solve important interdisciplinary problems in education no one can solve alone, and...

• Developing NICs powerfully amplifies your ability to solve important interdisciplinary problems in education no one can solve alone, and...

• Requires and grows champions

• Establishes Centers of Excellence

• Seeds business development (partners and funders come back to us)

• Grows name and leadership recognition

• Supports recruitment


Cross-NICing and Meta-NICs?Cross-NICing and Meta-NICs?

• Large challenges due to demanding natures of NICs and domain differences

• NOTE: each of our education NICs had different mechanisms and solutions:

• TAPPED IN: virtual places; member services

• PALS: web-based digital library and training tools

• OERL: digital library of case-study models, and tools

• CILT: F2F workshops; CILT-KN; partnership seed funding

• ESCOT: interoperability testbed; POW integration teams

• All needed activities for integrating knowledge and incenting contributions and participation.

• Large challenges due to demanding natures of NICs and domain differences

• NOTE: each of our education NICs had different mechanisms and solutions:

• TAPPED IN: virtual places; member services

• PALS: web-based digital library and training tools

• OERL: digital library of case-study models, and tools

• CILT: F2F workshops; CILT-KN; partnership seed funding

• ESCOT: interoperability testbed; POW integration teams

• All needed activities for integrating knowledge and incenting contributions and participation.

copyright © 2000. sri international. all rights reserved. session 6 the unfinished revolution - ii...

Documents

education community

effective learning

university of california

education professionals

online learning experiences

learning technologies

isolated improvement

california pt3