damian bebell- '1:1 computing research: informing effective policy development

Technology Assessment Study Collaborative

Damian J. BebellAssistant Research ProfessorTechnology and Assessment Study CollaborativeLynch School of EducationBoston College

www.intasc.org

www.jtla.org

June 2010 Summit: 1:1 Research

IntroductionIntroduction

“All of us, professionals as well as laymen, must consciously break the habits we bring to thinking about the computer. Computation is in its infancy. It is hard to think about computers of the future without projecting onto them the properties and the limitations of those we think we know today. And nowhere is this more true than in imagining how computers can enter the world of education.”

—Seymour Papert, Mindstorms, 1980

Technology is changing society and schooling…Technology is changing society and schooling…

Computers and digital technology have transformed many aspects of 21st century life (and economies)

Increasing presence within schools, although access and use have rarely been robust…

Generation of “digital natives”

Increased public and political interest in educational reform

Purported Benefits/Reform Model

Increase resources and information

Exposure to computer literacy, technology, new learning (21st century skills)

Improve teaching (lesson plans, communication, instruction)

Motivate and engage students

Movement towards student-centered classrooms

Streamline record keeping

Special needs/accommodations

Differentiate Instruction/learning

Improve student learning and test scores

History and Trends in Educational Technology

Radical transformation of teaching and learning?

“Jet powered stagecoach”


Standard Metric of Technology Access

Student : Computer RatiosU.S. Average: 1983 = 125:1 1998 = 6:1 2007= 4.1

Source: Market Data Retrieval, Public School Technology Survey 1998-2006; Education Week, 2007

The rise of 1:1 Computing

20 years of 1:1 computing programs continuously evolving

Widespread proliferation across world

Few programs have more than 5 years experience

Fewer still have invested in research and evaluation efforts

So, lots of interest but little empirical research


Shared access

Sporadic use

Limited impact

1:1 Computing

Summary of 1:1 Research: www.jtla.org

Common Trends and Findings: Inputs and Setting

• Research and evaluation efforts were systemically underfunded and lacked rigor

• 1:1 defines an access ratio, little else

• Despite common access, significant variations in the type and levels of use across schools and classrooms

• Numerous obstacles to implementation

• Pivotal role of teacher

• Importance of building level leadership

• Curricular and technical support (PD)


More general summary of research from BECTA

“A new realism has emerged about the impact of technology on educational outcomes. Technology is no longer seen as a ‘silver bullet' but as a facilitating factor that can enhance teaching and lead to more effective learning.

A sharpened focus on the learner and the teacher, however, has established that learners' engagement with their own learning and teachers' willingness to challenge and stretch learners, are central to raising standards. Technology adds to the effectiveness of the learning process by providing tools for teachers to tailor challenge to the individual pupil.

At the learner-level technology allows students to produce professional outputs, the reticent to contribute to group discussions and, for those left behind, to practice core skills so that they can rejoin their peers. Technology also enables the student to gain formative feedback quickly and this has been shown to bring about changes in attitudes to school which, in turn, have stimulated more positive behaviours”.

http://www.becta.org.uk/impact.php


Common Research Trends and Findings: Outcomes (1)

• Technology can be adopted quickly into classrooms

• Implementation and findings often vary across and within schools

• Pedagogical beliefs can shape teachers’ technology use in the classroom.

• Importance of building level leadership• “I don’t think anyone ever expected how we’d

manage the 500+ laptops that kids were taking back and forth to their homes each day.”

• Concern and intimidation by sustainability• “Transform/Revolutionize teaching and

learning” is a very lofty goal

• Lots of positive teaching and learning results

• Long-term outcomes


Common Research Trends and Findings: Outcomes (2)

• Substantial increase in student and teacher technology use for a wide variety of purposes

• Documented changes in teaching and learning practices for most participants

• Increased student engagement, motivation

• Improvements in student products

• 3 of the 4 research studies examined student achievement


Teacher Attitudes After 3 Years of 1:1 Computing Teacher Attitudes After 3 Years of 1:1 Computing

Common Trends and Findings: Student Achievement

Standardized Test Scores has become the de facto measure of success

• CA: Compared 4th grade ELA test scores for 1:1 laptop students to a similar group of students in a traditional program from the same school district. Authors found that after two years, students in the 1:1 program outperformed the comparison group. Specifically, the 1:1 students had higher gains on the ELA test and on the subtests related to writing strategies and literary response and analysis than the non-1:1 students. Their findings suggest that “laptops may have a small effect on increasing such scores, with particular benefits in the areas of literary response and analysis and writing strategies”

• MA: Compared 7th grade students in their second year of the 1:1 program to students in matched comparison schools. Authors found statistically significant gains on student level ELA state assessment scores compared to non-1:1 students, after controlling for prior ELA achievement and SES. Authors additionally examined and found positive correlations between specific teacher and student practices and different subscales of the MCAS.

• TX: In one of the only experimental design studies of 1:1 computing, authors found implementation strength of Student Access and Use (of technology) was a consistently positive predictor of students’ TAKS reading and mathematics scores” and that students’ use of their laptop for learning at home was the “strongest implementation predictor of students’ TAKS reading and mathematics scores”.

• ME: David Silvernail has authored numerous studies and evaluation of MLTI and associated projects. March 2009, authors summarized 3 impact studies

• Causal comparison study of Creating Better Writers: 1/3 SD improvement in writing scores

• RCT 2 year Math PD program: Students in experimental group classroom scored significantly higher on: significantly higher on: 1. Tests specifically designed for the study 2. Statewide mathematics test

• Quasi-Experiment Hard Fun in Science: Using animation for science


Common Trends and Findings: Student Achievement

What added value does a 1:1 student to laptop ratio bring to technology-supported teaching and learning?

M. Dunleavy, W. F. Heinecke (2007)

Journal of Computer Assisted Learning

http://pkp.sfu.ca/ojs/demo/present/index.php/jce/article/view/171/56

Randomly selected students from an urban middle school population participated in 1:1 laptop classrooms or non-1:1 laptop classrooms. The researchers discovered a significant increase in overall science test scores as well as a significant gender interaction whereby boys had a much greater increase in scores in the 1:1 laptop program than girls.


““And as long as schools confine the technology to simply improving And as long as schools confine the technology to simply improving what they are doing rather than really changing the system, nothing what they are doing rather than really changing the system, nothing very significant will happen.” -Seymour Papertvery significant will happen.” -Seymour Papert


http://www.mff.org/edtech/article.taf?_function=detail&Content_uid1=106

Problems With Using Test Scores as a Primary Outcome Problems With Using Test Scores as a Primary Outcome Measure/success IndicatorMeasure/success Indicator

Schools serve multiple purposes, should not focus exclusively on one as others get “left behind.”

State constitutions, courts, parents, business leaders, and mission statements of schools themselves all attest to the often broad and multifaceted roles of American schools.


“The One Laptop Per Child initiative, whose green and white laptops are now ubiquitous in Uruguayan schools as a result of Plan Ceibal, likes to bill itself as an 'education project’… Plan Ceibal, on the other hand, does not see itself as an 'education project' per se, but rather as a project to help transform larger society, with the education system as just the initial vector through which the project hopes to infect all of Uruguay with a new level of 'connectedness'.”

-Michael Trucano, World Bank

Problems With Using Test Scores as a Primary Outcome Problems With Using Test Scores as a Primary Outcome Measure/Success Indicator (2)Measure/Success Indicator (2)

•Current tests are a poor/limited measure of academic/cognitive ability

•Teachers and principals in 1:1 MA pilot overwhelmingly reported that “students would use technology more if there was less pressure to perform on standardized test”

•Outdated turn around times•Huge amount of paper and mailings•Accommodations conditions uneven

•Mode of administration problem (for example ELA writing assessments)Example of computer writing study (7th grade MCAS) MA 1:1 Pilot

1:1 Students with laptops wrote longer essays that were scored statistically significantly higher than their peers using paper and pencil.

# of students Topic score Conventions score Word Count

Computer 310 7.2 5.6 388

Paper 141 6.6 5.3 302


Role and Potential for ResearchRole and Potential for Research

• Help us to ask better (and bigger) questions

• Inform and reflect on evolving practices (formative)

• Help convince teachers and others of ICT efficacy

• Determine what works best where (summative)

• Share success and overcome obstacles

• Establish a culture of reflection

“The process is continual and cyclical moving from the collection of data, to reporting and analysis and finally to using data for targeted interventions.” -Consortium of School Networking (COSN)


Role and Potential for Research in 1:1Role and Potential for Research in 1:1

• The field of assessment and educational measurement and research are completely reliant on and determined by technology.

• The IBM Optical Scanner allowed the bubble sheet and the rise of the 5 response question

• To this day, the artifacts of this primitive computer technology restrict imagination and practice


In our own research and evaluation work we have leveraged new and emerging technologies to advance the state of the art.

Bebell, D., O’Dwyer, L. M., Russell, M., & Hoffman, T. (2010). Concerns, considerations and new ideas for data collection and research in educational technology studies. Journal of Research on Technology in Education, 43(x).

1:1 Opportunities: Cutting edge assessments


New approach to testing accessibility uses principles of universal design

Build accommodations into test at outset of test development, eliminating the need for costly retrofits

Parting words…Parting words…

One of the underlying problems today is that most children (and many educators) have fully embraced the possibilities and opportunities afforded by computer-based technologies—whereas traditional school environments have not been provided adequate resources, time, and initiative to properly do so.

Research and reflection can be easily integrated and afforded through the ubiquitous technology, there is great potential to better serve students, teachers, schools and community. Reflections to date have only scratched the surface.

"We cannot solve our problems with the same thinking we used when we created them.” -Albert Einstein


Contact InformationContact Information

Email: [email protected]

www.intasc.org

www.jtla.org


mailto:[email protected]

http://www.intasc.org/

http://www.jtla.org/

damian bebell- '1:1 computing research: informing effective policy development

Documents

digital technology

impact of technology

teachers technology

teacher technology use

learnerlevel technology

new learning

learning practices

effective learning