DefiningData Habit of Mind

Saad Chahine Mount Saint Vincent University

Push to Use DataExternal pressures for educators to use data

from large- scale assessments

Very little is known about the process by which educators interpret and make judgments based on reports from large-scale testing

Potential compromise in validity when educators interpret and make judgments about students from large-scale assessment reports

π

Data Habit of Mind π

Methods Designed instruments to measure Statistical Literacy

and Score Report Interpretation (2 mock score reports)

Conducted pilot test with 12 pre-service students and refined instruments and interview procedure

Recruited educators through advertising

Conducted cognitive interviews to examine 20 educators’ data habit of mind

Used protocol analysis procedure to examine cognitive interview data (i.e., educators’ cognitive processing when interacting with score reports)

π

Sample Snap Shot All elementary (Gr. 1-8) teaching experience

with provincial testing

13 Female, 7 male

1-21+ years of experience

17 had undergrad focus in Humanities or Social Science and 3 had undergraduate focus in Sciences

Majority had B.Ed. (13); others had higher degrees (1 Ph.D., 6 M.Ed.)

π

Statistical Literacy

π

π

Results: Statistical Literacy

Min Max Mean SD

Statistical Literacy

Describing 3.00 4.00 3.77 0.24

Organizing & Reducing 2.00 4.00 2.89 0.65

Representing 1.67 4.00 3.33 0.65

Analyzing and Interpreting 2.00 4.00 3.38 0.53

π

Score Report Interpretation

π

π

Results: Score Report Interpretation

Cooper’s Fitness Test

Min Max Mean SD

Describing 4.00 4.00 4.00 0.00

Summarizing 3.00 4.00 3.75 0.44

Questioning 1.00 4.00 2.10 0.91

Proposed Application 1.00 4.00 2.45 1.10

Jabberwocky Test

Describing 4.00 4.00 4.00 0.00

Summarizing 3.00 4.00 3.75 0.44

Questioning 1.00 4.00 1.95 0.98

Proposed Application 1.00 4.00 2.00 0.97

π

Data Habit of Mind

π

π

Categories Based on Data Habit of Mind Skills π

Example: Questioning

Questioning Patterns for Cooper’s Fitness Test Score Report

π

Example: Questioning

Questioning Patterns for Jabberwocky Test Score Report

π

Influences of Data Habit of Mind

π

Results: Educational History & Comfort Levels

Expected that educational history and comfort levels (information, math, and stats) would be meaningfully related to Data Habit of Mind

However, found no meaningful relationships…

Examination of interview transcripts gives us glimpse of insight into what other factors may be meaningfully related to Data Habit of Mind…

π

Educator Beliefs About the Use of Assessment Data

Participant E18: “Assessment is probably, for me, has always been in my annual learning goal...my life-long learning goal in my formal years of education. It has been a constant pursuit of mine to understand, advance and implement assessment. We are a very data-driven school and I didn’t have to come to a data-driven school to be hungry for data. It already was in anything that I did...You keep yourself alive by having data. It just goes hand in hand. We are a very…we are a school that is constantly pushing for data.”

Educator with Higher Statistical Literacy and Higher Score Report Interpretation

π

Educator Beliefs About the Use of Assessment DataParticipant E13: “I’ve done a few workshops on DRA and reading, in math, like effective instruction. I’ve done a lot of reading myself there and we have had a lot of in-service training. That being said, it is an area that is still a little bit hazy because we are going along in our lessons and I don’t necessarily assess them. We do the evaluation side to see what we can get in marks. That’s why I’m interested in the whole literacy thing because we assess the kids in the reading and see where we need to go from there.”

Educator with Higher Statistical Literacy and Lower Score Report Interpretation.

π

Educator Beliefs About the Use of Assessment DataParticipant E2: “Ha ha, very high … Well, I think it’s kind of the package, frankly. I have my reading specialist, I have my religion specialist, which is essential for the Catholic school board. I do have a Master’s degree. I also think the fact that I’ve taught all three divisions helps a lot. Um, I am, in fact, I was an adjunct professor, and you have to mentor the young teachers... I’ve gone anywhere from basal readers to whole language to comprehensive literacy. I’ve kind of seen everything. That goes for math education, as well.

Educator with Lower Statistical Literacy and Higher Score Report Interpretation.

π

Educator Beliefs About the Use of Assessment Data

Participant E11: “Assessment to me ... it’s not always like paper and a test, a hard assessment that way. Sometimes I think you can get enough of an assessment by watching, observing and talking to the students. Some might not be able to articulate on paper and pencil what they do by talking.”

Educator with Lower Statistical Literacy and Lower Score Report Interpretation.

π

Future Research More research needed to examine categories

Double coding or multiple raters

Make links with medical diagnostic methods

Include more interactive reporting methods

Make links with cognitive human-computer Interface work (e.g., Ware’s work)

Conduct large-scale study of reporting systems and improvements

π

saad.chahine@msvu.ca

π