Zeros or No-zeros?

Stats from 6 EPSB high schools

--No Zero Policy-- Evidence It Does NOT Work

An initial investigation into the relationship between a no-zero policy and 5-year longitudinal data for school performance, diploma completion rate and

other factors.

While the 8 graphs could stand alone as reason to question the practice, explanatory notes have been added

so parents, students and the general public may gain a better understanding.

EMAIL: critically.analyzed@gmail.com

Zeros or No-zeros?

Stats from 6 EPSB high schools

A colleague challenged me to research and consider the effectiveness and soundness of a reformed grading policy as presented by Ken O’Conner AND others. Finding little more than anecdotal evidence and O’Conner’s own admissions on this ―The Grade Doctor‖ site that he is not aware of any hard-core research, I concluded I had to do my own study. Please consider it as only a starting point for a re-evaluation of the this new assessment model that is apparently being imposed on EPSB high schools.

Zeros or No-zeros?

Stats from 6 EPSB high schools

My Research Evolved Into Three Realizations: • • I was surprised by my findings. Some of the “reformed” practices have been conducted by caring, master teachers for decades and agree with O’Connor’s Grading for Learning type of philosophy. But searches reveal that the policy is certainly not “accepted” by all educators and is clearly “controversial”. Evidence is not hard evidence, only anecdotal.

• Although I am frustrated, I’m trying to get beyond the “no-zero”/“zero” argument to find middle ground here and identify root causes so we could work together at restoring a healthy educational environment. (The 8 charts included are clearly very telling--so what’s wrong?)

• A “new operating system of management” proposed by Daniel Pink (quoted as an authority on motivation by supporters) suggests autonomy

be given to students to promote mastery and sense of purpose. (Shouldn’t this autonomy be given back to teachers as well for the same reason?)

Zeros or No-zeros ?

Stats from 6 EPSB high schools

There was a total vacuum when I tried to find some real evidence that supported the use of codes vs. zeros as motivators. Did using zeros as placeholders discourage students and make them drop out? Anecdotal evidence was all that I could get, and it was typically standards based stuff at elementary and may not be best suited to high schools--especially in Math and Sciences.

While monitoring his blog called The Grade Doctor, even O’Conner himself admits several times that he knows of no research to support the movement and when he did mention ―empirical evidence‖ he does not offer links either in his blog or on his website.

Zeros or No-zeros ?

Blog Comment 1

Zeros or No-zeros ?

Blog comment 2

Zeros or No-zeros ?

Blog comment 3

Zeros or No-zeros ?

Stats from 6 EPSB high schools

How did the known hard-core “no zero” or “code policy” Edmonton schools rate province-wide on the standardized provincial diploma exams? I decided to check the Fraser Institute’s Alberta Report Card . http://alberta.compareschoolrankings.org/ChooseReport.aspxf

(Provincial Diploma exams are a very good mirror for students, parents, teachers and secondary institutions since it the ONLY authentic, trustworthy mark indicating that students are being marked consistent with provincial standards. The great inflation of marks of the past 4 years are making the diploma exams LOOK like they are of too high a standard, but the difficulty level of these exams has been fairly consistent. It’s the school mark worth 50% that is questionable sometimes due to principals changing marks or teachers being told to do so.)

(The linked document explains the Fraser Institute’s methodology and how, in their final rating, actually compares schools only with schools in the province of the same socio-economic level and similar course offerings).

Zeros or No-zeros ?

Stats from 6 EPSB high schools

I used the Frazer’s comparison tool to produce the following charts for the 5 yr span 2006-2010 for 6 schools only (a limit of the comparison tool):

•One school where teachers were instructed by an administration implementing a strict interpretation of the reformed grading philosophy for the 5 year period (they do NOT use any zeros but use codes instead -- Eastglen H.S.)

•One school where the administration openly professes that they leave such decisions up to teachers since they are in the best position to evaluate the students (Some teachers there claim that the use of zeros builds responsibility and citizenship -- Strathcona H.S.)

•Ross Sheppard and 3 other high schools with varied implementation of the new philosophy. (Ross Shep had JUST started the policy in the spring of 2011 and many teachers were not using it--or were confused)

The same comparison was made with M.E.Lazerte (as non-zero) with similar results.

Zeros or No-zeros ?

Stats from 6 EPSB high schools

NOTE 1: I do NOT criticize the teachers… they may well be the victims, having had their autonomy and professional discretion removed. NOTE 2: As you review the charts, consider the trend over the 5 years. Eastglen, being a school from a lower socio-economic area, may have different statistics. However, if there is a significant change in values over the 5 years, one has to consider the possible reasons for the change. NOTE 3: Definition of terms and more detailed data for each school are on the Fraser Institute Website and in the Appendices. http://alberta.compareschoolrankings.org/ChooseReport.aspx

Which school uses O’Conner non-zero? (and which encourages zeros?)

Zeros or No-zeros ?

Stats from 6 EPSB high schools

Notice that the course load is significantly smaller for Eastglen. Likely more students may have jobs because of its socio-economic location. But shouldn’t the boost in motivation from the ―no zero‖ code policy have had students enjoying school more and taking more, not fewer, classes?

No wonder it shows that MORE students there are in school for much longer as shown in the ―Delayed Advancement Rate‖ chart on the following slide. Is that frustrating for students? Is it expensive for taxpayers? (Side issue – MacLean’s Raising the dropout age won’t work)

Which school uses O’Conner non-zero? (and which encourages zeros?)

Zeros or No-zeros ?

Stats from 6 EPSB high schools

Perhaps the next slide is the most unsettling of the lot. I concede that one doesn’t expect as many students from a lower socio-economic school to get their High School Diploma. But after 5 years of ―code policy‖ and supposedly facilitating success, note that the next slide shows a significant trend. There is a DROP (red line) in the High School Diploma Completion Rate. (more dropouts with the no-zero policy?)

(Interesting… I wonder what Jasper Place (dark purple line) has done to greatly INCREASE their Diploma Completion Rate? I understand they have a modified/moderate ―zero‖ policy rather than a rigid, extreme no-zero policy.)

Which school uses O’Conner non-zero? (and which encourages zeros?)

Zeros or No-zeros ?

Stats from 6 EPSB high schools

The next two slides show the % of Diploma Exams failed and Diploma Mark over 5 yrs. The ―no-zero‖ school’s stats went from 18% failure rate to 33% of the students failing the Diploma Exams. (Province was about 15%-17%)

The school diploma exam averages plummeted. When using the codes, teachers have found that considerably MORE effort goes into tracking down, multiple times, only 5% to 10% of the students that always are ―absent‖ on exam days, or fail to turn in assignments etc. It has been reported by many teachers that putting in Zeros As Placeholders (ZAP) greatly reduces this task and more time for creativity. Could the other students be suffering since the teacher is pre-occupied?

Which school uses O’Conner non-zero? (and which encourages zeros?)

(diploma)

Which school used O’Conner non-zero? (and which encourages zeros?)

(diploma)

Zeros or No-zeros ?

Stats from 6 EPSB high schools

The next slide is of the most concern to me. It compares the school-awarded mark with the diploma exam mark (currently weighted 50/50). This is an indicator of totally unjustified mark inflation by the schools. The extreme upward trend is disturbing because it distorts the blended mark. You cannot trust it on it’s own. Is the use of codes inflating the marks?

(Unfortunately there is talk of reducing the value of the diploma to a 25/75 % blend making it even a more distorted, non-level playing field. Perehaps the diploma should be worth more than now because of rampant inflation of school-awarded marks. That would present a more authentic mark.)

Which school uses O’Conner non-zero? (and which encourages zeros?)

Zeros or No-zeros ?

Stats from 6 EPSB high schools

The Overall School Rating only compares schools with similar schools with the same average family income and programs. The rigid, strict non-zero / code policy school appears to be losing considerable ground. So… who is to say that noticeable drop in ratings are due to a ―non-zero‖ or a ―code policy‖? Maybe yes; maybe no. But something is sure affecting student performance. Inflated, non-authentic marks are unfair to students--to those that DO turn in all their assignments. (Post-Secondary Institutions really have a problem with this. Your mark shouldn’t depend on which school you attend ! Alberta Learning, ATA and school boards have to hold the principals and superintendents accountable and deal with this.)

Which school used O’Conner non-zero? (and which encourages zeros?)

Which school used O’Conner

non-zero?

OK… so this was only ONE ―reformed grading policy‖ school. How do I compare others? Clearly something has been driving the school-awarded marks upwards resulting in severe marks inflation. Recently the Calgary Herald offered an interactive database that allows one to list and rank the mark differential for all 200 high schools. I was SHOCKED to see 20%-40% inflation rates. On the next slide is the inflation ranking of Edmonton Catholic and Public Schools for Mr. Dorval’s subject (Physics 30). Do your homework and find out which schools have been rigidly adhering to the philosophy. Then looking at the next slide, you can decide for yourself if the marks are inflated for those schools.

Zeros or No zeros?

Marks Inflation ?

The Calgary Herald “Making the Grade” database has an

excellent user interface to compare schools in the province. At the top

of the page there is this excellent descriptor (see below). Ross

Sheppard (including Mr. Dorval) 3.3% discrepancy between the teacher

awarded mark and the diploma exam mark indicates that “students are

being marked consistent with the provincial standards.” (He’d been doing

that for 35 years… and didn’t want to compromise that by ignoring the fact that some

students were not doing all of the work). Sadly I present to you my final

chart, showing the travesty of marks inflation happening in this

province. (Why doesn’t Alberta Learning step in and sanction those schools?)

Zeros or No-zeros ?

Calgary Herald Database

Is there great inflation in

the school awarded

marks of the schools

known to be rigidly

adhering to the reformed

assessment philosophy?

Ross Shep had barely

started it’s policy by June

2011 and teachers were

(are?) resisting.

Zeros or No-zeros?

Rumours ?

Some Edmonton high schools have been using the reformed assessment policy for a number of years. Recurring and disturbing ―rumours‖ include…

• a ―new‖ initiative for cheaters get to Do-Over (the same exam with scrambled answers) instead of getting zero (Schoolhouse Consulting). (Cheating is in epidemic proportions already and this would not provide a

deterrent—there are no consequences for cheating.)

• At a north Edmonton school, students only get marked on attempted questions so if they only bubbled in 20 of 25 multiple choice questions teachers are told to consider that student’s mark is to be based out of 20 instead of 25! (with this logic a student can choose 5 questions they know

out of 25 on a test, leave the rest blank, and get 100%!

(I would request the board and media investigate these allegations)

Zeros or No-zeros ?

Possible Solution…

Those advocating the reformed grading philosophy often quote Daniel Pink (of TedTalks renown - a business management philosopher often quoted by educators) when it comes to the motivation of students and the move to make marks less important to learning. Pink's basic premise is: research has shown rewards are not the best motivators. Pink suggested a “new operating system of management”--shown to work--includes three elements: Autonomy , Mastery and Purpose. Autonomy meaning that individuals (students AND teachers) are given the opportunity to solve problems in their own unique way. Mastery refers to the phenomenon that autonomy leads to even better performance, with individuals continually working to improve. Purpose refers to the feeling that a person is working for a cause that really matters on a larger scale.

Zeros or No-zeros ?

Possible Solution…

Under some successful EPSB administrations in the past, Pink’s philosophy has indeed produced some master teachers.

Now this is being undone and nullified—ironically, one of these “master teachers” was suspended for not following orders by an administration that claims to be implementing a philosophy which quotes Pink when it comes to motivation !

Zeros or No-zeros ?

Possible Solution…

Is it possible that heavy-handed management in schools where administrations are making the reformed grading philosophy mandatory is the REAL cause for declining student performance shown by the graphs?

Perhaps the moderate administrative style at Strathcona school is what is EMPOWERING their teachers and students to do their best.

Zeros or No-zeros ?

Possible Solution…

After much controversy over evaluation reform, the Ontario Minister of Education has returned the issue of grading to teachers and their professional judgment. Texas school districts would no longer be able to force teachers to inflate students’ grades under a 2009 law.

EPSB Board Members TAKE YOUR POWER BACK !!!

Implement a proven management strategy and give the teachers back their autonomy to use their professional

discretion when assessing students

Daniel Pink –TedTalks ―On the Surprising Science of Motivation‖

Appendix – URLs to Linked References

The linked references in the PowerPoint version will not work in this video version. They are listed below

Fraser Institute Alberta Report Card http://alberta.compareschoolrankings.org/pdfs/Fraser_Institute_Report_Card_on_Alberta’s_High_Schools_2011.pdf

Fraser Institute Comparison Tool http://alberta.compareschoolrankings.org/ChooseReport.aspx

The Calgary Herald’s Data Centre ―Making the Grade‖

http://www.calgaryherald.com/news/data-centre/school-marks-vs-exams/index.html

Daniel Pink –TedTalks ―On the Surprising Science of Motivation‖ http://www.ted.com/talks/dan_pink_on_motivation.html

MacLean’s - Raising the dropout age won’t work http://www.calgaryherald.com/news/data-centre/school-marks-vs-exams/index.html

Schoolhouse Consulting http://educhatter.wordpress.com/2011/11/01/do-over-student-evaluation-what-lesson-does-pardoning-cheaters-teach/

Appendix – Contact Information

While I would prefer to come out of the closet and identify myself, I do have a great sense of self-preservation.

Unlike the brave Mr. Dorval, I have a family to support and still have to teach. I love my subject and sharing it through my interactions with my students. It’s supposed to be fun. If left alone to motivate my students as needed--it is fun.

I hope that you do not minimize what I am trying to convey because I have remained anonymous. I care deeply about this issue and re-instating the autonomy of the teacher.

EMAIL: critically.analyzed@gmail.com

Appendix – Case 2 (Which school uses non-zero and which encourages zeros?)

I criticized my own data saying that although there is a great negative correlation between no-zero policy and school performance etc. doesn’t mean that there is causation—but one certainly cannot ignore the fact that something is going terribly wrong in the strict no-zero schools. Before anyone charges that the data is only for ONE non-zero school that I have singled out (Eastglen), I now have produced the same charts for a SECOND known, strict no-zero school (M.E. LaZerte)

Four comparison charts follow without commentary

Appendix – Case 2 (Which school uses non-zero and which encourages zeros?)

(diploma)

Appendix – Case 2 (Which school uses non-zero and which encourages zeros?)

Appendix – Case 2 (Which school uses non-zero and which encourages zeros?)

Appendix – Case 2 (Which school uses non-zero and which encourages zeros?)

Appendix - Sample Report

Appendix - School Report

Appendix - School Report

Appendix - School Report

Appendix - School Report

Appendix - School Report

Appendix - School Report