DOES CRITERION HELP STUDENTS IMPROVE

THEIR WRITING?

Hank Delcore

Jennifer L. Ivie

Automatically-Generated Data

Students who made two submissions to ETS Criterion had higher average holistic scores than students who only made one submission.

Students with lower initial holistic scores were able to raise their scores, with subsequent submissions, to the same level as those whose initial holistic scores were higher.

Intervening variables?

Faculty Scoring Panel

Overall, student writing improved in classes that used ETS Criterion.

The greatest improvements from first to last submission were made in classes in which the instructors engaged students in substantial discussions of writing and how to use Criterion.

Intervening variables?

Music 187

Control group study across three sections of Music 187 (Jazz and Rock), an IC course.Group 1: no CriterionGroup 2: required 2-4 submissionsGroup 3: unlimited submissions

Group 2 has significantly higher instructor-assigned grades than Group 1.

Groups 1 and 3 had no significant difference in grades.

Collegiate Learning Assessment

A significant difference was found between Criterion users and Criterion non-users on the CLA performance task.

No significant differences were found between the two groups on any of the analytic writing task scores.

Faculty/Student Survey Highlights Students felt that Criterion

…helps with writing mechanics.…did not help them improve their writing skills.But, ESL students were more likely than English as First

Language students to respond that Criterion helped them improve their writing. And, they ESL students preferred Criterion feedback to instructor feedback.

Faculty felt that Criterion…did not reduce their workload.…did not do as much as they had hoped.…made grading simpler.…encouraged students to spend more time improving their

writing.… overall improved student writing.

Faculty Learning Community Faculty commented positively on

…the ease and convenience of using ETS Criterion.…the added ability it gave them to focus on course

content rather than grammar and mechanics.…the fact that students using Criterion were more

engaged with the writing process. Drawbacks included

…the inability of the program to recognize technical terms or subject-specific writing conventions.

…perceived inaccuracies in holistic scoring.…lack of compatibility with TurnItIn and Blackboard.

Conclusion

Some evidence that Criterion helps students improve writing, likely in combination with good instructor practices.

Criterion seems to help most with mechanics. Convergence of survey and FLC data: faculty

agree that students who use Criterion are more engaged with the writing process.

Next steps…