can third graders type?

Can Third Graders Type?

An examination of the fairness by grade of requiring constructed-response items on large-scale assessments administered via the computer By Steve MartinConnecticut State Department of EducationConnecticut Assessment Forum 8/13/12

Good AfternoonToday we will

Practice using responders. Use responders to get a sense of where the group is. Review where SBAC is on this question and cover

standards and other background on this issue. Discuss what other states are doing. Examine typed-response data from a recent CMT pilot test. Discuss possible contributing factors to trend data. Review where Connecticut is going in the short-term. Discuss reporting and further research possibilities. Q&A We will finish by exploring some free sample online

resources designed to help students learn to type.

Practice Using Responders Start->programs->2know toolbar

① To start your responder, first hit the on/off button on the bottom.



② You will know it is on when words appear in this screen.




③ Wait for the 2Know Toolbar to appear on this screen before you enter the session.




③ Wait for the 2Know Toolbar to appear on this screen before you enter the session.

④ Once you see the toolbar press ENTER.

⑤ When you see the network “Steve” on your responder, press ENTER again.

⑥ Once everyone is on we will practice this ABC question. When instructed, press A if you are MALE, and press B if you are FEMALE; and then press ENTER.

Question #1Q: What age were you when you

first learned to type?

A. Age nine or youngerB. Age 10 -12C. Age 13 -15D. Age 16 -17E. Age 18 or older

Question #2Please answer this question only if you have children of your own.

Q: At what age was your child(ren) when he/she/they first learned to type?

A. Age nine or youngerB. Age 10 -12C. Age 13 -15D. Age 16 -17E. Age 18 or older

Question #3If you have a child or children who are at least 13 years of age, (or at least grade 8) please answer this question about typing speed.

Q: I think that my child (or my children) can type…

A. Faster than I can.B. About the same speed as I can.C. Not as fast as I can.D. I don’t know how fast my child(ren) can type.

Question #4Please answer this question if you have recent experience with third graders using the computer.

Q: Which of the following is the fairest assessment of typical third-grade computer skill?

A. They cannot use the keyboard or mouse with any reliability.B. They can use the mouse to point and click reliablyC. They can use the mouse and keyboard reliably, but their

typing is very slow.D. They can use the mouse and keyboard reliably and they

can type words using the computer keyboard at least as fast as they can write with a pencil.

Question #5Q: Given that the statewide test will be

administered to third graders via computer beginning in the spring of 2015, which of the following test environments for them would you currently support?

A. Third graders should only need to point and click.B. Third graders should point and click AND type

responses to short-answer items.C. Third graders should point and click, type responses

to short answer items, AND type responses to extended response items (like the CMT writing sample).

Where SBAC StandsSteve: What research is planned on this issue?SBAC

Cognitive lab research is planned to learn how students across a range of grades approach and interact with different types of computer-administered items.

This is part of a larger effort to eliminate or minimize factors that could distract students from the ability to demonstrate their knowledge and skills.

For more information, please see the SBAC accessibility guidelines

Where SBAC StandsSteve: What about keyboarding skills specifically?SBAC: The Common Core State Standards emphasize proficiency with technology. For example, the 4th grade writing standard calls for students to “demonstrate sufficient command of keyboarding skills to type a minimum of one page in a single sitting.” Administering assessments by computer is a part of ensuring that schools and teachers can measure mastery of the full range of the standards.

Technology Standards for Grade 3 Students

What exactly are the CCSS Standards on the keyboarding issue for grades 3 and 4?

CCSS: Grade 3 Standard—W.3.6—With guidance and support from

adults, use technology to produce and publish writing (using keyboarding skills) as well as to interact and collaborate with others.

Grade 4 Standard—W.4.6.—With some guidance and support from adults, use technology, including the Internet, to produce and publish writing as well as to interact and collaborate with others; demonstrate sufficient command of keyboarding skills to type a minimum of one page in a single sitting.

What is Age Appropriate?Differing perspectives

“There is no right or wrong time to introduce your child to the family computer. Much like deciding when to buy the first bike, or when your child can go out alone with friends, the choice will be based on your parenting style, your values and, most importantly, your child’s readiness.”

Source:http://familyinternet.about.com/od/introtofamilycomputing/a/growthchart.htm

http://familyinternet.about.com/od/introtofamilycomputing/a/growthchart.htm

What is Age Appropriate?Differing perspectives

What about computers and Waldorf education?

Source: http://www.whywaldorfworks.org/02_W_Education/faq_about.asp

“Waldorf teachers feel the appropriate age for computer use in the classroom and by students is in high school. We feel it is more important for students to have the opportunity to interact with one another and with teachers in exploring the world of ideas, participating in the creative process, and developing their knowledge, skills, abilities, and inner qualities. Waldorf students have a love of learning, an ongoing curiosity, and interest in life. As older students, they quickly master computer technology, and graduates have successful careers in the computer industry.”

http://www.whywaldorfworks.org/02_W_Education/faq_about.asp

General Typing Speed Data

Source: Lynda S. Hartman rev 1/20/05 Permission to use if credit is retained. [email protected]

General Typing Speed Research


2002-2003 StudyIn the IllinoisSchool District U - 146

Researchers suggest that 10-20 wpm is sufficient in later elementary school (grades 3 – 6).

Some teachers, though, have made lower targets:

Others made their goal to be 2-3 times the students handwriting speed.

Grade Target in WPM

3 34 75 7


Buhler, KansasUSD – 313Conducted Research of student typing speed.

Task Force Report:

No recommendations for keyboarding speed until grade 8.

At this point, though, students are expected to type at a rate of 30 WPM with high accuracy.



Utah State Office of Education

Utah provides a detailed Web site in which keyboarding speed requirements are laid out very explicitly:

Grade WPM3 154 205 256 277 - 12 45 by end of

keyboarding course

http://www.schools.utah.gov/cte/keyboarding_standards.html


What Are Other States Doing?

I asked states that were listed in a State Educational Technology Directors Association 2011 report as having computer delivered assessmentsalready the following six survey questions:

1. Is your state’s summative test administered mostly via computer at any grade?

2. Is your state’s summative test administered mostly via computer to grade 3 students?

3. For how many years has your state allowed for grade 3 students to be administered a large-scale exam on computer?

4. Are there any computerized, grade 3, open-ended items requiring a keyboarded short-answer response on the statewide exam?

What Are Other States Doing?

Survey Questions 5 and 6:

5. Are there any grade 3, open-ended items requiring a keyboarded extended-answer response on the

statewide exam? (For purposes of this question, consider an “extended” response as typically requiring more than two sentences to receive the highest score.)

6.Has the issue of keyboard competence for younger students in any way shaped your state’s

decisions in the selection of computerized item-types?

What Are Other States Doing?1. Is your state’s summative test administered

mostly via computer at any grade?State ResponseDelaware YesGeorgia NoHawaii YesIdaho YesIndiana YesKansas YesMaryland Gr 5 &8scienceMinnesota YesNebraska YesOklahoma No grade 3 testTexas NoWest Virginia Yes

9/12 state’s surveyed administer some portion of their summative test via computer.

For Georgia, the computer version is an option for the grade 3 reading retest only.

Oklahoma reports that there is no grade 3 computer assessment in the state at all.

Texas administers an assessment in grades 2-12 online. TELPAS (Texas English Language Proficiency Assessment System) is for LEP students only. That is the only online assessment at grade 3 in Texas.

What Are Other States Doing?2. Is your state’s summative test administered

mostly via computer to grade 3 students? State ResponseDelaware YesGeorgia No*Hawaii YesIdaho Yes Indiana NoKansas YesMaryland NoMinnesota YesNebraska YesOklahoma NoTexas No*West Virginia Yes*

7/12 state’s surveyed administer their summative test via computer to grade 3.

For Georgia, the computer version is an option for the grade 3 reading retest only.

Texas LEP students only.

Limited to the Direct Assessment of Writing. The rest of the Westest 2 is administered via paper and pencil at this grade.

First draft can be done in pencil, but then it must be copied into the computer for a score.

What Are Other States Doing?3. For how many years has your state allowed for grade

3 students to be administered a large-scale exam on computer?

State ResponseDelaware 2yearsGeorgia 4 years*Hawaii 2yearsIdaho 3yearsIndiana 2yearsKansas 6yearsMaryland Was modified/ no longerMinnesota 3yearsNebraska 3yearsOklahoma NATexas NAWest Virginia 3years

9/12 state’s surveyed have administered some test to grade 3 students for 2 years or more.

Maryland gave a small number of students a modified assessment online, but they no longer administer a modified exam.

Kansas has been at this the longest having six years of computer-delivered testing.

Georgia makes this an option for grade 3 retesters.

Indiana—not the summative test.

What Are Other States Doing?4. Are there any computerized, grade 3, open-ended

items requiring a keyboarded short-answer response on the statewide exam?

State ResponseDelaware No all m-c and gridGeorgia NoHawaii Yes*Idaho NoIndiana NoKansas NoMaryland NoMinnesota Yes*Nebraska NoOklahoma NoTexas NoWest Virginia No*

Only 2/12 state’s surveyed require keyboarded short-answer responses from students at grade 3.

Hawaii gave a tentative “Yes” to this question. This still needs confirmation.

Minnesota administers the MCA in Mathematics to 80% of the students. The short-answer items are only numerical. MN ran cognitive labs and decided to limit students to just the numbers.

West Virginia only gives the Writing Sample online. No short-answer open-ended.

What Are Other States Doing?5. Are there any grade 3, open-ended items requiring a

keyboarded extended-answer response on the statewide exam?

State ResponseDelaware NoGeorgia NoHawaii Yes*Idaho No Indiana NoKansas NoMaryland NoMinnesota NoNebraska NoOklahoma NoTexas NoWest Virginia Yes*

Only 2/12 state’s surveyed require keyboarded extended-answer responses from students at grade 3.

Hawaii gave a tentative “Yes” to this question. This still needs confirmation.

West Virginia direct assessment of writing is keyboarded at grades 3-11, the rough draft can be by paper/pencil but final must be keyboarded by grade 3 students. When surveyed, 85% of 3rd graders said that they preferred the online version of the test over the traditional paper and pencil test.

What Are Other States Doing?5. Has the issue of keyboard competence for younger

students in any way shaped your state’s decisions in the selection of computerized item-types?

State ResponseDelaware NoGeorgia NoHawaii Yes*Idaho NoIndiana NoKansas Possibly*Maryland Yes*Minnesota Yes*Nebraska Yes*Oklahoma NoTexas NAWest Virginia

Yes*

6/12 state’s surveyed have made decisions based, at least in part on, keyboard competence.

Hawaii gives students three opportunities to test.

Kansas has statewide standards for keyboarding that begin in kindergarten.

MD moved away from administering online.

MN’s Cognitive Labs indicated that they should limit keyboarding to just the numbers.

Nebraska does paper/pencil until grade 4.

West Virginia did a survey, 85% of 3rd graders said that they preferred the online version of the test over the traditional paper and pencil test.

A Question for Your Responder

Given what you just learned about the experience in other states…

Q: which of the following test environments for third-grade students would you currently support?

A. Third graders should only need to point and click.B. Third graders should point and click AND type responses

to short-answer items.C. Third graders should point and click, type responses to

short answer items, AND type responses to extended response items (like the CMT writing sample).

March 2012 CMT DAW Pilot Test

Over a thousand CT students per grade (range 1,381 – 1,581) participated in an online Direct Assessment of Writing pilot which required them to keyboard their responses.

These responses are not reported back to schools or students. They don’t “count.”

The purpose of the pilot test is to determine the difference in difficulty between the two modes. If the two modes can be “post-equated” then a choice of modes can be offered before the 2015 online test.

The tests were recently scored. Both the pilot scores, and the observations of the scoring leads at the various Measurement Incorporated scoring centers were reported to the state.

Anecdotal Evidence From the Scoring Center

Mathematics: (Marianne Nasiatka, Grade 3 Scoring Director)

We saw evidence that some third graders don’t know how to use the shift key. An equal sign would appear in their responses instead of an addition symbol.


Reading and Science (Steve Zrimec, project monitor) The quality of the typewritten responses appeared to be equal to the handwritten responses from past projects. Keyboard unfamiliarity presented no obvious problems. There did not appear to be any obvious advantages for the students (i.e. spelling, grammar, and even ideas weren’t improved because of the format). And actually, this seemed true across all grades, not just Grade 3.


Direct Assessment of Writing (Sharon Williams, Grade 3 scoring director)

We were pleasantly surprised at the typing skills. There were few responses that indicated that the students did not know how to type. In other words, we saw very few "fingers on the wrong keys" gibberish responses but rather a fairly competent sample.

Readers noted (as did I) that an unusually high number of responses ended abruptly, as if the writers had run of time before finishing their narratives. We wondered if this was because the third graders are unable to type as fast as they can write. We also noted that many of the responses seemed brief and undeveloped compared to the responses we saw in the spring assessment. Very few typed responses went on to the second page, and many were only a half page long. This could be an issue with time and typing speed too.


Now that you have heard of the observations from the Scoring Centers…





CMT Typed Response Data

Grade

Score

Point

Grade 8 Grade 7 Grade 6 Grade 5 Grade 4 Grade 36

6.5

7

7.5

8

8.5

9

8.6

8.28.3

8.4

8.68.5

8.3

7.7

7.5

7.8

7.6

6.8

CMT Handwritten vs. TypedScore Difference by Grade

2012 CMT_DAW (HAND-WRITTEN CENSUS)

2012 MIST_DAW (COMPUTER PILOT)

Grade Difference

8 0.3


Grade

Score

Point


6.5

7

7.5

8

8.5

9

8.6

8.28.3

8.4

8.68.5

8.3

7.7

7.5

7.8

7.6

6.8




Grade Difference

8 0.37 0.5


Grade

Score

Point


6.5

7

7.5

8

8.5

9

8.6

8.28.3

8.4

8.68.5

8.3

7.7

7.5

7.8

7.6

6.8




Grade Difference

8 0.37 0.56 0.8


Grade

Score

Point


6.5

7

7.5

8

8.5

9

8.6

8.28.3

8.4

8.68.5

8.3

7.7

7.5

7.8

7.6

6.8




Grade Difference

8 0.37 0.56 0.85 0.6


Grade

Score

Point


6.5

7

7.5

8

8.5

9

8.6

8.28.3

8.4

8.68.5

8.3

7.7

7.5

7.8

7.6

6.8




Grade Difference

8 0.37 0.56 0.85 0.64 1.0


Grade

Score

Point


6.5

7

7.5

8

8.5

9

8.6

8.28.3

8.4

8.68.5

8.3

7.7

7.5

7.8

7.6

6.8




Grade Difference

8 0.37 0.56 0.85 0.64 1.03 1.7



3

4

5

6

7

8

9

10

11

12

8.68.2 8.3 8.4 8.6 8.58.3

7.7 7.57.8 7.6

6.8


2012 CMT_DAW (HAND-WRITTEN CENSUS)2012 MIST_DAW (COMPUTER PILOT)

Grade

Score

Point

Possible Contributing Factors to This Trend

Were the piloted (keyboarded) prompts more difficult?Was the novelty of computer testing a factor? Are the scorer’s showing a bias in favor of handwritten

text?Because of a lack of clear standards on the pilot, did the

time of administration of the pilot generally rob students of the full 45-minutes intended?

If time on task is a factor, will lengthening the standard time limit for a handwritten response be sufficient to overcome the differences for students who are typing?

Is keyboard familiarity (typing skill) a contributing factor?


Now that you have seen the data from the CMT DAW Pilot Test…





Connecticut’s Short-Term Plan

1) The Department would like to encourage equipped districts to administer more statewide tests on the computer before 2015.

2) Pilot data will show which of the test sessions can be equated to take into account the different difficulties of taking tests on the computer.

3) The Department plans to give districts some level of choice about which mode (paper-and-pencil vs. computer submitted) for students taking the census test.

Connecticut’s Short-Term Plan

4) Because selected forms will be equated, this means that a resulting scale score for the same test may have a different raw score as a result of mode. (written vs. computer submitted)

=

Further ResearchRESPONDER TIME: Should these CT findings be

reported to SBAC? Please use “TRUE” for YES and use “FALSE” for NO.

A repeated study could be done to see if the patterns are persistent over time. (Students may be improving on typing skill each year.)

It is possible to design a factor analysis to try to find out how much of the difference between the two modes is attributable to typing skill alone.

Online Resourceshttp://teachersites.schoolworld.com/webpages/y

ollis/typing.cfm Mrs. Yollis’ blog (above) sends you to these fine resources: Steve’s Review:

Learn to type AND Dance Mat Typing Game (same site)— By the BBC the best one here.I learn Tech…Website—hit the drop down “typing games.” the best ones are: typing chef keyboard climber game super hyperI also got addicted to “Typing Defense” Typer Shark game???—could not find it nor “Typing Lessons”

http://teachersites.schoolworld.com/webpages/yollis/typing.cfm

http://teachersites.schoolworld.com/webpages/yollis/typing.cfm

Dance Mat Typinghttp://www.bbc.co.uk/schools/typing/

Sequential lessons

Animal guides

Reward Screens

10-minute lessons

20 small typing tasks.

About a dozen lessons in the sequence.

http://www.bbc.co.uk/schools/typing/

Typing Chef http://www.sense-lang.org/typing/games/typingchef.php

Type the word in the bubble before it pops.

In later, more challenging screens other food falls into the pan.

Game works on speed of typing target words that get increasingly complex.

http://www.sense-lang.org/typing/games/typingchef.php

Keyboard Climber Gamehttp://www.tvokids.com/games/keyboardclimber

Help the monkey to climb ever higher by correctly typing the letter above him.

Clue to which hand to use based on left or right of screen.

Monkey falls to bottom of current level with each mistake.

Super Hyper Spider Typer http://funschool.kaboose.com/fun-blaster/games/game_super_hyper_spider_typer.h

tml Three levels of

difficulty.

Each progressively difficult within level.

Type the letters or words on the chameleons’ backs to protect the spider.

Different colored chameleons have different numbers of lives.

http://funschool.kaboose.com/fun-blaster/games/game_super_hyper_spider_typer.html

http://funschool.kaboose.com/fun-blaster/games/game_super_hyper_spider_typer.html

Typing Defensehttp://games.wordgames.com/media/typing-defe

nse.swfType the word

associated with each asteroid before it collides with the earth.

Planetary defense ray destroys correctly spelled asteroids.

Additional key words are earned like “slow, ice, and strike” as you increase in levels.

http://games.wordgames.com/media/typing-defense.swf

http://games.wordgames.com/media/typing-defense.swf


Now that you attended this session…Q: Will you fill out an evaluation sheet?

A. Yes!B. No!C. There is no evaluation sheet.

can third graders type?

Documents

older question

years of age

toolbar press

abc question

computer keyboard

graders type

onoff button

use responders