College Ready

Career Ready

National Adult Education College and Career Readiness Training Design Initiative

Next Generation AssessmentPresentersBonnie Goonen - bv73008@aol.comSusan Pittman-Shetler - skptvs@aol.com

Rationale underlying new assessment tools

Key points for discussion during training sessions

Webb’s Depth of Knowledge (DOK) and related tasks

Insight into how you can begin to apply the information to build on instructional approaches

Resources for trainer use during future sessions

Focus of the Train-the-Trainer Session – Part 1

2

• Changes occurring in the landscape of

education and the workforce require a new kind of test

• All to ensure that the adult education high school credential remains meaningful for adult learners, employers, and institutions

A new test in 2014: Why?

3

The Task Ahead

Steps I

Can Take

New Standards

New Assessment

2014

What are you doing to prepare for the implementation of college and career readiness standards?

How are you preparing instructors for the increased rigor of the new assessment?

What are your programmatic goals for change in 2013? 2014?

What do you need to help you move forward?

How will you make the difference?

It’s All in the Planning!Time Frame Objective to Meet

ConcernActivities Completed Additional Information

April – June 2013  Plan for professional development system for

transitioning instructors to college and career ready

standards-based instruction  

      

 July – December 2013        

January – June 2014

July – December 2014

NEW REALITY #1OUR WORLD HAS CHANGED

7

• Postsecondary education and training• Academic knowledge and skills• Practical literacies: The ability to use and apply the

knowledge of math, language arts, science, civics etc. to meet real-world challenges.

• Broader competencies: Critical thinking and problem solving, communications and collaboration, creativity, self-sufficiency etc.

More Important in the 21st Century

63% of all jobs will require some college or better by 2018

1973 1992 2007 20180%

10%20%30%40%50%60%70%80%90%

100%

Graduate's DegreeBachelor's DegreeAssociate's DegreeSome CollegeHigh School DiplomaNo High School Diploma

Labor force Labor force Labor force Labor force 91 million 129 million 154 million 166 million

Projections of Jobs and Education Requirements Through 2018. Center on Education and the Workforce (June 2010)

Education, Job Openings, and Unemployment in Metropolitan America. (2012). Brookings Institute

http://www.brookings.edu/research/papers/2012/08/29-education-gap-rothwell#M10420

Further evidence to support the need to educate adults so that they are well prepared for postsecondary education so that they can succeed in today’s workplace.

What does the workplace need?

Time Out for a Video

• For the workplace by connecting skills required for entry-level positions in the 21st century workplace to our curriculum

• For postsecondary education by connecting concepts learned to concepts necessary for successful entrance

• For real-world situations by actively engaging students in contextualized reading, mathematics, problem-solving, and communication activities

Our Goal: Preparing Students

• 60% of enrollees at community colleges need remediation (70% of those need math remediation)

• National studies have shown that two-thirds of students who take remedial classes never graduate

• Students needing one or more remedial math classes have a 90% drop-out rate

• Employers estimate that 39% of high school graduates who have no further education are not prepared for their current job and that 45% are under prepared for advancement.

Michael Kirst, Stanford University Study“Rising to the Challenge: Are high school graduates prepared for college and work?” Achieve,

Inc., 2005

Are Students Prepared?

ConnectionsThere is a strong correlation between education, training, career success,

satisfaction in life,

. . . and personal income.

1. To provide results leading to the award of a high school equivalency credential

2. To provide evidence of readiness to enter workforce training programs or postsecondary education

3. To provide actionable information about a candidate’s academic strengths and weaknesses

Purposes of the new GED® test

15

Improved

• What is the role of state staff in determining career and workplace needs?

• What is the role of the program manager in determining how the needs of the workplace will be met through instruction?

• What is the role of the teacher?

• What is the role of the student?

Next Steps - Different Roles

16

NEW REALITY #2TECHNOLOGY IS ESSENTIAL

17

• How many gadgets do you have?– mp3 or iPod– E-book reader– Tablet, such as an iPad– Laptop computer– Smart phone– Cell phone

New Realities

18

Would you really want to go back?

19

Technology is EVERYWHERE!• Today…

– Most job postings are exclusively online– Most job applications are completed online– Most job responsibilities have a technology

component built in• In the next decade…

– Career opportunities will be created by technological advances

Today’s Realities

20

Earning differentials of workers who use computers

21

Source: Rainie, L. Digital differences and money. Pew Research Center (2012)

• Enables measurement of concepts and/or skills that cannot be fully or appropriately captured by paper based tests (Bennett 2002; Parshall, Harmes, Davey, & Pashley, 2010)

• Improves measurement by increasing the precision or efficiency of the measurement process (Parshall, Spray, Kalohn, & Davey, 2001; van der Linden & Glas, 2000; Wainer, 1990)

Computer-based assessment . . .

22

New Realities: Question Types

23

GED® test HiSET™ TASCConstructed Response

Extended ResponseShort Answer

Multiple choice Technology-Enhanced Items

Fill-in-the-blank itemsHot-spot itemsDrag-and-drop itemsDrop-down selection items

EssayPersuasive

Multiple choice

Writing Prompt Informative/explanatory

Multiple choiceGridded response items

Computer Skills

Other Tools• Basic keyboarding• Cut• Copy• Paste• Undo/Redo• Insert• Enter – hard return• Spacing• Backspace• Highlight

Directional Tools• Previous/Next• Close• Minimize• Page tabsResource Tools

• Virtual Calculator• Calculator Reference Page• Formula Page• AE Symbol• Item Review/Flagging

Word Processing Skills

• Helps motivate students, especially Millennials and Gen Xers

• Builds collaboration skills for students• Requires higher-order thinking and problem-

solving skills• Develops skills for postsecondary education and

the workplace• Assists students in being successful on the high

school completion assessment

Why Integrate Technology?

25

Teaching technology is no more optional than teaching students how to use a pencil.

Technology in Today’s Classroom

26

27

“If a teacher today is not technologically literate - and is unwilling to make the effort to learn more - it's equivalent to a teacher 30 years ago who didn't know how to read and write.”

Karl Fisch (2007)

In order to teach it, we have to do it!

• Teach mouse and keyboarding skills• Integrate technology as a “normal” part of

the curriculum• Look at technology as more than just an

ability to use a computer– E-readers – Tablets – Smart phones

• Incorporate different question types using the technology of the 2014 GED® test and other computer-based assessment tools

A Few Strategies to Get Started

• What is the role of state staff in supporting the integration of technology state-wide?

• What is the role of the program manager in integrating technology into the adult education program?

• What is the role of the teacher?

• What is the role of the student?

Next Steps - Different Roles

29

NEW REALITY #3DIVERSE GENERATIONS

30

The Power of

Four!

Our current GED® test candidates

32

Millennials (16-30)75%

Gen Xers (30-47) 21%

Others (47+)4%2011 GED Testing Program® Statistical Report

What is a generation?

TraditionalistsBaby Boomer

Generation X

Generation Z

between 1965 & 1982

between 1927 & 1945between 1945 & 1964

between 1982 & mid 2000s

Millennial

G. I.’sbetween 1901 & 1926

between mid 2000s – 2020?

• ~40% of the population under 25• 95% of 18-29 year olds use the Internet• 81% of teens play games online• 76% of online teens get news online• 53% have made purchases online• 41% use the web to get health information

Pew Research Center (2009)

Coming to a Center Near You!

New Realities

35

Source: Zickuhr, K. & Smith, A. Digital differences. Pew Research Center (2012)

New Realities

36

Source: Zickuhr, K. & Smith, A. Digital differences. Pew Research Center (2012)

• Teachers tend to teach– in their personal learning style– by the methods by which they were taught– by the generation in which they were born

• Students prefer teachers who teach– the way they (the students) learn– by the techniques of the generation in which they

were born

What We Know

• What is the role of state staff in supporting the integration of differentiated instructional methods state-wide?

• What is the role of the program manager in determining what differentiated curriculum/ strategies/methods are most effective for different generations?

• What is the role of the teacher?

• What is the role of the student?

Next Steps - Different Roles

38

NEW REALITY #4STUDENTS MUST HAVE HIGHER-ORDER THINKING SKILLS

39

From Bloom to Webb – Cognitive Rigor

Cognitive Rigor: Blending the Strengths of Bloom's Taxonomy and Webb's Depth of Knowledge to Enhance Classroom-level Processes

http://standardsco.com/PDF/Cognitive_Rigor_Paper.pdf

What is Depth of Knowledge (DOK)?

41

Adapted from the model used

by Norman Webb to align standards with

assessment

Focuses on content standards

in order to successfully complete an assessment

item/task

Descriptive, not a

taxonomy

Not the same as

ability levels

Webb’s Depth of Knowledge Model

42

Level 1: Recall

Level 2: Skills and Concepts

Level 3: Strategic Thinking

Level 4: Extended ThinkingD

epth

of K

now

ledg

e

Level 2: Skill ConceptA. Focus is on applying skills and concepts (in a familiar/typical situation), relationships (compare, cause-effect), main ideas. B. Requires deeper knowledge than definitionC. Explaining how or why D. Making decisions E. Estimating, interpreting in order to respond F. One right answer

Level 4: Extended ReasoningA. Requires complex reasoning, planning, and thinking (generally over extended periods of time) for the investigation. B. Assessment activities have multiple steps with extended time provided. C. Students may be asked to relate concepts within the content area and among other content areas.D. Students make real-world applications in new situations.

Webb’s Depth of Knowledge

DOK is not about difficultyDifficulty is a reference to how many students answer a question correctly.• How many of you know the definition of exaggerate?

DOK 1 – recall– If all the students know the answer, then it is easy.

• How many of you know the definition of pellucid? DOK 1 – recall– If most do not know the definition, this question is

difficult, but that alone does not change the DOK level.

• The intended student learning outcome determines the DOK level.

• Instruction and classroom assessments must reflect the DOK level of the intended learning outcome.

DOK is about complexity

• DOK 1 requires recall of information, such as a fact, definition, term, or performance of a simple process or procedure.

• Answering a Level 1 item can involve following a simple, well-known procedure or formula.

Recall: DOK Level 1

45

• Recall facts • Apply a formula• Describe features or

characteristics• Perform a process or set

of procedures

DOK Level 1 Examples

Sample Level 1 GED® Question

47

List

Recall

• DOK 2 includes mental processing beyond recalling or reproducing a response. Items require students to make some decisions as to how to approach the question or problem.

• These actions imply more than one mental or cognitive process/step.

Skills/Concepts: Level 2

• Identify and summarize information from a text

• Compare and contrast

• Explain cause-effect

• Predict a logical outcome

• Classify geometrical figures

• Retrieve information from a graphic and use it to solve a problem requiring multiple steps

DOK Level 2 Examples

Sample Level 2 GED® Question

50

Comprehend

Synthesize

Describe (why)

• DOK 3 requires deep understanding as exhibited through planning, using evidence, and more demanding cognitive reasoning. The cognitive demands at Level 3 are complex and abstract.

• An assessment item that has more than one possible answer and requires students to justify the response they give would most likely be a Level 3.

Strategic Thinking: Level 3

• Analyze or evaluate the effectiveness of literary elements

• Solve a multiple-step problem and provide support

• Compare actions and analyze their impact

• Develop a model for a complex idea

• Propose and evaluate solutions

• Explain, generalize, or connect ideas, using supporting evidence

DOK Level 3 Examples

Sample level 3 GED® test question

53

Synthesize

Reason

Evaluate

Support

• DOK 4 requires high cognitive demand and is very complex. Students are expected to make connections - restate ideas with the content or among content areas- and have to select or devise one approach among many alternatives on how the situation can be solved.

• Due to the complexity of cognitive demand, DOK 4 often requires an extended period of time.

Extended Thinking: Level 4

DOK Level 4 Examples• Gather, analyze, organize, and interpret

information from multiple sources to draft a reasoned report

• Analyze author’s craft (e.g., literary techniques, point of view, etc.)

• Analyze and explain multiple perspectives or issues within or across time periods, events, or cultures

• Specify a problem, identify solution paths, solve the problem, and report the results

• Write and produce an original work

DOK Levels Can Be CumulativeStandard DOK Assessed DOK Needed

Analyze text(s) in order to identify, understand, infer or synthesize information

DOK 3 DOK 1 (read)DOK 2 (understand)DOK 3 (apply information)

Apply knowledge of sentence structure in composing or editing

DOK 2 DOK 1 (know parts)DOK 2 (write sentence/edit sentence)

Predict trends based on graphical representation

DOK 3 DOK 1 (determine how many)DOK 2 (compare)DOK 3 (make decisions)

Simplify and evaluate numerical and algebraic expressions

DOK 1 DOK 1 (solve)

Remember DOK is . . .

57

…a scale of cognitive demand…descriptive…NOT the same as difficulty…NOT the same as Bloom’s

Taxonomy

Can you identify the complexity of each of the following tasks?

Check Your Webb Knowledge

At what level would you be assessing students knowledge, if you had them . . .

Identify and summarize the major events, problem, solution, conflicts in a literary text.

Determine the area of a triangle given a drawing or labels.

Gather, analyze, organize, and interpret data from multiple sources to draft a reasoned report.

Analyze or evaluate the effectiveness of literary elements (plot, setting, conflict, point-of-view).

What’s the DOK Level?

2

1

4

3

• Use questions that require students to explain their answers

• Have students apply reading, writing, and mathematical skills using challenging content from all subject areas

• Use open-ended question formats

• Use and develop questions for class discussion and tests that are of the same cognitive rigor as the 2014 GED® test

A Few Strategies to Get Started

• What is the role of state staff in supporting the integration of DOK in both assessments and instruction?

• What is the role of the program manager in determining how DOK will be implemented in the classroom?

• What is the role of the teacher?

• What is the role of the student?

Next Steps - Different Roles

61

NEW REALITY #5MORE RIGOROUS CONTENT THAT MIRRORS THE REAL-WORLD NEEDS OF STUDENTS

62

New RealitiesAssessm

ents

Curriculum Design

Lesson Planning

Instruction

Student Learning

College and Career

Readiness Standards for

Adult Education

2014 GED® test OverviewModule

2002 TestTesting Time

2014 Test Testing Time

Reasoning Through Language Arts

185 minutes 150 minutes in 3 sections: [25 min + 45 min ER] +

[10 min. break] + [70 min]~51 items – 65 raw score points

Mathematics 90 minutes 90 minutes in 2 sections:[~12 min non-calc] + [~78 min]

~46 items – 49 raw score points

Science 80 minutes 90 minutes in 1 section~34 items – 40 raw score points

Social Studies 70 minutes 90 minutes in 2 sections:[65 min + 25 min ER]

~35 items – 44 raw score points

Total Battery ~ 7 hours ~ 7 hours

HiSET™ Overview

65

2002 Series GED® test

Test Time Number of Questions

Language Arts – Reading 65 minutes 40 questions

Language Arts – Writing Part 1 – 75 minutesPart 2 – 45 minutes

51 questionsEssay question

Mathematics 90 minutes 50 questions

Science 80 minutes 50 questions

Social Studies 70 minutes 50 questions

Shift 1: Complexity• Regular practice with complex text and its academic

language

Shift 2: Evidence• Reading, writing, and speaking grounded in evidence

from text, both literary and informational

Shift 3: Knowledge• Building knowledge through content-rich nonfiction

Shifts in CCR ELA/Literacy Standards

66

Shift 1: Focus• Focusing strongly where the standards focus

Shift 2: Coherence• Designing learning around coherent progressions level

to level

Shift 3: Rigor• Pursuing conceptual understanding, procedural skill and

fluency, and application – all with equal intensity

Shifts in CCR Mathematics Standards

67

Let’s Take a Closer Look

68

Creating a Master Curriculum Framework

Evidence of Learning

Content Areas and Topics

Essential SkillsAlignment

Instructional Plan Delivery Method(s)

Materials

Content Areas/Topics

KYAE Common Core State StandardsUnpacking Chart for Standards

Standards-in-Action: Innovations for Standards-Based Education, Unit 1, MPR Associates, Inc., Prepared for U.S. Department of Education, Office of Vocational and Adult Education, 2009

Unpack the Standards

70

1Standards

2Skills Included in Standard

3 Concepts Included in Standard

4Through a Particular Context

5 Cognitive Demand/ Levels of Thinking

6Sample Activity

Standard:

Developing Lessons

• What is the role of state staff in supporting the more rigorous College and Career Readiness Standards for Adult Education?

• What is the role of the program manager in determining how the College and Career Readiness Standards for Adult Education will be implemented in the classroom?

• What is the role of the teacher?

• What is the role of the student?

Next Steps - Different Roles

72

NEW REALITY #6RESOURCES PROVIDE SUPPORT FOR THE CLASSROOM

73

GEDTS - www.GEDtestingservice.com

74

Resources, Resources, Resources

75

Additional Resources

76

Online tutorials and training Test-taker resources One-stop shop for practice materials

Multimedia outreach Video profiles of success

Brand resources for local programs

Research on adult learners

HiSET™ - http://hiset.ets.org/

77

HiSET™ - http://hiset.ets.org

78

TASC

80

“High achievement always occurs in the framework of

high expectation.”Charles F. Kettering (1876-1958)

Our students need and deserve• A college and career ready credential• An educational environment where learning is

• accelerated;• contextualized;• results-oriented;• and leads students toward a career!

Remember, it’s all about our students . . .

Questions, insights, suggestions

82

83

Presenters

Bonnie GoonenTrainer/Consultant

bv73008@aol.com

Susan Pittman-ShetlerTrainer/Consultant

skptvs@aol.com

This workshop developed courtesy of GED Testing Service®.