1

Michigan Merit ExaminationEdward Roeber

Michigan Department of Education

2

Issues Leading Up to the MME

• High school principals organization felt MEAP testing took too long (days and hours)

• MEAP assessment program suffered an operational meltdown in 2003

• Although students can receive up to $3,000 towards post-secondary education, student didn’t take the MEAP test seriously

• Participation in the MEAP assessment was too low - especially in suburban districts

3

Proposed MME Legislation

• Students would take the following tests– College-entrance test and a work skills assessment

acceptable to employers, plus and Michigan items needed to secure Federal approval

– Testing would occur in the spring of the junior year– Retesting in fall and spring of senior year– One free retake

• College-entrance readiness tests (PLAN and PSAT) used for dual enrollment eligibility purposes

4

Arguments in Favor of MME

• Students will receive a “free” college-reportable score on a college entrance test - this would motivate students to participate in and do well on assessments

• Students will receive regular work skills scores in reading and mathematics - useful in employment

• Michigan Merit Exam scores could still be used to award post-secondary scholarship in Michigan

• Lower cost and less testing time

5

Arguments Against MME• Little standards-based information from a college

entrance and work skills tests - no data for student or school improvement

• Test administration procedures are very rigid • Can all students score “above average?”• Since ACT test scores have not changed much

over the past decade, how will schools make AYP?

• MME is far more expensive - $73 versus $19/student

6

MME Legislation• PA 592 of 2004 - MME inserted into Career and

Technical Preparation Act

• PA 593 of 2004 - Replaced high school MEAP with MME in School Aid Act

• PA 594 of 2004 - Replaced high school MEAP with MME in Postsecondary Enrollment Options Act

• PA 595 of 2004 - Replaced high school MEAP with MME for determining Merit Award Scholarships

• PA 596 of 2004 - Replaced the high school MEAP with MME in the State School Code

7

Steps Used to Implement MME

• Visited potential vendors to share ideas

• Competitively bid the MME program

• Reviewed the bids

• Selected new contractor(s) - Pearson/ACT

• Determined the content of each MME test

• Conducted pilot test of 37 high schools (statistical alignment)

8

Steps Needed to Implement MME

• Conducted additional content alignment studies to select MME items

• Prepared revisions to Michigan’s Accountability Workbook

• Submitted application to USED in July, 2006

• Conducted MME standard setting for NCLB purposes in July

• Set PLAN/PSAT cuts for dual enrollment

9

Steps Needed to Implement MME• November 1, 2006 was the “drop-dead date” for

the MME versus MEAP decision for Spring 2007– Receive approval from USED prior to drop-dead date,

OR

– continue MEAP High School Assessment (if approval is not received by that date)

• If approved by USED, implement the MME in grades 11 in the 2006-07 school year; maintain MEAP HST for Grade 12 only

10

Steps Needed to Implement MME

• Was the USED approved the MME?• November 1, 2006 letter from USED:

“. . . It is likely to be fully compliant. . However, additional evidence is needed to show how Michigan’s new assessments meet the standards and assessment requirements under ESEA.”

11

Evidence That Must be Provided

• Evidence of SBE adoption of achievement standards for the MME

• Documentation of technical quality indicators, except standard setting and materials previously submitted, as listed in the Peer Review Guidance for the MME as administered in spring 2007.

12

Evidence That Must be Provided

• Final technical manuals for the 2006-07 MME and MI-Access assessments.

• Actual MME assessment reports of results at the student, school, and state levels that have been provided.

13

So…it’s a Go!• First MME administration - Spring 2007• Students to be assessed: Grade 11

– All students enrolled in Grade 11 during March 2007 take the complete Michigan Merit Exam

– Including students who previously took the MEAP in grade 10 to qualify for dual enrollment

14

MME Assessment Design• Grade 10: College entrance readiness test (PLAN

and PSAT - local school/student decision)• Grades 11 and 12: Michigan Merit Examination

– English Language Arts - ACT, WorkKeys and a Michigan Social Studies item

– Mathematics - ACT mathematics, a few items from the ACT Science section, WorkKeys and some Michigan items

– Science - ACT test and Michigan items– Social Studies - all Michigan items

15

MME Assessment Design

Assessments Administered

Contributes to an English

Language Arts Score

Contributes to a Mathematics

ScoreContributes to

a Science Score

Contributes to a Social Studies

Score

•ACT Plus Writing

•WorkKeys Applied Mathematics & Reading for Information And Michigan Mathematics

•Michigan Science and Social Studies

Total ELA Score

Reading•ACT Reading (MC)•WorkKeys Reading for Information items (MC)

Writing•ACT English (MC)•ACT Writing (CR)•Michigan Social Studies CR item -writing score

•ACT Mathematics (MC)

•WorkKeys Applied Mathematics (MC)

•Selected ACT Science items (MC)

•Michigan items (15 MC)

•ACT Science (MC)

•Michigan Science (55 MC)

•Michigan Social Studies (57 MC and 2 CR - social studies score)

Note: The number of Michigan items may increase in order to field-test new items to replace those released annually.

16

MME Test Organization    

Day Test Subject Session PartsTotal Items or

PromptsTesting Time

(minutes)Total Admin Time

Day 1ACT Plus Writing

English

5

75 45

Reading 40 35

Writing 1 30

Mathematics 60 60

Science 40 40

Day 2

WorkKeys

Reading for Information

3

33 45  

Applied Mathematics 33 45  

MichiganMathematics 14 20  

Days 2,3 or 4

Michigan

Science 1 46 45  

Social Studies 2

32 50  

  27 50  

TOTAL Minutes 465

TOTAL hours 7.75

17

Special Populations

• Students with Disabilities• Students with Section 504 Plans• English Language Learners

18

Accommodations

• ACT-approved– Available to students with disabilities and

Section 504 plans– Apply only to the ACT

• State-allowed– Available to students with disabilities,

Section 504 plans, and English language learners

– Apply to all parts of the MME

19

Accommodation Requests

• ACT-approved versus State-allowed• For the ACT

– 7,934 ACT-approved– 456 denied– 1,565 state-allowed materials requested

• For State-allowed accommodations no approval is required

20

MME Accommodation Summary Table

21

MME Schedule for 2006-2007• MME assessment administration dates:

– MME administered statewide• March 13 - ACT • March 14 - WorkKeys and Michigan Mathematics• March 14-16 - Other Michigan tests

– Make-up testing scheduled two weeks later• March 27 - ACT • March 28 - WorkKeys and Michigan Mathematics• March 28-30 - Other Michigan tests

– Auxiliary test centers offered for non-public students

22

MME Activities for 2006-2007

• September-October, 2006

– Conducted high school video test briefing on September 12

– Established 1,111 high schools as test centers - public schools, charter schools, and private schools

– Schools identified key testing staff

• Test Supervisor

• Backup Test Supervisor, and

• Test Accommodations Coordinator

23

MME Activities for 2006-2007• November, 2006-February, 2007

– High school Test Supervisor, Backup Test Supervisor, and Test Accommodations Coordinator training at 12 sites (about 3,400 testing staff)

– Schools apply for accommodations for students with disabilities and English language learners (both ACT- and state-approved ones)

– ACT reviews accommodation requests and determines those eligible for college-reportable or state-reportable scores

– Auxiliary test centers set up for non-public school students who attend a non-public school that was not established

24

MME Schedule for 2006-2007• February-March, 2007

– Tests (ACT and Michigan) produced

– Accommodations determined and materials ordered

– Non-secure and secure materials shipped to high schools

– Students complete the demographic and non-cognitive sections of the answer folders for the MME

– High school Test Supervisors train room supervisors and proctors

– MME assessments administered

25

MME Schedule for 2006-2007

• April-July, 2007– Students and schools receive reports of results

• Regular ACT reports (college-reportable)

• WorkKeys reports

• MME score reports

– MME scores used for state accreditation and NCLB AYP accountability purposes

26

MME Schedule for 2006-2007

• August-November, 2007– Retake vouchers produced, sent to schools,

and given to students

– Students register for MME retake via ACT

– Retake on a single fall nationally-scheduled ACT testing Saturday administration - fourth Saturday in October

– WorkKeys and Michigan tests administered the following week in schools

27

MME Issues and Challenges• Getting USED to approve an ACT-based

assessment for NCLB AYP purposes– How much augmentation is needed?– Non-alignment of the WorkKeys tests

• Alignment of the MME tests to Michigan’s standards - determining needs to be added– Changing nature of the ACT tests

• Preparing to administer the ACT in Michigan’s schools– Test center establishment– Testing staff training– Accommodations

28

MME Issues and Challenges

• Preparing the IT work for blending multiple assessments into a coherent assessment– Different psychometric models – Two contractors’ systems with Michigan’s systems

• Assuring adequate assessment administration preparation - for very secure tests (ACT)– Moving from an ACT-administered test to Michigan

educator-administered tests

29

MME Issues and Challenges• Accommodations for students with disabilities

– What accommodations will be offered and under what conditions?

– How much work is it to get accommodations approved?

– Will MI-Access need to be modified and/or expanded?

• Participation of English language learners– What accommodations will be offered and under

what conditions?– Will any ELL accommodations result in college-

reportable ACT scores?– Why can’t ELLs use the same accommodations as

SWDs but not receive college-reportable scores?

30

MME Issues and Challenges• Working through the logistics of testing

– Multiple (3) answer folders– Inability to pre-ID students– Blended reporting of multiple tests– Alignment of the tests to a state’s standards

• Working with ACT– Attempting to modify established procedures

• Electronic registration of students, key testing staff and schools

• Updating information on staff and schools electronically• Considering modern psychometric methods• Blending ACT’s procedures with those of states

31

MME Issues and Challenges

• States using ACT have recently formed an “ACT Users Group” – Identified a number of challenges that

states would like to work on together– Identified several issues that states would

like to work on with ACT– Next meeting is May 17-18 in Detroit

32

For Questions and Comments

Edward Roeber(517) 373-0739 voiceRoebere@michigan.gov

Email Questions to mme@michigan.govMME Website: www.michigan.gov/mme