Item Selection and Ordering Information

September, 2014MAEIA Model HS Assessments

MAEIA Project Management Team2

Kathy Dewsbury-WhiteMAEIA Project Director

Ed RoeberMAEIA Assessment Director

Ana Luisa CardonaMAEIA Arts EducationConsultant Frank Stuart PhilipMAEIA AssessmentConsultant

Cheryl PooleMAEIA EditorKathy HumphreyMAEIA Field Test Coordinator and Project Management Support

Barb MicheluttiMAEIA PD and Project Support Director

Jason ODonnellMAEIA Administrative AssistantThree members were added to the Project Management Team in Year 2 once we realized the scale of the work ahead.2What is the Michigan Arts Education Assessment and Instruction Project (MAEIA)?3https://www.dropbox.com/s/zng8w2bu9pg3fbe/MAEIA%20Voiceover.mov

Open this link, download it to your desktop to see the 8.5 minute overview of the MAEIA Project. Be sure your sound is on. Youre part of something BIG!

4If you take out the brochure4

56This presentation is intended for use by teachers who registered for the 2014-15 MAEIA Model High School Arts Assessments field test, but were unable to attend a Briefing Session.

If you have not yet registered to participate in the field test, please do so by visiting our webpage.http://www.michiganassessmentconsortium.org/maeia-high-school-assessment-field-test-resources

Please review this entire presentation BEFORE you order assessment items.

This presentation refers to resources and documents located on the webpage noted above.

Information on who to contact for assistance is included at the end of this presentation.Important!Introduction to AssessmentStandardsDescription of Assessment TypesSamples of Teacher and Student BookletsExplanation of the Catalog of AssessmentsSelection of ItemsOrderingBefore, During, and After Assessment AdministrationReturn of MaterialsHow to Use the AssessmentsWho to Contact with Questions

Overview7Four disciplinesDance, Music, Theatre, and Visual ArtsAssessments can be used in several ways:To inform current instructionTo improve student learning and achievementFor future instructional program improvementAs one portion of educator evaluation

Intro to MAEIA Model Assessments8Based on MAEIA Performance StandardsAssess important content, knowledge and skills in the Michigan Arts Education Standards and Benchmarks 2011Organized as Create, Perform and Respond Performance StandardsItems also are aligned to the VPAA Credit Guidelines

Integral Role of Standards in the MAEIA Assessments910MAEIA Performance StandardsMichigan Content Standards and BenchmarksMichigan Grade Level Content Standards Layers of Standards

11The next six slides describes the types of assessments found in the catalogue of MAEIA Model Assessments.Coming up..12Performance Tasks Performance assessments carried out by individual students or small groups of students over time (days, weeks, months)

Performance Events Performance assessments that are administered on-demand, without any or just a brief amount of rehearsal time.

Types of Assessment ItemsChapter 4 Nature of the Assessment Items

Refer to the Art Education Hope and Visions Collage Performance Event we started with.1213Performance assessments carried out by individual students or small groups of students over time (days/weeks/months)Tasks may take place over an entire semester or trimester.Tasks are carried out in or out of class, and are very much related to instruction (e.g., class assignments)They measure essential outcomes in the content standards not easily measured in other waysPerformances are judged using one or more scoring rubrics

Performance Tasks14Performance assessments that are administered on-demand, with limited or no practice or rehearsal time (i.e., no more than a few minutes)May be individual or small group assessmentsTest administrator presents items to one student or a small group of students, who respond in real timePerformances are also judged using one or more scoring rubricsPerformance Events15Constructed ResponseItems in which students write a response to a prompt.Selected Response Items related to other items in which the student is given a prompt (a question or a statement) and answer choicesBoth C-Rs and S-Rs serve to illuminate declarative knowledge nested inside Tasks and Events.

Types of Assessment Items contdChapter 4 Nature of the Assessment Items

1516Items in which students write a response to a promptUsually can be administered to groups of students togetherSome type of stimulus (e.g., music selection, video, or picture) could be usedResponse may involve writing, sketching, constructing a table, as well as a written responseResponses are judged using one or more scoring rubrics

Constructed-Response Items17Items in which the student is given a prompt (a question or a statement) and answer choicesStudent has to select either the correct answer or the most correct answerMultiple-choice questions are the most popular form of these itemsIn MAEIA, these are used to tap the content knowledge or other procedural knowledge needed to respond to PT/PE/CR itemsSelected-Response Itemsalso say for CRWe follow Best Practice by following the professional measurement standards that govern the development & use of assessments:

Are they written so that teachers and students understand them?

Do they yield assessable information?

Do the items assess what they are intended to assess?Have the scoring guides been reviewed and revised (can they be strengthened)?

Any assessment with stakes for teachers and/or students should be piloted or field tested before it is used to make important decisions. (AERA/APA/NCME Standards for Educational and Psychological Testing)Why do we field test assessments?1819The next four slides provides a look at the cover and the first three pages of a Teacher Booklet of a MAEIA Model Assessment.

The Student Booklet is nearly identical to the Teacher Booklet but does not have the teacher-specific information or the answer keys for the selected-response questions. Coming up..20

Samples of a Teacher Booklet- Cover Page

21M.E421 This is the item label. It refers to Music.Event #421

Make Connections is the name of the assessment.

High School Level 3 means it is a high school item written for the students who have had 3 years of work in the discipline.

Under Teacher Booklet, you find a list of what is included in the booklet.

Decoding the Cover Page22

23The table at the top shows the standards being assessed by this booklet.

Following the table is an overview of the assessment activity.

Page 2 also shows the suggested duration of this assessment, the list of materials and the set-up required.

Decoding the 2nd Page of the Booklet24

25This page shows the teacher directions.

It also defines how many parts this assessment has and shows, in bold text, the beginning of the script written for the teacher to read to the students.

Decoding the 3rd Page of the Booklet26You will turn to the Assessment Administration Manual (AAM) to read the important considerations to be made when selecting which assessments fit into your curriculum and instruction.Assessment Administration Manual

Read Chapter 4, pages 14-16, of the AAM.

Now that youve seen a sample booklet27Selecting the Assessments to Administer

Please read Chapter 4 of the Assessment Administration Manual

4.1Catalog of Available Assessments144.2Fitting the Assessments Into Your Instructional Program144.3Selecting When to Use the Assessments154.4Adding Additional Assessments154.5Deleting Planned Assessments164.6Questions or Concerns1628COLUMN HEADINGS OF THE CATALOGUE SEE SLIDE 29, 30, & 31 FOR DEFINITION OF EACH COLUMNDisciplineDance, Music, Theatre, or Visual Arts

Item Type and NumberThe code for each task (T) or event (E). Note: all High School assessments are the 400 series. This will distinguish them from grade K-2 assessments (100s), grade 3-5 assessments (200s), and grade 6-8 assessments (300s).

Short Item DescriptorA brief phrase or title of the assessment, intended to convey the essence of the item.

Grade RangeThis is always set at 4, i.e., high school.

Levels 1, 2 and 3This designates the level or levels for which the item was written. Performance StandardPerformance standards are discipline-specific and condense the Michigan Merit Curriculums five content standards into the three overarching performance standards of Create, Perform, and Respond. These performance standards were developed to serve as the basis for MAEIA model assessments that measure student proficiency in each arts discipline at each grade span - K-2, 3-5, 6-8, and 9-12. Introduction to the Catalogue of MAEIA Assessment Items29See Section 4.1, page 14 of the Assessment Administration Manual, for a printed copy of slides 21-22.29Content Standard and BenchmarkThese are the Michigan State Board of Education-approved standards for arts education, taken from Michigan Arts Education Content Standards and Benchmarks for Dance, Music, Theatre and the Visual Arts (2011).

VPAA GuidelineThese are taken the Michigan State Board of Education-approved Michigan Merit Curriculum Credit Guidelines for the Visual, Performing, and Applied Arts (2006).

Requires RecordingThis is a signal to a teacher using this item about whether students responses will need to be electronically recorded.

Total TimeThis is an indication of the number of 50-minute class periods that the assessment requires. This number may not be precise for two reasons:Your class period length may differThe time it takes to record and download the performances of all students in larger classes may require an extra class period (or two)Introduction to the Catalogue of MAEIA Assessment Items30Because students participate in the arts for different reasons and durations, assessment tasks and events were developed for three different levels of HS arts students:

Level 1Students who fulfill their one credit visual, performing, or applied arts (VPAA) high school graduation requirement only, or who are in their first year of a multi-year VPAA program.Level 2Students who have already completed their first year in an arts discipline course and are now in their second year of instruction in the same arts discipline.Level 3Students who have already completed their first and second years in an arts disciplines courses and are now in their third year or fourth year of instruction in the same arts discipline.

If a student takes a year of instruction in one discipline (e.g., music) and another year of instruction in another discipline (e.g., theatre), they would participate in Level 1 assessments in each discipline. Teachers have the flexibility to adjust the assessments to match the instructional levels of the students being assessed.Assessment Levels31See Section 3.6, page 13, for a printed copy of the Levels. 3132Click on this link and scroll down to find the short description catalogue of items for your discipline: http://mi-arts.wikispaces.com/MAEIA+Project+Documents

Then follow directions for Steps 1-3 on slides 34 -36.Coming up..Step 1Look at the Catalog for your discipline and the Short Item Descriptor of the Tasks and Events. Make a notation of the Tasks or Events that interest or intrigue you, i.e., that you would like to know more about.

Steps in Selecting Items33Taken from Item Selection Guidance (ISG) document

33Step 2Now, click on the hyperlinks for the Short Item Descriptors in the Catalogue for the items you selected in Step 1.

Check the: Michigan Student Learning Standards Assessed, LevelOverview and Outline Suggested Total Time.

Select the Tasks and Events that logically fit into your existing curriculum and instructional plans.

Make a notation of the ones that you confirm fit your instruction, curriculum and assessment plan.

Steps in Selecting Items34Taken from Item Selection Guidance (ISG) document

34Step 3

Enter the item type, number, the course in which you plan to use it and the number of students in that course.

Check which instructional cycle you will be using for the assessment.

Check to see that the items you have selected will result in a reasonable assessment plan for your coursesThis is the assessment field test plan. Keep this for your records.Steps in Selecting Items35Taken from Item Selection Guidance (ISG) document

3536Read the review on slides 38 - 42. Just before orderingWe are expecting teachers to select a small number of Events and Tasks to use throughout the school year (OctoberApril).

We are suggesting that items be used within three instructional cycles, to help you pace your use of the Events and Tasks throughout the school year:October 15 December 15, 2014 (fall)January 7 February 27, 2015 (winter)March 2 April 17, 2015 (spring)

We are suggesting 1-2 Tasks and 1-2 Events per Instructional Cycle. This is a recommendation, not a requirement; field testers may select fewer (or more) items.

37In review

Taken from Item Selection Guidance (ISG) document. 3738After using and scoring the assessments yourself, you will return student responses to us to aid in refining the items and enhancing the scoring rubrics.

Your feedback on the uses and usefulness of the assessments is very important to us. We look forward to your feedback as you finish using the assessments during each instructional period and at the conclusion of the field test.

There is a separate Teacher Booklet for each assessment. Each item is self-contained.

You will be sent a sufficient number of Student Booklets (plus a small overage) and couple of copies of the Teacher Booklet for each assessment item you select

You will receive an Assessment Administration Manual (AAM) with overall field test directions in it. The AAM discusses general assessment topics, including actions you should take before, during, and after assessment. We encourage you to select a mix of Create, Perform and Respond Items. Combined, these strands are the creative process (in Visual Arts, we think of Perform as Present.)

Any additional materials needed to administer an item is noted.

Recording of student responses is also noted.39Taken from Item Selection Guidance (ISG) document, page 2.

3940Modifications are fair game.

You must record any modifications that you make and return those notes to MAEIA for refining the assessments. Use the Item Review and Comment Form

Examples of modifications might include:Changing genre of images, music, theatre and dance examplesAdding criteria to scoring rubrics Revising to better fit your instruction

Any modifications will be recorded on a form titled Item Review and Comment Form

Modifications of Items

41Item Review and Comment FormAttachment B in the Assessment Administration Manual.

4142Ready to order the items youve selected?Proceed to the next slide for the web address and directions for ordering. 43Ordering Your Items via an Online Order Formhttp://www.michiganassessmentconsortium.org Click on the MAEIA tab at the top.

Then click on the Field Test Resources link in the middle of the page

And finally, in the middle of that page, click on Model High School Assessment Order Form The next 5 slides will show you what you will see when you open the online order form.

When you click on the link for the Order Form, first you will see some introductory pages directions and a few words about the assessment items.Then you will see this page. This page is VERY IMPORTANT.4344

First, you will advance through two pages before you will enter contact and shipping information. This is very important!This screen continues in the actual order form.When you click on the link for the Order Form, first you will see some introductory pages directions and a few words about the assessment items.Then you will see this page. This page is VERY IMPORTANT.4445Youll be asked for your district and building codes. If you do not have them, leave this blank. 46Ordering Your Items

Next, you will select the discipline in which you will be field-testing assessment items.Next, you will select the ONE arts discipline in which you will be assessing students.4647Ordering Your Items

Select the tasks and events.Click on the little arrow that looks like a v to select number of students.Select instructional cycle(s).After selecting your discipline, you will see screens that look like this for all the Tasks and all the Events for the discipline you selected. TASKS are presented first; they are organized by the standards of perform, create and respond.Then, the EVENTS are presentedalso organized by perform, create, and respond.

For each item you want to order, select the instructional cycle(s) and select the number of students ACROSS all the cycles.

4748Ordering Your Items

This is the last screen you will see before you submit your order. Please be sure your order is correct before you click SUBMIT. Thank you!49Once youve submitted your order, a PDF of your order will appear on the screen.

You can download it to your computer and save it.50

Before leaving this presentation you must refer again to the Assessment Administration Manual (AAM). Click on the link below.

Assessment Administration Manual

Assessment Administration ManualCHAPTER 5 Read pages 17-26 for the overview and before, during and after assessment for step by step instructions for

Preparing for the assessment.Administering the assessment.Packing and returning student results, booklets and your comments and modifications.

51Taken from the Assessment Administration Manual.5152The next 15 slides will summarize the all-important Chapter 5 of the Assessment Administration Manual which provides the specifics of what you will do before, during and post administration of the assessments of your choice.

Coming up.Inventory the assessment materials you were sent. Do you have enough Student Booklets? Order extras from DRC is needed.

Read through each assessment one or more times.

Practice administering the assessment with a colleague or to yourself

Assemble other materials needed (if any).

Decide whether the assessment can be used as is or needs to be modified. You may modify them if necessary.

Determine how students responses will be recorded and downloaded (if necessary)

Determine when the assessment will be usedBefore Administering the Assessment53Stress that it is important for the teachers to be completely familiar with each assessment before they use them.

Teachers may modify the assessment to better fit their planned instruction.53Observe students during the assessment and note any issues, confusion, or questions on their part on the Item Review and Comment Form sent with the shipment of assessment materials

Collect the assessment materials between class periods and at the conclusion of the assessment

Organize the assessment materials for return to DRC

During the Assessment5455Assessments may require recording student responses in image, audio or video files. Set up recording devices when required.

Record audio, images and/or video with smart phones or other recording devise; send digital files via email to get them to your computer. Then copy to the flash drive sent with your assessment.

Be sure to record responses and save them during the assessment in preparation for submitting them at the conclusion of the assessment.

One flash drive is sent for every 25 students are sent in the assessment shipments for this purpose.

Recording Student Work56See Attachment D on page 13 of the Assessment Administration Manual for instructions for recording student results.

Recording Student Work

57Item Review and Comment FormAttachment B in the Assessment Administration Manual.

57Score students responses using the Teacher Scoring Rubric(s) provided; note any issues with the rubrics

Complete your feedback on the assessment using the Item Review and Comment Form

Mark up a copy of the Teacher Booklet and/or Student Booklet with your comments

Upload recorded student responses to the flash drive(s) provided (See Attachment D of AAM)These are secure assessment materialsPut all responses to one assessment item on a flash drive (one flash drive has been sent to you for each 25 students indicated on your original order)Put the flash drive(s) in the envelope provided; fill out the label(s) provided and seal the envelopePost Assessment Activities58Place the loose ancillary student responses in the envelope provided and complete the Ancillary Student Materials Return Envelope label.

Ancillary materials include the following:Assessment Questions pages that students are instructed to tear off and give to you at the start of many of the assessmentsDrawings/paintings done by studentsPhotographs of student work in lieu of return of large, bulky responses and/or 3-D objects (e.g., cans or kites)Any other student responses not done in their Student Booklets Post Assessment Activities59

60Use the appropriate, identified envelope when returning assessment results.Taken from page 23 of the Assessment Administration Manual.

60

61Taken from page 25 of the Assessment Administration Manual.

61

62Taken from page 24 of the Assessment Administration Manual.

62Place the Item Review and Comment Form and the marked up Teacher Booklet and/or Student Booklet with your comments in the envelope provided. Fill out the label to indicate the assessment

Include a copy of any revised Teacher Booklet and/or Student Booklet that you have created in the same envelopePost Assessment Activities63Returning materials to DRC64Packaging the materials for return to DRC:Use sturdy clean boxes to ship the materials via UPS (e.g., the boxes the assessment materials were sent to you in)Send each assessment after you have given it and scored students responsesUse the pre-paid labels to ship the materials to DRC at no cost to you or your schoolIf the assessment return requires more than one box, label each 1 of X, 2 of X, etc., with X = total number of boxesFill out a Materials Return Cover Sheet and place on top of the first box.65Materials Return Cover Sheet

Attachment C in the Assessment Administration Manual, page 1. 6566Modifications of ItemsDescribe in detail any modifications you make.

Attachment C in the Assessment Administration Manual, page 2.

6667

Order of loading materials in boxes:Taken from page 26 of the Assessment Administration Manual.6768The next 3 slides provide suggestions for and cautions about using the MAEIA Model Assessments for Educator Effectiveness. Coming UpHow the MAEIA Assessment Results Might Be Used for Educator EffectivenessIf you feel that the assessment represents a good assessment of students (that is, that students understood what to do and were able to respond to the assessment), then the assessment results may represent their current achievement

Teachers can select students responses to represent their work with the class (to show what the class was able to do) as well as individual students (those whose performance was exceptional or students who struggled but were still able to complete the assessment) How to Use the Results for Educator Effectiveness69Teachers can also select examples of student work from each instructional period to illustrate changes in students work over the school yearStudents whose performance was marginal in the fall who are now doing exceptional workStudents who struggled to perform at all who are now doing acceptable (or better) workStudent who still need help and how you are working with them

The teacher and the school administrator can use this work as one piece of evidence of student learning and achievementHow to Use the Results for Educator Effectiveness70Other sources of achievement data (e.g., students responses to other MAEIA assessments as well as to other measures or indicators of achievement) should be used as well

Prepare corresponding narratives regarding teacher practices to accompany student assessment results

These achievement data along with appropriate observational data should be used in the overall evaluation of an educator

The goal of such educator evaluation should be primarily improvement of educator practiceHow to Use the Results for Educator Effectiveness7172The next two slides provide final comments about appropriately using the results of the MAEIA Model Assessments.Comments and CautionsRemember, this is the first time these assessments have been given to students.

The assessments may have issues that prevent students from showing you (and us) what they have learned and can do. (The purpose of field testing is to uncover these issues and to correct them).

The assessment items as written may not fit well with the instruction planned and implemented by the educator.

Even if the assessment item worked well and fits with the educators instruction, no single data element should be used to make important decisions about educators or students.Cautions about Using the Results7374The MAEIA assessments are NOT designed to be used as pre and post assessments.Another Caution75Know that you are serving an important role by field testing the MAEIA High School Items.

You can expect your order of assessments within the next 7-10 days.

Check your order right away to make sure that youve received the correct item(s).

Retain the shipping boxes for returning the assessment results.

Contact us right away if you have questions or corrections. Now that youve orderedMAEIA Field Test Resource Pagehttp://www.michiganassessmentconsortium.org/maeia-high-school-assessment-field-test-resourcesJason ODonnell517.327.9224517.327.0779 (fax)jodonnell@michiganassessmentconsortium.org Jason will forward each message to the most appropriate person to answer your question(s).

Thank you for volunteering to participate in the field test!

Contact Us for Help:76