test, measurement & research services september 20, 2010

44
An Exploratory Teacher Survey Related to the American Diploma Project Algebra II End-of-Course Exam Test, Measurement & Research Services September 20, 2010

Upload: others

Post on 09-Jun-2022

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Test, Measurement & Research Services September 20, 2010

An Exploratory Teacher Survey Related to the

American Diploma Project Algebra II

End-of-Course Exam

Test, Measurement & Research Services

September 20, 2010

Page 2: Test, Measurement & Research Services September 20, 2010

2

Abstract

A survey-based exploratory study was conducted to better understand the gaps between

curriculum and instruction, and what content knowledge is expected of students on the

American Diploma Project (ADP) Algebra II End-of-Course Exam. Specifically, the

intent was to understand which of the ADP Algebra II exam standards and benchmarks

teachers are covering in their classrooms and why many students may be performing

poorly on the exam. Analyses were based on the responses of 324 teachers who taught

Algebra II in several ADP Consortium states (Arkansas, Hawaii, Kentucky, New Jersey,

and Ohio). The survey results revealed that teachers underestimated the rigor of the

constructed-response items on the exam and were not always clear on the kinds of items

that are being used to assess certain ADP Algebra II exam benchmarks. In general, these

survey results may suggest that the ADP Algebra II exam standards are still being

implemented in the curriculum and instructional practices of the schools that are giving

the exam, and that teachers will benefit from professional development activities aimed at

increasing understanding of the ADP Algebra II exam standards and how these standards

are reflected in the exam content.

Page 3: Test, Measurement & Research Services September 20, 2010

3

The ADP Assessment Consortium has succeeded in creating a rigorous, high-

quality Algebra II assessment that will not only inform students whether they have

mastered the content of the course they have just completed but also given an indication

of preparation for higher-level mathematics courses. Now that the rigorous exams with

high standards are in place, states are turning their attention to investigating the

consistency of content and rigor within and across states in their courses and seeking

strategies that can improve the quality of curriculum and instruction in Algebra II courses

within and across states.

To assist in these goals, Pearson developed and administered a survey to teachers

related to the content of the Algebra II exam. The purpose of the teacher content

evaluation study was to better understand the gaps between curriculum and instruction,

and what content knowledge is expected of students on the test. Specifically, the intent

was to understand which of the ADP standards and benchmarks teachers are covering in

their classrooms and why many students may be performing poorly on the exam. The

teacher content evaluation survey was administered online to Algebra II teachers in a

subset of the ADP Consortium states. The specific research questions were as follows:

1. Where are there gaps between curriculum and instruction, and the exam?

2. Why do teachers suspect students may not be able to answer some ADP Algebra II exam items correctly?

Page 4: Test, Measurement & Research Services September 20, 2010

4

Methods

Survey

An online survey was administered to Algebra II teachers May 3-28, 2010. The

survey took teachers approximately 30 minutes to complete and included three sections

that are outlined below. (See Appendix A for the complete survey.) Because released

items were used for the study, it was possible to administer the study through an easily

accessible online survey tool (Vovici), that teachers could access from work, home, or

other locations with internet access. The survey could only be completed once each from

a given computer. After the survey was closed out, the responses were downloaded into

Excel for analysis.

Demographics/Background. A series of questions were used to gather data on

the characteristics of the teacher and the school in which the teacher taught. Information

was collected on the following topics:

• Gender • Ethnicity • State in which currently teaching • School setting (e.g., urban, rural) • Number of years teaching mathematics • Highest degree obtained • Whether or not certified to teach mathematics at the secondary level • Level of Algebra II taught (e.g., honors) • Textbook used in Algebra II course • Whether or not course taught on block schedule, and type (if applicable)

Personal identifying information such as name of teacher, school, or district was

not collected so that respondents could feel free to answer questions honestly in

subsequent sections of the survey.

Page 5: Test, Measurement & Research Services September 20, 2010

5

Benchmarks. The ADP Algebra II End-of-Course Exam measures 41

benchmarks across the following five standards:

1. Standard O: Operations and Expressions

2. Standard E: Equations and Inequalities

3. Standard P: Polynomial and Rational Functions

4. Standard X: Exponential Functions

5. Standard F: Functional Operations and Inverses

To gain a better understanding of the extent to which the ADP Algebra II exam

benchmarks are being implemented, teachers were presented with the list of the

benchmarks and asked estimate the number of hours that he/she spent teaching that topic.

The following options were provided as response choices: a) 0 hours, b) 1-4 hours, c) 5-9

hours, d) 10-14 hours, and e) 15+ hours. (For a complete list of the Algebra II

benchmarks, see Appendix B.) Respondents who teach more than one type of Algebra II

course (e.g., regular, honors, etc.) were asked to refer to a single course for all of their

judgments.

Released Items Survey. Because the Algebra II End-of-Course Exam has proven

difficult for the students taking the exam, one area of interest was to understand why

many students are struggling to correctly answer items that appear on the test. To gather

additional data related to these issues, teachers were presented with 15 released items

from the Algebra II exam and asked to estimate the percentage of their students who

would be able to answer the item correctly (for multiple-choice items) or earn more than

half of the possible points (for constructed-response items) at the end of the course.

Response options included percentage ranges of a) 0 to 20, b) 21 to 40, c) 41 to 60, d) 61

Page 6: Test, Measurement & Research Services September 20, 2010

6

to 80, and e) 81 to 100. (Recall that the exam contains 1-point multiple-choice items, 2-

point short-answer items, and 4-point extended-response items.) There were two

multiple-choice items and one constructed-response item representing each of the five

standards. The items were chosen to reflect the typical level of complexity and content

coverage of the Algebra II exam. Respondents indicating that less than 61% of students

would successfully answer the item were asked to provide a free response explaining why

students would have difficulty answering the item correctly.

Sample

The sampling plan focused primarily on collecting responses from teachers in

those states that are administering the ADP Algebra II End-of-Course Exam. Currently,

Arkansas and Hawaii are the only Algebra II census-testing states so they were the focus

of this survey. As a result, the sampling target was to obtain responses from at least 25%

of Algebra II teachers in Arkansas and Hawaii. Additional data were collected from

Kentucky, New Jersey, and Ohio with a total of 25 teachers targeted in each of those

states. Targets were met in Arkansas, Hawaii, and New Jersey while Ohio and Kentucky

samples were one and two teachers short of their targets.

The methodology used for identifying and contacting teachers varied by state. The

state departments of education in Arkansas, Kentucky, and New Jersey invited teachers to

participate in the survey but Pearson sent out requests for participation to teachers in

Hawaii and Ohio. Since the target in Kentucky, New Jersey, and Ohio was only 25

teachers, not all teachers in the state were invited to participate but rather a subset who

had exposure to the ADP Algebra II End-of-Course Exam.

Page 7: Test, Measurement & Research Services September 20, 2010

7

Pearson offered to provide teachers with an honorarium of $20 per completed

survey, however due to state policies the honorarium count not be provided in Arkansas

or Hawaii. Because the survey itself contained no personal identifying information, state-

specific invitation letters for New Jersey, Kentucky, and Ohio asked respondents to

contact Pearson directly to obtain compensation.

Analyses

Prior to analysis, data were cleaned. There was a total of 348 responses to the

survey but those with no mathematics teaching experience or those not teaching Algebra

II were removed. Analyses were based on the remaining 324 responses.

Demographics

The frequencies of responses for the various demographic and background

questions were determined. These data are displayed in tables in terms of percentages of

the 324 respondents.

Benchmarks

Using the medians of the response options, weighted averages were calculated on

responses regarding the number of hours spent teaching each benchmark. A bar chart was

created to visually display the relative time spent teaching each benchmark.

Released Items

For responses estimating the percentage of students who would correctly respond

to the released items, weighted averages were calculated using the medians of the

response categories. Free responses indicating suspected reasons why students would not

get an item correct were synthesized into the following categories:

1. The item is too difficult or complicated.

Page 8: Test, Measurement & Research Services September 20, 2010

8

2. The material was not covered.

3. Students lack understanding.

4. The wording and/or set-up is unfamiliar.

Teachers were asked to comment if they estimated that 60% or fewer of their

students would correctly respond to an item. Thus, the number of possible comments

varied by item and not all teachers who were asked to respond did. Therefore, for ease of

interpretability, the percentage of teachers who were asked to respond was calculated as

well as the percentage of teachers whose responses fell into each of the four categories

listed above.

A table was created summarizing the several pieces of data pertaining to each

released item. These data included item characteristics (the item type, the benchmark it

measures, and whether calculator usage was allowed), the weighted average of teachers’

estimates of correct student responses, the weighted average of the estimated number of

hours spent teaching the benchmark that the item measures, and percentage of teachers

whose free responses fell into the aforementioned categories.

Statistics were available on the released items and were used to provide an idea of

how teacher estimates compared to actual student performance. Some of the released

items had been administered operationally and other had only been through field testing

so the statistics used in this study were those from the most recent administration.

Although the release of specific item statistics is restricted, the released items were

grouped according to item means on a 0/1 scale with possible ranges of 0-.20, .21-.40,

.41-.60, and .61-.80. In calculating item means, students who omitted the item were

excluded from calculations so that item means were not influenced by omit rates.

Page 9: Test, Measurement & Research Services September 20, 2010

9

Results

Demographics

The following table provides demographic distributions of responses. Of the

respondents, 83% were white and two-thirds were female. Teachers from five states

participated but most were from either Arkansas or New Jersey. The data pertaining to

textbook use were not analyzed as collection by way of free response resulted in a wide

array of responses. However, the data are available for future investigation.

Page 10: Test, Measurement & Research Services September 20, 2010

Table 1. Demographic Distributions

Demographic PercentageEthnicity

White 83%Asian/Pacific Islander 12%African American 2%Hispanic 1%Other/No Response 2%

GenderFemale 67%Male 32%

StateNew Jersey 42%Arkansas 30%Hawaii 14%Ohio 7%Kentucky 6%

Years Experience1 to 3 13%4 to 6 17%7 to 9 12%10 to 15 20%16 to 20 14%21 to 25 9%25+ 14%

Highest DegreeM.A./M.S. 55%B.A./B.S. 41%Doctoral 1%Other 4%

School SettingSuburban 41%Rural 32%Urban 24%Other 2%

Schedule StructureNot Block 58%Alternating Block 17%4x4 Block 10%Other/No Response 16%

*Percentages may not sum to 100% due to rounding.

10

Page 11: Test, Measurement & Research Services September 20, 2010

Benchmarks

The bar chart displayed in Figure 1 indicates the mean number of hours that

teachers spent teaching the Algebra II benchmarks. Although the information is presented

in terms of hours, it would be more appropriate for the data to be interpreted in a relative

sense rather than absolute. Teachers indicated that relatively more time is spent teaching

Standards O (Operations and Expressions) and E (Equations and Inequalities) compared

to the other three standards. Estimated time spent on two of the benchmarks in these

standards, O.3.d (perform operations on polynomial expressions) and E.2.a (solve single-

variable quadratic, exponential, rational, radical, and factorable higher-order polynomial

equations over the set of real numbers, including quadratic equations involving absolute

value), were noticeably higher than the other benchmarks.

11

Page 12: Test, Measurement & Research Services September 20, 2010

12

Figure 1. Hours Spent Teaching Algebra II Benchmarks

01

23

45

6

78

910

O1a

O1b

O1c

O2a

O2b

O3a

O3b

O3c

O3d

O3e O3f

E1a

E1b

E1c

E1d

E2a

E2b

E2c

E2d

E2e

P1a

P1b

P1c

P1d

P2a

P2b

P2c

P2d

P2e P2f

X1a

X1b

X1c

X1d

F1a

F1b

F2a

F2b

F3a

F3b

F3c

Standard O Standard E Standard P Standard X Standard F

Mea

n N

umbe

r of

Hou

rs

Page 13: Test, Measurement & Research Services September 20, 2010

13

Released Items

Figure 2 displays a bar chart that indicates the mean proportion of students that

teachers expected could respond correctly to each released item. On the survey, items

were presented in order of standard but in Figure 2, items are arranged into difficulty

ranges for ease of comparison between teacher estimates and actual student performance.

Recall that difficulty estimates are based on field test item means. Solid black lines were

added to highlight the median of the difficulty ranges. For example, for the difficulty

range of 0 to .20, a solid line was constructed at .10.

Figure 2 indicates that teachers tended to overestimate student performance across

all item types and item difficulties. With items arranged by field test difficulty, the largest

discrepancies between teacher estimates and actual student performance tended to be

with the five constructed-response items. These items were categorized by student

performance data as being very difficult but teachers’ estimates did not reflect this same

difficulty.

Page 14: Test, Measurement & Research Services September 20, 2010

14

Figure 2. Teachers’ Estimate of Performance on Released Items

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

SA ER ER SA SA MC MC MC MC MC MC MC MC MC MC

.00-.10 .11-.20 .21-.30 .31-.40 .41-.50

Item Type and Field Test Difficulty

Pro

po

rtio

n

Table 2 provides data regarding the free responses of teachers as to why students

may struggle with particular items. Recall that respondents were only asked to respond if

they indicated that 60% or fewer of their students would be able to respond correctly and

not all of whom were asked to respond did. The items are arranged in the same order as

in Figure 2, according to difficulty. Additional information in the table includes item

characteristics, mean proportion of students estimated to respond correctly, and mean

number of hours spent teaching the benchmark that the item measures.

The most frequently mentioned reasons for students not being able to respond

correctly were that the item was too difficult or complicated and that the material was not

taught. Item 6 had the highest number of free responses and interestingly, the responses

were spread across all four reasons. Several of the items frequently suggested by teachers

as too difficult or complicated for students to answer correctly (e.g., items 3, 7, and 8)

measured standards that received relative greater instructional time, (i.e., Operations and

Page 15: Test, Measurement & Research Services September 20, 2010

15

Expressions; Equations and Inequalities). On the other hand, teachers provided little

explanation for poor student performance on the two items corresponding to the

benchmarks shown in Figure 1 to have been associated with most instructional time

(O.3.d, measured by item 9, and E.2.a, measured by item 2). In general, the free response

data are difficult to interpret because in many cases teachers did not comply with the

request that they offer an explanation when they estimated 60 percent or fewer of

students would answer a question correctly.

Page 16: Test, Measurement & Research Services September 20, 2010

16

Table 2. Data on Released Items Item Mean on 0/1 Scale .41-.50Survey Order Item 6 Item 3 Item 12 Item 15 Item 9 Item 14 Item 10 Item 4 Item 5 Item 1 Item 11 Item 8 Item 7 Item 13 Item 2Item Characteristics:

Benchmark F,3,c E P X,1,d O,3,d X,1,c P,2,b F,1,b F,2,b E,1,a P,2,d O,3,c O,2,b X,1,b E,2,aItem type SA ER ER SA SA MC MC MC MC MC MC MC MC MC MCSession Calc No Calc Calc Calc No Calc No Calc No Calc Calc No Calc Calc Calc No Calc Calc Calc No Calc

Data from Teacher Survey:Estimate of proportion of correct responses 0.37 0.37 0.51 0.54 0.55 0.50 0.60 0.39 0.63 0.52 0.32 0.55 0.45 0.55 0.61Hours spent teaching benchmark 2.91 5.09 4.6 4.01 7.74 3.31 4.91 4.23 3.43 4.91 3.72 5.92 4.51 3.91 8.72Percent asked to comment on items* 74% 76% 55% 52% 49% 57% 37% 74% 33% 56% 81% 48% 61% 47% 38%Reasons for incorrect responses:

Too difficult/complicated 6% 15% 1% 0% 0% 4% 2% 12% 0% 2% 23% 14% 16% 7% 2%Not covered 17% 0% 2% 6% 5% 9% 9% 13% 4% 2% 11% 1% 6% 7% 5%Students lack understanding 8% 6% 5% 3% 2% 5% 1% 6% 4% 14% 0% 0% 3% 0% 0%Unfamiliar wording/set up 7% 11% 6% 0% 2% 0% 0% 0% 0% 3% 2% 0% 0% 2% 5%

*Teachers estimating correct responses of 60% or less were asked to comment on why students may struggle with the item.

.00-.10 .11-.20 .21-.30 .31-.40

Page 17: Test, Measurement & Research Services September 20, 2010

17

Spring 2010 Performance

To provide context for study interpretations, student performance on the Spring

2010 ADP Algebra II End-of-Course Exam will be summarized. The percentages

provided are at the consortium level but it is important to note the two states representing

the largest portion of examinees in the Spring 2010 administration were Arkansas and

Hawaii representing 63% and 18%, respectively. On the overall exam, 13% of students

scored at or above prepared and performance at the standard level is displayed in Table 3

indicating the percent of students scoring at or above mastery in each of the five

standards. These mastery determinations were established by linking the overall exam

proficiency level to a comparable level of performance on each standard. In that sense,

the differences in percentages of students achieving mastery across standards can be

considered relative to a common performance level.

Table 3. Standards Mastery Percentages

O E P X F16% 18% 19% 24% 22%

Standard Mastery

Table 3 indicates that the lowest performance was seen in Standards O and E,

which were also the standards indicated in the survey to have received the most

instructional time. One possible reason for this finding may be that the aspects of these

standards that are covered in classroom instruction are not sufficiently aligned with the

emphasis and rigor measured by the ADP Algebra II exam.

Discussion

In considering the survey results, two themes emerged that seem worthy of further

discussion. These included: 1) a tendency to over-estimate student performance on

Page 18: Test, Measurement & Research Services September 20, 2010

18

constructed response items; and 2) some evidence that instruction time is focused on

more basic Algebra I material within the standards. Each of these themes is discussed in

more detail below.

Overestimation of Student Performance on Constructed-Response Items

Teachers grossly overestimated student performance on the five constructed-

response items as shown in Figure 2. Students may not be accustomed to demonstrating

their knowledge and skills through constructed-response items. Perhaps teachers assume

that students can easily transfer knowledge and skills to different item formats without

exposing students to these types of experiences in the classroom. Challenging students

with constructed-response items more frequently in the classroom may allow students to

more easily demonstrate their knowledge and skills through a variety of item formats.

Instructional Time Focused on Algebra I Material

The response pattern displayed in Figure 1 regarding time spent teaching the

various benchmarks indicates that more time being devoted to Standards O and E and less

to Standards X and F with time spent on Standard P falling in between. Because

Standards O and E could be considered an overlap of Algebra I material, this result may

suggest that teachers are spending time re-teaching material that students did not

adequately learn in Algebra I. Given the spring performance of relatively lower

percentages of students being considered to have reached mastery in Standards O and E,

perhaps these two standards are being taught at more of an Algebra I level and not at the

depth of represented in the ADP Algebra II exam standards.

Page 19: Test, Measurement & Research Services September 20, 2010

19

Limitations and Conclusions

This teacher content survey was an initial attempt to explore questions related to

potential gaps between curriculum and instruction, and the ADP Algebra II exam. As

with any exploratory study, there were limitations that should be taken into account as

results are considered. First, due to time and resource limitations, it was not possible to

pilot the survey, and some aspects of the instrument could have been made better if

piloting had been feasible. For example, if the survey were to be re-administered, the

ranges for the hours spent teaching benchmarks would likely be refined to better reflect

the actual number of hours available in an instructional year. Second, the survey data

collected using the released item set were limited in that only 15 items were included, and

these items contained various item types measuring different standards within a wide

range of difficulties. All of these sources of variation provided confounding factors when

interpreting the data on the released items.

Despite limitations due to the exploratory nature of the study, the survey results

did reveal some insights about possible disparities between what is being taught and

emphasized instructionally, and what is being measured by the ADP Algebra II exam. In

particular, the survey data revealed that teachers underestimated the rigor of the

constructed-response items on the exam and may not have always been clear on the kinds

of items that are being used to assess certain ADP Algebra II exam benchmarks. In

general, these survey results suggest that the ADP Algebra II exam standards may still be

being implemented in the curriculum and instructional practices of the schools that are

giving the exam, and that teachers will benefit from professional development activities

Page 20: Test, Measurement & Research Services September 20, 2010

20

aimed at increasing understanding of the ADP Algebra II exam standards and how these

standards are reflected in the exam content.

Page 21: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

ADP Algebra II Teacher Content Evaluation Study

1) What is your gender?

MaleFemale

2) What is your race/ethnicity?

Caucasian/WhiteBlackAmerican IndianAsian/Pacific IslanderMultiracialHispanic/LatinoOther (please specify)

If you selected other, please specify:

3) I am currently teaching in the state of:

ArkansasHawaiiKentuckyNew JerseyOhioRhode IslandOther (please specify)

If you selected other, please specify:

4) I have ______ years of experience teaching mathematics.

1 to 34 to 67 to 910 to 1516 to 2021 to 2525+

5) The highest degree I have obtained is:

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (1 of 23) [8/2/2010 9:07:46 AM]

Appendix A

A1

Page 22: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

B.A./B.S.M.A./M.S.DoctoralOther (please specify)

If you selected other, please specify:

6) Is this degree in a mathematics-related field?

YesNoNot Sure

7) Are you certified to teach mathematics (or courses in mathematics-related areas) in your state at the secondary level?

YesNoOther (please specify)

If you selected other, please specify:

8) What level of Algebra do you teach? (Choose all that apply.)

Algebra IIAlgebra II Advanced/HonorsOther (please specify)

If you selected other, please specify:

9) Which textbook(s) are you currently using in your Algebra II class(es)? You may include title, author, publisher and/or edition if available.

10) If you teach on a block schedule, what type of block scheduling is used?

4 x 4Alternating DaysNot applicable—my course is not taught on a block scheduleOther (please specify)

If you selected other, please specify:

11) My school is located in a predominantly _____ area.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (2 of 23) [8/2/2010 9:07:46 AM]

A2

Page 23: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

UrbanSuburbanRuralOther (please specify)

If you selected other, please specify:

[page break]

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (3 of 23) [8/2/2010 9:07:46 AM]

A3

Page 24: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

The ADP assessment consortium created a rigorous, high-quality Algebra II End-of-Course assessment that will not only inform students as to whether or not they have mastered the content of the course they have just completed but also give them an indication of their preparedness for higher-level mathematics courses. With the standards and exam now in place, states are turning their attention to ensure consistency of content and rigor in Algebra II courses within their state, to improve the quality of curriculum and instruction.

The purpose of this teacher content evaluation study is to better understand the gaps between curriculum and instruction within and across states, and what is covered on the Algebra II exam. Specifically, the purpose is to understand which of the ADP Algebra II exam standards and benchmarks teachers are covering in their classrooms and the percentage of their students that teachers anticipate would be able to successfully answer ADP Algebra II exam items.

Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach.

12) If you teach more than one Algebra II course, please refer to one course in your judgments, and specify the name of the course below.

[page break]

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (4 of 23) [8/2/2010 9:07:46 AM]

A4

Page 25: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

13) Operations and Expressions - O1. Real numbers Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

0 1 to 4 5 to 9 10 to 14 15+

a. Convert between and among radical and exponential forms of numerical expressions.

b. Simplify and perform operations on numerical expressions containing radicals.

c. Apply the laws of exponents to numerical expressions with rational and negative exponents to order and rewrite them in alternative forms.

14) Operations and Expressions - O2. Complex numbers Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

0 1 to 4 5 to 9 10 to 14 15+

a. Represent complex numbers in the form a+bi, where a and b are real; simplify powers of pure imaginary numbers.

b. Perform operations on the set of complex numbers.

15) Operations and Expressions - O3. Algebraic expressions Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

0 1 to 4 5 to 9 10 to 14 15+

a. Convert between and among radical and exponential forms of algebraic expressions.

b. Simplify and perform operations on radical algebraic expressions.

c. Apply the laws of exponents to algebraic expressions, including those involving rational and negative exponents, to order and rewrite them in alternative forms.

d. Perform operations on polynomial expressions.

e. Perform operations on rational expressions, including complex fractions.

f. Identify or write equivalent algebraic expressions in one or more variables to extract information.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (5 of 23) [8/2/2010 9:07:46 AM]

A5

Page 26: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

16) Equations and Inequalities - E1. Linear equations and inequalities Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

0 1 to 4 5 to 9 10 to 14 15+

a. Solve equations and inequalities involving the absolute value of a linear expression.

b. Express and solve systems of linear equations in three variables with and without the use of technology.

c. Solve systems of linear inequalities in two variables and graph the solution set.

d. Recognize and solve problems that can be represented by single variable linear equations or inequalities or systems of linear equations or inequalities involving two or more variables. Interpret the solution(s) in terms of the context of the problem.

17) Equations and Inequalities - E2. Nonlinear equations and inequalities Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

*For choice c below, benchmark is as follows:

0 1 to 4 5 to 9 10 to 14 15+

a. Solve single-variable quadratic, exponential, rational, radical, and factorable higher-order polynomial equations over the set of real numbers, including quadratic equations involving absolute value.

b. Solve single variable quadratic equations and inequalities over the complex numbers; graph real solution sets on a number line.

*c. Use the discriminant, D = b squared - 4ac, to determine the nature of the solutions of the equation ax squared + bx + c = 0.

d. Graph the solution set of a two-variable quadratic inequality in the coordinate plane.

e. Rewrite nonlinear equations and inequalities to express them in multiple forms in order to facilitate finding a solution set or to extract information about the relationships or graphs indicated.

18) Polynomial and Rational Functions - P1. Quadratic functions Please read each benchmark as shown on the following screens and indicate the number of hours

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (6 of 23) [8/2/2010 9:07:46 AM]

A6

Page 27: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

0 1 to 4 5 to 9 10 to 14 15+

a. Determine key characteristics of quadratic functions and their graphs.

b. Represent quadratic functions using tables, graphs, verbal statements, and equations. Translate among these representations.

c. Describe and represent the effect that changes in the parameters of a quadratic function have on the shape and position of its graph.

d. Recognize, express, and solve problems that can be modeled using quadratic functions. Interpret their solutions in terms of the context.

19) Polynomial and Rational Functions - P2. Higher-order polynomial and rational functions Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

*For choice a below, benchmark is as follows:

0 1 to 4 5 to 9 10 to 14 15+

*a. Determine key characteristics of power functions in the form f(x) = axn , a≠0, for positive integral values of n and their graphs.

b. Determine key characteristics of polynomial functions and their graphs.

c. Represent polynomial functions using tables, graphs, verbal statements, and equations. Translate among these representations.

d. Determine key characteristics of simple rational functions and their graphs.

e. Represent simple rational functions using tables, graphs, verbal statements, and equations. Translate among these representations.

f. Recognize, express, and solve problems that can be modeled using polynomial and simple rational functions. Interpret their solutions in terms of the context.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (7 of 23) [8/2/2010 9:07:46 AM]

A7

Page 28: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

20) Exponential Functions - X1. Exponential functions Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

0 1 to 4 5 to 9 10 to 14 15+

a. Determine key characteristics of exponential functions and their graphs.

b. Represent exponential functions using tables, graphs, verbal statements, and equations. Represent exponential equations in multiple forms. Translate among these representations.

c. Describe and represent the effect that changes in the parameters of an exponential function have on the shape and position of its graph.

d. Recognize, express, and solve problems that can be modeled using exponential functions, including those where logarithms provide an efficient method of solution. Interpret their solutions in terms of the context.

f. Recognize, express, and solve problems that can be modeled using polynomial and simple rational functions. Interpret their solutions in terms of the context.

21) Functional Operations and Inverses - F1. Function operations Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

0 1 to 4 5 to 9 10 to 14 15+

a. Combine functions by addition, subtraction, multiplication, and division.

b. Determine the composition of two functions, including any necessary restrictions on the domain.

22) Functional Operations and Inverses - F2. Inverse functions Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

0 1 to 4 5 to 9 10 to 14 15+

a. Describe the conditions under which an inverse relation is a function.

b. Determine and graph the inverse relation of a function.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (8 of 23) [8/2/2010 9:07:46 AM]

A8

Page 29: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

23) Functional Operations and Inverses - F3. Piecewise-defined functions Please read each benchmark as shown on the following screens and indicate the number of hours you typically spend each year covering the topic for the Algebra II course you teach. If you teach more than one Algebra II course, please refer to one course in your judgments.

0 1 to 4 5 to 9 10 to 14 15+

a. Determine key characteristics of absolute value, step, and other piecewise-defined functions.

b. Represent piecewise-defined functions using tables, graphs, verbal statements, and equations. Translate among these representations.

c. Recognize, express, and solve problems that can be modeled using absolute value, step, and other piecewise-defined functions. Interpret their solutions in terms of the context.

[page break]

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (9 of 23) [8/2/2010 9:07:46 AM]

A9

Page 30: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

On the next screens are 15 released items that cover the Algebra II content standards. There are three items (two multiple-choice and one constructed-response items) for each of the five standards; these items were chosen because they reflect the typical level of complexity and content coverage on the Algebra II exam.

Please review each item and indicate the percentage of students who you believe would get this item correct (multiple-choice) or earn more than half of the possible points (constructed-response), or in other words, "be successful" if it was administered at the end of your course. If you believe that less than 60% of your students would be successful, please use the space provided to explain why believe they would be unable to get the item correct or earn more than half of the available points on a constructed-response item.

When addressing the rationale for your students lack of success, please keep in mind some of the following issues: Is the mathematical vocabulary used in the item also being used in the classroom? If there is context provided in the question, is it a context that students would be familiar with regarding that standard? Are students familiar with and assigned homework that involves constructed-response item types?

[page break]

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (10 of 23) [8/2/2010 9:07:46 AM]

A10

Page 31: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

24) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is B.

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

25) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is B.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (11 of 23) [8/2/2010 9:07:46 AM]

A11

Page 32: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

26) Please review the item below and indicate the percentage of students who you believe would earn more than half of the possible points (or in other words, "be successful") if it was administered at the end of your course. If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why believe they would be unable to earn more than half of the available points.

This item is worth a total of 4 points.

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

[page break]

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (12 of 23) [8/2/2010 9:07:46 AM]

A12

Page 33: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

27) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is B.

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

28) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is C.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (13 of 23) [8/2/2010 9:07:46 AM]

A13

Page 34: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

29) Please review the item below and indicate the percentage of students who you believe would earn more than half of the possible points (or in other words, "be successful") if it was administered at the end of your course. If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why believe they would be unable to earn more than half of the available points.

This item is worth a total of 2 points.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (14 of 23) [8/2/2010 9:07:46 AM]

A14

Page 35: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

[page break]

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (15 of 23) [8/2/2010 9:07:46 AM]

A15

Page 36: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

30) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is C.

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

31) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is D.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (16 of 23) [8/2/2010 9:07:46 AM]

A16

Page 37: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

32) Please review the item below and indicate the percentage of students who you believe would earn more than half of the possible points (or in other words, "be successful") if it was administered at the end of your course. If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why believe they would be unable to earn more than half of the available points.

This item is worth a total of 2 points.

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

[page break]

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (17 of 23) [8/2/2010 9:07:46 AM]

A17

Page 38: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

33) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is C.

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

34) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is C.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (18 of 23) [8/2/2010 9:07:46 AM]

A18

Page 39: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

35) Please review the item below and indicate the percentage of students who you believe would earn more than half of the possible points (or in other words, "be successful") if it was administered at the end of your course. If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why believe they would be unable to earn more than half of the available points.

This item is worth a total of 4 points.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (19 of 23) [8/2/2010 9:07:46 AM]

A19

Page 40: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

[page break]

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (20 of 23) [8/2/2010 9:07:46 AM]

A20

Page 41: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

36) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is B.

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

37) Please review the item below and indicate the percentage of students who you believe would get this item correct (or in other words, "be successful") if it was administered at the end of your course.If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why you believe they would be unable to get the item correct.

The correct answer for this item is C.

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (21 of 23) [8/2/2010 9:07:46 AM]

A21

Page 42: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

38) Please review the item below and indicate the percentage of students who you believe would earn more than half of the possible points (or in other words, "be successful") if it was administered at the end of your course. If you believe that less than 60% of your students would be successful, please use the comments field provided to explain why believe they would be unable to earn more than half of the available points.

This item is worth a total of 2 points.

0 to 20 percent21 to 40 percent41 to 60 percent61 to 80 percent81 to 100 percent

Additional comments:

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (22 of 23) [8/2/2010 9:07:46 AM]

A22

Page 43: Test, Measurement & Research Services September 20, 2010

ADP Algebra II Teacher Content Evaluation Study

Thank you for completing this survey.

If you have questions about this survey, please contact Anne Johnson at 319-358-4360 or [email protected].

If you have additional questions about the ADP program, please contact the ADP Customer Service Center at 1-866-688-9555 or via e-mail at [email protected].

11%

http://desktop.vovici.com/wswebtop.dll/WSPreview?v=0&pgid=0&renderprintdisplay=1&esid=317551 (23 of 23) [8/2/2010 9:07:46 AM]

A23

Page 44: Test, Measurement & Research Services September 20, 2010

B1

Appendix B

O1. Real numbersa. Convert between and among radical and exponential forms of numerical expressions.

b. Simplify and perform operations on numerical expressions containing radicals. c. Apply the laws of exponents to numerical expressions with rational and negative exponents to order and rewrite them in alternative forms.O2. Complex numbersa. Represent complex numbers in the form a+bi, where a and b are real; simplify powers of pure imaginary numbers.b. Perform operations on the set of complex numbers.

O3. Algebraic expressionsa. Convert between and among radical and exponential forms of algebraic expressions. b. Simplify and perform operations on radical algebraic expressions. c. Apply the laws of exponents to algebraic expressions, including those involving rational and negative

exponents, to order and rewrite them in alternative forms.d. Perform operations on polynomial expressions.

e. Perform operations on rational expressions, including complex fractions.f. Identify or write equivalent algebraic expressions in one or more variables to extract information.

E1. Linear equations and inequalitiesa. Solve equations and inequalities involving the absolute value of a linear expression.b. Express and solve systems of linear equations in three variables with and without the use of technology. c. Solve systems of linear inequalities in two variables and graph the solution set.

d. Recognize and solve problems that can be represented by single variable linear equations or inequalities or systems of linear equations or inequalities involving two or more variables. Interpret the solution(s) in terms of the context of the problem. E2. Nonlinear equations and inequalitiesa. Solve single-variable quadratic, exponential, rational, radical, and factorable higher-order polynomial equations over the set of real numbers, including quadratic equations involving absolute value.

b. Solve single variable quadratic equations and inequalities over the complex numbers; graph real solution sets on a number line. c. Use the discriminant, D = b 2 - 4ac , to determine the nature of the solutions of the equation ax 2 + bx + d. Graph the solution set of a two-variable quadratic inequality in the coordinate plane.e. Rewrite nonlinear equations and inequalities to express them in multiple forms in order to facilitate finding a solution set or to extract information about the relationships or graphs indicated.

P1. Quadratic functionsa. Determine key characteristics of quadratic functions and their graphs. b. Represent quadratic functions using tables, graphs, verbal statements, and equations. Translate among these representations.c. Describe and represent the effect that changes in the parameters of a quadratic function have on the shape

and position of its graph.d. Recognize, express, and solve problems that can be modeled using quadratic functions. Interpret their solutions in terms of the context.P2. Higher-order polynomial and rational functionsa. Determine key characteristics of power functions in the form f (x ) = ax n , a! 0 , for positive integral values b. Determine key characteristics of polynomial functions and their graphs.c. Represent polynomial functions using tables, graphs, verbal statements, and equations. Translate among these representations.d. Determine key characteristics of simple rational functions and their graphs. e. Represent simple rational functions using tables, graphs, verbal statements, and equations. Translate among these representations.f. Recognize, express, and solve problems that can be modeled using polynomial and simple rational

functions. Interpret their solutions in terms of the context.

X1. Exponential functionsa. Determine key characteristics of exponential functions and their graphs.

b. Represent exponential functions using tables, graphs, verbal statements, and equations. Represent exponential equations in multiple forms. Translate among these representations.

c. Describe and represent the effect that changes in the parameters of an exponential function have on the shape and position of its graph. d. Recognize, express, and solve problems that can be modeled using exponential functions, including those where logarithms provide an efficient method of solution. Interpret their solutions in terms of the context.

F1. Function operationsa. Combine functions by addition, subtraction, multiplication, and division.

b. Determine the composition of two functions, including any necessary restrictions on the domain.

F2. Inverse functionsa. Describe the conditions under which an inverse relation is a function. b. Determine and graph the inverse relation of a function.

F3. Piecewise-defined functionsa. Determine key characteristics of absolute value, step, and other piecewise-defined functions.b. Represent piecewise-defined functions using tables, graphs, verbal statements, and equations. Translate among these representations.c. Recognize, express, and solve problems that can be modeled using absolute value, step, and other piecewise-defined functions. Interpret their solutions in terms of the context.

Eq

uati

on

s an

d I

neq

uali

ties

Po

lyn

om

ial

an

d

Rati

on

al

Fun

ctio

ns

Exp

on

en

tial

Fu

nct

ion

sFu

nct

ion

al

Op

era

tio

ns

an

d I

nve

rses

ADP Algebra II End-of-Course Exam Content StandardsContent

StandardsContent

Benchmarks

Op

era

tio

ns

an

d E

xp

ress

ion

s