cabrillo college ctep english placement validation portfolio validity... · cabrillo college ctep...

Cabrillo College

CTEP English Placement Validation Portfolio

April 23, 2004

Planning & Research OfficeRichard C. Borden, Ph.D.Institutional Research Analyst

Introduction & Exhibit IndexCTEP English Placement Validation Portfolio

IntroductionIn Spring of 2003, Cabrillo College embarked upon a project to validate the Chancellor approved CTEP instrument for placing students intoits English course sequence: ENGL-255/290, ENGL-100, ENGL-1A. The goal was to switch over from assessing with the formerly usedEnglish Holistic Essay commencing with the Fall 2003 assessment term. That goal was, indeed, achieved.

The CTEP instruments ultimately selected for use by the English Faculty were:● CTEP Form A Part 1. Reading Comprehension (35 Questions)● CTEP Form A Part 2. Sentence Structure & Grammar (30 Questions)

The CTEP validation and monitoring process is an ongoing one. This document showcases the content of Cabrillo's validation portfolio todate. Portfolio items are presented in reverse order of completion so that those who have been participants in the process since itsinception will find the "freshest" cream on top.

Exhibit Description Page

CTEP Criterion Validity CTEP Faculty Derived Cut Scores● ENGL-255/290.......................................... 2 ● Cut Scores & Multiple Measures Used...... 15● ENGL-100................................................. 3● ENGL-1A...................................................4 CTEP Item Level Content Validity

● CTEP Part 1.............................................. 16Success Rates ● CTEP Part 2.............................................. 17

● Fall03 (CTEP) versus Fall02 (Essay).........5CTEP Domain Level Content Validity

Disproportionate Impact ● CTEP Part 1: ENGL-100........................... 18● Fall03 (CTEP)............................................6-9 ● CTEP Part 1: ENGL-1A............................. 19● Fall02 (Essay)........................................... 10-13 ● CTEP Part 2: ENGL-100........................... 20

● CTEP Part 2: ENGL-1A............................. 21CTEP Consequential Validity

● Student Readout....................................... 14● Instructor Readout.....................................14

Cabrillo College Page 1 of 21Planning & Research Office: 4/23/2004

Richard C. Borden, Ph.D.



Criterion Validity for English 255/290 Placements

y = 2.2556x + 20.532R2 = 0.1271

0

5

10

15

20

25

30

35

1 2 3 4 5

X = Instructor's Evaluation of Student's Prospects for Success (1=Low, 5=High)

Y =

Stud

ent's

Sco

re o

n C

TEP

Part

s 1

& 2

This analysis is limited to only those students who met the following profile:

● Assessed for the Fall 2003 term ● Assessed for English in the above term(s) ● Students who placed into ENGL-255/290 ● Only students who received instructor week #5 survey evaluations

● Enrolled that term in a Departmental course consistent with their Placement Level ● Enrollment status for the above term(s) was "1st Time Any College" ● Highest level of education they had attained was a High School Diploma ● Students who were members of the current year's graduating High School class

● Those who achieved their Placement Level without aid of Multiple Measures Points ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

The correlation between the 91 pairs of instructor evaluations & CTEP scores was 0.36 and, thus, exceeds the Chancellor's minimum of 0.35

1

1

1

1

11

12

22

11

2

1

11

52438421 2

2

31

1

21

12

11

12523

4112

1

1

1

1

Circled # next to data markers shows number of students who received that pair of scores.

Regression equation for the best fitting straight line through the data points.



Criterion Validity for English 100 Placements

y = 1.2122x + 38.018R2 = 0.1132

25

30

35

40

45

50

1 2 3 4 5


Y =

Stud

ent's

Sco

re o

n C

TEP

Part

s 1

& 2


● Assessed for the Fall 2003 term ● Assessed for English in the above term(s) ● Students who placed into ENGL-100 ● Only students who received instructor week #5 survey evaluations



The correlation between the 63 pairs of instructor evaluations & CTEP scores was 0.34 and, thus, for all intents and purposes meets the Chancellor's minimum of 0.35

2

1

3

23

2

11

1

1

1

2

2

1

1

14

4

3

3

3

3

3

1

2

21

3

2

2

1

1





Criterion Validity for English 1A Placements

y = 2.0769x + 45.949R2 = 0.2225

40

45

50

55

60

1 2 3 4 5


Y =

Stud

ent's

Sco

re o

n C

TEP

Part

s 1

& 2


● Assessed for the Fall 2003 term ● Assessed for English in the above term(s) ● Students who placed into ENGL-1A ● Only students who received instructor week #5 survey evaluations



The correlation between the 27 pairs of instructor evaluations & CTEP scores was 0.47 and, thus, exceeds the Chancellor's minimum of 0.35

2

1

2

1

2

5

1

1

1

1

4

2

2

1

1



English Success FA03 & FA02

Placement Test Analysis The analysis below is limited to only those students who met the following profile:Dept ENGL ● Assessed for English in the indicated termEnrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaGraduated HS this year? (All) ● All students regardless of whether they graduated from High School this yearENGL MM Upgrade? No ● Those who achieved their Placement Level without aid of Multiple Measures PointsASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

Dirterm Placement Test2003FA 2002FA

Data Course Placement CTEP1&2 EngEssay Success Rate ENGL-1A 74.4% 72.1% ● ENGL-1A success rate was 2.3% HIGHER in FA03 than in FA02.

ENGL-100 70.4% 73.0% ● ENGL-100 success rate was 2.6% LOWER in FA03 than in FA02.ENGL-255/290 65.0% 55.6% ● ENGL-255/290 success rate was 9.4% HIGHER in FA03 than in FA02.

#Enrolled ENGL-1A 176 86 ● Number of ENGL-1A students was 90 (105%) HIGHER in FA03.ENGL-100 416 437 ● Number of ENGL-100 students was 21 (5%) LOWER in FA03.ENGL-255/290 143 151 ● Number of ENGL-255/290 students was 8 (5%) LOWER in FA03.

Total Success Rate 70.3% 69.0% ● Overall success rate was 1.3% HIGHER in FA03 than in FA02.Total #Enrolled 735 674 ● Overall number of students was 61 (9%) HIGHER in FA03.



Disproportionate Impact ENGL 03

Gender Analysis The analysis below is limited to only those students who met the following profile:Dirterm 2003FA ● Assessed for the Fall 2003 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

Test Used GenderCTEP1&2

Data Course Placement Female MalePlacement Percent ENGL-1A 27.6% 24.1%

ENGL-100 52.4% 58.5%ENGL-255/290 20.0% 17.3%

Student Count ENGL-1A 102 96ENGL-100 194 233ENGL-255/290 74 69

Total Placement Percent 100.0% 100.0%Total Student Count 370 398

Majority Gender: MaleMajority group ENGL-1A placement rate: 24.1%

EEOC 80% Guideline: 19.3%

Per EEOC guidelines, any group that falls below 80% of the majority group's ENGL-1A placement rate (here 80% of 24.1% = 19.3%) has been disproportionately impacted.

Disproportionate Group: Female MaleImpact? No No

EEOC Gap:[Note: groups of fewer than 10 are inadequate for analysis.] Adequate sample size? Ok Ok

# Groups Impacted: 0 # Groups Impacted with Ok sample size: 0




Ethnicity Analysis The analysis below is limited to only those students who met the following profile:Dirterm 2003FA ● Assessed for the Fall 2003 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

Test Used EthnicityCTEP1&2

Data Course Placement Af. Am. Am. Ind. Asian Filipino Hispanic Other WhitePlacement Percent ENGL-1A 11.1% 0.0% 40.9% 21.1% 8.5% 16.7% 37.6%

ENGL-100 66.7% 100.0% 50.0% 52.6% 56.6% 58.3% 54.6%ENGL-255/290 22.2% 0.0% 9.1% 26.3% 34.9% 25.0% 7.8%

Student Count ENGL-1A 1 9 4 24 2 155ENGL-100 6 2 11 10 159 7 225ENGL-255/290 2 2 5 98 3 32

Total Placement Percent 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%Total Student Count 9 2 22 19 281 12 412

Majority Ethnicity: WhiteMajority group ENGL-1A placement rate: 37.6%



Disproportionate Group: Af. Am. Am. Ind. Asian Filipino Hispanic Other WhiteImpact? Yes Yes No Yes Yes Yes No

EEOC Gap: -19.0% -30.1% -9.0% -21.6% -13.4%[Note: groups of fewer than 10 are inadequate for analysis.] Adequate sample size? Under 10 Under 10 Ok Ok Ok Ok Ok

# Groups Impacted: 5 Af. Am. / Am. Ind. / Filipino / Hispanic / Other# Groups Impacted with Ok sample size: 3 Filipino / Hispanic / Other




Age Group Analysis The analysis below is limited to only those students who met the following profile:Dirterm 2003FA ● Assessed for the Fall 2003 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

Test Used Age GroupCTEP1&2

Data Course Placement < 18 18 - 20 21 - 25 26 - 30 31 - 40 41 - 50 51 - 60Placement Percent ENGL-1A 30.7% 24.8% 15.8% 36.4% 20.0% 42.9% 66.7%

ENGL-100 55.9% 56.5% 50.0% 45.5% 50.0% 42.9% 33.3%ENGL-255/290 13.4% 18.7% 34.2% 18.2% 30.0% 14.3% 0.0%

Student Count ENGL-1A 39 142 6 4 2 3 2ENGL-100 71 323 19 5 5 3 1ENGL-255/290 17 107 13 2 3 1


Majority Age Group: 18 - 20Majority group ENGL-1A placement rate: 24.8%



Disproportionate Group: < 18 18 - 20 21 - 25 26 - 30 31 - 40 41 - 50 51 - 60Impact? No No Yes No No No No

EEOC Gap: -4.1%[Note: groups of fewer than 10 are inadequate for analysis.] Adequate sample size? Ok Ok Ok Ok Ok Under 10 Under 10

# Groups Impacted: 1 21 - 25# Groups Impacted with Ok sample size: 1 21 - 25




Primary Disability Analysis The analysis below is limited to only those students who met the following profile:Dirterm 2003FA ● Assessed for the Fall 2003 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

Test Used Primary DisabilityCTEP1&2

Data Course PlacementAcquired

Brain InjuryLearning Disabled

Mobility Impaired

Not Disabled

Other Disability

Psychological

DisabilityVisually Impaired

Placement Percent ENGL-1A 0.0% 5.6% 33.3% 26.4% 0.0% 0.0% 0.0%ENGL-100 0.0% 50.0% 66.7% 55.6% 100.0% 100.0% 100.0%ENGL-255/290 100.0% 44.4% 0.0% 18.0% 0.0% 0.0% 0.0%

Student Count ENGL-1A 1 1 196ENGL-100 9 2 413 1 1 1ENGL-255/290 1 8 134


Majority Primary Disability: Not DisabledMajority group ENGL-1A placement rate: 26.4%



Disproportionate Group:Acquired

Brain InjuryLearning Disabled

Mobility Impaired

Not Disabled

Other Disability

Psychological


Impact? Yes Yes No No Yes Yes YesEEOC Gap: -21.1% -15.5% -21.1% -21.1% -21.1%

[Note: groups of fewer than 10 are inadequate for analysis.] Adequate sample size? Under 10 Ok Under 10 Ok Under 10 Under 10 Under 10

# Groups Impacted: 5 Acquired Brain Injury / Learning Disabled / Other Disability / Psychological Disability / Visually Impaired

# Groups Impacted with Ok sample size: 1 Learning Disabled




Gender Analysis The analysis below is limited to only those students who met the following profile:Dirterm 2002FA ● Assessed for the Fall 2002 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

Test Used GenderEngEssay

Data Course Placement Female MalePlacement Percent ENGL-1A 17.2% 12.9%

ENGL-100 65.6% 60.3%ENGL-255/290 17.2% 26.8%

Student Count ENGL-1A 64 42ENGL-100 244 196ENGL-255/290 64 87

Total Placement Percent 100.0% 100.0%Total Student Count 372 325

Majority Gender: FemaleMajority group ENGL-1A placement rate: 17.2%



Disproportionate Group: Female MaleImpact? No Yes

EEOC Gap: -0.8%[Note: groups of fewer than 10 are inadequate for analysis.] Adequate sample size? Ok Ok

# Groups Impacted: 1 Male# Groups Impacted with Ok sample size: 1 Male




Ethnicity Analysis The analysis below is limited to only those students who met the following profile:Dirterm 2002FA ● Assessed for the Fall 2002 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

Test Used EthnicityEngEssay

Data Course Placement Af. Am. Am. Ind. Asian Filipino Hispanic Other WhitePlacement Percent ENGL-1A 0.0% 20.0% 15.0% 7.1% 6.1% 0.0% 21.6%

ENGL-100 53.8% 60.0% 65.0% 71.4% 57.0% 77.8% 67.5%ENGL-255/290 46.2% 20.0% 20.0% 21.4% 36.9% 22.2% 10.8%

Student Count ENGL-1A 1 3 1 15 82ENGL-100 7 3 13 10 139 7 256ENGL-255/290 6 1 4 3 90 2 41


Majority Ethnicity: WhiteMajority group ENGL-1A placement rate: 21.6%



Disproportionate Group: Af. Am. Am. Ind. Asian Filipino Hispanic Other WhiteImpact? Yes No Yes Yes Yes Yes No

EEOC Gap: -17.3% -2.3% -10.2% -11.2% -17.3%[Note: groups of fewer than 10 are inadequate for analysis.] Adequate sample size? Ok Under 10 Ok Ok Ok Under 10 Ok

# Groups Impacted: 5 Af. Am. / Asian / Filipino / Hispanic / Other# Groups Impacted with Ok sample size: 4 Af. Am. / Asian / Filipino / Hispanic




Age Group Analysis The analysis below is limited to only those students who met the following profile:Dirterm 2002FA ● Assessed for the Fall 2002 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

Test Used Age GroupEngEssay

Data Course Placement < 18 18 - 20 21 - 25 26 - 30 31 - 40 41 - 50 51 - 60Placement Percent ENGL-1A 14.1% 15.3% 18.2% 6.7% 0.0% 33.3% 0.0%

ENGL-100 68.7% 64.2% 47.7% 46.7% 25.0% 50.0% 100.0%ENGL-255/290 17.2% 20.5% 34.1% 46.7% 75.0% 16.7% 0.0%

Student Count ENGL-1A 14 81 8 1 2ENGL-100 68 339 21 7 1 3 1ENGL-255/290 17 108 15 7 3 1


Majority Age Group: 18 - 20Majority group ENGL-1A placement rate: 15.3%



Disproportionate Group: < 18 18 - 20 21 - 25 26 - 30 31 - 40 41 - 50 51 - 60Impact? No No No Yes Yes No Yes

EEOC Gap: -5.6% -12.3% -12.3%[Note: groups of fewer than 10 are inadequate for analysis.] Adequate sample size? Ok Ok Ok Ok Under 10 Under 10 Under 10

# Groups Impacted: 3 26 - 30 / 31 - 40 / 51 - 60# Groups Impacted with Ok sample size: 1 26 - 30




Primary Disability Analysis The analysis below is limited to only those students who met the following profile:Dirterm 2002FA ● Assessed for the Fall 2002 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)

Test Used Primary DisabilityEngEssay

Data Course PlacementAcquired Brain Injury

Hearing Impaired

Learning Disabled

Mobility Impaired

Not Disabled

Other Disability

Psychological Disability

Visually Impaired

Placement Percent ENGL-1A 0.0% 0.0% 4.0% 0.0% 15.8% 0.0% 33.3% 0.0%ENGL-100 100.0% 100.0% 28.0% 40.0% 64.6% 100.0% 33.3% 0.0%ENGL-255/290 0.0% 0.0% 68.0% 60.0% 19.6% 0.0% 33.3% 100.0%

Student Count ENGL-1A 1 104 1ENGL-100 1 1 7 2 425 3 1ENGL-255/290 17 3 129 1 1

Total Placement Percent 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%Total Student Count 1 1 25 5 658 3 3 1

Majority Primary Disability: Not DisabledMajority group ENGL-1A placement rate: 15.8%



Disproportionate Group:Acquired

Brain InjuryHearing Impaired

Learning Disabled

Mobility Impaired

Not Disabled

Other Disability

Psychological


Impact? Yes Yes Yes Yes No Yes No YesEEOC Gap: -12.6% -12.6% -8.6% -12.6% -12.6% -12.6%

[Note: groups of fewer than 10 are inadequate for analysis.] Adequate sample size? Under 10 Under 10 Ok Under 10 Ok Under 10 Under 10 Under 10

# Groups Impacted: 6 Acquired Brain Injury / Hearing Impaired / Learning Disabled / Mobility Impaired / Other Disability / Visually Impaired

# Groups Impacted with Ok sample size: 1 Learning Disabled



Consequential Validity ENGL 03

Student Ratings of their Preparedness for the class. The analysis below is limited to only those students who met the following profile:Dirterm 2003FA ● Assessed for the Fall 2003 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)Who completed survey? (All) ● All students with personal and/or instructor week #5 survey evaluations

Test Used Data Course PlacementCTEP1&2Percentages Student Counts

Student evaluation of own preparedness for course ENGL-255/290 ENGL-100 ENGL-1A ENGL-255/290 ENGL-100 ENGL-1AI am Overqualified 5.9% 8.3% 6.1% 6 6 2I am Qualified ("just barely" to "fully") 92.1% 90.3% 93.9% 93 65 31I am Not Qualified 2.0% 1.4% 0.0% 2 1Grand Total 100.0% 100.0% 100.0% 101 72 33

● The Chancellor's threshold for an acceptable response is that at least 75% should feel qualified to be in the course in which they were placed.● That 75% standard is exceeded at all levels of the placement sequence.

Instructor Ratings of Prospects for Success in the class. The analysis below is limited to only those students who met the following profile:Dirterm 2003FA ● Assessed for the Fall 2003 termDept ENGL ● Assessed for English in the above term(s)Enrolled as Placed? Yes ● Enrolled that term in a Departmental course consistent with their Placement LevelEnroll Status 1st Time Any College ● Enrollment status for the above term(s) was "1st Time Any College"Highest Education HS Grad ● Highest level of education they had attained was a High School DiplomaENGL MM Upgrade? (All) ● All assessees regardless of whether Multiple Measures Points upgraded their Placement LevelASAP Q3 Is English your first (primary) language? (All) ● All assessees regardless of their answer to the CAPP/ASAP primary language question (Q3)Who completed survey? (All) ● All students with personal and/or instructor week #5 survey evaluations

Test Used Data Course PlacementCTEP1&2Percentages Student Counts

Instructor evaluation of students' ability ENGL-255/290 ENGL-100 ENGL-1A ENGL-255/290 ENGL-100 ENGL-1AModerate Ability or Greater (will pass) 80.8% 76.7% 95.5% 97 66 42Weak Ability or Less (may/will not pass) 19.2% 23.3% 4.5% 23 20 2Grand Total 100.0% 100.0% 100.0% 120 86 44

● The Chancellor's threshold for an acceptable response is that at least 75% should be rated as having ability sufficient for success in the course.● That 75% standard is exceeded at all levels of the placement sequence.



English CTEP Cut ScoresCABRILLO COLLEGEEnglish Test Project: CTEP Form A Part 1. Reading Comprehension (35 Questions)English Test Project: CTEP Form A Part 2. Sentence Structure & Grammar (30 Questions)

(Note: Although Form A Part 3 was also evaluated, the faculty elected not employ it for English placements.)FACULTY DERIVED CUT SCORES for the above two tests combined (65 Questions)

FacultyReviewer English 290/255 English 100 English 1A

Rater#1 (Parts 1 & 2) 26.40 45.25Rater#2 (Parts 1 & 2) 2.08 27.55 45.15Rater#3 (Parts 1 & 2) 32.45 46.10

Rater#4 (Part 1) Rater#5 (Part2)

24.55 40.85

Average RawFaculty-derived

cut2.08 27.74 44.34

Remaining Items 62.92 37.26 20.66

Remaining ItemsRight By Chance* 15.73 9.32 5.17

Average CorrectedFaculty-derived

cut17.81 37.05 49.50

Rounded Cut 18 37 50

Cut Ranges** Selected after

reviewing above data on 02-06-2003

00-34† 35-49 50-69

Present at the 2-6-03 review meeting were: W.Baer, G.Donatelli-Sardo, D.Putnam, R.Borden

* Note: Since each test question offers four alternative answers, a student who randomly selected answers totest questions would be expected to get roughly every fourth question correct by chance. This in turn impliesthat students who guessed their answers to all 65 questions would get an average of 65 ÷ 4 = 16.25 (roughly 16)test items correct by chance.

** Criterion Score includes:● a total of 65 possible CTEP Part 1 + Part 2 points; ● plus, a total of 4 possible multiple measures (MM) points

•► 1MM point if: CAPP Q9= "More than 10 years" out of school•► 1MM point if: CAPP Q11= "A" was grade received in last English class•► 1MM point if: CAPP Q12= "B to A-" OR "A- to A" was high school GPA•► 1MM point if: CAPP Q18= "Less than 6 units" were planned for next term

† Those scoring in the 00-17 range receive the following placement/comment: "English 255 or 290 with Reading and Lab corequisites; we recommend that you also enroll in Adult Education courses or ESL courses"

Cabrillo College Page 15 of 21 Planning & Research Office: 4/23/2004

ENGL Item Review CTEP1

CABRILLO COLLEGEEnglish Content Review: CTEP Part 1. Reading Comprehension

Summary of Average Item RatingsInstructions: Please rate each item’s importance for each English course using the scale below.

How important is the academic knowledge or skill measured by this item forsuccessful acquisition of the skills taught in this course?

5. Critically Important4. Important3. Moderately Important2. Of Slight Importance1. Not Relevant

Test Item Number

English 290/255 English 100 English 1A

# of Raters*: N=2 N=4 N=41 2.5 3.8 4.52 2.0 3.3 4.33 2.5 3.8 4.54 2.5 3.8 4.85 2.5 3.3 4.36 2.0 3.5 4.07 1.5 3.3 4.08 2.0 4.0 4.89 2.5 3.5 4.010 1.5 3.3 4.011 2.5 3.8 4.812 1.5 3.0 3.813 1.5 3.3 4.314 2.5 3.8 4.815 2.5 3.5 4.516 2.5 3.5 4.317 2.0 3.5 4.318 2.5 3.3 3.819 2.0 3.3 4.020 2.0 3.3 4.021 1.5 3.5 4.322 2.0 3.0 3.523 2.0 3.5 4.324 1.5 3.3 4.325 2.0 3.3 4.526 1.0 3.0 4.527 1.5 3.0 4.028 1.5 3.3 4.329 2.0 3.0 3.830 1.5 2.8 4.031 1.5 3.0 3.832 1.0 3.0 4.033 1.0 2.8 3.834 2.5 3.0 4.535 1.0 2.8 4.0

*Raters were: R1, R2, R3 and R4

Average Rating 1.9 3.3 4.2

#Items 4.0 or higher 0 1 29#Items 3.0-3.9 0 31 6#Items 2.0-2.9 21 3 0#Items below 2.0 14 0 0


Richard C. Borden, Ph.D,

ENGL Item Review CTEP2

CABRILLO COLLEGEEnglish Content Review: CTEP Part 2. Sentence Structure & Grammar

Summary of Average Item RatingsInstructions: Please rate each item’s importance for each English course using the scale below.

How important is the academic knowledge or skill measured by this item forsuccessful acquisition of the skills taught in this course?

5. Critically Important4. Important3. Moderately Important2. Of Slight Importance1. Not Relevant

Test Item Number

English 290/255 English 100 English 1A

# of Raters*: N=1 N=4 N=41 4.0 3.0 4.32 3.0 3.0 4.33 4.0 3.5 4.34 4.0 3.3 4.35 3.0 2.8 4.06 3.0 3.0 4.37 3.0 2.5 3.88 3.0 3.3 4.39 4.0 3.0 4.0

10 3.0 3.0 4.311 4.0 3.0 4.312 3.0 3.0 4.313 2.0 2.5 3.814 2.0 2.8 4.015 3.0 3.0 4.316 3.0 2.8 4.017 3.0 3.3 4.318 4.0 3.0 4.319 2.0 3.0 4.320 2.0 2.8 4.021 2.0 2.8 4.322 3.0 3.0 4.323 3.0 3.0 4.324 3.0 3.0 4.325 3.0 3.0 4.326 3.0 3.0 4.327 3.0 2.8 4.028 4.0 3.0 4.029 3.0 2.8 4.030 3.0 2.8 4.0

*Raters were: R1, R2, R3 and R5

Average Rating 3.1 2.9 4.1

#Items 4.0 or higher 7 0 28#Items 3.0-3.9 18 20 2#Items 2.0-2.9 5 10 0#Items below 2.0 0 0 0



ENGL-100 Domain Review CTEP1

CABRILLO COLLEGE English Content Review: CTEP Part 1. Reading ComprehensionSummary: English 100 Pre-Skills Match to Test Domains

English 100 Pre-skillsBased Upon

# of RatersCourse Content

%*4 100% 100% 50% 50% 50% 100%4 100% not 50% 75% 50% 100% not 75%4 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄4 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄

4 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄

4 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄4 100% not 100% not 100% not 100% not 75% 75%

4 75% not 50% 75% not 75% not 50% 75%

4 100% 100% 100% 100% 100% 100%

Interpretation: 50% - two raters agreed that the pre-skill was measured by the domain100% - all four raters agreed that the pre-skill was measured by the domain 75% not - three raters agreed that the pre-skill was not measured by the domain 75% - three raters agreed that the pre-skill was measured by the domain 100% not - all four raters agreed that the pre-skill was not measured by the domain

CTEP Part 1. Reading Comprehension Domains

English 290/255Main Idea Literal Comp. Inferential

Comp.Interp./Eval Compreh.

Vocabulary in Context

Check mark any CTEP domain(s) that, in your professional opinion, measure the numbered pre-skill.

1. Learn to address a topic clearly.2. Develop ideas using detail and evidence. 3. Write unified paragraphs.4. Begin to vary sentence structure

9. Show improvement in reading comprehension by learning to identify an author's main ideas and purpose and through vocabulary development.

5. Begin to identify and correct mechanical errors, including errors in punctuation, sentence structure, verb tense, pronoun and subject-verb agreement, and spelling6. Begin to eliminate frequent errors in idiomatic usage7. Learn to use the dictionary to enlarge vocabulary and improve spelling8. Learn to write essays with clear ideas supported by some evidence, a coherent sequence of paragraphs, competent control of language, and no pervasive pattern of errors

*Percentage of raters who indicated that the course pre-skill was measured by at least one of the CTEP Reading Comp. domains.



ENGL-1A Domain Review CTEP1

CABRILLO COLLEGE English Content Review: CTEP Part 1. Reading ComprehensionSummary: English 1A Pre-Skills Match to Test Domains

English 1A Pre-skillsBased Upon


%*4 100% 100% 50% 50% 50% 100%4 100% not 50% 75% 50% 75% not 75%4 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄4 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄


4 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄4 100% not 100% not 100% not 100% not 75% 75%

4 75% not 75% not 75% not 100% not 75% not 50%

4 100% 100% 75% 75% 75% 100%

4 75% 75% 50% 75% not 75% not 75%

4 75% not 50% 50% 50% 100% not 50%




4 100% not 50% 50% 100% 75% not 100%

4 100% not 100% not 100% not 100% not 75% 75%



4 100% 75% 75% 75% 75% not 100%

4 100% 75% 100% 75% 100% not 100%

4 75% 75% 50% 50% 100% not 100%


21. Learn to develop well-organized responses to readings and answers to questions in essay exams, drawing reasonable conclusions and using specific evidence to support main points.

17. Learn to write essays with clear ideas supported by some evidence, a coherent sequence of paragraphs, competent control of language, and no pervasive pattern of errors18. Learn to devise effective introductions and conclusions19. Learn to read actively by annotating, paraphrasing, and summarizing; learn to analyze and evaluate the ideas of other writers.

20. Learn to examine texts carefully to identify main points and the writer's point of view; see the connections between individual parts and the whole.

13. Learn to vary sentence length and establish connections among ideas, including the use of coordination and subordination.

14. Learn to write essays free of most distracting errors in syntax and mechanics.

15. Learn the importance of their writing of voice, tone, and careful diction in addressing an audience; in their reading, become aware of tone, implication, inference, and irony, and learn to distinguish fact from opinion. 16. Learn to use the dictionary to enlarge vocabulary and improve spelling


10. Learn to focus on a topic.11. Learn more sophisticated strategies to develop an essay, using a variety of narrative and expository techniques. 12. Learn to unify paragraphs and make clear transitions between them.



CTEP Part 1. Reading Comprehension Domains

English 100Main Idea Literal Comp. Inferential

Comp.Interp./Eval Compreh.

Vocabulary in Context


*Percentage of raters who indicated that the course pre-skill was measured by at least one of the CTEP Reading Comp. domains.



ENGL-100 Domain Review CTEP2

CABRILLO COLLEGE English Content Review: CTEP Part 2. Sentence Structure & GrammarSummary: English 100 Pre-Skills Match to Test Domains

English 100 Pre-skillsBased Upon

Parallelism Errors

Modifier Errors

Wordiness Errors


%*4 100% not 100% not 100% not 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄4 100% not 100% not 100% not 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄4 100% not 75% not 75% not 75% not 50% 75% not 50% 75% not 50%4 75% 50% 75% 75% 75% 50% 75% 75% 100%

4 100% 100% 100% 75% 75% 50% 75% 100% 100%

4 100% not 100% not 100% not 75% not 75% not 100% not 75% 100% not 100%4 100% not 100% not 100% not 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄

4 75% 75% 75% 75% 75% 50% 75% 75% 75%

4 100% not 100% not 100% not 100% not 100% not 100% not 50% 100% not 50%





CTEP Part 2. Sentence Structure & Grammar Domains

English 290/255Sentence Errors

Verb Errors Pronoun Errors

Semantic Errors

Punctuation Errors


*Percentage of raters who indicated that the course pre-skill was measured by at least one of the CTEP Sent. Struct. & Grammar domains.



ENGL-1A Domain Review CTEP2

CABRILLO COLLEGE English Content Review: CTEP Part 2. Sentence Structure & GrammarSummary: English 1A Pre-Skills Match to Test Domains

English 1A Pre-skillsBased Upon

Parallelism Errors

Modifier Errors

Wordiness Errors


%*4 100% not 100% not 100% not 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄4 100% not 100% not 100% not 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄4 100% not 75% not 75% not 75% not 50% 75% not 50% 75% not 50%4 75% 50% 75% 75% 75% 50% 75% 75% 100%

4 100% 100% 100% 75% 50% 50% 50% 75% 100%

4 100% not 100% not 100% not 75% not 75% not 100% not 75% 100% not 100%4 100% not 100% not 100% not 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄

4 100% 75% 75% 100% 75% 75% 100% 100% 100%


4 100% not 100% not 100% not 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄

4 75% not 100% not 75% not 75% not 100% not 75% not 75% not 75% not 50%


4 100% 75% 50% 100% 75% 50% 75% 100% 100%

4 100% 75% 75% 75% 75% 50% 100% 100% 100%



4 75% 75% 75% 100% 75% 100% 100% 100% 100%

4 75% not 75% not 75% not 100% not 75% not 50% 75% 75% not 75%


4 100% not 100% not 100% not 100% not 100% not 100% not 100% not 100% not 0% ◄◄◄



CTEP Part 2. Sentence Structure & Grammar Domains

English 100Sentence Errors

Verb Errors Pronoun Errors

Semantic Errors

Punctuation Errors


1. Learn to address a topic clearly.2. Develop ideas using detail and evidence. 3. Write unified paragraphs.4. Begin to vary sentence structure5. Begin to identify and correct mechanical errors, including errors in punctuation, sentence structure, verb tense, pronoun and subject-verb agreement, and spelling6. Begin to eliminate frequent errors in idiomatic usage7. Learn to use the dictionary to enlarge vocabulary and improve spelling8. Learn to write essays with clear ideas supported by some evidence, a coherent sequence of paragraphs, competent control of language, and no pervasive pattern of errors


10. Learn to focus on a topic.11. Learn more sophisticated strategies to develop an essay, using a variety of narrative and expository techniques. 12. Learn to unify paragraphs and make clear transitions between them. 13. Learn to vary sentence length and establish connections among ideas, including the use of coordination and subordination.

14. Learn to write essays free of most distracting errors in syntax and mechanics.

15. Learn the importance of their writing of voice, tone, and careful diction in addressing an audience; in their reading, become aware of tone, implication, inference, and irony, and learn to distinguish fact from opinion. 16. Learn to use the dictionary to enlarge vocabulary and improve spelling

21. Learn to develop well-organized responses to readings and answers to questions in essay exams, drawing reasonable conclusions and using specific evidence to support main points.

17. Learn to write essays with clear ideas supported by some evidence, a coherent sequence of paragraphs, competent control of language, and no pervasive pattern of errors18. Learn to devise effective introductions and conclusions19. Learn to read actively by annotating, paraphrasing, and summarizing; learn to analyze and evaluate the ideas of other writers.

20. Learn to examine texts carefully to identify main points and the writer's point of view; see the connections between individual parts and the whole.

*Percentage of raters who indicated that the course pre-skill was measured by at least one of the CTEP Sent. Struct. & Grammar domains.



cabrillo college ctep english placement validation portfolio validity... · cabrillo college ctep...

Documents