CAROLE BOGUE-FEINOUR, RETIRED VICE CHANCELLOR OF ACADEMIC AFFAIRS, CCCCO

MYRNA HUFFMAN, DIRECTOR MIS, CCCCO

JANET FULKS, ASCCC CURRICULUM CHAIR

JULY 2009CURRICULUM INSTITUTE

1

Measuring and Reporting Basic Skills Success; You

Need a Valid Instrument - CB 21 Coding

Background2

BSI directed attention to the ARCC dataSome measures did not make senseInvestigating the coding revealed inaccuraciesParticularly the

Basic Skills Progress Outcome ESL Progress Outcome

Progress for courses prior to transferNeeded to define the Supplemental Basic

Skills Report for accountability on the BSI funding and noncredit enhanced funding

“CB 21-Course Prior to Transfer Course Level”3

CB21 is code for the course “level”, in terms of number of levels below the transferrable levelHow many levels below transfer level is this course?

It is used primarily for basic skills but can be used for non-basic skills, degree-

applicable coursesIt is used only for English, writing, ESL,

reading, or mathematics (TOP codes)We are reporting credit courses; noncredit in

progress

MIS Data Element CB214

Is used for a lot of accountability reporting Which in turn is used to justify investments and

expenditures in basic skills ARCC Technical Advisory Group: defines metrics for

mandated reports

Is necessary to show student progress through basic skills curriculum 4…3…2…1…transferrable

Tracking Progress5

CB21 Course Progress to Transfer

Transfer Level Courses

Courses One Level Prior to Transfer

Courses Two Levels Prior to Transfer

Courses Three Levels Prior to Transfer

Courses Four Levels Prior to Transfer for Mathematics, English

and Reading. ESL has Five & Six levels prior to transfer.

Old Coding Instructions for Math6

Currently, CB21: A= 1 prereq. for transfer math

(Intermediate Algebra) B= 2 prereq./prep. for “A” (Algebra

I/Elem. Algebra) C= 3 prereq./prep. For “A/B”

(Arithmetic) Y= 4+ >3 levels below transfer level

(N/A)

Old Coding Instructions for English7

Currently, CB21: A= 1 prereq. for transfer Eng. Comp.

(Subject A) B= 2 prereq./prep. for “A” (Not available) C= 3 prereq./prep. For “A/B” (Not

available) Y= 4+ >3 levels below transfer level

(Not available)

Old Coding Instructions for Reading and ESL

8

Not addressed at all

Coding problems

No colleges had a common beginning; college level was different for every college – even colleges within a district

Some college coding had no progress – all courses one level

Some courses were in the wrong order of progression

Some courses were incorrectly identified as transfer

9

Sample Coding – All are Credit Courses

10

ElementaryGeometry DegApplicable Not Basic Skills 1 IntermediateAlgebra DegApplicable Not Basic Skills 1 ElementaryAlgebra DegApplicable Not Basic Skills 2 Pre Algebra ‐ NOT DegApplicable Basic Skills 3

ElementaryAlgebra DegApplicable Not Basic Skills 1 Arithmetic NOT DegApplicable Basic Skills 2

ElementaryAlgebra DegApplicable Not Basic Skills 1 ElementaryAlgebra 1st Half ‐ DegApplicable Not Basic Skills 1 ElementaryAlgebra 2nd Half ‐ DegApplicable Not Basic Skills 1 Intermediate Algebra DegApplicable Not Basic Skills 1

Sample Data – All are Credit Courses11

IntermediateAlgebra DegApplicable Not Basic Skills 1

Basic Mathematics NOT DegApplicable Basic Skills 3

Beginning Algebra NOT DegApplicable Basic Skills 4+

Sample Data – All are Credit Courses12

Course Id Title

Basic Skills Status

Transfer Status Level

ESLV33 Intermed Reading Comprehension B C BESLV01 Low-Beginning ESL B C CESLV40 Vocab&Communication: Nonnative B C CESLV34A Low-Begin Read Comprehension B C CESLV15 Inter ESL Listening & Speaking B C CESLV06 High-Advanced ESL B C CESLV05 Low-Advanced ESL B C CESLV03 Low-Intermediate ESL B C CESLV14 Begin ESL Listening & Speaking B C CESLV12 English Proficiency: Citizen B C CESLV02 High-Beginning ESL B C CESLV34B High-Begin Read Comprehension B C CESLV07 Reading Skills LEP B C CESLV04 High-Intermediate ESL B C CESLV08 Writing Skills LEP B C CESLV16 Adv ESL Listening & Speaking B C C

Sample Data13

TitleCredit Status

Basic Skills Status

Transfer Status Level

ELEMENTARY ALGEBRA D N C YELEMENTARY ALGEBRA II D N C YELEMENTARY ALGEBRA I D N C YINTERMEDIATE ALGEBRA D N C YMATH: OPERATIONS WITH WHOLE NUMBERS C B C YMATH: OPERATIONS WITH DECIMALS C B C YMATH: OPERATIONS WITH RATIONAL NUMBERS C B C YOPERATIONS WITH FRACTIONS, DECIMALS AND % C B C YPRE-ALGEBRA C B C YBASIC MATHEMATICS SKILLS C B C Y

Enrollments(SX)

StudentDemographics

(SB)Sections

CoursesFin.Aid

Assess.

PBS

VTEA

Matric.

Pgm.Awds.

Emp.Demo.

Sessions

Calendar Assignments

EOPSDSPS

Emp.Assign.

Cal-WORKs

CCC MIS Database

15

Why do we code courses?

What are the data uses?

16

Research Questions

• Legislative Analyst Office • Department of Finance • California Postsecondary

Education Commission • California Student Aid

Commission• Public Policy Institute• UC/CSU• Legislature – Committees and

individual members• Community College

Organizations• Newspapers• Labor Unions

Data Matches

• Transfer to UC/CSU/NSC match

• Dept. of Social Services • EDD/UI Match/Wage Study

Accountability Reporting

Justification & Funding • Matriculation• EOPS• DSPS

Career Technical EducationPerkins Core Indicator ReportsPerkins Allocations

BOGW Administrative Funding

Federal Integrated Postsecondary

Education Data System (IPEDS) Reporting

CCC Data Mart

Annual Staffing Report

MIS Data Element CB2117

Chancellor’s Office MIS system collects all course information each term

Courses are coded for identification purposesTOP code, credit status, transfer status,

units, basic skills status, SAM/voc code, etc.

CB21=Course Prior to College Level (Current)

MIS Data Element CB2118

Last changed in 1994

Defined number of “codeable” levels at 5 (xfer + 4 below)

Is used across math/English/reading/writing/ESL

Has little curricular definition of levels

ARCC Reporting19

2009 Statewide ARCC Data

What CB21 is used for20

Basic Skills Improvement Rate (ARCC)◦ Credit courses only◦ Completed (A,B,C,CR, P) any math/Eng basic

skills course at 2 or more levels below◦ Within 3 years, successfully completed a higher

level basic skills course of same discipline Anywhere in the system where SSN’s are

reported

◦ Current data range: 24%-62%, ◦ avg 49%.

What CB21 is used for21

ESL Improvement Rate (ARCC) Credit ESL courses only Completed (A,B,C,CR, P) any ESL course

at 2 or more levels below Within 3 years, successfully completed a

higher level ESL course Anywhere in the system where SSN’s are

reported Current data range: 0% to 81%, avg. 42%

The Strategy - Establishing a Rubric22

Is not standardizationDoes not drive curricular changesIs not common course numbering or

articulation

IS a mapping exercise designed to maximize our ability to show student progress AND your good work

Newly Designed CB 21 – Attached to Rubrics

23

Each college retains their own curriculumDoes not affect degree applicabilityAllows for uniformity throughout stateWill enhance collection of accurate and

comparable dataCan be used to collect assessment and

placement data

The Process24

140 FacultyResearch for standards, outcomes and exit

skillsDivided by DisciplineDeveloped rubrics based on need and

curriculumUniversally acceptable, not comprehensive

The results25

Rubrics created using 4 levels below transfer◦ Reading◦ Writing◦ Math,

ESL created 3 rubrics and needed 6 levels below transfer◦ ESL Writing◦ ESL Reading◦ ESL Listening and Speaking◦ **ESL Integrated

Vetting the Rubrics26

Demographics of Discipline Faculty Taking Part in the Vetting Process Faculty experience teaching basic skills Courses taught transfer and below

(select all that apply)

Full load 76 32.6% transfer 153 65.7%

> 50% of load 71 30.5% 1 level below 204 87.6%

25-50% load 67 28.8% 2 levels below 207 88.8%

<25% 9 3.8% 3 levels below 151 64.8%

no basic skills *10 4.3% 4 levels below 82 35.2%

totals 233 100% 5 levels below 30 12.9%

coding issues -teach geometry but not classified as basic skills etc 6 levels below 21 9.0%

assignment issues: adjuncts have varying loads, senate pres 7 levels below 10 4.3%

*some are chairs this semester 8 levels below 3 1.3%

Vetting the Rubrics27CB 21 Rubric-Vetting Data Results as of February 18, 2009

Discipline Number of respondents

Approval Contingent

Approval

Doesn’t work

other 4 levels adequate

6 levels adequate

comments on levels

other comments

Math 79 59 (74.7%)

18 (22.8%)

1 (1.3%)

1 (1.3%)

86.1% none 6.3% wanted 3 levels, 5.1% five levels

need to deal with geometry 1 or 2 levels below

Reading 29 12 (42.9%)

14 (50%)

1 (3.6%)

2 (7.1%)

50% few 25% want only 3 levels & 10.7% more than 4

78.6% (22) want grade levels included

ESL 35

ESL writing 21 9 (42.9%)

10 (47.6%)

1 (4.8%)

1 (4.8%)

3 (14.3%)

16 (76.2%)

some decisions based on local course offerings

including word count 61.9% yes 38.1% no

ESL listening & speaking

10 5 (50%)

5 (50%)

0 0

100%

ESL reading 4 3(75%) 1(25%) 0 0 0 75% 25% (1) 5 levels

English 69 38 (55.1%)

27 (39.1%)

3 (4.3%)

1 (1.4%)

30 (43.5%)

3 (4.5%)

much relates to ESL & student population 40.6% (28) 3 levels

word counts 52.2% helpful; 47.8% not helpful

other 21 These people answered some but not all of the questions within specific disciplines

total 233

Moving on to noncredit28

ABE/ASEMathEnglishESL integrated

Next Steps29

What will this mean?How will it change ARCC reporting?

Will the data suddenly show a big difference due to recoding?

How will you proceed?

Next Steps & Timeline30

Coding instructions CIOs and Researchers meetings and

listservsPossible web trainingContact people for questionsAll recoding must be done by

November 30

Questions31