illinois higher education performance funding model

IBHE Presentation 1

Illinois Higher EducationPerformance Funding Model

Steering Committee MeetingMay 8, 2013

Dr. Alan Phillips

Purpose

• The purpose of this presentation is to identify and address issues associated with the refinement of the performance funding model that will allocate funding based on performance as a part of the FY 2015 IBHE Higher Education budget submission.

• Accomplish this in accordance with the intent of Public Act 97-320 (HB 1503), the Performance Funding legislation, and the goals of the Illinois Public Agenda.

IBHE Presentation 2

Performance Funding Objective

• To develop performance funding models for public universities and community colleges that are…

– Linked directly to the Goals of the Illinois Public Agenda and the principles of Public Act 97-320

– Equipped to recognize and account for each university’s mission and set of circumstances

– Adjustable to account for changes in policy and priorities

– Not prescriptive in how to achieve excellence and success

3IBHE Presentation

• Performance Metrics Shall:– Reward performance of institutions in advancing the

success of students who are:• Academically or financially at risk.• First generation students.• Low-income students.• Students traditionally underrepresented in higher education.

– Recognize and account for the differentiated missions of institutions of higher education.

– Focus on the fundamental goal of increasing completion.– Maintain the quality of degrees, certificates, courses, and

programs.– Recognize the unique and broad mission of public

community colleges.

IBHE Presentation 4

Public Act 97-320 (HB 1503)

Considerations

• All things being equal, simple is always better.

• The model has to be complex enough to do what it needs to do and yet be simple enough that it can be explained to all stakeholders

• The more measures we add, the more we dilute the weights for the existing measures.

• This is not an exact science, and in some cases we are using proxies for what we really want to measure because direct measures are not available.

• You cannot capture the infinite number of variables with nine measures and five sub-categories.

• Data availability continues to be a problem, so we will use the best data we have.

IBHE Presentation 5

FY13 Performance Measures

IBHE Presentation 6

Measure Source• Bachelors Degrees (FY07-09) IPEDS• Masters Degrees (FY07-09) IPEDS• Doctoral and Professional Degrees (FY07-09) IPEDS• Undergraduate Degrees per 100 FTE (FY07-09) IPEDS• Education and General Spending per Completion (FY09-11) RAMP• Research and Public Service Expenditures (FY09-11) RAMP

FY13 Sub-Categories

IBHE Presentation 7

Sub-Category• Low Income (Pell/Map Eligible)

• Adult (Age 25 and Older)

• Hispanic• Black, non-Hispanic

• STEM & Health Care (by CIP Code)

FY14 Refinement Goals• Refine the existing measures and sub-categories to the extent

possible or find replacement measures that capture what we are trying to measure in a better way (i.e. Research Expenditures, Low Income Students, Cost per FTE, etc.).

– Identify additional measures and sub-categories to add to the model.

– Identify better and more current sources of data.

– See if there is a better way to scale (normalize) the data.

– Discuss ways to account for other factors (i.e. Hospitals, Medical Schools, Dental Schools, etc.)

IBHE Presentation 8

FY14 Changes to the Performance Measures

• Deleted: – Education and General Spending per Completion

(RAMP)

• Added:– Cost per Credit Hour. (Cost Study)– Cost per Completion. (Cost Study)– Credit Hour Accumulation. (Institutional Data)– Graduation Rates (Institutional Data)

IBHE Presentation 9

FY14 Changes to the Sub-Categories

• Did not change the sub-categories.• Changed some of the sources of data.• Sub-Categories were:

– Low Income (Pell/MAP Eligible) – Institutional Data– Adult (Age 25 and Older) – CCA/ILDS – Hispanic - IPEDS– Black, non-Hispanic - IPEDS– STEM & Health Care (By CIP Code) – HLS + CIP 51

IBHE Presentation 10


Performance Funding Model (FY14)

4-Year Public Universities

Performance Funding Model Steps(4-Year Public University)

• Step 1 – Identify the performance measures or metrics that support the achievement of the state goals.

• Step 2 – Collect the data on the selected performance measures• Step 3 – Award an additional premium (i.e. 40%) for the production of certain

desired outcomes such as completions by underserved or underrepresented populations

• Step 4 – Normalize (scale) the data, if necessary, so it is comparable across variables.

• Step 5 – Weight each of the Performance Measures to reflect the priority of the Measure to the mission of the institution.

• Step 6 – Multiply and sum the Scaled Data times the Weight to produce the Weighted results.

• Step 7 – Add an adjustment factor for high cost entities (i.e. Hospitals, Medical, Dental, and Veterinary Schools).

• Step 8 – Use the final Weighted results (or Total Performance Value) to distribute performance funding.


Performance Measures


Measure Source• Bachelors Degrees (FY09-11) IPEDS• Masters Degrees (FY09-11) IPEDS• Doctoral and Professional Degrees (FY09-11) IPEDS• Undergraduate Degrees per 100 FTE (FY09-11) IPEDS• Research and Public Service Expenditures (FY10-12) RAMP• Grad Rates 100%/150%/200% of Time (Fall 02-04 Cohort) Institutional Data• Persistence (Completed 24/48/72 Semester Hours) (FY07-09) Institutional

Data• Cost per Credit Hour (FY09-11) Cost Study• Cost per Completion (FY09-11) Cost Study

Step 1 – Identify the performance measures or metrics that support the achievement of the state goals.

Step 2 – Collect the data on the selected performance measures (3-year averages)

Sub-Categories


Sub-Category Weight• Low Income (Pell/Map Eligible) 40% - Institutional

Data• Adult (Age 25 and Older) 40%• Hispanic 40%• Black, non-Hispanic 40%• STEM & Health Care (by CIP Code) 40% - HLS + CIP 51

Step 3 – Award an additional premium for the production of certain desired outcomes such as completions by underserved or underrepresented populations

Scaling Factors

• Averaged the measures across all of the institutions.• The average number of bachelors degrees will serve as the

base value.• Determine a scaling factor that will normalize the rest of the

averages to the average number of bachelors degrees.• Adjust the scaling factors as appropriate (i.e. Masters &

Doctorates).• Multiply all of the initial data by the scaling factor to

normalize the data.


Step 4 – Normalize (scale) the data, if necessary, so it is comparable across variables.

Scaling Factors


Step 4 – Normalize (scale) the data, if necessary, so it is comparable across variables.

Measure Universities 1-12 (Avg) Scaling Factor Adjusted Scaling Factor• Bachelors Degrees (FY09-11)• Masters Degrees (FY09-11)• Doctoral and Professional Degrees (FY09-11) • Undergraduate Degrees per 100 FTE (FY09-11)• Grad Rates 100% of Time (Fall 02-04 Cohort)• Grad Rates 150% of Time (Fall 02-04 Cohort)• Grad Rates 200% of Time (Fall 02-04 Cohort) • Persistence (Completed 24 Semester Hours) (FY07-09)• Persistence (Completed 48 Semester Hours) (FY07-09)• Persistence(Completed 72 Semester Hours) (FY07-09)• Cost per Credit Hour (FY09-11) (Cost Study)• Cost per Completion (FY09-11) (Cost Study)• Research and Public Service Expenditures (FY09-11)

2,8221,042

22725274650

1,6441,4531,350

34636,566

112,914,667

1.02.7

12.4112.6104.4

60.957.0

1.71.92.18.1

.1.00002

112

200505050

222

-8-.050

.00005

Performance Measure Weights


Step 5 – Weight each of the Performance Measures to reflect the priority of the Measure to the mission of the institution.

Doctoral/ Research-Very High Research-High Research

Measure• Bachelors Degrees• Masters Degrees• Doctoral and Professional Degrees• Undergraduate Degrees per 100 FTE • Grad Rates 100% of Time• Grad Rates 150% of Time • Grad Rates 200% of Time • Persistence (Completed 24 Semester Hours) • Persistence (Completed 48 Semester Hours) • Persistence (Completed 72 Semester Hours) • Cost per Credit Hour • Cost per Completion• Research and Public Service Expenditures

UIUC UIC NIU SIUC ISU17.0% 18.0% 28.0% 28.0% 33.0%14.0% 15.0% 15.0% 14.0% 23.0%13.0% 14.0% 10.0% 8.0% 6.0%

4.0% 4.0% 11.0% 13.0% 12.0%1.5% 1.5% 1.5% 1.5% 2.0%1.0% 1.0% 1.0% 1.0% 1.5%0.5% 0.5% 0.5% 0.5% 0.5%0.5% 0.5% 0.5% 0.5% 0.5%1.0% 1.0% 1.0% 1.0% 1.5%1.5% 1.5% 1.5% 1.5% 2.0%0.5% 0.5% 1.0% 0.5% 1.5%0.5% 0.5% 1.0% 0.5% 1.5%

45.0% 42.0% 28.0% 30.0% 15.0%100.0% 100.0% 100.0% 100.0% 100.0%

Performance Measure Weights


Step 5 – Weight each of the Performance Measures to reflect the priority of the Measure to the mission of the institution.

Masters Colleges & Universities (Large)

Measure• Bachelors Degrees• Masters Degrees• Doctoral and Professional Degrees• Undergraduate Degrees per 100 FTE • Grad Rates 100% of Time• Grad Rates 150% of Time • Grad Rates 200% of Time • Persistence (Completed 24 Semester Hours) • Persistence (Completed 48 Semester Hours) • Persistence (Completed 72 Semester Hours) • Cost per Credit Hour • Cost per Completion• Research & Public Svc Expenditures

SIUE WIU EIU NEIU CSU GSU UIS42.0% 40.0% 40.0% 40.0% 40.0% 45.0% 43.0%28.0% 25.0% 26.0% 26.0% 25.0% 27.0% 27.0%

2.5% 1.0% 0.0% 0.0% 1.0% 1.0% 1.0%12.0% 13.0% 13.0% 13.0% 13.0% 5.0% 8.0%

2.0% 2.5% 2.5% 2.5% 2.5% 0.0% 2.5%1.5% 2.0% 2.0% 2.0% 2.0% 0.0% 2.0%1.0% 1.0% 1.0% 1.0% 1.0% 0.0% 1.0%1.0% 1.0% 1.0% 1.0% 1.0% 5.0% 1.0%1.5% 2.0% 2.0% 2.0% 2.0% 7.0% 2.0%2.0% 2.5% 2.5% 2.5% 2.5% 0.0% 2.5%1.5% 4.0% 4.0% 4.0% 4.0% 4.0% 4.0%1.5% 4.0% 4.0% 4.0% 4.0% 4.0% 4.0%3.5% 2.0% 2.0% 2.0% 2.0% 2.0% 2.0%

100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%

Performance Value Calculation


Step 6 – Multiply and Sum the Scaled Data times the Weight to produce the Performance Value for each institution.

Measure• Bachelors Degrees• Masters Degrees• Doctoral and Professional Degrees• Undergraduate Degrees per 100 FTE • Grad Rates 100% of Time• Grad Rates 150% of Time • Grad Rates 200% of Time • Persistence (Completed 24 Semester Hours) • Persistence (Completed 48 Semester Hours) • Persistence (Completed 72 Semester Hours) • Cost per Credit Hour • Cost per Completion• Research & Public Svc Expenditures

2,8221,042

22725274650

1,6441,4531,350

34636,566

$112,914,667

3,5221,454

240 25274650

1,6441,4531,350

34536,566

$112,914,667

Data Data + Premium112

200505050

222

-8-.050

.00005

(Data+Premium)x Scale3,5221,454

4805,0001,3502,3002,5003,2882,9062,700

-2,760-1,8285,646

30.0%25.0%

5.0%10.0%

1.5%1.0%0.5%1.0%1.5%2.0%1.5%1.0%

20.0%100.0%

Total PerformanceValue

1,057 364

24 500

20 23 13 33 44 54

-41 -18

1,1293,200

xWeight = Scale

Performance Value Calculation


Step 7 – Add an adjustment factor for high cost entities (i.e. Hospitals, Medical, Dental, and Veterinary Schools)

• Divide the amount of the university GRF appropriation allocated to fund the high cost entity by the total university GRF appropriation.

• Multiply this factor by the university performance value and add the result back to the performance value.

• This results in a total performance value for institutions with these high cost entities.

• Example: $20M/$200M = .10

.10 X 3200 (PV) = 320

320 + 3200 = 3520 = Total Performance Value


Percentages for Distribution

Total Performance Value 10,840 4,435 3,200 17,302Percentage of Total 58.7% 24.0% 17.3% 100%

Distribution: Pro Rata $587,000 $240,000 $173,000 $1,000,000

University 1 University 2 University 3 Total

Funding AllocationBased on Performance

Step 8 – Use the Weighted results (or Total Performance Value) to distribute funding based on a Pro Rata Share of the total amount of funds set aside for performance funding.

Results for FY14

• Performance funding values increased for all twelve of the four-year public universities from FY13 to FY14.

• Assuming a .5% funding set-aside and a 4.95% reduction in GRF Funding:

– Variance in funding allocations due to performance ranged from +.21% to -.22%.

– The actual funding amount variance ranged from +$70K to -$101K.



FY14 Performance Funding Allocation Performance Funding

($ in thousands) FY2013 FY2014 FY2014 FY2014 FY2014 FY2014 FY2013 - FY 2014

Appropriation Budget

Reduction % Change Set Aside*Performance

Funds Net Change Final Allocation $ Change %

Change

$ 1,230,092.0 $ 1,169,202.4 (4.95) % $ 5,866.3 $ 5,866.3 $ 0.0 $ 1,169,202.4 $ (60,889.6) (4.95)

Chicago State University 36,805.6 34,983.7 (4.95) 175.4 136.9 -38.5 34,945.2 (1,860.4) (5.05)

Eastern Illinois University 44,041.1 41,861.1 (4.95) 210.2 244.9 34.7 41,895.8 (2,145.3) (4.87)

Governors State 24,650.5 23,430.3 (4.95) 117.7 140.7 23.0 23,453.3 (1,197.2) (4.86)

Illinois State University 74,082.4 70,415.3 (4.95) 353.3 359.2 5.9 70,421.2 (3,661.2) (4.94) Northeastern Illinois University 37,807.6 35,936.1 (4.95) 180.4 217.9 37.5 35,973.6 (1,834.0) (4.85)

Northern Illinois University 93,470.2 88,843.4 (4.95) 445.6 390.1 -55.4 88,788.0 (4,682.2) (5.01)

Western Illinois University 52,147.8 49,566.5 (4.95) 248.7 258.1 9.4 49,575.8 (2,572.0) (4.93)

Southern Illinois University ** 204,693.8 194,561.5 (4.95) 975.8 870.1 -105.7 194,455.7 (10,238.1) (5.00)

Carbondale 145,316.4 138,123.2 (4.95) 692.8 591.8 -101.0 138,022.2 (7,294.2) (5.02)

Edwardsville 59,377.4 56,438.2 (4.95) 283.0 278.2 -4.7 56,433.5 (2,943.9) (4.96)

University of Illinois *** 662,393.0 629,604.5 (4.95) 3,159.3 3,248.5 89.3 629,693.8 (32,699.2) (4.94)

Chicago 305,571.6 290,445.8 (4.95) 1,457.4 1,479.2 21.8 290,467.6 (15,104.0) (4.94)

Springfield 23,152.7 22,006.6 (4.95) 110.5 108.0 -2.5 22,004.1 (1,148.6) (4.96)

Urbana/Champaign 333,668.7 317,152.1 (4.95) 1,591.4 1,661.3 70.0 317,222.1 (16,446.6) (4.93)

FY2014 Set Aside for allocation per performance funding is based on a 0.5% reallocation of a $1.173 billion budget for public universities. Governor's budget proposal cuts public university funding by 4.95% from the FY2013 appropriation. SIU Administration is allocated on a pro-rated basis to each campus, SIU School of Medicine is included with the Carbondale Campus. UI Administration is allocated on a pro-rated basis to each campus.

Refinement Issues for FY15

• What is the best way to account for the difficulty of getting underrepresented students through to completion throughout the model?

• What is the best way to account for less prepared students in the model?

• Are there differences in the cost per completion for different sub-categories of students (i.e. is cost for completion for an adult student different than that of a STEM student)? Should that be integrated in the model?

• What is the best way to address the issue of transfer students and part-time students?



• Are there other high value degrees and programs, in addition to the STEM programs, that we should add to the model?

• Are we giving enough priority to measures of efficiency?

• What is the best way to account for high cost entities (i.e. Hospitals and Medical, Dental, and Veterinary schools)?

• Are we adequately accounting for institutional improvement from year to year?

• Do we change the measures or the sub-categories?

• Do we change the sources of the data for the model?


Refinement Issues for FY15• What is the best way to account for the difficulty of getting

underrepresented students through to completion throughout the model?

– Four of the five sub-categories address underrepresented students and these sub-categories are weighted in the Bachelors, Masters, and Doctoral completion measures.

– They are, however, not weighted in the Cost per Completion or the Completion per 100 FTE measures.

– Do we weight the completions in these two categories, and if so, do we readjust the sub-category weights so as to not overweight these two categories in the model?

– Are there other ways to account for the challenge of educating these students that we should consider?

• Refinement Committee: Take a look at the results of the current methodology relative to the results of weighting these students throughout the model to determine which method would produce better results.



• What is the best way to account for less prepared students in the model?

– We do not currently have a good way to track these students.

– These students tend to be underrepresented students which are already captured in the sub-category weightings.

– One possibility would be to add another subcategory for remedial students

• (defined as first-time undergraduate students who complete remedial education courses in math, English/reading, or both, and are awarded a bachelors degree – (CCA Definition))

• Refinement Committee – No change to the model. These students are adequately accounted for in the existing model.



• Are there differences in the cost per completion for different sub-categories of students (i.e. is cost for completion for an adult student different than that of a STEM student)? Should that be integrated in the model?

– There are differences in the costs between sub-categories.

– The costs for each sub-category would vary by institution.

– There is no data available to determine those costs.

– We do not know how we would incorporate these cost differentials into the model.

• Refinement Committee – No change to the model. Would significantly complicate the model, with marginal benefit.


Refinement Issues for FY15• What is the best way to address the issue of transfer students

and part-time students?– By definition, IPEDS data does not include transfer or part-time students

in the calculation of graduation rates or retention measures.

– Based on survey input provided by the institutions, with few exceptions, part-time students were low density students and did not significantly affect the model outcome.

– For transfer students, it might be possible to use CCA transfer categories (i.e. 30 or fewer credits, 31 to 59 credits, or 60 or more credits)

• Refinement Committee – Part-time student numbers will not be included due to their low density. Transfer student numbers will be incorporate in the Graduation Rate and Persistence Measures. As these numbers are not reflected in IPEDS, transfer student numbers will be provided by the institutions.



• Are there other high value degrees and programs, in addition to the STEM programs, that we should add to the model?

– The current STEM program list consists of the Homeland Security (HLS) STEM program list and the CIP Code 51.

– There are other programs that could be added to the list of STEM programs such as behavioral or health/nutrition related fields.

– When you begin to move away from clearly defined criteria, the determination of what should or should not be STEM, becomes much less clear and much more subject to interpretation.

• Refinement Committee – Stay with the current list of STEM programs (i.e. HLS List + CIP 51).


Refinement Issues for FY15• Are we giving enough priority to measures of efficiency?

– Current measures of efficiency include:

• Undergraduate Degrees per 100 FTEs

• Graduation rates per percentage of time

• Cost per Credit Hour

• Cost per Completion

– Should these measures be given increased weighting?

– Should we add additional efficiency measures?

• Refinement Committee – No change to the model. Given there are four efficiency measures, the Refinement Committee viewed the existing weighting to be sufficient and expressed concern that the addition of another efficiency measure would only serve to dilute the weighting of the other existing performance measures.


Refinement Issues for FY15• What is the best way to account for high cost entities (i.e.

Hospitals and Medical, Dental, and Veterinary schools)? – Need to be accounted for in the model in some way.

– Current methodology - Does not account for all of the costs.

– Complete Carve-Out – However, some schools benefit in the model from these entities (i.e. Completions and Public Service Expenditures).

– Is there another method that would better account for these non-performance costs in the model?

– Are their other high cost entities that should be added to the list?

• Refinement Committee – A complete carve-out would create additional problems and issues. Therefore, we will work to develop a better methodology to account for these entities. No other high cost entities should be added at this time.


Refinement Issues for FY15• Are we adequately accounting for institutional improvement from year to

year?– Every institution improved its performance funding scores from FY13 to

FY14.– There has been some discussion that each institution should be measured

against itself.• Until the performance funding model is stabilized, the scores from the current year are

not directly comparable to the scores from the previous year due to changes in the model from year to year.

• In the future, the performance value for each institution could be compared against the previous performance value for that institution.

• The scores for each individual measure at each institution can be compared against the previous scores for those measures at each institution where they are consistent from year to year.

• Refinement Committee – No change to the model at this time. We can relook this issue once we get to a point where the model does not change from year to year.


Refinement Issues for FY15• Do we change the measures or the sub-categories

– Do we change the Graduation Rate Measure(s)?– Do we change the Credit Hour Accumulation Measure(s)?– Do we add to the sub-categories (i.e. a remediation measure)?

• Do we change the sources of data for the model?– IPEDs– Survey– CCA– Cost Study

• Refinement Committee – Use the 150% of Time, Graduation Rate Measure, but incorporate

transfer students per the CCA definitions. (Survey Data)– Use the 24 Semester Credit Hours Completed in the First Year Measure,

to include credits earned at other institutions. (Survey Data)– Do not add any additional sub-categories.– Continue to use the existing data sources.


Performance Funding Model Steps(4-Year Public University)

• Step 1 – Identify the performance measures or metrics that support the achievement of the state goals.

• Step 2 – Collect the data on the selected performance measures.• Step 3 – Award an additional premium (i.e. 40%) for the production of certain

desired outcomes such as completions by underserved or underrepresented populations

• Step 4 – Normalize (scale) the data, if necessary, so it is comparable across variables. – (No change to the Scaling Methodology)

• Step 5 – Weight each of the Performance Measures to reflect the priority of the Measure to the mission of the institution.

• Step 6 – Multiply and sum the Scaled Data times the Weight to produce the Weighted results.

• Step 7 – Add an adjustment factor for high cost entities (i.e. Hospitals, Medical, Dental, and Veterinary Schools).

• Step 8 – Use the final Weighted results (or Total Performance Value) to distribute performance funding.

Bold - Indicates Steps that are under review by the Refinement Committee


FY15 Performance Measures


Measure Source• Bachelors Degrees (FY10-12) IPEDS• Masters Degrees (FY10-12) IPEDS• Doctoral and Professional Degrees (FY10-12) IPEDS• Undergraduate Degrees per 100 FTE (FY10-12) IPEDS• Research and Public Service Expenditures (FY11-13) RAMP• Graduation Rate - 150% of Time (Fall 03-05 Cohort)* Institutional

Data• Persistence-Completed 24 Semester Hours in One Year (FY08-10)* Institutional

Data• Cost per Credit Hour (FY10-12) Cost Study• Cost per Completion (FY10-12) Cost Study

* Incorporate transfers per the CCA transfer category definitions.

FY15 Sub-Categories


Sub-Category Weight• Low Income (Pell/Map Eligible) 40% - Institutional

Data• Adult (Age 25 and Older) 40%• Hispanic 40%• Black, non-Hispanic 40%• STEM & Health Care (by CIP Code) 40% - HLS + CIP 51

Other FY15 Issues Yet to be Resolved

• Determine whether or not to carry the weighted sub-categories all the way through the model.

• Develop a better methodology to account for the high cost entities in the model.

• Continue to work with the institutions to improve the quality of available data for use in the performance funding model.


Next Steps

• Steering Committee Meeting – May 8, 2013.

• Incorporate input from the Steering Committee.

• Continue work to resolve the issues.

• IBHE Board Meeting – June 4, 2013.

• Recommend changes to the model at the next refinement committee meeting – June 27, 2013.

• Continue work to resolve the issues.

• Steering Committee Meeting – July 17, 2013


Questions/Comments?


illinois higher education performance funding model

Documents

existing measures

ipeds education

public universities

additional measures

replacement measures

direct measures

illinois public agenda

principles of public