Linking Strategic Planning, Budgeting, and Assessment

Rhode IslandJuly 14, 2006

National Center for Higher Education Management Systems3035 Center Green Drive, Suite 150 Boulder, Colorado 80301-2251

Dennis Jones, president (phone: 303-497-0315, email: dennis@nchems.org)Aims McGuinness, senior associate (phone: 303-497-0350, email: aims@nchems.org)

Karen Paulson, senior associate (phone: 303-497-0350, email: aims@nchems.org)

2

How Assessment/Evaluation, Planning, and Budgeting

Are Linked

3

The Management Cycle

Strategic Management —The allocation of resources to programmed activities calculated to achieve a set of goals.

Assessment

Planning

Resource Allocation

4

The Management Cycle in a Public Institution

State

Planning

Institution

State

Institution

Resource Allocation

State

Assessment

Institution

5

Overview of Rhode Island and Higher Education

How Does Each Institution Fit

into this Larger Picture?

6

Selected Data About Rhode Island and Its System of

Higher Education

National Collaborative on Postsecondary Education

7

Rhode Island Population Growth, 1990-2000

Source: U.S. Census Bureau, Brookings Institute

44,855 45,068

-213

-5,000

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

50,000

Total Population Hispanic Non-Hispanic

8

Projected Change in Rhode Island Population by Age and Race/Ethnicity, 2000-20

Source: U.S. Census Bureau

-26,577

5,256

35,347

22,021

5,303

1,909

1,806

4,273

1,637

24,274

9,001

17,754

16,888 9,656

10,044 3,525

6,114

6,251

3,113

-50,059-60,000

-40,000

-20,000

0

20,000

40,000

WhiteAfrican AmericanHispanicOther

0-17 18-24 25-44 45-64 65+

9

Rhode Island Educational Attainment by Race/Ethnicity (Percent)

84.5 86.8

30.8

82.8

71.3

37.6

73.5 74.1

17.6

56.552.2

8.7

57.2

50.3

7.3

0

20

40

60

80

100

Age 18-24 with High School Diploma Age 25-64 with High School Diploma Age 25-64 with Bachelor's or Higher

WhiteAsianAfrican AmericanHispanicOther

Source: U.S. Census Bureau

10

Projections of High School Graduates to 2018 By Race/Ethnicity—Rhode Island

0

2,000

4,000

6,000

8,000

10,000

12,000

2000-01

2001-02

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

2012-13

2013-14

2014-15

2015-16

2016-17

2017-18

White (6,988 to 5,828) Hispanic (771 to 2,713) Black (547 to 1,240)Asian/PI (273 to 335) AI/ AK Native (33 to 120)

Source: WICHE Projections of High School Graduates

11

Percent of African-Americans and Hispanics at Each Stage of the Education Pipeline—Rhode Island

5.3 5.5 5.8

5.0 4.8

11.6

8.4

6.1

4.7

3.9

0

3

6

9

12

18-Year-Olds High School Graduates First-Time CollegeFreshmen

All OtherUndergraduates

Credentials andDegrees Awarded

African-American Hispanic

Source: U.S. Census Bureau, NCES-Common Core Data, NCES-IPEDS Enrollment and Completions Surveys

12

2002 2004

28%35%81%

19%

18%

$4,000

28%35%78%

21%

18%

$3,997

U.S. Average

2004

Top States 2004

22%28%68%

40%

14%

$3,344

15%16%32%

89%

7%

$2,619

13

Affordability in Rhode Island—Percent Change from 2002-03 to 2003-04

[1] State appropriations had dropped in the previous year ($174 million in 2001-02 to $169 million 2002-03). Although the appropriations increased 2% (particularly when negative growth is common elsewhere), Rhode Island’s state appropriation per $1,000 personal income is the 42nd lowest in the nation.

[2] State grant aid in Rhode Island is entirely need-based and serves undergraduates only.[3] A rescission of 10% is expected according to the survey respondent, which would bring the appropriation figures down to $8.9 million, 82% increase from

the previous year.

2002-03 ($) 2003-04 ($) Change (%)

Tuition – Public 2-Year 2,014 2,120 5(23rd highest) (national avg. = 10)

Tuition – Public 4-Year 3,761 3,995 6(23rd) (national avg. = 12)

Per capita income 30,256 31,107 3(2001) (2002)

State appropriations [1] 169.6 million 172.8 million 1.9(national avg. = 2)

State grant [2] 4.9 million 9.9 million 102(actually spent) (appropriated) [3]

14

Percent of Total Gross State Product by Industry and Comparison to U.S.—Rhode Island

1 0

20

6 5

9

2120

12

1 0

5

11

65

9

30

22

12

1 1

5

14

87

9

2022

12

5

0

10

20

30

Agriculture,Forestry, and

Fishing

Mining Construction Manufacturing Transportationand Public

Utilities

Wholesale Trade Retail Trade Finance,Insurance, and

Real Estate

Services Government

Rhode Island 1990Rhode Island 2000U.S. 2001

Source: Bureau of Economic Analysis, www.bea.doc.gov

15

Net Migration by Degree Level and Age Group, 1995-2000—Rhode Island

Source: U.S. Census Bureau, 2000 Census; 5% Public Use Microdata Sample (PUMS) Files

22- to 29-Year-Olds 30- to 64-Year-Olds

Less than High School

High School

Some College

Associate

Bachelor’s

Graduate/Professional

Total 7,065

727

746

366

433

228

4,565

0 2,500 5,000 7,500

3,

-706

-2,804

600

2,141

1,551

2,812

-4,000 -2,000 0 2,000 4,

594

000

16

Rhode Island Occupations with High Net Imports and Exports, 1995-2000—Residents Age 22-29 with College Degrees

Source: U.S. Census Bureau, 2000 Census; 5% Public Use Microdata Sample (PUMS) Files

-384-343

-321-315-309

-296-253

-241-206

7199105107

119121129136

168419

-336

-500 -400 -300 -200 -100 0 100 200 300 400 500

Advertising, Mktg., Promotions, Public Rel., & Sales ManagersMedia & Communication WorkersInformation & Record ClerksPrimary, Secondary, & Special Education TeachersBusiness Operations SpecialistsEngineersSales Representatives, ServicesComputer SpecialistsHealth Diagnosing & Treating PractitionersFinancial Specialists

Water Transportation WorkersVehicle & Mobile Equip. Mechanics, Installers, & Repairers

Other Military OccupationsLaw Enforcement

Supervisors, Food Preparation & ServingOther Teachers & Instructors

Construction TradesNursing, Psychiatric, & Home Health Aides

Food & Beverage ServingPostsecondary Teachers

17

Rhode Island Occupations with High Net Imports and Exports, 1995-2000—Residents Age 30-64 with College Degrees

Source: U.S. Census Bureau, 2000 Census; 5% Public Use Microdata Sample (PUMS) Files

-282-234

-201-151

-115-107-106-103

-92138138143

159159

204208

215269272

-210

-300 -200 -100 0 100 200 300

Primary, Secondary, & Special Education TeachersOther Management OccupationsSupervisors, Food Preparation & ServingFinancial SpecialistsBusiness Operations SpecialistsPersonal Appearance WorkersArchitects, Surveyors, & CartographersComputer SpecialistsInformation & Record ClerksFirst-Line Supervisors/Mgrs., Protective Service Workers

Life, Physical, & Social Science TechniciansOther Personal Care & Service Workers

Health Diagnosing & Treating PractitionersDrafters, Engineering, & Mapping Technicians

Postsecondary TeachersMilitary Officer Special & Tactical Operations Leaders/Managers

Media & Communication WorkersEngineers

Operations Specialties ManagersAdvertising, Marketing, Promotions, Public Relations, & Sales Managers

18

Federal R&D Expenditures Per Capita, 2003

Source: National Science Foundation; U.S. Census Bureau

$0

$75

$150

$225

124.2

85.0

23.9

272.8

MarylandMassachusettsNew HampshireRhode IslandConnecticutHawaiiAlaskaColoradoPennsylvaniaVermontUtahNew MexicoNew YorkNorth DakotaW

ashingtonNorth CarolinaIowaMontanaW

isconsinAlabamaMissouriCaliforniaDelawareOregonUnited StatesMichiganIllinoisGeorgiaMississippiTexasTennesseeVirginiaOhioArizonaNebraskaMinnesotaKansasSouth CarolinaIndianaLouisianaNevadaW

yomingKentuckyNew JerseyIdahoW

est VirginiaFloridaSouth DakotaOklahomaArkansasMaine

19

Percent of Federal R&D Expenditures Generated by Private Sector,2001

Source: National Science Foundation; U.S. Census Bureau

87.0

57.5

39.1

0.00

20

40

60

80

100%

MassachusettsNew YorkConnecticutMissouriMarylandNew HampshireTennesseeRhode IslandIllinoisPennsylvaniaGeorgiaNorth CarolinaUnited StatesCaliforniaOhioLouisianaNew JerseyFloridaTexasW

isconsinIndianaNebraskaOklahomaVirginiaOregonUtahMaineVermontAlabamaColoradoSouth CarolinaMinnesotaIowaMichiganW

est VirginiaMississippiKentuckyW

ashingtonAlaskaArizonaArkansasDelawareHawaiiIdahoKansasMontanaNevadaNew MexicoNorth DakotaSouth DakotaW

yoming

20

Suggested Themes

Two Rhode Islands

Urban Core—Rest of State

Growing Income Disparities

Overall Slow Population Growth—Dramatic Increase in Latino Population

Education Attainment/Adult Literacy

Gaps in Preparation(continued)

21

Suggested Themes (continued)

Disparities in ParticipationGeographicEthnic

Gaps in Affordability

Gaps in Completion, Especially for Growing Latino Population

Need to Diversify Economy and Link R&D to Future Competitive Position of Rhode Island

22

Strategic Management Success Factors

Administrative Commitment

Organization-Wide Goals

Climate of Understanding and AcceptanceInstitutional LeadershipDepartments

Capture Control SystemsInformationRewardBudget

23

Planning

Making Decisions About Organizational Priorities that Specify What Changes in Intentions, Competencies or Behaviors the Organization Will Pursue

The Mechanism to Affirm Organizational Values and Methods by Which They Are Identified

24

Strategic Planning

Make Decisions (Now) Concerning the Desired Future

Develop Optimal Relationship Between the Organization and Its Environment

Provide Unity and Direction to All Organizational Decisionmaking—A Context for Operational Decisions

Objectives:

(continued)

25

Strategic Planning (continued)

Planning Must Result in Decisions

Planning Is Fundamentally a Change Process

26

Strategic Decision Areas

Basic Mission

Clientele

Program/Service Mix

Comparative Advantage

Assets

Objectives

Basic Purposes of the Enterprise and Its Guiding Principles for Behavior

Target Audiences of the Institution

Program Offerings and Priorities of the Institution

“Differential Advantage” Sought Over Other Organizations Engaged in Similar Activities

Changes Needed in Human, Physical, Information or Intangible Assets of the Enterprise

What the Organization Must Accomplish in Order to Move from Existing to Desired State of Affairs

27

Review of Mission Statements

28

Elements of Good Mission Statement

Intended Good Effects

for an

Identified Clientele

Within Specified or Assigned Constraints

or

With Specified Value Perspectives

29

Questions to Ask About a Mission Statement

Does It Contain the Three Key Ingredients?

Does Specification of Good Effects Provide Guidance for Assessment?

Does Statement Provide Broad Guidance for Resource Allocation?

Are Priorities Identifiable?

Does It Provide Guidance for Saying “No”?

30

The Domain of Community and Technical College Services

COMMUNITY ANDTECHNICAL COLLEGE

SERVICESIn-School Youth

(Secondary Education)

Recent High School Graduates Adults Employers

CLIENTS

Remedial and Developmental Education

General Education

Transfer Preparation

Career Preparation

Customized Training, Rapid Response Workforce Development

Community Service (Non-Credit and Other Services to the Community)

Brokering and Serving as a Delivery Site for Other Providers

31

Types of Mission Statements*

A “What We Do” Mission Statement

The College’s Mission Has Two Fundamental and Interrelated Components: the Provision of the Highest Quality Education and Dissemination of New Knowledge.

A “Who We Serve” Mission StatementThe Mission of College X Is to Improve the Lives and Personal Success of Our Students, Associates, and the Larger Community by Providing Superior Learning Opportunities in the Liberal Arts and Sciences.

*Chaffee, E. E. (1998). “Listening to the People We Serve” in The Responsive University. W. G. Tierney (Ed.). Baltimore, MD: The Johns Hopkins University Press.

32

Basic Mission

Clients to Be Served

Program/Service Mix

Comparative Advantage

Assets

Important to Begin with Actual or “Behavioral” Mission as Defined by Information on:

33

B R E A K

34

How Do You Know Your Institution’s Strengths and

Weaknesses?

35

Institutional Effectiveness/Assessment

How Does Your Institution Know Who It Really Is in Terms of Students, Programs, Faculty, etc.?

How Does Your Institution Know if It Has Achieved Its Goals?

36

Effectiveness as a Multi-Faceted Concept

Two Paradigms of Effectiveness (Both Are Important)

Goal Attainment

Maintenance and Condition of the Asset

Three Units of Analysis for Assessing Effectiveness

Program/Administrative Unit or Function

Institution

System/State/Society

(continued)

37

Effectiveness as a Multi-Faceted Concept (continued)

Five Perspectives on Effectiveness

Internal/Academic Perspective

Client/Consumer Perspectives

Students

Employers

“Investors” and the Public

38

Assessing Effectiveness: Initial Considerations

Assessment as a “Second Look” at GoalsThe Multiplicity of GoalsGoals as “Promises”Goals and Evidence

Achievement of Institutional PurposesTypes and Sources of EvidenceOrganized Around “Thematic Goals”

(continued)

39

Assessing Effectiveness: Initial Considerations (continued)

Assessment of Long-Term Viability

Maintaining Critical Assets

Goals and Changing Environments

Assessment of Learning Outcomes

What Is the Curriculum Really?

Which Outcomes Are “Collective Property?”

40

Key Points About Assessment

Start with What You Do

Implied and Stated Goals

Existing Sources of Evidence

Reexamine Information Assets

Data Audits

Decentralize Where Possible, but Avoid “Anything Goes”

(continued)

41

Key Points About Assessment (continued)

Use Results to Start Discussions, not “Give Answers”

Indicators (and Their Limits)

Forums for Collective Discussion

Emphasize Fixing Problems, not Creating “Winners and Losers”

Links to Resource Allocation

The Need for Consistency

42

Often Encountered Problems with Assessment

Measuring Everything that Moves

Need for Priorities (“What You Measure Is What You Value”)

“Evidence” not “Measurement”

The Perfect Data Fallacy

Recognize that All Data Are “Bad”

How Good Is Good Enough to Make a Decision?

(continued)

43

Often Encountered Problems With Assessment (continued)

Linear Thinking

Goals Don’t Always Come First

Visible Application and Action Is Needed at Every Stage of Development

Fixed Targets and “Standards”

Look at Multiple Dimensions of Performance

Emphasize Continuous Improvement

44

Linking Assessment to Decisionmaking...

Assessment Is Typically Conducted After the Fact and at the Aggregate Level

Decisionmaking Is in Real Time and Is Typically Focused on the Detailed Operational Level

…Is So Difficult Because:

45

Specific Opportunities for Linkage

Harnessing Accreditation Self-Study

Building an Infrastructure

Keeping Planning Assets Intact

Tracking the Plan

Forums and Retreats

Indicators and Updates

(continued)

46

Specific Opportunities for Linkage (continued)

The “Results” Dimension of Program ReviewAnnual IndicatorsMulti-Year Cycles

Assessment and BudgetingA “Real” Role for Assessment CommitteesFixing Problems or Rewarding Performance

Packaging Decisions Around Assessment Results

47

Assessing Learning: Key Points

Start with the Curriculum (and Its Implied Goals and Purposes)

Syllabus AuditsPrerequisite Sequences

Examine Processes as Well as OutcomesThree Curricula—Designed, Delivered, and ExperiencedPatterns of Coursetaking and Longitudinal Tracking

(continued)

48

Assessing Learning: Key Points (continued)

Principal Assessment Methods

Cognitive Measures> Constructed (Tests)> Naturally-Occurring (Work Samples, Portfolios)

Self Reports> Surveys> Behavioral Outcome Measures

Emphasize Existing Points of Contact with Students

49

Bottom Line for Any Quality Assessment Mechanism at the Institutional Level

Must Be Conditioned by a Conscious Link to Resource Allocation

Must Recognize the Needs and Contexts of Both the Academy and Society

50

What Ongoing Evaluation and Feedback Processes Does

Your Institution Have and Use?

51

Potential Clients

Students

Local Government

State Government

Agencies of Federal Government

Employers in Selected Industries (Both Public and Private Sectors)

Labor Organizations

Civic Groups

52

Questions Regarding Students

Who Are Your Students (Really)?How Many Student Bodies? Disaggregate to Identify Distinctive Clientele and Population Groups

What Do Students Do (Really)?How Many Patterns of Enrollment?What Are the Impacts of These Patterns?Do They Square with Aspirations and Expectations?

(continued)

53

Questions Regarding Students (continued)

What Is Your Student Retention Rate? Which Students Are Dropping Out? What Can You Do To Prevent That?

What Demands Do Students Make on the Institution?

54

Sources of Evidence

Admissions Records

Registration/Transcripts

Student Surveys

Assessment Databases

Individual “Service Office” Records

55

How Clients Are Served

Instructional ProgramsTransferVocationalAdult Literacy

Short-Course Training

Service—Problem-Solving Assistance

Support Services(continued)

56

How Clients Are Served (continued)

Intended Outcomes

Knowledge and Skills to Be Acquired

Preparation for What?

Method of Delivery

Time and Place of Delivery

57

What Is the Gap Between Myth and Reality in Program/Service Mix?

Trends in Student Enrollments and Degrees Granted by:

Department/FacultyLevel of Instruction

Trends in Research Productivity by Department/Faculty

Trends in Enrollments by Modes of Delivery

58

Sources of Evidence

Student Transcripts

Student Evaluations

Testing and Evaluation Data

Syllabi Analyses

Faculty Personnel Files

59

Questions Regarding Service

What Service Work Is Done at Your Institution?

Who Are the Clients?

Do You Know?

60

Asset-Related Issues

Faculty and Staff Development

Program/Curriculum Development

Deferred Maintenance

Obsolescence/Sufficiency of Equipment

Currency/Accessibility of Information Resources

Change/Repair of:ImageGood Will

61

B R E A K

62

How Does Your Institution Make Adjustments so

It Can Enhance Its Areas of Comparative Advantage?

63

Strategic Planning—Levels of Strategy

Strategic

Themes

Match

WHO WE ARE WHO WE WANT TO BE

Opportunities

Threats

1. Do we wantto change?

2. Must wechange?

3. Can wechange?

Strengths

Weaknesses

Mission

Clientele

Program Mix

Comparative Advantage

Assets

Mission

Clientele

Program Mix

Comparative Advantage

Assets

External Environment

Internal Environment

Organizational Culture

Level I – Strategic Themes

64

Key Point

To Ensure a Link Between

Strategic Planning and Strategic Budgeting,

Funds Should Be Budgeted for

Contingencies and New Initiatives

(Right Side of Strategic Planning Chart)

Linked to the Strategic Themes for Change

65

Linking Planning and Budgeting...

Planning Is Typically Conducted at the Strategic Level

Budgeting Is Typically Focused at the Operational Level

…Is So Difficult Because:

We Need to Develop an Approach to Strategic Budgeting.

66

Central Considerations in Budget Process

Creation/Maintenance of Institutional Assets

Allocation/Use of These Assets in Such a Way that Priority Objectives Are Achieved

67

Assets

The Focus of Budget Decisions

What Kinds Will We Have/Own?

In What Quantities/How Many?

With What Characteristics?QualityAppropriateness

At What Price?

What Are Assumptions About Levels of Utilization?

(continued)

68

The Focus of Budget Decisions (continued)

PersonnelFacultyAdministratorsSupport

FacilitiesClassroomsLabsOfficesLibrary

Equipment/Technology

Collections

Students/Clientele

Finances (Endowments/Reserves)

Program/Curricula – Program Mix/Learning Modules

Intangible Assets

Types of Assets

(continued)

69

The Focus of Budget Decisions (continued)

Consumables (Utilities, Insurance, Supplies, etc.)In What Quantities?At What Prices?

RevenuesIn What Amounts?From Which Sources?

AllocationTo Which Units?To What Ends? (the Basis of Accountability)

70

One Set of Budget Trade-Offs

Quantity Quality Utilization Unit Total Total Sources

AssetsPersonnelFacilitiesEquipmentCollectionsStudentsFinancesProgramIntangible

ConsumablesServicesSuppliesUtilities

Contingency

New Initiatives

=

TuitionAppropriationsStudent FeesState BudgetGovt. GrantsPrivate GiftsEndowmentSales & ServicesOther RevenuesReserves

Costs Revenues

71

Where Are Your Institution’s Points of Leverage?

72

T H A N K Y O U!