Weaving Assessment and Accountability into the

Institutional FabricWhat if Everyone Actually Bought In?

Victor M. H. Borden, Ph.D.Associate Vice President

University Planning, Institutional Research, and Accountability (IU)Associate Professor of Psychology (IUPUI)

Past President (AIR)Devoted Fan (M.O.U.S.E.)Former Spy (U.N.C.L.E)

2006 Arizona AIR Conference – March 31 – Phoenix

2006 Arizona AIR Conference – March 31 – Phoenix

OverviewWhat the H-E- is he talking

about? Is there some kind of a problem?Bought into what?

What is it we do again and why?What might we be doing and

how?

2006 Arizona AIR Conference – March 31 – Phoenix

What’s Going On?Is your workload picking up?Is there less, if any down time?Do you have time to clean up

and decompress/reflect between projects?

Is there such a thing as “between projects” or just “among projects?”

2006 Arizona AIR Conference – March 31 – Phoenix

The Challenge Increased demands for what we do

External demands Improved literacy

Increased expectations as to what can be done Technological advancements Expectations for self-service Our own effectiveness

No notable increase in resources

2006 Arizona AIR Conference – March 31 – Phoenix

What Exactly Do We Do?What is IR?Who uses IR?For what?

2006 Arizona AIR Conference – March 31 – Phoenix

The Many Faces of IR

2006 Arizona AIR Conference – March 31 – Phoenix

Arizona Faces of IR

2006 Arizona AIR Conference – March 31 – Phoenix

As We See Ourselves?

2006 Arizona AIR Conference – March 31 – Phoenix

As Others See Us?

2006 Arizona AIR Conference – March 31 – Phoenix

What is IR?

…research conducted within an institution of higher education to provide information which supports institutional planning, policy formation and decision making.Saupe, 1990 – The functions of IR

2006 Arizona AIR Conference – March 31 – Phoenix

Brought Into What?Using data, information,

evidence, analysis to…Plan – Figure out what to do Implement – Figure out how to do itEvaluate – Determine how well it is

being done Improve – Figure out how to do it

better

2006 Arizona AIR Conference – March 31 – Phoenix

Who Uses It? Senior administrators Administrative directors Administrative staff Campus committees Deans and department chairs Faculty senates/councils Faculty Students and student groups State, federal and commercial agency staff

2006 Arizona AIR Conference – March 31 – Phoenix

Who Could Use It?

2006 Arizona AIR Conference – March 31 – Phoenix

Used for What HE Functions?

Enrollment management Resource allocation Management Program improvement Process improvement Planning and budgeting Accountability Climate assessment Student learning outcomes

assessment Environmental scanning

Economic impact Faculty salary equity Space utililization Grant development Marketing Knowledge management Compliance reporting Data administration and

warehousing Student engagement Alumni engagement and

cultivation

2006 Arizona AIR Conference – March 31 – Phoenix

What Can IR Be Used For?

2006 Arizona AIR Conference – March 31 – Phoenix

What is it we Really Do?Sit at our desks, producing tables

and charts to hand off to those who requested them?

Facilitate organizational learning

2006 Arizona AIR Conference – March 31 – Phoenix

The IR Credo

I realize that I will not succeed in answering all of your questions. Indeed, I will not answer any of them completely. The answers I provide will only serve to raise a whole new set of questions that lead to more problems, some of which you weren’t aware of in the first place. When my work is complete, you will be as confused as ever, but hopefully, you will be confused on a higher level and about more important things

2006 Arizona AIR Conference – March 31 – Phoenix

A New Definition of IR

The primary function of institutional research is to facilitate organizational learning for the continuous improvement of higher education institutions and systems.

2006 Arizona AIR Conference – March 31 – Phoenix

What’s important is not what the data say, It’s what we say and

do about the data~Victor Borden

2006 Arizona AIR Conference – March 31 – Phoenix

A Learning Paradigm Planning, evaluation, and improvement

all fit within a rational model Learning incorporates uncertainty,

ambiguity, and multiple styles Individual learning and organizational

learning are compatible concepts Learning is mission critical for all

educational institutions

2006 Arizona AIR Conference – March 31 – Phoenix

Norton & Kaplan’s BSCFinancial performanceCustomer service and

satisfactionProcess effectiveness and

efficiencyOrganizational learning

2006 Arizona AIR Conference – March 31 – Phoenix

Senge’s 5 Dimensions of Organizational Learning

Personal masteryMental modelsBuilding shared visionTeam learningSystems thinking

2006 Arizona AIR Conference – March 31 – Phoenix

Single- and Double-Loop Learning

Learning is the detection and correction of error (unintended consequences)

“Governing Variables” are those things what we feel are important to keep within acceptable limits

“Action Strategy” is what we do or plan to do to keep the governing variables within limits

“Consequences” are the intended and unintended outputs and outcomes Intended: confirm our theory in use Unintended: suggests error in our theory in use

2006 Arizona AIR Conference – March 31 – Phoenix

Single-Loop LearningGoverning variables not called

into questionAdjustments made to action

strategies at bestDefense mechanisms can readily

arise to maintain single-loop learning

GoverningVariables

ActionStrategies

Conse-quences

2006 Arizona AIR Conference – March 31 – Phoenix

Double-Loop Learning Questioning the role of the framing

and learning systems which underlie actual goals and strategies

Reflection is fundamental Basic assumptions are confronted Hypotheses publicly tested Falsification is sought Ego is laid aside

GoverningVariables

ActionStrategies

Conse-quences

2006 Arizona AIR Conference – March 31 – Phoenix

Model I and II Org Learning Single- and double-loop learning at

the organizational level Model I: Organizational members

prescribe to a common theory in use Organizational policies and practices

inhibit change Model II: Governing values, policies,

and practices promote double-loop learning

2006 Arizona AIR Conference – March 31 – Phoenix

A Model I Learning Organization

Governing Variables Tow the line Win at all costs Suppress negative feelings Emphasize rationality

Action Strategies Control environment and task unilaterally Protect self and others unilaterally Discourage inquiry

Consequences Defensive relationships Low freedom of choice Reduced production of valid information Little public testing of ideas

2006 Arizona AIR Conference – March 31 – Phoenix

A Model II Learning Organization

Governing Variables Valid information is most important Free and informed choice Shared internal commitment

Action Strategies Shared control Participation in design and implementation of

action Consequences

Minimally defensive relationships High freedom of choice Public testing of ideas

2006 Arizona AIR Conference – March 31 – Phoenix

Domains of HEI Information Use

Two dimensionsCommonality of goals and objectives Intentionality of information use

Four QuadrantsModel IModel IIDisciplinary ScholarshipNothing doing

2006 Arizona AIR Conference – March 31 – Phoenix

Commonality of Goals and Objectives

Individual/PrivateEach person works toward their own

purposes

Everyone works toward the same purposesOrganizational/Shared

2006 Arizona AIR Conference – March 31 – Phoenix

Intentionality of Evidence Use

Subjective/biasedLooks for and accepts evidence that

supports a given conclusion

Objective or balanced approach to identifying and examining evidence

Objective/unbiased

2006 Arizona AIR Conference – March 31 – Phoenix

HEI Information Use Domains

Intentionality of Information Use

Com

mon

ality

of G

oals

/Obj

ectiv

es

Indi

vidu

al/p

rivat

eO

rgan

iz’l/

shar

ed

2006 Arizona AIR Conference – March 31 – Phoenix

Beyond Information ProviderIt is one thing to give people the

information they think they need to make decisions

It is quite another to engage our colleagues in using information to figure out how to improve our collective lot

2006 Arizona AIR Conference – March 31 – Phoenix

How Might we Expand Our Impact

Borrowing from colleague’s models

Structural arrangementsBrokering

2006 Arizona AIR Conference – March 31 – Phoenix

Colleague’s Models Library

Collections; reference; information literacy; credibility assessment; online tools

IT Centralized – Standards and supported

platforms Decentralized – Local Support Providers Usability analysis Stewardship

Professional Development (CTL) Workshops; consulting; faculty fellows

2006 Arizona AIR Conference – March 31 – Phoenix

Other Useful ConceptsOutsourcing

Noel-Levitz; College BoardNSSE; CSRDE; Delaware

Structural arrangementsSplit positions; hosted staff;

exchangesContracted services

2006 Arizona AIR Conference – March 31 – Phoenix

Still Other Useful ConceptsNetworking Structures

Coordinating committees IR CouncilQuality facilitators

BrokeringLinking to expert resources Internal and external

2006 Arizona AIR Conference – March 31 – Phoenix

Summary We are victims of our own success

We da man / We go, girl The more we do to respond to

demand, the more demand we create We must go from providing fish to

teaching people how to fish We must learn together with our

colleagues how to improve our fishing yield

2006 Arizona AIR Conference – March 31 – Phoenix

Implications We need to expand our ideas about

deploying IR products and services More emphasis on coordination and

collaboration We need to develop ourselves as

more informed and expansive higher education administrators

What doesn’t kill us will surely make us stronger

2006 Arizona AIR Conference – March 31 – Phoenix

Good to Great (Social Sector)

What you are good at

What you are passionate about

What you have sustainable resources to support

Jim Collins

2006 Arizona AIR Conference – March 31 – Phoenix

What is IR? What are we good at?

Figure out how to cull use evidence from the information droppings all around us

What are we passionate about? Getting people to use evidence…

What do we have resources to support By ourselves, not very much With our colleagues, an H-E- of a lot

more