Download - Metrics and Money:
![Page 1: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/1.jpg)
1
Metrics and Money:The Process and Politics of
Accountability
Stephen Daigle, Ph.D, California State UniversityMichael Large, Ph.D, Social and Behavioral Research Institute, CSU San MarcosPatricia Cuocco, California State University
![Page 2: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/2.jpg)
2
Overview
• ITS and the Measures of Success (MOS)
• MOS Research Agenda
• Findings
• Questions (and, we hope, answers)
![Page 3: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/3.jpg)
3
Integrated Technology Strategy
• Early 1990s – Presidential Interest in Using Technology As a Strategic Tool Driven By:– Immense Growth in Technology, – Antiquated Legacy Systems, – Increasing Demand for Access– Fiscal Constraints
• Mid 1990s – Develop Integrated Technology Strategy Framework– Iterative Process, Constituent Input and Approval
![Page 4: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/4.jpg)
4
ITS – The Process
• 1996 Accepted by Board of Trustees• Living Framework – Not Written Plan• 2003 – Technology Decisions Still
Governed by ITS
![Page 5: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/5.jpg)
5
What Is ITS
• Outcome Driven– Excellence in Learning and Teaching– Quality of the Student Experience– Administrative Quality and Productivity– Personal Productivity
![Page 6: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/6.jpg)
6
ITS – the Icon
![Page 7: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/7.jpg)
7
ITS – How It Fits
• Personal Productivity - Attained Through Infrastructure Initiative – Minimum Baseline at All CSU Campuses
• The Infrastructure Is Critical If ITS Benefits Are to Extend to All Students, Faculty and Staff
• The Infrastructure Requires Money ($250M) = PROBLEM
![Page 8: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/8.jpg)
8
Getting the Infrastructure• Legislature Challenged CSU to Be
Creative• CSU Was Too Creative • Public/private Partnership Made
Legislature Uncomfortable• Californians Passed a Bond• Presidents Decided Infrastructure Had
Priority Over New Buildings
![Page 9: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/9.jpg)
9
Getting to “Yes” *
• Bond Expenditures Require Legislative Approval• Legislature Wanted to Tie Investment in
Infrastructure to “Learning Outcomes”• Not What ITS Is About• Infrastructure Enables ITS Initiatives Which, in
Turn, Enable Outcomes
Apologies to Fisher, Ury and Patton
![Page 10: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/10.jpg)
10
Negotiating Accountability
• Worked With Legislature – Agreed on What Could Be Measured
• Developed 10 Year Process for Measures of Success
• Framework • Baseline• Changes to the Baseline
![Page 11: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/11.jpg)
11
Types Of Data Collected
• System• Campus• Individual (Student, Faculty, Staff)• External
![Page 12: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/12.jpg)
12
Scope Of Data Collection
• 23 Campuses; 1,000 Miles• 400,000 Students• 35,000 Faculty And Staff• 10 Year Commitment• Institutional Comparisons Across Time• S, F, S Comparisons Across Time
![Page 13: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/13.jpg)
13
Technology User Survey Samples
• Students: Campus, Class Level, Ethnicity; 23 X 5 X 9 Cells
• Faculty: Campus, Rank, Discipline; 23 X 4 X 8 Cells
• Staff: Campus, Job Classification; 23 X 7 Cells
![Page 14: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/14.jpg)
14
CATI LOGISTICS(COMPUTER ASSISTED TELEPHONE INTERVIEWING)
• No Self-select As With Mail• About 100 Questions; 20-30 Minutes• Importance Of Skip Facility• Instant Database Creation
![Page 15: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/15.jpg)
15
CATI (Continued)
• Standardized Context Provided (E.G., High Speed Network)
• Trained Interview Staff, Monitoring, Evaluation
• Refusals Less Than 2 Percent; Still Over Sample
• $75 To $100k Per Survey, But Systemwide
![Page 16: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/16.jpg)
16
User Outcomes
Hardware Software Network Support Training
ACCESS
USE
SATISFACTION
![Page 17: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/17.jpg)
17
Examples of Metrics• Institutional
– Library Cost Avoidance– Smart Classrooms– Data Center Savings– Training and Support Spending
• Individual– Computer Ownership– Network Use– Help Desk Satisfaction– Use of Administrative Systems
![Page 18: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/18.jpg)
18
Findings and Data
![Page 19: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/19.jpg)
19
Findings
Hardware Software Network Support Training
Access
Use
Satisfaction
![Page 20: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/20.jpg)
20
Hardware
• Access– Students’ access to hardware
• Use– Percent of Faculty Requiring Computer Use
• Satisfaction– Faculty satisfaction with Teaching in Computer La
bs
![Page 21: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/21.jpg)
21
Software
• Access– Faculty access to software
• Use– Percent of Students Using Components of Their St
udent Information System• Satisfaction
– Staff Satisfaction with Software
![Page 22: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/22.jpg)
22
Network
• Access– Students’ access to wireless networking
• Use– Staff use of network from off-campus
• Satisfaction– Staff satisfaction with e-mail
![Page 23: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/23.jpg)
23
Student Access to a University-provided Computer
Access to a University-Provided Workstation
YesNo
Perc
ent
100
80
60
40
20
0
Administration
2001
2003
94
6
94
6
![Page 24: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/24.jpg)
24
Percent of Faculty Requiring Students to Use a Computer
Required Students to Use Computer
YesNo
Perc
ent
100
80
60
40
20
0
ADMINISTRATION
2000 AY
2002 AY
85
15
64
36
![Page 25: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/25.jpg)
25
Faculty Satisfaction with Giving Instruction in a Computer Lab
SURVEY ADMINISTRATION
2002 AY2000 AY
Mea
n Sa
tisfa
ctio
n10.0
9.0
8.0
7.0
6.0
5.0
4.0
3.0
2.0
1.00.0
Technical Support
Pedagogical
Effectiveness
7.87.4
6.76.4
![Page 26: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/26.jpg)
26
Faculty Access to University-provided Software Needed
Access to University-Provided Software
YesNo
Perc
ent
100
80
60
40
20
0
ADMINISTRATION
2000 AY
2002 AY
94
6
96
![Page 27: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/27.jpg)
27
Percent of Students Using Their Campus Student Information System
Administration
20032001
Perc
ent U
sing
Stu
dent
Info
Sys
tem
100
80
60
40
20
0
Information about
Registration
Information about
Grades
Information about
Financial Aid
Information about
Billing
Degree Progress
Information
37
22
40
20
41
29
88
71
86
67
![Page 28: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/28.jpg)
28
Staff Satisfaction with University-provided Software Available
Satisfaction with Software
Completely Satisfied
987654321Not at All Satisfied
Perc
ent
40
30
20
10
0
29
21
25
12
55
![Page 29: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/29.jpg)
29
Percent of Students with Laptops and Wireless Capability
Wireless NetworkingOwns a Lap-Top
Perc
ent
100
90
80
70
60
50
40
30
20
10
0
17
36
![Page 30: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/30.jpg)
30
Staff Use of University Network from Off-campus
Accessed Network from Off-Campus
YesNo
Perc
ent
70
60
50
40
30
20
10
0
Year of Study
2000
2002
60
40
5050
![Page 31: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/31.jpg)
31
Staff Satisfaction with E-mail Services
Q4B1B Satisfaction with Campus E-mail Services
8.74 2224 1.476
8.86 2123 1.344
8.80 4347 1.415
ADMIN Year of Study1 2000
2 2002
Total
Mean N Std. Deviation
![Page 32: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/32.jpg)
32
Importance of Providing Electronic Access to Course Instruction
Any Time and Place
Academic Rank
Assistant ProfessorAssociate Professor
Professor
Mea
n Im
porta
nce
10.0
9.0
8.0
7.0
6.0
5.0
4.0
3.0
2.0
1.00.0
5.15.05.0
![Page 33: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/33.jpg)
33
Comparison between Faculty and Students in Perceived Importance of Providing “Any-time, Any-place” Instruction
Q1A3 Importance of Providing Electronic Access to Course Instructionfor Students at Any Time and Place
5.07 3167 3.040
8.11 3086 2.156
GroupFaculty - 2002
Students - 2003
Mean N Std. Deviation
![Page 34: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/34.jpg)
34
Uses of the Data
• Accountability• Description of the Population of Interest• Change Over Time• Subgroup Comparisons• Cross-group Comparisons
![Page 35: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/35.jpg)
35
Conclusion
• Expensive – but You Get What You Pay For: Valid
• Reliable• High Level of Confidence• Negotiate – Don’t Be Passive – Be Proactive• Accountability Provides Cover• Infrastructure Is a Utility and a Prerequisite• Strategic Planning Is Dynamic – Change/add
![Page 36: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/36.jpg)
36
Look For Yourself
http://its.calstate.edu/documents/Data_Collection/I_Reports_MOS/Measure_of_Success.shtml
![Page 37: Metrics and Money:](https://reader035.vdocuments.mx/reader035/viewer/2022062410/56815e00550346895dcc43dd/html5/thumbnails/37.jpg)
37
Copies of the Presentation
http://its.calstate.edu