using analytics for institutional transformation - dr. yvette mozie-ross - university of maryland...

Using Analytics for Institutional Transformation

Yvette Mozie-Ross, Ph.D.Blackboard Teaching and Learning Conference, Asia

October 6, 2016

On your mobile device, please browse to:

responseware.turningtechnologies.com

Enter Session ID: bblearn

50.umbc.edu | #UMBC50

Highlights

• About Us

• How We Got Here

• Analytics on the Analytics

• Data Driven Decision-Making

• Impact and Implications

• What’s Next?


About UMBC


UMBC: An Honors University in Maryland

Student Enrollment, Fall 2016Undergraduate: 11,142Graduate: 2,498Total: 13,640

Freshman Class Profile, Fall 2016SAT Average: 1217Grade Point Average: 3.7Living on Campus: 75%

15 NCAA Division I Sports (America East Conference)

Faculty:527 Full-time298 Part-time

Staff:1,288

UG Programs:62 Majors; 41 Minors; 22 Certificates

Grad Programs: 52 Masters; 29 Doctoral; 26 Certificates


President Freeman A. Hrabowski

Carnegie Classification – Doctoral Universities (Higher Research Activity)

What significant milestone is UMBC celebrating this year?


Recent AccoladesTop 5 Most Innovative Schools

US News and World Report

Top Universities for Undergraduate EducationPrinceton Review’s The Best 381 Colleges

Top 100 Public UniversitiesForbes Top Colleges

Leaders in Supporting Low Income StudentsUS Department of Education

Outstanding Academic Workplace (Honor Roll Distinction)Chronicle of Higher Education

President’s Cup Qualifier (Final Four of College Chess) - A Record 15 Times!National Collegiate Team Chess Championship


How We Got Here


Brief History of REX (Report Exchange)2006UMBC purchased a data warehouse product from a small local company, iStrategy. The product loads data from People Soft Student Administration (PSSA) into a set of Microsoft tables designed to facilitate reporting (came with many delivered reports).

2007Converted critical legacy data into a “virtual” People Soft database, thereby getting the warehouse and its reporting tool to work.

2009Went live with PS Student Administration (Recruitment, Admissions, Financial Aid, Registration and Student Billing)

2010Converted census (“frozen”) data from 1995 to present into data warehouse.


Brief History of REX (Report Exchange)

2011Blackboard acquired iStrategy and renamed the product line, Blackboard Analytics (BbA).UMBC begins beta testing the BbA modules for PeopleSoft Financial Aid and PeopleSoft Finance

2012UMBC migrates the Student Admin module to BbA version 4.0

2013REX web site is born and the data warehouse is upgraded to SQL Server 2012


5 Key Elements to Success


Key Element #1

The Right People

Champions (Campus and Division/Unit)Users (Inquisitive Nature, Data-Driven)

Technical (for Implementation)


Key Element #2

Spirit of Collaboration

Shared Data WarehouseOne Source

Transparency


Key Element #3

A Well-Defined Access and Training Strategy

Skill Levels of Report UsersLearning Styles and Modes of Delivery

Training Mechanism Sustainable Over Time


How many REX Guided Report Users?


Snapshot of Current REX Users

657 Guided Report Users (Parameter Driven MS Reporting Services Reports)President, Provost, Deans, Department Chairs, Back offices - Undergraduate Admissions, Undergraduate and Graduate Program Directors, Advising Coordinators

50 Exploratory Users (Pyramid)Enrollment Management Directors, Associate Deans, Academic ProgramSpecialists/Coordinators, Admissions Counselors

15 Report Builders (Directly Against Cube or Dimensional Model)Institutional Research Analysts, Division of Information Technology Analysts, Department IT Analysts

Key Element #4

A Comprehensive Communications Plan

Endorsed by LeadershipEnable/Enhance a Data Driven Culture

Include Outreach and FeedbackBranding


Key Element #5

An Effective Governance Structure

Fluid Not RigidOrganically Created

Team-based


Key Elements Summary

1. The Right People

2. Spirit of Collaboration

3. A Well-Defined Access and Training Strategy

4. A Comprehensive Communications Plan

5. An Effective Governance Structure


Analytics on the Analytics


Number of Distinct Users Since January 1, 2016?

A. 58B. 448C. 726

Number of Overall Report Runs Since January 1, 2016?

A. Less than 100,000B. 100,000 to 175,000C. Over 175,000

Peak Day of Week for REX Usage?

A. MondayB. WednesdayC. Friday

Peak Time of Day for Rex Usage?

A. 10:00 AMB. 12:00 PMC. 4:00 PM

Data Driven Decision-Making


UG Program Directors and Chairs Want to Know…

What are the admission and yield rates of students from XYZ High School?

How are students from XYZ High School performing in key gateway courses in the majors? Are there subject areas or courses of particular concern (high DFW rates)?

What are the graduation and retention rates of students from XYZ High School matriculating at UMBC?


Report Highlights:• Last 5 Years of Data• 5 Year Applicant and Enrollment Trends• Student Profile – GPA and Test Scores• Retention/Graduation Rates• Degrees Awarded• Segment Further by School Control and

County

Report Highlights:• DFW Counts and Rates• DFW Rate Comparison by

Institution, College, and Subject• Grade Distribution• Segment Further by School Control

and County

UG Admissions Wants to Know…

Which recruitment events appear to have the greatest impact on yield?

Do specific events appear to impact specific demographic populations differently?

What do we know about our event participants? How did they get to our doorsteps? (Prospects, Inquirers, Score Senders, Stealth Applicants, etc.)


Report Highlights:• Event Attendance Comparison• Yield by Initial Recruiting Status• Segment Further by

Race/Ethnicity, Gender, and Residency

Enrollment Management and Deans Want to Know…

Are we offering enough of the types of classes our students need to progress to degree completion in a timely fashion?

Where is there evidence of enrollment pressure and unmet demand?

Are we maximizing use of existing resources? Where is there evidence of underutilization of courses?


Report Highlights:• Current Year

Comparison to PY Current, Day 1, and Census

• Over/under utilized courses

• Waitlist Totals• Further Segment

by Course Attribute (i.e. Gen Eds)

Academic Advisors Want to Know…

How do students’ performance in course A relate to their performance in course B?

What are the outcomes (persistence, retention, graduation) based on student performance in a course?


Report Highlights:• Course Descriptive

Statistics• Comparison to

“Course Not Taken” Population

• Segment further by Course Section

Report Highlights:• Outcomes by Grade• Further Segment by

Course Section• Display by Admit Type,

Race/Ethnicity, Grades• Useful for assessing

effectiveness of Interventions

Faculty Want to Know…

How does my Bb LMS course activity (accesses, interactions, submissions) change over the course of the semester?

How does my Bb LMS course activity compare with others?


Report Highlights:• My Course Activity Average

v. All Course Activity Average

• Averages by Type of Activity

• Useful for Assessing Course Redesign Impact

Students Want to Know…

How am I doing in XYZ course relative to my peers?

How have students with comparable activity performed in XYZ course?


Report Highlights:

• Student’s Activity Level and Activity Rank

• Course Activity (Average Usage)

• Student Usage by Tool• Course Usage by Tool

Impact

• Changed the nature of the questions

• Allowed for more substantive conversations

• Improved data quality and confidence in data

• Enabled responsiveness to constituents

• Supported more evidence-based decisions


Examples of ImpactEnrollment Management/Course Demand Committee –

Made the Case for the Allocation of $250K - $300K Annually in Enrollment Pressure Funding to Departments to Support the Addition of Course Sections

Undergraduate Admissions – Confirmed Decision to Discontinue Hosting Regional Reception for Admitted Students Resulting in $40K

Saving In Recruitment Expenditures

Academic Advising and Financial Aid – Strategically Identified Students Not Yet Registered Due to Outstanding Balance; Leveraged Senior

Completion Fund ($50K) to Assist 26 Students w/ Outstanding Balances Resulting in Increased Enrollment, Increased Net Revenue and Increased Degrees Awarded

Departments/Colleges – Inspired Course Redesign in Gateway Chemistry and Mathematics Courses


Implications

• Demand – Experienced Significant Increase (”Success Breeds Success”)

• Staffing – Need More Technical Staff (Currently Supported by Staff of 5)

• Policy – Need to Establish Policy w/ Respect to Access and Use of Data

• Training – Need for Increased Training on FERPA and Acceptable Use


What’s Next?• Co-Curricular Analysis - Bringing Other Data Into the Warehouse ( i.e. Event Attendance, Co-

Curricular Involvement, Living-Learning Communities)

• Assessment - Using the Warehouse to Assess Effectiveness of Interventions (i.e. Supplemental Instruction, Tutoring)

• Predictive Analytics – Using the Warehouse to Feed and Receive Data Back from Predictive Tools (i.e. Civitas, Blackboard Predict)

• Administrative Reporting – Increasing Availability of Finance and HR Data and Reports

• Documentation/Training – Developing a More Comprehensive User Resources Library (i.e. Data Dictionary)


Questions?
