combining the streams: university-wide admissions analytics
TRANSCRIPT
Combining the Streams: University-wide Admissions Analytics
MIKE SALISBURY
UNIVERSITY OF ROCHESTER
ASSISTANT DIRECTOR, UNIVERSITY ANALYTICS
1
Topics
What were the project goals?
Who were the target user roles and their requirements?
What analytics and data models did we build?
What reports and dashboards did we build?
How did we handle training and change management?
2
About Me and Our Team
Joined UR Data Warehouse team in 2017
Spent previous 9 years at Ellucian as product
owner for Banner ODS/EDW, Performance
Applications, Ellucian Analytics
Working our way through Student lifecycle
Financial Aid (PowerFaids) - 2017
Admissions - (Slate and others) – 2019
Student – (1st R1 Go-live Workday Student) – 2020
Student Financials - 2021
Student Retention and Time to Degree – 2021
Data Warehouse subject areas also include
Finance/Procurement (Workday), Grant Administration (Coeus), HR (PeopleSoft), Student Career Services (Handshake), Advancement (Ellucian Advance) Corporate Relations, more
500+ active Cognos and Tableau users and 300+ authors
3
Admissions DW Development Team
Admissions User Leads from each school
John Podvin and Thisie Schisler-do from Office of
Institutional Research
Deena Rocco – DW development lead
Rashmee Shrestha – Cognos reports, Slate
Extracts and system testing
Hillary Lincourt – Tableau dashboards and system
testing
Jose Delacruz – DW development
Charlie Rosenberg – DW development
Mike Salisbury – Project manager, requirements,
data model, testing, training
Project Goals – Admissions Reporting
Aggregate admissions data from across the University in to the data
warehouse for ease of reporting and analysis.
Provide a single, trusted version of the truth for admissions data
across the University.
Lighten the load on individual schools to produce their own reports
and visualizations.
Improve the consistency and quality of admissions data.
4
Project Goals – Institutional Research Reporting
Make it easier to report admissions repeatable and reliable
numbers. (Board, senior leadership, deans, external entities
(IPEDS, COFHE, AAUDE, CGS, US News).
Improve student life cycle analysis beyond what each
admissions office could do on its own.
Example: Combine with fin aid PowerFAIDS data, enrolled student
retention and graduation rates)
Improve trend analysis and year over year point in time
reporting.
5
Enrollment Funnel Metrics (Noel Levitz)
6
Admissions Funnel
7
• Enable headcount and yield metrics
• Individual schools and divisions may have additional funnel states being tracked• Each school has their funnel states mapped to these states
• Track current funnel status as well as timing of funnel state transitions• Enable YOY and YTD comparisons and trend analysis
• Can be used to develop forecasting models
• Tracking of prospects/inquiries in DW has been deferred by most schools
Prospects/Suspects
InquiriesApplicants
Admits AcceptsEnrollments
Admissions Lifecycle
Examples of Admissions Analysis Requirements
Role Examples of Analysis Questions
Institutional
Researchers
• What is the relationship between students’ academic qualifications and subsequent first year college performance and retention to second year?
• What are the trends YOY in institutional aid distribution by demographic and geographic factors?
School Deans
and staff
• How do admissions rates and enrollment yields vary by quality attributes, demographic factors, and financial need/aid levels?
• What are the inquiry, applicant and enrollment trends for various
schools/programs? Year to Date compared to previous cycles?
School Admissions
Analysts
• What is the status of our funnel this year to date compared to the last two years this time? Over the last four weeks compared to previous years?
• What recruitment strategies are resulting in more inquiries, applicants, and movement of undergrad/grad prospective students through the funnel?
• What are the conversion and yield rates by source?
Financial Aid • How do the financial aid packages compare between applicants who enrolled and did not enroll?
• How do the financial aid package levels compare with applicant quality metrics?
Office of
Global Engagement
• What are the international prospect, applicant and enrollment trends for various schools/programs? Year to Date compared to previous cycles?
• What is the composition of the international prospective student funnel by region, country, city, school, recruitment source? 8
Analytics Enabled by Admissions Integration
9
Measures Multiple DimensionsRecruiting and Admissions
• Current/Total Enrollment Funnel Counts
• Application/Applicant counts
• Application Decisions
• Enrollment Yield
• Admissions Rate
Financial Aid for New Enrollments
• % Receiving Aid
• % of Need Met
• % Receiving Institutional Aid
• Gross and Unmet Need (FM/IM)
• Cost of Attendance
• Projected Tuition Discount
Student Success
• Enrollment Counts
• Retention rates
• Graduation rates
• Academic performance
• Admissions/Academic Calendars, YOY, YTD
• Academic Level (Undergraduate, Graduate)
• Student Population (first-year, transfer)
• Admit/Enroll Degree program
• Application Decision/Admissions Funnel Status
• Test Scores/Test score ranges
• Secondary School/Post Performance GPA ranges
• Diversity/Demographic
• Geographic Region/Country/State/City/Zip
• Resident/Non Resident
• School last attended
• Financial Aid Recipient/Financial Aid Type/Source
• Recruitment Source
• Recruitment Event attendance
Analyze by
Tableau and Cognos tools, templates, reports are available to report authors and analysts
Consumers have personalized access to Cognos reports and Tableau dashboards through
UR Data Analytics Portal
Slate operational reports used primarily by Admissions
10
Data Warehouse Dashboards and Reports
Capability Tools Senior Leadership
Managers Staff Authors and Analysts
IT
Report & Dashboard
Consumers
UR Data Analytics Portal x x x
Cognos Bursted Reports x x x
Slate Operational Reports/Dashboards
x x x
Ad-hocReports and Analysis
Cognos Report Studio x x x
Tableau Web Author x x
Data Exports and Datasets x x
Dashboard & Report Authoring
Tableau Desktop x x
Cognos Report Studio x x
Slate Query/Report Tool x x
Tableau Dashboards Cognos Reports
Admissions Funnel TrendsAdmissions Funnel Trends
(Printable)Admissions YOY Trend by
School and DegreeAccepts and Current Funnel
Status by Entry Term
Admissions Program Trends Admissions Pool Composition Potential Student Profile Potential Student List
Admissions Degree Trends Admissions GPA TrendsAdmissions Deferrals YOY
Trend by SchoolCognos report templates
International Admissions Dashboard
Admissions SAT Trends
Country and City Admissions Dashboard
Admissions Funnel YOY Trend
Admissions Trend By Prior Education
Admissions Dashboards and Reports
11
UR Data Analytics Portal Home Page
12
Access via browser on PC,
Tablet
Browse galleries of content by
domain/ subdomain, role
Quick navigation to personal favorites,
categories of content
Find content searching
metadata like name, description,
author, data elements
Personalized access
Personalized content lists
Data Governance Certification
Levels
Admissions Trends Dashboard
13
Admissions Pool Demographic Composition
14
Admissions Trends by Program
15
Admissions Funnel Year over Year Trend Comparison
16
Admissions Trends by Prior Education Geographically
17
International Admissions By Country and City
18
Fact grain: application/decision/admissions cycle
Admissions Cycles from each school
Funnel Status Group rolls up individual school funnel states to standard categories
Used bridge table to handle schools with multiple applied programs on application
Individual school application attributes available in application dimension
Highest test scores pivoted to slotted test score summary table
Current and Initial Funnel Status Indicators on Fact
Enrolled Funnel Status inserted based on student system enrollment data
19
Admissions Funnel History Data Model
Data Extracts
❖ Slate extracts built using Slate Report/Query Tool
❖ 5 CSV Extracts run nightly (Potential Students, Applications, Decisions, Prior Education, Test Scores)
❖ Currently 5 different Slate instances, 1shared by 3 Grad Schools
❖ Med School and Pre-2021 Business School data extracted from local databases
Integration Process
❖ Use Dell Boomi to launch ETL processes
❖ ETL processing by Oracle Data Integrator
❖ Nightly reload process
❖ Separate initial data staging where extract file has different format
❖ 1 fact table, 12 dimension tables
❖ Reference tables maintained for Funnel Status, Applied Program, Admissions Cycle
Admissions Data Integration
Architecture
20
Tableau
Server
ESM Slate
SFTP
Bo
om
i
Ato
msp
he
re
UR DMZ/File Server
Oracle Data Warehouse
Ora
cle
Da
ta
Inte
gra
tor U
R D
ata
Ce
nte
rGrad Slate
UG ASE Slate
Nursing Slate
SimonMed School
Cognos
Server
UR Data
Analytics Portal
UR Admissions DW Project Timeline
▪ Integrated admissions data from multiple, differently-configured instances
of Slate, Hobsons/Campus Management, and Med school systems
▪ Used Agile Scrum with 2-week sprints
▪ Development structured into 3 rounds
Round 1 (September – February)
- Enrollment Funnel History requirements, data model, dashboards, reports
- Graduate AS&E, Grad School of Medicine, Warner Education, Eastman Slate integration
Round 2 (March – May)
- Undergraduate AS&E integration (Slate and historical)
- Integrated student data for sourcing enrollments
- Test Scores and Prior Education attributes
Round 3 (April – July)
- Simon Business Hobsons, Medical School system and Nursing Slate integration
- Additional dashboards for international and domestic admissions, test score summary
- Prepared nightly data integration schedule
User Acceptance Testing (July - September)
- Worked with individual school admissions offices on data validation and cleanup.
- Obtained acceptance from each school.
- Deployed Tableau dashboards and Cognos reports in the UR Data Analytics Portal for access by Senior Leadership and their support teams
User Acceptance Test Plan and Results
User Acceptance Test Summary
Each school needed to provide source for
comparison
DW team provided analysis of discrepancies
and root causes to user leads
Multiple rounds of testing required
Results:
22/24 metrics for 2015-2019 Application, Admits,
Accepts, Enroll Counts match or within 1%
Discrepancies are known and reflect historical
data updates and data cleanup
Admissions DW is more accurate then current
GRAD Slate due to having more current data
UAT Team
Admissions User Leads from each school
Rashmee Shrestha
Hillary Lincourt
Mike Salisbury
Admissions % Var
Enrolled
% Var
Admit
% VarApplic.
Variance Reasons
ASE Graduate 1.4% 0.1% 0.1% Updates in Slate
for historical
data (defers)
ASE Undergrad 0.7% 0.01% 0.01% N/A
Eastman 0.8% 0.8% 0.5% N/A
Grad SMD 1% 3.3% 0.8% Some Slate
decision data
cleanup
Warner 4.6% 0.8% 0.28% Updates in Slate
for historical
data (defers)
Simon 0.05% 0.5% 0.0% N/A
Nursing 0.0% 0.0% 0.0% Only 2019 data
Med School 0.0% 0.0% 0.0% N/A
Student Reporting Community Engagement and Training
23
Sprint Reviews (Bi-weekly)
❖ Requirements and Design feedback
❖ Demonstrations of work in progress
Student Reporting Committee Meetings (Monthly)
❖ Reviewed progress and plans for user acceptance testing, training
❖ Discussion of reporting data definitions and governance
UR Admissions DW Report Author Data Model Training
❖ Instructor-led Zoom session with handouts
Report Authoring Office Hours (weekly)
❖ 2 90-minute Sessions per week via Zoom
❖ Supported by 2-4 DW team members
❖ Admissions and Student Report Authors bring questions or challenges with their Tableau/Cognos report building, Cognos 11 questions, etc…
❖ Current invite list includes 50+ report authors across each school, provost office/IR, ISO, IT
