effectively sharing your program’s story · qar appendices. appendix a: candidate recruitment,...
Post on 14-May-2020
4 Views
Preview:
TRANSCRIPT
Effectively Sharing Your Program’s Story
Writing the AAQEP Quality Assurance Report
February 27, 2020AAQEP Staff
Always Improving Together
Agenda: Sharing Your Program’s Story
I. Standards & Key QuestionsII. Quality Assurance Report ComponentsIII. Presenting Your Case for Quality: The Quality
Assurance Report
2
Inquiring Minds…
What are your QARquestions?
1. Download Socrative Student2. Enter Classroom Code WTIDVIM
3
Standards & Key Questions
AAQEP Standards: Dimensions of Quality
5
Four Key QuestionsStandard 1
At the end of the program, are completers ready to fill their target professional role effectively?
Standard 2 Were completers prepared to work in diverse contexts, have they done so successfully, and are they growing as professionals?
What evidence supports these claims?6
Four Key Questions (cont.)Standard 3
Does the program have the capacity (internally & with partners) to ensure that completers are prepared and succeed professionally?
Standard 4Is the program engaged in strengthening the education system in conjunction with its stakeholders and in keeping with its institutional mission?
What evidence supports these claims?
7
Evidence Requirements and Priorities
8
Multiple measures with reasonable continuity
Quality of evidence must be investigated and shared
Priority is given to direct performance measures
Indirect and downstream measures must be considered
Differentiation of evidence by license, level, location, and mode
Assessments appropriate to the program and context
Quality Assurance Report Components
Quality Assurance Report at a Glance
10
• Presents your case for quality• Explains abbreviated data table(s) w/ evidence to
support each individual aspect• Includes links to complete data tables/appendices• Provides analysis of data (often based on multiple
measures informing the same aspect)
• Present key program practices that correspond to aspects of Standards 3 and 4
• Describe and document key program practices• Provide full data tables (where appropriate)
QAR Narrative
Appendices
QAR Outline Introduction and Overview of the Program and the Study The Case for Standard 1: Candidate/Completer Performance The Case for Standard 2: Completer Competence and Growth The Case for Standard 3: Quality Program Practices The Case for Standard 4: Program Engagement in
Improvement Conclusion: Findings and Commitments Appendices
11
QAR Outline: Narrative IntroductionIntroduction and Overview of the Program and the Study Institution/program origin, context, commitments Program logic (rationale, standards alignment, curricular
coherence) Program overview (Program Specification Table, program
strands/options, enrollment, staffing) Self-study overview (summary of method, who participated, etc.)
Set the stage for your story—your case for quality.Does the introduction help a reviewer understand your program’s context?
12
QAR Outline: The Standards The Case for Standard 1: Candidate/Completer Performance
The Case for Standard 2: Completer Competence and Growth
The Case for Standard 3: Quality Program Practices
The Case for Standard 4: Program Engagement in Improvement
√ Clear presentation of evidence* supporting each aspect.√ Account for the evidence requirements for each standard.√ Articulate your program’s criteria for success.
*Standards 2 & 4: identify planned-for measures13
QAR Outline: ConclusionIn addition to summarizing what your evidence tells you about your program, the conclusion provides:
an explanation of any planned changes to the program based on the evidence presented
identification of new areas of investigation or inquiry, data sources to collect, strategies to explore, benchmarks for tracking progress
documentation of new initiatives or innovations, program commitments
This section will frame your Annual Report.14
QAR AppendicesAppendix A: Candidate Recruitment, Selection, and MonitoringAppendix B: Completer Support and Follow-Up PracticesAppendix C: Program Capacity and Institutional CommitmentAppendix D: Internal Audit of the Quality Control SystemAppendix E: Evidence of Data Quality
Appendices support the narrative by documenting key program practices.
15
A Review of the AppendicesFor each appendix we will cover: Focus Purpose Relationship to the narrative and
standard/aspect
16
Appendix A: Candidate Recruitment, Selection, and Monitoring
17
A program’s admissions, monitoring, and support process based on empirical evidence of the provider’s own choosingFocus
• To describe the cycle of admissions and monitoring throughout the program and
• To evaluate the effectiveness of the systemPurpose
Candidate Success
Are you admitting and supporting candidates in such a way that they can succeed?
In contrast to Appendix D (internal audit), is the system operating successfully (as intended)?
18
QAR (S3) – Appendix A Relationship
19
QARBrief narrative to describe
program’s process regarding:
Recruitment/Admissions
Selection (Program/state criteria)
Monitoring/Support
Appendix AProvides the necessary
documentation/evidence of:
Recruiting efforts, admissions data, demographics
Data on program acceptances
Data on benchmarks, gate keeping, tracking, advising, completion, etc.
Appendix B: Completer Support and Follow-Up Practices
20
Services offered to program completers to support their career entry and ongoing professional growth
Focus
To show how the provider supportscompleters as they enter and grow in the professional workplace
Purpose
QAR (S4) - Appendix B RelationshipQAR
Brief narrative to describe how the program:
Supports completers
Follows up with completers
Appendix BProvides the necessary
documentation/evidence of:
Innovations (what’s new to your program?)
Data on completer support and follow-up
Appendix C: Documentation of Program Capacity and Institutional Commitment
22
The program’s capacity and institutional commitment to deliver quality preparation through curricula, faculty, facilities, fiscal support student support services, student feedback, etc.
Focus
• To inform an accreditation decision based on both quality and confidence
• To make the case that program quality can continue between accreditation reviews
Purpose
Appendix C- QAR Relationship to Standard 3QAR
Brief narrative to describe how:
PART I: Evidence of capacity for maintaining program quality
PART II: Evidence of institutional commitment
Appendix cProvides the necessary evidence of:
DIMENSIONS of CAPACITY Curricula alignment with state/national standards: tableQualified faculty: table, CVs Adequate facilities: table/description Fiscal support: refer to parity table Support services: description/policies Means for student feedback: description/ policies
Parity table: is your program comparable to other programs? (next slide)
Faculty Qualifications Table (sample)Name Current
Rank/Title
Terminal degree
(Y/N)
Institution Granting Degree
Year of degree granted
Field of degree/ area of special-ization
Related qualifi-cations(link to
CV)
Coursenumbers
of regularly assigned courses
Years at provider
FACULTY
INSTRUCTIONAL STAFF
ADJUNCTS/LECTURERS
24
Parity Table (sample)Demonstrates Institutional Commitment
25
Capacity Aspect
Preparation Program
Institutional Norm or Comparable
Program
Analysis of Difference
CurriculumFaculty (ranks, load)Facilities Fiscal supportSupport ServicesStudent feedback
A Pathway to the Internal Audit: Appendix D Appendix D invites you
to: describe the quality
control processes used to evaluate and improve program quality
audit the system to ensure that it is working as intended
26
Appendix A:specified
admissions and
monitoring processes
Appendix B: described completer
support and follow up
efforts
Appendix C:
documented dimensions of program capacity
Appendix D: Internal Audit of the Quality Control System
27
• The quality control processes used to evaluate and improve program quality
• An audit to ensure the system is working as intended
Focus
To demonstrate the program’s capacity to monitor and maintain quality and engage in continuous improvementPurpose
Internal Audit: Appendix DTwo optional approaches
Broad-based approach10-20 program completers
Focused approach when a particular problem has already been
documented by evidence
28
QAR(S3) – Appendix D Relationship
29
QARBrief narrative to describe
program’s process:
Provide an overview of the internal audit,
process, and outcomes.
Appendix DProvides the necessary
documentation/evidence of:
DESCRIBE policies and practices that ensure quality
SAMPLE completers and audit the system starting with them.
Sample purposefully.
REPORT degree to which processes/polices were followed,
explain discrepancies, improvements
Internal Audit Map (sample)
30
Admissions
What process/ criteria
are used?
Advising
Expect-ations for
advancing through the program?
Monitoring
What are the criteria along the way? (with diversity of
placements)
Final Clinical
Placement
Were criteria met?
Completion Check
Process followed?
Criteria met?
Completer Follow-up
Who will be contacted
post-completion
and did that
happen?
For a sample of completers
Sample coursesWhat process/criteria are used?
Sample of placements, partnerships, supervisorsCriteria for placement/personnel/training
Appendix E: Evidence of Data Quality
31
Data quality of the programs’ measures/instrumentsFocus
To ensure the quality of evidence on which decisions are based is credible
Purpose
QAR (S3) – Appendix E Relationship
32
QARNarrative to describe:
Overview of how the program ensured data quality through
validity, reliability, trustworthiness, and fairness
Links to copies of locally developed measures
Appendix ESupporting evidence/
documentation addresses:
Program’s formal process to establish validity, reliability, trustworthiness, and fairness
How were faculty, internal and external stakeholders engaged in evaluating instruments and ensuring reliable administration?
What were the results of these investigations?
Aspects of Data QualityEvidence regarding assessments should be appropriate to type
ReliabilityReliable and reliably collected. How
are raters calibrated?
ValidityHow have/ might content validity be
established?
TrustworthinessFor qualitative data,
how is analysis done?
FairnessAddressing bias – do measures work
well for all candidates?
Data Quality
Presenting Your Program’s Story
Making Your Case for Quality:The Quality Assurance Report
How Many QARs Should I Write?Things to consider…
All in One
Coherent evidence set*
*Evidence must be disaggregated by level,
location, mode
More Than OneDifferent evidence sets
Varied governance
Autonomy of programs, use of feedback
35
Report FormatRequired components include:
Introduction, Standards, Conclusion, AppendicesNo page or character requirements/limitations, but
do remember that:Reviewers are humanQuality is more important than quantity
No one product design/presentation**but do consider the report will need to be archived
36
Perspective and How to Gain It…Who is your audience?Remember that your reviewers and the Accreditation
Commission will likely be unfamiliar with your context.
Review the QRT Report template. Does your self-study presentation help or hinder a reviewer?
Use your cohort members as critical friends (introduction!).
37
Anticipate Reviewer Needs
title sections embed links in narrative
file names, page
numbers
provide portions of data tables
terms, acronyms defined
graphics, visual relief
38
Is navigating your QAR intuitive?
A Note About DataWhen feasible, provide sample size and basic descriptive
statistics (range, mean, standard deviation) and the program’s definition of success
De-identify your dataData must be disaggregated by: Level, license, certificate program LocationMode of delivery
39
QAR Ideas to Remember Use the narrative to tell your story and make your case.
Although standards will be evaluated holistically, the aspects need to be clearly addressed by the evidence put forth.
If data is not yet available for aspirational Standards 2 and 4, provide newly or soon-to-be implemented plans to address these particular aspects.
Consider the evidence set provided for each aspect. Be judicious in selecting the best evidence to make your case.
40
QAR Ideas to Remember (cont.)Attention to limitations in the evidence (small sample
size, not fully validated measures) is expected and providers are encouraged to address these issues in continuous improvement plans.
Explanation and analysis of the evidence and how the program plans to use this evidence for program improvement. (What, why, how!)
41
QAR Ideas to Remember (cont.)Power in the “conclusion”: changes based on
evidence, new areas of investigation because of evidence, documentation of new initiatives and innovations.
Conclusion helps to shape the Annual Report. The analysis and implications guides your action plan.
42
And Finally…
The QAR is a celebration of your program’s work and a sign of your program’s commitment to
continuous improvement!
43
Questions?
44
Follow AAQEP on Social Media
45
twitter.com/aaqep1
linkedin.com/company/aaqep
THANK YOU!
Visit www.aaqep.org for updates and opportunities
Send questions and ideas to: aaqep@aaqep.org
46
top related