Center on Postsecondary Education and Disability, ; AHEAD 2013
1
Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature
Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus
University of Connecticut
Presentation at the Association on Higher Education and Disability Annual Conference
Baltimore, MDJuly 11, 2013
Center on Postsecondary Education and Disability, ; AHEAD 2013
2
About Us
• Allison– Assistant Professor, Department of Educational Psychology,
Neag School of Education, UConn– Research Associate, Center on Postsecondary Education and
Disability (CPED)• Adam– Doctoral Student, Department of Educational Psychology,
Neag School of Education, UConn• Joe– Associate Professor , Department of Educational Psychology,
Neag School of Education, UConn– Director of CPED, Neag School of Education, UConn– Member, NPSO Advisory Board
Center on Postsecondary Education and Disability, ; AHEAD 2013
3
Special Thanks
• Lyman L. Dukes III• Michael Faggella-Luby• Nicholas Gelbar• Jennifer S. Kowitt• Melissa Root
Center on Postsecondary Education and Disability, ; AHEAD 2013
4
Session Objectives
• To explain the rationale for conducting this comprehensive literature review
• To explain the background and methods used• To present specific key findings to date• To present suggestions for future research• To facilitate discussion regarding future
research directions
Center on Postsecondary Education and Disability, ; AHEAD 2013
5
Project Background
• The Vocational Rehabilitation Act of 1973 required all institutions of higher education to ensure access to qualified students with disabilities (SWD).
• Forty years after the passage of the Act, 11% of college freshmen report having a disability (U.S. G.A.O., 2009).
• The profession of disability services is now longer a nascent field in higher education
Center on Postsecondary Education and Disability, ; AHEAD 2013
6
Project Background
• To date, a comprehensive analysis of the literature dealing with disability and higher education has not been conducted
• This literature is broad in scope and dispersed across a variety of disciplines (e.g., special education, higher education, psychology, sociology)
• The 40-year anniversary of the passage of Section 504 provides an anniversary to review the field’s literature:– What topics have been studied?– What populations have been studied?– What methodologies have been employed?– How much of the literature is research? How much is not research? – What aspects of the field have substantial evidence and support?– What aspects are lacking substantial evidence and support?– What research areas within the field are likely to receive greater attention in coming
years?• Relevance to practitioners, researchers, policy makers
Center on Postsecondary Education and Disability, ; AHEAD 2013
7
Project Background• Initially looked to a process used in secondary transition by the National
Secondary Transition Technical Assistance Center (NSTTAC: Test, Fowler, Kohler & Kortering, 2010)
• To review evidence-based practices in secondary transition based on quality experimental studies
• Method:– Procedures: (a) electronic search, (b) reviewing reference lists, (c) hand
searches of journals, and (d) updating by replicating the initial procedures
– Criteria: (a) publ. after 1984, (b) SWD in subjects 11-22 yrs, (c) IV or DV aligned to five areas of Kohler’s Taxonomy
– Focus: Included systematic reviews and group or single subject design studies
– Lens: Applied NSTTAC decision rules for determining levels of evidence (Strong, Moderate, Potential)
– Total: 240 reviews and intervention studies
8
•Are based on rigorous research designs
•Have demonstrated a record of success for improving student outcomes
•Have undergone systematic review process using quality indicators to evaluate level of evidence
Evidence-Based Practices
•Are based on rigorous research designs
•Have demonstrated a record of success for improving student outcomes
Research-Based Practices
•Are based on research
•Have demonstrated limited success
•Have used a ‘weak’ research design
Promising Practices
•Are not based on research
•Have no data to support effectiveness
•Based on anecdotal evidence and/or professional judgment
Unestablished Practices
(Helsel, Hitchcock, Miller, Malinow, & Murray, 2006; Twyman, 2008)Broad Definitions
From Cameto, Mazzotti, & Test (2011)
Center on Postsecondary Education and Disability, ; AHEAD 2013
9
Findings• Identified 33 evidence-based practices in secondary Transition• Categorized using Kohler’s Taxonomy for Transition Programming
– 3 in Student-Focused Planning (e.g., involving students in IEP) – 26 in Student Development (e.g., life skills, purchasing skills) – 1 in Family Involvement (training parents about transition)– 3 in Program Structure (extending services beyond secondary school)– No practices identified in the area of Interagency Collaboration
• Only 2 evidence-based practices have a strong level of evidence:– teaching life skills, teaching purchasing skills
• 28 practices had a moderate level of evidence• For more information, see: Test, D. W., Fowler, C. H., Richter, S. M., White, J.,
Mazzotti, V., Walker, A. R., & Kortering, L. (2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32(2), 115-128.
Center on Postsecondary Education and Disability, ; AHEAD 2013
10
Project Background
• Genesis was a request from NSTTAC to present information about evidence-based practices regarding:– Successful transition to postsecondary education– Success in postsecondary education
• Our plan:– Initially, to follow the NSTTAC meta-analysis procedures– But, postsecondary education lacks a taxonomy for the literature– Postsecondary education does not use the evidence based
practice standards required in secondary education– No prior sorting of the literature, either by topical or research
categories– Required a regrouping and new direction
Center on Postsecondary Education and Disability, ; AHEAD 2013
11
Our Method
• Began with review of 80+ JPED articles from 2000-2010– Identifying common themes and topics– Development of broad content “domains”• Identification of respective sub-domains
• Domains sent to past two JPED editors for feedback• An electronic rating form was developed and revised• JPED articles from 10 issues reviewed by four coders• Reliability determined, team debriefing, further
refinement of domains, rating form
Center on Postsecondary Education and Disability, ; AHEAD 2013
12
Initial Domain DescriptionsDomain Name Domain Description
Student Support and Services
Transition into college, student retention, access to accommodations, access to assistive technology, determining eligibility, assessment, evaluation, and diagnosis, coaching
Student Learning and Experiences
Teaching students study skills, learning strategies, teaching self-determination, teaching self-advocacy, teaching students about legal rights and responsibilities, knowledge attitudes and beliefs (KAB), experiences as person with disability
Postsecondary program structure
Program development, program evaluation, policies and procedures, fit within the institution, collaboration with other campus services, legal compliance, determining eligibility
Postsecondary Outcomes
Transition to career or graduate school, employment outcomes
Faculty/Staff Support
Faculty development and training, faculty KAB; Staff development and training, staff KAB
No Fit Studies that do not relate to any of the above domains
Center on Postsecondary Education and Disability, ; AHEAD 2013
13
Our Method
• Discussion revealed overlaps, determination of key terms– e.g., “policies and procedures”, “experiences”– Where does eligibility “belong”?– Difference between institutional and program legal compliance?– What about studies of instruments and proposed constructs?
• Domains collapsed and updated:– Student level – Program level– Faculty/staff level– Construct level
• JPED articles from an additional 5 issues reviewed by four coders• Reliability determined at 75%-85%• Debriefing lead to 100% agreement; refinement of terms, inclusion
and exclusion criteria
Center on Postsecondary Education and Disability, ; AHEAD 2013
14
Our Method
• Concurrently, 500 articles from other sources collected• Sorted into domains; reliability measured
– 88% - 96% for sorting
• Articles provided a broader perspective and lead to further refinement of the subdomains
• Validity check by 8 former editors or co-editors of JPED– Measured the clarity of domain definitions
• all were strongly agree or agree that the definition is clear
– Requested suggestions for missing domains– Fit of the subdomains– Suggestions for missing subdomains and clarification of subdomains
• (e.g., legal compliance at the program or institutional level)
Center on Postsecondary Education and Disability, ; AHEAD 2013
15
Inclusion and Exclusion Criteria• Inclusion criteria:
1. The article is about Postsecondary Education for Students with Disabilities (broadly considered to include faculty, disability services, etc.)2. The article is about one of the following topics/populations:a. Programs for accepted students into degree granting programs at a 2 or 4 year college or universityb. Programs, services, or experiences of matriculated studentsc. Articles about the experiences of students with disabilities who have dropped out of degree granting programs at a 2 or 4 year college or universityd. Articles about the experiences of students with disabilities who are graduates of degree granting programs at a 2 or 4 year college or university
• Exclusion criteria1. Articles that are primarily about secondary students in transition or transition aged programs.
Center on Postsecondary Education and Disability, ; AHEAD 2013
16
Domain DescriptionsDomain Name Domain Description
Student-Level Studies
Experiences and perceptions of students with disabilities in and after higher education.
Program or Institution-Level
Studies
Service provision by the disability services office in a higher education institution. Can also relate to institutional policies and procedures pertaining to students with disabilities.
Faculty/Non-Disability Support Staff-Level Studies
Knowledge, attitudes, beliefs of faculty and non-disability services personnel to enhance access to higher education for students with disabilities. Also education or support for faculty and staff in this practice.
Construct Development-Level
Studies
Development, evaluation, or validation of a variable, including development/validation of assessment instruments, evaluation metrics, theoretical models of service delivery, standards of practice, or ethics. The variable must be under proposal, in development, or being used in practice to gather empirical evidence.
No Fit Studies that do not relate to any of the above domains.
Sub DomainsStudent Level Studies
• Access (physical, cognitive, attitudinal)• Assistive technology use• Career development• Experiences, perceptions, knowledge, attitudes, or beliefs of students with
disabilities• Learning/using study skills, learning strategies • Mainstream technology use• Meeting institutional requirements (e.g., degree requirements, foreign language
requirements, math requirements)• Post-undergraduate program experiences and/or outcomes (e.g., graduate school,
employment)• Profiles of students (e.g., diagnostic profiles, profiles of successful and/or
unsuccessful students)• Requesting or using accommodations (e.g., assistive technologies, separate testing
location, course substitutions)• Self-determination skills (e.g., self-advocacy, student goal attainment, self-
disclosure, self-management, legal rights and responsibilities)• Statistics on students with disabilities (e.g., rate of access to postsecondary
education, student retention, graduation rate, statistics on accommodation use)
Sub DomainsProgram/Institutional Level Studies
• Collaboration with faculty or academic departments• Collaboration with other campus services • Experiences, perceptions, knowledge, attitudes, or beliefs of disability service
providers• General or specific descriptions of disability programs and resources and/or
recommended program components• Institutional Policies/Procedures• Legal compliance (institutional specific)• Legal compliance (program specific)• Program development• Programs for incoming students (e.g., freshmen, transfer students)• Programs for students transitioning to graduate school or employment• Programs for specific cohorts of students (e.g., LD, Aspergers, etc)• Policies and procedures (e.g., determining student eligibility for services,
determining reasonable accommodations, determining access to assistive technology)
• Professional development/training for disability services staff • Program evaluation (e.g., student retention, student use of program related
services, student graduation rates)• Program fit within the institution (e.g., student affairs v. academic affairs)
Sub DomainsFaculty/Non-Disability Support Level Studies
• Campus staff development and training• Campus staff knowledge, attitudes and beliefs (e.g., about
students with disabilities)• Campus staff practices• Faculty development and training• Faculty knowledge, attitudes and beliefs (e.g., about students
with disabilities; about providing accommodations)• Faculty teaching practices
Sub DomainsConstruct Development Studies
• Assessment instruments (development, validation, use to develop diagnostic profiles)
• Conceptual models or discussion of issues in disability services (e.g., eligibility for services)
• Conceptual models of service delivery (e.g., Universal Design, other models)
• Conceptual models of instruction/assessment of learning• Evaluation metrics or methods• Instructional practices• Standards of practice, performance or ethics.• Other (including disability studies)
Method
Search Terms Included:• Academic Accommodation• Accommodation• ADD• ADHD• Blind• College• College Admission• College Student• Deaf• Disabilities• Disability• Disabled• Dyslexia• Dyslexic
• Handicap• Handicapped• Hearing Impairment• Postsecondary Education• Mental Illness• Mobility Impairment• Postsecondary Education• Student Affairs• Student Personnel• Student Services• University• University Student• Visual Impairment
Continued literature review and collection of articles
Center on Postsecondary Education and Disability, ; AHEAD 2013
22
Method
• 1,210 articles identified by searches of multiple data bases (e.g., Academic Search Premier, EBSCO, PSYCinfo)
• Published between 1955 and 2012• Articles grouped into domains, reliability measured– Coding resulted in some articles shifting domains
• Will focus on student-level, program or institution-level, faculty/non-disability-level studies today
• Future steps will focus on coding construct development studies
Center on Postsecondary Education and Disability, ; AHEAD 2013
23
Method
• An electronic database was developed that included the reference citation for each article and unique alpha-numeric codes (e.g., 201225Shaw4).
• The codes were designed to be entered into the electronic coding instrument, so that coding could be linked back to the reference citation.
• Each article randomly assigned to two coders• Reliability determined for each domain• Discrepancies discussed
Center on Postsecondary Education and Disability, ; AHEAD 2013
24
Instrument
• An electronic coding instrument was designed and refined with two pilots, multiple coders.
• The instrument allowed for the researchers to code:– Did the article meet inclusion criteria?– Did the article present original data?– If not research based, what type? (e.g., lit review, legal
analysis)– If research based, what type? (with multiple layers)– What was the setting? (US, Canada, international, 2- or 4-year)– Sample information? (numbers, gender, disability, race, etc..)– Domain and sub-domain
• Across coding sheet, 148 choices were possible• To achieve agreement, coders selections must be exact
Center on Postsecondary Education and Disability, ; AHEAD 2013
25
Inter-Rater Reliability
• For today’s presentation, three subsets of articles (Domains 1, 2, and 3) were analyzed.
• Each article coded twice to check for inter-rater reliability.
• Discrepancies discussed and reconciled
Frequency and Reliability by DomainDomain n Overall
ReliabilityDomain 1: Student-Level Studies
376 82%
Domain 2: Program or Institution-Level Studies
235 86%
Domain 3: Faculty/Non-Disability Support Staff-Level Studies
110 86%
Center on Postsecondary Education and Disability, ; AHEAD 2013
26
Articles by Domain (Initial Sort)
Domain Name N
Student-Level Studies 500
Program or Institution-Level Studies
290
Faculty/Non-Disability Support Staff-Level Studies
125
Construct Development-Level Studies
134
No Fit 161
Center on Postsecondary Education and Disability, ; AHEAD 2013
27
Frequency of Articles by Domain Over Time
1951-1955
1956-1960
1961-1965
1966-1970
1971-1975
1976-1980
1981-1985
1986-1990
1991-1995
1996-2000
2001-2005
2006-2010
2011-20120
20
40
60
80
100
120
140
Student LevelProgram/InstitutionFaculty/Staff
Center on Postsecondary Education and Disability, ; AHEAD 2013
28
Journals with the Highest Frequency of Articles About Higher Education and Disability
JPED
Journ
al of L
D
Col Stu
d Journ
al
J Col S
tud Dev
LD R &
P
J Voc R
ehab Couns
Exceptional
Children
Disabilit
y & So
ciety
J Col S
tud Psyc
hotherap
y
LD Q
uarterly
0
50
100
150
200
250
# Articles
Unique Journals: 305
Center on Postsecondary Education and Disability, ; AHEAD 2013
29
Journals with the Highest Frequency of Student-Level Articles
JPED
Journ
al of L
D
J Col S
tud Dev
Disabilit
y & So
ciety
College
Student J
ournal
LD R &
PCDTEI
Exceptional
Children
Dyslexia
J Col S
tud Psyc
hotherap
y0
102030405060708090
# Articles
Unique Journals: 172
Center on Postsecondary Education and Disability, ; AHEAD 2013
30
Journals with the Highest Frequency of Program/Institutional-Level Articles
JPEDJ o
f LD
J of V
oc Rehab
New Direct
Stud Se
rv
Col Stu
d Journ
al
Exceptional
Children
New Direct
Higher E
d
TEACHING Exce
pt Child
Disabilit
y & So
c
J Deve
lop Ed
LD R &
P0
10
20
30
40
50
60
70
# Articles
Unique Journals: 109
Center on Postsecondary Education and Disability, ; AHEAD 2013
31
Journals with the Highest Frequency of Faculty/Non-Disability Staff-Level Articles
JPED
Col Stu
d Jounral
Col Teac
hing
Journ
al of L
D
J Voc S
pecial N
eeds Ed
J Col C
ounselin
g
J Col S
tud Psyc
hotherap
y
NACADA Journ
al
J Stu
d Aff R & P
New Direct
Stud Se
rv
Teaching i
n Higher E
d0
5
10
15
20
25
30
# Articles
Unique Journals: 71
Center on Postsecondary Education and Disability, ; AHEAD 2013
32
Journals with the Highest Frequency of Construct Development-Level Articles
JPED
Journ
al of L
D
Equity &
Excelle
nce in
Ed
Col Stu
d Journ
al
J Voc R
ehab
New Direct
Higher E
d05
101520253035
# Articles
Unique Journals: 70
Center on Postsecondary Education and Disability, ; AHEAD 2013
33
Proportion of Data-Based vs. Non-Data-Based Articles for Student-Level Studies
85%
15%
Data-Based (n = 318)Non-Data-Based (n = 58)
Center on Postsecondary Education and Disability, ; AHEAD 2013
34
Frequency of Data-Based vs. Non-Data-Based Student-Level Studies Over Time
1976-1980
1981-1985
1986-1990
1991-1995
1996-2000
2001-2005
2006-2010
2011-2012
0
20
40
60
80
100
120
Data-BasedNon-Data-Based
Center on Postsecondary Education and Disability, ; AHEAD 2013
35
Proportion of Student-Level Studies by Research Methodology
54%32%
5%9%
Quantitative (n = 171)Qualitative (n = 103)Mixed Methods (n = 16)Coder Disagreement (n = 28)
Center on Postsecondary Education and Disability, ; AHEAD 2013
36
Proportion of Student-Level Studies With and Without Control/Comparison Groups
17%
35%
48%With Control/Comparison Group (n = 8)Without Control/Compar-ison Group (n = 16)Coder Disagreement (n = 22)
Center on Postsecondary Education and Disability, ; AHEAD 2013
37
Proportion of Student-Level Studies Containing a Control/Comparison Group by Experimental vs.
Quasi-Experimental Methodology
17%
50%
33%Experimental (n = 2)Quasi-Experimental (n = 6)Coder Disagreement (n = 4)
Center on Postsecondary Education and Disability, ; AHEAD 2013
38
Proportion of Student-Level Studies by Location
53%
8%
31%
8%
4-year (n = 190)2-year (n = 29)International (n = 112)Coder Disagreement (n = 28)
Note: 17 studies include multiple locations
Center on Postsecondary Education and Disability, ; AHEAD 2013
39
Location of Student-Level Studies Over Time
1976-1980
1981-1985
1986-1990
1991-1995
1996-2000
2001-2005
2006-2010
2011-2012
0
10
20
30
40
50
60
4-year2-yearInternational
Center on Postsecondary Education and Disability, ; AHEAD 2013
40
Proportion of Student-Level Studies Including Clear Sample Size Data
71%
19%
10%
Studies Including Student Sample Size (n = 267)Studies Not Including Student Sample Size (n = 73)Coder Disagreement (n = 36)
Center on Postsecondary Education and Disability, ; AHEAD 2013
41
Proportion of Student-Level Studies Including Data on the Race/Ethnicity of Participants
17%
75%
8%
Studies Clearly Provide Data on Race of Partic-ipants (n = 53)Studies Do Not Clearly Provide Data on Race of Participants (n = 240)Coder Disagreement (n = 25)
Mean = 6,136SD = 91,021Min = 1Max = 1,502,658
Center on Postsecondary Education and Disability, ; AHEAD 2013
42
Proportion of Student-Level Studies Including Disability-Related Demographic Information
68%
21%
11%
Study Provided Frequen-cies of Disability Type (n = 217)Study Did Not Provide Frequencies of Disability Type (n = 66)Coder Disagreement (n = 35)
Center on Postsecondary Education and Disability, ; AHEAD 2013
43
Proportion of Student-Level Articles Including Data About the Gender of Participants
51%
35%
14%
Reported Gender (n = 161)Did Not Report Gender (n = 111)Coder Disagreement (n = 46)
Center on Postsecondary Education and Disability, ; AHEAD 2013
44
Proportion of Student-Level Articles Including Data on the Class Standing of Participants
14%
76%
11%
Reported Class Standing Data (n = 43)Did Not Report Class Standing Data (n = 241)Coder Disagreement (n = 34)
Center on Postsecondary Education and Disability, ; AHEAD 2013
45
Proportion of Student-Level Articles Including Data About the Non-Student Participants
7%
89%
3%
Included Non-Student Par-ticipants (n = 23)Did Not Include Non-Stu-dent Participants (n = 284)Coder Disagreement (n = 11)
Center on Postsecondary Education and Disability, ; AHEAD 2013
46
Thirteen Subdomains of Student-Level Studies and Their Frequencies
(Articles could be coded as multiple subdomains)
• Experience, perception, knowledge, attitude of SWD (n = 161)
• Profiles of SWD (n = 54)
• Learning/using learning strategies (n = 30)
20 ≤
• Statistics on SWD (n = 19)
• Requesting or using accommodations (n = 18)
• Self-determination (n = 17)
• Access (physical/cognitive/attitudinal) (n = 14)
• Assistive technology use (n = 11)
10 – 19
• Career development (n = 9)
• Mainstream tech use (n = 6)
• Other (n = 6)
• Meeting institutional requirements (n = 5)
• Post-undergraduate outcomes (n = 1)
≥ 9
Center on Postsecondary Education and Disability, ; AHEAD 2013
47
Proportion of Data-Based vs. Non-Data-Based Program/Institution-Level Studies
39%
61%
Data-Based (n = 92)Non-Data-Based (n = 143)
Center on Postsecondary Education and Disability, ; AHEAD 2013
48
Frequency of Data-Based vs. Non-Data-Based Program/Institution-Level Studies Over Time
1951-1955
1956-1960
1961-1965
1966-1970
1971-1975
1976-1980
1981-1985
1986-1990
1991-1995
1996-2000
2001-2005
2006-2010
2011-20120
5
10
15
20
25
30
35
40
Data-BasedNon-Data-Based
Center on Postsecondary Education and Disability, ; AHEAD 2013
49
Proportion of Program/Institution-Level Studies by Research Methodology
60%27%
1%12%
Quantitative (n = 55)Qualitative (n = 25)Mixed Methods (n = 1)Coder Disagreement (n = 11)
Center on Postsecondary Education and Disability, ; AHEAD 2013
50
Proportion of Program/Institution-Level Studies With and Without Control/Comparison Groups
14%
86%
With Control/Comparison Group (n = 0)
Without Control/Compar-ison Group (n = 1)
Coder Disagreement (n = 6)
NOTE: All 6 disagreements exist be-cause one coder indicated that the study was an article without a control/comparison group while the other coder left the data point blank.
Center on Postsecondary Education and Disability, ; AHEAD 2013
51
Proportion of Program/Institution-Level Studies by Location
47%
24%
13%
16%
4-Year (n = 64)2-Year (n = 33)International (n = 18)Coder Disagreement (n = 22)
Note: 42 studies include multiple locations
Center on Postsecondary Education and Disability, ; AHEAD 2013
52
Location of Program/Institution-Level Studies Over Time
1951-1955
1956-1960
1961-1965
1966-1970
1971-1975
1976-1980
1981-1985
1986-1990
1991-1995
1996-2000
2001-2005
2006-2010
2011-20120
2
4
6
8
10
12
14
16
18
4-Year2-YearInternational
Center on Postsecondary Education and Disability, ; AHEAD 2013
53
Proportion of Program/Institution-Level Studies Including Data About the Non-Student
Participants
29%
65%
5%
Included Non-Student Par-ticipants (n = 27)Did not Include Non-Student Participants (n = 60)Coder Disagreement (n = 5)
Center on Postsecondary Education and Disability, ; AHEAD 2013
54
Sixteen Subdomains of Program/Institution-Level Studies and Their Frequencies
(Articles could be coded as multiple subdomains)
• Institutional policies/procedures(n = 51)
• Prof. development/training for DSPs (n = 40)
• Program policies and procedures(n = 35)
• Legal compliance (Program specific) (n = 23)
• Experience, knowledge, attitudes,, beliefs of DSPs (n = 20)
20 ≤1
0 – 19
• Legal compliance (Institution specific) (n = 9)
• Programs for incoming students (n = 8)
• Progs for students transitioning to grad school/employment (n = 8)
• Collaboration with other campus services (n = 7)
• Programs for specific cohorts of SWD (n = 7)
• Descriptions/recommendations of disability programs/resources ( n = 5)
• Program evaluation (n = 5)
• Program fit within institution (n = 3)
• Program development (n = 2)
• Other (n = 1)
≥ 9
Center on Postsecondary Education and Disability, ; AHEAD 2013
55
Proportion of Data-Based vs. Non-Data-Based Faculty/Non-Disability Support Staff-Level
Studies
60%
39%
1%
Data-Based (n = 66)Non-Data-Based (n = 43)Coder Disagreement (n=1)
Center on Postsecondary Education and Disability, ; AHEAD 2013
56
Frequency of Data-Based vs. Non-Data-Based Faculty/Non-Disability Support Staff-Level
Studies Over Time
1986-1990
1991-1995
1996-2000
2001-2005
2006-2010
2011-2012
0
5
10
15
20
25
Data-BasedNon-Data-Based
Center on Postsecondary Education and Disability, ; AHEAD 2013
57
Proportion of Faculty/Non-Disability Support Staff-Level Studies by Research Methodology
53%
20%
9%
18%
Quantitative (n = 35)Qualitative (n = 13)Mixed methods (n = 6)Coder disagreement (n = 12)
Center on Postsecondary Education and Disability, ; AHEAD 2013
58
Proportion of Faculty/Non-Disability Support Staff-Level Studies With and Without Control/Comparison Groups
67%
33%With Control/Comparison Group (n = 0)Without Control/Compar-ison Group (n = 2)Coder Disagreement (n = 1)
NOTE: The 1 disagreement exists because one coder indicated that the study was an article without a control/comparison group while the other coder left the data point blank.
Center on Postsecondary Education and Disability, ; AHEAD 2013
59
Proportion of Faculty/Non-Disability Support Staff-Level Studies by Location
65%
18%
16%1%
4-year (n = 44)2-year (n = 12)International (n = 11)Coder Disagreement (n = 1)
Note: 7 studies include multiple locations
Center on Postsecondary Education and Disability, ; AHEAD 2013
60
Location of Faculty/Non-Disability Support Staff-Level Studies Over Time
1976-1980
1981-1985
1986-1990
1991-1995
1996-2000
2001-2005
2006-2010
2011-2012
0
2
4
6
8
10
12
14
4-year2-yearInternational
Center on Postsecondary Education and Disability, ; AHEAD 2013
61
Proportion of Faculty/Non-Disability Support Staff-Level Studies Including Data About the
Non-Student Participants
85%
9%6%
Included Non-Student Par-ticipants (n = 56)Did Not Include Non-Student Participants (n = 6)Coder Disagreement (n = 4)
Center on Postsecondary Education and Disability, ; AHEAD 2013
62
Six Subdomains of Faculty/Non-Disability Staff-Level Studies and Their Frequencies
(Articles could be coded as multiple subdomains)
• Faculty knowledge, attitudes and beliefs (n = 45)
• Faculty teaching practices (n = 23)
20 ≤
• Faculty development and training (n = 17)
• Campus staff knowledge, attitudes, and beliefs (n = 15)
10 – 19
• Campus staff practices (n = 9)
• Campus staff development and training (n = 6)
≥ 9
Center on Postsecondary Education and Disability, ; AHEAD 2013
63
Discussion• Articles on higher education and disability have
been published in 305 unique journals• These journals have a range of purposes, styles,
level of rigor• Articles at the Student-Level are by far the most
common (n = 376) followed by Program/Institutional-Level (n = 235)
• The overall number of published articles in the field increased through the ‘90’s and early ‘00’s, but is declining in past 5 year period
Center on Postsecondary Education and Disability, ; AHEAD 2013
64
Discussion –Good News
• JPED leads the way, but the field is represented in a multi-disciplinary range of journals
• 85% of the articles at the Student-Level are data-based• The data-based studies in this area increased steadily over the
past 30 years (e.g., less than 5 in early 80’s to 100 in 2005-2010)• Number of studies at international institutions have steadily
increased over time• Student-Level studies feature a range of ethnicities• A range of methods are used:
– Quantitative – 54%– Qualitative – 32%
Center on Postsecondary Education and Disability, ; AHEAD 2013
65
Discussion –Concerns (or opportunities!)
• Very limited number of experimental or quasi-experimental studies (n = 8)
• Limited number of studies with control groups (n = 8)• Only 8% of the Student-Level studies are at 2-year
institutions– Trend line has not increased over time
• Better descriptions of samples needed– Size, race, class standing
• Need more data-based studies at the Program/Institutional level (61% are non data-based)– Trend increased in the 1990’s, then dropped through the 2000’s
Center on Postsecondary Education and Disability, ; AHEAD 2013
66
Discussion –More Opportunities…
• Most studies at the Student-Level are on experiences, perceptions, KAB’s (n = 161)
• More articles needed on:– Learning strategies (n=30)– Self-determination (n=17)– Access (n=14)
• More studies needed:– Program/Institutional-Levels– Faculty/Staff-Level
Center on Postsecondary Education and Disability, ; AHEAD 2013
67
Discussion –Limitations
• Not possible to gather every published article– Search terms as broad as possible– Use of a range of data-bases
• Domains and codes for data-collection determined by the research team– Iterative process– Examined multiple journals– Feedback from outside experts
• Coding errors– Each article double coded– Reconciliations
Center on Postsecondary Education and Disability, ; AHEAD 2013
68
Next Steps• Code articles for Domain 4• Additional data-cleaning• Additional analysis to observe frequencies,
trends over time, trends by journals, by locations, etc…
• Deep drill downs into specific areas to identify evidence-based practices, promising practices, etc…
ReferencesCameto, R., Mazzotti, V. L., & Test, D.W. (2011, April). High-quality research in secondary transition: Current status and future need. DCDT Showcase presented at the Council for Exceptional Children Annual Convention, Nashville, TN.
Helsel, F. K. I, Hitchcock, J. H., Miller, G., Malinow, A., & Murray, E. (2006). Identifying evidence-based, promising and emerging practices that use screen-based technology to teach mathematics in grades K-8: A research synthesis. Presented at AERA 2006 Meeting, San Francisco, CA.
Rehabilitation Act of 1973, 29 U.S.C. § 701 et seq.
Test, D. W., Fowler, C., Kohler, P., & Kortering, L. (2010, August). Evidence-based practices and predictors in secondary transition: What we know and what we need to know, Executive Summary.(Revised). Charlotte, NC: National Secondary Transition Technical Assistance Center. Available athttp://www.nsttac.org/sites/default/files/assets/pdf/pdf/ebps/ExecsummaryPPs.pdf
Test, D. W., Fowler, C. H., Richter, S. M., White, J., Mazzotti, V., Walker, A. R., & Kortering, L. (2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32(2), 115-128.
U.S. Department of Education, National Center for Education Statistics. (2012). Digest of Education Statistics, 2011 (2012-001)
Center on Postsecondary Education and Disability, ; AHEAD 2013
70
Q & AThank you!!