our literature, our field: findings and trends from postsecondary disability literature allison r....

70
Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University of Connecticut Presentation at the Association on Higher Education and Disability Annual Conference Baltimore, MD July 11, 2013 Center on Postsecondary Education and Disability, ; AHEAD 2013 1

Upload: irene-rustan

Post on 15-Dec-2015

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

1

Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature

Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus

University of Connecticut

Presentation at the Association on Higher Education and Disability Annual Conference

Baltimore, MDJuly 11, 2013

Page 2: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

2

About Us

• Allison– Assistant Professor, Department of Educational Psychology,

Neag School of Education, UConn– Research Associate, Center on Postsecondary Education and

Disability (CPED)• Adam– Doctoral Student, Department of Educational Psychology,

Neag School of Education, UConn• Joe– Associate Professor , Department of Educational Psychology,

Neag School of Education, UConn– Director of CPED, Neag School of Education, UConn– Member, NPSO Advisory Board

Page 3: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

3

Special Thanks

• Lyman L. Dukes III• Michael Faggella-Luby• Nicholas Gelbar• Jennifer S. Kowitt• Melissa Root

Page 4: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

4

Session Objectives

• To explain the rationale for conducting this comprehensive literature review

• To explain the background and methods used• To present specific key findings to date• To present suggestions for future research• To facilitate discussion regarding future

research directions

Page 5: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

5

Project Background

• The Vocational Rehabilitation Act of 1973 required all institutions of higher education to ensure access to qualified students with disabilities (SWD).

• Forty years after the passage of the Act, 11% of college freshmen report having a disability (U.S. G.A.O., 2009).

• The profession of disability services is now longer a nascent field in higher education

Page 6: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

6

Project Background

• To date, a comprehensive analysis of the literature dealing with disability and higher education has not been conducted

• This literature is broad in scope and dispersed across a variety of disciplines (e.g., special education, higher education, psychology, sociology)

• The 40-year anniversary of the passage of Section 504 provides an anniversary to review the field’s literature:– What topics have been studied?– What populations have been studied?– What methodologies have been employed?– How much of the literature is research? How much is not research? – What aspects of the field have substantial evidence and support?– What aspects are lacking substantial evidence and support?– What research areas within the field are likely to receive greater attention in coming

years?• Relevance to practitioners, researchers, policy makers

Page 7: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

7

Project Background• Initially looked to a process used in secondary transition by the National

Secondary Transition Technical Assistance Center (NSTTAC: Test, Fowler, Kohler & Kortering, 2010)

• To review evidence-based practices in secondary transition based on quality experimental studies

• Method:– Procedures: (a) electronic search, (b) reviewing reference lists, (c) hand

searches of journals, and (d) updating by replicating the initial procedures

– Criteria: (a) publ. after 1984, (b) SWD in subjects 11-22 yrs, (c) IV or DV aligned to five areas of Kohler’s Taxonomy

– Focus: Included systematic reviews and group or single subject design studies

– Lens: Applied NSTTAC decision rules for determining levels of evidence (Strong, Moderate, Potential)

– Total: 240 reviews and intervention studies

Page 8: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

8

•Are based on rigorous research designs

•Have demonstrated a record of success for improving student outcomes

•Have undergone systematic review process using quality indicators to evaluate level of evidence

Evidence-Based Practices

•Are based on rigorous research designs

•Have demonstrated a record of success for improving student outcomes

Research-Based Practices

•Are based on research

•Have demonstrated limited success

•Have used a ‘weak’ research design

Promising Practices

•Are not based on research

•Have no data to support effectiveness

•Based on anecdotal evidence and/or professional judgment

Unestablished Practices

(Helsel, Hitchcock, Miller, Malinow, & Murray, 2006; Twyman, 2008)Broad Definitions

From Cameto, Mazzotti, & Test (2011)

Page 9: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

9

Findings• Identified 33 evidence-based practices in secondary Transition• Categorized using Kohler’s Taxonomy for Transition Programming

– 3 in Student-Focused Planning (e.g., involving students in IEP) – 26 in Student Development (e.g., life skills, purchasing skills) – 1 in Family Involvement (training parents about transition)– 3 in Program Structure (extending services beyond secondary school)– No practices identified in the area of Interagency Collaboration

• Only 2 evidence-based practices have a strong level of evidence:– teaching life skills, teaching purchasing skills

• 28 practices had a moderate level of evidence• For more information, see: Test, D. W., Fowler, C. H., Richter, S. M., White, J.,

Mazzotti, V., Walker, A. R., & Kortering, L. (2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32(2), 115-128.

Page 10: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

10

Project Background

• Genesis was a request from NSTTAC to present information about evidence-based practices regarding:– Successful transition to postsecondary education– Success in postsecondary education

• Our plan:– Initially, to follow the NSTTAC meta-analysis procedures– But, postsecondary education lacks a taxonomy for the literature– Postsecondary education does not use the evidence based

practice standards required in secondary education– No prior sorting of the literature, either by topical or research

categories– Required a regrouping and new direction

Page 11: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

11

Our Method

• Began with review of 80+ JPED articles from 2000-2010– Identifying common themes and topics– Development of broad content “domains”• Identification of respective sub-domains

• Domains sent to past two JPED editors for feedback• An electronic rating form was developed and revised• JPED articles from 10 issues reviewed by four coders• Reliability determined, team debriefing, further

refinement of domains, rating form

Page 12: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

12

Initial Domain DescriptionsDomain Name Domain Description

Student Support and Services

Transition into college, student retention, access to accommodations, access to assistive technology, determining eligibility, assessment, evaluation, and diagnosis, coaching

Student Learning and Experiences

Teaching students study skills, learning strategies, teaching self-determination, teaching self-advocacy, teaching students about legal rights and responsibilities, knowledge attitudes and beliefs (KAB), experiences as person with disability

Postsecondary program structure

Program development, program evaluation, policies and procedures, fit within the institution, collaboration with other campus services, legal compliance, determining eligibility

Postsecondary Outcomes

Transition to career or graduate school, employment outcomes

Faculty/Staff Support

Faculty development and training, faculty KAB; Staff development and training, staff KAB

No Fit Studies that do not relate to any of the above domains

Page 13: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

13

Our Method

• Discussion revealed overlaps, determination of key terms– e.g., “policies and procedures”, “experiences”– Where does eligibility “belong”?– Difference between institutional and program legal compliance?– What about studies of instruments and proposed constructs?

• Domains collapsed and updated:– Student level – Program level– Faculty/staff level– Construct level

• JPED articles from an additional 5 issues reviewed by four coders• Reliability determined at 75%-85%• Debriefing lead to 100% agreement; refinement of terms, inclusion

and exclusion criteria

Page 14: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

14

Our Method

• Concurrently, 500 articles from other sources collected• Sorted into domains; reliability measured

– 88% - 96% for sorting

• Articles provided a broader perspective and lead to further refinement of the subdomains

• Validity check by 8 former editors or co-editors of JPED– Measured the clarity of domain definitions

• all were strongly agree or agree that the definition is clear

– Requested suggestions for missing domains– Fit of the subdomains– Suggestions for missing subdomains and clarification of subdomains

• (e.g., legal compliance at the program or institutional level)

Page 15: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

15

Inclusion and Exclusion Criteria• Inclusion criteria:

1. The article is about Postsecondary Education for Students with Disabilities (broadly considered to include faculty, disability services, etc.)2. The article is about one of the following topics/populations:a. Programs for accepted students into degree granting programs at a 2 or 4 year college or universityb. Programs, services, or experiences of matriculated studentsc. Articles about the experiences of students with disabilities who have dropped out of degree granting programs at a 2 or 4 year college or universityd. Articles about the experiences of students with disabilities who are graduates of degree granting programs at a 2 or 4 year college or university

• Exclusion criteria1. Articles that are primarily about secondary students in transition or transition aged programs.

Page 16: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

16

Domain DescriptionsDomain Name Domain Description

Student-Level Studies

Experiences and perceptions of students with disabilities in and after higher education.

Program or Institution-Level

Studies

Service provision by the disability services office in a higher education institution. Can also relate to institutional policies and procedures pertaining to students with disabilities.

Faculty/Non-Disability Support Staff-Level Studies

Knowledge, attitudes, beliefs of faculty and non-disability services personnel to enhance access to higher education for students with disabilities. Also education or support for faculty and staff in this practice.

Construct Development-Level

Studies

Development, evaluation, or validation of a variable, including development/validation of assessment instruments, evaluation metrics, theoretical models of service delivery, standards of practice, or ethics. The variable must be under proposal, in development, or being used in practice to gather empirical evidence.

No Fit Studies that do not relate to any of the above domains.

Page 17: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Sub DomainsStudent Level Studies

• Access (physical, cognitive, attitudinal)• Assistive technology use• Career development• Experiences, perceptions, knowledge, attitudes, or beliefs of students with

disabilities• Learning/using study skills, learning strategies • Mainstream technology use• Meeting institutional requirements (e.g., degree requirements, foreign language

requirements, math requirements)• Post-undergraduate program experiences and/or outcomes (e.g., graduate school,

employment)• Profiles of students (e.g., diagnostic profiles, profiles of successful and/or

unsuccessful students)• Requesting or using accommodations (e.g., assistive technologies, separate testing

location, course substitutions)• Self-determination skills (e.g., self-advocacy, student goal attainment, self-

disclosure, self-management, legal rights and responsibilities)• Statistics on students with disabilities (e.g., rate of access to postsecondary

education, student retention, graduation rate, statistics on accommodation use)

Page 18: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Sub DomainsProgram/Institutional Level Studies

• Collaboration with faculty or academic departments• Collaboration with other campus services • Experiences, perceptions, knowledge, attitudes, or beliefs of disability service

providers• General or specific descriptions of disability programs and resources and/or

recommended program components• Institutional Policies/Procedures• Legal compliance (institutional specific)• Legal compliance (program specific)• Program development• Programs for incoming students (e.g., freshmen, transfer students)• Programs for students transitioning to graduate school or employment• Programs for specific cohorts of students (e.g., LD, Aspergers, etc)• Policies and procedures (e.g., determining student eligibility for services,

determining reasonable accommodations, determining access to assistive technology)

• Professional development/training for disability services staff • Program evaluation (e.g., student retention, student use of program related

services, student graduation rates)• Program fit within the institution (e.g., student affairs v. academic affairs)

Page 19: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Sub DomainsFaculty/Non-Disability Support Level Studies

• Campus staff development and training• Campus staff knowledge, attitudes and beliefs (e.g., about

students with disabilities)• Campus staff practices• Faculty development and training• Faculty knowledge, attitudes and beliefs (e.g., about students

with disabilities; about providing accommodations)• Faculty teaching practices

Page 20: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Sub DomainsConstruct Development Studies

• Assessment instruments (development, validation, use to develop diagnostic profiles)

• Conceptual models or discussion of issues in disability services (e.g., eligibility for services)

• Conceptual models of service delivery (e.g., Universal Design, other models)

• Conceptual models of instruction/assessment of learning• Evaluation metrics or methods• Instructional practices• Standards of practice, performance or ethics.• Other (including disability studies)

Page 21: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Method

Search Terms Included:• Academic Accommodation• Accommodation• ADD• ADHD• Blind• College• College Admission• College Student• Deaf• Disabilities• Disability• Disabled• Dyslexia• Dyslexic

• Handicap• Handicapped• Hearing Impairment• Postsecondary Education• Mental Illness• Mobility Impairment• Postsecondary Education• Student Affairs• Student Personnel• Student Services• University• University Student• Visual Impairment

Continued literature review and collection of articles

Page 22: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

22

Method

• 1,210 articles identified by searches of multiple data bases (e.g., Academic Search Premier, EBSCO, PSYCinfo)

• Published between 1955 and 2012• Articles grouped into domains, reliability measured– Coding resulted in some articles shifting domains

• Will focus on student-level, program or institution-level, faculty/non-disability-level studies today

• Future steps will focus on coding construct development studies

Page 23: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

23

Method

• An electronic database was developed that included the reference citation for each article and unique alpha-numeric codes (e.g., 201225Shaw4).

• The codes were designed to be entered into the electronic coding instrument, so that coding could be linked back to the reference citation.

• Each article randomly assigned to two coders• Reliability determined for each domain• Discrepancies discussed

Page 24: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

24

Instrument

• An electronic coding instrument was designed and refined with two pilots, multiple coders.

• The instrument allowed for the researchers to code:– Did the article meet inclusion criteria?– Did the article present original data?– If not research based, what type? (e.g., lit review, legal

analysis)– If research based, what type? (with multiple layers)– What was the setting? (US, Canada, international, 2- or 4-year)– Sample information? (numbers, gender, disability, race, etc..)– Domain and sub-domain

• Across coding sheet, 148 choices were possible• To achieve agreement, coders selections must be exact

Page 25: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

25

Inter-Rater Reliability

• For today’s presentation, three subsets of articles (Domains 1, 2, and 3) were analyzed.

• Each article coded twice to check for inter-rater reliability.

• Discrepancies discussed and reconciled

Frequency and Reliability by DomainDomain n Overall

ReliabilityDomain 1: Student-Level Studies

376 82%

Domain 2: Program or Institution-Level Studies

235 86%

Domain 3: Faculty/Non-Disability Support Staff-Level Studies

110 86%

Page 26: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

26

Articles by Domain (Initial Sort)

Domain Name N

Student-Level Studies 500

Program or Institution-Level Studies

290

Faculty/Non-Disability Support Staff-Level Studies

125

Construct Development-Level Studies

134

No Fit 161

Page 27: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

27

Frequency of Articles by Domain Over Time

1951-1955

1956-1960

1961-1965

1966-1970

1971-1975

1976-1980

1981-1985

1986-1990

1991-1995

1996-2000

2001-2005

2006-2010

2011-20120

20

40

60

80

100

120

140

Student LevelProgram/InstitutionFaculty/Staff

Page 28: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

28

Journals with the Highest Frequency of Articles About Higher Education and Disability

JPED

Journ

al of L

D

Col Stu

d Journ

al

J Col S

tud Dev

LD R &

P

J Voc R

ehab Couns

Exceptional

Children

Disabilit

y & So

ciety

J Col S

tud Psyc

hotherap

y

LD Q

uarterly

0

50

100

150

200

250

# Articles

Unique Journals: 305

Page 29: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

29

Journals with the Highest Frequency of Student-Level Articles

JPED

Journ

al of L

D

J Col S

tud Dev

Disabilit

y & So

ciety

College

Student J

ournal

LD R &

PCDTEI

Exceptional

Children

Dyslexia

J Col S

tud Psyc

hotherap

y0

102030405060708090

# Articles

Unique Journals: 172

Page 30: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

30

Journals with the Highest Frequency of Program/Institutional-Level Articles

JPEDJ o

f LD

J of V

oc Rehab

New Direct

Stud Se

rv

Col Stu

d Journ

al

Exceptional

Children

New Direct

Higher E

d

TEACHING Exce

pt Child

Disabilit

y & So

c

J Deve

lop Ed

LD R &

P0

10

20

30

40

50

60

70

# Articles

Unique Journals: 109

Page 31: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

31

Journals with the Highest Frequency of Faculty/Non-Disability Staff-Level Articles

JPED

Col Stu

d Jounral

Col Teac

hing

Journ

al of L

D

J Voc S

pecial N

eeds Ed

J Col C

ounselin

g

J Col S

tud Psyc

hotherap

y

NACADA Journ

al

J Stu

d Aff R & P

New Direct

Stud Se

rv

Teaching i

n Higher E

d0

5

10

15

20

25

30

# Articles

Unique Journals: 71

Page 32: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

32

Journals with the Highest Frequency of Construct Development-Level Articles

JPED

Journ

al of L

D

Equity &

Excelle

nce in

Ed

Col Stu

d Journ

al

J Voc R

ehab

New Direct

Higher E

d05

101520253035

# Articles

Unique Journals: 70

Page 33: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

33

Proportion of Data-Based vs. Non-Data-Based Articles for Student-Level Studies

85%

15%

Data-Based (n = 318)Non-Data-Based (n = 58)

Page 34: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

34

Frequency of Data-Based vs. Non-Data-Based Student-Level Studies Over Time

1976-1980

1981-1985

1986-1990

1991-1995

1996-2000

2001-2005

2006-2010

2011-2012

0

20

40

60

80

100

120

Data-BasedNon-Data-Based

Page 35: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

35

Proportion of Student-Level Studies by Research Methodology

54%32%

5%9%

Quantitative (n = 171)Qualitative (n = 103)Mixed Methods (n = 16)Coder Disagreement (n = 28)

Page 36: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

36

Proportion of Student-Level Studies With and Without Control/Comparison Groups

17%

35%

48%With Control/Comparison Group (n = 8)Without Control/Compar-ison Group (n = 16)Coder Disagreement (n = 22)

Page 37: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

37

Proportion of Student-Level Studies Containing a Control/Comparison Group by Experimental vs.

Quasi-Experimental Methodology

17%

50%

33%Experimental (n = 2)Quasi-Experimental (n = 6)Coder Disagreement (n = 4)

Page 38: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

38

Proportion of Student-Level Studies by Location

53%

8%

31%

8%

4-year (n = 190)2-year (n = 29)International (n = 112)Coder Disagreement (n = 28)

Note: 17 studies include multiple locations

Page 39: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

39

Location of Student-Level Studies Over Time

1976-1980

1981-1985

1986-1990

1991-1995

1996-2000

2001-2005

2006-2010

2011-2012

0

10

20

30

40

50

60

4-year2-yearInternational

Page 40: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

40

Proportion of Student-Level Studies Including Clear Sample Size Data

71%

19%

10%

Studies Including Student Sample Size (n = 267)Studies Not Including Student Sample Size (n = 73)Coder Disagreement (n = 36)

Page 41: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

41

Proportion of Student-Level Studies Including Data on the Race/Ethnicity of Participants

17%

75%

8%

Studies Clearly Provide Data on Race of Partic-ipants (n = 53)Studies Do Not Clearly Provide Data on Race of Participants (n = 240)Coder Disagreement (n = 25)

Mean = 6,136SD = 91,021Min = 1Max = 1,502,658

Joe Madaus
could this be right?
Page 42: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

42

Proportion of Student-Level Studies Including Disability-Related Demographic Information

68%

21%

11%

Study Provided Frequen-cies of Disability Type (n = 217)Study Did Not Provide Frequencies of Disability Type (n = 66)Coder Disagreement (n = 35)

Page 43: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

43

Proportion of Student-Level Articles Including Data About the Gender of Participants

51%

35%

14%

Reported Gender (n = 161)Did Not Report Gender (n = 111)Coder Disagreement (n = 46)

Page 44: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

44

Proportion of Student-Level Articles Including Data on the Class Standing of Participants

14%

76%

11%

Reported Class Standing Data (n = 43)Did Not Report Class Standing Data (n = 241)Coder Disagreement (n = 34)

Page 45: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

45

Proportion of Student-Level Articles Including Data About the Non-Student Participants

7%

89%

3%

Included Non-Student Par-ticipants (n = 23)Did Not Include Non-Stu-dent Participants (n = 284)Coder Disagreement (n = 11)

Page 46: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

46

Thirteen Subdomains of Student-Level Studies and Their Frequencies

(Articles could be coded as multiple subdomains)

• Experience, perception, knowledge, attitude of SWD (n = 161)

• Profiles of SWD (n = 54)

• Learning/using learning strategies (n = 30)

20 ≤

• Statistics on SWD (n = 19)

• Requesting or using accommodations (n = 18)

• Self-determination (n = 17)

• Access (physical/cognitive/attitudinal) (n = 14)

• Assistive technology use (n = 11)

10 – 19

• Career development (n = 9)

• Mainstream tech use (n = 6)

• Other (n = 6)

• Meeting institutional requirements (n = 5)

• Post-undergraduate outcomes (n = 1)

≥ 9

Page 47: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

47

Proportion of Data-Based vs. Non-Data-Based Program/Institution-Level Studies

39%

61%

Data-Based (n = 92)Non-Data-Based (n = 143)

Page 48: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

48

Frequency of Data-Based vs. Non-Data-Based Program/Institution-Level Studies Over Time

1951-1955

1956-1960

1961-1965

1966-1970

1971-1975

1976-1980

1981-1985

1986-1990

1991-1995

1996-2000

2001-2005

2006-2010

2011-20120

5

10

15

20

25

30

35

40

Data-BasedNon-Data-Based

Page 49: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

49

Proportion of Program/Institution-Level Studies by Research Methodology

60%27%

1%12%

Quantitative (n = 55)Qualitative (n = 25)Mixed Methods (n = 1)Coder Disagreement (n = 11)

Page 50: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

50

Proportion of Program/Institution-Level Studies With and Without Control/Comparison Groups

14%

86%

With Control/Comparison Group (n = 0)

Without Control/Compar-ison Group (n = 1)

Coder Disagreement (n = 6)

NOTE: All 6 disagreements exist be-cause one coder indicated that the study was an article without a control/comparison group while the other coder left the data point blank.

Page 51: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

51

Proportion of Program/Institution-Level Studies by Location

47%

24%

13%

16%

4-Year (n = 64)2-Year (n = 33)International (n = 18)Coder Disagreement (n = 22)

Note: 42 studies include multiple locations

Page 52: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

52

Location of Program/Institution-Level Studies Over Time

1951-1955

1956-1960

1961-1965

1966-1970

1971-1975

1976-1980

1981-1985

1986-1990

1991-1995

1996-2000

2001-2005

2006-2010

2011-20120

2

4

6

8

10

12

14

16

18

4-Year2-YearInternational

Page 53: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

53

Proportion of Program/Institution-Level Studies Including Data About the Non-Student

Participants

29%

65%

5%

Included Non-Student Par-ticipants (n = 27)Did not Include Non-Student Participants (n = 60)Coder Disagreement (n = 5)

Page 54: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

54

Sixteen Subdomains of Program/Institution-Level Studies and Their Frequencies

(Articles could be coded as multiple subdomains)

• Institutional policies/procedures(n = 51)

• Prof. development/training for DSPs (n = 40)

• Program policies and procedures(n = 35)

• Legal compliance (Program specific) (n = 23)

• Experience, knowledge, attitudes,, beliefs of DSPs (n = 20)

20 ≤1

0 – 19

• Legal compliance (Institution specific) (n = 9)

• Programs for incoming students (n = 8)

• Progs for students transitioning to grad school/employment (n = 8)

• Collaboration with other campus services (n = 7)

• Programs for specific cohorts of SWD (n = 7)

• Descriptions/recommendations of disability programs/resources ( n = 5)

• Program evaluation (n = 5)

• Program fit within institution (n = 3)

• Program development (n = 2)

• Other (n = 1)

≥ 9

Page 55: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

55

Proportion of Data-Based vs. Non-Data-Based Faculty/Non-Disability Support Staff-Level

Studies

60%

39%

1%

Data-Based (n = 66)Non-Data-Based (n = 43)Coder Disagreement (n=1)

Page 56: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

56

Frequency of Data-Based vs. Non-Data-Based Faculty/Non-Disability Support Staff-Level

Studies Over Time

1986-1990

1991-1995

1996-2000

2001-2005

2006-2010

2011-2012

0

5

10

15

20

25

Data-BasedNon-Data-Based

Page 57: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

57

Proportion of Faculty/Non-Disability Support Staff-Level Studies by Research Methodology

53%

20%

9%

18%

Quantitative (n = 35)Qualitative (n = 13)Mixed methods (n = 6)Coder disagreement (n = 12)

Page 58: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

58

Proportion of Faculty/Non-Disability Support Staff-Level Studies With and Without Control/Comparison Groups

67%

33%With Control/Comparison Group (n = 0)Without Control/Compar-ison Group (n = 2)Coder Disagreement (n = 1)

NOTE: The 1 disagreement exists because one coder indicated that the study was an article without a control/comparison group while the other coder left the data point blank.

Page 59: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

59

Proportion of Faculty/Non-Disability Support Staff-Level Studies by Location

65%

18%

16%1%

4-year (n = 44)2-year (n = 12)International (n = 11)Coder Disagreement (n = 1)

Note: 7 studies include multiple locations

Page 60: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

60

Location of Faculty/Non-Disability Support Staff-Level Studies Over Time

1976-1980

1981-1985

1986-1990

1991-1995

1996-2000

2001-2005

2006-2010

2011-2012

0

2

4

6

8

10

12

14

4-year2-yearInternational

Page 61: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

61

Proportion of Faculty/Non-Disability Support Staff-Level Studies Including Data About the

Non-Student Participants

85%

9%6%

Included Non-Student Par-ticipants (n = 56)Did Not Include Non-Student Participants (n = 6)Coder Disagreement (n = 4)

Page 62: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

62

Six Subdomains of Faculty/Non-Disability Staff-Level Studies and Their Frequencies

(Articles could be coded as multiple subdomains)

• Faculty knowledge, attitudes and beliefs (n = 45)

• Faculty teaching practices (n = 23)

20 ≤

• Faculty development and training (n = 17)

• Campus staff knowledge, attitudes, and beliefs (n = 15)

10 – 19

• Campus staff practices (n = 9)

• Campus staff development and training (n = 6)

≥ 9

Page 63: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

63

Discussion• Articles on higher education and disability have

been published in 305 unique journals• These journals have a range of purposes, styles,

level of rigor• Articles at the Student-Level are by far the most

common (n = 376) followed by Program/Institutional-Level (n = 235)

• The overall number of published articles in the field increased through the ‘90’s and early ‘00’s, but is declining in past 5 year period

Page 64: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

64

Discussion –Good News

• JPED leads the way, but the field is represented in a multi-disciplinary range of journals

• 85% of the articles at the Student-Level are data-based• The data-based studies in this area increased steadily over the

past 30 years (e.g., less than 5 in early 80’s to 100 in 2005-2010)• Number of studies at international institutions have steadily

increased over time• Student-Level studies feature a range of ethnicities• A range of methods are used:

– Quantitative – 54%– Qualitative – 32%

Page 65: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

65

Discussion –Concerns (or opportunities!)

• Very limited number of experimental or quasi-experimental studies (n = 8)

• Limited number of studies with control groups (n = 8)• Only 8% of the Student-Level studies are at 2-year

institutions– Trend line has not increased over time

• Better descriptions of samples needed– Size, race, class standing

• Need more data-based studies at the Program/Institutional level (61% are non data-based)– Trend increased in the 1990’s, then dropped through the 2000’s

Page 66: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

66

Discussion –More Opportunities…

• Most studies at the Student-Level are on experiences, perceptions, KAB’s (n = 161)

• More articles needed on:– Learning strategies (n=30)– Self-determination (n=17)– Access (n=14)

• More studies needed:– Program/Institutional-Levels– Faculty/Staff-Level

Page 67: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

67

Discussion –Limitations

• Not possible to gather every published article– Search terms as broad as possible– Use of a range of data-bases

• Domains and codes for data-collection determined by the research team– Iterative process– Examined multiple journals– Feedback from outside experts

• Coding errors– Each article double coded– Reconciliations

Page 68: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

68

Next Steps• Code articles for Domain 4• Additional data-cleaning• Additional analysis to observe frequencies,

trends over time, trends by journals, by locations, etc…

• Deep drill downs into specific areas to identify evidence-based practices, promising practices, etc…

Page 69: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

ReferencesCameto, R., Mazzotti, V. L., & Test, D.W. (2011, April). High-quality research in secondary transition: Current status and future need. DCDT Showcase presented at the Council for Exceptional Children Annual Convention, Nashville, TN.

Helsel, F. K. I, Hitchcock, J. H., Miller, G., Malinow, A., & Murray, E. (2006). Identifying evidence-based, promising and emerging practices that use screen-based technology to teach mathematics in grades K-8: A research synthesis. Presented at AERA 2006 Meeting, San Francisco, CA.

Rehabilitation Act of 1973, 29 U.S.C. § 701 et seq.

Test, D. W., Fowler, C., Kohler, P., & Kortering, L. (2010, August). Evidence-based practices and predictors in secondary transition: What we know and what we need to know, Executive Summary.(Revised). Charlotte, NC: National Secondary Transition Technical Assistance Center. Available athttp://www.nsttac.org/sites/default/files/assets/pdf/pdf/ebps/ExecsummaryPPs.pdf

Test, D. W., Fowler, C. H., Richter, S. M., White, J., Mazzotti, V., Walker, A. R., & Kortering, L. (2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32(2), 115-128.

U.S. Department of Education, National Center for Education Statistics. (2012). Digest of Education Statistics, 2011 (2012-001)

Page 70: Our Literature, Our Field: Findings and Trends From Postsecondary Disability Literature Allison R. Lombardi, Adam R. Lalor, & Joseph W. Madaus University

Center on Postsecondary Education and Disability, ; AHEAD 2013

70

Q & AThank you!!