Annual BEAP ReportMarch 20, 2010

27th Annual BPD Conference

Atlanta, GA

Introduction

Foundation Curriculum Assessment Instrument

Purpose of the FCAI

1. Provides Pre/Post test in seven major curricular areas of a BSW program

2. Seeks to help the program assess “value added” from entry to exit

3. Assists with identification of curricular areas that may need attention

4. Provides national comparative data

Curricular components Curriculum Area Number of

Questions

Practice 13

Human Behavior & Social Environment 10

Policy 9

Research 9

Ethics and values 8

Diversity 8

Social and Economic Justice 7

Sample HBSE Question

• The concept “person-in-environment” includes which of the following:a. Clients are influenced by their environment

b. Clients influence their environment

c. Behavior is understood in the context of one’s environment

d. All of the above

Sample Practice Question

• Determining progress toward goal achievement is one facet of the _____ stage.

– a. Engagement – b. Evaluation – c. Assessment – d. Planning

Testing History – 2006-2010 • Version 3 1 Test 65 ?s n= 305• Version 4 1 test 55 ?s n= 381• Version 5 1 test 82 ?s n= 286• Version 6 1 test 74 ?s n= 36• Version 7 1 test 72 ?s n= 318• Version 8 1 test 64 ?s

n=1576

TOTAL =2902

Reliability Testing – Version 8

–Tested in two junior practice classes–Students tested twice, 2 weeks apart–Pearson’s correlation coefficient: r

= .86

– a

Item difficulty index

• Overall difficulty or average should be around .5 (Cohen & Swerdlik, 2005)

• FCAI = .523 (n=415)

• “This is a very good difficulty level for the test.Not likely to misrepresent the knowledge level of test takers”.

Expansion beyond BSW

• Based upon CSWE assertions related to educational levels in social work education, we expanded testing to three additional groups: – MSW foundation students:

• entering • exiting

– Advanced standing students: • entering

Mean scores by Educational level – Version 8

Program level

N Mean Score

Std Dev Min Max

BSW entering 441 31.34 7.27 10 57

BSW exiting 522 40.62 7.25 9 58

MSW F entering 277 37.01 7.40 16 54

MSW F exiting 248 41.15 7.84 11 59

Advanced Standing entering

88 41.57 6.24 25 55

Instrument Discriminant Analysis

Pre/post test for differenceBSW & MSW Foundation

Groups by pre/post scores

Mean Pre test Score

Mean Post test score

SD

t-value

Sig.

BSW ENTERING/BSW EXITING

31.34 40.62 7.3

19.76

.000

MSW F ENTERING/MSW F EXITING

37.01 41.15 7.6

6.219

.000

Tests of Difference by Educational Level

Groups by Educational level

Mean MSW Exit Score

Mean BSW Exit Score

SD t-value

Sig/P-value

41.15 40.62 7.5 .924 Not Sig

Tests of Difference by Educational level

Groups by Educ. Level

BSW exiting

Advanced Standing

SD t-value Sig.

40.62 41.57 6.7

1.157 Not Sig.

BSW Entering BSW Exiting t-test p-value

441 522

Total Score (64)

31.734 40.62 -19.77 .000

Practice (13) 6.91 9.51 -20.01 .000

HBSE (10) 5.54 6.65 -10.25 .000

Policy (9) 3.49 4.71 -10.98 .000

Research (9) 3.41 4.84 -12.89 .000

Ethics (8) 4.18 5.47 -13.72 .000

Diversity (8) 3.94 4.71 -7.44 .000

Social & Econ Justice (7)

3.88 4.73 -8.35 .000

Domain Comparisons- Version 8 Overall

MSW F Entering

MSW F Exiting

t-test p-value

N 277 248

Total Score (64) 37.01 41.15 -6.22 .000

Practice (13) 8.44 9.44 -5.69 .000

HBSE (10) 6.09 6.69 -4.05 .000

Policy (9) 4.36 5.01 -3.40 .000

Research (9) 4.39 5.19 -4.87 .000

Ethics (8) 4.65 5.21 -4.95 .000

Diversity (8) 4.68 4.52 1.17 .244

Social & Econ Justice (7)

4.39 5.09 -5.00 .000

Domain Comparisons- Version 8 Overall

BSW Exiting

Advanced Standing

t-test p-value

N 522 88

Total Score (64) 40.62 41.57 -1.16 .248

Practice (13) 9.51 9.65 -0.61 .544

HBSE (10) 6.65 6.62 0.11 .913

Policy (9) 4.71 4.90 -0.93 .352

Research (9) 4.84 4.72 0.61 .539

Ethics (8) 5.47 5.60 -0.85 .396

Diversity (8) 4.71 4.98 -1.48 .140

Social & Econ Justice (7)

4.73 5.10* -2.23 .026

Domain Comparisons-Version 8 Overall

BSW Exiting

MSW F Exiting

t-test p-value

N 522 248

Total Score (64) 40.62 41.15 -0.92 .356

Practice (13) 9.51 9.44 0.47 .640

HBSE (10) 6.65 6.69 -0.34 .733

Policy (9) 4.71 5.01* -2.19 .029

Research (9) 4.84 5.19* -2.52 .012

Ethics (8) 5.47* 5.21 2.57 .011

Diversity (8) 4.71 4.52 1.60 .109

Social & Econ Justice (7)

4.73 5.09* -3.08 .002

Domain Comparisons- Version 8 Overall

Field Practicum/Placement Assessment Instrument

(FPPAI)

• In response to evaluate social work program outcomes related to the 2008 EPAS competencies and practice behaviors

• The need for a field/practicum assessment that measures competencies related to practice behaviors

Need for Field Evaluation

• Initial Piloting was conducted May 2008

• Second Piloting was conducted in Fall 2008 /Spring 2009

• Third Piloting phase was conducted Fall 2009

• Full implementation: Fall 2010

FPPAI Piloting Phases

• 55 Likert Scale questions measuring practice behaviors linked to the EPAS 2008 competencies.

• Qualitative feedback form for each domain available for program use.

• Will be available online and in print format.

• Individual program outcomes report with national comparisons.

• Individual program outcomes report with national comparisons for EPAS 2008 Competencies & Practice Behaviors

• Can be used as a final field assessment and mid-test/post test design.

• MSW pilot in the planning stages.

Assessment Methodology

FPPAI Scale

FPPAI: Instrument Sample (Quantitative)

FPPAI Sample: Qualitative Feedback Form

Educational Policy

2.1.1—Identify as a professional social worker and conduct oneself accordingly

Practice Behavior

use supervision and consultation

Measures

BEAP-Exit

BEAP-FPPAI

BEAP-Employer

BEAP-Graduate

BEAP & EPAS 2008

• 204 participants from 18 schools and 18 states participated in the FPPAI pilot project.

• Overwhelming positive feedback from participating schools.

• Incorporated pilot feedback survey results into final pilot revisions.

• Cronbach’s Alpha reliability test of internal consistency was completed, (n=204, .962) reporting excellent internal consistency.

• Fall online MSW foundation pilot

FPPAI: Overview & Key Findings

If your program is interested in being a part of the national ”MSW Foundation Online” pilot, please access the FPPAI website:

http://www.lcsc/bchristenson/fpai.htm

Dr. Brian Christenson:– blchristenson@lcsc.edu

Dr. Ruth Gerritsen-McKane:– Ruth.Gerritsen-McKane@socwk.utah.edu

Call for Foundation Year MSW Participation

http://beap.socwk.utah.edu/

http://www.lcsc.edu/bchristenson/fpai.htm

2009 Report on the Revised BEAP Instruments

Sample Profile Overview of all Respondents

2000-2006

Respondents

2007

Respondents

2008

Respondents

2009

Respondents

Overall

Respondents

Entrance 24,977 3,319 2,287 524 31,107

SWVI at Entrance 25,446 3,309 1,611 348 30,714

Exit 14,304 2,546 2,157 761 19,768

SWVI at Exit 11,936 2,283 1,551 423 16,193

Alumni 4,488 428 337 96 5,349

Employer 1,613 181 118 23 1,935

Totals 82,764 12,066 8,061 2,175 105,066

Sample vs. Population Program Type

2000-2006

Entrance

2007

Entrance

2008

Entrance

2009

EntranceOverall

Program

TypeN % N % N % N % BEAP % CSWE%

BSW Only 747 71.3 98 74.2 60 73.9 83 86.6 72.0 70.6

Combined

Programs

177 28.7 15 25.8 10 26.1 17 13.4 27.8 29.3

Total 924 100 113 100 70 100 100 100 99.8 100

Sample Profile Response Rates

Instrument2000-2006

Average %

2007

Average %

2008

Average %

2009

Average %

Overall

Average %

Entrance 96.03 96.17 96.10 91.42 96.10

Exit 93.56 93.98 94.01 91.98 93.55

Alumni 43.41 44.54 45.06 36.39 42.81

Employer 30.57 29.42 31.40 24.40 30.31

Auspices of College or University(Overall-Entrance)

Auspice Program Percentage Student Percentage

Public(#schools)

51 68.8

Private Denominational(#schools)

34 19.7

Private Non-Denominational(#schools)

15 11.6

Ethnicity at EntranceEthnicity

2000-2006 %

N= 24,977

2007 %

N= 3,319

2008 %

N= 2,287

2009 %

N= 524

Overall %

N= 31,107

European/

Caucasian67.1 62.2 66.9 57.1 65.6

African/Black 17.6 21.2 16.5 25.2 18.4

Native American 9.9 8.1 8.7 5.9 9.5

Other 3.8 3.7 3.8 2.7 3.9

Other Hispanic 3.8 5.6 4.0 6.5 4.2

Asian/Pacific Islander 3.4 3.5 3.9 2.9 3.4

Chicano/Mexican 3.0 3.5 3.5 4.6 3.1

Puerto Rican 1.6 1.7 1.6 3.6 1.6

Note: Percentages equal more than 100% due to multiple self-reported ethnicities.

Financial Aid at Entrance

Aid Source2000-2006 %

N= 24,977

2007 %

N= 3,319

2008 %

N= 2,287

2009 %

N= 524

Overall %

N= 31,107

Self Work/Savings 78.8 76.8 81.0 75.0 77.7

Loan* 77.9 78.5 81.0 89.7 77.0

Grant* 76.3 78.4 72.4 82.0 74.9

Family Assistance 44.0 43.0 41.1 39.5 43.3

*Includes Federal and State only.

Plan to be Employed During BSW Education

(Reported at Entrance)

2000-2006 %

N= 24,977

2007 %

N= 3,319

2008 %

N= 2,287

2009 %

N= 524

Overall %

N= 31,107

Plan to be Employed 79.4 81.0 80.5 73.7 78.9

Educational

Plans

2000-2006 %

N= 14,304

2007 %

N= 2,546

2008 %

N= 2,157

2009 %

N=761

Overall %

N=19,768

Have Future Educational Plans

82.9 83.2 84.6 86.5 82.2

MSW 75.7 77.8 79.4 80.7 75.6

Other MA 13.5 12.0 11.3 12.6 12.8

Other BA 2.9 3.0 2.9 3.0 2.9

Note: Percentages equal more than 100% due to multiple self-reported future educational plans.

Future Educational Plans-Exit

KSV- Exit

Mean Score 2000-2006 2007 2008 2009 Overall

Knowledge 7.28 7.38 7.38 7.25 7.30

Skills 7.35 7.43 7.48 7.31 7.37

Values 8.64 8.69 8.73 8.56 8.65

(Scale of 0-10)

Exit Advising Scores

2000-2006

N= 14,304

2007

N= 2,546

2008

N= 2,157

2009

N= 761

Overall Average

N= 19,768

Course selection/

curriculum planning6.82 7.00 6.95 6.71 6.84

Professional advising 6.66 6.80 6.85 6.44 6.68

Career planning 6.07 6.28 6.29 5.94 6.11

(Scale of 0-10)

Overall Advising Scores by Auspice- Exit

Course Selection and Curriculum Planning

Career Planning Professional Advising

Private Denominational

College or University

(# Schools)

7.46 6.75 7.33

Private-Non

Denominational College or University

(# Schools)

7.20 6.56 7.22

Public College or

University

(# Schools)6.61 5.86 6.40

(Scale of 0-10)

Overall Advising Scores by Program Type-Exit

Course Selection and Curriculum

Planning*

Career Planning* Professional

Advising*

BSW Only 7.08 6.43 7.01

Co-located Programs 6.14 5.18 5.71

N= 20,833t- test*p= .000

(Scale of 0-10)

Current Employment Information Primary Function - Exit

n= number reporting

Primary Function2000-2005 %

n=3536

2006 %

n=688

2007 %

n=731

2008 %

n=563

Overall %

n=5857

Generalist Practice 39.2 38.2 35.3 38.7 38.5

Direct Practice with Individuals, Families, or Groups

25.0 25.7 26.1 25.8 25.0

Training 10.8 11.3 12.2 11.7 11.3

Administration 8.5 7.3 8.3 7.1 8.3

Other Functions 5.0 4.5 6.4 6.2 5.3

Current Employment Information Primary Role - Exit

Primary Role2009 %

n=224

Advocate/Activist 33.5

Broker 22.3

Enabler/Facilitator 16.1

Administrator 9.4

Counselor/Clinician 6.7

Initiator 5.8

Coordinator 3.6

Case Manager 1.3

Convener/Mediator 1.3

n= number reporting

Client Systems - Alumni/ae

2000-2006 %

N= 4,488

2007 %

N= 428

2008 %

N= 337

2009 %

N= 96

Overall %

N= 5,349

Individual Clients 66.1 63.8 67.7 67.4 65.3

Family Clients 61.7 61.7 61.7 62.0 61.2

Group Clients 34.6 36.4 34.7 34.0 34.4

Organization Client 29.3 25.5 28.5 26.3 28.0

Community Clients 24.1 23.8 25.2 23.2 23.9

Other Client systems 2.9 3.0 2.7 2.5 2.9

Income Level Most Represented by Clients - Alumni/ae

Income Level

2000-2006 %

N= 4,488

2007 %

N= 428

2008 %

N= 337

2009 %

N= 96

Overall %

N= 5,349

Poverty or Below 52.0 54.2 54.1 54.5 52.0

No Typical Income 25.2 24.5 18.7 23.8 25.2

Above Poverty and Below Middle Income

17.8 16.7 13.9 16.4 17.8

Middle Income

or Above5.0 4.6 6.1 5.3 5.0

Professional Development During the Last Year

(Reported at Alumni/ae)

2000-2006

N= 4,488

2007

N= 428

2008

N= 337

2009

N= 96

Overall

N= 5,349

Mean # of times used evaluation of client progress 17.31 14.5 14.96 16.27 16.78

Mean # of times used program evaluation 6.21 5.41 5.88 6.36 6.08

Mean # of times used other research techniques 5.44 4.86 5.49 4.16 5.26

Mean # of professional workshops attended 5.11 5.02 4.78 4.55 4.86

Mean # of professional conferences attended 2.58 2.38 2.34 2.70 2.50

Current Employment Information- Field of Practice

Overall Exit

n=7,577

Overall Alumni/ae

n= 4,083

Corrections/Criminal Justice

23.1%

Child Welfare/Child Protection

19.4%

Child Welfare/Child Protection

14.6%

Mental/Behavioral/CMH

15.4%

Youth Services

8.4%

Aging and Gerontological Social Work

10.9%

Mental/Behavioral/CMH

7.1%

Health/Medical

7.6%

Aging and Gerontological Social Work

5.9%

Family Service

7.5%

n= number of responses

Current Employment Information 2009 Exit-Field of Practice

2009 Exit

n= 200

Overall Exit

n=7,577

Child Welfare/Child Protection

16.0%

Corrections/Criminal Justice

23.1%

Aging and Gerontological Social Work

10.0%

Child Welfare/Child Protection

14.6%

Youth Services10.0%

Youth Services

8.4%

Mental Retardation/Developmental Disabilities

9.5%

Mental/Behavioral/CMH

7.1%

Family Service

8.5%

Aging and Gerontological Social Work

5.9%

Current Employment Information 2009 Alumni/ae - Field of Practice

2009 Alumni/ae

n= 58

Overall Alumni/ae

n= 4,083

Child Welfare/Protection

29.3%

Child Welfare/Child Protection

19.4%

Aging and Gerontological Social Work

15.5%

Mental/Behavioral/CMH

15.4%

Alcohol, Drug, or Substance Abuse (and)

Mental Retardation/Developmental Disabilities

6.9%

Aging and Gerontological Social Work

10.9%

Housing (and)

School Social Work

5.2%

Health/Medical

7.6%

Family Service

7.5%

Mean* Knowledge Scores—at ExitBSW only BSW & MSW

Co-locatedTest**

Public7.35 6.82 p<.001

Private-

Denominational 7.59 6.77 p< .001

Private-

Non Denominational

7.65 7.26 p< .001

* Mean score on 10 point scale ** t-test, N= 19,768

Mean* Skill Scores—at ExitBSW only BSW & MSW

Co-locatedTest**

Public

7.41 6.84 p< .001

Private-

Denominational 7.71 6.86 p< .001

Private-

Non Denominational

7.77 7.46 p= .001

* Mean score on 10 point scale

** t-test, N= 19,768

Mean* Value Scores—at ExitBSW only BSW &

MSW Co-located

t-Test**

Public

8.71 8.30 p<.001

Private-

Denominational 8.82 8.36 p=.001

Private-

Non Denominational

8.78 8.73 p=.523

* Mean score on 10 point scale

** t-test, N= 19,768

Region I—Southeast*

Knowledge, Skills and Value Scores by Deciles

*Alabama, Arkansas, Mississippi, Georgia, Florida, South Carolina, Louisiana, Tennessee

Percentile Group

Knowledge Skills Values

Mean Mean Mean

10 5.64 5.62 7.00

20 6.17 6.16 8.00

30 6.75 6.85 8.00

40 7.17 7.31 8.77

50 7.67 7.77 9.00

60 8.00 8.00 9.50

70 8.25 8.31 9.80

80 8.75 8.85 10.00

90 9.33 9.38 10.00

Region 1 Public BSW Only Programs

What BEAP can do for your program

• Student demographics• Assessment on KSV• Employment information• Advising feedback• Professional development of students &

alumni/ae• Peer comparison by region, program type,

auspice & nationally• …and more

BEAP Instruments & EPAS Competencies/Practice Behaviors

2.1.6—Engage in research-informed practice and practice-informed research.

A.Use practice experience to inform scientific inquiry

B. Use research evidence to inform practice

Graduate

C19, 20, 21

E4

Exit

D20

D19

Employer

Q14

FCAI

31-39

FPPAI

23,36,37,38

23,36,37,38

BEAP & EPAS 2008

http://beap.socwk.utah.edu/

http://beap.socwk.utah.edu/site/annual.htm