measuring the effectiveness of college & career readiness

8/20/2019 Measuring the Effectiveness of College & Career Readiness

1/37

Measuring the Effectivenessof College & Career

Readiness InterventionsCAROLYN BERGER, PH.D.

NOVA SOUTHEASTERN UNIVERSITY

[email protected]


2/37

Overview

Importance of CCR data

Deciding what data to collect

Established CCR assessments

Developing an action plan

Presenting data to stakeholders

Access full presentation at:

http://bit.ly/EBSC-2016-Berger


3/37

What do you think??

School Counselors must collect all tydata on their counseling program.

Do you think collecting CCR data is

especially important?

Why or why not?


4/37

Importance of CCR data

National attention around CCR issues (Reach Higher, North StarGoal, etc.)

The Center on Education and the Workforce reported that:

By 2020, 65% of jobs in the nation will require some form ofpostsecondary education.

At our current rate of educational attainment, the U.S. will be short5 million workers for jobs that require postsecondary credentials by2020.

School counselors need to demonstrate they can help worktowards national and state goals in CCR areas. It’s our time toshine!

DATA is essential to demonstrate our role in this importantinitiative.


5/37


There are a lot of different options for CCR data

See following slides for data that corresponds with the 8NOSCA Components of CCR


6/37


7/37


8/37


9/37

Pair & Share

How can school counselors figure out which CCdata their principal or school district would mostwant to see improve?


10/37


Look at school improvement plan (i.e., school goals

Collect all 3 types of data:

Process (# students)

Perception (e.g., surveys)Outcome (e.g., test scores, grades, attendance, etc

Think about what stakeholders want to see improve

Outcome data is preferred but perception can beused to support


11/37


Process

Participation in extracurriculars &enrichment activities

# students in leadership positions

Participation in career/interestassessments

Participation in financial aid talks

Student enrollment in advanced courses

Outcome

Attendance

Discipline

Promotion

GPA

Dropout

Reading level

Standardized tests

Enrollment in AP

NOSCA includes a wide variety of PROCESS DATA & OUTCOME DATA


12/37


Perception data is not included in NOSCA’s list

Several types of assessments that can gather perceptiodata will be addressed

Perception data isn’t usually as preferred by administratas outcome data

However, perception data can give a lot of usefulinformation to best meet students’ CCR needs


13/37

Why is perception data so useful?

Do students perceive that the counseling interventionsare helpful?

Do students perceive that their cognitive skills, attitudesor knowledge levels have been positively impacted as

a result of a counseling intervention? Perception data is useful to demonstrate:

Knowledge of CCR skills (e.g., how to apply, requirements topursue postsecondary education, etc.)

Confidence levels

Career maturity


14/37

Established CCR Perception Assessme(click on links)

Career Beliefs Inventory (CBI)- 96 items, grades 8-12, 5 categories, (e.g. Factoinfluence my decisions)

Career Decision Making Self-Efficacy Scale (CDSE)- long or short forms, high scbroad areas (e.g., self-appraisal)

Career Thoughts Inventory (CTI)- 48 items, 3 scales (e.g., commitment anxiety)school

College Going Self-Efficacy Scale (CGSES)- 30 items, measures beliefs about cattendance and persistence, MS and HS

Career Development Inventory (CDI)- 80 items, four aspects of career maturitycareer planning), 1 hour, high school

Career Maturity Inventory (CMI)- 24 items (agree or disagree), 6th-12th grade, mstudents’ maturity towards making career decisions

http://www.mindgarden.com/78-career-beliefs-inventoryhttp://www.mindgarden.com/78-career-beliefs-inventoryhttp://www.mindgarden.com/79-career-decision-self-efficacy-scalehttp://www.mindgarden.com/79-career-decision-self-efficacy-scalehttp://www.mindgarden.com/79-career-decision-self-efficacy-scalehttp://www4.parinc.com/Products/Product.aspx?ProductID=CTIhttp://www4.parinc.com/Products/Product.aspx?ProductID=CTIhttp://www.strivetogether.org/sites/default/files/images/16%20College%20Going%20Self-Efficacy%20Scale.pdfhttp://www.strivetogether.org/sites/default/files/images/16%20College%20Going%20Self-Efficacy%20Scale.pdfhttp://www.vocopher.org/ncda2008/data/resources/CDI%20User%20Manual.pdfhttp://www.vocopher.org/ncda2008/data/resources/CDI%20User%20Manual.pdfhttp://www.vocopher.com/ms/cmic/CMI_C_Master.pdfhttp://www.vocopher.com/ms/cmic/CMI_C_Master.pdfhttp://www.vocopher.com/ms/cmic/CMI_C_Master.pdfhttp://www.vocopher.org/ncda2008/data/resources/CDI%20User%20Manual.pdfhttp://www.strivetogether.org/sites/default/files/images/16%20College%20Going%20Self-Efficacy%20Scale.pdfhttp://www4.parinc.com/Products/Product.aspx?ProductID=CTIhttp://www.mindgarden.com/79-career-decision-self-efficacy-scalehttp://www.mindgarden.com/78-career-beliefs-inventory


15/37

Career Beliefs Inventory

96 items

grades 8-12

Five Scales:

1. My Current Career Situation2. What Seems Necessary for My Happiness

3. Factors That Influence My Decisions

4. Changes I Am Willing to Make

5. Effort I Am Willing to Initiate


16/37

Career Beliefs Inventory

Manual = $50; Reproducible PDF = $2 per assessment; Online survey = $2 p


17/37

Career Decision Making Self-EfficacyScale

long or short forms (50 vs. 25 items)

high school

5 broad areas:

Self-Appraisal: The ability to accurately appraise one's own abilitiinterests, and values

Occupational Information: The ability to locate sources of informabout college majors and occupations

Goal Selection: The ability to match one's own characteristics to demands and rewards of careers

Planning: Knowing how to implement an educational or career c

Problem Solving: Being able to figure out alternative plans or cop

strategies when plans do not go as intended.


18/37

Career Decision-Making Self Efficacy S

Manual = $50; Reproducible PDF = $2 per assessment; Online survey = $2.4


19/37

Career Thoughts Inventory

Assist in career problem solving and decision making in college students, and (upper) high school students

$59 manual

$74/25 text booklets

Likert scale: Strongly Agree- Agree- Disagree- Strongly D

48 Items; Three scales:

1. Decision Making Confusion

2. Commitment Anxiety

3. External Conflict


20/37

CTI Sample Items


21/37

College-Going Self-Efficacy Scale

Two scales: Attendance Scale & Persistence Scale

30 Items

Links:

Free access to CGSES Information about reliability & validity

http://www.strivetogether.org/sites/default/files/images/16%20College%20Going%20Self-Efficacy%20Scale.pdfhttp://www.strivetogether.org/sites/default/files/images/16%20College%20Going%20Self-Efficacy%20Scale.pdfhttps://libres.uncg.edu/ir/uncg/f/L_Borders_Measure_2010.pdfhttps://libres.uncg.edu/ir/uncg/f/L_Borders_Measure_2010.pdfhttps://libres.uncg.edu/ir/uncg/f/L_Borders_Measure_2010.pdfhttp://www.strivetogether.org/sites/default/files/images/16%20College%20Going%20Self-Efficacy%20Scale.pdf


22/37

CGSES Sample Items


23/37

College-Going Self-Efficacy Scale

For attendance beliefs, school counselors can examine coabout finances, abilities, family, decision-making, and genfeelings about attending college.

For persistence beliefs, school counselors can focus on abfinances, family responsibilities, life skills, and general feelinabout continuing in college.


24/37

Career Development Inventory

Free on Vocopher.com (register for account online); includes a UseManual and research on validity & reliability

80 items

Four Subscales:

1. Career Planning

2. Career Exploration

3. World of Work knowledge

4. Decision Making


25/37

CDI

The Career Development Inventory was created to:

Measure career maturity

Address career concerns in adolescence


26/37

CDI Sample Items


27/37

Career Maturity Inventory

24 Items

Free on Vocopher.com (register for account online)

Surveys attitudes which are important in making career decisions

Asks about attitudes and feelings toward making a career choiceand entering the world of work

Helps counselors see how mature their students are and identifyspecific areas of concern


28/37

CMI Sample Items

ITEM Response

Once you choose a job, you can't chooseanother one.

AGREE orDISAGREE

In order to choose a job, you need to know whatkind of person you are.

AGREE orDISAGREE

I plan to follow the line of work my parentssuggest. AGREE orDISAGREE

I guess everyone has to go to work sooner thanlater, but I don't look forward to it.

AGREE orDISAGREE

You can do any kind of work you want to as longas you try hard.

AGREE orDISAGREE


29/37

CMI Sample results


30/37

Others??

Structured Career Development Interview (Lapan, 2000

Work Adjustment Inventory (Gilliam, 1994)

Occupational Self-Efficacy Scale (Schyns & von Collani

More?


31/37

Recommendations for ChoosingAssessments

Take them yourself in advance

Make sure they measure what you intend tomeasure: what is the main purpose of your

intervention? Pilot with a few students of the same age as

your intended population


32/37

What if I can’t find the assessment I ne

Consult with counselor educators who maybe able to assist you

You may wish to consider Instrument

development but be aware that it is a multi-step process

Important to establish reliability and validity


33/37

Developing an Action Plan

Pair up with a fellow attendee and discuss your ownplans for measuring CCR data at your school

Which types of CCR data do you think youradministration and other stakeholders want to see?

What types of data will you collect?

What assessments do you want to research further andperhaps implement?


34/37

Presenting Data to Stakeholders

Closely tie results to school goals for CCR

Go back to the question, “what does my administrationto see improved?”

If you cannot demonstrate clear results using outcome

utilize perception data to show how the intervention he Always discuss next steps when presenting data

If it works, discuss plan to do more of it.

If it didn’t work that great, what will you change?


35/37


36/37

Bowers, J. & Hatch, C. (2012) – Third Edition. The ASCA National Model - A framcounseling programs. Alexandria, VA: The American School Counselor Associa978-1-929289-32-5

The College Board (2010). The College Board National Office for School CounsEight components of college and career readiness counseling. Retrieved frommedia.collegeboard.org/digitalServices/pdf/nosca/11b_4416_8_Components

f Curry, J. & Milsom, A. (2014). Career Counseling in P-12 Schools. New York, NY:

Publishing.

Hatch, T. (2014). The Use of Data in School Counseling. Thousand Oaks, CA: Co

Stone, C. B. & Dahir, C. A. (2011). – Third Edition. School counselor accountabiof student success. Upper Saddle River, NJ: Prentice Hall.

References

https://secure-media.collegeboard.org/digitalServices/pdf/nosca/11b_4416_8_Components_WEB_111107.pdfhttps://secure-media.collegeboard.org/digitalServices/pdf/nosca/11b_4416_8_Components_WEB_111107.pdfhttps://secure-media.collegeboard.org/digitalServices/pdf/nosca/11b_4416_8_Components_WEB_111107.pdfhttps://secure-media.collegeboard.org/digitalServices/pdf/nosca/11b_4416_8_Components_WEB_111107.pdfhttps://secure-media.collegeboard.org/digitalServices/pdf/nosca/11b_4416_8_Components_WEB_111107.pdf


37/37

Thank you for coming!!

Full presentation accessible at:

http://bit.ly/EBSC-2016-Berger

Feel free to contact Carolyn Berger at [email protected]
http://bit.ly/EBSC-2016-Bergerhttp://bit.ly/EBSC-2016-Bergerhttp://bit.ly/EBSC-2016-Berger

measuring the effectiveness of college & career readiness

Documents