cathleen l. smith, ph.d. professor emerita of psychology portland state university

Does Contact with Advisors Predict Advising Learning Outcomes?

A Multi-Institutional Study(CODE: 186)

National NACADA ConferenceOctober 5, 2012

Cathleen L. Smith, Ph.D. Professor Emerita of Psychology

Portland State University

[email protected] Janine M. Allen, Ph.D. Professor Emerita of Education

Portland State University

[email protected]

Acknowledgements

Grant from the NACADA Foundation

Support of members of our research collaborative and their institutional research offices

Agenda

Describe the advising curriculum and what we would expect students to learn from advising encounters

Present a study that examines advising learning of students from nine institutions

Discuss implications of findings

Advising Learning: A New Emphasis

New emphasis: What students should learn in academic advising encounters

NACADA Concept of Academic Advising: – Advising is “integral to fulfilling the teaching

and learning mission of higher education”– And, as such, has its own curriculum,

pedagogy, and student learning outcomes (NACADA, 2006)

Learning-Centered Advising:State of the Literature

More conceptual than empirical

Focused on: Identifying the advising curriculum (e.g., Hemwall &

Trachte, 2005; Lowenstein, 2005) and learning outcomes advising should produce in students Distinguishing between learning-centered advising and more traditional approaches (i.e., prescriptive and developmental advising)


Focused on: Speculating about the mechanisms by which learning takes place in advising encountersDifferentiating learning outcomes from other aspects of advising (e.g., student responsibilities) Advocating for the adoption and use of a learning-centered advising paradigm


A logical next step in the evolution of this new advising paradigm is to gather empirical data on the learning outcomes that are thought to arise from participation in advising encounters

Deriving our Advising Learning Outcomes

In formulating our learning outcomes, we began with our conception of quality academic advising as a multi-dimensional process encompassing five domains

– Integration– Referral– Information– Individuation– Shared responsibility


Integration of the student’s academic, career, and life goals with each other and with aspects of the curriculum and co-curriculum

Referral to campus resources for academic and non-academic problems

Provision of information about degree requirements and how the university works with regard to policies and procedures


Individuation, or consideration of students’ individual characteristics, interests, and skills

Shared responsibility, or encouraging students to assume responsibility for their education by providing them with opportunities to develop and practice planning, problem-solving, and decision-making skills

Advising Curriculum

Advising Content:

Integration, Referral, Information

Advising Pedagogy:

Individuation, Shared Responsibility

Advising Content: Information

Our past research (Allen & Smith, 2008; Smith &

Allen, 2006) has shown the primary importance to students of the information domain

Thus it was represented by two learning outcomes


Advising assists students in understanding the multitude of requirements they face in order to successfully complete their program of study

Information: 1st Learning Outcome

1. Knows RequirementsCommunity college students: “I know what requirements (e.g., prerequisites, general education, transfer requirements) I must fulfill at name of community college in order to meet my educational goals”

University students: “I know what requirements (e.g., major, general education, other university requirements) I must fulfill in order to earn my degree”


Advising helps students navigate their complex institution by assisting them in understanding how things work with regard to its timelines, policies and procedures

Information: 2nd Learning Outcome

2. Understands How Things Work

“I understand how things work at name of institution (timelines, policies, and procedures with regard to registration, financial aid, grading, graduation, petition and appeals, etc.)”

Advising Content: Referral

Advising is a conduit through which the student becomes aware of resources at the institution that assist with – Academic problems (e.g., writing, test

anxiety, tutoring) – Non-academic problems (e.g., child care,

financial, physical and mental health)

Referral: Learning Outcome

3. Knows Resources

“When I have a problem, I know where at name of institution I can go to get help”

Advising Content: Integration

Advising promotes connected learning: – One of the primary goals of liberal

education (Cronon, 1998)

– Central to developmental advising– Considered by students as especially

influential (Light, 2001)

Integration: Learning Outcome

4. Understands Connections

“I understand how my academic choices at name of institution connect to my career and life goals”

Advising Learning Outcomes = Retention Predictors

Advising may be implicated in retention Having a plan to achieve one’s

educational goals Having a significant relationship with

faculty or staff on campus

Retention-Related Learning Outcomes

5. Has Educational Plan“I have a plan to achieve my educational goals”

6. Has Significant Relationship“I have had at least one relationship with a faculty or staff member at name of institution that has had a significant and positive influence on me”

Affective Learning Outcomes

Advising may change students’ values We wanted to measure not only what students

know and can do, but also what they might appreciate or value, as a result of participation in advising

We wanted outcomes that might reflect that students who received quality academic advising benefited from it and thought others might too

Affective Learning Outcomes

7. Values Advisor/Advisee Relationship“It is important to develop an advisor-advisee relationship with someone on campus”

8. Supports Mandatory Advising“There should be mandatory academic advising for students”

What are the Advising Learning Outcomes Measuring?

All eight learning outcomes are measures of students’ meta-cognition

Meta-cognition: What students know about their own knowledge and values

Multi-Institutional Study:Nine study institutions in Oregon

Institution Carnegie Classification

Community Colleges Community College 1 Associate’s/Public/Urban Serving/Multi-campus Community College 2 Associate’s/Public/Rural Serving/Large

Private Universities Private University 1 Master’s (larger programs) Private University 2 Master’s (larger programs)

Public Universities Public University 1 Research University (very high research activity) Public University 2 Research University (very high research activity) Public University 3 Research University (high research activity) Public University 4 Master’s (medium programs) Public University 5 Master’s (small programs)

Method

Online administration of the Inventory of Academic Advising Functions – Student Version

Administered in 2010 or 2011 Students invited to participate:

– Universities: All fully admitted students – Community colleges: All students enrolled in

credit-bearing classes

Method

To ensure that all students in the study had similar educational goals– We selected students at the two

community colleges who indicated that their main reason for attending the college was to earn credit toward a bachelor’s (4-year) degree

Research SampleInstitution

Number of Participants

Participation Rate

Community Colleges

Community College 1 6013 33.7

Community College 2 1159 21.1

Private Universities

Private University 1 437 43.1

Private University 2 1599 52.5

Public Universities

Public University 1 4026 22.1





Total 22,366 26.1

Respondent DemographicCharacteristics

64.5% Female

33.1% New Students (enrolled at their institution for the first time during the academic year in which the survey was administered)

76.6% White

Mean age 25.3 years (SD 8.5 years)

Research Questions

Does advising learning vary as a function of:

Frequency of contact with advisors in the formal advising system?

Source of information students use to choose required classes?

Research Question 1

Does advising learning vary as a function of frequency of contact with advisors in the formal advising system?

Are scores on the 8 advising learning outcomes higher for students who have contacted advisors than for those who have not? Among students who have contacted advisors, are scores higher for those who have more contacts than for those with fewer encounters?

Research Question 1:Formation of Groups

We grouped students based upon their responses to two survey items


1st Survey Item: “Which of the following describes where at name of institution you get your PRIMARY academic advising, that is, the advising you consider most central to your academic progress?”

– Institutional representatives common to all institutions

(e. g., “faculty advisor in my program of study”) – Advising offices unique to each institution

(e. g., “advising center”)– No advising option

“I have not received academic advising from faculty or staff at name of institution”


2nd Survey Item: “On average, how often do you get advice from your primary source of advising, that is, the advising you consider most central to your academic progress?”

– At least once per term– At least twice per year– At least once per year– “I’m not currently getting academic advising from faculty or

staff at name of institution.”


We assigned students to one of three groups:

1.Not advised (n = 3443)(Had not received or were not currently getting advising)

2.Advised occasionally (n = 3538)(Advised at least once per year)

3.Advised frequently (n = 14,886)(Advised at least twice per year or at least once per term)

Measures of Advising Learning Outcomes

8 advising learning outcomes, each measured by a 6 point Likert-type scale– 1 = Strongly Disagree– 6 = Strongly Agree

Research Question 1 To examine the relationship between frequency

of contact (independent variable) and advising learning outcomes (dependent variables)

We used ANCOVA, controlling for – Institution– Institution size– Enrollment status (new vs. continuing)– GPA

I know what requirements I must fulfill in order to meet my educational goals / earn my degree

1=strongly disagree 6=strongly agree

Results of ANCOVA: F(2, 20551) = 195.78, MSE = 1.35, p < .001, η2 = .02

Results of post hoc analysis: All three groups significantly different from each other.

Advised Frequently

Advised Occasionally

Not Advised

Knows Requirements

5.09a (1.11) 4.97b (1.19) 4.56c (1.44)

I understand how things work at name of institution




Advised Frequently


Not Advised

Understands How Things Work

4.59a (1.24) 4.43b (1.31) 4.23c (1.40)

When I have a problem, I know where at name of institution I can go to get help1=strongly disagree 6=strongly agree



Advised Frequently


Not Advised

Knows Resources

4.52a (1.36) 4.02b (1.46) 3.69c (1.59)

I understand how my academic choices at name of institution connect to my career and life

goals 1=strongly disagree 6=strongly agree



Advised Frequently


Not Advised

Understands Connections

5.04a (1.07) 4.83b (1.19) 4.64c (1.29)

I have a plan to achieve my educational goals 1=strongly disagree 6=strongly agree

Results of ANCOVA: F(2, 21418) = 120.75, MSE = .81, p < .001, η2 = .01


Advised Frequently


Not Advised

Has Educational Plan

5.48a (0.84) 5.36b (0.94) 5.21c (1.11)

I have had at least one relationship with a faculty or staff member at name of institution that has had a

significant and positive influence on me1=strongly disagree 6=strongly agree



Advised Frequently


Not Advised

Has Significant Relationship

4.51a (1.51) 4.12b (1.67) 3.79c (1.76)

It is important to develop an advisor-advisee relationship with someone on campus




Advised Frequently


Not Advised

Values Advisor/Advisee Relationship

5.20a (1.01) 4.79b (1.19) 4.42c (1.39)

There should be mandatory academic advising for students




Advised Frequently


Not Advised

Support Mandatory Advising

4.51c (1.47) 4.04b (1.60) 3.89a (1.72)

Frequency of Contact:Summary of Results

Advised Frequently Advised Occasionally Not Advised

Knows Requirements 5.09a (1.11) 4.97b (1.19) 4.56c (1.44)


4.59a (1.24) 4.43b (1.31) 4.23c (1.40)

Knows Resources 4.52a (1.36) 4.02b (1.46) 3.69c (1.59)


5.04a (1.07) 4.83b (1.19) 4.64c (1.29)

Has Educational Plan 5.48a (0.84) 5.36b (0.94) 5.21c (1.11)


4.51a (1.51) 4.12b (1.67) 3.79c (1.76)


5.20a (1.01) 4.79b (1.19) 4.42c (1.39)


4.51a (1.47) 4.04b (1.60) 3.89c (1.72)

Research Question 2

Does advising learning vary as a function of the source of information students use to choose required classes?

Are scores on the 8 advising learning outcomes higher for students who have relied on advisors for help in choosing required classes than for students who have self-advised using official advising materials or advice from informal sources (friends/other students or family members)?

Research Question 2: Formation of Groups

We grouped students based upon their responses to one survey item “Please select the circle that best describes where at name of institution you get most of your information about classes to take to meet degree requirements.”

– institutional representatives and advising offices – institutional tools students might use to self-advise

(“catalog,” “advising website,” “advising guide”)– members of the student’s informal social network

(“friend(s)/other student(s),” “family member(s)”)

Research Question 2: Formation of Groups

We assigned students to one of three groups:

1. Advisor (n = 12,957) (students who selected institutional representatives or advising offices.)

2. Advising tools (n = 7210) (students who selected institutional tools)

– Informal social network (n = 1245) (students who selected “friend(s)/other student(s)” or “family member(s)”)

Research Question 2

To examine the relationship between source of information (independent variable) and advising learning outcomes (dependent variables)

We used ANCOVA, controlling for – Institution– Institution size– Enrollment status (new vs. continuing)– GPA

I know what requirements I must fulfill in order to meet my educational goals / earn my degree




Advisor Advising Tools Informal/Social Network

Knows Requirements

5.09a (1.11) 4.92b (1.25) 4.39c (1.46)

I understand how things work at name of institution






4.61a (1.24) 4.42b (1.31) 3.97c (1.40)

When I have a problem, I know where at name of institution I can go to get help1=strongly disagree 6=strongly agree




Knows Resources

4.54a (1.36) 4.03b (1.49) 3.62c (1.52)

I understand how my academic choices at name of institution connect to my career and life goals






5.60a (1.07) 4.81b (1.19) 4.51c (1.34)

I have a plan to achieve my educational goals 1=strongly disagree 6=strongly agree

Results of ANCOVA: F(2, 21157) = 75.77, MSE = .81, p < .001, η2 = .01



Has Educational Plan

5.48a (0.85) 5.35b (0.95) 5.18c (1.10)

I have had at least one relationship with a faculty or staff member at name of institution that has had a

significant and positive influence on me1=strongly disagree 6=strongly agree





4.51a (1.52) 4.10b (1.67) 3.92c (1.72)

It is important to develop an advisor-advisee relationship with someone on campus






5.20a (1.02) 4.73c (1.26) 4.85b (1.21)

There should be mandatory academic advising for students


Results of ANCOVA:

F(2, 20318) = 174.31, MSE = 2.31, p < .001, η2 = .02




4.52a (1.47) 4.04c (1.63) 4.22b (1.57)

Source of Information:Summary of Results


Knows Requirements 5.09a (1.11) 4.92b (1.25) 4.39c (1.46)


4.61a (1.24) 4.42b (1.31) 3.97c (1.40)

Knows Resources 4.54a (1.36) 4.03b (1.49) 3.62c (1.52)


5.06a (1.07) 4.81b (1.19) 4.51c (1.34)

Has Educational Plan 5.48a (0.85) 5.35b (0.95) 5.18c (1.10)


4.51a (1.52) 4.10b (1.67) 3.92c (1.72)


5.20a (1.02) 4.73c (1.26) 4.85b (1.21)


4.52a (1.47) 4.04c (1.63) 4.22b (1.57)

Summary of Findings Scores on all eight learning outcomes were

significantly higher for students who had been advised than for those who had not

Among students who had seen advisors, scores were significantly higher for those who had more contacts than for those who had fewer encounters

Students who got most of their information about required classes from advisors scored significantly higher on all eight learning outcomes than those who self-advised using advising materials or who relied on advice from family or friends

Implications of Findings

The more advising, the more learning

Institutions need to ensure that all students have frequent contact with advisors

Implications of Findings

The advising relationship matters It can be supplemented, but not

supplanted, by web sites, advising guides, etc.

Institutions need to ensure that students see advisors and have access to quality advising tools

Limitations of the Study

Students were not randomly assigned to frequency of contact or source of information groups

All measures were self-reported

Discussion

Questions

Comments

References Allen, J. M., & Smith, C. L. (2008). Faculty and student perspectives on advising:

Implications for student dissatisfaction. Journal of College Student Development, 49, 609-624.

Cronon, W. (1998). Only connect: The goals of a liberal education. The American Scholar, 67(4), 73-80.

Hemwall, M. K., & Trachte, K. C. (2005). Academic advising as learning: 10 organizing principles. NACADA Journal, 25(2), 74-83.

Light, R. J. (2001). Making the most of college: Students speak their minds. Cambridge, MA: Harvard University Press.

Lowenstein, M. (2005). If advising is teaching, what do advisors teach? NACADA Journal, 25(2), 65-73.

National Academic Advising Association. (2006). NACADA concept of academic advising. Retrieved from www.nacada.ksu.edu/Clearinghouse/AdvisingIssues/Concept-Advising.htm

Smith, C. L., & Allen, J. M. (2006). Essential functions of academic advising: What students want and get. NACADA Journal, 26(1), 56-66.

cathleen l. smith, ph.d. professor emerita of psychology portland state university

Documents