a competency-based approach to the master’s …journal of case studies in accreditation and...

Journal of Case Studies in Accreditation and Assessment Volume 4 – December, 2014

A competency-based approach, page 1

A competency-based approach to the master’s degree preparation of

higher education professionals

Molly Ott

Arizona State University

Evelyn Baca


Jesus Cisneros


Evan Bates

Michigan State University

ABSTRACT

The purpose of this manuscript is to describe a competency-based approach to designing

and assessing master’s level professional preparation programs in the field of higher education

administration. Given the absence of a universal set of competencies defined for HEA master’s

degree programs, the authors draw from the CAHEP (2010) and Wright (2007) general

guidelines for HEA graduate programs, and the literature pertaining to graduate competencies

and outcomes that are specific to early career student affairs positions. The competency-based

approach described here contributes to the literature on competency-based assessment models for

master’s level graduate programs, and has utility for faculty and administrators seeking to assess

professional master’s programs, especially those that lack nationally defined standards or

certification.

Keywords: higher education administration, competency-based assessment, graduate programs

Copyright statement: Authors retain the copyright to the manuscripts published in AABRI

journals. Please see the AABRI Copyright Policy at http://www.aabri.com/copyright.html.

http://www.aabri.com/copyright.html



While the quality of undergraduate education in the United States is the source of

substantial debate among policymakers, scholars, and the general public (e.g., Arum & Roksa,

2011), master’s degrees receive considerably less attention. Yet their popularity is on the rise.

In 1987, U.S. universities awarded 290,000 master’s degrees, a number that more than doubled

to 670,000 by 2009 (Jaquette, 2011). Although over two fifths of contemporary undergraduates

go on to receive master’s degrees (Jaquette, 2011), whether these degree programs effectively

prepare students for success in their fields is an open question.

Ascertaining master’s program quality and associated student outcomes is critical,

especially given increasing external pressures for colleges and universities to be accountable for

all students’ success (Palomba & Banta, 2001). Yet the vast majority of models to assess

academic programs focus on the undergraduate level. Maki and Brokowski (2006) recently drew

attention to the need to better document doctoral student learning and career outcomes, but

despite the expanding enrollments in master’s degree programs, consideration of how to define

and document student success at this level has been comparatively scant (Delaney, 1998).

Professional fields, which include business, social work, engineering, journalism, public

administration, and education, represent the vast majority of master’s degrees (Conrad et al.,

1998; Conrad, Haworth, & Millar, 1993; Sun, 2004). Some institutions assess graduate students

using common standards, regardless of field (e.g., Khan, Khalsa, Klose, & Cooksey, 2012).

Many experts, however, contend the academic experiences and expected outcomes of graduate

students are heterogeneous, and useful assessment processes must be attentive to field or

disciplinary differences (Dougan, 1996; Dunbar et al., 2006; Nesheim et al., 2006)

This manuscript focuses on professional master’s degree programs in higher education.

Education programs are lightening rods for public critique (Borko, Liston, & Whitcomb, 2006).

For instance, a 2009 Center for American Progress policy brief characterized master’s programs

in education schools as “a notoriously unfocused and process-dominated course of study,”

lacking in quality and true impact that results in improved professional practice (Roza & Miller,

2009, p. 1). While not substantiated empirically, this sentiment reflects the need for education

graduate schools to assess students’ outcomes associated with master’s degree programs to

demonstrate efficacy, as well as identify areas needing improvement. The purpose of this

manuscript, therefore, is to describe a competency-based approach to designing and assessing

master’s level professional preparation programs in the field of higher education administration.

COMPETENCY-BASED APPROACH TO PROFESSIONAL EDUCATION

In contrast to the liberal arts and social sciences (Voorhees, 2001), professional academic

fields are characterized by “specialized competence, acquired as the result of intellectual

training” (Carr-Saunders & Wilson, 1933, p. 307). Graduate professional schools are charged to

“inculcate knowledge of the theory and practice so that the candidates of the profession are

sufficiently competent for practice in the respective field” (Sun, 2004, p. 6). In the professions,

this notion of competency is a central principle to guide academic program development, as well

as the evaluation of student learning and success.

Competencies are defined as the “integration of skills, abilities, and knowledge as

focused on a particular task” (U.S. Department of Education, 2001, p. 1) and are easily measured

(Voorhees, 2001). Although the terms “competency” and “outcome” are often used

interchangeably (Banta, 2001) and some argue that no real differences exist between outcome-

versus competency-based education in practice (e.g., Morke, Dornan, & Eika, 2013), the model



described here focuses on competencies for three reasons. First, in higher education, student

outcomes often also encompass retention, graduation, and placement rates, which do not

necessarily reflect mastery of skills or knowledge (Voorhees, 2001). Second, “competency”

implies expertise that is directly transferable to a specific employment field (Banta, 2001), which

is why it lends itself especially well to professional graduate programs (Bilder & Conrad, 1996).

Finally, curriculum design and assessment can be integrated with one another around

competencies, creating a coherent and consistent language from which teachers and learners can

work (Voorhees, 2001).

Implementing a competency-based approach to professional academic programs begins

by conceptually defining intended competencies. These definitions should be specific enough to

facilitate assessment (Voorhees, 2001). Often, competencies are designed around standards

identified by national professional organizations, such as the Council of Social Work Education

for Master’s in Social Work degrees (Meyer-Adams, Potts, Koob, Dorsey, & Rosales, 2011), the

American Society for Training & Development for Master’s in Workforce Training and

Development degrees (Gaudent, Annulis & Kmiec, 2008), and the Council for the Advancement

of Standards (CAS) in Higher Education for Master’s in Student Affairs degrees (Kuk &

Banning, 2009). For fields lacking national guidelines, however, intended outcomes and

competencies are less prescribed. Also, while such external guidelines provide helpful

benchmarks, they are not necessarily ongoing or fully reflective of an individual program’s own

goals, as well as those set forth by the department, college, and institution. Program faculty and

other stakeholders must therefore draw from external, as well as internal resources to reach

consensus about the competencies that graduates should possess (Voorhees, 2001).

After defining competencies, the next step is to determine how they will be developed in

learners. Faculty should map them onto courses and associated learning activities, and ensure

that academic content is aligned with the competencies (Bers, 2001). Identifying how

competencies will be measured is critical. Professional graduate degree program administrators

and faculty often rely on external evaluation metrics (e.g., rankings, accreditation, program

reviews, certification or licensure examinations) (Funk & Klomparens, 2006). In addition,

programs (especially those in fields without nationally defined competency standards and

associated measures) often rely on internally defined competencies and evaluations to collect

data pertaining to the types of competencies described above; that is, student, alumni, and/or

faculty self-reported perceptions of learning (e.g., Delaney, 1997; Gaudet, Annulis, & Kmiec,

2008; Meyers-Adams et al., 2011) and/or program quality (e.g., Delaney, 1997; Ketefian &

Hagerty, 1987). While self-reports promote reflective practice (Walser, 2009), many students

are not aware of the full benefits associated with their graduate degree experience until after

graduation (Bilder & Conrad, 1996). For this reason, supplementing perceptions and satisfaction

with direct measures of competency is essential to comprehensive program evaluation (Maki,

2001).

The peer-reviewed scholarship pertaining to designing and implementing competency-

based assessment models for master’s level graduate programs, especially those that focus on

professional practice (e.g., social work, education, and nursing), is limited (Kaylor & Johnson,

1994). The competency-based approach described here contributes to this gap and has utility for

faculty and administrators seeking to assess professional master’s programs, especially those that

lack nationally defined standards or certification. The remaining discussion is organized as

follows: after a brief explanation of the field of Higher Education Administration, the authors

describe their specific program, how they developed expected competencies for the program,



identified where the competencies would be addressed in the curriculum, and designed methods

to measure the competencies. The conclusion emphasizes the importance of communicating

with internal and external stakeholders about the model, with an aim towards developing a

competency-based program culture.

HIGHER EDUCATION ADMINISTRATION GRADUATE EDUCATION

The first degree program in Higher Education was implemented at Clark University in

1893 (see Goodchild, 2013 for history of the field). No external accrediting body exists for the

field, but today there are approximately 161 universities in the U.S. that offer a doctoral and/or

master’s degree in Higher Education Administration (Jensen, 2013). Broadly speaking, Higher

Education Administration (HEA) is a specialized field of study that is “concerned with the

behavioral interaction of students, faculty, and administrators within the context of a college or

university environment, and the interrelationship of this environment with the larger society”

(Wright, 2007, p. 19). Typically, coursework encompasses the history and philosophy of higher

education, administration and leadership, finance, law, policy, and organizational

change/development (CAHEP, 2010; Hyle & Goodchild, 2013). Master’s degree graduates enter

a range of positions within two- and four-year colleges and universities, including admissions,

orientation, advising, human resources, financial aid, intercollegiate athletics, and student affairs,

as well as other for-profit and not-for-profit organizations with higher education-related foci.

Beginning in 1964, what is known as the Council for the Advancement of Standards

(CAS) in Higher Education established guidelines specific to masters-level college student

personnel and student affairs professional preparation programs (COSPA, 1964).1 While the

outcomes associated with these standards have been studied extensively (e.g., Burkard, Cole, Ott

& Stoflet, 2004; Cuyjet, Longwell-Grice, & Molina, 2009; Herdlein, 2004; Kretovics, 2002;

Richmond & Sherman, 1991; Waple, 2006), including surveys of faculty about their graduates’

outcomes (e.g., Dickerson, Hoffman, Anan, Brown, Vong, Bresciani, Monzon, & Oyler, 2011;

Herdlein, Kline, Boquard, & Haddad, 2010; Kuk, Cobb, & Forrest, 2007), there has been little

empirical consideration of the application and use of these standards within master’s degree

programs themselves (Creamer, 2003). DiRamio (2013) sampled 44 student affairs graduate

programs and observed less than half advertised on their websites that they used the CAS

standards. After reviewing syllabi, he found even fewer (29%) explicitly identified and used

CAS in course designs.

Even less attention has been given to systematically defining or studying the

competencies associated with master’s level HEA degree programs as distinct from student

affairs professional programs. Wright (2007) proposed a set of guidelines to define the HEA

knowledgebase. Based on Bloom’s Taxonomy of Educational Objectives, she organized

competencies into three domains: conceptual skills, (i.e., decision-making, research and

1 An important clarification is that Higher Education Administration programs are distinct from

Educational Administration programs designed to prepare primary and secondary level

principals, superintendents, and other K-12 leaders. Another more nuanced – and underspecified

(Wright, 2007) – distinction within the universe of education graduate degree programs is made

between those that emphasize postsecondary administration (which is the focus of this

manuscript) and those that are college student affairs specific (CAS, 2006) or community college

specific (CAHEP, 2010).



evaluation, resource allocation behavior, entrepreneurial behavior, and introspective behavior),

human relations skills (i.e., leadership, problem solving, conflict resolution behavior, and self-

development), and technical skills (i.e., fiscal management, communication skills, planning, and

technology). Wright’s taxonomy does not distinguish between competencies expected of

master’s versus doctoral level graduates, and she acknowledged, “much work remains to be done

in terms of identifying requisite skills and competencies expected of the effective higher

education administrator, manager, and leader; related indicators of performance; and outcomes

evaluation” (p. 30). In 2010, the Council for the Advancement of Higher Education Programs

(CAHEP) released a set of baseline recommendations for master’s programs in Higher Education

Administration, (CAHEP, 2010). While they propose general curricular content and that

programs should define outcomes that are consistent with their visions/missions, CAHEP’s

guidelines do not articulate specific competencies. To date, no model for programs to implement

these guidelines, define competencies at the HEA master’s degree level, or suggest strategies to

assess competencies has been published.

CASE BACKGROUND

Arizona State University’s (ASU) master’s degree program in Higher and Postsecondary

Education (HED) is part of the Mary Lou Fulton Teachers College, the top-ranked graduate

school of education in Arizona and 18th

nationally (U.S. News & World Report, 2014). The

HED program was established in 1964 and over the course of its history has awarded Ed.S.,

Ph.D., M.A., and Ed.D. degrees. Currently, the program offers an M.Ed. degree only.

Enrollment-wise, ASU is among the largest face-to-face master’s programs in the HEA field,2

with approximately 140 degree-seeking students. The M.Ed. consists of 30 credits, and most

students complete the degree in one and a half years.

COMPETENCIES OF HIGHER EDUCATION ADMINISTRATION MASTER’S

DEGREE GRADUATES

While the best practice is to align an assessment program with standards set by

professional or disciplinary associations (Rivas et al., 2010), as discussed above, HEA does not

have a universal set of competencies defined for master’s degree programs. Its faculty therefore

drew from several sources to define those for their program: the CAHEP (2010) and Wright

(2007) general guidelines for HEA graduate programs, and the literature pertaining to graduate

competencies and outcomes that are specific to early career student affairs positions (see

Herdlein, Riefler, & Mrowka, 2013 for a recent meta-analysis of this literature). In this research,

competencies are commonly described from the perspective of employer representatives such as

chief student affairs officers or published position descriptions (Burkard et al., 2004; Herdlein,

2004; Hoffman & Bresciani, 2012) or based on the opinions of faculty (Arellano & Martinez,

2009; Dickerson et al., 2011; Herdlein, Kline, Boquard, & Haddad, 2010; Kuk, Cobb, & Forrest,

2007).

2 According to the authors’ analyses of 2011-12 IPEDS completion data, the largest two

programs are Drexel University (131 degrees) and Capella University (88 degrees). Both are

online master’s degree programs. Arizona State University is a face-to-face program and was the

third largest in 2011-12 completions, awarding 73 degrees.



Consistent with the suggestion of Maki (2001) and others (Bresciani, 2011; CAHEP,

2010), the authors tailored the competencies to ensure they were directly aligned with the

program’s mission/purpose statement. Similar to other professional fields, they equally

emphasized acquisition and application (Delaney, 1997) and organized the competencies

according to the knowledge and skills that successful higher education practitioners should

demonstrate at the conclusion of their master’s degree, as indicated in Figure 1. (Appendix A.).

METHODS AND CRITERIA TO ASSESS COMPETENCIES

This assessment approach blends direct and indirect methods. Direct assessment involves

experts reviewing artifacts produced by students (e.g., exams, papers, portfolios, projects, and

presentations) to determine whether they demonstrate mastery of competencies, while indirect

assessments capture students’ self-reported perceptions of their skills, knowledge, and learning

experiences (e.g., surveys, focus groups, and exit interviews) (Maki, 2010).

Direct Methods of Assessment

Arizona State University’s master’s program curriculum is divided into a core of twelve

required credits and a menu of structured electives from which students may choose the

remaining eighteen credits. The authors mapped the competencies onto the requirements,

ensuring that the main course assignments (as indicated in Figure 2., Appendix A.) reflected the

identified knowledge and skills.

The authors developed three rubrics to directly assess how well students’ three core

assignments demonstrated the competencies (Mertler, 2001). The rubrics rely on a non-linear

design to assess the extent to which student performance is related to multiple broad standards or

competencies. They were designed to be criterion referenced, such that level of competence is

not evaluated according to fellow students’ performance (i.e., norm-referenced) but instead

reflect the individual’s independent performance according to standards set by the program

leadership (Mager, 1997). All components of the rubric measure the objective of a specialized

knowledge or skill associated with the core curriculum (i.e., content validity), and clear

distinctions are made between performance levels for each of the criteria being assessed to help

ensure accurate, consistent, and fair assessment. The rubrics are provided in Appendix B.

Indirect Methods of Assessment

Many institutions use commercially developed indirect assessment tools, such as the

National Survey of Student Engagement, or internally developed instruments like course

evaluations or exit surveys (Hogan, Lusher & Mondal, 2012). These measures do not typically

focus on field-specific competencies, so the authors created a survey instrument tailored to the

program’s needs. The instrument design began with a series of focus groups with students and

alumni to better ascertain their experiences in the program, generally, as well as identify possible

competencies not initially identified through the literature review and faculty discussions. The

multiple iterations of the instrument, which included student feedback and pilot testing,

ultimately produced a four part online survey that sought information from students on their



perceptions of competencies, as well as professional experiences, career plans and aspirations,

and demographics. The items pertaining to competencies are listed in Appendix C.

The authors developed two versions: a pre-test for incoming students and a post-test for

graduating students. Students are invited to participate in the pre-survey prior to the start of their

first semester in the program. Likewise, graduating students are asked to complete the post-

survey at the conclusion of their final semester of the program. Program faculty then compare

the corresponding responses to determine whether significant changes in self-perceived

competency occurred.

Creating a Competency-Based Culture: The Importance of Communication

In some professions, including HEA, no separate accrediting agency or certification body

exists to define expected student outcomes associated with master’s degrees. Creating a culture

that values defining and assessing competencies is nonetheless imperative for programs in these

fields, especially given public attention to hold postsecondary education writ large more

accountable. To promote a competency-based culture, implementing the type of assessment plan

described here should be accompanied by strategies to communicate with external as well as

internal stakeholders (Bers, 2001).

External audiences with an interest in program competencies include prospective

students, employers, policymakers, and the general public. To inform these groups, the ASU

HEA program’s promotional materials and public website includes an explanation of the

competencies. When recommending students for jobs in conversations with employers or in

written letters, faculty and program administrators describe the competency model and the types

of skills and knowledge that graduates possess. The program also provides a summary of

assessment results in annual reports that are publicly available. Strategies to communicate with

internal stakeholders are similarly important. Faculty are stewards of student learning who

should have ongoing input in how competencies frame the curriculum. At the beginning of each

academic year, the ASU HEA master’s program convenes a meeting of all tenure and non-tenure

line faculty to solicit feedback on how well the competencies align with their individual courses

and whether adjustments are necessary. Being able to define, explain, and demonstrate

competencies is valuable for students as they seek employment. In the ASU HEA master’s

program, faculty socialize their students to this model early. Prospective students are provided

with an overview of the competencies in information sessions and program materials. The

required new student orientation includes a discussion of what the program competencies are,

how they are developed and assessed across coursework, and how they connect to career success

post-graduation. All new students subsequently are asked to complete the pre-program survey

(one of the indirect assessment measures). They are enrolled in the Introduction to Higher

Education course in their first semester, where the final project and its associated rubric are

aligned with the program competencies.

The competency-based approach described here offers a template for higher education

administration master’s degree programs, as well as those in other fields that lack nationally set

standards. Expanding enrollments, coupled with intense calls for accountability at all levels of

higher education, require master’s program faculty and administrators to ensure that students

possess demonstrable knowledge, skills, and abilities that will lead to career success.



REFERENCES

Arellano, E., & Martinez, M. (2009). Does educational preparation match professional practice:

The case of higher education policy analysts. Innovative Higher Education, 34(2), 105-

116.

Arum, R., & Roksa, J. (2011). Academically adrift: Limited learning on college campuses.

Chicago, IL: University of Chicago Press.

Banta, T. (2001). Assessing competence in higher education. In C. Palomba & T. Banta,

Assessing student competence in accredited disciplines: Pioneering approaches to

assessment in higher education (pp. 1-12). Sterling, VA: Stylus Publishing.

Bers, T. (2001). Measuring and reporting competencies. New Directions for Institutional

Research, 110, 29-40.

Borko, H., Liston, D., & Whitcomb, J. (2006). A conversation of many voices: Critiques and

visions of teacher education. Journal of Teacher Education, 57(3), 199-204.

Bresciani, M. (2011). Identifying barriers in implementing outcomes-based assessment program

review: A grounded theory analysis. Research and Practice in Assessment, 6, 5-16.

Burkard, A., Cole D.C., Ott, M., & Stoflet, T. (2004). Entry-Level competencies of new student

affairs professionals: A Delphi Study. NASPA Journal, 42(3), 283-309.

Conrad, C.G., Haworth, J.G., & Millar, S.M. (1993). A Silent Success: Master’s Education in the

United States. Baltimore, MD: John Hopkins University Press.

Conrad, C.F., Duren, K.M., & Haworth, J.G. (1998). Students perspectives on their master’s

degree experiences: Disturbing the conventional wisdom. New Directions for Higher

Education, 101, 65-76.

Council of Student Personnel Associations (COSPA). (1964). A proposal for professional

preparation in college student personnel work. Unpublished manuscript, Indianapolis:

Author.

Council for the Advancement of Academic Standards (CAS). (2006). CAS professional

standards for higher education (6th ed.). Washington, DC: Author.

Council for the Advancement of Higher Education Programs (CAHEP). (2010). A commitment to

quality: Guidelines for higher education administration and leadership preparation

programs at the masters degree level. Las Vegas, NV: Association for the Study of

Higher Education.

Cuyjet, M.J., Longwell-Grice, R., & Molina, E. (2009). Perceptions of new student affairs

professional and their supervisors regarding the application of competencies learned in

preparation programs. Journal of College Student Development, 50(1), 104-119.

Delaney, A. M. (1997). Quality Assessment of professional degree programs. Research in

Higher Education, 38(2), 241-264.

Dickerson, A., Hoffman, J., Anana, B., Brown, K., Vong, L., Bresciani, M., Monzon, R. &

Oyler, J. (2011). A comparison of senior student affairs officer and student affairs

preparatory program faculty expectations of entry-level professionals’ competencies.

Journal of Student Affairs Research Practice, 48(4), 463-479.

DiRamio, D. (2013). Professional Competencies and Standards of Practice: The Student Affairs

Perspective From the Student Affairs Administrators in Higher Education, College

Student Educators International, and Council for the Advancement of Standards in

Higher Education. In S. Freeman, L. Hagedorn, L. Goodchild & D. Wright (Eds.),



Advancing higher education as a field of study: In quest of doctoral degree guidelines

(pp. 77-92). Sterling, VA: Stylus.

Dougan, A. M. (1996). Student assessment by portfolio: One institution’s journey. The History

Teacher, 29, 171-178.

Dunbar, N., Brooks, C., & Kubicka-Miller, T. (2006). Oral communication skills in higher

education: Using a performance-based evaluation rubric to assess communication skills.

Innovative Higher Education, 31(2), 115-128.

Funk, K., & Klomparens, K. L. (2006). Using the assessment process to improve doctoral

programs. In P. Maki & N. Borkowski (Eds.), The assessment of doctoral education:

Emerging criteria and new models for improving outcomes (pp.145-162). Sterling, VA:

Stylus Publishing.

Gaudet, C. H., Annulis, H. M., & Kmiec, J. J. (2008). Building an evaluation framework for a

competency-based graduate program at the University of Southern Mississippi.

Performance Improvement, 47(1), 26-36.

Goodchild, L. (2013). Higher education as a field of study: Its history, degree programs,

associations, and national guidelines. In S. Freeman, L. Hagedorn, L. Goodchild & D.

Wright (Eds), Advancing higher education as a field of study: In quest of doctoral degree

guidelines (pp. 13-50). Sterling, VA: Stylus.

Herdlein, R. J. (2004). Survey of chief student affairs officers regarding relevance of graduate

preparation of new professionals. NASPA Journal, 42(1), 51-71.

Herdlein, R. J., Riefler, L., & Mrowka, K. (2013). An integrative literature review of student

affairs competencies: A meta-analysis. Journal of Student Affairs Research and Practice,

502(3), 250-269.

Herdlein, R., Kline, K., Boquard, B., & Haddad, V. (2010). A survey of faculty perceptions of

learning outcomes in master’s level programs in higher education and student affairs.

College Student Affairs Journal, 29(1), 33-45.

Hogan, E., Lusher, A., & Mondal, S. (2012). Development of an instrument for indirect

assessment of college business programs. Journal of Case Studies in Accreditation and

Assessment, 2. Retrieved from http://www.aabri.com/manuscripts/11999.pdf

Hyle, A., & Goodchild, A. (2013). Contemporary condition of higher education programs in the

United States and Canada: An initial 2012 report. In S. Freeman, L. Hagedorn, L.

Goodchild & D. Wright (Eds.), Advancing higher education as a field of study: In quest

of doctoral degree guidelines (pp. 51-76). Sterling, VA: Stylus.

Jaquette, O. (2011). In pursuit of revenue and prestige: The adoption and production of master's

degrees by U.S. colleges and universities. (PhD Dissertation), Center for the Study of

Higher and Postsecondary Education, School of Education, University of Michigan, Ann

Arbor.

Jensen, D. (2013). A demographic analysis of graduate programs in higher education

administration in the United States. Academy of Educational Leadership Journal, 17(3),

1-19.

Ketefian, S., & Hagerty, B.K. (1987). Putting master’s programs to the test. Nursing & Health

Care, 8(9), 502-507.

Khan, R., Khalsa, D., Klose, K., & Cooksey, Y. (2012). Assessing graduate student learning in

four competencies: Use of a common assignment and a combined rubric. Research and

Practice in Assessment, 7, 29-41.

Kretovics, M. (2002). Entry-level competencies: What student affairs administrators consider



when screening candidates. Journal of College Student Development, 43, 912-920.

Kuk, L., & Banning, J. (2009). Student affairs preparation programs: A competency based

approach to assessment and outcomes. College Student Journal, 43(2), 492-453.

Kuk, L., Cobb, B., & Forrest, C. (2007). Perceptions of competencies of entry-level practitioners

in student affairs. NASPA Journal, 44(4), 664-691.

Mager, R. F. (1997). Measuring instructional results or got a match? (3rd ed.). Atlanta, GA: The

Center for Effective Performance, Inc.

Maki, P. (2010). Assessing for learning: Building a sustainable commitment across the

institution. Sterling, VA: Stylus.

Maki, P., & Borkowski, N. (2006). The assessment of doctoral education: Emerging criteria and

new models for improving outcomes. Sterling, VA: Stylus Publishing.

Mertler, C. (2001). Designing scoring rubrics for your classroom. Practical Assessment,

Research & Evaluation, 7(25). Retrieved from

http://PAREonline.net/getvn.asp?v=7&n=25

Morcke, A., Dornan, T., & Eika, B. (2013). Outcome (competency) based education: An

exploration of its origins, theoretical basis, and empirical evidence. Advances in Health

Science Education, 18(4), 851-863.

Myers-Adams, N. Potts, M. K., Koob, J. J. Dorsey, C. J., & Rosales, A. M. (2011). How to

tackle the shift of educational assessment from learning outcomes to competencies: One

program’s transition.

Palomba, C., & Banta, T. (2001). Assessing student competence in accredited disciplines:

Pioneering approaches to assessment in higher education. Sterling, VA: Stylus

Publishing.

Richmond, J., & Sherman, K. J. (1991). Student development preparation and placement: A

longitudinal study of graduate students’ and new professionals’ experiences. Journal of

College Student Development, 32, 8-16.

Roza, M., & Miller, R. (2009). Separation of degrees: state-by-state analysis of teacher

compensation for master’s degrees. Center for American Progress. Retrieved from

http://www.americanprogress.org/wp-

content/uploads/issues/2009/07/pdf/masters_degrees.pdf

Sun, J. B. (2004). Professional schools: research and assessment involving multiple

constituencies. New Directions for Institutional Research, 124, 5-29.

Nesheim, B., Guentzel, M., Ganswemer-Topf, A., Ross, L., & Turrentine, C. (2006). If you want

to know, ask: Assessing the needs and experiences of graduate students. New Directions

for Student Services, 115, 5-18.

Voorhees, R. (2001). Competency-based learning models: A necessary future. New Directions

for Institutional Research, 110, 5-13.

Walser, T.M. (2009). An action research study of student self-assessment in higher education.

Innovative Higher Education, 34, 299-306.

Waple, J.N. (2006). An assessment of skills and comptencies for entry-level student affairs work.

NASPA Journal, 43(1), 1-18.

Wright, D. (2007). Progress in the development of higher education as a specialized field of

study. In D. Wright & M.T. Miller (Eds.) Training Higher Education Policy Makers and

Leaders: A Graduate Program Perspective p.19-34. Charlotte, NC: Information Age

Publishing.

http://pareonline.net/getvn.asp?v=7&n=25

http://www.americanprogress.org/wp-content/uploads/issues/2009/07/pdf/masters_degrees.pdf

http://www.americanprogress.org/wp-content/uploads/issues/2009/07/pdf/masters_degrees.pdf



APPENDIX A.

Figure 1. Master’s Degree in Higher & Postsecondary Education Core Competencies

Specialized Knowledge Associated with Core Curriculum

K1. Mission and Values of Higher Education: Analyze the mission, purpose, and goals of higher

education and how administration helps to advance institutional and enterprise-wide

outcomes (Wright, 2007).

K2. History of Higher Education: Demonstrate mastery of the historical foundations of

American higher education and explain how historical influences can be observed in

administrative practice today (CAHEP, 2010; Herdlein et al., 2013; Herdlein et al., 2010;

Wright, 2007).

K3. Higher Education Stakeholders: Understand the roles that faculty, administrators and staff,

and students play in the functioning of American colleges and universities, and analyze

contemporary issues confronting each of these stakeholder groups (Herdlein et al., 2013).

K4. Higher Education Administrative Theory: Recognize, explain, and apply the key theoretical

frameworks that guide administrative practice in higher education (CAHEP, 2009; Herdlein

et al., 2013).

K5. Ethics of Practice: Recognize, explain, and apply the ethical codes of conduct that guide

administrative practice in higher education (Herdlein et al., 2013; Herdlein et al., 2010).

K6. Methods of Inquiry: Demonstrate knowledge of action-based methods of inquiry and how to

use these to design best-practice, innovative solutions in higher education administrative

practice (Herdlein et al., 2013; Herdlein et al., 2010; Wright, 2007).

Specialized Skills and Abilities Associated with Core Curriculum

SA1. Data-driven decision-making and problem-solving: Ability to identify problems of

practice within a higher education functional unit (e.g., department, office) and analyze

available data as well as action-based methods of inquiry to implement best-practice,

creative solutions (Burkard et al., 2004; Herdlein et al., 2013; Wright, 2007).

SA2. Innovation: Ability to be innovative in administrative practice, applying new technology

and ideas to success in the profession (Wright, 2007).

SA3. Communication: Demonstrate effective spoken and written communication skills, as

well as listening skills, that attend to a variety of audiences, including college students,

faculty, and administrators (Burkard et al., 2004; Herdlein et al., 2013; Herdlein et al., 2010;

Wright, 2007).

SA4. Collaboration: Ability to collaborate across functional lines and diverse backgrounds, as

well as cultivate professional relationships across institutions and organizations (Burkard et

al., 2004; Herdlein et al., 2013; Herdlein et al., 2010).

SA5. Reflective practice: Ability to engage in critical self-reflection and commit to ongoing

professional improvement (Herdlein et al., 2013; Herdlein, 2004; Wright, 2007).



Figure 2. Core Assignments for Direct Assessment of Competencies

Course Title Artifact Description

Introduction to

Higher

Education

Higher

Education

Presentation

This project traces the history, present status, and

future trends pertaining to (a) an important current or

emerging topic in higher education, or (b) a practice-

based area of higher education that they aspire to or

currently work in. It takes the form of a 20-30 minute

presentation recorded and uploaded to the internet,

requiring students to innovatively incorporate the use

of technology.

Practicum Practicum

Placement

Reflection Paper

This is a substantive paper that is reflective of content

learned during structured practical experience in a

college or university setting, supervised by practitioner

and/or faculty member with whom the student works

closely. The Reflection Paper describes each

individual’s journey through the experience and how it

affected his/her professional identity development.

Applied Inquiry

& Project

Action-Oriented

Research Project

This project is a small-scale applied study performed in

a higher education setting. The student either: 1– has an

opportunity to initiate a small change (the action) and

evaluate the immediate local consequences (the

research), or 2– has an opportunity to investigate a

practice-based problem (the research) and suggests

specific opportunities for practitioners to ameliorate the

empirically defined issues (the action). The results of

the Applied Project are documented through a research

paper (similar to a professional conference paper),

which describes the setting, the need, the action taken,

and the short-term results of the study, as well as a

poster presentation.



APPENDIX B.

Core Project Rubrics

Core Project #1: Higher Education Presentation

Content &

Associated

Competency

1 - Inadequate 2 - Acceptable 3 - Excellent

Introduction

Provides

introduction to

topic/ practice-

based area

K3, SA3

Has basic understanding of

the topic/ practice-base area

Describes the topic/

practice-base area with

some comprehension of

level of complexity

Describes the topic/

practice-base area with a

full understanding of level

of complexity

Historical

Connections

Traces the history

(past and present)

of the topic/

practice-base area

K2

Recognizes basic historical

elements pertinent to the

topic/ practice-base area

Provides some evidence of

understanding patterns,

cause/effect relationships

and critical indicators

related to the topic/

practice-base area

Understands and articulates

patterns, cause/effect

relationships and critical

indicators related to the

topic/ practice-base area

Literature

Review

Incorporates and

addresses relevant

connections to key

sources.

SA1

Makes relevant connections

to a convenient sample of

sources, however the

sources presented are

notably incomplete or

lacking key information

Makes relevant

connections to various

sources, however the

presentation of pertinent

information is missing

some key aspects

Makes relevant

connections to a

comprehensive collection

of key sources

Originality and

Innovation

Identifies potential

future trends of

the topic/ practice-

base area

K2, SA2

Minimally identifies

implications,

recommendations, and

future trends. Ideas do not

evidence originality of

thought

Is able to predict and

evaluate a limited scope of

implications,

recommendations and

future trends. Ideas

evidence original thinking

but may not be fully

developed or clearly

articulated


evaluate a broad range of

implications,

recommendations and

future trends. Impressive

originality of ideas

Conclusion

Provides synthesis

and summary to

conclude

presentation.

SA3

Provides minimal

conclusive synthesis and

summary of gathered

information

Summarizes key aspects of

topic and provides some

conclusive synthesis of

gathered information

Fully synthesizes gathered

information and provides

well-developed summary

and conclusions

Organization &

Delivery

Organizes and

delivers

presentation

effectively.

Notable absence of

organization and lack of a

logical sequence of clearly

expressed ideas. Student

fails to capture and

maintain audience attention

Some lack of organization

and/or occasional lack of

logical sequence of ideas;

student does not

consistently capture

audience attention

Presentation is well

organized and follows a

logical sequence of ideas

throughout; student

establishes and maintains

audience attention



SA3 throughout presentation throughout

Use of

Technology

Uses appropriate

and creative

audiovisuals

(A/V)

SA2

A/V lacks evidence of

preparation and creativity

A/V shows some

preparation and creativity

A/V is effective and

carefully prepared,

creativity evidenced

Instructor Comments:

______________________________________________________________________________

______________________________________________________________________________



Core Project #2: Practicum Placement Reflection Paper

Content &

Associated

Competency


Stakeholder

Analysis

Understanding of

practicum

placement

K3

Has basic recognition of

organizational structure

and stakeholders with

minimal evidence of

higher level

understanding

Recognizes organizational

structure and stakeholders

with some comprehension

of level of associated

complexities

Describes organizational

structure and stakeholders

with a deep and nuanced

understanding of

associated complexities

Higher

Education

Theory &

Practice

Makes

connections to

program content

within higher

education

practice.

K1, K4, K5

Response is incomplete,

lacking in connections to

important ideas, concepts,

and previous knowledge

of issue pertaining to

higher education theory

and practice

Response is adequate,

with some meaningful

connections to important

ideas and concepts.

Student response alludes

to previous knowledge or

larger issues pertaining to

higher education theory

and practice

Response is thorough with

meaningful connections to

important ideas and

concepts. Student

integrates previous

knowledge and connects

to larger issues pertaining

to higher education

Reflective

practice

Engages in self-

assessment

SA5, K5

Response provides

information about how

understanding has

changed, but lacks

examples or comparison

Response describes, with

some examples or

comparisons how

understanding has

changed

Response describes, with

specific detail and

examples and

comparisons how

understanding has

changed

Professional

Development

Commits to

ongoing

professional

development

SA4, SA5

Response alludes to

questions or topics for

future learning. However,

student does not provide

in-depth reflections on

collaborative experiences,

work environment, or the

cultivation of professional

relationships

Response mentions new

topics or questions for

future learning. Student

generally reflects on

collaborative experiences,

work environment, and

the cultivation of

professional relationships

Response raises important

questions for further

exploration, learning, or

understanding. Student

response shows evidence

of ability to collaborate

with others and cultivate

professional relationships.

Critical analysis of work

environment evidenced

APA Style

Uses APA style

(SA1)

Frequent lack of

appropriate APA

formatting

Occasional lapses in use

of APA format

Consistent use of APA

format


______________________________________________________________________________

______________________________________________________________________________



Core Project #3: Action-Oriented Research Project

Content &

Associated

Competency


Research Problem

Identifies a problem,

opportunity or

challenge within the

research literature.

SA1

Has basic recognition of a

problem, opportunity, or

challenge with minimal

detail and understanding

Recognizes a problem,

opportunity or challenge

with some comprehension

of level of complexity

Describes the problem and

all component pieces with

a full understanding of

level of complexity

Theoretical

Framework

Conceptually frames

the research problem

within an appropriate

theoretical

framework

K4

Minimally incorporates

theoretical framework in

presentation of research

problem; conceptual

framing lacks

comprehensiveness and

connection to overall paper

(data analysis, discussion,

and conclusions)

Incorporates theoretical

framework in presentation

of research problem;

appropriate application

conceptual framing with

some details and areas of

connection in need of

further development

Appropriately applies

theoretical framework to

research problem; fully

develops conceptual

framing with clear

connections to overall

paper (data analysis,

discussion, and

conclusions)

Critical Analysis

Analyzes the

elements of a

specific research

problem

SA1

Recognizes elements

pertinent to the research

problem, however there are

frequent lapses in critical

thinking skills (analysis,

synthesis, evaluation,

problem solving skills, able

to support opinions and

justify choices)

Provides some evidence of

understanding patterns,



related to the research

problem with only

occasional lapses in

critical thinking skills

(analysis, synthesis,

evaluation, problem

solving skills, ability to

support opinions and

justify choices)

Understands and

articulates patterns,



related to the current

research problem.

Shows evidence of well-

developed critical thinking

skills (analysis, synthesis,

evaluation, problem

solving skills, ability to

support opinions and

justify choices)

Literature Review

Gathers relevant

research literature

SA1

Collects relevant research

from a convenient sample

of sources, however the

reviewed literature has a

notable absence of key

information

Collects relevant research

from various resources

and databases, however

some aspects of the review

are absent

Collects a comprehensive

collection of relevant

research

Data Analysis

Interprets data

effectively relative to

the research problem

SA1

Provides minimal

integration of gathered

information with elements

and facts pertaining to the

research problem

Provides some integration

of gathered information

with elements and facts

pertaining to the research

problem

Supports arguments with

relevant data when

applicable. Fully

integrates gathered

information with elements

and facts pertaining to the

research problem

Discussion and

Conclusions

Predicts and

evaluates

Minimally identifies

implications,

recommendations, and

conclusions


evaluate a limited scope of

implications,

recommendations and


evaluate a broad range of

implications,

recommendations and



implications,

recommendations

and conclusions

SA1

Ideas frequently

underdeveloped

conclusions. Ideas not

always fully developed

conclusions; Ideas fully

developed

Use of Academic

English

Organizes ideas well

and uses appropriate

academic register.

SA1

Notable absence of

appropriate organization

and structure; flow of ideas

is at times incoherent;

frequently lacks

appropriate, formal and

precise register

Occasional lack of

organization and

appropriate structure; At

times struggles with flow

of ideas; and uses

appropriate, formal and

precise register most of

the time

Well-organized and

structured throughout; the

flow of ideas is

consistently logical;

Regular use of formal and

precise register

APA Style

Uses APA style

(SA1)

Frequent lack of

appropriate APA

formatting

Occasional lapses in use

of APA format

Consistent use of APA

format


______________________________________________________________________________

______________________________________________________________________________



APPENDIX C.

Survey Instrument Items: Perceived Higher Education Administration Competencies and

Skills

Note. Students are given two sets of response options for each item stem. The first set of

response options (Part I) asks about self-rated knowledge of competencies, and the second set of

response options (Part II) asks about importance given to each competency.

Part I: Based on your current understanding of higher education competencies, how would you

rate your knowledge of the areas listed below? Please select the option that best reflects your

current knowledge.

Response options for each item

Not knowledgeable - Somewhat knowledgeable - Knowledgeable - Very

knowledgeable

Part II: Based on your career aspirations when you complete your Master’s Degree in Higher

Education Administration, how important are the following competencies to you? Please select

the option that best reflects your opinion.

Response options for each item

Not important - Somewhat important - Important - Very important

Item Stems for Parts I and II:

1. The mission, purpose, and goals of higher education. (K1)

2. The historical foundations of American higher education (K2)

3. The roles that faculty play in the functioning of American colleges and universities. (K3)

4. The roles that administrators and staff play in the functioning of American colleges and

universities. (K3)

5. The roles that students play in the functioning of American colleges and universities. (K3)

6. The ethical codes of conduct guiding higher education administration professional practice.

(K5)

7. How to network and cultivate professional relationships in the field of higher education

administration (SA4).

8. How to conduct literature searches to identify scholarship that might be helpful in

ameliorating an issue or question related to higher education practice. (SA1)

9. How to write a literature review on a higher education topic. (SA3)

10. How to do a presentation on a higher education topic. (SA3)

11. The components of a well-designed and well-executed higher education research study. (K6)

12. How to propose research questions related to an issue or problem in higher education. (K6,

SA1)

13. Common quantitative methodologies used in higher education research. (K6)

14. Common qualitative methodologies used in higher education research. (K6)



15. How higher education practitioners design and conduct research to better understand a

problem of practice. (K6, SA1)

16. The ethical issues related to the study of human subjects, especially college students. (K6)

17. Leadership practices in higher education (K4)

18. Finance and budgeting in higher education (K1, K4)

19. Legal issues in higher education (K1, K4)

20. Diverse student populations in higher education (K1, K3)

a competency-based approach to the master’s …journal of case studies in accreditation and...

Documents