learning outcomes: comparing student perceptions of skill level and importance

http://jmd.sagepub.com/Journal of Marketing Education

http://jmd.sagepub.com/content/24/3/203The online version of this article can be found at:

DOI: 10.1177/0273475302238043

2002 24: 203Journal of Marketing EducationCharles R. Duke

Learning Outcomes: Comparing Student Perceptions of Skill Level and Importance

Published by:

http://www.sagepublications.com

can be found at:Journal of Marketing EducationAdditional services and information for

http://jmd.sagepub.com/cgi/alertsEmail Alerts:

http://jmd.sagepub.com/subscriptionsSubscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

http://jmd.sagepub.com/content/24/3/203.refs.htmlCitations:

What is This?

- Dec 1, 2002Version of Record >>

at Universitats-Landesbibliothek on November 14, 2013jmd.sagepub.comDownloaded from at Universitats-Landesbibliothek on November 14, 2013jmd.sagepub.comDownloaded from

http://jmd.sagepub.com/


http://jmd.sagepub.com/content/24/3/203

http://jmd.sagepub.com/content/24/3/203



http://jmd.sagepub.com/cgi/alerts

http://jmd.sagepub.com/cgi/alerts

http://jmd.sagepub.com/subscriptions

http://jmd.sagepub.com/subscriptions

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.com/journalsPermissions.nav

http://www.sagepub.com/journalsPermissions.nav

http://jmd.sagepub.com/content/24/3/203.refs.html

http://jmd.sagepub.com/content/24/3/203.refs.html

http://jmd.sagepub.com/content/24/3/203.full.pdf

http://jmd.sagepub.com/content/24/3/203.full.pdf

http://online.sagepub.com/site/sphelp/vorhelp.xhtml

http://online.sagepub.com/site/sphelp/vorhelp.xhtml





DECEMBER 2002JOURNAL OF MARKETING EDUCATION

Learning Outcomes: Comparing StudentPerceptions of Skill Level and Importance

Charles R. Duke

As an example of the assessment of learning outcomes, stu-dent perceptions were obtained for learning outcome impor-tance as well as their own abilities to perform learning out-come skills. Development of outcomes involved multiplestakeholder groups and extensive reviews. Rankings weredeveloped for highest importance and highest skill level as away to evaluate outcomes needing emphasis. A simple prior-ity index demonstrated one technique of allocating resourcesin program changes. In this example, highest priorities con-verged around issues of interpersonal, leadership, globaleconomy, and interacting in the business environment. Exten-sive comparisons were made between class level as well asbetween marketing majors and other business disciplines.Although lists of outcomes, student perception ratings, andpriorities will vary among institutions, this example indicateda beginning point for rating criteria.

THE NEED FOR REVISED OUTCOMES

Continuing calls for revisions to marketing education haveprovided alternative ideas to meet the needs of both graduatesand their future employers (cf. Chonko 1993; Mason 1995).The alteration of American Assembly of Collegiate Schoolsof Business (AACSB) guidelines almost a decade ago hasgenerated discussion of the pros and cons of varied methodsof achieving the intent of the accreditation approach (AACSB1993; Geiger and Dangerfield 1997; Pharr and Morris 1997).

Regardless of these calls for immediate change, marketingdegree programs have not changed rapidly and are relativelyconsistent in course content and structure (Turnquist,Bialaszewski, and Franklin 1991; Webb, Mayer, and Piche1999). Although some institutions have modified their pre-sentation of content into nontraditional formats, relativelystandardized programs appeal to colleges seeking to facilitateaccreditation, transfer students, or hire new professors (cf.Koch 1997; Pharr and Morris 1997; Ruhland 1991). With thisconsistency, marketing content knowledge might be ade-quately covered, but other learning outcomes and critical hir-ing criteria may not be so obviously addressed (cf. Lamb,Shipp, and Moncrief 1995). To fully meet the needs of stu-

dents, modification of the outcomes from marketing pro-grams may be needed (cf. Chonko 1993).

A key part of a program’s evolution is understanding per-ceptions of the importance of learning outcomes and percep-tions of the skill level achieved (cf. Marshall, Laask, andGoolsby 1996). Useful perceptions can be gathered fromemployers, students, and faculty. This study provides anexample of the process used to develop institution-specificoutcomes along with analysis of student perceptions. Contin-ual assessment is needed beyond this starting point to ensurethat goals are being achieved.

Shifting Assessment Processes

The move from a “teaching orientation” to a “learning ori-entation” is a significant shift in the educational approach forprofessors as well as for curricula (cf. Chonko 1993). As apart of curriculum review and design, attention to studentlearning styles and to experiential education methods canimprove student engagement and increase learning (Dukeand Reese 1995; Roach, Johnston, and Hair 1993).

Outcome-based education uses learning goals and assess-ment to evaluate program performance expressed as skillknowledge of the graduate (Lamont and Friedman 1997).From these assessment results, decisions on changes may bemade to attain the expressed learning goals (cf. Owlia andAspinwall 1996). There does not appear to be a definitivestudy that suggests a universal list of learning outcomes,especially since outcomes are institution specific. The pro-cess used here has been recommended over many years todevelop those institutionally specific outcomes (Chonko1993; Lamb, Shipp, and Moncrief 1995; Lamont and Fried-man 1997).

203

Charles R. Duke is professor of marketing in the College of Business andBehavioral Sciences at Clemson University.

Journal of Marketing Education, Vol. 24 No. 3, December 2002 203-217DOI: 10.1177/0273475302238043© 2002 Sage Publications

at Universitats-Landesbibliothek on November 14, 2013jmd.sagepub.comDownloaded from



OUTCOMES DEVELOPMENT AND ANALYSIS

A general analytic framework for considering academiccourse contribution to learning outcomes follows a classicproject management approach of commitment to the process;generation of a skills list; identification of methods of skilldevelopment; understanding what is currently being used;determining deficiencies between current and desired skilllevels; and finally analyzing, monitoring, and assessing skilllevel in a revised curriculum (cf. Floyd and Gordon 1998).Specific skills can be introduced or reinforced in a course orin other programmatic features (cf. Arora and Stoner 1992;Giacobbe and Segal 1994). Deficiencies in student develop-ment can be identified and addressed.

Both academic and popular literature on hiring criteria canbe used as sources in developing learning outcomes. Lists ofhiring criteria have been relatively consistent over time (cf.Kelley and Gaedeke 1990; McDaniel and White 1993;Tomkovick, Erffmeyer, and Hietpas 1996). Generally, com-munications, motivation and personal attributes, problemsolving, leadership, and marketing skills are near the top ofthese lists. Techniques for improving student achievementhave been discussed for some of these outcomes with exam-ples for written and oral communications (Corbin and Glynn1992; Shipp, Lamb, and Mokwa 1993), intuition (Shipp,Lamb, and Mokwa 1993), creativity (Ramocki 1994; Gilbert,Prenshaw, and Ivy 1996; Shipp, Lamb, and Mokwa 1993),computer usage (Shipp, Lamb, and Mokwa 1993), informa-tion usage (Sterngold and Hurlbert 1998), technical compe-tence (Gomes, Pickett, and Duke 1992), effective listening(Cost, Bishop, and Anderson 1992), ethics (Martin 1990),and international competence (Hopkins 1999; Kaynak,Yucelt, and Barker 1990; Turley and Shannon 1999).

Outcomes Development

Developing a list of outcomes relevant to each individualuniversity or marketing program requires commitment to theprocess and participation from multiple stakeholders (Lamb,Shipp, and Moncrief 1995). The process for identifying theappropriate learning outcomes involves review of existinginformation along with participation of employers, students,faculty, and others interested in student development.Learning outcomes may include general concepts relevant toall schools, criteria related to the special nature of the pro-gram (such as specialties or targeted hiring industries), andissues related to the special culture of the institution (cf.Giacobbe and Segal 1994; Gomes, Pickett, and Duke 1992;Koch 1997). Several sources contribute to criteria development:

• AACSB guidelines ensure that the core will meet accredita-tion needs.

• Mission statements for the college ensure compatibility andinsight into areas of need.

• General publications and business education literature pro-vide comments and trends for curriculum issues, learningoutcomes, and hiring criteria.

• Benchmark surveys detail extensive comparisons to otheruniversities.

• Curricula of other business schools provide other approachesto business education.

• Exit surveys provide unique insight into the process of curric-ular flow as well as perception of skills developed by the mar-keting program.

• Alumni surveys provide direct evaluation of graduates’ per-ception of their preparation.

• Business firms and recruiters indicate what they expect fromnew employees.

Analysis Options

Different analyses provide a choice of individual versuscomposite scores used in various methods to make compari-sons between cross-sectional or temporal classifications.

Individual outcomes. All of the outcomes can be listed inorder of perceived importance or of perceived skill level (cf.Chonko 1993; Lamb, Shipp, and Moncrief 1995). This showsthe relative ranking of issues and may provide more structureto the analysis.

Composite categories. Perceptions of performance andimportance of general issues can be evaluated by aggregatingthe scores of the individual criteria supporting each compos-ite outcome category as illustrated by SERVQUAL(Zeithaml, Berry, and Parasuraman 1996) and SERVPREF(Cronin and Taylor 1994). The analyst could then considerthe composite ratings or the relative ranking of the categoriesto determine priorities for improvement.

Analysis of variance. Comparisons across many differentclassifications could provide insight for program develop-ments. For each learning outcome, comparisons can be usedto determine differences among business majors that mayindicate relative strengths and weaknesses of each discipline.In addition, comparisons across the class level of the studentmay indicate how well the upper division has improved thestudents’ confidence and skill level beyond common coreclasses.

Several other options are available to make cross-sectionalor temporal comparisons of assessment data. Comparingmarketing or business results with results from nonbusinessdisciplines may be important if marketing programs are incompetition with other disciplines for funding. Assessmentmight be done using many other stakeholder groups (cf.Arora and Stoner 1992; Glynn, Rajendram, and Corbin 1993;Heinfeldt and Wolf 1998). Employer perceptions affect thestudents’ prospects for jobs after graduation and the reputa-tion of the institution in attracting support (Floyd and Gordon1998; Lundstrom and White 1997; McMartin 1999). Alumni

204 DECEMBER 2002




perceptions offer the opportunity to gain input from thosealready in business with the background and experience in theacademic programs being evaluated (Duke and Reese 1995).Faculty members bring the experience of program develop-ment and discipline expertise (Lundstrom and White 1997).An approach that uses several stakeholder groups may deter-mine differences not only in delivery of the marketing pro-gram but also in its perceived effectiveness among these dif-ferent stakeholder groups (Floyd and Gordon 1998; Lamb,Shipp, and Moncrief 1995).

Factor analysis. Using factor analysis, researchers areable to examine potential groupings of individual items thatmight indicate underlying dimensions not otherwise obvious.Factor analysis dimensions are expected to reflect the com-posite areas already developed because the individual itemsare normally generated to describe or measure a facet of thosedimensions (cf. Hair et al. 1998). For this exploratory study,the analysis searches for the initial structure of a set of itemsgenerated for institution-specific measurement.

Student Perceptions

Student perception of the importance of learning out-comes may point to a shift in needs from employers, or it maysignal a weakness in relaying and emphasizing the need to thestudent (Duke and Reese 1995; Lamont and Friedman 1997;Lundstrom and White 1997). Perceptions of the achievementof learning outcomes may suggest how well the program isdeveloping the student (cf. Duke and Reese 1995; Glynn,Rajendram, and Corbin 1993).

Although students may not have a firm grasp on the reali-ties of the workplace (cf. Hall et al. 1995), they have a reason-able grasp on the quality of their curriculum and programwith some biases toward wanting more from their institution(cf. Glynn, Rajendram, and Corbin 1993; Turley and Shan-non 1999). Understanding student perception is crucial fortactical evaluation of classroom performance, general curric-ulum flow and value, as well as alumni reflections on thevalue of the components of their educational programs (Dukeand Reese 1995).

Student perceptions, although not the only measurementmethod or group to be used, are a crucial component to suc-cessful program assessment and revision (Duke and Reese1995; Lamont and Friedman 1997; Lundstrom and White1997). Using the concept of developing institution-specificlearning outcomes and using student perceptions as one criti-cal stakeholder evaluation, this study (1) reports the generalmethod used by one business program and marketing depart-ment to develop programmatic learning outcomes and (2)demonstrates the analysis of student perceptions of theimportance of those outcomes along with perception of theirskill level for each outcome.

PERFORMANCE EVALUATION APPROACHES

The basis of perceptual outcome assessment has evolvedfrom multiattribute attitude models (cf. Fishbein and Ajzen1975) through performance importance (Kotler 2000;Martilla and James 1977) to multiple discussions of servicequality (cf. Zeithaml, Berry, and Parasuramum 1996; Croninand Taylor 1994). In all these conceptualizations, both per-ceptions of performance and importance are provided by cus-tomers. Just as these service quality and preference scalesrequire adjustment for each specific application and for var-ied stakeholders, educational assessment tools require adjust-ment for progress to be tracked (Owlia and Aspinwall 1996).Differences in disciplines and in programs suggest differentelements and emphasis areas for evaluation (Ward and Chan-dler 1999).

Performance Importance

Performance importance charts are used to consider howwell a criterion is being met compared to how important it isto the user (Duke and Persia 1996; Kotler 2000; Martilla andJames 1977). Plots can be made of consumer ratings with per-formance measured on one axis and importance on the other,with both constructs measured on the same numerical scale.The assumed ideal allocation of resources and effort wouldresult in performance equal to importance, with these scoresfalling along a 45-degree diagonal across the chart (Martillaand James 1977). For example, high performance on high-importance criteria indicate the need to “keep up the goodwork” and appropriate resources being used for importantissues. And low performance on low-importance criteria indi-cates low-priority issues. Conversely, high performance oncriteria of low importance indicates “possible overkill,”whereas low performance on high-importance criteria indi-cates issues that need attention (Martilla and James 1977).

Service Quality

Service quality considers the success or progress of anorganization in delivering the service that was intended (Fisk,Grove, and John 2000; Palmer and Cole 1995). TheSERVQUAL approach uses scales designed to measure cus-tomer perceptions of service quality along five key dimen-sions: reliability, tangibles, assurance, empathy, and respon-siveness of the service provider. It evaluates both process andoutcomes (cf. Fisk, Grove, and John 2000). SERVQUAL-styled systems use expectations as a comparison level for theperceived performance (Zeithaml, Berry, and Parasuramum1996), whereas SERVPREF-styled systems use a single mea-sure for performance comparing evaluations over time(Cronin and Taylor 1994). In both systems, importanceweights are used to indicate priorities and to construct com-posite evaluation measures.

JOURNAL OF MARKETING EDUCATION 205




Using Importance Scores

Intuitively, higher importance scores indicate attributes orcriteria that require more effort or resources to ensure meet-ing the needs of the consumer. Lower importance criteriaindicate attributes or criteria that require less effort orresources since they are not crucial to the consumer’s choiceprocess (Kotler 2000; Martilla and James 1977).

Opinions vary on using importance scores to weight per-ceptions. Using weighted scores implies that customers’ per-ceptions are related to the strength of their beliefs of eachattribute’s importance multiplied by the perception of perfor-mance for the attribute. Some researchers suggest that impor-tance weighting helps to estimate the complex attitudes forservices (cf. Carman 1990). Without attribute importance,there is no indication of the relative importance that custom-ers attach to evaluation criteria or performance.

METHOD

Student perceptions of learning outcomes were sought as apart of evaluation for changing the business common core.Following a general evaluation model (cf. Lamb, Shipp, andMoncrief 1995), the project intended to identify what stu-dents should know (expected learning outcomes), how theprogram currently meets these criteria, and the changes tocurricula to meet learning outcomes.

Outcomes Development

Generating a list of outcomes (skills) required extensiveinput and evaluation. Accreditation guidelines were reviewedto assure compliance. Lists of hiring criteria and success fac-tors were compiled from both academic and business educa-tion literature to provide trends and specific issues to be con-sidered. Benchmark studies provided confirmation of someissues and expansion of others. Reviews of business curriculafrom other institutions indicated that all were similar,although each had a slightly different implementation inmeeting learning outcomes.

Exit surveys gave specific and crucial insight into programsuccesses and deficiencies in meeting learning outcomes.Alumni surveys, primarily from those 1, 2, and 5 years aftergraduation, provided direct evaluation of some learning out-comes and information to develop others. As a preliminaryassessment, alumni felt that their education prepared themwell for their careers along many of the outcomes and thatimprovement was needed on others. Preliminary compositecategories were recognized, and other facets of these catego-ries were developed with new criteria to assess outcomesmore comprehensively.

After using these sources to develop preliminary lists ofoutcomes, individual depth interviews and focus groups of

employers and advisers were used to gain additional insightand confirmation of the appropriate items to use in assessingoutcomes. In these discussions, common themes emerged forthe grouping of some outcomes and development of new onesto encompass broader composite categories of learning out-comes. In addition, lists of outcomes were reviewed forappropriateness and comprehensiveness among groups offaculty in each of the business disciplines as well as amonguniversity faculty involved with outcomes assessment.

Multiple items were generated within each of the compos-ite categories. As criteria for assessment of content and levelof educational achievement, the learning outcomes weredeveloped using verbs and adjectives suggested to fit withBloom’s (1956) Taxonomy of Educational Objectives. Thetaxonomy suggests that education level increases as studentspass through the stages of knowledge, comprehension, appli-cation, analysis, synthesis, and evaluation. This led to devel-opment of assessment measures to evaluate how well learningoutcomes were attained.

Questionnaire Construction

Each of the learning outcome criteria was rated on anunforced 5-point Likert-type scale of importance (“In myopinion, this skill is important to my future job”) with optionfor “don’t know.” The performance of the program in provid-ing the outcome was similarly rated on a 5-point Likert-typescale (“In my opinion, the courses have provided me with ahigh level of this skill”). The performance rating was a forcedscale since the respondents were fully capable of answeringthe question. In addition, each student provided informationon his or her class status, major, and overall grade pointaverage.

Pretest of the preliminary questionnaire was performed by43 students, not used in later data analysis, and indicated aneed to drop, consolidate, expand, or reword many of the indi-vidual items. The final result was a set of nine categories and atotal of 45 items, each selected for its appropriateness,uniqueness, and ability to convey different facets of the out-come areas identified (Churchill 1979). The categories devel-oped for this institution-specific measurement were leader-ship, communication, interpersonal, analytic, decisionmaking, technological, global economy, ethics, and businesspractices. The individual items that were developed in thisprocess are listed in Table 1.

Sampling

To maintain some control over duplication and to increasethe probability of response rate, cluster sampling in class-rooms was used. Course sections were identified to collectevaluations from students who were (1) just completing lowerdivision core requirements and (2) graduating seniors just

206 DECEMBER 2002




completing the entire curriculum. Questionnaires were dis-tributed during class time, collected as students finished,coded by the researchers, entered by skilled input specialists,and then analyzed. Total sample size was 502 students (allmajors), with 358 business students (186 seniors and 172completing lower division). Marketing majors accounted for159 of the business majors with 199 in other business majors.

USING RESULTS FOR PROGRAM ANALYSIS

Ranking Composite Categories

Combined category item ratings of student perceptionsgive an indication of the perceptions for broad categories oflearning outcomes and skill levels (Table 2). Note that theobjective in these groupings was to organize the criteria ratherthan to effectively develop cohesive scales. Factor analysis ofthe criteria to confirm category groupings and to exploreother underlying dimensions is discussed after individual out-come results are explored.

Category means for importance ratings were limited to anarrow range (Table 2). Students perceived technologicalskills and interpersonal skills to be the most important as wellas highest in skill level. Global economy and analytic impor-tance were the lowest rated. The relative rankings of the cate-gories between importance and perceived skill were some-what consistent, but not exactly the same. Analytic skills,decision-making skills, and ethics were ranked higher in skillthan in perceived importance. The most obvious difference isthat interpersonal skills were ranked dramatically higher inimportance than skill level, indicating perhaps the greatestpriority as perceived by students.

Ranking with a “Priority Index”

Decisions on which criteria to address first should con-sider both perceived skill levels and importance level. Thedifferent rank orders of the two measures presents somedilemma in setting priorities. The scores can be combined todevelop an index that indicates the position of the score rela-tive to the ideal position on the 45-degree diagonal on a per-formance importance chart. Depending on the interval scalesused, the user (or researcher) can develop a priority index thatdevelops a comparison between the individual outcomes andbetween the outcome categories. One simple index using therating scales in this project considers the ratio of the skill levelto importance. When the ratio is low, then the outcome needsattention, and when it is high, then the outcome might beoveremphasized. By subtracting this result from a value of1.0, a priority index can be developed that shows at a glancethose criteria that need the greatest attention (higher positivescores), while lower priority criteria will have lower scores.


TABLE 1LEARNING OUTCOMES

A. Leadership skillsA1 Ability to serve as a team leaderA2 Ability to use different leadership stylesA3 Ability to support shared team valuesA4 Ability to facilitate conflict resolution

B. Communication skillsB1 Ability to write clearlyB2 Ability to speak effectively to groupsB3 Ability to use active listening skillsB4 Ability to explain technical concepts to nontechnical peopleB5 Ability to communicate at the correct level of detailB6 Ability to manage communication flow in teamsB7 Ability to write an executive summary

C. Interpersonal skillsC1 Comprehension of differences among peopleC2 Ability to relate to people with diverse backgroundsC3 Ability to build effective teamsC4 Ability to solve conflicts

D. Analytic skillsD1 Comprehension of quantitative problem-solving techniquesD2 Ability to apply the right tools to business problemsD3 Comprehension of the accuracy and reliability of dataD4 Ability to think systematicallyD5 Ability to identify relationships between problems and/or

issuesE. Decision-making skills

E1 Ability to use decision-making techniques to solve problemsE2 Knowledge of negotiating skills and techniquesE3 Ability to anticipate and provide alternative solutionsE4 Ability to identify central issues of a problemE5 Ability to incorporate market and competitor information into

analysisE6 Ability to evaluate risk involved in decisions

F. Technological skillsF1 Ability to use word processingF2 Ability to use spreadsheetsF3 Ability to use databasesF4 Ability to prepare multimedia presentationsF5 Ability to search and integrate multiple data sourcesF6 Ability to communicate electronically

G. Global economyG1 Comprehension of cultural differencesG2 Comprehension of economic differencesG3 Comprehension of the global business environmentG4 Comprehension of the impact of other economic systems

on the U.S. economyH. Ethics

H1 Ability to recognize ethical conflicts in personal situationsH2 Ability to recognize ethical conflicts in business situationsH3 Ability to make ethical decisions

I. Business practicesI1 Ability to conduct a business meetingI2 Ability to analyze industry trendsI3 Comprehension of market-based economiesI4 Knowledge of the interdependence of business functionsI5 Comprehension of basic business practicesI6 Ability to focus on customer needs




Negative scores indicate “overkill” and might be consideredfor reduction in resources. The formula becomes:

PI = [1 – (SL / Imp)],

where PI = priority index, SL = skill level, and Imp = impor-tance. Higher priority criteria are considered for improve-ment and greater resources. And lower priority criteria (espe-cially negative ones) are considered for reduced resources orat least no larger infusion of resources. Because the index inthis example is developed from interval data with an arbitraryzero point and the scales can be changed to suit the needs ofdifferent university departments, each project must considerwhether a different index needs to be developed if the scalesvary. However, if scales are consistently applied within a pro-ject, then any index developed will consistently indicate therelative priority of each outcome.

In this case, interpersonal and leadership skills appear tohave the highest priority, whereas analytic, technological, anddecision-making skills and ethics seem to be lower in priority(Table 2). Note that, in this case, the indexes as well as themeans are very close, and all of the outcome categories areconsidered to need more resources and effort. The priority

ranking is one method of determining which area is addressedfirst.

Ranking Individual Outcome Perceptions

Expanding beyond category analysis, each of the individ-ual learning outcomes may be ranked by its importance ratingand, separately, by its perceived skill level (Table 3). Althoughthis may show a complex mixture of the various categoryitems, the specific outcomes can be more closely examinedfor improvement. Convergence of rankings for the two sets ofratings might indicate an appropriate level of effort on the partof the program. Skill-level ratings indicate the students’com-fort level with their capabilities. However, students may alsoconsider that they do not have the experience to rate them-selves at a higher level of capabilities. Reasons for rankingscan be investigated in continuing program evaluations.

Student perceived importance ratings ranged from a low of4.03 to a high of 4.71 (Table 3), with a wider range of skill-level ratings from 3.43 to 4.25. Importance perceptions pro-vide the students’ outlook on what is necessary for employ-ment. In this case, consistent ratings for both importance andskill (performance) indicate that some criteria seem to beappropriately taught in the curriculum. Some of the many dif-ferent communications issues were rated highly along withsome other interpersonal issues.

Conversely, divergence of rankings may indicate an areafor attention. For example, “ability to conduct a businessmeeting” was rated as very important (ranked 7) but receiveda relatively lower perceived skill level (ranked 45), whichmight indicate an area for improvement in skill training. Asanother example, “identifying the central issues of a prob-lem” was ranked as 30 in importance but higher in perceivedskill at 8, which offers potential for either improving studentperceptions of importance or improving their perception oftheir own more modest ability to identify central issues. It iscrucial to understand that this step in the program’s analysismust be followed by more detailed investigation into under-standing the reasons for the ratings.

Priority indexes for each individual criterion can also becalculated (Table 4). In this case, students perceived theirgreater priorities for individual criteria to be in various inter-personal, communications, and leadership skills. The smallerpriorities were in analytic techniques, decision making, busi-ness practices, and computer skills.

Comparing Marketing with Other Disciplines

Many marketing degree programs are closely aligned inacademic and/or curriculum structure with other business dis-ciplines, whereas some marketing departments are relativelyautonomous in their program decisions. The differences mayindicate that one discipline might share its success with otherdisciplines or needs to look at other disciplines for practical

208 DECEMBER 2002

TABLE 2COMPOSITE OUTCOME CATEGORIES

RATINGS—MARKETING MAJORS

Importance a Skill Level b Priority Index c

Outcome Category M Rank M Rank PI Rank

A. Leadership skills 4.44 5/6 3.57 7 .196 2B. Communication

skills 4.44 5/6 3.61 6 .187 4C. Interpersonal

skills 4.48 2 3.53 8 .212 1D. Analytic skills 4.27 9 3.68 3/4 .138 9E. Decision-making

skills 4.45 4 3.70 2 .169 6/7F. Technological

skills 4.53 1 3.86 1 .148 8G. Global economy 4.35 8 3.52 9 .191 3H. Ethics 4.43 7 3.68 3/4 .169 6/7I. Business practices 4.46 3 3.65 5 .182 5

a. Importance: Each outcome is rated on a Likert-type scale (1 =strongly disagree, 5 = strongly agree) unforced with option of “don’tknow.” Response is to the statement “In my opinion, this skill is impor-tant to my future job.”b. Skill level: Each outcome is rated on a forced Likert-type scale (1 =strongly disagree, 5 = strongly agree). Response is to the statement“In my opinion, the courses have provided me with a high level of thisskill.”c. Priority Index (PI) = (1 – (Skill Level / Importance)). Higher magni-tude indicates higher priority. Positive values indicate higher prioritydue to underperformance. Negative values indicate lower prioritiesfrom overperformance.




advise on improvement. In this example, summaries of signif-icant differences between marketing students and those inother business disciplines were developed (Tables 5 and 6).

Marketing majors perceived higher importance for com-municating correct level of detail, incorporating competitorinformation, as well as comprehending cultural and eco-nomic differences. On the other hand, marketing studentsdecreased in their perceived importance of managing teamcommunication flow from lower division to seniors. Thismay indicate that practicing team communications duringtheir senior year reduced their anxiety about the process. Or, itmay indicate that other issues replaced this criterion in theirpriority scheme.

Skill perceptions may vary across majors. In this study,marketing students had higher skill perceptions than othermajors for ability to use decision-making techniques, provid-ing alternative solutions, listening skills, and conductingbusiness meetings. Marketing students rated themselveslower in perceived skill for quantitative problem-solvingtechniques, systematic thinking, comprehension of data, abil-ity to evaluate risk, use of spreadsheets, and preparing multi-media presentations.

Evolution from Lower Division to Seniors

Rating changes between students completing lower divi-sion core courses with graduating seniors who have com-pleted almost all of the degree program indicates the successor need for improvement of the upper division of the program


TABLE 3MARKETING STUDENT PERCEPTIONS

OF LEARNING OUTCOMES

Importance a Skill Level b

Learning Outcome/Criterion M Rank M Rank

B2 Ability to speak effectively in groups 4.71 1 3.87 6F6 Ability to communicate electronically 4.65 2 4.25 1F1 Ability to use word processing 4.63 3 4.23 2I6 Ability to focus on customer needs 4.62 4 4.08 3B1 Ability to write clearly 4.60 5 3.81 7B3 Ability to use effective listening

techniques 4.57 6 3.75 6I1 Ability to conduct a business

meeting 4.57 7 3.13 45A1 Ability to serve as a team leader 4.54 8 3.77 12C2 Ability to relate to people with

diverse backgrounds 4.53 9 3.48 36E6 Ability to evaluate risk involved

in decisions 4.52 10 3.67 22I5 Comprehension of basic business

practices 4.52 11 3.98 4C4 Ability to solve conflicts 4.51 12 3.57 30F4 Ability to prepare multimedia

presentations 4.51 13 3.60 27H3 Ability to make ethical decisions 4.51 14 3.75 15F2 Ability to use spreadsheets 4.50 15 3.79 9F5 Ability to search and integrate

multiple data sources 4.49 16 3.76 13E5 Ability to incorporate market and

competitor information into analysis 4.48 17 3.77 11A4 Ability to facilitate conflict resolution 4.47 18 3.45 39E1 Ability to use decision-making

techniques to solve problems 4.47 19 3.90 5C3 Ability to build effective teams 4.47 20 3.48 37F3 Ability to use databases 4.43 21 3.56 33C1 Comprehension of differences

between people 4.43 22 3.57 29E2 Knowledge of negotiating skills

and techniques 4.43 23 3.35 43H2 Ability to recognize ethical conflicts

in business situations 4.42 24 3.72 16B5 Ability to communicate at the

correct level of detail 4.42 25 3.56 31D2 Ability to apply the right tools to

business problems 4.42 26 3.70 18G3 Comprehension of the global

business environment 4.41 27 3.62 25G4 Comprehension of the impact of

other economic systems on theU.S. economy 4.41 28 3.48 38

E3 Ability to anticipate and providealternative solutions 4.41 29 3.71 17

E4 Ability to identify central issuesof a problem 4.41 30 3.79 8

I2 Ability to analyze industry trends 4.40 31 3.55 34B6 Ability to manage communication

flow in teams 4.40 32 3.68 20A3 Ability to support shared team values 4.38 33 3.70 19A2 Ability to use different leadership

styles 4.37 34 3.36 42H1 Ability to recognize ethical conflicts

in personal situations 4.37 35 3.58 28

D5 Ability to identify relationshipsbetween problems and/or issues 4.35 36 3.78 10

I3 Comprehension of market-basedeconomies 4.32 37 3.54 35

D4 Ability to think systematically 4.31 38 3.61 26G2 Comprehension of economic

differences 4.31 39 3.56 32I4 Knowledge of the interdependence

of business functions 4.30 40 3.63 24G1 Comprehension of cultural

differences 4.26 41 3.43 40D3 Comprehension of the accuracy

and reliability of data 4.24 42 3.64 23B7 Ability to write an executive summary 4.20 43 3.41 41B4 Ability to explain technical concepts

to nontechnical people 4.18 44 3.19 44D1 Comprehension of quantitative

problem-solving techniques 4.03 45 3.67 21

a. Importance: Likert-type scale (1 = strongly disagree; 5 = stronglyagree) unforced with option of “don’t know.” Response is to the state-ment “In my opinion, this skill is important to my future job.”b. Skill level: Likert-type scale (1 = strongly disagree; 5 = stronglyagree).Response is to the statement “In my opinion, the courses haveprovided me with a high level of this skill.”

TABLE 3 (continued)

Importance a Skill Level b

Learning Outcome/Criterion M Rank M Rank




(Tables 5 and 6). In this example, seniors perceived a higherimportance for speaking in groups, applying the right tools toproblems, identifying the relationships between problems,integrating multiple data sources, communicating electroni-cally, comprehending the global environment, and conduct-ing a business meeting.

Seniors also perceived that their skills were higher inexplaining technical concepts, managing team communica-tions, writing executive summaries, applying the right tools,comprehending data accuracy, incorporating competitorinformation, evaluating risks, using all computer packages,searching and integrating multiple data sources, compre-

210 DECEMBER 2002

TABLE 4PRIORITY INDEXES FOR OUTCOME IMPORTANCE ANDSKILL-LEVEL RATINGS FOR MARKETING STUDENTS

Learning Outcome/Criterion Importance a Skill b PI c Priority

I1 Ability to conduct abusiness meeting 4.57 3.13 .314 1

E2 Knowledge of negotiatingskills and techniques 4.43 3.35 .243 2

B3 Ability to explain technicalconcepts to nontechnicalpeople 4.18 3.19 .236 3

A2 Ability to use differentleadership styles 4.37 3.36 .232 4

C2 Ability to use differentleadership styles 4.53 3.48 .232 5

A4 Ability to facilitate conflictresolution 4.47 3.45 .228 6

C3 Ability to support sharedteam values 4.47 3.48 .220 7

G4 Comprehension of impactof other economic systemson the U.S. economy 4.41 3.48 .212 8

C4 Ability to facilitate conflictresolution 4.51 3.57 .209 9

F4 Ability to prepare multimediapresentations 4.51 3.60 .201 10

F3 Ability to use databases 4.43 3.56 .197 11G1 Comprehension of cultural

differences 4.26 3.43 .194 12C1 Ability to serve as a team

leader 4.43 3.57 .194 13I2 Ability to analyze industry

trends 4.40 3.55 .193 14B5 Ability to communicate at

the correct level of detail 4.42 3.56 .193 15E6 Ability to evaluate risk involved

in decisions 4.52 3.67 .189 16B7 Ability to write an executive

summary 4.20 3.41 .188 17I3 Comprehension of

market-based economies 4.32 3.54 .180 18H1 Ability to recognize ethical

conflicts in personal situations 4.37 3.58 .180 19B3 Ability to use active listening

skills 4.57 3.75 .179 20G3 Comprehension of the global

business environment 4.41 3.62 .179 21B2 Ability to speak effectively to

groups 4.71 3.87 .178 22G2 Comprehension of economic

differences 4.31 3.56 .174 23B1 Ability to write clearly 4.60 3.81 .172 24A1 Ability to serve as a team

leader 4.54 3.77 .170 25H3 Ability to make ethical

decisions 4.51 3.75 .167 26B6 Ability to manage

communication flow in teams 4.40 3.68 .164 27D4 Ability to think systematically 4.31 3.61 .163 28F5 Ability to search and integrate

multiple data sources 4.49 3.76 .163 29

D2 Ability to apply the right toolsto business problems 4.42 3.70 .161 30

H2 Ability to recognize ethicalconflicts in business situations 4.42 3.72 .160 31

E5 Ability to incorporate marketand competitor informationinto analysis 4.48 3.77 .159 32

E3 Ability to anticipate andprovide alternative solutions 4.41 3.71 .159 33

F2 Ability to use spreadsheets 4.50 3.79 .158 34I4 Knowledge of the

interdependence ofbusiness functions 4.30 3.63 .156 35

A3 Ability to support sharedteam values 4.38 3.70 .155 36

D3 Comprehension of theaccuracy and reliability of data 4.24 3.64 .142 37

E4 Ability to identify centralissues of a problem 4.41 3.79 .140 38

D5 Ability to identify relationshipsbetween problems and/orissues 4.35 3.78 .129 39

E1 Ability to use decision-makingtechniques to solve problems 4.47 3.90 .128 40

I5 Comprehension of basicbusiness practices 4.52 3.98 .119 41

I6 Ability to focus on customerneeds 4.62 4.08 .117 42

D1 Comprehension of quantitativeproblem-solving techniques 4.03 3.67 .089 43

F1 Ability to use word processing 4.63 4.23 .087 44F6 Ability to communicate

electronically 4.65 4.25 .084 45

a. Importance: Likert-type scale (1 = strongly disagree, 5 = stronglyagree) unforced with option of “don’t know.” Response is to the state-ment “In my opinion, this skill is important to my future job.”b. Skill level: Likert-type scale (1 = strongly disagree, 5 = stronglyagree).Response is to the statement “In my opinion, the courses haveprovided me with a high level of this skill.”c.Priority index (PI) = (1 – (Skill Level / Importance).Higher magnitudeindicates higher priority. Positive values indicate higher priority due tounderperformance. Negative values indicate lower priorities fromoverperformance.

TABLE 4 (continued)

Learning Outcome/Criterion Importance a Skill b PI c Priority




hending global issues and economic impact, analyzing indus-try trends, comprehending business practices, and focusingon the consumer. Seniors perceived lower skills in explainingtechnical concepts and managing communication flows pos-sibly as a result of greater direct experience with these issues.

Interactions of Major and Class

There is a potential that students in difference majors mayevolve differently in their perceptions as they progressthrough their programs. In this study, marketing students

were generally stable or declining across classes in theirimportance perceptions, whereas other majors increased theirimportance perceptions for using different leadership styles,speaking more effectively in groups, and managing commu-nication flow in teams (Tables 5 and 6).

Changes in perceived skill levels across class varied acrossmajors. Marketing majors increased their perception of abil-ity in listening skills and were stable on conflict resolutionabilities (Tables 5 and 6). However, they decreased in theirperceptions of using leadership styles and comprehension ofquantitative techniques.


TABLE 5SUMMARY OF ANOVA DIFFERENCES FOR IMPORTANCE BETWEENMARKETING AND OTHER BUSINESS STUDENTS AND CLASS LEVEL

Effects: Class,Learning Outcome / Criterion Major, Interaction Description

Leadership skillsA2 Ability to use different leadership styles Interaction Marketing stable; others increase

Communication skillsB1 Ability to write clearly Major Management lowerB2 Ability to speak effectively to groups Class Seniors higher

Interaction Others increased more than marketing over classB4 Ability to explain technical concepts to Interaction Finance increase; others decrease over class

nontechnical peopleB5 Ability to communicate at the correct level of detail Major Accounting lower; marketing and

finance higherB6 Ability to manage communication flow in teams Interaction Accounting and management increase; marketing

and finance decrease over classInterpersonal skills

C2 Ability to relate to people with diverse backgrounds Major Accounting lowerAnalytic skills

D1 Comprehension of quantitative problem-solving techniques Major Accounting higher; marketing and management lowerD2 Ability to apply the right tools to business problems Class Seniors higherD3 Comprehension of the accuracy and reliability of data Major Accounting and finance higherD5 Ability to identify relationships between problems Class Seniors higher

and/or issuesDecision-making skills

E2 Knowledge of negotiating skills and techniques Major Finance higher; accounting lowerE5 Ability to incorporate market and competitor Major Marketing higher; accounting lower

information into analysis.Technological skills

F5 Ability to search and integrate multiple data sources Class Senior higherF6 Ability to communicate electronically Class Senior higher

Global economyG1 Comprehension of cultural differences Major Marketing higher; accounting lower

Interaction Accounting stable; others increased over classG2 Comprehension of economic differences Major Marketing and finance higherG3 Comprehension of the global business environment Class Senior higher

Major Marketing and finance higherG4 Comprehension of the impact of other economic Major Marketing and finance higher

systems on the U.S. EconomyEthics No significant differencesBusiness practices

I1 Ability to conduct a business meeting Class Senior higherI2 Ability to analyze industry trends Major Finance higher; accounting and marketing lowerI3 Comprehension of market-based economies Major Finance higher; management lowerI4 Knowledge of the interdependence of business Major Finance higher; management lower

functions




212 DECEMBER 2002

TABLE 6SUMMARY OF SIGNIFICANT DIFFERENCES FOR SKILL LEVEL BETWEEN

MARKETING AND OTHER BUSINESS STUDENTS AND CLASS LEVEL

Effects: Class,Learning Outcome / Criterion Major, Interaction Description

Leadership skillsA2 Ability to use different leadership styles Interaction Marketing decreases over class; others are stable or

increaseA4 Ability to facilitate conflict resolution Interaction Marketing stable; others increase over class

Communication skillsB3 Ability to use active listening skills Interaction Marketing and finance increased; management

decreased over classB4 Ability to explain technical concepts to nontechnical people Class Senior lowerB6 Ability to manage communication flow in teams Class Senior lowerB7 Ability to write an executive summary Class Senior higher

Interpersonal skills No significant differencesAnalytic skills

D1 Comprehension of quantitative problem-solving techniques Interaction Finance and management higher; marketing andaccounting lower

D2 Ability to apply the right tools to business problems Class Seniors higherD3 Comprehension of the accuracy and reliability of data Class Seniors higher

Major Accounting higher; marketing lowerD4 Ability to think systematically Major Accounting and finance higher; marketing and

management lowerD5 Ability to identify relationships between problems and/or Major Finance higher

issuesDecision-making skills

E1 Ability to use decision-making techniques to solve problems Major Marketing and accounting higher; finance andmanagement lower

E3 Ability to anticipate and provide alternative solutions Major Marketing higherE5 Ability to incorporate market and competitor information Class Senior higher

into analysis Major Management lowerE6 Ability to evaluate risk involved in decisions Class Senior higher

Major Accounting and finance higher; marketing andmanagement lower

Technological skillsF1 Ability to use word processing Class Senior higherF2 Ability to use spreadsheets Class Senior higher

Major Accounting higher; marketing lowerF3 Ability to use databases Major Marketing and finance lowerF4 Ability to prepare multimedia presentations Class Senior higher

Major Management highest; marketing lowestF5 Ability to search and integrate multiple data sources Class Senior higher

Global economyG2 Comprehension of economic differences Class Senior higher

Major Finance higher; accounting lowerG3 Comprehension of the global business environment Class Senior higher

Major Finance higher; accounting lowerG4 Comprehension of the impact of other economic systems Class Senior higher

on the U.S. economy Major Finance higher; accounting lowerEthics

H1 Ability to recognize ethical conflicts in personal situations Major Accounting highest; management lowestH3 Ability to make ethical decisions Major Accounting highest; management lowest

Business practicesI1 Ability to conduct a business meeting Major Marketing highest; management lowestI2 Ability to analyze industry trends Class Senior higher

Major Accounting highest; management lowestI3 Comprehension of market-based economies Major Finance highest; management lowestI5 Comprehension of basic business practices Class Senior higher

Major Accounting higher; management lowerI6 Ability to focus on customer needs Class Senior higher

Major Management lower





TABLE 7FACTOR LOADINGS FOR A PRIORI CATEGORIES

Importance Skill Level

Coefficient Factor Coefficient FactorLearning Outcome / Criterion (N = 461, all majors) Alpha Loading Alpha Loading

A. Leadership skills .75 .79A1 Ability to serve as a team leader .56 .57A2 Ability to use different leadership styles .72 .59A3 Ability to support shared team values .66 .62A4 Ability to facilitate conflict resolution .68 .68

B. Communication skills .79 .78B1 Ability to write clearly .61 .51B2 Ability to speak effectively to groups .57 .43B3 Ability to use active listening skills .48 .45B4 Ability to explain technical concepts to nontechnical people .17 .44B5 Ability to communicate at the correct level of detail .06 .50B6 Ability to manage communication flow in teams .27 .40B7 Ability to write an executive summary .28 .42

C. Interpersonal skills .79 .81C1 Comprehension of differences among people .42 .70C2 Ability to relate to people with diverse backgrounds .42 .78C3 Ability to build effective teams .47 .47C4 Ability to solve conflicts .49 .43

D. Analytic skills .68 .80D1 Comprehension of quantitative problem-solving techniques .50 .56D2 Ability to apply the right tools to business problems .59 .56D3 Comprehension of the accuracy and reliability of data .73 .59D4 Ability to think systematically. .36 .59D5 Ability to identify relationships between problems and/or Issues .53 .57

E. Decision-making skills .83 .81E1 Ability to use decision-making techniques to solve problems .49 .33E2 Knowledge of negotiating skills and techniques .67 .63E3 Ability to anticipate and provide alternative solutions .66 .52E4 Ability to identify central issues of a problem. .52 .34E5 Ability to incorporate market and competitor information into analysis. .34 .27E6 Ability to evaluate risk involved in decisions. .48 .31

F. Technological skills .86 .87F1 Ability to use word processing .72 .66F2 Ability to use spreadsheets .77 .79F3 Ability to use databases .69 .75F4 Ability to prepare multimedia presentations .75 .76F5 Ability to search and integrate multiple data sources .70 .66F6 Ability to communicate electronically .66 .26

G. Global economy .88 .82G1 Comprehension of cultural differences .71 .70G2 Comprehension of economic differences .71 .79G3 Comprehension of the global business environment .77 .70G4 Comprehension of the impact of other economic systems on the U.S. economy .69 .63

H. Ethics .86 .89H1 Ability to recognize ethical conflicts in personal situations .75 .78H2 Ability to recognize ethical conflicts in business situations .84 .86H3 Ability to make ethical decisions .72 .78

I. Business practices .81 .83I1 Ability to conduct a business meeting .14 .49I2 Ability to analyze industry trends .68 .66I3 Comprehension of market-based economies .77 .68I4 Knowledge of the interdependence of business functions .69 .70I5 Comprehension of basic business practices .72 .61I6 Ability to focus on customer needs .57 .47

NOTE: Figures in italics indicate questionable loading for a priori dimensions.




Factor Analysis

An exploratory factor analysis would help to confirm thecomposite categories developed, and it might also show someunderlying dimensions not obvious in the scores. Followingconventional procedures (cf. Hair et al. 1998), the factor anal-ysis considered the difficulties of a large number of variables(45 in each procedure) and a moderately high number of a pri-ori dimensions (nine expected). With a rule of thumb that thesample size needed to support a factor analysis should be aminimum of 5 (more acceptably 10) times the number of vari-ables (Hair et al. 1998), the full sample of all students, includ-ing nonbusiness majors, seemed likely to support factoring.Because external employers were heavily involved in out-come development and the literature is consistent on hiringcriteria without regard to major, using all of the students in thesample for exploratory factor analysis was considered appro-priate. The sample of all respondents (N = 502) was reducedby incomplete data to a total of 461.

With a few exceptions, nine factors emerged, suggestingthat the a priori composite categories were relatively distinctfor both perception of skill level and importance of the skills(Table 7). Exceptions were most obvious for the importanceratings of communications where mixed results indicated thatboth “ability to communicate correct level of detail” and“ability to explain technical concepts to nontechnical” loadedon the leadership dimension, although the loadings were lessthan the .30 criterion. However, the criterion “ability to man-age communication flow in teams” clearly loaded on the lead-ership dimension at .37. The criterion “ability to write anexecutive summary” loaded on the communications dimen-sion, but its loading of .28, although the highest loading of anyfactor, did not meet the minimum criteria. The same lack ofsufficient loading on any dimension occurred for the businesspractices criterion of “ability to conduct a meeting” and forthe technology skill level of “ability to communicate elec-tronically.” But the decision-making skill level of “ability toincorporate market and competitor information” loaded nicely(.43) on the dimension of business practices.

Factor reliability was evaluated using coefficient alphacalculations (Table 7). Each of the outcome categories showeda coefficient alpha of .75 or greater for both importance andskill level except for analytic skills importance (.68). Withoutthe criterion “ability to think systematically,” the coefficientrises to the level of the other categories. To further examinestability of the factors, split sample factor analyses yieldedmixed results. This led to the conclusion that this number ofvariables (45) and factors (9) required larger numbers ofrespondents to remain stable than was available in one-half ofthis sample (230 students in each half). However, there weresome strong consistencies. Within all of the analyses per-formed, composite learning outcome areas that were univer-sally factored were leadership, technological, global econ-

omy, ethics, and business practices. Depending on the split ofthe sample, elements of communications, interpersonal, ana-lytic, and decision-making skills might load on other dimen-sions or combine among themselves.

DEVELOPING MANAGERIAL ACTIONS

Composite Outcome Categories

The composite outcome categories were generally closetogether in their ratings, but clear differences exist betweenstudent perceived importance and perceived skill levelachieved. Curriculum evaluators can derive reasonable sug-gestions for managerial actions to support broad-based goalsfrom the results of the composite category ratings and rank-ings (Table 2).

Interpersonal, leadership, global economy, and communi-cation skills appear to be higher needs defined by this sample.Students seemed to feel that they were closer to the level ofskill needed for analysis, technology, and decision making.Prior emphasis on these issues has made students more confi-dent of their abilities. These serve as the basis of the businesspractices that were considered an area of moderate need withpriority indexes near the middle of the ranking. The issues inthe middle group also appear to have some connections whereproper ethics and a good grasp of global issues will aid goodbusiness decisions. Emphasis of these areas seems to be pre-ferred by this sample before more emphasis is placed on ana-lytic areas.

The factor analyses (Table 7) provided some confidencethat the composite issues were distinct, but the variations withlower numbers of subjects suggests that some of the dimen-sions were somewhat related. Communications and interper-sonal facets might naturally fit together in curricular develop-ment as might analytic and decision-making items. Also byproviding more training in communications and personalinteractions, the students may feel more capable whenpresented with leadership training especially for groupprocesses.

Individual Outcomes

The highest ranked importance-performance issues indi-cated a mixture of original categories but point toward theability to operate in work environments. Various interper-sonal, communication, and leadership skills seem to beemphasized for items of highest need. Curriculum developersmight consider programmatic changes that introduce theseindividual outcomes to the students early in their academicexperience. Policies and techniques of reinforcing theseissues through the students’ curricula can be established toprepare them for the substantial load of group work normallyexpected in senior marketing courses. A good example of apotential model to follow is the “communications across the

214 DECEMBER 2002




curriculum” concept, whereby English writing and speakingare emphasized by courses throughout the course of study (cf.Corbin and Glynn 1992). Such issues as conflict resolution,negotiating, leadership styles, and team leadership might bereasonably introduced early in a student’s career and empha-sized during upper division courses. Similarly, understandinggeneral business practices might be viewed as an evolutionfrom an introduction into the interactions of business prac-tices in a lower division course. More interaction with realbusinesses will help make connections between classroomwork and actual business practices, but this interaction is notalways practical with larger classes or campuses not locatedin urban areas.

Skills related to the analysis of information and its use inmaking decisions are lower priorities for this sample of stu-dents. Even so, the differences between importance and per-ceived skill indicated that greater training is needed. Focus onthe outcomes derived from employers may provide opportu-nity to shift resources in favor of these items rather than con-tinuing to concentrate on items not highly valued by thosewho hire the students at the end of their academic careers.

Using scores for each criterion rated, the learning out-comes could be evaluated and addressed in a curricularchange. But noncurricular options include a training programthat is provided without grade but required as a part of studentdevelopment. Some MBA programs have required modulesthat introduce students to such diverse issues as industrybriefings, interviewing skills, table manners, and businessdress etiquette. The intent of these programs is not only todeliver content knowledge but also to increase student self-confidence. The key is that creative solutions may not be lim-ited to conventional curriculum courses.

LIMITATIONS AND FUTURE RESEARCH

Each application of outcome development and assessmentpresents challenges as well as opportunities for new investi-gations. In this case, issues for limitations and future researchincluded ensuring the inclusion of crucial specific criteria(like creativity), student assessments, comparisons with otherstakeholder groups, comparisons across time, comparisonswith other universities, sample sizes, and verification ofdimensions.

In the process of developing outcome criteria, some issuesthat are crucial to student success may not emerge from theconventional group processes. Prior to involving other multi-ple sets of stakeholders, these crucial criteria must be identi-fied for inclusion. During the processes for this study, intu-ition and creativity did not expressly emerge in thedevelopment of outcome items. The importance of theseissues has been developed in prior studies (Gilbert, Prenshaw,and Ivy 1996; Ramocki 1994; Shipp, Lamb, and Mokwa1993). Future research should expressly impose these issues,and others that may be identified as crucial, in evaluation,

even though they may not emerge from companies or facultysuggestions.

Student perspectives may be less developed by their lim-ited exposure to work environments, but their perceptions arecrucial to outcomes assessment. Confidence in a skill area is agood indicator if and only if the students understand theissues involved. Importance for each criteria can indicate theneed for emphasis to be added (or subtracted) from the curric-ulum elements. Caution must be exercised in making assump-tions on these ratings without examining the underlyingissues that influence the opinions. Other indications, such asexit surveys or alumni assessments, are often available for aquick reference, but some issues may require added researchfrom focus groups of students or other qualitative methods touncover important issues.

Additional assessment methods can contribute signifi-cantly to program assessment by comparing student percep-tions with various other groups. Multiple years of student per-ceptions can be used to track trends in perceptions due toeither curricular changes or external influences on studentopinion. As one measure of effectiveness, this tends to showthe improvement or degradation of the program on each crite-rion. Another comparison is with the perception of otherstakeholder groups such as employers/recruiters, faculty, oragencies who have interest in learning outcomes. Perceptionsof these groups can show both content needs and communica-tion needs. That is, employers may see needs for more ana-lytic training, whereas faculty or advisory boards may feelthat interpersonal skills need improvement. The results mayshow a true dilemma in program resource allocation, or theymay show a lack of good communications with stakeholdergroups. Comparisons with other programs within the sameinstitution or programs in other universities are always attrac-tive for benchmarks of improvement. Again, caution must beused when questions or groups are not precisely comparable.

Sample size requirements vary depending on the use of thedata. In this case, basic exploratory comparisons of meanswere supported by the number of responses obtained. Moresophisticated analyses (such as factor analysis) may requiremore respondents when reliability tests (such as split sampletechniques) with a large number of criteria and a prioridimensions require large sample sizes. Advanced techniquesof data analysis may help to evaluate the potential for under-lying structure for any set of criteria developed.

SUMMARY

This study discusses the process used for developinglearning outcomes and the illustrative analysis of student per-ceptions as one step in the process of implementing a learningoutcomes approach to curricular design. Developing learningoutcomes for the composite marketing program is an impor-tant step in evaluating the success and the relevance of theprogram. Following the steps outlined in previous literature





helps to begin developing the outcomes. This study illustrateda potential method of applying the analysis of student percep-tions within the evaluation process. Applying this methodol-ogy can improve the faculty’s understanding of the students’feelings about the program and can point toward improvedcommunication of successful learning outcomes as well asareas for improvement with students and other stakeholders.Although this is not the only evaluation technique andalthough this technique is not always definitive in its out-come, it can significantly contribute to program assessment.

REFERENCES

American Assembly of Collegiate Schools of Business (AACSB). 1993.Achieving quality and continuous improvement through self-evaluationand peer review: Standards for accreditation in business administrationand accounting. St. Louis, MO: American Assembly of CollegiateSchools of Business.

Arora, Raj, and Charles Stoner. 1992. The importance of skills of MBA stu-dents seeking marketing positions: An employer’s perspective. Journalof Marketing Education 14 (summer): 2-9.

Bloom, Benjamin. 1956. Taxonomy of educational objectives. New York:McKay.

Carman, James M. 1990. Consumer perceptions of service quality: Anassessment of the SERVQUAL dimensions. Journal of Retailing 66 (1):33-55.

Chonko, Lawrence B. 1993. Business school education: Some thoughts andrecommendations. Marketing Education Review 3 (spring): 1-9.

Churchill, Gilbert A., Jr. 1979. A paradigm for developing better measures ofmarketing constructs. Journal of Marketing Research 16 (February): 64-73.

Corbin, Steven B., and Karen A. Glynn. 1992. Writing in the marketing cur-riculum. Journal of Marketing Education 14 (summer): 46-52.

Cost, Doris L., Marcia H. Bishop, and Elizabeth Scott Anderson. 1992.Effective listening: Teaching students a critical marketing skill. Journalof Marketing Education 14 (spring): 41-45.

Cronin, J. Joseph, Jr., and Steven A. Taylor. 1994. SERVPERF versusSERVQUAL: Reconciling performance based and perceptions-minus-expectations measurement of service quality. Journal of Marketing 58(1): 125-31.

Duke, Charles R., and Margaret A. Persia. 1996. Performance-importanceanalysis of escorted tour evaluations. Journal of Travel and TourismManagement 5 (3): 207-23.

Duke, Charles R., and Richard M. Reese. 1995. A case study in curriculumevaluation using strategic and tactical assessments. Journal of Educationfor Business 70 (6): 344-47.

Fishbein, Martin, and Icek Ajzen. 1975. Belief, attitude, intentions, andbehavior. Reading, MA: Addison-Wesley.

Fisk, Raymond P., Stephen J. Grove, and Joby John. 2000. Interactive servicelearning. Boston: Houghton Mifflin.

Floyd, Callum J., and Mary Ellen Gordon. 1998. What skills are most impor-tant? A comparison of employer, student and staff perceptions. Journal ofMarketing Education 20 (2): 103-9.

Geiger, Joseph J., and B. Dangerfield. 1997. An analysis of integrated curric-ulum models in U.S. colleges of business. Proceedings of the AnnualMeeting: Western Decision Sciences Institute 26:128-33.

Giacobbe, Ralph W., and Madhav N. Segal. 1994. Rethinking marketingresearch education: A conceptual, analytical, and empirical investiga-tion. Journal of Marketing Education 16 (spring): 43-58.

Gilbert, Faye W., Penelope J. Prenshaw, and Thomas T. Ivy. 1996. A prelimi-nary assessment of the effectiveness of creativity training in marketing.Journal of Marketing Education 18 (fall): 46-57.

Glynn, Karen A., K. N. Rajendram, and Steven B. Corbin. 1993. Perceptual-based student outcomes assessment process in the marketing curriculum.Journal of Education for Business 69 (1): 11-15.

Gomes, Roger, Gregory M. Pickett, and Charles R. Duke. 1992. Broadeningthe marketing curriculum with high technology: An academic responseto “world class” industrial evolution. Journal of Marketing Education 14(fall): 15-23.

Hair, Joseph F., Rolph E. Anderson, Ronald L. Tatham, and William C.Black. 1998. Multivariate data analysis. Upper Saddle River, NJ:Prentice Hall.

Hall, Mark, Gerald Stiles, John Kuzma, and Kevin Elliot. 1995. A compari-son of student and employer expectations with regard to business intern-ships. Marketing Education Review 5 (fall): 43-50.

Heinfeldt, Jeff, and Fran Wolf. 1998. Re-engineering the business curricu-lum: A stakeholder paradigm. Journal of Education for Business 73 (4):198-201.

Hopkins, J. Roy. 1999. Study abroad as experiential learning. Liberal Educa-tion 85 (summer): 36-41.

Kaynak, Erdener, Ugur Yucelt, and A. Tansu Barker. 1990. Internationallyoriented marketing curriculum development: A comparative study ofCanada, the U.S.A., and New Zealand. Journal of Marketing Education12 (fall): 53.

Kelley, Craig A., and Ralph M. Gaedeke. 1990. Student and employer evalu-ation of hiring criteria for entry-level marketing positions. Journal ofMarketing Education 12 (fall): 64-71.

Koch, Akam J. 1997. Marketing curriculum: Designing its new logic andstructure. Journal of Marketing Education 19 (fall): 2-12.

Kotler, Philip. 2000. Marketing management: Analysis, planning implemen-tation, and control. Upper Saddle River, NJ: Prentice Hall.

Lamb, Charles W., Jr., Shannon H. Shipp, and William C. Moncrief III. 1995.Integrating skills and content knowledge in the marketing curriculum.Journal of Marketing Education 17 (summer): 10-20.

Lamont, Lawrence M., and Ken Friedman. 1997. Meeting the challenges toundergraduate marketing education. Journal of Marketing Education 19(fall): 17-30.

Lundstrom, William J., and Steven White. 1997. A gap analysis of profes-sional and academic perceptions on the importance of international mar-keting curriculum content and research areas. Journal of Marketing Edu-cation 19 (summer): 16-24.

Marshall, Greg W., Felicia G. Laask, and Jerry R. Goolsby. 1996. Integratingquality improvement tenets into the marketing curriculum. Journal ofMarketing Education 18 (summer): 28-36.

Martilla, John A., and John C. James. 1977. Importance-performance analy-sis. Journal of Marketing 41 (1): 77-80.

Martin, James H. 1990. Integrating ethics into the marketing curriculum.Journal of Education for Business 65 (7): 322-27.

Mason, J. Barry. 1995. Marketing education in the 1990’s: A dean’s perspec-tive and prospective view. Marketing Education Review 5 (spring): 1-8.

McDaniel, Stephen W., and J. Chris White. 1993. The quality of the aca-demic preparation of undergraduate marketing majors: An assessment bycompany recruiters. Marketing Education Review 3 (fall): 9-16.

McMartin, Flora. 1999. A collaborative role for industry in assessing studentlearning. Annual Forum of the Association for Institutional Research.Seattle, WA: Association for Institutional Research.

Owlia, Mohammad S., and Elaine S. Aspinwall. 1996. A framework for thedimensions of quality in higher education. Quality Assurance in Educa-tion 4 (2): 1-20.

Palmer, Adrian, and Catherine Cole. 1995. Services marketing: Principlesand practice. Englewood Cliffs, NJ: Prentice Hall.

Pharr, Steven, and Linda J. Morris. 1997. The fourth generation marketingcurriculum: Meeting AACSB’s guidelines. Journal of Marketing Educa-tion 19 (fall): 31-40.

216 DECEMBER 2002




Ramocki, Stephen P. 1994. It is time to teach creativity throughout the mar-keting curriculum. Journal of Marketing Education 16 (summer): 15-25.

Roach, Scott S., Mark W. Johnston, and Joseph F. Hair Jr. 1993. An explor-atory examination of teaching styles currently employed in marketingeducation: Developing a typology and its implication for marketing stu-dents. Journal of Marketing Education 15 (fall): 31-38.

Ruhland, Sheila K. 1991. Marketing education program evaluation: Imple-menting recommendations. Marketing Educators’Journal 17 (1): 29-41.

Shipp, Shannon, Charles W. Lamb Jr., and Michael P. Mokwa. 1993.Developing and enhancing marketing students’ skills: Written and oralcommunication, intuition, creativity, and computer usage. MarketingEducation Review 3 (fall): 2-8.

Sterngold, Arthur H., and Janet M. Hurlbert. 1998. Information literacy andthe marketing curriculum: A multidimensional definition and practicalapplication. Journal of Marketing Education 20 (fall): 244-49.

Tomkovick, Chuck, Robert C. Erffmeyer, and Gregg Hietpas. 1996. Evalu-ating entry-level sales applicants: An application of policy capturing bycollegiate recruiters. Marketing Education Review 6 (fall): 29-40.

Turley, L. W., and J. Richard Shannon. 1999. The international marketingcurriculum: Views from students. Journal of Marketing Education 21(3): 175-80.

Turnquist, Philip H., Dennis W. Bialaszewski, and LeRoy Franklin. 1991.The undergraduate marketing curriculum: A descriptive overview. Jour-nal of Marketing Education 13 (spring): 40-46.

Ward, Bart, and William D. Chandler. 1999. Applying quality managementconcepts to managing business schools. S.A.M. Advanced ManagementJournal 64 (fall): 21-24.

Webb, Marion S., Kenneth R. Mayer, and Virginie Pioche. 1999. An analysisof U.S. business schools’ catalogs, application packages, and programmaterials from and international perspective. Journal of Marketing forHigher Education 9 (fall): 39-47.

Zeithaml, Valarie A., Leonard L. Berry, and A. Parasuraman. 1996. Thebehavioral consequences of service quality. Journal of Marketing 60(April): 31-46.





learning outcomes: comparing student perceptions of skill level and importance

Documents