Transcript

Empiricism and the Art of Teaching

Josef M. Broder*

“Teaching is a messy, indeterminate, inscrutable, often intimidating, and highlyuncertain task. ”

Richard Elmore

Introduction

Effective teaching is a recurring topic offaculty discussion and disagreement. The title ofmy address suggests that effective teaching has twocomponents. First and increasingly important,teaching has an empiricaJ component. Theempiricism of teaching asserts that there areidentifiable traits of effective teaching that can beused to improve one’s teaching experience. I wantto share with you some insights we have gainedfrom recent empiricaJ studies on teaching and theteaching evaluation process. Second, there is the artof teaching or the intangible and creative componentof teaching. I will speak on how the art of teachingcan be refined. This I will do by way of offeringsome personal teaching tips that have at least mademy teaching experience more enjoyable, Mypresentation will proceed M follows. I will beginwith one of my more disheartening experiences asa graduate teaching assistant. Next, I will discusssome basic assumptions about teaching at researchinstitutions, Here, I discuss the unique role ofteaching, its critics, its limitations, and sources ofimprovement. Next, I will review some of themajor findings of empirical studies. Last, I willoffer personal and professional recommendations forteaching effectiveness,

On this reluctant occasion I am remindedof my early teaching experiences in graduate school.I volunteered to be a teaching assistant in amarketing class for Michigan State University’s

two-year AG TECH Program. I remember thesestudents as being rather atypical college students bytheir willingness to express their opinions ofclassroom affairs, In this particular class I hadassigned a term project. Students were asked toresearch a topic, write a report and make formalclass presentations. These tasks proved ratherdifficult for one of my more outspoken students,Tim, I believe was his name. At first he resistedthe assignment and later had a penchant of askingsome “off the wall” questions. I encouraged himto do his best, despite my lack of teaching

experience. The day came for Tim to give hisformal class presentation. A hectic day I recall. Istayed late for one of my own classes and came tenminutes late for class. When I arrived, the classwas waiting patiently. Tim’s presentation was thelast of four that day. Time expired and Tim’spresentation had to be cut short. His presentationwas good, but the bell rang before class discussion.I thought nothing more about it until I received myend-of-term evaluations. My teacher ratings weregood with one rather startling exception. Onestudent rated me poor on alt categories, andprovided one of the most discouraging assessmentsof my teaching career. The evaluation read,

“This is the worst teacher I haveever had in my entire life. Infact, this teacher is so bad 1wouldn ’t even recommend him tomy dog!”

*Josef M. Broder is professor, Department of Agricultural and Applied Economics, Umversity of Georgia, Athens.Presidential address to the Southern Agricultural Economics Association Meeturg, Nashville, Tennessee, Februaw 6-9, 1994. The author would like to thank Bob Shulstad, Steve Turner, Fred Wh]te, Helen Fosgate, Ralph Christy,Oral Capps, and Sandra Batie for their helpful comments and suggestions.

.J. Agr. and Applied &on. 26 (l), July, 1994: 1-18Copyright 1993 Southern Agricultural Economics Association

. thder Elnpmcwn and t)le AM OJTeackingL

With these comments one wonders why I havespent the better part of my professional career inteaching, Curiosity, embarrassment, guilt,frustration, or revenge, I am at a loss to say, Irmvcr got to thank that particular student for his

mslghts, It may not even have been Tim. Adversilyis truly one of life’s greatest teachers, Today’spresentation is motivated by the spirit ofovercoming the frustrations of teaching.

Basic Assumptions About Instruction

The environment m which teaching isconducted at research r.tmversitics ignites the crmx

of higher education and provides a setting for itslimitations and source of improvement. Some basic

assumptions about this environment are presented.

Teaching is the only non-proprietaryactivity of higher education, William Prokasy, ourVice President for Academic Affairs, argues that[post-secondary] teaching is the exclusive domain ofuniversities and colleges. As monopolists, wc havedone little to improve the quality of our teachingproduct, Research and service activities areconducted by other public and private institutions.Because of competition from other institutions, wehave been far more aggressive and innovative in ourresearch and service activities than in our teachingaetlvmes.

Public scrutiny of public institutions, ingeneral, and universities, in particular, hasintensi$ed. The assessment arena for highereducation is no longer the exclusive domain offaculty members. The universities’ monopoly onteaching is being challenged. Here are someexamples. 1 worry when legislators learn that I’min the classroom only two hours each day; whenlegislators oppose sabbatical leaves on the groundsthat the ~axpayer should not pay university facultyto take vacations; when Susie has to buy a sectionenrollment card from another student to enroll in arequired course; when Tim cotnplains about hisclass of 300 students and his TA who has troublespeaking English; when transfer students strugglethrough our basic courses, despite having had theprerequisites; when business schools don’trecognize agricultural economics courses as meetingthe necessary business requirements; and when

agribusiness employers recruit from businessschools instead of from our department.

The documentation of teaching quality inagricultwal economics departments is inadequatefor the promotion and tenure process at mostuniversities (Kahl and Williams). The problemsand limitations of documenting teaching quality inour departments result from the following. First,tcachmg performance receives much poorerdocumentation than does research in promotion andtenure dosslcrs (Louis). Second, the process ofusing student evahmtions for measuring teachingperformance is highly subjective and controversial(Machina). Third, the teaching evaltmtions used inagricultural economics and related departments arebetter suited for personnel decisions than forimproving teaching performance of individualfaculty members, Fourth, the methods and rigoragricultural economists have applied to problems ofagriculture, food, and rcsourccs have not beenapplied to their teaching evaluation process.

Good classes are distinguished from poorones not by instructional magic, but by a set ofidentifiable practices (Garvin, p. xxii). While theelements of good classes and good teaching cannotbe completely codified, certain practices andtechniques seem to be more effective than others.Knowing how various techniques affect teachingquality is the motivation for the empiricism ofteaching. This assertion does not dismiss the artisticpart of teaching, but suggests that the ant andscience of teaching are interdependent.

The Southern Agricultural EconomicsAssociation (SAEA) should take an active role inpromoting teaching qua/ity. In recent years theSAEA has taken an active role in promotingteaching quality. Yet, the proportion of theAssociation’s resources devoted to teachingactivities is small relative to faculty appointments inteaching. With the recent change in the name ofour journal, the SA13A is in a unique position totake a more aggressive role in promotmgprofessional dialogue on teaching.

The Origins of Poor Teaching

Richard Elmore of Harvard’s GraduateSchool of Education writes that universities seek

J. Agr. and Applied Econ,, July, 1994 3

shelter from public scrutiny of teaching quality byconstructing the following defenses (p, x),

1. Professors are hired to profess, not reallyto teach. Unlike primary and secondaryschool teachers, we are trained almostexclusively in subject matter and notmethods of teaching. Since we know somuch, our task is to tell students everything we know. Of course, we assume thatstudents want to learn everything theprofessor knows or has to say.

2, College studenti are motivated more by adesire to master the subject matter andless by how the subject matter ispresented. We assume that students whoattend college are motivated by anappreciation and respect for knowledge.While bells and whistles are needed tomotivate students in primary and secondaryschools, our students are only interested inthe subject matter.

3. Teaching is a g~t that descends fromheaven onto the shoulders of a fewamong us. That is, good teaching is suchan integral part of one’s personality that itcarmot be taught. Consequently, as long asa department has its share of goodteachers, other faculty shouldn’t be overlyconcerned with teaching quality. Granted,while some faculty are better teachers thanothers, there is no reason why all facultycan’t be good teachers,

4. D~ferences in teaching are matters oftaste and style. That is, differences inopinion on teaching quality are highlysubjective and matters of personalpreference. When teaching performance isdefined in this context, faculty are reluctantto criticize a colleagues’ style of teaching,even when that particular style is thesource of poor teaching performance.

The origins of poor teaching can be tracedto periods of prosperity and expansion in highereducation. A period when students and fundingwere plentiful, A period when there was littlepublic scrutiny of higher education. Times have

changed. Universities have become big business.Robert Hemenway, Chancellor of the University ofKentucky, has argued that modem researchuniversities are at risk of losing their intellectual,political, and moral authority. Increasingly, wehave to compete with pollsters, celebrities, and talkshow hosts in the intellectual marketplace.Politically, we compete with prisons and highwaysfor public monies. Much of the public’s knowledgeof Universities is limited to football statistics. Welost the moral high ground when we purged ourstudies of ethical and normative questions in thename of scientific rigor. The political correctnessthat has filled this moral void has garnered littlesupport from the public. Universities have passedinto a new age of accountability, an agc in whichpoor teaching cannot be defended.

Measuring Teaching Quality?

Despite a growing concern for teachingquality, there is a reluctance to ask the difficultquestions as to what makes up good and poorteaching. How does an individual or departmentadopt a strategy for effective teaching? Muchdispute arises over the qualifications of theevaluator or who is best qualified to judge teachingquality. Here, reasonable educatom will disagree.A few years ago, Bill Taylor and I took acomprehensive look at how agricultural economicsdepartments evaluate teaching (Broder and Taylor).Table 1 shows that student evaluations of teaching(SETS) are the primary method used to evaluateteaching in the Southern Region and the U.S. as awhole. Despite the wide-scale use of studentevaluations, the use of student evaluations formeasuring teaching performance is highly subjectiveand controversial (Machina). Critics of studentevaluations have argued that there is little evidencethat teaching evaluation forms and proceduresactually measure or contribute to teaching quality(Braskamp, et al.). One of our concerns was tobetter understand why student evaluations are metwith such skepticism. To answer this question, weexamined the larger incentive and reward structureof our universities. We believe that studentevaluations are discounted in the promotion andtenure process much like teaching is discounted inthat process. The tenuous role of teaching atresearch universities is explored in the followingconceptual discussion.

4 Broder Empiricism and the Art of Teaching

Table 1. Methodsof TeachingEvaIustionUsed by AgriculturalEconomics snd RelatedDepartments in the Southern Region

Frequency

Method Frequently Ckcusionally By Request Never

--------------------------- percent ---------------------------

Student 100.0 -- -- -.

Peer 7.7 23.1 46.2 23.1

Administrative 23.1 23.1 23.1 30.1

AtUnmi 0.0 23.1 7.7 61.5

Industry 0.0 15.4 15.4 61.5

Conceptual Framework

Teaching activities can be examined in thelarger context of allocating faculty resources.Faculty resources are allocated at the margin basedon the perceived costs and benefits of facultyactivities. Yet, the costs and benefits of facultyactivities can’t be known with certainty.Uncertainties lead to market imperfections thatdistort the resource allocation process betweenteaching, rese~ch, and other faculty activities(Broder and Taylor), First, information costs ofdocumenting teaehing and research are different orasymmetrical. That is, research productivity iseasier to document than teaching productivity.Second, unlike research, the benefits of teachingactivities are not fully translated into departmentalrevenue accounts. That is, the benefits of teachingare undervalued compared with those of research.My earlier work with Rod Ziemer showed that theearnings of teachers are significantly less than thoseof researchers, ceteris paribus (Broder and Ziemer).This finding has puzzled me for most of my

professional career,

Many faculty members allocate their timeamong several activities, For example, assume afaculty member allocates his/her time betweenteaching and research activities as shown by thetransformation curves in figure 1. We assume thatfaculty members operate on the frontier of theirtransformation curves rather than at some interiorpoints such as point D. As faculty members move

through the professional ranks we assume theybecome more productive and move to highertransformation curves, Of course, we assume that[full] professors operate on the highest of thesetransformation curves (Pc -Pc’). Teaching andresearch activities generate revenues or rewards thatare shown by the price or revenue line. Optimumallocation of faculty resources is defined by thepoint of tangency between the transformation curveand the price line. Here, the marginal rate oftransformation is equal to the inverse price ratio.

With perfect information and markets forfaculty resources, faculty activities shouldapproximate the theoretical optimum. Distortions ormisallocation of faculty resources occur when thebenefits of teaching are poorly documented or whendepartments and universities cannot capture thebenefits of their teaching activities. In both cases,these market distortions lead to a steeper price lineand a smaller allocation of faculty resources toteaching activities. That is, market distortions mayresult in an under-allocation of faculty resources toteaching. Improved documentation of teaching andits benefits rotates the price line clockwise andresults in more faculty resources devoted toteaching. In the long run, effective documentationof teaching can affect the faculty member’s career(or expansion) path from one devoted primarily toresearch @-R) to one with a greater role forteaching (O-T). Documenting teaching effectivenessas a graduate teaching assistant or assistantprofessor can have an enduring influence on one’sprofessional career.

J. Agr. and Applied Econ., July, 1994

Figure 1. Optimum Allocation of Faculty Resources

Research

Ie

Id

IC

Pc

Do c

Pb

Pa

o Pa’ Pb’ Pc’ Id’

Source: Broder and Taylor

Does the theoretical model conform toreality? Our survey results and personal experiencelend support to this model. Our survey examinedhow departments used student evaluations in theirday-to-day activities. The findings, shown in table2, raise doubts about the Southern Region’scommitment to teaching and its process formeasuring teaching quality, In this survey, weasked department heads to rate their teachingevaluation process. When the Southern Regionwas compared to other regions we noted thefollowing.1 Departments in the Southern Regiontended to be more critical of their studentevaluations than that of all other regions. Southerndepartments were less satisfied with their studentevaluation forms and information obtained fromthese forms.2

If student evaluations are met with suchskepticism, why don’t departments turn to othermethods for evaluating teaching? Second infrequency to student evaluations are peerevaluations. We expected peer evaluations ofteaching to complement student evaluations ofteaching. Instead, we found that student evaluationsare competitive with peer evaluations or thatdepartments that relied more on peer evaluations

10’ ic’ Teaching

were less dependent on student evaluations andvisa-versa (Broder and Taylor). So, why aren’t peerevaluations used more extensively in ourprofession? A survey of the literature offers someexplanations. While peers are well-qualified toassess a teacher’s knowledge of subject, they areless able to judge teaching effectiveness or howwell the teacher is viewed by the class. Peerevaluations give formative information about coursecontent and structure while student evaluations offersummary information about teacher and courseeffectiveness. Of the teaching evaluationinstruments, student evaluations are unique in beingable to measure the affective results of teaching orhow students feel/appreciate the teacher or class.

We were puzzled to learn that, especiallyin the South, student evaluations tended to havemore influence on promotions and tenure decisionsat the department level than at the college level andeven less so at the university level. Kenneth ClrOs

Louis offers some explanation for this phenomena.He argues that the validity of student evaluationsdiminishes as dossiers move through the promotionand tenure process because teaching excellence isdifficult to ascertain from student evacuations alone.

Broder: Empiricism and ~heArt of Teaching

Table 2. Assessments of Student Evaluations of Teaching by Agricultural Economrcs and RelatedDepartments in the Southern Region

Region

Item Southern Other t-score’

---- Likert scale’ ----

SETS are your primary methodfor evaluating teaching 7.31 7.78 0.75

SETS influence promotion and

tenure decisions at the:

Department level 6.85 7.57 1.00

College level 5.92 7.29 1.90*

University level 5.23 6.81 1.97*

SETS influence salary adjustments 6.38 7.31 1.44

SETS influence teaching awards 8.08 8.44 0.63

Faculty are satisfied with SET form 5.67 6.60 1.62

Faculty are satisfied with procedurefor administering SETS 7.25 8.08 1.98*

Statistical summaries provide usefulinformation 7.OQ 8.17 1,85*

Your department assigns teacherson basis of SETS 4.42 5.17 0.84

Teachers are sensltlve to SET results 6.92 7.42 0.72

Teuching is the primary missionof your department 4.31 5.25 1.16

‘Means differences are statistically significant at the alpha = 0.10 level (*)

bBased on a scale of 1 to 10 where 10 = strongly agree and 1 = strongly dwigree

My experience on college and university promotioncommittees shows:

1. A lack of uniformity in studentevaluation forms and procedures

2. A lack of standards or guidelinesin reporting student evaluationresults

3. The tendency for all faculty tohave above-average studentevahfations

4. Under-reporting of teachingactivities relative to researchactivities

5. Faculty skepticism about thevalidity of student evaluation data

The low priorities that research universitiesplace on teaching may result more from poor andunimaginative documentation of teaching than frominherent biases against teaching. We simply have tolearn to do a better job documenting excellence inteaching.

Refining Student Evaluations

In surveying student evaluations acrossdepartments, the variety in student evaluationsforms, questions, and statistics is striking. Why isthere such variety in the student evaluation processthat purports to measure essentially the same thing,teaching quality? Upon closer inspection, studentevaluations in the Southern Region ask twenty

J. Agr, and Applied Econ., July, 1994 7

questions, including ten on the instructor, six on thecourse, three on student impact, and two on studentcontrol (table 3). Students responding to thesequestions (table 4) were given from four to tenresponse options, were asked to rate, agree, orchoose the description that best fit the question.Responses were alphabetic (a, b, c), mnemonic(good, fair, poor), and numeric (5, 4, 3); and wereon descending or ascending scales. If I didn’t knowbetter, I would guess that the basic studentevaluation forms were designed by a reluctantcommittee of agricultural economics faculty, sometwenty years ago, and have been revised on an adhoc basis by succeeding committees. Since manyagricultural economists lack training in educationalassessment, their efforts are limited by theirtraining.

Once these student evaluation data arecollected, departments produce a paJtry level ofstatistics from these data. Most, but not all,departments produce mean and frequencydistributions of student evaluations (table 5). Halfreport standard deviations and less than half reportcomparative statistics at the department or collegelevel. Given the lack of uniformity acrossdepartments, the lack of comparative statistics is tobe expected. Statistical summaries of studentevaluation data often fail to identify sources orreasons for good or poor results. For example, theteacher might like to know if she was pitching theclass to majors versus non-majors or if studentevaluations were correlated with student effort,expected grades, or class standing. Suchinformation is crucial to developing strategies forimproved teaching. Yet, with few exceptions,departments do little to diagnose their studentevaluations. This is why we argue that studentevaluations are used more for personnel decisionsthan for helping faculty improve their teachingefforts.

What’s Important to Students?

After asking faculty how they felt about theteaching evaluation process, we torned next tostudents. For the most part, students are given noinput into how teaching evacuations are designed oradministered. For example, none of the southernschools made student evaluation data available tostudents, as one might expect from a profession that

teaches the benefits of market information (table 6).The reluctance to involve studenm in the processmay arise because students are not perceived asbeing capable of evaluating teachers. To addressthis issue, Jeff Dorfman and I examined studentevacuations from our department (Broder andDorfman). We began by a rather extensive reviewof the literature. Of course, the experts disagree asto the validity of student evacuations. Much of theliterature is concerned with identifying sources ofbias in student evaluations, including student,teacher, and course characteristics,

The literature offers two responses tostudent evaluation bias. The first response is thatstudent evaluation bias is inevitable and comparableto that found in other forms of evaluation. Criticsargue that student evaluations should be used withcaution in the promotion and tenure process becauseof inherent biases in the process. The secondresponse is that student evacuation bias should beidentified and controlled through statisticaltechniques or replaced with other methods ofteaching evaluation (as though other methods arefree of bias). Our spin on this debate was todistinguish between externaJ and internal bias, andfocus on the latter. External bias refers to student,teacher, or course characteristics that influencestudent evaluations. Intemat bias refers to studentperceptions of teacher and course attributes thataffect their overall student evaluations. That is,what teacher and course characteristics areimportant to students?

We assumed that responses to studentevaluations are influenced by the students’ reasonfor being in the class. Following the human capitalframework, students were assumed to be bothproducers and consumers in the educational process.As producers, students attend class as an investmentin human capitat to enhance their future productiveand earnings capacity, As producers, students’evaluations are based on how much they learn inclass (Nimmer and Stone). We expected studentevaluations to be correlated with questions onlearning such as new knowledge gained and amountof maierial covered. As consumers of education,students derive utility from class attendance and areexpected to evaluate teaching by how much utilitythey gain from being in class, We expected studentevacuations to be correlated with questions on

Broder: Empirici.rmand the Art of Teaching

Table 3. Questions on SET Forms Used by Agricultural EcOnO~cs ~d Relat~ Departmentsin the Southern Region

Region

Questions Southern Other t-score’

Number of questions on:

Instmctor 9.7 12.1 0.42

Course 5.9 8.6

Shrdent Impact 3.4 2.9

Class Control 2.1 3.2

.30

.46

.46

Total 20.4 27.3 1.56

“Means different at alpha = 0.10 level (*)

Table 4. Response Options on SET Forms Used By Agricultural Econonucs and RelatedDepartments in the Southern Region

Region

Nature of Response Southern Other

----- percent -----

Response Options3 0.0 3.2

4 10.0 0.0

5 80.0 67.76 0.0 9.7

7 0.0 16.1

10 10.0 3.2

Response Style:Llkert 10.0 22.6

Rating 70.0 61.3

Mixed 0.0 9.7

Descriptive 20.0 6.5

Response Type:

Alphabetic 30.0 22.6

Mnemonic 40.0 25.8

Numeric 30.0 32.3

Mixed 0.0 9.7

None 0.0 9.7

Respnrw Scale:Ascending 0.0 54.8

Descending 40.0 35.5Mixed 0.0 9.7

J, Agr. and Applied Econ,, July, 1994 9

Table 5. Statistics Derived From SET Data By Agricultural Econormcs and RelatedDepartments in the Southern Region

Region

Statistics Southern Other

Mean respnnse

Median respense

Standard Deviation

Frequency Dlstributlon

Correlation Analysis

Graphical Analysis

Comparisons to:DepartmentCollegeUniversity

----- percent -----

75.0 87.5

12.5 8.7

50.0 52.2

75.0 87.5

14.3 0.0

14.3 0.4

37.5 21,737.5 17,412.5 17,4

Table 6. Availabihty of SET Summaries in Agricultural Economics and RelatedDepartments in the Southern Region

Region

Item Southern Other

Summaries of SET’sare made available to:

Individual faculty 92.3 94.4

Other faculty 7.7 16,7

Department head 84.6 100,0

Deans 38.5 36.1

Students 0.0 8,3

teacher’s enthusiasm and abili~ to stimulatethinking. We used these theoretical expectations tolearn which teacher and course evaluations are mostcorrelated with overall student evaluations andwhether students apply these criteria consistentlyacross teachers and courses, That is, whetherstudent evaluations are arbitrary and capricious.

We examined 200 end-of-term studentevaluations. Ordinary least squares models for

overall evaluations of teacher and course wereestimated. The thirty-four questions on our studentevaluation form were narrowed by the followingcriteria and used as explanatory variables. First, weincluded questions common to student evaluationsat other universities. Second, we included questionsthat captured the theoretical framework. Third, weeliminated redundant questions. Explanatoryvariables were instructor’s knowledge of subject,preparation for class, ability to maintain interest and

10 Broder: Empiric~m and the Art 0$ Teaching

stimulate study, ability to explain subject,organization of lectures, tying information together,and coverage of subjects on exams. To identify therelative contribution of these teacher-courseattributes, we restricted the explanatory variablessuch that their coefficients summed to one. Ourresults are shown in figure 2. We found that 81percent of the explained variation in overall ratingsof the teacher was associated with four instructorattributes: enthusiasm (24!ZO), ability to stimulatethinking (14%), knowledge of subject (23%), andtying information together (20%). We also foundthat while students rate some teachers higher thanothers, they did, in fact, apply these criteriauniformly across faculty. An implication of theteacher model is that students value those attributesthat contribute to their enjoyment of the learningprocess.

Next, we found that 91 percent of theexplained variation in course ratings was associatedwith the following variables (figure 3): newknowledge gained (35%), tying information together(31%) and amount of subject matter presented(25%). As in the teacher model, students appliedthese standards uniformly across courses. Theimplications of these findings are that studentsexpect courses to teach them something. That is,students place value on the course’s conmibution totheir human capital and future earnings capacity.Despite the misgivings of some faculty, studentappear to be rational and consistent in theirevaluations of teaching.

What’s Important to the Profession?

I would like to examine the role of theSAEA in promoting teaching quality. Several yearsago I addressed this group on the SAEA’S role inresident instruction, In that address, I reported onthe status of SAEA’S involvement in teaching-related activities. Five categories of activities wereidentified:

1. Recognition of outstandingteaching

2. Seminars and workshops onteaching

3. Publications on teaching relatedactivities

4. Undergraduate and graduatestudent activities

5. Standing or ad hoc committees oninstruction

Ten years ago, the SAEA’S involvement inteaching-related activities was virtually nonexistent.At that time the SAEA’S involvement in teachingwas the occasional publication of teaching-relatedarticles. A summary of these articles then and noware shown in table 7.

Since that time, the SAEA has taken amore active role in teaching. We now have anannual teaching award and a graduate student papercompetition. Recipients of these awards are askedto showcase their accomplishments in a poster. Iwould like to commend the organizers forestablishing these awards, especially for asking therecipients to share their works with the profession.We are pleased to report that this year’s outstandinggraduate student paper will be published in the Julyissue of our Journal. In 1991, the SAEA sponsoreda pre-conference workshop on teaching, the first ofits kind to my knowledge, Finally, the number ofteaching-related articles in the Journal has tripledfrom our earlier study. To date, the SAEA does nothave a standing committee on teaching. Aspresident, I plan to appoint an ad hoc committee toexplore ways the SAEA can better promote qualityinstruction in our Association.

How good are the teachers in the SouthernRegion? Assessment data on teaching is difficult toobtain. One measure of teaching quality, however,does speak well of the Southern Region. Iexamined the number of AAEA teaching awards byregion over the past fifteen years. As shown intable 8, the South received more than half (53%) ofthe AAEA Teaching Awards for Less than 10 Yearsand one third of the Awards for More than 10Years. Overall, our region has dominated theseawards. The only discouraging note is that we havenot fared as well in the senior award category. Iam concerned that many of our outstanding youngteachers do not sustain their interest and momentumin teaching in their latter careem, This apparentattrition of good teachers should be of concern tothe Association.

J. Agr. and Apphed Econ.,July, 1994

Figure 2. Student Ratings of Instructor Relative Importance of Attributes

Knowledge of subject23%

,::.:.:::::::..F /’ Tying info together

Stimulate thinking14%

Figure 3. Student Ratings of Course Relative Importance of Attributes

Subject matter25%

200/0

‘ledge

Other9%

11

Tying info together31?40

12 Broder: Empiricism and the Ari of Teaching

Table 7. Teacbiog Publications in Journal of Agricuhural and Applied Economicr

Item 1980-84 1989-93

Number of IssuesBeginning Volume - NumberEnding Volume - Number

Total Number ofArticlesAuthorsPages

Teacfring-RelatedArticles (%)Authora (%)Pagea (%)

912-116-1

204422

1498

1.91.81.3

921-125-1

212439

2012

6.64.84.7

Table 8, Amerrcan Agricultural I+xnromrcs Association Teaching Awards by Region

south Northeast North Central West

Number of Departments 13 12 8 17

Number of AwardsJuniofl 8 2 2 3SeniO& 5 1 8 1

Percent of AwardsJunior 53 13 13 20Senior 33 ‘1 53 7

Per DepartmentJunior 0.62 0.16 0.25 0.17Senior 0.38 0.08 1.00 0.06

‘Junior Award is for less than ten years experience.bSenior Award M for ten or more years experience.

The Art of Teaching

When drafting this address I had originallyplanned to close at this time. My colleagues notedthat the address had been devoted to what othersthought about teaching, with few insights from mypersonal experience and observations. My initialresponse was that you would not be interested inpersonal experiences alone, In the end mycolleagues prevailed and convinced me to sharesome personat insights into effective teaching. Indoing so, I recognize that teaching is a highly

personal experience and not often transportable.What works for one individual may spell disasterfor another individual. Or in the words of aneditor’s rejection letter, teaching techniques tend tobe instructor, class, course, or university specific.Given that many of us older teachers are set in ourways, I offer these observations to young teachers.

Graduate Teaching Experience

On the subject of teaching professors howto teach, Elmore writes (p.xi):

J. Agr. and Applied Econ., July, 1994 13

“Professors spend most of theirgraduate education preparing toconduct research, their onlypreparation for teaching is theirown, largely unexamined,experience as students. In thepeculiar world of universities, oneis expected to know how to teachas a condition of employment, butthe practical problems of teachingare almost never discussed. ”

Unlike teachers in primary and secondaryeducation, we receive no formal training on how toteach. Granted, we do more than teach, but theimpacts of our teaching efforts are immense. In oursurvey of teaching evaluations, we found almost noinvolvement by graduate students in the teachingprograms (Broder and Taylor). I find this lack ofinvolvement by graduate students to be a disserviceto young would-be teachers, the department and thestudents themselves. Some of you might argue thatgraduate students lack experience and should not beallowed in the classroom. Our studies have shownthat what graduate students lack in experience, theymake up for in enthusiasm. In an earlier study,Michael Wetzstein and I found no significantdifferences in student ewduations between facultyand graduate students (Broder, et al.)

My experience as a graduate teachingassistant was invaluable and had a profound andlasting impression of what it means to be aprofessor, a scholar, and an agricultural economist.My advice to young faculty is to get involved inteaching while in graduate school. Don’t wait untilyou receive your faculty appointment and have toteach for the first time while struggling to publishand develop new research projects. There areadditional benefits to graduate teaching experience.Graduate teaching assistants do far better on theiroral exams and defenses than their counterparts.Learning and being comfortable with economicdialogue is a valuable asset in our profession.Believe me, I’ve watched some of our top graduatestudents fail oral exams because they had never hadto express their economics verbally.

Rely on Experience

I’ve watched many young and some not-so-young faculty struggle in the classroom. They

suffer, the students suffer, and I have to hear aboutit as Undergraduate Coordinator. Occasionally, Iam asked to diagnose the problem. Often theproblem results from the faculty trying to resolveissues on their own, Often, it’s not sheer studentboredom that causes students to become hostile, it’sa violation of the student contract. Changing therules of the game, the initial contract, being toodogmatic, inflexible, not giving students any benefitof your doubt, is asking for trouble. How can youavoid these pitfalls? My advice is to recognize

1. that you are not the only teacherin the department,

2. that others have probably taughtyour course,

3. that others have more experiencein teaching than you, and

4. your colleagues are more thanglad to help.

Don’t worry about losing control, power, orfeeling inadequate. Its amazing what secondopinions can do in solving our teaching dilemmas.Even department heads can offer some goodinsights into the teaching process. Recognize thatteaching styles are unique to individual personalitiesand avoid the temptation of adopting unfamiliarteaching styles. Recognize and adopt a teachingstyle that is comfortable and seems natural.

Of course, you should also rely on yourown experience. As you gain teaching experienceand accumulate student evaluations, you have avaluable source of data to diagnose your teachingperformance. The literature is quick to point outthat student evaluations alone may offer fewinsights into how to improve one’s teaching(Braskamp, et al.). Ask colleagues to help youinterpret your evaluations, Don’t be fixated by theoutliers, the extreme comments, but focus on themean or trend. If possible, gain access to the rawdata and generate a richer set of statistics. If youare having problems, try to identify which groups ofstudents are being the most critical. Try to identifywho is your audience, or which audiences you havebeen ignoring, Of course, bear in mind that overallstudent evaluations are largely determined by a fewteacher and course characteristics.

14 Broder: Empiricism and the Art of Teaching

Beyond Lecture

A common mistake in many classrooms isthe overuse of lectures to the exclusion of otherteaching techniques. Many of us were taught inlectures and we continue to use this technique, evenin presidential addresses. I am reminded of AlSchmid’s comments on the lecture method, He saysthat lectures are artifacts of the middle ages, beforethere were printing presses. Without printedmateriats, students merely transcribed the teachers’or speakers’ comments. This was not learning buttranscription. Yet today, the practice continuesalmost unabated, while learning suffers and studentsbecome transcribers, and inefficient ones at that.The next time you lecture, take a moment to readsome of the notes your students are taking. You’llbe surprised at the inaccuracies and gaps in theirnote-taking.

My advice to young teachers is to varyyour style. Lectures are effective to a point.Fifteen minutes into the lecture, you have lost halfof your class. Add some variety and diversity intoyour delivery. Ask questions, have students writeand react, be quiet, have students speak, show avideo, or use visual aids. Recognize that studentsget tired of any one teaching technique and tend tolearn more if they can see an idea from differentmediums.

Engage Students

Many of us are overly concerned with thedelivery of factual content. A number of studieshave found that, when lecturing is the dominantmode of teaching, students forget up to 50 percentof course content within a few months (Garvin,p,4). Elmore writes (p. xii), “we have knowledge,only as we actively participate in its construction,”Students learn by engaging, with other students andwith the teacher, in a process of inquiry, criticaldiscourse, and problem-solving. When teachers failto engage students, the entire class experiencesuffers from a tack of what Fred White calls classenergy (White).

Teaching effectiveness also suffers from alack of ownership. Teachers deliver factualmaterials, students transcribe notes, and no oneclaims ownership or responsibility for the processor

the outcome, A classic problem of commonproperty, lack of ownership results in a lack ofresponsibility. Students learn from being activelyinvolved in the learning process and not frommemorizing facts and figures presented in soon-to-be-forgotten lectures, Active learning begins withthe premise that knowledge is not the exclusiveproperty of the faculty member or that knowledge isto be rationed according to rules in a coursesyllabus (Elmore, p.xvi)

,,,, The main value that students

take away from our classes is nottheir knowledge of the subject,but a predisposition to learn. ”

Here, my colleague, Steve Turner, has longargued that to instill in students the predisposition tolearn we should teach them the capacity to learn byconducting independent research (Turner).

In practical terms, I advise you to engageyour students, make them partners in the educationprocess, share the responsibility for learning. Listento what student have to say, don’t assume thatknowledge is a one-way affair, Some of my bestand worst students now serve in the StateLegislature, and I’m glad that they were in myclass, and that they passed the course. Asdescribed earlier, students are rational individualsand attend class for rational reasons. Giving thema voice in the learning process is essential, anddoesn’t hurt your teacher ratings either. Let megive you an example. We have a bright instructorwho was struggling in the classroom. Class moralehad deteriorated to the point where one studentpublicly berated the teacher for a long list ofshortcomings. The teacher was devastated, the classwas in a trauma, and the teaching moment wasgone. He was advised to deal with the crisis and, inthe future, shift the burden of learning from theteacher to the students, He incorporated somemarketing games that engaged the students. Thestudents enjoyed the experience, His student ratingsimproved drastically, not so much for what theteacher had done, but what the students had done,He also gave them the option to drop some of theirworst grades, and that seemed to help.

J. Agr, and Applied Econ., July, 1994 15

Sense of Humor

I’ve been accused by some for not takingsome my professional duties too seriously. At mylast high-school reunion, my teachers were suqwisedto learn that I joined the teaching profession. Ithink a key element of effective teaching is thatteachers maintain a sense of humor. By this I don’tmean telling jokes which have become dangerous inthis age of political sensitivity. By humor I meana state-of-mind that recognizes the limitations of ourknowledge. Humor is a relief from dogma whichstudents find to be difficult, Humor and irony canbe powerful teaching tools when used to challengeand present conflicting points-of-view. Humormakes you appear more human. Without a sense ofhumor, it would be difficult to survive the rigors ofteaching at a research university.3

Civility

Students accept the responsibilities oflearning if the class environment is open and basedon mutual respect. Unknowingly, some facultymembers lose empathy for students that areconfused or slow to comprehend. Some teachingmethods reIy heavily on fear, intimidation, andanxiety. While this approach may work for basictraining in the military, I have serious reservationsabout its usefulness in college education. Negativetechniques may have short term effects on classattendance, i.e. too many absences and the studentfails the course. Classes that use negativetechniques are often unpleasant to both the teacherand students.

In contrast to negative techniques are thepositive techniques that afford students a measure ofrespect and trust. The adage, what goes around,comes around has merit here, If you treat yourclass with respect, they are more likely to directtheir energies at learning than at being hostiletoward you, the text, the course, or the university.By respect, I don’t mean (1) that you should avoidhard choices, (2) that every student will be pass thecourse, or (3) that you defer to the whims of theclass. Instead, you treat them like equal partners inthe educatioml experience. Despite your civility,some will fail your course, but at least they will failwith respect and no animosity toward you,

What have I done lately to enhance mutualrespect in my classes? First, I’ve started takingclass photographs and make an honest effort tolearn more about my students. Class photographsextend the contract beyond the class period and givea sense of permanence to the teaching experience.Second, I have replaced pop quizzes which seemrather punitive, with asking the class what theylearned today and what confused them. Third, Iroutinely ask students about their study habits andrun regression models that explain their examscores. How is your exam score affected byabsences, prerequisites, study habits, grade pointaverage, etc.? Fourth, I make an effort to write abrief note to the parents of my top students,congratulating them on their child’s performance inmy class. The parents and students truly appreciatethis gesture. Follow-up letters from these parentshave been some of the most rewarding mementos ofmy teaching career.

Pedagogy

Having reviewed a number of agriculturaleconomics textbooks, I am struck by their lack ofimagination. This is not a criticism of the authorsbut a problem of conveying the subject matter to anincreasingly diverse student body. These studentshave little or no feeling for the technical side of ourprofession. Tying information logether, relating ourideas to the real world can be frustrating.Fortunately, there are publications on pedagogy thatoffer a variety of insights and approaches toteaching complex economic concepts, Thepublication of which I am most familiar is the GreatIdeas for Teaching Economics, by Byrns and Stone.I highly recommend this book to teachers who wantto revitalize their teaching skills. Other publicationsinclude Journal of Economics Education and theNACTA Journal. In addition, the AAEA and SAEAconduct occasioned teaching workshops that servethis purpose.

Teaching-Related Research

Teaching excellence at- many researchuniversities goes beyond routine classroomexperience. Professional contributions to teachingextend the teaching experience beyond theclassroom. I have had both successes and failuresin publishing teaching-related research. The low

16 Broder: Empiricism and the Art of Teaching

number of such publications in our journals is duemore to a lack of submissions than to any overt biasagainst such articles. Editors and reviewers demandmuch from these articles, They demand that thesearticles (1) be relevant to the profession, (2) bebased on a theoretical framework, (3) be approachedwith scientific (statistical) rigor, and (4) not beinstructor, class, course, or university specific.Quite frankly, I feel than many of our editors wouldlike to publish more teaching-related articles buthave trouble accepting manuscripts that don’t meetthe standards of research articles (for lack of abetter term).

Faculty Renewal

Lastly, I feel that faculty renewal is criticalfor faculty members with heavy teaching loads.Renewal activities range from periods away fromthe classroom, to attending workshops, to sabbaticalleaves. A well-planned sabbatical can be mosteffective. If you are willing to teach some courseswhile on sabbatical leave at another university, yourchoice of leave sites might be greatly enhanced,especially if you are willing to teach theintroductory level courses, Some of you might say,why go on leave just to teach? Hopefully, youwould not teach for the duration for your leave andhave free time to pursue your interests,

Attending teaching workshops can also bequiet rewarding, On occasions you should seek outworkshops outside our profession, Its not that wedon’t have something to say about teaching, but theinteraction with other disciplines is especiallyenlightening. Last year I attended a week-longprogram on Teaching by the Case Method atHarvard University. This was a fine educationalexperience and one that I would highly recommendto faculty wanting to learn more about case methodteaching.

Closing Remarks

in this address, I have stressed the value ofempiricism in teaching excellence. While teachinginvolves personality and behavior that cannot becompletely quantified, much can be learned fromsystematic analysis of the teaching process. First,

we examined how departments assess teachingquality, next we identified teaching characteristicsthat are important to students, and finally, weexamined the SAEA’S contributions to teachingactivities. The SAEA has some outstandingteachers. All of us take pride in doing a good jobin the classroom. Yet, teaching excellence is notthe exclusive domain of the Southern Region, norcan we expect to sustain our reputation without acommitment of professionat resources. I leave youwith some personal observations andrecommendations.

I encourage young faculty to use theiranalytical skills to diagnose their teaching efforts.Discover your comparative strengths andweaknesses. Don’t be afraid to ask your studentsabout your teaching performance. Their input cangreatly enhance the teaching experience for you andthe class, Keep in mind that many of our studentsare surprisingly rational. They attend class for apurpose and they appreciate the high qualityteaching that helps them achieve that purpose.

Agricultural economics and relateddepartments play a vital role in recognizing,supporting, and rewarding teaching. Reasonablefaculty will disagree about how teaching should beevaluated and rewarded. Current methods for doingso are unnecessarily ad hoc and too quicklydiscounted. Departments should take a closer lookat their teaching evaluation process. Dissatisfactionwith the process undermines faculty efforts toimprove their teaching performance. Facultyparticipation in the teaching evaluation processshould be encouraged. Evaluating teaching shouldbe a shared responsibility between teachers,students, and administrators.

I want to commend the SAEA forrecognizing and promoting teaching excellence.The teaching mission is a fundamental part ofagricultural economics. The Association depends onthe financial support of our universities.Undergraduate students and programs drive ourdepartments and, in turn, give purpose to ourAssociation, Activities that promote teaching alsopromote and ensure the long-term viability of theAssociation.

J Agr. and Applied Econ,, July, 1994 17

References

Braskamp, L.A., D.C. Brandenburg, E, Kohen, J.C. Ory, and P.W, Mayberry. Evaluating TeachingEffectiveness: A Practical Guide. Beverly Hills, California: Sage Publications.

Broder, Josef M. and Jeffrey H. Dorfman, “Determinants of Teaching Quality: What’s Important toStudents.” Research in Higher Education, 35-2, (in press).

Broder, Josef M., John W. McClelland, and Michael Wetzstein. “Teacher Popularity and StudentEvaluation of Teachers: Some Preliminary Results.” J. Econ. & Finance, Vol. 6, 1982, pp.471-474.

Broder, Josef M. and William J. Taylor. “Teaching Evaluation in Agricultural Economics and RelatedDepartments,” Amer. J. Agr. Econ., 76-2(Februmy 1994): (in press).

Broder, Josef M. and Rod F. Ziemer. “Determinants of Agricultural Economics Faculty Salaries,”Amer. J. Agr. Econ., Vol. 64, No. 2, 1982, pp. 301-303.

Byms, Ralph T. and Gerald W. Stone (editors). Great Ideas for Teaching Economics, Fourth Edition,Glenview, Illinois: Scott, Foresman, and Company, 1989.

Elmore, Richard R. “Forward.” Education for Judgement, Boston, Massachusetts: Harvard BusinessSchool Press, 1991, pp. ix-xix.

Garvin, David A. “Barriers and Gateways to Learning.” in Education for Judgement, Boston,Massachusetts: Harvard Business School Press, 1991, pp.3- 13.

Hemenway, Robert. “The Basic Secret of Life: Proposals for the University of the 21st Century.” inRethinking the University: Educating for the 21 Century - Proceedings of the University of GeorgiaAcademic Affairs Faculty Symposium, May 7, 1993.

Kahl, Kandice H. and Fred E, Williams. “Panel on Measuring Teaching Quality: Survey and Summary.”in Proceedings of the American Agricultural Economics Association Teaching Workshop, AAEAResident Instruction Committee, (July 1986):27-29,

Louis, Kenneth Gros. “The Renewal and Evaluation of Teaching - Preparing a Teaching Dossier.” inProceedings of the American Agricultural Economics Association Teaching Workshop, AAEAResident Instruction Committee, (July 1986):30-38.

Machina, Kenton. “Evaluating Student Evaluations.” Academe 73-3(May-June 1987):19-22.

Nimmer, J.G. and E,F. Stone (1991), “Effects of Grading Practices and Time of Rating on StudentRatings of Faculty Performance and Student Learning.” Research in Higher Education, 32:195-215.

Prokasy, William F., Address to Faculty Recognition Ceremony, June 6, 1989, University of Georgia,Center for Continuing Education, Athens, Georgia.

Turner, Steven C, “Morale and Faculty Development in Agricultural Economics: Discussion,” J. Agr.Applied Econ., 25- l(July 1993):24-26.

White, Fred C. “Enhancing Class Attendance,” NACTA Journal, 36-4(December 1992):13-15.

18 Broder: Empiricism and lhe Art of Teaching

Endnotes

1. States in the Southern Region were: Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana,Mississippi, North Carolina, Oklahoma, South Carolina, Tennessee, Texas, and Virginia.

2, Mean values for the Southern Region were compared to the mean values in all other regions combinedusing student t-tests.

3. Humor is not exclusive to faculty. Elements of humor are found in some of the more common questionsasked by students. My top-ten list of (unintended) humorous questions asked by students are given below,with all due respect to my past, current, and future students.

10.9.8.7.6.5.4.3,2.1.

“Do we have to buy the book?”

“Do you grade on a curve?“Can we drop a grade?“Do we have to take the final exam?“1 knew the material, I just couldn’ t give it back to you on the exam. ”“Is this going to be on the test?”“I can’ t be in be in class tomorrow, I have to register for classes. ”“I got the exact same answer as my classmate but she got a higher grade. ”“IS this score on my exam, the number right or the number wrong?“I can’ t be in class tomorrow, will you be talking about anything important?”


Top Related