yarbrough and henley

308 JRME 1999, VOLUME 47, NUMBER 4, PAGES 308-318

Seven rehearsal excerpts demonstrating research-identified teaching characteristics were presented to university music majors (N = 176) for ratings. Subjects watched either a videotape focused on the teacher or another focused on students. Numerical

ratings from 1 to 10 were assigned by subjects to 10 categories of student and teacher behavior: time use, musicianship, accuracy of presentation, student attentiveness, student performance quality, enthusiasm, intensity, pacing, personality, and overall

effectiveness. Results demonstrated that subjects gave their highest ratings when view-

ing the teacher and lowest ratings when viewing students. The highest-rated excerpt contained a low percentage of student off-task behavior (6.53 %), a high percentage of approvals (71 %), moderate eye contact (27.30%), many activity changes (27), a

high percentage of student response time (66 %), and rapid pacing, as indicated by an average length of both teacher and student activities of 5-6 seconds.

Cornelia Yarbrough and Paul Henley Louisiana State University

The Effect of

Observation Focus on

Evaluations of Choral

Rehearsal Excerpts Previous observation and evaluation studies have dealt primarily

with teacher effectiveness rather than student performance quality and attentiveness (or motivation and commitment) in rehearsal. To date, few studies have included student achievement as a variable in the study of teacher or conductor effectiveness. Instead, research has been focused on defining, measuring, and evaluating skills that

might demonstrate effective teaching. In most evaluative situations, an observer uses a checklist of

teacher behaviors to determine teaching behaviors. There is an

assumption that the teacher is in control of the teaching and learn-

Cornelia Yarbrough is the Haymon Professor of Music Education at the School of Music, Louisiana State University, Baton Rouge, LA 70803-2504. Paul Henley is an assistant professor of music education in the Department of Music at Western Montana College, 710 Atlantic Avenue, Dillon, MT 59725. Copyright @ 1999 by MENC-The National Association for Music Education.

JRME 309

ing process and that if the teacher demonstrates "good teaching," the students will demonstrate "good learning." There have been few

investigations of the question of whether good teaching can exist in the presence of poor student responses. Is it possible for a teacher to

get a good teaching evaluation if the performance of the students is

poor? In music research attempting to isolate the effect of teaching be-

havior on student achievement, attentiveness, and attitudes, Yar-

brough (1975) concluded that a combination of an approving facial

expression, eye contact, verbal approval, and approach body movement could perhaps be the combination of behaviors resulting in the

highest magnitude and resulting in higher on-task percentages. Price (1983) showed that band directors using sequential patterns with immediate, related feedback were effective in producing good performance while maintaining high student attentiveness and positive student attitude.

Sims (1986) varied teacher affect (high/low) and found that

changes in pupil off-task behavior were more obvious when high affect was followed by low affect than under the opposite condition.

Jellison and Kostka (1989) demonstrated that elementary school students recalled more specific musical information than nonspecific social information when teaching patterns were used. In addition, subjects in both of these studies rated teachers using patterns as

"good teachers," compared to those who did not use them. In a study of the assessment of effective teaching by instrumental

music student teachers and experts, student teachers who identified a large percentage of student on-task behavior in their rehearsal were likely to be rated more effective by music experts than those student teachers who had identified student on-task percentage as being lower (Madsen, Standley, Byo, & Cassidy, 1992). An earlier study sug- gested that attentiveness in rehearsals may be related to teacher attention to students in the form of eye contact (Yarbrough & Price, 1981).

Music education research has demonstrated that when watching videotaped music teaching, observers' reactions vary greatly and appear to be influenced by several factors. The position of the camera (e.g., whether it is focused on the student, the teacher, or both) may affect perception of the feedback that the teacher gave (Duke & Prickett, 1987). Even when instructed to notice both the student and the teacher, observers often pay greater attention to the teacher's behavior than to the student's responses (Duke, 1987; Standley & Greenfield, 1987). Additionally, Prickett and Duke (1992) showed that observers' evaluations of teaching may be affected differently by the observation tasks; that is, both the format and the focus of obser- vations may affect the evaluations.

310 YARBROUGH/HENLEY

The present study is the third in a series of studies designed to study effective choral rehearsal techniques. The first study described teacher behaviors, student attentiveness, and student performance quality across an entire semester of rehearsals (Yarbrough, Dunn, & Baird, in press). Results demonstrated that less off-task behavior was related to higher performance ratings. For the second study, subjects viewed seven videotaped excerpts focused on the teacher and gave their highest ratings to the excerpt demonstrating low off-task student behavior, a high percentage of teacher approvals, and rapid pacing (Yarbrough & Madsen, 1998). The purpose of the present study was to determine whether focusing the attention of evaluators on students versus teacher would affect the assessment of teaching in choral rehearsal situations.

METHOD

Subjects (N = 176) were university music education majors from four large state university schools of music. They were graduate (n =

57) and undergraduate (n = 119) students, vocal (n = 89), instrumental (n = 81), or both vocal and instrumental (n = 3), and males (n = 64) and females (n = 111).1 They were randomly assigned to one of two experimental groups: one group (n = 89) observed and evaluated seven videotaped choral rehearsal excerpts in which the camera focused on the conductor/teacher; the second group (n = 87) observed and evaluated the same excerpts except that the camera was focused on the students singing in the choral ensemble.

Videotaped excerpts were extracted from previously taped rehearsals across one complete semester. One member of a universi-

ty choral music faculty and one university level choral ensemble (n =

75) were preparing music for a concert at the end of the semester. Members of the ensemble were freshman and sophomore music

majors and nonmusic majors at all levels. Excerpts from the rehearsals of two contrasting choral pieces were selected. These pieces were a lyrical folk-song setting of "Shenandoah," arranged by James Erb (SSAATTBB-Lawson-Gould Music Publishers, Inc., Octavo No. 51846), and an up-tempo spiritual, "Daniel, Daniel, Servant of the Lord," arranged by Undine Smith Moore (SATB-Warner Bros. Publications, Inc., Octavo No. W 3475).

Seven excerpts were selected from the videotapes of these rehearsals. Two cameras were used to videotape the rehearsals. One camera was focused on the conductor/teacher; the other was focused on the students in the ensemble. The camera focused on the students was rotated every 15 seconds to capture a different group of students. For each videotape, each excerpt was analyzed in terms of the observation categories given in Table 1. Excerpts were then

JRME 311

Table 1

Comparison of Excerpts by Category

Excerpts

Category: 1 2 3 4 5 6 7

Piece: Daniel Daniel Shenan Shenan Daniel Shenan Shenan Rehearsal Type: R eading Drill Concept Structured Drill Structured Structured

practice practice practice

Student off-task behavior: 11.63% 20.00% 11.63% 14.28% 6.53% 14.49% 15.91%

Rehearsal time: Teacher talk 17.10% 62.50% 58.69% 23.08% 34.04% 26.93% 29.17% Student response 82.90% 37.50% 41.31% 76.92% 65.94% 73.07% 70.83%

Approval/ Disapproval ratio: 50/50% 7/93% 20/80% 60/40% 71/29% 57/43% 40/60%

Nonverbal teacher behavior:

Body movement 4.50% 4.50% 0.00% 58.30% 0.00% 5.00% 45.40% Expressive

conducting 95.50% 0.00% 88.90% 55.50% 0.00% 37.50% 87.50%

Eye contact 4.50% 100.00% 52.00% 20.00% 27.30% 12.00% 33.30% Facial approvals 9.10% 60.00% 4.00% 27.30% 9.10% 4.00% 0.00%

Musical concepts taught: 13 13 27 8 7 15 15

Pacing characteristics:

Activity changes 11 19 9 5 27 19 11 Teacher activities 6 10 5 3 14 10 6 Student activities 5 9 4 2 13 9 5

Mean activity time* 28.00 5.00 34.00 39.00 5.00 18.00 16.00

Mean teacher time 8.67 6.00 35.80 15.00 4.57 9.40 8.33 Mean student time 51.60 3.67 31.50 75.00 5.70 28.33 20.00 Length of excerpt 305.00 88.00 305.00 195.00 138.00 349.00 168.00

Note. Musical works used in this study: Daniel = "Daniel, Daniel, Servant of the Lord"; Shenan = "Shenandoah."

* Time is measured in seconds.


scripted and coded using operational definitions and codes for

sequential patterns. Each coded segment was timed. Excerpts were

categorized as either reading, drill, structured practice, or concept rehearsal. A reading rehearsal was defined as sight-reading the piece without stopping for rehearsal of it; a drill rehearsal was defined as

working a specific, and small portion of the piece over and over until it was solidified; a structured practice rehearsal was defined as one in which the conductor/teacher gave tnd that spot and singing until the conductor stopped and gave another direction as to where to

begin again; and a concept rehearsal was defined as working on specific concepts such as rhythm, dynamics, or intonation throughout the entire piece. Subsequent data demonstrated that the characteristics of the excerpts were as presented in Table 1.

Subjects in groups of 20 or fewer watched either a stimulus tape of the seven excerpts focusing on the conductor/teacher or one focus-

ing on the students in the ensemble. Evaluation forms distributed to

subjects asked for demographic data concerning graduate or under-

graduate status, male or female gender, and instrumental or vocal

major. Subjects were asked to rate the conductor/teacher or the ensemble on 10 scales for each excerpt on a scale from 1 (poor, low, slow, or dull depending on the characteristic rated) to 10 (superb, high, fast, or sparkling). The characteristics rated from poor to

superb included use of rehearsal time, musicianship, accuracy of instruction, student attentiveness, student performance quality, and overall teaching effectiveness. The characteristics rated low to high included enthusiasm and intensity; the characteristic rated slow to fast was pacing; and that rated dull to sparkling was personality. After

giving a numerical rating for each category, subjects were given the

opportunity to write comments about each excerpt. The same evaluation form was used for both videotapes.

RESULTS

Data analyzed represented mean evaluations of the student and teacher focus tapes for the 10 categories of student and teacher behavior and for the seven excerpts. In addition, subjects' comments taken from the evaluation forms were categorized as positive or negative and then counted and summarized. Thus, independent variables for this study included focus of evaluator attention (students or

teacher), gender of evaluator (male or female), level of evaluator

(graduate or undergraduate), major of evaluator (vocal or instru-

mental), university, and the seven different excerpts. Dependent variables were mean ratings for the seven excerpts and mean ratings for the 10 categories (obtained by averaging the ratings of each of the 10 categories across the seven excerpts).

JRME 313

Three separate analyses of variance (ANOVAs) with repeated measures on the mean ratings for the seven excerpts were calculated for

comparisons by (1) gender, (2) major (vocal or instrumental), and (3) level (graduate or undergraduate). An additional three separate ANOVAs with repeated measures on the mean ratings for the 10 cat-

egories were calculated for comparisons by (1) gender, (2) major (vocal or instrumental), and (3) level (graduate or undergraduate). No significant differences were found in any of these six comparisons (p > .05).

Next, an ANOVA was computed comparing mean ratings of the seven excerpts by observation focus (teacher versus student) and the four universities with repeated measures on excerpt ratings. Data demonstrated a significant difference between excerpt ratings given by subjects who viewed the teacher versus those given by subjects who viewed the students in the ensemble, F (1, 168) = 31.96, p = .0001. In addition, there was a significant difference among excerpt ratings by the four universities, F (3, 168) = 6.67, p = .003, and among the seven

excerpt ratings, F (6, 1008) = 48.90, p = .0001. Significant interactions were found between excerpts and focus, F (6, 1008) = 7.03, p = .0001, between excerpts and universities, F (18, 1008) = 2.05, p = .006, and

among excerpts, focus, and universities, F (18, 1008) = 2.59, p = .0003.

Further examination of means for the significant main effects and for the significant interactions revealed the following:

1. Subjects' ratings for the excerpts were significantly higher for the teacher focus tape (M = 6.79) than for the student focus tape (M = 5.83).

2. Ratings by university A (M = 6.77) were significantly higher than

ratings by universities C (M = 5.98) and D (M = 5.90) but not significantly different from university B (M = 6.41); no other significant differences among means were found.

3. The highest rating was given to Excerpt 5 (M = 7.36) and the lowest rating to Excerpt 6 (M = 5.75); the highest rated excerpt contained a low percentage of student off-task behavior (6.53%), a high percentage of approvals (71%), moderate eye contact (27.30%), many activity changes (27), a high percentage of student response time (66%), and rapid pacing as indicated by an average length of both teacher and student activities of 5-6 seconds; the lowest rated

excerpt contained a high percentage of student off-task behavior

(14.49%), a low percentage of approvals (57%), a moderate number of activity changes, a high percentage of student response time

(73%), and slow pacing, with the average length of teacher and student activities of 9-28 seconds (see Table 1).

4. Regardless of observation focus (teacher versus student), the highest rating was given to Excerpt 5; the lowest rating for the


teacher observation focus was given to Excerpt 3; and the lowest rat-

ing for the student observation focus was given to Excerpt 1. 5. The highest rated excerpt for all universities was Excerpt 5;

there was little agreement among the four universities regarding the lowest rated excerpt with Excerpts 1, 3, and 6 achieving lowest ratings.

Finally, an ANOVA was computed comparing ratings of the 10 categories by observation focus (teacher versus student) and the four universities with repeated measures on category ratings. Data demonstrated significant differences between the student and teacher observation focus, F (1, 168) = 32.11, p = .0001, among the four universities, F (3, 169) = 6.81, p = .0002, and among the 10 categories, F (9, 1512) = 50, p = .0001. Also, there was a significant interaction effect between categories and observation focus, F (9, 1512) = 7.65, p < .0001.

Examination of means demonstrated that ratings for every one of the 10 categories were higher for the teacher observation focus than for the student observation focus. Significant differences among universities replicated results described for excerpts (see Item 2 above).

Further scrutiny of the significant interaction effect between focus of observation and categories is demonstrated in Table 2, which shows rank order of the ratings of each category by observation focus. The category of student attentiveness was rated higher under the teacher observation focus than under the student observation focus. Also, regardless of observation focus, the categories of time use, accuracy of task presentation, performance quality, and musicianship ranked in the top five ratings, and those for pacing, intensity, enthusiasm, and personality fell in the bottom five ranks.

Finally, subjects' comments were categorized and counted. There were 149 positive and 286 negative comments. Overall, there were more comments about the teacher than the students regardless of focus of observation. Of the 286 negative comments, 213 were about the teacher and 73 were about students in the ensemble. Of the 149 positive comments, 117 were about the teacher and 32 were about the students in the ensemble (see Table 3).

DISCUSSION

It seems clear that evaluations are indeed different when the focus of observation changes. This replicates prior research (Duke, 1987; Duke & Prickett, 1987; Prickett & Duke, 1992; Standley & Greenfield, 1987). Ratings in this study were lower across all categories for both student and teacher behavior when the evaluators' attention was focused on the students in the ensemble.

Interestingly, evaluators in the teacher focus group expressed frus-

JRME 315

Table 2 Mean Category Ratings Ranked by Observation Focus

Student focus Teacher focus

Category Mean Rank Mean Rank

Use of rehearsal time 6.42 2 7.24 1

Musicianship 6.27 3 7.07 4

Accuracy of instruction 6.49 1 7.16 2

Student attentiveness 5.38 8 7.02 5

Student performance 5.84 5 7.09 3

Enthusiasm 5.29 10 6.38 9

Intensity 5.52 7 6.40 8

Pacing 5.81 6 6.55 7

Personality 5.32 9 6.13 10

Overall 6.01 4 6.90 6

tration at being asked to evaluate students without the ability to see them. On the other hand, evaluators in the student focus group seemed not at all troubled by the inability to see the teacher. This is

perhaps a result of the traditional approach to teaching evaluation that places the complete responsibility for teaching and learning on the teacher.

Choral conductors in secondary and higher education may pro- ceed with at least two contradictory assumptions regarding teaching effectiveness in rehearsal situations. One assumption might be that the quality of the final performance is the result of the effectiveness of the rehearsals, and thus the teacher, leading up to the concert. On the other hand, the assumption might be that if the performance quality is not there at the beginning (that is, if the bad performers have not been weeded out in the audition process), there is little the conductor can do to produce a high-quality final performance. Thus, if the first assumption is true, the responsibility for quality may rest on the conductor; on the other hand, if the second assumption is true, the responsibility for quality may rest on the members of the ensemble.


Table 3 Comments about Teacher/students by Focus of Observation

Comments Student focus Teacher focus Total

Positive about teacher 25 92 117

Negative about teacher 52 161 213

Positive about students 24 8 32

Negative about students 58 15 73

Total positive 49 100 149

Total negative 110 176 286

Total 159 276 435

While it is clear that both teacher and student excellence are required to achieve quality in musical performance, achieving reci- procity in musical communication between teacher and student seems to be a neglected aspect of music teacher training. Often, young conductors are taught to concentrate on the musical score, thereby neglecting the individual members within the ensemble. In this study, subjects' comments contained many references to lack of eye contact and lack of positive facial expressions when the teacher interacted with the ensemble. Developing the ability to both concentrate on the score and pay attention to interpersonal interactions is paramount for good teaching to occur in ensemble rehearsals.

In evaluating teaching, two approaches are prominent. In one, the evaluator describes in detail what she or he sees without making a value judgment regarding whether it is right or wrong, good or bad, or better or worse. In the other approach, the evaluator does place a value judgment on what is observed and also makes suggestions regarding how to fix problems. Thus, the evaluator makes it clear what he or she would do to make things better.

Written comments made by subjects after viewing each excerpt reflected these two evaluation approaches. Some comments simply stated what the subject saw in the excerpt. For example, subjects said, "She lost the beat a couple of times," "All of her attention was on the score," and "Nice smile!" Other comments stated what the subject would have done to make the rehearsal better. For example, subjects said, "Get rid of the pencil," "In the 4 pattern, you should have made

JRME 317

the 3rd beat clearer," and "I think she should be working on diction now."

The statistically significant differences among the mean evaluations of the four universities were expected. This result is not remark- able when the question we wanted to answer is considered. What we wanted to know was whether subjects from different universities would evaluate the excerpts and the categories within the excerpts similarly. Indeed, this was the case. The relationship among the rat-

ings for the excerpts across the four universities was similar. That is, the highest-rated excerpt for all the universities was the same-

Excerpt 5. There was little agreement among the four universities

regarding the lowest-rated excerpt, with Excerpts 1, 3, and 6 achiev-

ing lowest ratings; however, these three excerpts were consistently rated lowest across all other independent variables.

The most important result of this study is that the highest rating was given to the excerpt demonstrating a low percentage of student off-task behavior, a high percentage of approvals, and rapid pacing (i.e., 5-6 seconds of alternating student performance and teacher talk), thus replicating prior research (Yarbrough, Dunn, & Baird, in

press; Yarbrough & Madsen, 1998). Also, the lowest ratings were

given to the excerpt demonstrating a high percentage of student off- task, a low percentage of approvals, and slow pacing (i.e., 9-28 seconds of alternating student performance and teacher talk).

These three variables-student attentiveness, positive reinforcement, and pacing--are often the most difficult to teach prospective teachers to understand. Maintaining student attentiveness requires teacher attention to the students rather than to the musical score.

Genuinely positive reinforcement requires practice and commitment to the belief that such reinforcement does work. Appropriate pacing can be achieved only when the teacher has analyzed the musical task

thoroughly and knows what he or she wants to hear in the final performance. Regardless of the difficulties, there is now enough extant research showing the importance and effectiveness of these variables to more vigorously encourage their emphasis in music teacher train-

ing programs.

NOTE

1. Three students did not declare their majors, and one student did not declare his or her gender.

REFERENCES

Duke, R. A. (1987). Observation of applied music instruction: The percep- tions of trained and untrained observers. In C. K. Madsen and C. A.


Prickett (Eds.), Applications of research in music behavior (pp. 115-124). Tuscaloosa: University of Alabama Press.

Duke, R. A., & Prickett, C. A. (1987). The effect of differentially focused observation on evaluation of instruction. Journal of Research in Music Edu- cation, 35, 27-38.

Jellison, J. A., & Kostka, M. (1989). Student or teacher as focus of attention and elementary students' written recall of teaching content within music teaching units. In R. A. Duke (Ed.), Texas music education research (pp. 43-49). Austin, TX: Texas Music Educators Association.

Madsen, C. K., Standley,J. M., Byo,J. L., & Cassidy, J. W. (1992). Assessment of effective teaching by instrumental music student teachers and experts. Update: The Applications of Research in Music Education, 10 (2), 20-24.

Price, H. E. (1983). Effect of conductor academic task presentation, conductor reinforcement, and ensemble practice on performers' musical achievement, attentiveness, and attitude. Journal of Research in Music Education, 31, 245-257.

Prickett, C. A., & Duke, R. A. (1992). Evaluation of music instruction by musi- cians and nonmusicians assigned differential observation tasks. Bulletin of the Council for Research in Music Education, no. 113, 41-50.

Sims, W. L. (1986). The effect of high vs. low teacher affect and passive vs. active student activity during music listening on preschool children's attention, piece preference, time spent listening and piece recognition. Journal of Research in Music Education, 34, 173-191.

Standley, J. M., & Greenfield, D. G. (1987). The effect of a focused observation task and its transfer on the analysis of a kindergarten music class by pre-senior versus pre-internship music education/therapy majors. In C. K. Madsen and C. A. Prickett (Eds.), Applications of research in music behavior

(pp. 99-114). Tuscaloosa: University of Alabama Press. Yarbrough, C. (1975). Effect of magnitude of conductor behavior on per-

formance, attentiveness, and attitude of students in selected mixed cho- ruses. Journal of Research in Music Education, 23, 134-146.

Yarbrough, C., Dunn, D. E., & Baird, S. L. (in press). Longitudinal study of

teaching in a choral rehearsal. Southeastern Journal of Music Education.

Yarbrough, C., & Madsen, K. (1998). The evaluation of teaching in choral rehearsals. Journal of Research in Music Education, 46, 469-481.

Yarbrough, C., & Price, H. E. (1981). Prediction of performer attentiveness based on rehearsal activity and teacher behavior. Journal of Research in Music Education, 29, 209-217.

Submitted October 12, 1998; accepted July 22, 1999.

yarbrough and henley

Art & Photos