comparative learning and retention of conventional and instructional tv methods

Comparative Learning and Retention of Conventional Instructional TV Methods

and

KENNETH D. HOPKINS

D. WELTY LEFEVER

There is an abundance of evidence showing that pupils receiving televised instruction tend to achieve at least as well as conventionally taught pupils (2, 2, 6). Few studies, however, have ex- plored possible retention or forgetting differentials. Of the few retention studies which have been reported, most have been conducted at the college level and have used relatively short periods between immediate and delayed posttests. In these investigations a general trend of no significant differences has been evidenced (6). Kanner (4) found, however, in a military setting that face- to-face (conventional) instruction was superior to television in the immediate posttest, but that this difference was no longer present on a retention test administered one month later. Al- though the difference scores, per se, were not tested for significance, it was suggested that the TV group retained proportion- ately more. Schramm (7) has noted that the relative efficacy of

Kenneth D. Hopkins is an associate professor of educational psychology at the University of Southern California. D. Welty Lefever is a professor of educational psychology at the same university. Both authors have served as research consultants for the Anaheim Instructional Television Project for the past five years. They are presently writing a monograph presenting the results from their five years of research with this project.

28

ITV VS. CONVENTIONAL METHODS : 2 9

PROCEDURE

instructional television (ITV) differs for the various levels of education, with ITV having produced a relatively better showing at the elementary grades. Therefore, it would seem inappropri- ate to infer that results from retention studies with highly select college students apply to elementary school pupils or to gener- alize findings from one- or two-month delayed posttest compari- son to longer time intervals.

The purposes of the present investigation were to compare the achievement and long-term retention of elementary school pupils taught conventionally and with ITV. In 1959 the Anaheim City School District initiated a dosed-circuit ITV project. The School of Education of the University of Southern California was contracted to conduct an impartial evaluation. The study continued over a five-year period; the present study represents one small facet of the total evaluation, described fully in a University of Southern California, School of Educa- tion, monograph in preparation.

The curricular area designated as the basis for the retention research was social studies. Since the subject matter of each grade level in social studies is highly independent of that of other grades, a minimum of review and relearning would be expected from grade to grade. This nonoverlapping, noncontinuous nature of the social studies curricula reduces the confounding which would operate in such areas as science and arithmetic. In the Cali- fornia public schools the respective content areas for grades four, five, and six, are California, the United States, and Central-South America.

To control administration variance and in order that knowledge in social studies would not be confounded by reading ability, all tests in the study were administered over television by the same member of the evaluation staff. While the pupil followed his printed copy of the test, each item was read twice. A pause then provided opportunity for the examinee to respond on his separate IBM answer sheet.

The study includes comparisons of the achievement and retention in social studies between the three instructional patterns:

SC-TV Pupils in regular "self-contained" classrooms receiving ITV and related classroom teaching.

Rdp-TV Pupils in large classes (75-85 pupils) with two teachers in which ITV and related classroom teaching occur simul- taneously with the SC-TV group. For instruction in non-

AV COMMUNICATION REVIEW : 30

STUDY I

televised, basic skill subjects the pupils are deployed into small classes of approximately 25.

Control Pupils in conventional classes receiving no televised instruction.

The self-contained TV (SC-TV) and the control classes remain with a single teacher for the entire day, with the difference between them that the SC-TV group receives televised instruction in social studies, science, Spanish, music, and arithmetic enrich- ment. The redeployment-TV (Rdp-TV) and SC-TV groups re- ceive the identical televised instruction, differing only in classroom organization and in the nature of the teacher's "follow-up" instruction at the conclusion of the telecast. Shanks (8, 9) gives a more complete description of the method groups. All three instruction groups were compared in Study I; only SC-TV and control were involved in Study II. During the final month of the 296o-6z school year, aU fifth grade pupils (N = 2,284, absentees excluded) were given a 5o-item, multiple-choice examination covering the entire year's curriculum, with special emphasis given to the content of the last semester. (A previous examination over the first semester's work had been given, but will not be reported here since it was not directly involved in the retention study.) The examination was constructed around district objectives using the scope-and-se- quence curricular guides and state-approved social studies text- books as the basis for item development.

A 3 x 3 x 2 analysis of covariance (method group x level of intelligence x sex) was employed utilizing the IBM 7o94 com- puter at the Western Data Processing Center (for which grati- tude is hereby expressed). CTMM language IQ's (z957-S form) served as the covariate with the values of 9 ~ and zzo dividing the groups into the three intelligence levels. An analysis of variance was also performed to corroborate the fact that the groups were highly comparable, not requiring the covariance adjust- ment, which was to be expected since the schools were assigned randomly to method groups. The CTMM mean language IQ's for the Rdp-TV, SC-TV, and control groups were 2o8.2, 2o9.3, and 2o9.7, respectively. The results from these analyses are given in Table -r. Only unadjusted values are given since they differ only fractionally from the adjusted means because the groups were virtually equal on the covariate. Since the design employed a fixed effects model, only the F-ratios and the error mean square

ITV VS. CONVENTIONAL METHODS : 3I

are given because all other variance estimates can be reproduced from these values. Homogeneity of variance was insured by using the two most extreme cell variances for the F-test (F = 2.54, p > .oi), hence making the more rigorous Bartlett test unneces-

sary.

TABLE 1. F-Ratios

and Error

F-ratios Source of Variation df Covariance Variance

Mean Squares for the

Analyses of Variance and

Covariance for the

End-of-Term Fifth Grade

Social Studies Test a

Method groups 2 3.62* 4.08* Levels of Intelligence b 2 6.09** 161.32"* Sex ~ 1 54.85** 50.96** Method x Intelligence 4 1.13 0.44 Method x Sex 2 0.48 0.41 Intelligence x Sex 2 1.25 1.37 Methods x Intelligence x Sex 4 0.59 0.60

1,165 37.858 Error MS 1,166 43.958

aKR 21rtt~.83 . b Means and N's: low: 22.14 (N--- 112); avg.: 25.53 (N = 479); high:

32.04 (N = 593). c Means and N's: boys: 30.45 (N -- 620); girls: 26.29 (N = 564). * p < .05.

** p < .001.

As shown in Table I , there were significant differences on this immediate posttest between the method groups means, yet no significant interactions between the three factors. (The significant F-ratios for level of intelligence and sex are of only inci-

dental interest to the study and were included only for purposes of evaluating possible interactions with method groups.) The means, standard deviations, and number of pupils participating in both the immediate postest and the retention test administered nine months later are given in Table z by method group.

The sixth grade teachers were not made aware that the sched- uled social studies test was to be a retention test, a repeat of the fifth grade final examination, in order to minimize any review efforts on the part of teachers unfamiliar with the function of the study. It is important to note that there were no control classes for grade six, only Rdp-TV and SC-TV. Therefore, the fifth grade control pupils were in either Rdp-TV or SC-TV classes for grade six; many SC-TV pupils were also placed in Rdp-TV

AV COMMUNICATION REVIEW : .32-

TABLE 2

Means, Standard

Deviations,

classes for grade six. For this reason differential review of method/group would be unlikely.

Method Group

Immediate Delayed N X X s s r t

1 2 1 2 12 (gain)

and Rdp-TV Intercorrelations SC-TV

for the Control Immediate

350 28.89 29 .65 8 .08 7 .88 .893 3.84**

543 28 .24 2 8 . 7 7 7 .60 7.91 .786 2.43* 264 26.22 26 .80 7 .63 7.97 .699 1.55

and Delayed Posttests

on the Fifth Grade

Social Studies Test by

Method Group

* p < .05. ** p < .01.

Although the analysis of covariance had indicated there were significant differences between the means of the three method groups, it was necessary to use Kramer's (5) extension of Dun- can's new multiple range to groups of unequal numbers of replication to identify the particular differences evidencing significance. It was found that both of the ITV method groups were superior (p. < .o2) to the control group on both the immediate and delayed posttest. The differences between Rdp-TV and SC- TV groups were not significant.

Unexpectedly the means for all groups were higher on the retention test than on the immediate posttest. Examined separately, the Rdp-TV and SC-TV groups had significant gains whereas the control group did not, yet the control group increased numerically more than SC-TV (cf., Table 2). The apparent inconsistencies resulted from larger N's and higher correlations coefficients between the tests noted with the TV groups. Although the mean gain for Rdp-TV (.76) was larger than that of both SC-TV (.53) and control (.58), the differences between the group gains were not significant (t = .78 and t = .43, respectively). Obviously the difference between SC-TV and control was also not significant (t = .oi2).

The hypothesized factors which contributed to the increase in scores on the delayed tests are as follows: (a) the tests' emphases on items requiring reasoning and interpretation in addition to knowledge and hence may have measured mental or reasoning ability as well as achievement, (b) the possible review which may have followed the immediate posttest by some grade five teachers who took advantage of the opportunity to use the test as an instructional device; hence learning and overlearning would


STUDY II

have taken place in some pupils after the immediate test, (c) the slight benefit from practice effect, and (d) incidental learning.

It is interesting to note that the stability coefficients given in Table z differed significantly for the various groups; Rdp-TV showing the greatest stability (r = .89), followed by SC-TV (r = .79), which in turn showed greater consistency than did the control group (r = .7o). All differences were significant beyond the .oi level. These stability differences would seem to have related to the degree of instructional and organizational change typically experienced by the various groups. In grade six there was no control group; consequently all pupils in control classes for grade five were assigned either to Rdp-TV or SC-TV sixth grade classes, and consequently all control pupils experienced change. Since the number of Rdp-TV classes was increased after the immediate posttest, many SC-TV pupils at grade five were placed in Rdp-TV classes for grade six. Most Rdp-TV pupils continued in the same group for grade six.

The principal conclusions from Study I are (a) that the general pattern of results of Rdp-TV, SC-TV > control was noted on both the immediate posttest and retention test and (b) the methods groups did not differ in retention rates. The second study examined the achievement patterns immedi- ately after instruction and again following a two-year (26- month) interval. An examination covering the first semester of the fourth grade social studies curriculum, a study of California, was administered to all pupils in the district's fourth grades (N = ~,375, absentees excluded). The 45-item, multiple-choice test was designed and administered in the manner previously described in Study I. The immediate posttest occurred during the middle of grade four, the retention test during the seventh school month of grade six.

Only two method groups, SC-TV and control, operated at the fourth grade level for the ~96o-6-f school year. For the subse- quent grades, five and six, all control and SC-TV pupils were placed in either Rdp-TV or SC-TV classes.

The z x 3 x 2 analysis of covariance (method groups x intelligence levels x sex) of the immediate posttest was conducted in the same manner as previously described in Study I. The results are given in Table 3- F-ratios for an analysis of variance are also given to illustrate the comparability of the groups and the consistency of the findings with and without the covariance adjust-

AV COMMUNICATION REVIEW : 34

TABLE 3

F-Ratios and Error Mean

Squares for the

merit. The mean CTMM language IQ's for the SC-TV and control

groups were 5o5.o and :to6.2, respectively. Homogenei ty of cell

variances was confirmed when the two extreme values were used

(F = L77 , p > .05).

Grade 4 Grade 6 Covari- Vari- Covari- Vari-

Source of Variation df ance b ance df ance e ance

Analyses of Variance and

Covariance for the

Social Studies Examination" Administered

at Grades 4 and 6

Method groups 1 10.88"* 12.20"* Levels of Intelligence d 2 6.24** 74.65** Sex e 1 22.73** 16.89"* Method x Intelligence 2 2.77 2.21 Method x Sex 1 0.66 0.53 Intelligence x Sex 2 1.19 0.43 Methods x

Intelligence x Sex 2 0.43 0.21 1,362 33.87

Error MS

1 2.56 2.11 1 12.46"* 66.43** 1 27.11"* 45.48** 1 0.09 0.56 1 3.40 5.10 1 0.05 .01

1 0.01 .72 1,007 16.75

1,363 36.73 1,008 32.85

"KR 21 rtt = .78. b CTMM language IQ's served as the covariate. e Total score from grade four used as covariate; homogeneity of variance

was evidenced (F = 1.28, p > .01). F-ratio from covariance analysis must be interpreted cautiously as correlations between the covariate and the de- pendent variable were significantly different for the two groups. The change in df for intelligence resulted from a division into two groups at IQ 100.

dMeans and N's: low: 21.84 (N = 235; avg.: 24.43 (N ~ 590); high: 28.51 (N "- 550).

"Means and N's: boys: 26.16 (N = 707); girls: 25.04 (N = 668). ** p < .001.

The results from the immediate posttest, as shown in Table 3,

are in consistent agreement with the related findings of Study I. The results of particular relevance for the study were again as

follows: (a) superiority of the self-contained TV group over the

control group and (b) the absence of significant interactions between method, intelligence, and sex.

The identical fourth grade social studies test was repeated 26

months later near the end of the pupils' sixth grade work. The

test was administered in the same manner as the initial test. The

nature of the test was unannounced to prevent any at tempt to

review on the part of teachers who might be unfamiliar with the

intent of the study. The analyses of covariance and variance

results for all students taking both tests are given in Table 3.

Table 4 gives the composite results for the two method groups.


TABLE 4

Means, Standard

Deviations, and

Immediate Delayed

Method Group N X X s s r t 1 2 1 2 12 (gain)

Intercorrelations for the

Immediate and Delayed

SC-TV 820 26.16 28 .35 6 .67 6.22 .761 15.82"* Control 196 24 .14 27 .58 6 .79 6.65 .663 13.33"*

2.73**

Posttests for the

Fourth Grade Social Studies

Test by Method Group

** p ~ .01.

The means for both groups significantly increased over the two-year period, although the difference between them was no longer significant on the retest. The fact that the increase was larger than for the one-year retention study would tend to sup- port the maturational hypothesis, i.e., the items required reasoning ability as well as knowledge. In addition, since the test was originally administered near the middle of grade four, second semester learning would also be evidenced in the gain.

Consistent with the relationship noted in Study I, the stability coefficient for SC-TV was significantly greater than that for the control group (t = 2.73 ). This finding was again attributed to the greater consistency in instructional method for the SC-TV group for the two-year period, i.e., method x pupil interactions would be expected to be less with SC-TV since less instructional change occurred with this group over the two-year period. (All pupils of the study were in either SC-TV or Rdp-TV situations for grades five and six.)

In addition to the analysis of total test scores, an attempt was made by the two investigators to classify the individual test items as to the type of process required and to make a separate analysis for each of the three item types: Type A items requiring only factual knowledge (N = 58); Type B items requiring the com- parison and interrelating of facts or events (N = 58); Type C items requiring the drawing of inferences, applying a principle, generalizing, and extrapolating (N = 9). The response of each examinee on each item was punched into data processing cards so that the five possible permutations of response patterns for each item could be ascertained, i.e., RR: right responses on both the immediate and delayed tests; RW: right response on immediate test with wrong response on delayed test; WR: wrong response on initial test with correct answer on the delayed test;

AV COMMUNICATION REVIEW : 3 6

TABLE 5 Percentage of Correct Responses

on the

WW: the same wrong answer on both tests; and Ww: wrong but different responses given on both occasions.

Several significant chi square values were obtained between the response patterns of the two groups for items from each of the three-item types. The Kolmogorov-Smirnov goodness-of-fit test was applied to the obtained distribution of item chi square values for each item type to ascertain whether there was a significant departure from the theoretically experted chance chi square distribution. For the i8 type A items, no significant trend was evidenced (D = .233 , p > .~to). Item type B (D = .45 8, p < .oo5) and C (D = .394, P < .oz5), however, were associated with larger chi square values than reasonably could be accounted for by chance alone. The significance for the 28 type B items (requiring comparisons of facts or events) resulted primarily from the relatively larger percentage of RR responses in the SC- TV group. The significant difference in the response patterns of the two groups on type C items resulted almost entirely from the larger proportion of WR responses in the control group. The relationships are clarified by Table 5 which presents the findings in terms of percentage of correct items for each group and item type.

Method Group

Type A Type B Type C Total

Immed. Immed. Immed. Immed. Delayed De layed Delayed Delayed (n=18) (n=18) (n=9) (n=45)

Immediate and Delayed

Posttests for Each Item

Type and Group

SUMMARY

SC-TV 62.86 63.92 51.77 62.52 59.28 61.39 57.70 62.81 Control 59.55 61.47 44.72 60.22 54.28 66.22 52.63 61.92

For each item type the control group improved more than did the SC-TV, although only with item Type C did the control group exceed SC-TV in any of the six comparisons. The relatively greater gain by the control groups, in part, was a reflection of their greater opportunity for improvement since the initial scores were lower. In addition, the mean IQ for the control group was fractionally higher than for the SC-TV group; consequently slightly greater gain would be anticipated on reasoning type items. The factual items showed the least gain, consistent with the maturational hypothesis. Each of the 2o elementary schools in the Anaheim City School District were randomly assigned to one of the three method

ITV VS. CONVENTIONAL METHODS : 37

REFERENCES

groups. The present studies included the entire group of fourth and fifth grade pupils who received instruction either conventionally or with one of the two ITV methods. The two studies compared pupils' achievement and retention by method group, intelligence, and sex. Groups receiving the televised social studies instruction performed significantly better on the end-of-term examinations than did the control groups. On the one- and two- year delayed posttests, contrary to expectation, all groups had significantly higher means than on the immediate posttests. The basic explanation for the phenomenon was thought to be matura- tion since many items appear to have tapped reasoning ability as well as content knowledge. This interpretation was supported by the fact that greater gain was noted on the two-year delayed posttest than on the one-year detention measure. The hypothesis was further strengthened by the analysis by item type, the factual items showing the least increase over the time interval. No differential change or gain existed for method groups for the one- year study. In the two-year study the control group showed greater gain, but its mean remained below that of the TV group.

I. Allen, William H. Television for California Schools. Sacramento: California State Department of Education, ~959.

2. Allen, William H. "Audio-Visual Communication." Encyclopedia of Education Research. (Edited by C. W. Harris.) New York: Mac- millan Co., x96o. pp. ~15-37.

3. Benschoter, R. D., and Charles, D. C. "Retention of Classroom and Television Learning." Journal of Applied Psychology 41: 253-56; ~957.

4. Kanner, J. H., and others. Television in Army Training. Evalua- tion of "Intensive" Television for Teaching Basic Electricity. New York: Army Pictorial Center, 1958.

5. Kramer, C. Y. "Extension of Multiple Range Tests to Group Means with Unequal Numbers of Replication." Biometrics I2: 3o7-3Io; I956.

6. Schramm, Wilbur, editor. The Impact of Educational Television. Urbana: University of Illinois Press, I96o.

7. Schramm, Wilbur. "Learning from Instructional Television." Re- view of Educational Research 32: i56-67; April ~962.

8. Shanks, Robert E. "Closed-Circuit TV in Anaheim City Elemen- tary School District." School Board Journal 548: 34-38; February z964 .

9. Shanks, Robert E. "The Anaheim Approach to Closed-Circuit Tele- vision." A Guide to Instructional Television. (Edited by Robert W. M. Diamond.) New York: McGraw-Hill Book Co., ~964. pp. ~56- 79.

comparative learning and retention of conventional and instructional tv methods

Documents