title a review of the literature related to the cloze review of the literature related to the cloze...
TRANSCRIPT
f
I'c
DOCUMENT RESUME
ED 075 785 CS 000 469
AUTHOR Frame Ralph D.TITLE A Review of the Literature Related to the Cloze
Procedure.PUB DATE 72NOTE 264p.; M.Ed. Thesis, Boston University School of
Education
EDRS PRICE MF-S0.65 HC-$9.87DESCRIPTORS *Cloze Procedure; *Literature Reviews; Readability;
*Reading; Reading Ability; Reading Achievement;*Reading Comprehension; *Reading Research, eadingSkills; Reading Tests
ABSTRACTThis review of the literature related to the cloze
procedure includes most of the studies reported from 1953 to 1971.One hundred ninety-eight documents are included and are divided intofour categories, studies in which the cloze procedure has beenapplied (1) as a measure of readability, (2) as a measure ofcomprehension, (3) as a measure of language facility, and (4) as aninstructional technique. An appendix and bibliography are included.(NP)
U.S DEPARTMENT OF HEALTH.EDUCATION & WELFAREOFFICE OF EDUCATION
THIS DOCUMENT HAS BEEN REPRO-DUCED EXACTLY AS RECEIVED FROMTHE PERSON OR ORGANIZATION ORIGINATING IT POINTS OF VIEW OR OPINIONS STATED DO NOT NECESSARILYREPRESENT OFFICIAL OFFICE OF EDUCATION POSITION OR POLICY
BOSTON UNIVERSITY
SCHOOL OF EDUCATION
Thesis
A REVIEW OF THE LITERATURE RELATED TO
PERMISSION TO REPRODUCE THIS COPY-RIGHTED MATERIAL HAS BEEN GRANTEDBY
Ralph D. Fram
TO ERIC AND ORGANIZATIONS OPERATINGUNDER AGREEMENTS WITH THE U S OFFICEOF EDUCATION FURTHER REPRODUCTIONOUTSIDE THE ERIC SYSTEM REQUIRES PERMISSION OF THE COPYRIGHT OWNER
THE CLOZE PROCEDURE
by
Ralph D. Fram
Bachelor of TeachingThe University of New Brunswick
1967
Bachelor of ArtsThe University of New Brunswick
1969
submitted in partial fulfillment of the
requirements for the degree of
4- Master of Education
19724
*FILMED FROM BEST AVAILABLE COPY
First Readers
Second ReadersThomas E. Culliton, Jr.Professor of Education
ft 111/4.0
Ro elmina IndrisanoAssociate Professor of Education
ii
sriire*el
Acknowledgments
The author acknowledges with grateful appreciation:
- family and friends for their abiding interest,understanding, and encouragement,
- Dr. Helen A. Murphy and Dr. Agnes Matthews fortheir stimulating enthusiasm and inspiration,
- Dr. John R. Bormuth and Dr. Wilson L. Taylorfor papers and profitable suggestions,
- Dr. Thomas E. Culliton, Jr. for his unselfishguidance, and
- his advisor, Dr. Roselmina Indrisano, for hervalued and scholarly counsel.
TABLE OF CONTENTS
INTRODUCTION1
Chapter
I. THE CLOZE PROCEDURE: AN OVERVIEW 2
II. THE CLOZE PROCEDURE AS A MEASUREOF READABILITY 23
III. THE CLOZE PROCEDURE AS A MEASUREOF COMPREHENSION 70
IV. THE CLOZE PROCEDURE AS A MEASUREOF LANGUAGE FACILITY 139
V. THE CLOZE PROCEDURE AS ANINSTRUCTIONAL TECHNIQUE 178
APPENDIX 232
BIBLIOGRAPHY 244
INTRODUCTION
This thesis presents a review of the literature
related to the cloze procedure, and includes all the studies
reported from 1953 to 1971, the exceptions being those docu-
ments unavailable at the time of this writing.
A total of two hundred twenty-two documents were
examined. Of these, one hundred ninety-eight were selected
as being related in some respect to the cloze procedure.
One hundred twenty-six abstracts have been included in the
body of the thesis and the remaining seventy-two references
have been included in the Appendix.
References in Section I of the Appendix are classi-
fied to correspond with the five chapters of the thesis.
Section II contains twelve additional references assumed
to be of significance but which were unavailable at the
time this thesis was prepared.
The Bibliography, a composite of all one hundred
twenty-six abstracts, is also classified to correspond with
the five chapters.
1
CHAPTER I
THE CLOZE PROCEDURE: AN OVERVIEW
In 1953 Taylorl introduced a technique for measur-
ing readability which he named "the cloze procedure." The
name of this new technique was derived from the Gestalt
concept of "closure" which hypothesizes that the reader
completes a structural whole by filling in a missing gap.
Taylor defines cloze as:
A method of intercepting a message from a 'trans-mitter' . . . mutilating its language patterns bydeleting parts, and so administering it to 're-ceivers' . . . that their attempts to make the pat-terns whole again potentially yield a considerablenumber of cloze units.2
A cloze unit is
. . . any single occurrence of a successful attemptto reproduce accurately a part deleted from 'a mes-sage' . . . by deciding from the context that re-mains, what the missing part should be.3
AlthodWthe cloze procedure per se is a relatively
new concept, word deletion techniques were used by
'Wilson L. Taylor, "'Close Procedure': A New Toolfor Measuring Readability," Journalism Quarterly, 30 (1953),414-438.
2Ibid., 416.
3wg.
2
Ebbinghaus, as reported by Rankin,1
in 1897 in the con-
struction of an intelligence test. Applications of the word
deletion technique may be observed in several current psy-
chological instruments and in the use of incomplete sen-
tences in measures of scholastic achievement. Vocabulary
tests utilize. the word deletion technique to determine the
reader's sensitivity to context clues. The doze proce-
dure, however, should not be confused with the latter word
deletion techniques.
Omitted words in sentence completion tests and
vocabulary tests requiring the use of context clues are de-
termined by the test constructor in terms of specific mean-
ing, context, and the purpose of the test. In contrast,
doze deletions may occur at Any point in a continuous
passage by means of a specific, objective criterion. Suc-
cess or failure in predicting omitted words may or may not
influence subsequent success or failure. Contextual clues
in a doze test may be far removed from a doze deletion
and not immediately surround it, as they would in a vocabu-
lary test of seasitivity to context clues.
Cloze deletions are made oy means of a specific,
1Earl F. Rankin, Jr., "The Cloze Procedure - Its
Validity and Utility," Startin and Im rovin Colleffet Read-ing Programs, ed. by Oscar S. Causey and Will am Eller,Eighth Yearbook of the National Reading Conference (Milwau-kee, Wisconsin: National Reading Conference, 1959), pp.131-144.
i.
objective criterion which takes one of two forms: an every
"nth" word deletion or the deletion of every word of a par-
ticular syntactic type. The former is sometimes referred
to as a structural deletion and the latter as a lexical
deletion. Structural deletions may be made on the basis
of every second, or fifth, or tenth word, for example, or
by a more sophisticated randomizing procedure of the re-
searcher's choice. Lexical deletions are usually limited
to nouns or verbs, although any syntactic element may be
chosen as the deletion criterion. The first and last sen-
tences of the passage are not generally included in the
procedure in order to permit the reader to become familiar,
to some degree, with the topic of the passage.
Deleted words are replaced by a blank line of
standard length. Scoring is computed on the basis of either
the number of correct exact replacements, the number of
acceptable synonym replacements, or the number of correct
form class replacements.
Studies investigating the effects of different
cloze test forms and different scoring systems have been1
conducted by Bormuth. The use of exact-length blanks in
IJohn R. Bormuth, "Experimental Applications of
Cloze Tests," Improvement of Reading Through ClassroomPractice, ed. by J. Allen Figurel, Proceedings of the Inter-national Reading Association (Newark, Delaware: Interna-tional Reading Association, 1964), pp. 303-3061 John R.Bormuth, "Validities of Grammatical and Semantic Classifi-
5
cloze tests of reading comprehension have been investi-
gated by Anderson.1
Various theoretical and practical as-
pects of the cloze procedure have been discussed by Culhanes
Hafner; Ohnmacht, Weaver, and Kohler; Rankin; Taylor;
Tuinman and Blanton; and Weaver.2
cations by Cloze Test Scores," Reading and Inquiry, ed. byJ. Allen Figurel, Proceedings of the International ReadingAssociation (Newark, Delaware: International Reading Asso-ciation, 1965), pp. 283-286.
1Jonathan Anderson, "A Report of Research on Compre-
hension in Reading" (paper presented at the Third Interna-tional Reading Association World Congress on Reading, Syd-ney, Australia, August 7-9, 1970).
2Joseph W. Culhane, "Cloze Procedures and. Ccmprehen-
sion," The Reading Teacher, 23 (1970), 410-413, 464; Law-rence E. Hafner, "Relationships of Various Measures to the'Cloze'," New Concepts in College- Adult Reading, ed. by EricL. Thurston and Lawrence E. Hafner, Thirteenth Yearbook ofthe National Reading Conference (Milwaukee, Wisconsin: Na-tional Reading Conference, 1963), pp. 135-145; Lawrence E.Hafner, "Implications of Cloze," The Philosophical andSociological Bases of Reading, ed. by Eric L. Thurston andLawrence E. Hafner, Fourteenth Yearbook of the NationalReading Conference (Milwaukee, Wisconsin: National ReadingConference, 1965), pp. 151-158; Fred Ohnmacht, Wendell W.Weaver, and Emmett T. Kohler, "Cloze and Closure: A Fac-torial Study," The Journal of Psychology, 74 (1970), 205-217s Earl F. Rankin, Jr., "The Close Procedure - Its Valid-ity and Utility," gjnrgovirgStartinandlr*ColleeFteadirgiPrograms, ed. by Oscar S. Causey and William Eller, EighthYearbook of the National Reading Conference (Milwaukee,Wisconsin: National Reading Conference, 1959), pp. 131-144; Earl F. Rankin, Jr., "Closure and the Cloze Procedure,"College and Adult Reading, ed. by David M. Wark, Confer-ence Proceedings of the North Central Reading Association(Minnepolis, Minnesota: University of Minnesota, 1964),pp. 54-84; Earl F. Rankin, Jr., "Research Design and theCloze Procedure," Vistas in Reading, ed. by J. AllenFigurel, Proceedings of the International Reading Associa-tion (Newark, Delaware: International Reading Association,
6
Since its introduction by Taylor in 1953 the cloze
procedure has been applied with increasing frequency to
studies concerned with readability, comprehension, language
facility, and educational methodology. Subsequent chapters
of this thesis present the literature pertinent to these
four categories.
1966); Wilson L. Taylor, "Recent Developments in the Use of'Clone Procedure,'" Journalism Quarterly, 33 (1956), 42-48;Wilson L. Taylor, "Relative Influence of Preceding, Follow-ing, and Surrounding Contexts on Cloze Performance" (paperpresented at the Annual Meeting of the National Reading Con-ference; Tampa, Florida, December 3, 1971); J. Jaap Tuinmanand B. Elgit Blanton, The Cloze Procedure: Corollaries ofa Search Hypothesis (Bloomington, Indiana: School of Educa-tion, University of Indiana, 1971); Wendell W. Weaver,"Theoretical Aspects of the Cloze Procedure," The Philosophi-cal and Sociological Bases of Reading, ed. by Eric L.Thurston and Lawrence E. Hafner, Fourteenth Yearbook of theNational Reading Conference (Milwaukee, Wisconsin: NationalReading Conference, 1965), pp. 115-132.
7
ANDERSON, JONATHAN. A REPORT OF RESEARCH ON COMPREHENSION INREADING. PAPER PRESENTED AT THE THIRD INTERNATIONALREADING ASSOCIATION WORLD CONGRESS ON READING, SYDNEY,AUSTRALIA, AUGUST 7-9, 1970.
Purpose: The use of exact-length blanks in cloze tests ofreading comprehension was investigated.
Principles, Concepts, or Procedures: Subjects were indigenousprimary-school pupils in New Guinea for whom English was aforeign language.
Conclusions: No significant difference was found between themean scores of subjects on cloze tests using blanks of a uni-form length and the mean scores of subjects on cloze tests us-ing blanks of the same length as the deleted words. Both ver-siuns of cloze tests were equally valid as measures of generalreading comprehension. The question of length of blank to usein cl6ze tests has practical as well as theoretical signifi-cance for teachers ire constructing cloze tests when they wishto use photocopying facilities. The implication of these find-ings is that by the use of a photocopy procedure, such factorsas size of print, illustrative material, and page layout maybe included in the estimate of the ease or difficulty ofprinted material as measured by cloze tests. Tables and refer-ences are included.1
litesearch in Education (Washington, D.C.: United StatesGovernment Printing Office, 1971), Vol. 6, No. 4, p. 111.(SD 045 323, 13 pagesi.
8
BORMUTH, JOHN R, "EXPERIMENTAL APPLICATIONS OF CLOZE TESTS."IMPROVEMENT OF READING THROUGH CLASSROOM PRACTICE.EDITED BY J. ALLEN FIGUREL. PROCEEDINGS OF THE INTERNA-TIONAL READING ASSOCIATION. NEWARK, DELAWARE: INTERNA-TIONAL READING ASSOCIATION, 1964, 303-306.
Purpose: The author states three purposes for the study.
1. to discover if the five forms of a cloze test madefrom the same passage differ in difficulty.
2. if they do, by how much.
3. to determine if these differences were influenced bythe number of items iris .uded in the forms.
Principles, Concepts, or Procedures: Five cloze tests of 50items each were made over each of 20 passages. Deletions weremade so that every word in a 250 word section of every passagebecame a deletion item in one of the five forms. The passagesranged in Dale-Chall readability from approximately 4.0 to 8.0and were chosen from textbooks and literary works. The 695subjects were organized into five groups on the basis ofStanford Reading Achievement Test scores. A different formof the cloze test was assigned to each group. Only exact-wordreplacements were accepted.
Conclusions: The following conclusions were reached.
1. The means on different cloze test forms that aremade from the same passage differ significantlyfor tests of 50 items or less. . . .
2. The differences in difficulty among test formsmade from the same passage tend to diminish asmore items are included in the test forms.
3. The practice of using only a single cloze testform over each of the passages that a researcheris studying should probably be avoided where pre-cise determinations of passage difficulties areneeded and especially when the passages are short.In such experiment.; all five forms should be used.
9
4. When single forms are used in experiments it isnecessary to show that the observed differencesare sufficiently large to assure that they didnot arise solely because of the test forms .
(306)
10
BORMUTH, JOHN R. "VALIDITIES OF GRAMMATICAL AND SEMANTICCLASSIFICATIONS OF CLOZE TEST SCORES." READING AND IN-QUIRY. EDITED BY J. ALLEN FIGUREL. PROCEEDINGS OF THEINTERNATIONAL READING ASSOCIATION. NEWARK, DELAWARE:INTERNATIONAL READING ASSOCIATION, 1965, 283-286.
Purpose!:
. . . doze test responses are classified accord-ing to their semantic and grammatical relationships
the deleted word and scores based on each ofthese categories are studied to find out which arethe most valid when the tests are used to measurereading ability and passage difficulty. (283)
Principles, Concepts, or Procedures: A 52-item cloze test wasconstructed over each of 20 passages. These were administeredto 50 fifth and sixth grade subjects. Two tests per testperiod were given during a ten day period. There were no timelimits. The Stanford Achievement Test: Reading was givenprior to the cloze testing.
Conclusion:::
1. When cloze tests are used as measures of individ-ual differences in reading ability, scores obtainedby counting responses exactly matching the deletedwords seem to yield the most valid scores.
2. When cloze tests are used in readability studiesas measures of the comprehension difficulties ofpassages, scores obtained by counting responses ex-actly matching the deleted words seem to yield thegreatest amount of discrimination among passage dif-ficulties. (285)
11
CULHANE, JOSEPH W. "CLOZE PROCEDURES AND COMPREHENSION." THEREADING TEACHER, 23 (1970), 410;413, 464.
Purpose: The author discusses the history of the cloze proce-dure, as well as the construction, scoring, and uses of clozematerials.
Principles, Concepts, or Procedures: The studies made of thecloze procedure since 1953 have demonstrated its usefulness tothe reading field particularly in the area of teaching andtesting comprehension.
Once the deletion pattern has been decided, a researcherdeletes words in every sentence of a passage, except the firstand last sentence, and replaces each deleted word with ablank space. The blanks should be of uniform length in test-ing situations. However, in teaching situations, they may beas long as the deleted word and may even begin with the firstletter of the deleted word. Exact word replacements shouldbe required when the exercises are used as a test device.Synonyms should be accepted in teaching situations and discus-sion of answers should take place.
Cloze is a useful technique for use with word callers,for it forces attention to context as the passage is read andnot afterwards. Cloze is also useful in promnting betterlanguage patterns.
Conclusions: The cloze technique is easily learned and put touse by both teachers and pupils. To complete deletions cor-rectly, careful reading and the use of context are required.
12
HAFNER, LAWRENCE E. "RELATIONSHIPS OF VARIOUS MEASURES TO THECLOZE." NEW CONCEPTS IN COLLEGE -ADULT READING. EDITEDBY ERIC L. THURSTON AND LAWRENCE E. HAFNER. THIRTEENTHYEARBOOK OF THE NATIONAL READING CONFERENCE. 'MILWAUKEE.WISCONSIN: NATIONAL READING CONFERENCE, 1963, 135..445.
Purpose: This researcher investigated the relationships ofvarious measures (vocabulary, intelligence, information, achieve-ment, personality, and reasoning) to pre-reading cloze.
Principles, Concepts or Procedures: Subjects were thirty-twomale and female senior college students enrolled in a readingmethods course.
A 50-item cloze test using every-fifth-word deletions wasdeveloped from an article on educational television. Amongother instruments used were: Michigan Vocabulary Profile Test;Otis Quick-Scoring Mental Ability Tests Gamma; Cattell 16 Per-sonality-Factor Questionnaire; and Wechsler-Bellevue Intelli-gence Scale, Information sub-test.
Conclusions:
1. Cloze scores correlated positively and significantlywith vocabulary, intelligence, information, achievement, andreasoning. They were not related to introversion.
2. Cloze scores compared favorably with standard predic-tors as a predictor of course grades.
13
HAFNER, LAWRENCE E. "IMPLICATIONS OF CLOZE." THE PHILOSOPHICALAND SOCIOLOGICAL BASES OF READING. EDITED BY ERIC L.THURSTON AND LAWRENCE E. HAFNER. FOURTEENTH YEARBOOK OFTHE NATIONAL READING CONFERENCE. MILWAUKEE, WISCONSIN:NATIONAL READING CONFERENCE, 1965, 151-158.
Purpose: The author discusses the implications .)f the clozeprocedure.
Principles. Concepts, or Procedures: The author indicatesthat cloze tests are valid measures of comprehension but shouldnot be used as replacements of informal inventories. Each cancomplement the other.
Cloze is useful for improving the writing of materials.To be intelligible, materials require an amount of redundancyas opposed to the terseness so often found in concept-ladenmaterial.
The author suggests that cloze can be modified to showthe role of reasoning in reading and to discover the dynamicsof the thinking process in individual students.
In addition to studies of readability, cloze has applica-tion to language creativity.
Cloze tests utilizing letter deletions as opposed to worddeletions yielded high correlations with elementary readingtests.
Conclusions: A center for the collection and dissemination ofcloze knowledge would be useful to education.
14
OHNMACHT, FRED; WEAVER, WENDELL W.; AND KOHLER, 51METT T."CLOZE AID CLOSURE: A FACTORIAL STUDY." THE JOURNAL OFPSYCHOLOGY, 74 (1970', 205-217.
Purpose: The authors explored
. the relationship of flexibility of closureand speed of closure to a number of cloze tasksrepresenting structural, lexical, concrete, andabstract deletions. (206)
Principles. Concepts, or Procedures: The Hidden Patterns Testand the Hidden Figures Test, and the Gestalt Com letion Testand the Concealed Words Test were administered to determineflexibility and speed of closure respectively. Measures ofassociational fluency and verbal comprehension were deter-mined from the Controlled Associations Test and the AssociationIV, and the Wide Range Vocabulary Test and the Advanced Vocabu-lary Test respectively.
The close tests were developed from materials previouslygraded for difficulty.1 The structural cloze tests deletedevery fifth word regardless of function. Lexical cloze testswere made by deleting every fifth noun, verb, or adjective.Abstract noun and concrete noun forms were developed from therationale that abstract nouns describe "objects" not perceiv-able by the senses while concrete nouns describe words whichare perceivable. Every-fifth-word deletions were also used inthese two forms.
Subjects were 115 high school students who were adminis-tered the test battery over a two week period. The close testscoring required exact replacements.
Conclusions:
1. Structural and concrete cloze tasks yielded higher meansthan did lexical and abstract ones.
2. Some relationship exists between flexibility and speedof closure and cloze tasks. Flexibility, however, is moreclosely related to cloze tasks.
1See G. R. Miller and E. B. Coleman, "A Set of Thirty-SixProse Passages Calibrated for Complexity," p. 61 of this thesis.
15
RANKIN, EARL F., JR. "THE CLOZE PROCEDURE - ITS VALIDITY ANDUTILITY." STARTING AND IMPROVING COLLEGE READING PRO-GRAMS. EDITED BY OSCAR S. CAUSEY AND WILLIAM ELLER'.EIGHTH YEARBOOK OF THE NATIONAL READING CONFERENCE. MIL-WAUKEE, WISCONSIN: NATIONAL READING CONFERENCE, 1959.131.444.
Purpose: The author
1. considers the validity of the cloze procedure as ameasure of readability, intelligence, pre-reading knowledge,and aspects of reading comprehension.
2. offers suggestions concerning the use of the procedure.
Principles, Concepts, or Procedures: An explanation of thecloze procedure and a comparison with other techniques such assentence completion tests are given initially.
As measures of readability, the traditional formulae areinsensitive to short, abstract words and sentences, and toirregular grammatical characteristics of some sentences.Cloze is a more reliable measure of readability, for it de-letes systematically words from sentence patterns so that thereader can replace them because of his ability to predict fromthe surrounding redundancy.
Cloze appears to be a measure of intelligence only whenall word classes have an equal chance of being deleted. Dele-tions of nouns, verbs, and adjectives yield very low correla-tions because of the difficulty in replacing them when allwords of those classes are missing from context.
As a measure of knowledge before reading, cloze is a validpredictor when a second test is used as a criterion measure.Deletions of nouns, verbs, and adjectives are most highly dis-criminating here.
Recent research indicates high correlations between clozescores and general reading comprehension. Lexical comprehen-sion can also be measured by cloze when high meaning units suchas nouns and verbs are deleted. High correlations have beenreported between cloze scores and immediate recall. Cloze is
16
presumably a valid measure of post-reading comprehension.
The usefulness of cloze tests can readily be seen when thetime required for writing objective questions is considered.Equivalent forms can be made from the same materials. Howeverthere is some evidence that subjects with above average anxietyperform poorly on cloze tests.
The classroom teacher will find cloze useful in determiningthe readability of textbook material, as a measure of generalcomprehension, and as a diagnostic tool where discussion of re-placed words occurs.
Conclusions: Many aspects of the cloze procedure require in-vestigation -.:(3 fully realize its potentialities.
RANKIN, EARL F. "CLOSURE AND THE CLOZE PROCEDURE." COLLEGEAND ADULT READING. EDITED BY DAVID M. WARK. CONFERENCEPROCEEDINGS OF THE NORTH CENTRAL READING ASSOCIATION.MINNEAPOLIS, MINNESOTA: UNIVERSITY OF MINNESOTA, 1964,54-84.
Purpose: The author reviews selected studies on closure rele-vant to the cloze procedure.
Principles, Concepts, or Procedures: Rankin begins by definingthe cloze procedure and tracing its rationale from Gestaltpsychology.
The nature of closure is next considered. The role of pre-vious experience and the implications of neurological theoryare discussed. The research studies of Street, Thurstone,Rimaldi, Bechtoldt, Pemberton, and Mooney are discussed rela-tive to speed and flexibility of closure. The relationship ofclosure to cognition, personality, reading, and age and sexdifferences are considered with reference made to research.
Rankin believes ". the cloze procedure probably in-volves cognitive more than visual closure." (74) Auditory-motor closure may, in some instances, be involved. These threebehavioral levels are not necessarily related.
Personality factors such as rigidity and impulsivity,social context, particularly anxiety, and age differences arecited as important considerations relative to closure, cloze,and the total reading process.
Conclusions: Rankin hopes a degree of closure will resultwithin individuals reading this paper.
A bibliography of sixty-seven references is included.
18
RANKIN, EARL F., JR. "RESEARCH DESIGN AND THE CLOZE PROCEDURE."VISTAS IN READING. EDITED BY J. ALLEN FIGUREL. PROCEED-INGS OF THE INTERNATIONAL READING ASSOCIATION. NEWARK,DELAWARE: INTERNATIONAL READING ASSOCIATION, 1966, 489-491.
Purpose, The author discusses two approaches to improving re-search design in studies utilizing the cloze procedure.
Principles. Concepts, or Procedures: Rankin contends that com-prehension is usually measured as an "after reading" product.Since individuals approach reading tasks with some previousknowledge it is desirable to know what information was gainedfrom the passage alone. The cloze procedure is an effectivedevice for measuring residual gain.
Because the skills required for success in the cloze pro-cedure are teachable, cloze has many possibilities as a teach-ing device.
Conclusions; The cloze procedure is a powerful research toolwith many possibilities yet to be realized.
19
TAYLOR, WILSON L. "RECENT DEVELOPMENTS IN THE USE OF 'CLOZEPROCEDURE.'" JOURNALISM QUARTERLY, 33 (1956), 42-48.
Purpose: The author reports results from recent studies heconducted to investigate the appropriateness of cloze as ameasure of comprehension in both English and Koread, of lis-tenability news broadcasts, and of intelligence, existingknowledge, and learning.
Principles, Concepts, or Procedures: Subjects in the SampsonAir Base study were administered two comprehension tests, twodoze tests, and an intelligence test to determine "before"and "after" learning as well as intelligence.
Easy, medium, and hard passages in theKorean languagewere adapted to doze and administered to Korean college stu-dents.
A "good" and a "bad" script both utilizing doze, wereprepared for two stories which were read by an experiencedannouncer to determine listenability.
Conclusions:
1. Cloze is a reliable measure of comprehension, and,hence, readability.
2. Learning and intelligence also seem to be measured.3. Cloze is a discriminator between the listenabili-ties of two oral presentations.
Among other conclusions were:
1. "There seems to be little advantage in preclassify-ing words and limiting deletions to them . ." (48)2. There was no advantage to scoring synonyms.3. Every fifth-word deletion located blanks as farapart as they needed to be.4. A series of about 50 deletions was sufficient toyield a stable score of the difficulty of a givenpassage.
20
TAYLOR, WILSON L. "RELATIVE INFLUENCE OF PRECEDING, FOLLOWING,AND SURROUNDING CONTEXTS ON CLUE PERFORMANCE." PAPERPRESENTED AT THE ANNUAL MEETING OF THE NhTIONAL READINGCONFERENCE, TAMPA, FLORIDA, DECEMBER 3, 1971.
Purpose: The influences of "preceding," "following," and"surrounding" contexts on cloze responses were investigated.
Principles. Concepts, or Procedures: Forty-five prose frag-ments were drawn frlm a short story, "The Egg," by SherwoodAnderson. Each fragment consisted of eleven words. Themiddle word in each fragment--the sixth--was deleted. Threeforms for each fragment were prepared; one presented both thefive "preceding" and "following" contexts while the remainingtwo presented either the "preceding" or "following" contexts.
The resulting 135 fragments were organized into threeassemblies of forty-five fragments each so that fifteen ofeach of the three versions of context appeared.
Subjects were nineteen college students. Each responsewas scored as like or unlike the original word and the re-sponses of other subjects.
Conclusions:
1. "Preceding" and "following" contexts were found to beabout equal in their influences on cloze responses.
2. Taylor notes the limitations of the study and suggestsreplication incorporating other subjects and materials.
21
TUINMAN, J. JAAP, AND BLANTON, B. EIGIT. THE CLOZE PROCEDURESCOROLLARIES OF A SEARCH HYPOTHESIS. BLOOMINGTON, INDIANA:SCHOOL OF EDUCATION, UNIVERSITY OF INDIANA, 1971.
Pumse: In order to determine whether the kind of process un-derlying cloze responses is indeed a systematic and exhaustivesearch, a study was conducted exploring some corollaries tosuch a search hypothesis.
Principles. Concepts, or Procedures: It was assumed that sub-jects would generate responses representing a number of wordtypes, that some of these word types would be sensible and somenonsensical, and that responses would be representative of theentire body of possible response words. Five versions of a300-word cloze passage, every fifth word deleted, were admin-istered to 390 junior-high-school students who were randomlyassigned to one of the versions. Protocols were hand scoredand success probabilities were calculated. A correlationmatrix among seven variables was calculated and analyzed usinga stepwise regression program.
Conclusions: Significant correlations were noted among theseven variables, with the highest correlation appearing betweensize of response body related to success probability. It wyclconcluded that the general search hypothesis appeared to besustained since distribution of responses was related to suc-cess probability and since the ratio of nonsense to sensibleresponses was relevant to that distribution. It was impliedthat a search process could be characterized as systematic inpart. Tables and references are included.1
1Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1971), Vol. 6, No. 6, pp. 104-105.(ED 047 913, 13 pages).
22
WEAVER, WENDELL W. "THEORETICAL ASPECTS OF THE CLOZE PROCEDURE."THE PHILOSOPHICAL AND SOCIOLOGICAL BASES OF READING.EDITED BY ERIC L. THURSTON AND LAWRENCE E. HAFNER. FOUR-TEENTH YEARBOOK OF THE NATIONAL READING CONFERENCE. MIL-WAUKEE, WISCONSIN: NATIONAL READING CONFERENCE, 1965, 115-132.
Purpose: The author discusses various theoretical aspects ofthe cloze procedure, particularly that of language and thought.
Principles, Concepts, or Procedures: Weaver questions whetherthe rationale for the cloze procedure is, in fact, Gestaltistin nature. Since there is no evidence that cloze units arecompleted in a perceptual fashion, he prefers to view the pro-cedure as a coding operation.
Language and thought are complexly bound together andform an important attribute of the cloze procedure. One meansof studying cloze in this situation is by making structuraland lexical deletions. To these deletions, syntdctic and se-mantic constraints are closely related.
Structural cloze scores are better predictors of reada-bility probably because they are dealing with the absencerather than the presence of redundancy. The cloze procedureis essentially a searching procedure which again emphasizesthe importance of language and thought.
Conclusions: Diverse areas of education and the social scienceshave made use of the cloze procedure. An area requiring studyis that of syntactic and semantic effects of context on particu-lar language units.
t
CHAPTER II
THE CLO2E PROCEDURE AS A MEASURE
OF READABILITY
The initial research related to the cloze procedure
as a measure of readability was undertaken by Taylor1who
found that prose passages written by Stein, Caldwell, and
Joyce were assessed more accurately by the cloze procedure
than by the Dale-Chall and Flesch formulae. Moreover,
Taylor found that selected passages were ranked in exactly
the same order of difficulty by these three methods.
Bormuth2
also investigated the validity of assess-
ing readability by means of cloze tests. The influences
of sample size and number of items on the precision of
cloze tests used for readability purposes were also inves-
tigated by Bormuth.3
Later, Bormuth4isolated factors of
1Wilson L. Taylor, "'Cloze Procedure': A New Toolfor Measuring Readability," Journalism Quarterly, 30(1953), 414-438.
2John Robert Bormuth, "Cloze Tests as Measures of
Readability and Comprehension Ability" (unpublished Ed.D.dissertation, Indiana University, 1962).
,Bormuth, "Optimum Sample Size and Cloze TestLength in Readability Measurement," Journal of EducationalMeasurement, 2 (1965), 111-116.
4Bormuth, "Readability: A New Approach," Reading
Research Quarterly, 1 (1966), 79-131: John R. Bormouth.
23
24
importance in the development of readability formulae.1
Bormuth and MacDonald tested the hypothesis that
cloze test scores are positively correlated with the ability
to detect an author's style.
Specific instructional materials designed for use
by students have been assessed for readability by means of2
the cloze procedure. Gallant estimated the readability of
Covington3assessed theprimary grade reading materials.
readability of third, and fourth4
textbooks, while Kulm estimated
tary algebra textbooks.
grade modern mathematics
the readability of elemen-
"New Data on Readability," Forging Ahead in Reading, ed. byJ. Allen Figurel, Proceedings of the International ReadingAssociation (Newark, Delaware: International Reading Asso-ciation, 1967), pp. 488-492; Cloze Readability Procedure,(Los Angeles: Center for the Study of Evaluation of In-structional Programs, University of California, 1967).
1John R. Bormuth and 0. L. MacDonald, "Cloze Testsas a Measure of Ability to Detect Literary Style," Readingand Inquiry, ed. by J. Allen Figurel, Proceedings of theInternational Reading Association (Newark, Delaware: Inter-national Reading Association, 1965), pp. 287-290.
2Ruth Margaret Frances Gallant, "An Investigation
of the Use of Cloze Tests as a Measure of Readability ofMaterials for the Primary Grades" (unpublished Ed.D. dis-sertation, Indiana University, 1964).
3Richard John Lee Covington, "An Analysis of Reada-bility of Third and Fourth Grade Modern Mathematics Text-books Using the Cloze Procedure" (unpublished Ed.D. disser-tation, University of California, Los Angeles, 1966).
4Gerald Kulm, "Measuring the Readability of Elemen-
tary Algebra Using the Cloze Technique" (paper presentedat the Annual Meeting of the American Educational ResearchAssociation, New Yol N. Y., February 4-7, 1971).
25
The readability of encyclopedia content designed
for elementary school pupils has been assessed by Liske.1
2Froese compared Dale-Chall readability ratings of sixth
grade science textbooks with cloze results. The readabil-
ity of certain high school textbooks was determined by
Beard.3
The cloze procedure was utilized by Froelich in
mPasuring the readability of selected college level elec-
tronics textbooks, and by Houska5in estimating the reada-
bility of industrial education content material at the high
school level.
The development of other readability scales has
ZWilfred Wallace Liske, "The Cloze Procedure forDetermining Readability of Encyclopedia Material for Ele-mentary School Pupils" (unpublished Ed.D. dissertation,University of Maryland, 1968).
2Victor Froese, "Cloze Readability Versus the Dale-
Chall Formula" (paper presented at the Annual Meeting ofthe International Reading Association, Atlantic City, NewJersey, April 19-23, 1971).
3Jacob G. Beard, "Comprehensibility of High SchoolTextbooks: Association With Content Area," Journal ofReading, 11 (1967), 229-234.
4Donald Max Froelich, "A Comparison of Two Methods
of Assessing Textbook Readability of Selected CollegeLevel Electronics Textbooks" (unpublished Ed.D. disserta-tion, University of Missouri - Columbia, 1970).
5Joceph Thomas Houska, "The Efficacy of the ClozeProcedure as a Readability Tool on Technical Content Ma-terial as Used in Industrial Education at the High SchoolLevel' (unpublished Ed.D. dissertation, University ofIllinois at Urbana-Champaign, 1971).
26
been facilitated by the cloze procedure. Miller and Cole-
man1utilized three different cloze techniques to develop
a set of prose passages ranging in difficulty from first
grade reading material to technical material. The result-
ing Miller-Coleman Readability Scale has been validated by2
Aquino.
Fry3 utilized the doze procedure as one of three
criterion measures in assessing the validity of the Fry
Readability Graph. Noun frequency counts in _combination
with the doze procedure have been used by Elley to mea-
sure readability.
With respect to readability, stndies have focused
on the difficulties within prose selections. Taylor5
1G. R. Miller and E. B. Coleman, "A Set of Thirty-
six Prose Passages Calibrated for Complexity," Journal ofVerbal Learning and Verbal Behavior, 6 (1967), 851-8547--
2Milagros Aquino, "The Validity of the Miller-Coleman Readability Scale," Reading Research Quarterly, 4(1969). 342-357.
3Edward B. Fry, The Readability Graph Validated
mat:PrUumjraAlq (Cambridge, Massachusetts: ComputerAssisted Instruction Laboratory, Graduate School of Educa-tion, Harvard University, 1969).
4Warwick B. Elley, "The Assessment of Readability
by Noun Frequency Counts," Reading Research quarterly, 4(1969), 411-427.
5Wilson L. Taylor, "Application of 'Cloze' andEntropy Measures to the Study of Contextual Constraint inSamples of Continuous Prose," (unpublished Ph.D. disser-tation, University of Illinois, 1954).
27
and MacGinitieI
investigated the effects of contextual con-
straint in English prose paragraphs on readability and
comprehension, while Aborn, Rubenstein, and Sterling and
Blake3
investigated contextual constraint upcn words within
sentences.
4Jefferson analyzed lexical and structural cloze
deletions as measures of readability. Rentel5assessed the
effects of combining a word classification system with word
length to measure the difficulty of certain word categories.
1Walter Harold MacGinitie, "Contextual Constraint
in English Prose" (unpublished Ph.D. dissertation, ColumbiaUniversity, 1960).
2Murray Aborn, Herbert Rubenstein, and Theodor D.Sterling, "Sources of Contextual Constraint Upon Words inSentences," Journal, of Experimental Psycholomr, 57 (1959).171-180.
3James Herbert Blake, "Cloze Scores, AgreementScores, and Approximated Equivalence Scores as Indices ofContextual Constraint" (unpublished Ph.D. dissertation,Syracuse University, 1970).
4George Lee Jefferson, Jr., "Lexical and Structural
Items as Predictors of Readability for High and Low AbilityReaders" (unpublished Ph.D. dissertation, University ofGeorgia, 1969).
5Victor Michael Rentel, "Form Class and Word Lengthas Indices of the Difficulty of Predicting Cloze Entries ina Sample of Continuous Prose" (unpublished Ph.D. disserta-tion, University of South Carolina, 1969).
28
ABORN, MURRAY; RUBENSTEIN, HERBERT; AND STERLING, THEODOR D."SOURCES OF CONTEXTUAL CONSTRAINT UPON WORDS IN SEN-TENCES." JOURNAL OF EXPERIMENTAL PSYCHOLOGY, 57 (1959),171-180.
Purpose: "This study investigated the constraint upon wordsattributable to the length, distribution, and structure ofcontext consisting of incomplete sentences." (171)
Principles, Concepts, or Procedures: From about 3000 sen-tences drawn at random from a number of popular magazines,1380 were selected.
41.
One word was omitted from each sentence in a waythat yielded three treatments of sentence length,within which were four treatments of position ofomission, within which were six treatments of wordclass of the omitted word. (179)
Subjects were 24 second-semester freshmen enrolled atthe University of Alabama. There were twenty 90 minute ses-sions lasting four, weeks. Only exact-word replacements wereaccepted.
Conclusions: Among the conclusions were:
1. The length, distribution, and grammatical struc-ture of context are all independently effectivesources of constraint on words in sentences.
2. The predictability of words belonging to a givenword class is, in general, inversely related tothe size of that class.
3. Increasing the context beyond 10words does notincrease predictability. The length at which con-text attains maximum effectiveness lies between5 and*10 words.
4. A bilaterally distributed context exerts greaterconstraint than a totally preceding or totallyfollowing context of the same length. . . . (179)
29
AQUINO, MILAGROS. "THE VALIDITY OF THE MILLER-COLEMANREADABILITY SCALE." READING RESEARCH QUARTERLY, 4(1969), 342.-357.1
Purpose: The author reports an investigation to validatethe Miller-Coleman passages calibrated for complexity, other-wise known as the Miller-Coleman Readability Scale.
Principles, Concepts, or Procedures: Fourteen adult subjectsread six passages for six consecutive working days. One minutewas allowed for each passage to be read. Subjects were thenrequired to recall the passage orally or in writing. Correctnoun, verb, adjective, and adverb replacements were scored.
At the conclusion of this phase of the study, subjectswere required to place the 36 passage cards in five stacksaccording to difficulty. Subjects then ranked each stack ofcards for relative difficulty.
Conclusions: Word-for-word recall and judged difficulty cor-related highly with both the bilateral and unilateral clozeaspects of the scale.
/See.also G. R. Miller, and B. B. Coleman "A Set ofThirty-Six Passages Calibrated for Comprehensibility,"p. 61 of this thesis.
30
BEARD, JACOB G. "COMPREHENSIBILITY OF HIGH SCHOOL TEXTBOOKS:ASSOCIATION WITH CONTENT AREA." JOURNAL OF READING, 11(1967), 229-234.
Purpose: The author investigated ". . . differences in com-prehensibility of textbooks associated with school subjectmatter content areas." (229)
Principles, Concepts, or Procedures: Ten passages of 250words each were randomly selected from each of four areas:biology, chemistry, American government, and world history.
Subjects were 250 tenth grade students in Tampa, Floridawho were sampled from five ability levels according to verbalscores received on the School and College Ability Test.
Five forms of a cloze test were prepared for each pas-sage. Since every fifth-word deletions were used, every wordwas a restorable item in each form. Only exact replacementswere scored correct.
Comprehensibility was also measured by using the averagenumber of words per T-unit. Each passage was also
analyzed to determine the extent of use of10 elements of expressional style: monosyllablewords, sentence length, first person pronouns,second person pronouns, third person pronouns,different words, complex sentences, infinitivephrases, words beginning with the letter "i," andabstract nouns. (233-234)
Conclusions: "Small and insignificant differences werefound for the means of the comprehensibility measures . . ."(234) and for all the structural variables except "i" wordsand infinitive phrases.
It was concluded that the comprehensibility ofprose used in current high school textbooks is,on the average, about the same for Americangovernment, world history, biology, and chemistry.(234)
31
BLAKE, JAMES HERBERT. "CLOZE SCORES, AGREEMENT SCORES, ANDAPPROXIMATED EQUIVALENCE SCORES AS INDICES OF CONTEXTUALCONSTRAINT." UNPUBLISHED PH.D. DISSERTATION, SYRACUSEUNIVERSITY, 1970.
Purpose: Problems in indexing a language variable, contextualconstraint, were considered in this study. The contextual con-straint of a position in connected discourse is the degree towhich the words elsewhere in the discourse determine what wordoccupies the position.
Principles, Procedures: Three indices were con-sidered: the cloze score, the agreement score, and the approxi-mated equivalence score. These are scores that can be calculatedfrom data obtained by means of a deletion-test item.
The cloze score is computed according to the followingformula:
Cloze score = P1
in which P1 is the relative frequency of the response identicalto the deleted word.
The agreement score is the ratio of the amount of agree-ment among the responses to a deletion-test item and the great-est amount of agreement possible. As a computational proce-dure, the agreement score is identical to the Shannon redun-dancy index. . .
The subjects were undergraduate history students who par-ticipated in two testing sessions. During the first sessionthey responded to a variety of deletion-test items. Duringthe second session they responded to a variety of free-associa-tion-test items. The stimuli for these latter items were wordsthat the subjects had offered frequently in response to thedeletion-test items.
Conclusions: The cloze score and the agreement score werecompared with respect to detecting known differences in contex-tual constraint. Approximations to English were the sourcesof these known differences in contextual constraint. The
32
agreement score was found to be more sensitive than the clozescore to these differences.
The cloze score and the agreement score were each used inattempts to detect differences in the contextual constraintswithin twenty-five-word sentences and the contextual con-straints within eleven-word sentences. Regardless of the in-dex used, no differences were detected. It could not be de-termined whether there were in fact no differences or whetherboth indices were too insensitive to detect whatever differ-ences were present.
The cloze score and the approximated equivalence scorewere compared with respect to estimating contextual constraint.Estimates were higher by small amounts when approximated equi-valence scores indexed contextual constraint than when clozescores estimated contextual constraint. Even when contextualconstraints were known to be very small, approximated equiva-lence scores provided higher estimates than cloze scores.
The cloze score and -he approximatel equivalence scorewere each used to estimate the contextual constraints upon se-lected positions in an ecperimental passage. Scores exceededa 0.100 criterion for a greater number of positions whenapproximated equivalence scores indexed contextual constraintthan when cloze scores indexed contextual constraint. Thus, agreater number of positions were judged to be under substan-tial contextual constraint when approximated equivalencescores were used.
It was concluded that when researchers must detect differ-ences in contextual constraint, the cloze score is not the mostappropriate index. In such situations, the researcher shoulduse an index more sensitive to differences in contextual con-straint. The agreement score apparently is more sensitive thanthe doze score. In the future, however, other indices mayprove even more sensitive than the agreement score.
In addition, it was concluded that when researchers mustdetect even small amounts of contextual constraint, the dozescore is not the most appropriate index. The approximatedequivalence score is more appropriate because it is less likelyto underestimate contextual constraint. In the future otherindices may prove to be even more appropriate than the approxi-mated equivalence score in such situations.
Future research projects might involve a search for more
33
easily computed indices that have the desirable properties ofagreement scores and approximattql equivalence scores. Inaddition, future projects might use cloze scores, agreementscores, and approximated equivalence scores to predict theincidence of substitution-type word recognition errors. Pre-dictions about the words substituted and predictions aboutthe effects of each substitution on comprehension might bemade. Such predictions could be compared with actual readingperformances.1
110slrlterDertationAbstznationalle Humanitiespnd Social Sciences (Annbor,tArdichigan:University Micro-films, 1971), Vol. 31, No. 11, pp. 5923-A, 5924-A. (OrderNo. 71-10,892, 331 pages).
34
BORMUTH, JOHN ROBERT. "CLOZE TESTS AS MEASURES OF READABIL-ITY AND COMPREHENSION ABILITY." UNPUBLISHED ED.D. DIS-SERTATION, INDIANA UNIVERSITY, 1962.1
Purpose: The purpose of this study was to investigate the va-lidity of cloze tests as measures of the readability of thepassages from which they are made. Since the readability ofa passage is defined by the amount of comprehension withwhich it is read, it was necessary to investigate three as-pects of cloze tests. These were (1) their validities asmeasures of comprehension ability, (2) their efficiencieswhen used for the purposes of discriminating among the diffi-culties of tests and the abilities of individuals, and (3)their validities as measures of the amounts of comprehensionwith which passages are read.
Principles, Concepts, or Procedures: Nine passages were used.These were equally divided among three classes of subject mat-ters and three levels of difficulty. A 31 item comprehensiontest was written over each passage. Each test contained itemsmeasuring each of seven comprehension skills. By counting asubject's correct responses to the items measuring one skillin the entire set of tests, a measure of ability in that skillwas obtained. Cloze tests were made over each passage by re-placing every fifth word with a blank until 50 items were ob-tained. Only when a subject could write in the blank the exactword deleted was his response scored correct. All tests wereadministered to three groups of 50 subjects each. 4 group wascomposed of children enrolled at one grade level. The gradelevels of the groups were grades 4. 5, and 6.
Conclusions: The findings supported the hypothesis that clozetests are valid measures of comprehension ability. Scores oneach of the nine cloze tests correlated with the scores oneach of the seven tests of comprehension skills. Analysis ofthe variance of cloze scores showed that they were affected bythe variables which affect comprehension scores. A similaranalysis was applied to the scores of comprehension tests.The proportion of the variances due to each variable was foundto be similar on the cloze and comprehension tests. All cor-relations between cloze and comprehension scores on tests over
/See also John R. Bormuth, "Cloze as a Measure of Readabil-ity," Appendix.
35
the same passages were significant. The correlations between'he totals of the comprehension and cloze scores reached .946.
The findings supported the hypothesis that cloze testsare efficient when used for the purposes of measuring read-ability and comprehension ability. The difficulty values ofthe items on each test were distributed over the entire rangeof difficulties. The items had high correlations with thetotal scoroa on the tests which indicated that they tended tobe intercorrelated. When test reliabilities were estimatedby the method of rational equivalence, the median reliabilitywas .837. When reliability was estimated from correlationsamong tests over passages in the same class of subject matter,the medial reliability was .802.
The findings supported the hypothesis that clozz, testsmeasured the readability of the passages in this study. Thedifficulty rankings of the cloze tests were correlated withthe difficulty rankings of the comprehension tests over thesame passages. All correlations were significant. The com-prehension test rankings were loss reliable than the clozetest rankings. When the difficulty rankings of the compre:Ien-sion tests were taken from the scores of the total group,their correlations with the cloze test rankings by each groupranged from .883 to .929.1
1Dissertation Abstracts (Ann Arbor, Michigan: UniversityMicrofilms, 1963), Vol. 23, No. 11, pp. 4218-4219. (Order No.63-2586. 205 pages).
36
BORMUTH, JOHN R. "OPTIMUM SAMPLE SIZE AND CLOZE TEST LENGTHIN READABILITY MEASUREMENT." JOURNAL OF EDUCATIONALMEASUREMENT, 2 (1965), 111-116.
Purpose: The purpose of the study was ". to estimatethe number of subjects and number of items required toattain various levels of precision in readability experi-ments involving cloze I sts." (111)
Principles. Concepts. or Procedures: The procedure reportedhere is the same as that reported in "Experimental Applica-tions of Cloze Tests."1
Conclusions: "Decisions of optimum research design musttake into account theoretical, statistical, and economic fac-tors." (115) The table of average standard errors developedas an adjunct to the experiment ". . shows that error isreduced more by adding a given number of items than by add-ing the same number of subjects." (115)
1See page 8 of this thesis.
3?
BORMUTH, JOHN R. "READABILITY: A NEW APPROACH." READINGRESEARCH QUARTERLY, 1 (1966), 79-131.
Purpose: "Five problems basic to the development of precisereadability fcmulae were investigated." (79)
Principles, Concepts or Procedures: The procedure reportedhere is the same as that reported in "Experimental Applica-tions of Cloze Tests."'
Conclusions:
1. Nonlinear correlation techniques are required in somereadability formulae.
2. Readability formulae can predict difficulty for thestudents at all levels of ability.
3. Difficulties inherent in words, independent clauses,and sentences can be predicted by readability formulae.
4. The high correlation between linguistic variables andpassage difficulty (.934) indicates that the validity of for-mulae based on linguistic variables can be improved.
5. The development of more sophisticated linguisticvariables will yield the greatest improvement in readabilityprediction.
See page 8 of this thesis.
38
BORMUTH, JOHN R. "NEW DATA ON READABILITY." FORGING AHEAD INREADING. EDITED BY J. ALLEN FIGUREL. PROCEEDINGS OF THEINTERNATIONAL READING ASSOCIATION. NEWARK, DELAWARE:INTERNATIONAL READING ASSOCIATION, 1967, 488-492.
Purpose: The author describes recent research in readability.
Principles, Concepts, or Procedures:
1. Measuring the difficulty of reading materials is con-
sidered first. Traditionally multiple choice tests have beenused for this purpose. Because one can never be sure that heis not just measuring the difficulty of the test questionsand because word, phrase, and sentence difficulties within thepassage are thereby overlooked, multiple choice tests are notsatisfactory measures. The cloze procedure, however, providesa valuable alternative, for the aforementioned difficultiesare not existent therein.
2. The measurement and description of language is nextdiscussed. "Present investigators are probing more deeplyinto the question of what makes a word difficult to under-stand." (489) Both word length and morphological complexityhave been shown not to be sources of difficulty. The fre-quency with which a word is used may be of significance. "The
degree of intricacy of the grammatical relationships betweenthe parts of a sentence . ." is ". . . an important sourceof difficulty in understanding the sentence." (490) Syntacticdepth, modifier distance, and transformational complexity arehighly related to sentence difficulty. "Modern researchersare looking beyond the word and the sentence to find the fea-tures of language that operate over longer segments of text toinfluence comprehension." (491)
3. The progressive development of readability formulae isoutlined. An important finding has been that ". . . regardlessof a person's reading ability, the same features of languagethat caused difficulty for him caused the same amount of diffi-culty for others." (491) Future research must examine trans-formations of wards and also yield profiles of difficulty foreach of the language features in a passage.
39
Conclusions: Because psychologists and linguists have de-veloped new tools and because of increased understanding ofmathematics, rapid strides have been and will continue to bemade in readability research.
40
BORMUTH, JOHN R. CLOZE READABILITY PROCEDURE. LOS ANGELES:CENTER FOR THE STUDY OF EVALUATION OF INSTRUCTIONAL PRO-GRAMS, UNIVERSITY OF CALIFORNIA, 1967.1
Purpose: The research related to the cloze procedure as ameasure of comprehension difficulty of written materials wasreviewed.
Principles, Concepts, or Procedures: The characteristics, thevalidity, and the application of the cloze procedure are dis-cussed. Among the topics are the following:
1. Measurement of past-reading knowledge.2. Measurement of knowledge gain.3. Measurement of passage difficulty.4. Cloze test reliability.5. Designing the testing procedure.6. Deletion and scoring procedures.7. Test administration.8. Interpretation of scores.
Conclusions: The cloze procedure yields results which are morevalid than those available from readability formulae. The pro-cedure is applicable to a wide range of evaluation tasks.
1See also John R. Bormuth, "The Implications and Uses of
the Cloze Procedure in the Evaluation of Instructional Pro-grams," Appendix.
41
BORMUTH, JOHN R., AND MACDONALD, 0. L. "CLOZE TESTS AS A MEA-SURE OF ABILITY TO DETECT LITERARY STYLE." READING ANDINQUIRY. EDITED BY J. ALLEN FIGUREL. PROCEEDINGS OFTHE INTERNATIONAL READING ASSOCIATION. NEWARK, DELAWARE:INTERNATIONAL READING ASSOCIATION, 1965, 287-290.
Purposes Two hypotheses were tested.
1. there is a positive correlation betweensubjects' scores on cloze tests, and their scoreson tests of ability to detect an author's style.
2. scores on the test of ability to detectthe author's style . . . have a higher correlationwith the scores on the cloze test given after thesubjects . read the book than with the scoreson the test given to the subjects before theyread the book. (289)
Principles, Concepts, or Procedures: Tests designed to measurethe literary styles of Trollope and Steinbeck were constructed.Chosen were 100 sample passages of about 40 words each, all ofwhich represented Trollope's work. An additional 100 sampleswere drawn from the works of other authors. The 200 passageswere typed on cards which were later shuffled. The subjectswere 150 female college students who were required to read eachpassage and indicate which were written by Trollope. The sameprocedure was used with Steinbeck's writings.
The subjects were given a cloze test of 100 items madeover Barchester Towers by Trollope before being assigned thebook to read over a four week period. The author, his style,and the period were discussed in class. A second cloze testwas given at the end of this period. The same procedure wasused with respect to Of Mice and Men by Steinbeck.
Conclusions: The first hypothesis was significant at the .01level and accepted, while the second hypothesis was not con-firmed. It may be confidently assumed that a reader's sensi-tivity to style is an important variable in determining how wellthe reader will perform on a cloze test. Further, subjects whohave not read the materials from which cloza tests were madeseem to do as well as those who have studied the materials.This latter finding seemingly contradicts earlier studies byRankin, and Weaver and Kingston.
42
COVINGTON, RICHARD JOHN LEE. "AN ANALYSIS OF READABILITY OFTHIRD AND FOURTH GRADE MODERN MATHEMATICS TEXTBOOKSUSING THE CLOZE PROCEDURE." UNPUBLISHED ED.D. DISSER-TATION, UNIVERSITY OF CALIFORNIA, LOS ANGELES, 1966.
Purpose: The purpose of this study has been to employ thedoze procedure of test construction and evaluation in assess-ing the readability of two separate modern mathematics text-books, one a third grade textbook and the other a fourth gradetextbook, both published by Science Research Associates.
Principles, Concepts, o:were taken from each telfifth word basis. In ad _
used in which Form A ofword one, Form B with woemaking 50 deletions, Aeach deletion.
zedures: Six 250-word passages-A and words deleted on an every`ion, three deletion patterns were
test deleted words starting withthree, and Form C with word five,ndard length blank was used for
The textbook passages were divided among three testtypes: word problems, statements of directions which tellpupils to perform some behavior in arithmetic activities, andstatements of instructions which present presumably new mathe-matical concepts.
Pupils were divided into three ability groups. The firstgroup included pupils on the California Test of Mental Matu-rity publisher's standardized stanines 1, 2, and 3. Thesecond group from stanines 4, 5, and 6; and the third abilitygroup from stanines 7, 8, and 9. Group two indicates pupilsof average ability and group three above average. Group one,because of extremely small N's, was dropped from statisticalanalysis.
Conclusions: Assessment of readability was based on two mea-surements. The first was a count of those pupils' tests scor-ing 17 or more correct or 16 or less correct on each form andin each ability group. The other measure was the mean correctresponses for each :Form, each test, and by ability group.
On the basis of the assumptions for the study, thirdgrade second ability group pupils did not meet criterion scoreson any tests. Third grade third ability group and all fourthgrade pupils met or exceeded criterion scores only on wordproblems.
43
Therefore, by inference, the third and fourth gradeGreater Cleveland Mathematics Program textbooks are not read-able for third and fourth grade pupils, except for word prob-lems.1
1Dissertation Abstracts A: The Humanities and SocialSciences (Ann Arbor, Michigan: University Microfilms, 1067),Vol. 27, No. 10. (Order No. 67-4509, 108 pages).
44
ELLEY, WARWICK B. "THE ASSESSMENT OF READABILITY BY NOUN FRE-QUENCY COUNTS." READING RESEARCH QUARTERLY, 4 (1969), 411 -427.
Purpose: The author describes the assessment of readabilityusing noun frequency counts.
Principles, Concepts, or Procedures: Ten passages of approxi-mately 150 words were devised. Every sixth word was deleted.Subjects were seventy secondary school pupils. Replacementsof verbs, pronouns, adjectives, adverbs, conjunctions, andprepositions were superior in percentage scores to noun re-placements.
These results and similar findings.by other researchersprompted the author to consult teachers' and pupils' ratingsas criterion measures. Noun frequency levels were consistent-ly better predictors of difficulty.
Conclusions: The author recommends further investigation ofthe use of cloze scores as criterion measures.
45
FROELICH, DONALD MAX. "A COMPARISON OF TWO METHODS OF ASSESS-ING TEXTBOOK READABILITY OF SELECTED COLLEGE LEVEL ELEC-TRONICS TEXTBOOKS." UNPUBLISHED ED.D. DISSERTATION,UNIVERSITY OF MISSOURI - COLUMBIA, 1970.
Purpose: The purpose of this study was to compare the clozereadability technique with the Flesch Reading Ease Formula toascertain the effectiveness of each in assessing the readabil-ity of selected college level electronics textbooks. The com-parison was made in terms of student achievement on a multiplechoice test over a passage selected from a college level elec-tronics textbook.
Principles, Concepts, or Procedures: The study was conductedin three state colleges in Missouri and involved students en-rolled in a basic electronics course in the Department of Indus-trial Education in each school.
The Flesch Reading Ease Formula was chosen for comparisonwith the cloze procedure in this study because of its generalacceptance by textbook publishers and others as a convenientreadability assessment technique.
Pearson product-moment correlations were computed to showthe relationship between the cloze tests and the achievementtest. T-tests of the difference between two means for corre-lated samples and for independent samples were computed toanalyze the difference of the measures at both levels of reada-bility.
A criterion cloze test score was empltyed to assess thewritten material as acceptable or not acceptable in comparisonwith reading abilities of the students.
Conclusions: Cloze test scores identified the readabilitylevels of written technical material in a manner more consis-tent with the abilities of college students to comprehend thematerial as assessed by scores on an achievement test over thewritten technical material than the readability level assess-ment made by application of the Flesch Reading Ease Formula.
An analysis of the mean cloze test scores and of the meanachievement test scores revealed that although the Flesch Read-ing Ease Formula rated the technical material at the ninth and
46
at the fifteenth grade levels of readability, the ability of thestudents to comprehend the written material, as assessed by theresults of the achievement test, was not predicted by the reada-bility levels as assessed by the Flesch Reading Ease Formula.
The Flesch Reading Ease formula did not identify the reada-bility of written technical material in a manner that was con-sistent with the ability of the college student to comprehendthe material as assessed by an achievement test over the samematerial.
The results from an analysis of the mean scores of theachievement test revealed that no significant difference existedbetween the two levels of readability of the written materialassessed by the Flesch Reding Ease Formula. In addition, theanalysis of the achievement test scores revealed that there wasa significant difference in the ability of the students to com-prehend the written material taken from the first one-half andfrom the second one-half of the selected textbook passage. TheFlesch Reading Ease Formula failed to identify the differencein content difficulty of the material in each one-half of theselected textbook passage.
The use of the close readability procedure in assessingthe readability of college level electronics textbooks in termsof student comprehension of the material may be considered to bea reliable technique.
Comparison of the results in analyzing the difference ofthe mean close test scores over both halves of the selected text-book passage and the difference of the achievement test scoresover both halves of the selected passage indicated that thecloze readability procedure and the achievement test were inagreement in assessing the written technical material selectedfrom the college level electronics textbook.
In view of the evidence presented in this study, it isapparent that factors in addition to those included in theFlesch Reading Ease Formula should be included in the assess-ment of the readability of college level electronics textbooks.
1Dissertation Abstracts International A: The Humanitiesand Soc a ciences Ann Arbor, Mic gan: in vers tirMicro-films, 1971), Vol. 31, No. 8, p. 3917-A. (Order No. 71 -3329,165 pages).
FROESE, VICTOR. "CLOZE READABILITY VERSUS THE DALE-CHALL FOR-MULA." PAPER PRESENTED AT THE ANNUAL MEETING OF THE IN-TERNATIONAL READING ASSOCIATION, ATLANTIC CITY, NEW JERSEY,APRIL 19-23, 1971.
Purpose: The validity of the Dale-Chall readability ratings forsixth-grade science textbooks when compared to an independentcriterion of language difficulty expressed in cloze units wasexamined.
Principles, Concepts, or Procedures: The Canadian Lore- Thorn-dike IQ Test and the Canadian Test of Basic Skills, Vocabularyand Comprehension subtests, were administered to 366 sixth gra-ders. Mean IQ score was 102, while mean scores for vocabularyand comprehension were 6.19 and 6.15, respectively. Passagesfrom 12 science textbooks were subjected to the cloze techniqueand administered to the subjects.
Conclusions: Findings revealed that (1) the 12 cloze passagesyielded a reliability coefficient of .791 (2) cloze scores cor-related from .55 to .85 with the intelligence measure; (3)cloze scores correlated from .64 to .86 with vocabulary, andfrom .52 to .85 with comprehension; (4) the cross-validationcoefficient of .90 obtained by Dale and Chall and the cross-validation coefficient of .29 obtained in this study are sig-nificantly different beyond the .01 level of confidence. Majorconclusions were (1) that the cloze tests are reliable measuresof language difficulty and (2) that the Dale-Chall ReadabilityFormula is not a valid measure of sixth-grade science textbookmaterials when the cloze procedure is,used as a criterion. Re-ferences are included,1
1Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1971), Vol. 6, No. 10, p. 76.(ED 051 975, 12 pages).
48
FRY, EDWARD B. THE READABILITY GRAPH VALIDATED AT PRIMARYLEVELS. CAMBRIDGE, MASSACHUSETTS: COMPUTER ASSISTEDINSTRUCTION LABORATORY, GRADUATE SCHOOL OF EDUCATION,HARVARD UNIVERSITY, 1969.
Purpose: The validity of the Fry Readability Graph was com-pared with a readability formula, the doze technique, andoral reading.
Principles, Concepts, or Procedures: Passages from seven pri-mary level books were ranked by the Fry Readability Graph, theSpache Readability Formula, and oral reading error a:ores, aswell as the doze procedure.
In the latter instance thirty second and third gradepupils read aloud from the seven book passages from whichevery fifteenth word was deleted if it was classified as anoun, verb, adverb, or adjective. Words of these classesnearest the fifteenth were deleted if the criterion couldnot be met, Scores were also obtained for the oral readingperformance of the children when the words were not deleted.
Conclusions: All four methods ranked the passages well. "Inmy opinion the cloze method was the most accurate a:id made thefinest distinctions." (6)
49
GALLANT, RUTH MARGARET FRANCES. "AN INVESTIGATION OF THE USEOF CLOZE TESTS AS A MEASURE OF READABILITY OF MATERIALSFOR THE PRIMARY GRADES." ,UNPUBLISHED ED.D. DISSERTATION,INDIANA UNIVERSITY, 1964.1
Purpose: This study dealt with two related problems, (1) thereliability and validity of cloze tests as a measure of readingcomprehension for pupils in the primary grades, and (2) theeffect of increased sentence length on the readability of ma-terials designed for use with these pupils.
Principles, Concepts, or Procedures: The procedure for problemone involved the rewriting of the paragraph section of theMetropolitan Reading Achievement. Tests for grades one, two, andthree. A modified form of cloze procedure was devised for usewith grade one. In regular cloze procedure, every fifth wordis replaced by a blank line on which the subject writes his re-sponse. In the revised form used with grade one, three multiple-choice responses were provided for every deleted fifth word inthe passages.
The cloze tests and a comparable form of the Metropolitan,Test were administered to 273 pupils. Pearson product-momentcorrelation was used to determine if the rankings of pupilson the cloze test corresponded with thy -ankings on the stan-dardized achievement test. Within each grade level, the cor-relation obtained for the boys was compared to that of thegirls. A two-way analysis of variance was used to test thesignificance of the differences between the cloze test scoreswhen the pupils within each grade were subdivided by sex andclassified by tested reading achievement levels. The analysiswas calculated for second and third grade scores, each gradelevel handled as a separate group.
For the second problem, two sets of passages were writ-ten, each consisting of five levels of reading difficulty asmeasured by the Spache readability formula. The revisedpassages were similar to the basal passages in number of wordsper passage and difficulty of vocabulary. Sentence length hadbeen increased in the revised passages so that the Spachcreadability level ranged from three months to one year higherthan that of the basal passages. Cloze tests over both setsof passages were administered to the 273 pupils. Differences
1See also Ruth Gallant, "Use of Cloze Tests as a Measure
of Readability in the Primary Grades," Appendix.
50
in mean cloze performance were tested for significance of eachgrade level.
Conclusions: Findings were as follcwst the correlations be-tween the cloze tests and the standardized reading tests foreach grade in total and for each grade subdivided by sex weresignificant. The analysis of variance corroborated the va-lidity of cloze scores as a measure of reading comprehension.The effect of sex on cloze test performance was shown to besignificant for the second and third grade scores. Althoughthe relationship of cloze performance to the standardized testscores was essentially the same for both sexes, the analysisindicated that it was taking place at a higher level for thegirls. The reliability coefficients for the cloze tests rangedfrom .90 to .97.
Differences in mean cloze test performance for the basaland revised passages were significant at the .01 level forboth sexes and the total groups in grades one and two.
At the third grade level the difference between the meanswas not significant for the boys on either set of scores, exactor substitute. The difference was significant at the .05 levelfor the exact scores of the girls and the total exact scores.
It was concluded that cloze tests were valid and reliablemeasures of readability for the primary grades. Increase insentence length appeared to increase the difficulty level ofmaterials for first and second grade pupils, but this increasedid not hold true for all subgroups of grade three.'
1Dissertation Abstracts (Ann Arbor, Michigan: UniversityMicrofilms, 19657, Vol. 25, No. 11, pp. 6431-6432. (Order No.65-2370, 152 pages).
51
HOUSKA, JOSEPH THOMAS. "THE EFFICACY OF THE CLOZE PROCEDUREAS A READABILITY TOOL ON TECHNICAL CONTENT MATERIAL ASUSED IN INDUSTRIAL EDUCATION AT THE HIGH SCHOOL LEVEL."UNPUBLISHED ED.D. DISSERTATION, UNIVERSITY OF ILLINOISAT URBANA-CHAMPAIGN, 1971.
Purpose: To ascertain the efficacy of the Cloze Procedure asa readability tool on technical content material.
Principles, Concepts, or Procedures: Nine technical contentpassages (three from each field) were drawn from the automo-tive, woods, and electronics industrial education fields. Eachpassage was approximately 350 words in length. An assessmentof readability of each passage was made by use of the FleschReading Ease Formula. Then, by manipulation of the two factorsof the Flesch formula, the three passages of each technicalfield were re-written to 7th, 10th, and 13th grade levels ofreadability. A cloze test and a 22-item comprehension test wasdeveloped over each passage. The cloze-test was of the every5th word deletion type while the 22-item comprehension test con-sisted of four measures of specific comprehension skill, i.e.,meaning of technical and non-technical vocabularies, recogni-tion of factual statements, and relationships.
The subjects (N=99) were obtained from the larger indus-trial education population (N=223) on the basis of their com-pleting all nine cloze and comprehension tests and havingavailable standardized measures of IQ and reading achievement.A t test between means of the sample population and the largerpopulation, on IQ and reading achievement test scores, indi-cated the study population to be representative of the largerpopulation. Furthermore, on the basis of the standardizedreading achievement scores, the sample population was dividedinto three groups of high, medium, and low in reading ability.
The administration of the tests was conducted in theregular classroom meeting. The cloze test was presented first,before the subjects had read the passage. Eleven days later,the subjects read each passage and responded to the 22-itemcomprehension test over the passage. Analysis of the hypoth-eses of this study was based on the results of these two tests.
oclusions: Split-half test reliabilities were computed foreach of the nine comprehension and cloze tests, and when cor-rected for attenuation by the Spearman-Brown Prophecy formula,
52
yielded reliabilities from .58 to .88 for the comprehensiontests and .73 to .92 for the cloze tests. Pearson r correla-tion coefficients were computed between cloze and comprehen-sion tests over the same technical content passage (r=.383 to.609), corrected for attenuation); between cloze tests andmeasures of specific comprehension skills (r=.115 to .497);and between cloze tests and IQ (r=.29 to .52) and cloze testsand reading achievement (r=.27 to .57). All were significantat the .05 level.
Spearman rho rank order correlation coefficients werecomputed between comprehension and cloze tests rank orderingof the nine passages and across levels of students' readingability (rhos = .86 to .98)1 between cloze test and FleschReading Ease Formula rank ordering of the three passages ofeach technical field, (rhos = .50 to 1.00) and between thecomprehension and cloze tests rank ordering of the three dif-ferent technical fields used in this study (rho = 1.00). Asinple one-way analysis test, followed by a Newman-Keuls test,was computed across levels of reading ability for each clozetest to verify if significant differences existed in the rankoracring process by the different ability groups. All F ratioswere significant at the .05 level.
The cloze procedure, when used on technical content ma-terials, was demonstrated to:
1. identify the relative readabilities of the passagessimilar to the comprehension test techniques
2. identify the relative readabilities of the passagessimilar to the Flesch Reading Ease Formula;
3. discriminate between readers of varying readingabilities:
4. identify which technical content fields were rela-tively_more readable;
5. relate, with low correlations, with standardizedmeasures of students' abilities.1
1Dissertation Abstracts International As The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-IIINT471W2771731.732, No. 8, pp. 4500-A, 4501-A. (Order No.72-6957, 199 pages).
53
JEFFERSON, GEORGE LEE, JR. "LEXICAL AND STRUCTURAL ITEMS ASPREDICTORS OF READABILITY FOR HIGH AND LOW ABILITYREADERS." UNPUBLISHED PH.D. DISSERTATION, UNIVERSITYOF GEORGIA, 1969.1
Purpose: This study is an attempt to clarify the use of clozetechniques in the measurement of reading comprehension and ofwritten language difficulty.
Principles, Concepts, or Procedures: The three variablestested were Reading Ability, Readability, and Deletion Type.The deletion categories were nth (every-fifth word), lexical(nouns, main verbs, adjectives), and structural (the tradition-al parts of speech not categorized as lexical). These threekinds of deletions were made over each of the three passageswhich had been constructed from easy, middle difficulty, andhard reading material. The passages used were those whichBormuth (1966) ranked for difficulty using averaged nth unitforms over each passage.
Four hundred and sixty-six subjects, eighth throughtwelfth grades were stratified into three reading abilitylevels, and randomly assigned to one of nine treatment com-binations within each reading ability level.
Conclusions: The data were analyzed by a three-way analysisof variance, based on assumptions of fixed effects model.There were significant main effects of reading ability and ofdeletion types. There was one significant interaction betweendeletion types and reading difficulty. This was not predicted.
The findings of this study strongly suggest that thecloze is potentially a strong research tool in the study oflanguage and reading, but at present it is not a highly validmeasure of reading comprehension or of passage difficulty.The significant differences among the three deletion typesshow that the language categories in this study represent atleast two major distinct linguistic variables, and the lexi-cal and structural units operate in basically different ways.
1See also George L. Jefferson, Jr., "Lexical and Struc-
tural Items as Predictors of Readability for High and LowAbility Readers," Appendix.
The significant interaction between deletion type andreadability strongly urges that the use of nth word clozetests, averaged across a passage, does not reflect the essen-tial differences between content (semantic meaning) andstructural (relationship meanings) units in the language.And further, no meaningful estimate of comprehension ur pas-sage difficulty may be made from averaged nth unit deletionscores. The use of other linguistic variables rank orderthe passages in contradictory ways from the nth-unit averagedscore ranking.
The results of this study, using random assignment ofsubjects and non-repeated measures, suggest a need for carefulexperimental manipulation of the language variables which cor-relational techniques have pointed out. The kind of designand the type of statistical treatment employed need carefulscrutiny if language experiments with the gloze are to makeuseful contributions to language research.'
ID ssertati n Abstkacts Inte national A: The Humanitiesand Social Sciences Ann Arbors Michigan: University Micro-films, 1970), Vol. 30, No. 12, p. 5287-A, (Order No. 70-10,200,152 pages).
55
KULM, GERALD. "MEASURING THE READABILITY OF ELEMENTARY ALGEBRAUSING THE CLOZE TECHNIQUE." PAPER PRESENTED AT THE ANNUALMEETING OF THE AMERICAN EDUCATIONAL RESEARCH ASSOCIATION,NEW YORK, N. Y., FEBRUARY 4-7, 1971.
Purpose: The relationship to readability of ten variablescharacterizing structural properties of mathematical prose wasinvestigated in elementary algebra textbooks.
Principles. Concepts, or Procedures: Readability was measuredby algebra students' responses to two forms of cloze tests.Linear and currilinear correlations were calculated betweeneach structural variable and the cloze test.
Conclusions: The results indicated that explanatory material,sentence length, percentage of mathematical symbols and per-centage of difficult words had significant linear ccrrelationswith readability. In illustrative material, percentage ofmathematical symbols and percentage of mathematics vocabularyhad significant linear correlations with readability. Depar-tures from linearity were significant for sentence length andpercentage of reader-directed sentences in explanatory material,and percentage of difficult words in illustrative material.got available in hardcopy due to marginal legibility of origi-nal documentg`
1Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1971), Vol. 6, No. 6, pp. 112-113.(ED 047 972, 24 pages).
56
LISKE, WILFRED WALLACE. "THE CLOZE PROCEDURE FOR DETERMININGREADABILITY OF ENCYCLOPEDIA MATERIAL FOR ELEMENTARY SCHOOLPUPILS." UNPUBLISHED ED.D. DISSERTATION, UNIVERSITY OFMARYLAND, 1968.
Purpose: The purpose of the study was (1) to employ the clozeprocedure of test construction and evaluation in assessing thereadability of selected juvenile encyclopedia material forchildren in grades four, five, and six, and (2) to compare theresults of cloze evaluation with readability formula grade-placement levels of the same material.
Principles, Concepts, or Procedures: Fifteen encyclopedia ar-ticles corresponding to social studies and science topicsstudied in grades four, five, and six were selected at randomfrom the 1967 edition of the World Book. Fifty-item cloze testswere constructed on the beginning, middle, and ending sectionsof each article.
The forty-five tests were assigned randomly to 1,199 pupilsdrawn at random from the total fourth, fifth, and sixth-gradepopulation in two public school districts in South West Mohi-can. The tests were administered by the classroom teachers inMarch, 1968. Mean cloze scores were calculated for each gradeand each article.
Grade-placement levels were estimated for each selectedencyclopedia passage by the application of the Yoakam andDale-Chall readability formulas.
Conclusions: To fulfill the purpose of the study, answers weresought for two major questions and two sub-questions.
1. Does readability as measured by cloze procedure indi-ate that selected materials from the World Book Encyclopedia,are readable for children in grades four, five, and six?
A readability criterion of 75/75 was set by this study,i.e., seventy-five percent of the pupils in a grade were re-quired to reach a minimum cloze score corresponding to seventy-five percent correct responses on a multiple-choice comprehen-sion test. According to the criterion, the selected referencematerials were not readable for the population te..sd. Thepupils that were successful in reaching the minimum (twenty-
57
two percent of the total population) had a mean IQ (Lorge-Thorndike) of 114 and a mean reading achievement level (Stan-ford) of 1.4 grades above the mean reading achievement levelof the total population.
Sub-Questions one (a) and (b) asked whether there weresi ificant differences amon and within the selected enc clo-pedia articles. Earlier research had indicated a wide rangeof grade-placement levels among this type of material. Pub-lishers of World Book claim that they adapt longer articles toa wide range of readers by lowering the readability difficultyof the beginning sections. No significant differences (two-way analysis of variance test, p < .01) were found among themean cloze test scores of the fifteen articles. Significantdifferences (p < .01) were found within articles. Two-wayanalysis of variance test on the mean reading achievementlevels showed no significant differences in reading achieve-ment among the groups completing the cloze tests on the vari-ous articles (p < .01).
2. Does readability of selected World Book Encyclopediamaterial as indicated by cloze test results correspond withthe grade-placement level of the same material as indicated by,readability formulas? In answer to this question, PearsonProduct-Moment correlation coefficients were calculated be-tween cloze test results and formula grade-placement levelsaccording to Dale-Chall and Yoakam readability formulas. Anegative correlation was expected since higher grade-place-ment levels should result in lower cloze scores. Significantcorrelations (p < .01) of from -.58 to -.70 were found formiddle and ending sections of articles, but not for beginningsections. The following conclusions resulted:
1. Only the more able readers were able to cope with theselected World Book Encyclopedia (1987 edition) reference ma-terials.
2. The readability difficulty among the selected articlesdoes not vary to a significant extent.
3. The claim of the publishers of World Book that thereadability difficulty increases within the total article wassubstantiated.
4. Cloze procedure used in evaluating encyclopedia mate-rial substantiated the claims of readability research thatcloze tests are reliable and valid for measuring readabilityof prose passages.
1
58
5. There were some significant correlations betweencloze test results and readability formula grade placementlevels. The correlations were not high possibly due to thelimitations of the readability formulas and the differentapproaches used. Cloze procedure seems to follow more con-sistently the actual difficulty of the reading material.1
1Dissertation Abstracts A: The Humanities and SocialSciences (Ann Arbor, Michigan: University Microfilms, 1969),Vol. 29, No. 12, Part 1, pp. 4189-A, 4190-A. (Order No.69-9590, 160 pages).
59
MACGINITIE, WALTER HAROLD. "CONTEXTUAL CONSTRAINT IN ENGLISHPROSE." UNPUBLISHED PH.D. DISSERTATION, COLUMBIA UNI-VERSITY, 1960.1
Purpose: Restorations of words and groups of words omittedperiodically from English prose were studied as indicators ofthe way in which the choice of words is determined by context.
Principles, Concepts, or Procedures: Since this form of thecompletion method has received extensive recent use with littlerecognition of many similar past applications, some of the moreimportant developments in the history of the completion methodwere outlined. Previous developments in the study of languageredundancy were also described.
The patterns of omission which were used permitted com-parison of the predictability of given ...fords when differentamounts of context were removed as periwito omissions in para-graphs. The usefulness of various criteria of correct predic-tion was studied in terms of frequency and reliability, and,as a result of this evaluation, two different criteria werechosen for use in the analysis of the data. Restorations werecounted correct in one case only if they were exact restitu-tions of the original words, and, in the other case, all ra-tionally and grammatically acceptable restorations were counted.
Words were restored in two descriptive paragraphs differ-ing in style and difficulty. Results, based on different sub-jects, were substantially the same for both paragraphs. Eachdifferent set of omitted words was restored by 20 differentcollege students.
Conclusions: A comparison of the restorations in the differentomission patterns led to the following results: (1) There wasno reliable difference in difficulty of restoration when every24th, 12th, or 6th word was omitted, but omitting every 3rdword did make restoration more difficult. (2) When words wereomitted in pairs, there was no reliable difference in diffi-culty of restoration, regardless of whether 22, 10, or only 4words of context were left between each omitted pair. (3)
1See also Walter H. MacGinitie, "Contextual Constraint inEnglish Prose Paragraphs," Appendix.
60
Words omitted in groups of four were more difficult to restorethan words omitted in pairs, and pairs were in turn more dif-ficult to restore than single words. (4) Omitting an adjacentword made restoration more difficult, but it made no differencewhether the additional omitted word preceded or followed theword to be restored. When each omitted word was scored sepa-rately, the above results were obtained regardless of whetherall acceptable restorations were counted correct or whether re-turn of the original word was required. When each group ofadjacent omitted words was scored as a unit, the above resultswere also obtained for both scoring criteria, except that in-creasing size of omitted word-group had little effect on thenumber of exact plus acceptable restorations.
These findings were interpreted as indicating that con-textual constraints upon words in English prose declinerapidly with increasing distance of the context and that con-text beyond about five words has little effect on word choice.Some implications of this conclusion for the relation of con-text to understanding were discussed.
A consideration of the relation of context to the abilityof subjects to supply a word which is both rationally and gram-matically acceptable in the rest of the passage suggested thatneither the form nor the content of written English may bemuch constrained beyond about five words and that units inwhich thoughts are composed may seldom be greater than thislength.l
1DissertatipMicrofilms, 1960)No. Mic 60-2020,
n Abstracts (Ann Arbor, Michigan: University, Vol. 21, No. 1, pp. 126-127. (L. C. Card171 pages).
61
MILLER, G. R., AND COLEMAN, E. B. "A SET OF THIRTY-SIX PROSEPASSAGES CALIBRATED FOR COMPLEXITY." JOURNAL OF VERBALLEARNING AND VERBAL BEHAVIOR, 6 (1967), 851-854.
Purpose: A set of prose passages was calibrated for complexityby means of the cloze procedure.
Principles, Concepts, or Procedures: Thirty -six passagesdrawn mainly from Standard Test Lessons in Reading were alteredso that each contained exactly 150 words and ranged in diffi-culty from first-grade to technical prose. For each passagethree types of cloze tests were constructed.
The first utilized every-fifth-word deletion. Five formswere constructed, each beginning with a different deletion sothat each word became a cloze item eventually. Six bookletsof six tests each were compiled, alternating easy and difficultpassages.
The second type of cloze test allowed the subject to useall the constraint in a passage to determine the deletion, for150 forms were made with only one deletion in each.
The third type demanded that the subject predict each ofthe 150 words beginning with the first. Each correct word wasrevealed after a response had been made.
Subjects were 479 college students. The first cloze testhad time limits. Correct exact replacements were accepted aswell as synonyms and form classes. Subjects given test twowere allowed unlimited time but had to replace the word exactly.In test three a new word was exposed every seven seconds. Onlyexact replacements were scored correct.
Conclusions:
1. The correlation of .99 between the weighted scores andthe exact replacement scores in test one indicates that thesimpler method of accepting only exact replacements is a validand reliable one.
2. Test three yielded the lowest percentage of correctresponses and test two, the highest.
62
3. All three tests ranked the passages for difficulty inapproximately the same manner.
4. The lack of increase in correct responses from word20 to word 140 in test three indicates that the responses arecon3trained very slightly, if at all, by words from othersentences.
63
RENTEL, VICTOR MICHAEL. "FORM CLASS AND WORD LENGTH AS IN-DICES OF THE DIFFICULTY OF PREDICTING CLOZE ENTRIES IN ASAMPLE OF CONTINUOUS PROSE." UNPUBLISHED PH.D. DISSER-TATION, UNIVERSITY OF SOUTH CAROLINA, 1969.
Purpose: The study examined the difficulty of predicting fourcategories of words at three levels of word length on a clozetest of reading comprehensions.
Principles, Concepts, or Procedures: Grammatical classificationwas based on that of C. C. Fries with Form Class I roughly cor-responding to nouns, pronouns, and nominalizations; Form Class2 to verbs; Form Clads 3 to adjectives; and Function Words tothose words not capitalized in titles. Category limits forword length were defined by letters per word (2-4, 5-7, 8-16).One hundred and fifty subjects were selected on a conveniencebasis from among juniors and seniors majoring in education en-rolled at both the University of South Carolina and the Univer-sity of Tennessee. A selection of 5,218 words of continuousprose from an essay entitled "How to Be an Employee" compriseda controlled (Flesch Reading Ease Formula - .69) languagesample.
Scores representing the difficulty of each grammaticalcategory, each word length category, and the combination ofboth were compared using a multivariate analysis of variancefor a repeated measures, multifactor experiment. Data analy-sis was performed by means of a computer program developedspecifically for this study for use on the IBM 7040 computer.The program made overall tests on within-subjects effectsthrough the use of Hotelling's T2 statistic, a multivariateanalogue of the square of t. All d4fferences were significantbeyond the .005 level.
Conclusions: Analysis indicated that as words increased inlength, the difficulty of predicting their occurrence in apassage increased significantly across nJ1 language categoriesbut not uniformly so. Form Class 3 appears least affected byincreasing word length and more significantly difficult topredict than were other categories. But for words over sevenletters long, Form Class 2 was slightly more difficult to pre-dict than Form Class 3, and both were significantly more dif-ficult to predict than was Form Class 1. Significant inter-actions occurred between word length and Form Class categories.
64
Typically, these interactions were wholly the result of varia-tions in the differences between categories. Significant in-teraction effects were mainly accounted for by the difficulty4 slsed on prediction by Form Class 3. For all categories ofwords over seven letters in length, length appeared to takeprecedence as a source of difficulty and was affected onlyslightly by variation in Form Class.
Warnings against generalization from these results bothto other levels of passage difficulty and to other subjectswere stressed.
Appendices include raw cloze test scores, lists of wordsdeleted in each category, a sample cloze test, summary sourcetables, a description of the nonprofile data analysis, illus-trative scatter plots, and a table of test reliabilities foreach of the eleven categories.'
1Dissertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 197017W1730, No. 10, p. 4284-A. (Order No. 69-20,113, 138 pages).
65
TAYLOR, WILSON L. "CLOZE PROCEDURE': A NEW TOOL FOR MEASUR-ING READABILITY." JOURNALISM QUARTERLY, 30 (1953), 414-438.
Purposes:
In Experiment One, the author sought to determine if clozescores would "rank" passages taken from the writings of Boswell,Huxley, and James as did the Flesch and Dale-Chall formulae.
In Experiment Two, the author hypothesized that
:1.. the results of Experiment One would be validated.2. the readability order of eight passages taken fromthe writings of Caldwell, Stein, and Joyce would bepredicted by the rank order of the cloze scores giventhem by a panel of subjects.3. the cloze procedure would handle materials eideror both the Flesch and Dale-Chall formulae coule not.
Principles, Concepts, or Procedures:
Experiment Ones Three passages were adminis+cred to 24college juniors and seniors. All were approximately 175 wordsin length and were mutilated by three systems. Scores werecomputed on the basis of exact-word and synonym replacement.Pilot studies were run in advance.
Experiment Two: Eight passages were administered to 72subjects each of whom completed two--an easy one and a harderone. Other procedural aspects were similar to Experiment One.
Conclusions:
Experiment Ones1. Cloze scores consistently ranked the passages asdid the formulae. Qualitatively the three mutila-tion systems yielded the same results. Quantitative-ly, however, there were some differences. Exact-word and synonym scoring yielded virtually identicalresults.
66
Experiment Two:1. The results of Experiment One were validated.2. The ranking given by the panel almost perfectlypredicted those given subjects in the experiment.3. Cloze was superior to either of the formulaein predicting readability.
Though the traditional formulae are reasonably accurateand reliable and do possess a number of advantages over thecloze procedure, certain language elements are not handled aswell by the former as the latter.
67
TAYLOR, WILSON LEWIS. "APPLICATION OF 'CLOZE' AND ENTROPYMEASURES TO THE STUDY OF CONTEXTUAL CONSTRAINT IN SAMPLESOF CONTINUOUS PROSE.". UNPUBLISHED PH.D. DISSERTATION,UNIVERSITY OF ILLINOIS, 1954.
Purpose: Relationships between the author's "cloze procedure"and the entropy or "uncertainty" measure of modern informationtheory were explored in an experiment based on the attempts ofsubjects to replace words that had been mechanically deletedfrom meaningful prose material. 7f...,rtain aspects of the appli-cation, nature, and Methodology of the doze method also wereinvestigated.
Principles, Concepts, or Procedures: Two sample passages,each containing 175 experimental words in sequence, were takenfrom each of two essays, X and Y. Previous pilot studies hadindicated that X was significantly "easier" to comprehend thanY, as gauged by the larger proportion of missing X words cor-rectly replaced.
Each passage was mutilated by counting out every fifthword, or 35 words altogether. The operation was repeated fourtimes, beginning with different words, to yield five deletionversions that together took out all 175 words. Each versionof each passage was reproduced on a separate page with a stan-dard-size blank in place of each missing word. A test assemblyincluded one version of all four passages and contained 140(4 x 35) blanks. The five deletion versions and the alterna-tion of the order in which essays were represented produced tenkinds of assemblies, which were randomly distributed amongabout 300 freshman rhetoric students.
Assemblies returned by 287 subjects were considered suffi-ciently filled in for the first stage of analysis; those thatexhibited a whole passage with all blanks left unfilled werediscarded.
A subject's guess on any particular blank was scored asright only if it "closed the gap" in the word sequence by pro-posing the same word as the original one deleted. Any sub-ject's doze score was simply the total of his correct guesses.The sum of its subjects' scores was the doze score of any setof material.
Initial analysis showed: (1) No.significant difference
68
between X-then-Y and Y-then-X orders of presentation; (2) sig-nificantly higher scores for subjects exposed to either essaynearly two months before than for those not previously exposedto it: (3) significantly higher scores for X than Y, as pilotstudies had predicted; (4) a significant overall differenceamong deletion versions for each essay.
The subject total was reduced to 250 - 50 in each dele-tion-version group - by discarding assemblies with the mostblanks unfilled. Then, it addition to scoring subjects andmaterials, cloze scores were allocated to individual blanks:that is, each of the 175 words was assigned the proportion ofsubjects (out of 50) that correctly guessed its identity fromcontext.
Conclusions: Subsequent analysis indicated: (1) a tendencyfor the accumulated percentage scores for each passage's fivedifferent deletion versions to converge as more successivewords were taken into account: (2) a tendency for the mean cumu-lative percentage score of all deletion versions to stabilizewithin the first 20 to 40 words of each passage; (3) approxi-mately normal distribution of the cloze scores of subjects foreach essay; (4) abnormal distribution of the cloze scoresallocated to blanks (missing words), with more scores fallingat the extremes than in the middle of the distribution for anypassage.
Exploration of the relationships between cloze and en-tropy measures was confined to the 250 subjects' performanceon one passage from X essay. Each blank and its context ofsurrounding words constituted an individual "system," the en-tropy of which was computed from the relative frequencies withwhich each alternative was proposed by 50 subjects trying toguess the missing word. Computation used the basic formulafor "average entropy":
H = - pilog2pii
The correlation scattergram between cloze and uncertaintymeasures for each of the 175 words was abnormal. The rank-difference correlation between the distributions reached ahighly significant value of -.87.
When, for each word in series, the cloze percentage andits paired relative uncertainty estimate were plotted in oppo-
69
sition on a 100-point scale, about half the pairs exhibited aroughly complementary relationship. Exploration revealedthat any cloze percentage is precisely complementary to the"maximum remaining relative uncertainty" of the system con-cerned. Such "maximum remaining" values were computed by re-moving the proportion of cloze responses from any set ofresponse alternatives, calculating the maximum absolute un-certainty of the possible alternatives remaining (as if all"wrong" alternatives), and then dividing by the maximum un-certainty for the whole system that included the cloze pro-portion.
Many of the 175 word-response systems yielded actualremaining uncertainty values much smaller than their corre-sponding maximums. The difference between the observed andmaximum Nalues was assumed to be that part of any system'stotal response performance not explained by organizationtoward production of the "correct" response, or by mere dis-organization (uncertainty), or even by both together. Thedifference appeared to involve an active influence that mis-directed, that tended to make subjects agree on a "wrong"response, and this difference was postulated to be an "indexof misdirection." The following formula was proposed to ex-press total performance with regard to any blank's system:
Cloze % + Rem.Rel.H + Misdirection = 1.00
All three sorts of measures were computed for each wordin series, and analysis showed all three distributions to benormal. The rank correlation coefficient of the cloze andremaining relative uncertainty distributions was -.83.
Various ways in which cloze procedure could be appliedto communication problems, at various places in the processthat links encoder and decoder, were illustrated and dis-cussed.1
'Dissertation Abstracts (Ann Arbor, Michigan: Univer-sity Microfilms, 1955), 15, No. 3, pp. 464-465. (Pub-lication No. 10,554, 127 pages).
f.
CHAPTER III
THE CLOZE PROCEDURE AS A MEnURE
OF COMPREHENSION
The relationship of numerous aspects of comprehen-
sion to the cloze procedure has been variously examined.1
Bormuth assessed the factor validity of cloze
tests as measures of reading comprehension, while Cranney
compared two types of cloze reading tests for validity and
reliability.
Numerous studies have examined the cloze procedure
as a technique for measuring reading comprehension.
Jenkinson, as reported by Rankin, 3obtained correlations
with cloze scores of .78 and .73 with the vocabulary and
comprehension sections of the Cooperative Reading Test.4
Using the same test, Fletcher obtained correlations of
Bormuth, "Factor Validity of Cloze Tests as Mea-sures of Reading Comprehension Ability," Reading ResearchQuarterly, 4 (1969), 358-365.
2Adelbert Garr Cranney, Jr., "A Comparison of aFree-Response and Multiple-Choice Cloze Reading Test" (un-published Ph.D. dissertation, University of Minnesota,1968). .
3Rankin, "The Cloze Procedure - Its Validity and
Utility," p. 138.
4Junior Eugene Fletcher, "A Study of the Relation-
ships Between Ability to Use Context as an Aid in Readingand Other Verbal Abilities" (unpublished Ph.D. disserta-tion, University of Washington, 1959).
71
.63 and .55 respectively.
1Using the Diagnostic Reading Test, Rankin found
correlations of .68 and .60 between vocabulary and compre-2
heneon, and cloze scores. Ruddell obtained correlations
ranging from .61 to .74 between cloze scores and the Stan-
ford Achievement Test, Paragraph Meaning sub-test.
In her study, Gallant3found correlations ranging
from .65 to .81 with the Paragraph Reading section of the
Metropolitan Achievement Test, Primary Batteries I and II
and Elementary Reading Test.
4Using every-tenth-word deletions, Schneyer found
correlations of .74 with vocabulary and .68 with comprehen-
sion when the Gates Reading Survey was used. When noun-
verb deletions were made, the correlations were .63 and
.60 respectively.
1Rankin, "An Evaluation of the Cloze Procedure as a
Technique for Measuring Reading Comprehension" (unpublishedPh.D. dissertation, University of Michigan, 1958).
2Robert Byron Ruddell, "An Investigation of the
Effect of the Similarity of Oral and Written Patterns ofLanguage Structure on Reading Comprehension" (unpublishedEd.D. dissertation, Indiana University, 1963).
3Gallant, "An Investigation of the Use of ClozeTests as a Measure of Readability of Materials for the Pri-mary Grades."
4J. Wesley Schneyer, "Use of the Cloze Procedure
for Improving Reading Comprehension," The Reading Teacher,19 (1965), 174-179.
72
As a measure of specific comprehension, cloze
scores have been correlated with scores from comprehension
tests based on the same material. Taylor1found correla-
tions ranging from .51 to .92, while Jenkinson, as reported
by Rankin,2
found a single correlation of .82. Bormuth3
found a single correlation of .93.
The cloze procedure as a technique for measuring
reading comprehension in specific situations has been the
subject of much investigation.
The effects on comprehension tests of various lexi-
cal deletions were studied by Bradley.
Swalm5 investigated the effects on reading compre-
1Taylor, "Recent Developments in the Use of 'Cloze
Procedure"; Wilson L. Taylor, "Cloze Readability Scoresas Indices of Individual Differences in Comprehension andAptitude," Journal of Applied Psychology, 41 (1957), 19-26.
2Rankin, "The Cloze Procedure - Its Validity and
Utility," p. 138.
3Bormuth, "Cloze Tests as Measures of Readabilityand Comprehension Ability (unpublished Ed.D. dissertation,Indiana University, 1962).
Michael Bradley, "Effects on Reading Tests ofDeletions of Selected Grammatical Categories" (paper pre-sented at the National Reading Conference, Atlanta,Georgia, December 4-6, 1969.
5James E. Swaim, "Comparison of Oral Reading, Si-lent Reading, and Listening Comprehension Assessed byCloze" (unpublished Ed.D. dissertation, Rutgers University,The State University of New Je'sey, 1971).
?3
hension, as measured by the cloze procedure, of the oral
reading, silent reading, and listening of second, third, and1
fourth grade students, while Daugherty measured the reading
comprehension of fourth grade students.2
Sikes utilized the cloze procedure to assess fifth
and sixth grade children's comprehension of original and
published materials.
The cloze procedure as a measure of the reading com-
prehension of selected Mexican-American and Anglo-American
elementary school children was analyzed by Crawford.3
4Geyer utilized the cloze procedure as a predictor
of the ability of junior high school students to comprehend
social studies materials.
The reading performance of selected students ua
1Joan Daugherty, "The Effect of the Cloze Procedure
and Interspersed Questions as an Aid to Reading Comprehen-sion" (unpublished Ph.D. dissertation, the University ofAkron, 1971).
2Helen Cruse Sikes, "A Comparative Study of Cloze
Procedure Scores on Original and Published Materials" (un-published Ed.D. dissertation, University of SouthernMississippi, 1971).
3Alan Neal Crawford, "The Cloze Procedure as a Mea-
sure of the Reading Comprehension of Elementary Level Mexi-can-American and Anglo-American Children" (unpublishedEd.D. dissertation, University of California, Los Angeles,1970).
4James Rush Geyer, "The Cloze Procedure as a Pre-
dictor of Comprehension in Secondary Social Studies Mate-rial" (unpublished Ed.D. dissertation, University of Mary-land, 1970).
occupation information materials written at various reada-1
bility levels was determined by Knight.2
Hater assessed the comprehensibility of mathemati-
cal English in grades seven through ten.
Greene3and Rankin evaluated the cloze procedure
as a technique for measuring the reading comprehension of
college students.
Fletcher5utilized the cloze technique to measure
the ability of college freshmen to use context, while6
Feldman assessed the learning accrued from programmed and
1David W. Knight, "The Reading Performance of Stu-dents With Ninth Grade Reading Achievement on OccupationalInformation Materials Written With Various Levels of Reada-bility" (unpublished Ph.D. dissertation, The Florida StateUniversity, 1966).
2Sister Mary Ann Hater, "The Cloze Procedure as aMeasure of the Reading Comprehensibility and Difficulty ofMathematical English" (unpublished Ph.D. dissertation,Purdue University, 1969).
3Frank Pierrepont Greene, "A Modified Cloze Proce-
dure for Assessing Adult Reading Comprehension" (unpub-lished Ph.D. dissertation, The University of Michigan,1964).
4Rankin, "An Evaluation of the Cloze Procedure as
a Technique for Measuring Reading Comprehension."
5Fletcher, "A Study of the Relationships BetweenAbility to Use Context as an Aid in Reading and Other Ver-bal Abilities" (unpublished Ph.D. dissertation, Universityof Washington, 2959).
6Margaret Eichbauer Feldman, "The Effects of Learn-
ing by Programed and Text Format at Three Levels of Diffi-culty" (unpublished Ph.D. dissertation, Cornel University,1964),
75
text presentations of an introductory college psychology
program.1
Guthrie studied the effect of delayed, immediate,
and omitted cloze testing on the comprehension of prose sen-2
tences by college students, and Sawyer and Braam investi-
gated the effect of instruction in speed of comprehension
upon reading comprehension developed from c.7.oze exercises.
The relationships between cloze scores and selected
variables, such as intelligence, have been studied by Hefner,
Gwaltney, and Robinson; Kohler; Lisman; Taylor; and Weaver3
et al.
1John T. Guthrie, Motivational Effects of Feedbackin Reading (Baltimore, Maryland: Center forthi-Study ofSocial Organization of Schools, Johns Hopkins University,1971).
2Rita Sawyer and Leonard S. Braam, "An Experiment
Combining Cloze Procedure With Speed of Comprehension In-struction," Proceedings of the College Reading Association,ed. by Clay A. Ketcham (Syracuse, New York: College Read-ing Association, 1964, pp. 27-33.
3Lawrence E. Hefner, Wayne Gwaltney, and RichardRobinson, "Reading in Bookkeeping: Predictions and Per-formance," Journal of Reading, 14 (1971), 537-546; EmmettTheodore Kohler, "An Investigation of Cloze Scores in Termsof Selected Cognitive Variables" (unpublished Ed.D. disser-tation, The Florida State University, 1966); Linda C. Lisman,"Vowel D'Aetion and Cloze Tests Compared With a ReadingAbility Test" (unpublished Master's thesis, Rutgers, TheState University of New Jersey, 1971); Wilson L. Taylor,"Cloze Readability Scores as Indices of Individual Differ-ences in Comprehension and Aptitude," Journal of AppliedPsychology, 41 (1957), 19-26; and Wendell Weaver, Albert J.Kingston, A. C. Bickley, and William F. White "Information-Flow Difficulty in Relation to Reading Comprehension,"Journal of Reading Behavior, 1 (1969), 41-49.
76
Comparable cloze and multiple-choice scores have
been investigated by Bormuth; Mosberg, Potter, and Cornell;1
and Rankin and Culhane.
The concept of information gain has been studied by2
means of the cloze procedure. Carver examined the exact
scoring of cloze replacements as measures of information
gained from reading.
Coleman and Miller3and Rankin and Dale
4studied
information gain by using the cloze procedure with college
students.
Information gain when individualized and tradition-
1Bormuth. "Comparable Cloze and Multiple-ChoiceComprehension Test Scores," Journal of Readings 10 (1967),291-299; Ludwig.Mosberg, Thomas C. Potter, and Robert K.Cornell, The Relation Between Cloze and Multiple-ChoiceTest Scores as a Function of Relative Paragraph Difficultyand Grade Level (Inglewood, California: Southwest RegionalLaboratory for Educational Research and Development, 1968);and Earl F. Rankin and Joseph W. Culhane, "ComparableCloze and Multiple-Choice Comprehension Test Scores," Jour-nal of Reading, 13 (1969), 193-198.
2Ronald P. Carver, "What Is Reading Comprehensionand How Should It Be Measured?" (paper presented at theNational Reading Conference, Atlanta, Georgia, December 4-6,1969.
3E. B. Coleman and G. R. Miller, "A Measure of In-formation Gained During Prose Learning," Reading_ResearchQuarterly, 3 (1968), 369-386.
'Earl F. Rankin and Lothar H. Dale, "Cloze ResidualGain - A Technique for Measuring Learning Through Reading,"The Psychology of Reading Behavior, ed. by G. B. Schickand M. M. May, Eighteenth Yearbook of the National ReadingConference (Milwaukee, Wisconsin: National Reading Crnfer-ence, 1968), pp. 17-26.
77
al textbook approaches to science instruction were used1
with fifth-grade students was investigated by Charles.
The influences of several properties of the English2
language on comprehension have been examined. Bormuth in-
vestigated the relationships between mean word depth and
comprehension difficulty, as well as the influence of length
of independent clauses on these two variables.
Coleman3
studied the effects of shortening sen-4
tences on comprehension of college students, while Darnell
assessed the influences of sentence order on the comprehen-
sion of college students,
Smith5
investigated the relationships between the
lEdward Charles, "An InNestigation of the Use ofCloze Tests to Compare Gain Scores of Students in ScienceWho Have Used Individualized Science Materials and ThoseWho Have Used Traditior.al Textbook Materials" (unpublishedEd.D. dissertation, Lehigh University, 1970).
2Bormuth, "Mean Word Depth as a Predictor of Compre-hension Difficulty," California Journal of Educational Re-search, 15 (1964), 226":277 John R. Bormuth, "New Measuresof Grammatical Complexity," The Psycholinguistic Nature ofthe Readink Process, ed. by Kenneth S. Goodman (Detroit:Wayne State University Press, 1968), pp. 239-253.
3E. B. Colerian, "improving Comprehensibility byShortening Sentences," Journal of Applied Psychology, 46(1962), 131-134.
4Donald K. Darnell, "The Relation Between SentenceOrder and Comprehension," Speech Monographs, 30 (1963),97-100.
5William Lewis Smith, "The Effect of Syntax onReading" (unpublished Ph.D. dissertation, The Florida StateUniversity, 1969).
78
maturity of syntactic structures and reading comprehension
at selected grade levels, while the effect of the simi-
larity of oral and written patterns of language structure1
on reading comprehension was studied by Ruddell.
The effect of interest on reading comprehension
was measured, in part, by the cloze procedure in a study2
undertaken by Henry.
Rankin and Hess3assessed internal reading flexi-
bility, and Bloomer and Heitzman examined the effects of
cloze pre-testing on reading comprehension.
1Ruddell, "An Investigation of the Effect of theSimilarity of Oral and Written Patterns of Language Struc-ture on Reading Comprehension."
2Peggy Elaine Henry, "The Effect of Interest onReading Comprehension as Measured by Cloze and MultipleChoice Tests" (unpublished Ph.D. dissertation, The Univer-sity of Iowa, 1969).
'Earl F. Rankin and Allen K. Hess, "The Measure-ment of Internal (Intra-Article) Reading Flexibility"(paper presented at the National Reading Conference, Atlan-ta, Georgia, December 4-6, 1969).
4Richard H. Bloomer and Andrew J. Heitzman, "Pre-
testing and the Efficiency of Paragraph Reading," Journalof Reading, 8 (1965), 219-223.
79
BLOOMER, RICHARD H., AND HEITZMAN, ANDREW J. "PRE-TESTING ANDTHE EFFICIENCY OF PARAGRAPH READING." JOURNAL OF READING,8 (1965), 219-223.
Purpose: The author examined ". . . the relationship betweeninformation presented to the student in short selections andhis comprehension of the material." (219)
Principles, Concepts or Procedures: Materials were five ofthe most difficult passages in the McCall-Crabbs Standard TestLessons in Reading.
Subjects were 146 students enrolled in six eighth gradeclasses.
Treatment Group A was given the pre-test followedby the reading selection, followed in turn by a post-test which was identical to the pre-test. TreatmentGroup B was given the reading selection followed by thepost-test. Treatment Group C was presented with thereading selection adapted for Cloze procedure by theomission of every tenth word, followed by the post-test.Treatment Group D was given the pre-test, followed bythe Cloze procedure reeding materials, followed by thepost-test. (219)
Conclusions: Among the conclusions were the following:
1. The response strength of the wrong response was de-creased when it was made in the pre-test.
2. Pre-testing did not appear to improve compensation- -attending more carefully to the unfamiliar material.
3. ". . the pre-test basically reduces the tension onthe individual, reducing his motivation to learn." (223)
80
BORMUTH, JOHN R. "MEAN WORD DEPTH AS A PREDICTOR OF COMPREHEN-SION DIFFICULTY." CALIFORNIA JOURNAL OF EDUCATIONAL RE-SEARCH, 15 (1964), 226-231.
Purpose: The purpose of the study was
to test a new method of measuring the grammati-cal complexity of sentences to see if it could beused to predict the difficulty children have in com-prehending written verbal materials. (226)
Principles, Concepts, or Procedures: The author tested threehypotheses.
1. Over a set of passages as a whole there is a cor-relation between mean word depth and comprehensiondifficulty.
2. There is a correlation between comprehension dif-ficulty and mean word depth when the subject mattersof the passages are held constant and the Dale-Challreadabilities are systematically varied. Further,this correlation is higher than a similarly obtainedcorrelation between comprehension difficulty and theDale-Chall predicted difficulties.
3. There is a correlation between mean word depth andcomprehension difficulty when the Dale-Chall pre-dicted difficulties of the passages are held c,lstant,and the subject matters of the passages are varied.(227-228)
The nine passages used in this study were equallydivided among three subject matter areas--literature,science, and social studies. One passage in eachsubject matter area had a Dale-Chall readabilitylevel of 4.5, 5.5, and 6.5. (228)
For each passage a 50-item doze test was constructed.The subjects mere 150 children enrolled in Grades , 5, and6 of a single school. The group's mean score on each dozetest was used as the measure of comprehension difficulty ofthe passages which were accordingly assigned ranks. "Thedepth of each word in each passage was determined. .
81
Ranks were assigned to the passages according to the relativesizes of the means of their word depths." (228)
Conclusions: The first hypothesis was found to be significantat the .05 level. The Spearman rho correlation was .77. Thesecond hypothesis yielded a Kendall W correlation of .33 be-cause of the perfect correlation between mean word depth andpredicted difficulty. The third hypothesis yielded a KendallW correlation of .78. Because this study demonstrated thatmean word depth can predict differences in difficulty amongpassages differing on content, ". . . concept difficulty oridea density is, at least in part, no more than a tendency forauthors to use different sentence structures when they writeabout different subject matters." (230)
82
BORMUTH, JOHN R. "COMPARABLE CLOZE AND MULTIPLE-CHOICE COM-PREHENSION TEST SCORES." JOURNAL OF READING, 10, 291-299.
Purpose: The author reports a study designed to build a"frame of reference by which the size of the cloze score canbe interpreted" (291) when used as a measure of the comprehen-sion difficulties of a passage.
Principles, Concepts, or Procedures: Nine passages of approxi-mately 275 words each were used and ranged in Dale-Chall reada-bility from 4.5 to 6.5.
A 50-item cloze test and a 31-item multiple-choicetest were made over each of ghil nine passages. (293)
The tests were administered under un.imed conditionsto 100 pupils enrolled in Grades 4 and 5. All thecloze tests were administered before any of the mul-tiple-choice tests were given and before the subjectswere -,)ermitted to read the passages from which thetests were made. Subjects were instructed to guesson all items where they did not know the correctanswer;. Approximately three days after a cloze testwas administered, the multiple-choice test over thesame passage was given. The subject first read thepassage and then, without being allowed to refer tothe passage, took the test over it. (293-294)
To determine if the multiple- choice ecores were unusuallyinflated because of this procedure, a second study was run.Two new samples, consisting of 32 and 27 subjects from Grades4 and 5, were drawn,
. . each group took one of the composite multiple-choice tests after taking a cloze test over it andanother without a cloze test preceding it. The testconditions and instructions were the same as thoseused in the first part of the study. No effort wasmade to match the groups, since the tests werematched for difficulty, (294)
83
Conclusions:
If the conventional readability standards are ac-cepted, a passage on which a student receives acloze score of 38 percent is sufficiently under-standable to him to be used in his instruction.That is, he can correctly answer about 75 percentof the multiple-choice items that can be writtenover that passage. On the other hand, if one de-mands as a criterion a multiple-choice score cor-rected for guessing, a cloze test score of 43 per-cent is required to reach the same criterion level.(298)
The second study demonstrated that the multiple-choicescores had not been unusually inflated because of the designof the first study.
84
BORMUTH, JOHN R. "NEW MEASURES OF GRAMMATICAL COMPLEXITY."THE PSYCHOLINGUISTIC NATURE OF THE READING PROCESS.EDITED BY KENNETH S. GOODMAN. DETROIT: WAYNE STATEUNIVERSITY PRESS, 1968, 239-253.
Purpose: The purpose was
1. ". to investigate the sizes of the correlationsbetween comprehension difficulty and . . ." (239) frequency,mean word depth, and length of independent clauses.
2. to discover ". . . if relationships between compre-hension difficulty and measures of grammatical complexitychange as a function of the reading ability of the subject."(239)
Principles: Concepts, or Procedures: The procedures reportedhere are the same as reported in "Experimental Applicationsof Cloze Tests. "l
Conclusions:
1. There was a significant correlation between allthree measures of grammatical complexity and comprehensiondifficultly.
2. Measures of mean word depth and length of independentclauses did not appear dependent on reading ability. However,independent clause frequency was slightly significant as read-ing achievements increased.
1See page 8 of this thesis.
85
BORMUTH, JOHN R. "FACTOR VALIDITY OF CLOZE TESTS AS MEASURESOF READING COMPREHENSION ABILITY." READING RESEARCHQUARTERLY, 4 (1969), 358-365
Purpose: The author reports a study designed to examine thefactor validity of cloze tests as measures of comprehension.
Principles, Concepts; or Procedures: Nine passages of 250words were chosen from various subject areas and ranged inDale-Chall readability from 4.5 to 6.5. A fifty-item clozetest which utilized every-fifth-word deletions was constructed.Seven multiple-choice tests designed to measure vocabulary,facts; sequences, relationships, main ideas. inferences, andauthor's purpose were written and validated by judges.
Subjects were 150 fourth, fifth, and sixth grade stu-dents who were administered the cloze tests first. The sub-jects were later permitted to read the passage and.then com-pleted the multiple-choice tests. Only exact replacementswere scored correct.
Conclusions: The correlations among cloze tests were high, aswere most among all tests in the correlation matrix.
It appears, . . ; that cloze tests made by deletingevery fifth word measure skills closely related oridentical to those measured by conventional multiple-choice comprehension tests. Little or no evidencewas found to suggest that the two types of tests mightmeasure different skills. (365)
86
BRADLEY, MICHAEL. "EFFECTS ON READING TESTS OF DELETIONS OFSELECTED GRAMMATICAL CATEGORIES." PAPER PRESENTED AT THENATIONAL READING CONFERENCE, ATLANTA, GEORGIA, DECEMBER4-6, 1969.
Purpose: An investigation was made of the effects of the re-moval of certain grammatical categories from standardizedreading comprehension test paragraphs on the reading compre-hension of 180 third, fourth, and fifth graders enrolled in arural Georgia school system.
Principles, Concepts, or Procedures: Each grade level con-tained a high and low reading ability group, based on stan-dardized reading test scores. The materials used were para-graphs rnd the accompanying multiple-choice items fromvarious standardized reading tests appropriate for thesegrade levels. The paragraphs were altered by deletion of thefollowing grammatical elements: (1) all nouns, (2) all ad-jectives, (3) all verbs, (4) all function words, (5) all words,and (6) no words.
Conclusions: The deletion of nouns had a significant effectat all three grade levels, but the deletion of adjectives hada significant effect only for fifth grade. It was also foundthat the subjects could answer the multiple-choice items with-out the aid of the accompanying reading paragraph at a levelgreater than c:.ance. Therefore, it was concluded that the read-ing comprehension test was measuring other variables such aspast learning and word association in addition to ability totake cues directly from the reading paragraph. Tables and re-ferences are included.'
'Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1970), Vol. 5, No. 8, p. 88.(ED 038 242, 12 pages).
87
CARVER, RONALD P. "WHAT IS READING COMPREHENSION AND HOWSHOULD IT BE MEASURED?" PAPER PRESENTED AT THE NATIONALREADING CONFERENCE, ATLANTA, GEORGIA, DECEMBER 4-6, 1969.
Purpose: Four conceptually different variables in reading:(1) knowledge gained (new information), (2) amount comprehended(degree of understanding), (3) reading aptitude (differencesbetween individuals), and (4) reading improvement (within in-dividual changes as a result of treatment) are discussed.
Principles, Concepts or Procedures: Existing measures ofreading comprehension are evaluated in terms of their potentialfor measuring these variables. It was concluded that most com-mercially available reading comprehension tests and the exactscoring close technique were inadequate measures or iCacantsof knowledge gained, amount comprehended, or reading improve-ment and should be regarded and used as measures of readingaptitude only. The Tinker Speed of Reading Test was recom-manded as a measure of reading improvement and reading aptitude.
Conclusions: The Carver-Darby Chunked Reading Test which wasdesigned and developed to measure the changes that take placees a result of reading was described and recommended as astandardized indicant of knowledge gained, amount comprehended,reading aptitude, and reading impro-1.1ent. Tables and refer-ences are included.1
'Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1970), Vol. 5, No. 8, p. 88.(ED 038 243, 15 pages).
88
CHARLES, EDWARD. "AN INVESTIGATION OF THE USE OF CLOZE TESTSTO COMPARE GAIN SCORES OF STUDENTS IN SCIENCE WHO HAVEUSED INDIVIDUALIZED SCIENCE MATERIALS AND THOSE WHO HAVEUSED TRADITIONAL TEXTBOOK MATERIALS." UNPUBLISHED ED.D.DISSERTATION, LEHIGH UNIVERSITY, 1970.
Purposes: The main purpose of this study was to determine ifcloze tests are sensitive to gain measurement when an individ-ualized approach and a traditional textbook approach to scienceinstruction are used.
A secondary purpose of this study was to determine whatrelationships exist among the scores made by fifth grade chil-dren in both the experimental and control groups on standard-ized tests of reading and science ability and gain scores asmeasured by cloze test procedures.
Principles. Concepts, or Procedures: A sample of three hundredchildren, one hundred fifty girls and one hundred fifty boys,was randomly selected from a population of five hundred andtwenty fifth grade students in Centennial School District, War-minster, Pennsylvania.
The following tests were administered: The SequentialTests of Educational Pro ress. Readin Level nd Science,l'itrel4B.wereusedtoProvidemeasuresofreading and scienceachievement. The investigator constructed cloze tests frominuividualized science activities and science textbooks.
The instructional program covered three weeks which in-cluded one hundred sixty minutes of science instruction perweek for both groups. Control group teachers taught tradi-tionally using a basal science textbook. Experimental groupteachers used individualized science activities developed bya Bethlehem Area School District ESEA Title III program,Keys for Interdisciplinary Tutorial Study.
The differences between gain scores made by pupils taughtby the individualized method and those made by pupils taughtby the traditional method were determined by a two factoranalysis of variance. The relationships among general readingability, science ability and gain scores on cloze tests weredetermined by the Pearson Product Moment coefficient of cor-relation.
89
Conclusions: Analysis of the data showed the following:
1. The gain scores of the experimental group were sig-nificantly higher than the gain scores of the control group onthe cloze tests constructed from individualized science ma-terials with P. < .001.
2. The gain scores of the control group were significant-ly higher than the gain scores of the experimental group onthe cloze tests constructed from the science textbook materialswith P. < .01.
3. Gain scores made by girls in both the experimentaland control groups were significantly higher than gain scoresmade by boys in both groups on the cloze tests constructed fromindividualized science materials with P. < .01.
4. There were no significant differences in the gainscores made by girls and boys in the experimental and controlgoups on the cloze tests constructed from science textbookmaterials with P. > .05.
5. With the experimental group, the highest relationshipswere found between the post-cloze test constructed from in-dividualized science materials and STEP Reading. The relation-ships between gain scores and all other variables were at bestonly moderately high.
6. With the control group, the highest relationshipswere found among the pre-cloze test constructed from sciencetextbook materials, STEP Reading, and the post-cloze test con-structed from individualized science materials. The relation-ships between gain scores and all other variables were at bestonly moderately high.
The following conclusions were drawn on the 'basis of theanalysis of the data.
1. Cloze test instruments are sensitive to gain measure-ment.
2. The individualized approach to science instructionis not superior to the traditional science textbook approachin terms of cloze test gain scores after a three week period.
3. Fifth grade girls scored significantly higher thanfifth grade boys on the cloze test constructed from individ-
90
ualized science materials probably because of superior read-ing ability.
4. Fifth grade boys will do as well as fifth grategirls in science provided the materials they are expected toread are suitable for them in terms of their level of readingachievement.
5. Cloze test,results are valid measures of reading com-prehension ability.'
'Dissertation Abstracts International As The Humanitiesand Social Sciences Ann Arbor, Michigan: University Micro-films, 1971), Vol. 31, No. 10, p. 5026-A. (Order No. 'p1-10,501, 89 pages).
91
COLEMAN, E. B. "IMPROVING COMPREHENSIBILITY BY SHORTENING SEN-TENCES." JOURNAL OF APPLIED PSYCHOLOGY, 46 (1962), 131-134.
Purpose: The effect of shortening sentenceson compreYgmsibil-ity was investigated.
Principles, Concepts, or Procedures: Three ten-sentence pas-sages were chosen from The Human Species by Geldard for re-writing into two versions containing six and fifteen sentences.
Subjects were ninety college undergraduates who weredivided into five groups. Since every-fifth-word deletionswere used, every word became a test item. Subjects were givenfifty seconds to read each passage before completing it. Onlyexact replacements were scored correct.
Conclusions: The shorter sentences were more comprehensiblethan the longer ones, but the magnitude of improvement wassmall. Four hypotheses are suggested by the author for fur-ther study.
92
COLEMAN, E. B., AND MILLER, G. R. "A MEASURE OF INFORMATIONGAINED DURING PROSE LEARNING." READING RESEARCH QUAR-TERLY, 3 (1968), 369-386.
Purpose: The authors describe two experiments designed tomeasure the degree to which a passage transmits new informa-tion.
Experiment I
Principles, Concepts, or Procedures: Twenty college under-graduates were matched with the subjects who had been admin-istered five forms of a cloze test in a previous study.Since the subjects in the previous study had not been allowedto read the material before taking the cloze test, the sub-jects in this study were allowed to read the material beforetesting in order to compare the two scores to determine theamount of information gain.
Materials and procedures were identical in both studies,except that in the second each subject was administered only18 instead of 36 passages. Sixty seconds were allowed forstudy and six minutes for completion of each passage.
Conclusions: Because of the high correlation of .93 betweenthe two scores, it appears that the cloze score before readingir measuring essentially the same information as the clozescore after reading.
Experiment II
Principles, Concepts, or Procedures: The materials used wereidentical to those in Experiment I. Subjects were-nine collegeundergraduates who were tested as a single group. .3y means oftransparencies and an overhead projector, the experimenter re-vealed one word at a time after the subjects had written theirprediction. Because the procedure was repeated, the differencein correct words was assumed to be a measure of informationgained.
93
Conclusions: The mean gain per hundred words was 38.93, andthe "before" and "after" correlation was .57. This procedureappears to be an accurate measure of information gain. Pas-sages containing few abstract nouns, few verb-nominalizations,few kernels, and many short words transmitted maximum infor-mation gain.
oh
CRANNEY, ADELBERT GARRT JR. "A COMPARISON OF A FREE-RESPONSEAND MULTIPLE-CHOICE CLOZE READING TEST." UNPUBLISHEDPH.D. DISSERTATION, UNIVERSITY OF MINNESOTA, 1968.
Purposes: The cloze procedure consists of the presentation ofa reading passage with deleted words. The reader is requiredto supply the missing words from the remaining context. Th2.9procedure has recently been applied to the mcmsurement of read-ing comprehension.
Traditional reading tests are difficult to construct andtend to divert the attention of the reader to comprehensionquestions rather than to the reading of the passage itself.Cloze tests in both respects seem to offer an improved methodof measuring reading comprehension.
The purpose of the study was to compare two types ofcl.-dze reading tests with respect to validity and reliability.A second purpose of the study was to investigate the effectson reliability and validity of eliminating ineffective itemsfrom the two cloze tests.
Principles, Concepts, or Procedures: Two, 300 item cloze testswere constructed. The first was an every 10th deletion, clozetest based on six college textbook passages. The second testwas based on the same material but was multiple-choice in form.Distractors for the correct responses on this test were general-ly synonyms. On the basis of item analysis techniques, 165 in-effective items were eltlinated from each test providing a longand short form of each test. Two groups of 100 college sopho-mores were administered the long and short forms of one ofthese tests and also the Cooperative Reading Test. Item selec-tion was based on two other groups of 400 students each.
Comparisons among the several test forms were based onreliability coefficients and validity correlations of clozescores with Cooperative Reading. Test comprehension scores.Tests of significance employed the t, F, or z statistics andthe .01 level of confidence.
1. Reliability coefficients on all test forms ranged from.83 to .93 and validity coefficients with the Cooperative Read-ing Test comprehension scores ranged from .51 to .54.
2. The comparison of the free-response and multiple-choice test forms resulted in no significant differences inreliability. One significant validity difference was found
95
in favor of the multiple-choice form. This form also containedabout 30% more difficult items and was the better test in re-spect to the distribution and range of item correlations withtotal cloze test scores.
3. The shortening of each long test significantly de-creased the reliabilities of each of the shorter forms but didnot significantly change validity coefficients with the Coopera-tive Reading Test comprehension scores.
4. Comparison between the short forms of the two testsresulted in a significant validity difference in favor of themultiple-choice test form, but no significant differences be-tween the reliabilities of the two forms.
Conclusions:
1. It was concluded that all cloze test forms were validmeasures of reading comprehension having about 26% of commonvariance with the Cooperative Reading Test.
2. All test forms were also considered reliable measuresof reading comprehension for group comparisons but not ade-quate for individual comparisons.
3. Item analysis and selection procedures used in thisstudy do not improve validity coefficients of either test.
4. The two cloze tests constru7ted in this study testeddifferent abilities and are, therefore, not comparable.1
1Dissertation Abstracts A: The Humanities and SocialSciences (Ann Arbor, Michigan: University MicrofITIT477568),Vol. 29, No. 3, p. 811-A. (Order No. 68-12,251, 169 pages).
96
CRAWFORD, ALAN NEAL. "THE CLOZE PROCEDURE AS A MEASURE OF THEREADING COMPREHENSION OF ELEMENTARY LEVEL MEXICAN-AMERI-CAN AND ANGLO-AMERICAN CHILDREN." UNPUBLISHED ED.D. DIS-SERTATION, UNIVERSITY OF CALIFORNIA, LOS AI4GELES, 1970.
Purpose: The purpose of this study was to investigate the va-lidity, reliability, and appropriateness of cloze tests as ameasure of the reading comprehension of third and sixth gradeMexican-American and Anglo-American children.
The three hypotheses and two related questions involvedfour basic problems: (1) the correlations between cloze testscores and scores on standardized reading tests and oral read-ing tests for groups at each grade level: (2) the differencesamong cloze test scores for groups at each grade level; (3)the reliability coefficients of cloze test scores for groupsat each grade level; and (4) the degree of correspondence be-tween instructional reading levels determined by cloze testsand by standardized reading tests, oral reading tests, andteacher judgment, for groups at each grade level.
Principles Concepts, or Procedures: The randomly selectedpopulation of 180 subjects consisted of third and sixth gradeMexican-American and Anglo-American children whose dominanthome language was English, and Mexican-American children whosedominant home language was Spanish. Each of the resultinggroups, which were designated MAE, MAS, and AE at each gradelevel, consisted of 30 subjects. On the basis of standardizedreading test scores, the subjects at each grade level wereclassified as more able or less able readers.
The reading subtests of the Stanford Achievement Test,Primary II, Form Y, and a cloze test at the second gradelevel were administered to all third grade subjects. Thereading subtests of the Comprehensive Tests of Basic Skills,Level 2 Form R, and a cloze test at the fourth grade levelwere administered to all sixth grade subjects. The GilmoreOral Reading Test, Form C, was administered to selected sub-jects in each group at both grade levels. Additional clozetests at other grade levels from first to seventh were admin-istered to all subjects in order to determine their instruc-tional reading levels.
97
Conclusions: Correlations between scores on cloze tests and onstandardized reading tests and oral reading tests were signifi-cant for MAE, MAS, and AE subjects at the third and sixthgrata levels. Correlations were significant for groups classi-fied as more able or less able except where low variability wasproduced by the restriction on the range of scores. A signifi-cant difference among cloze test scores of the MAE, MAS, andAE groups at the third grade level did not appear at the sixthgrade level. The reliability coefficients of all groups exceptthe sixth grade MAE group were above .70. Reliability coeffi-cients for groups classified as more able and less able andfor the sixth grade MAE group were affected by low variabilityproduced by the restriction on the range of scores; resultswere therefore inconclusive. Instructional reading levels de-termined by cloze tests corresponded closely with those deter-mined by the other measures.
It was concluded that cloze tests were a valid and re-liable means of testing reading comprehension and determininginstructional reading level for these third and sixth gradeMexican-American and Anglo-American children.1
1Dissertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1971), Vol. 31, No. 7, p. 3162-A. (Order No. 71-610.99 pages).
98
DARNELL, DONALD K. "THE RELATION BETWEEN SE'ITENCE ORDER ANDCOMPREHENSION." SPEECH MONOGRAPHS, 30 (1963), 97-100.
Purpose: This study investigated the relationship betweenthe accuracy of predictions about the missing parts of a mes-sage and the order of the sentences within the message.
Principles, Concepts, or Procedures: The experimental messagecomprised fifteen sentences--a thesis, two major contentions,two subcontentions for each, and two assertions for each sub-contention. These sentences were systematically rearrangedinto six other orders.
The seven resulting forms--ranging from "correct order"to "most mixed"--together with a control message were admin-istered to 140 college undergraduates. Each form was adminis-tered to twenty subjects. Time limits were imposed. Thecloze deletion system is not described.
Conclusions: While disorder did affect comprehension in thisstudy, this finding is not generalizable. While showing thatloss of clarity becomes greater as disorganization increases,the author cites several related areas requiring research.
99
DAUGHERTY, JOAN. "THE EFFECT OF THE CLOZE PROCEDURE AND IN-TERSPERSED QUESTIONS AS AN AID TO READING COMPREHENSION."UNPUBLISHED PH.D. DISSERTATION, THE UNIVERSITY OF AKRON,1971.
Purpose: The purpose of this study was: (1) to compare dif-ferent modes of presentation of reading material and evaluatetheir effects on reading comprehension as measured by retentionon a multiple-choice test: (2) to ascertain if different pre-sentations of reading material produce different results inretention of Knowledge-, Comparehension-, and Application-typequestions as defined in Bloom's Taxonomy of Educational Objec-tives; (3) to substantiate previous research findings that theplacement of questions does facilitate learning.
Principles, Concepts. or Procedures: The subjects were 197fourth grade pupils whose mean IQ score was within the normalrange of intelligence (90-110) as measured by the CaliforniaTest of Mental Maturity. The sample was limited to thoseschools located in geographical areas defined as white, middle-class socio-economic districts.
The six treatment groups were as follows:
1. Knowledge-type questions were interspersed in thereading passage after each paragraph. The subjectswere required to answer each question mentally.
2. Comprehension -type questions were interspersed inthe reading passage after each paragraph. The sub-jects were required to answer each question mentally.
3. The cloze procedure using a pattern of deletion of10 per cent of the noun determiners on each page ofthe reading passage required the subjects to replacethe deletions.
4. The doze procedure using a pattern of deletion ofevery tenth word in the reading passage required thesubjects to replace the deletions.
5. An irrelevant reading passage was included to estab-lish a baseline of learning.
6. The reading passage of the control group had neitherdeletions nor interspersed questions.
100
The reading material was in booklet form with the mul-tiple-choice test at the end of the booklets. The data col-lected was analyzed statistically by analysis of variance.
Conclusions: The results indicated there was no significantdifference between the mean score of the experimental treat-ment groups and the control group on the twenty-item multiple-choice test. On the subtests which consisted of eight Knowl-edge, seven Comprehension, and five Application questions, nosignificant difference existed.
It was concluded that with the sample used in thisstudy, neither interspersed questions nor word deletionsaided comprehension. However, additional research should beconducted before definite decisions are made on which modeof presentation of reading material is most effective in in-suring that the maximum level of comprehension is reached.1
1Dissertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1971), Vol. 32, No. 2, p. 664-A. (Order No. 71-20,275,82 pages).
101
FELDMAN, MARGARET EICHBAUER. "THE EFFECTS OF LEARNING BYPROGRAMED AND TEXT FORMAT AT THREE LEVELS OF DIFFICULTY."UNPUBLISHED PH.D. DISSERTATION, CORNELL UNIVERSITY,1964,1
Purposes: The present study was designed to investigate twoaspects of the stimulus characteristics of learning materials:first, programed or text format; and second, differing diffi-culty levels of the reading materials. One characteristic ofthe learner, namely his verbal ability, and the interactionsamong all three of these variables were also investigated.
Principles_,_ Concepts, or Procedures: Three versions of a be-ginning psychology program were prepared by increasing thereading difficulty level from the original "standard - fairlydifficult" level to one of "difficult" and another of "verydifficult" reading level. Using these materials as a base,three text versions were prepared, one for each of the threedifficulty levels, by filling in the blanks and organizingthe materials into paragraphs. Half of a sample of 144 collegestudents had been selected for high verbal. ability and half forlow. Half of each of these groups took one or the other of thethree difficulty levels of program and the other half took oneof the three levels of text material. Tests of learning wereby means of cloze tests which used material identical to thatbeing learned, and transfer tests which asked students totransfer their learning to new problems.
Conclusions: Using a cloze test measure of gains, no signifi-cant differences were found between learning by program or text.Using a transfer measure of learning, a significant differencewas found in favor of text for low verbal ability subjects,with a trend in the same direction for high ability subjects.These findings lend support to the conclusion that format isnot a critical determiner of learning with material similar tothat being tested but that the larger "wholes" of the textappear to provide better transfer than the "parts," as repre-sented by the program items. It may be hypothesized that pro-gramed format destroys organizational patterns of the learnerby the constant interruption by calls for response and may
1See also Margaret E. Feldman, "Learning by Programmedand Text Format at Three Levels of Difficulty," Appendix.
102
lead to premature clozure.
No significant differences in learning were found at-tributable to the three difficulty levels of the learning ma-terials, although the materials were perceived subjectivelyby the subjects to conform to the expected difficulty levels.It may be that other values of the try-out procedure used tocreate programs, such as better organization and more repeti-tions, may be more critical for learning than the readabilitylevel.
No significant differences were found between high andlow verbal ability groups on gains as measured by the clozetest, but a significant difference was found in favor of thehigh ability group on the transfer test.
A significant interaction was found between verbal abil-ity and difficulty level of the material on the cloze testmeasure of gains. Subjects made more gains when the materialwas at a level of challenge but still not too difficult. Noother interactions were found to be significant.
Perhaps the significant contribution of programed learn-ing will be an understanding of how to write textbooks whichwill communicate well and still offer the possibilities fororganization by the student as provided by a text.l
1Dissertation Abstracts (Ann Arbor, Michigan: UniversityMicrofilms, 1964), Vol. 25, No. 3, p. 1738. (Order No. 64-8105, 125 pages).
103
FLETCHER, JUNIOR EUGENE. "A STUDY OF THE RELATIONSHIPS BE-TWEEN ABILITY TO USE CONTEXT AS AN AID IN READING ANDOTHER VERBAL ABILITIES." UNPUBLISHED PH.D. DISSERTA-TION, UNIVERSITY OF WASHINGTON, 1959.
Purpose: For the mature reader, . general context, totalword form, and detailed word parts all function in word recog-nition. While the role of total word form and detailed wordparts in the recognition of printed symbols had been ratherthoroughly studied, the importance of the contribution madeby context to the whole perceptual process of reading had notbeen as systematically investigated. It was the purpose ofthis study, therefore, to gather data of a quantitative naturewhich would give more specific information as to the impor-tance of the role played by ability to use context in wordrecognition.
Principles, Concepts, or Procedures: To do this, the abilityto use context of 174 University of Washington freshmen wasmeasured using an adaptation of Taylor's "cloze" procedure.A best was devised using three reading selections, each approxi-mately 250 words in length, in which every fifth word was re-placed by a blank. The subjects were instructed to fill inthe blanks with the words they thought belonged there on thebasis of the surrounding verbal context. Scores were givenon the basis of the number of blanks filled with words whichmatched those used by the author of the selection.
Reading abilities were measured with the CooperativeEnglish Test, Reading Comprehension and the Rate of Compre-hension Test, Dvorak-Van Wagenea Examination of Silent Read-ing Abilities. These provided scores for reading vocabulary,level of comprehension, and two rate of comprehension scores.In addition, the American Council on Education Examinationof College Freshmen (ACE) was used to measure general verbalability.
Conclusions: When test results were correlated, substantialrelationvhips were disclosed between ability to use contextas measured by the "cloze" procedure test and all of the testsof reading abilities and also the test of general verbal abil-ity. The two tests designed to measure speed of comprehension,the Dvorak-Van Wagenen Rate of Comprehension Test and the Co-operative English Test, Speed of Comprehension, showed practi-cally the same correlations with the -"cicze" procedure test,590 and .570, respectively, ,hile their intercorrelation was
104
.560. The Cooperative English Test, Level of Comprehensionshowed a correlation of .547 with the "cloze" procedure testwhile the Cooperative English Vocabulary gave a correla-tion of .630. The highest correlation, .718, was between theACE, Linguistic Section and the "Cloze" procedure test.
A study was made of the characteristics of the "cloze"procedure test, particularly to determine its reliability.Reliabilities calculated using both the Kuder-RichardsonFormula 20 and a split-half technique were the same, .87,which also indicated a highly homogeneous test. An examina-tion of the test items indicated a good spread of difficultylevel and showed that almost all of them discriminated posi-tively between low and high scorers on the test.
The substantial relationships between general ability touse context and reading adlities, as shown in this study,lent support to the conclusion that ability to use context asan aid in reeding is a significant underlying factor in thereading proficiency of the mature reader.
It was recommended that the effects of variations inmake-up, content, and method of administration of "cloze"procedure materials be further investigated: that studies bemade at lower grade levels to gather information about thegrowth of ability to use context; and that similar studiesusing "cloze" materials and reading tests containing materialclosely comparable as to level of difficulty, type, and sub-ject matter content be carried out.'
1Dissertation Abstracts (Ann Arbor, Michigan: Univer-sity Microfilms, 1960), Vol. 20,.No. 7, pp. 2675-2676. (L. C.Card No. Mic 59-5483, 104 pages).
105
GEYER, JAMES RUSH. "THE CLOZE PROCEDURE AS A PREDICTOR OFCOMPREHENSION IN SECONDARY SOCIAL STUDIES MATERIAL."UNPUBLISHED ED.D. DISSERTATION, UNIVERSITY OF MARYLAND,1970.1
Purpose. The major purpose of this study was to investigatethe effectiveness of the cloze procedure as a predictor ofstudents' abilities to comprehend social studies materials.A secondary investigation dealt with the effectiveness ofutilizing social studies materials written on lower readabil-ity levels to improve comprehensibility.
Principles, Concepts, or Procedures: The following hypotheseswere investigated:
A. Cloze Procedure
1. Cloze test scores predict students'comprehend sc,:ial studies materialsdo standardized reading test scores
2. Cloze test scores predict students'comprehend social studies materialsdo I.Q. scores.
3. Cloze test scores predict students'comprehend social studies materialsdo previous social studies grades.
B. Rewritten Materials
abilities tobetter than
abilities tobetter than
abilities tobetter than
Students achieve higher scores on the criterion mea-sure, a multiple-choice test, when reading socialstudies materials written on a low readability levelthan when reeding social studies materials writtenon a more difficult level.
Instructional materials utilized in this study were twojunior high school American history texts. Both texts con-tained identical topics and visual aids; however, readabilitylevels rated by the Dale-Chall readability formula were dif-ferent.
1See'also James R. Geyer, "Cloze Procedure as a Predic-tor of Comprehension in Secondary Social Studies Materials,"Appendix.
106
The data consisted of scores from the pre-reading clozetests, an I.Q. test, previous social studies grades, a stan-dardized reading test, and the criterion measure, a multiple-choice test. The subjects were 201 eighth-grade stuffentsfrom a junior high school population. All subjects completedone of two cloze test forms. Then they read the chapter inthe text from which the cloze test had been developed and com-pleted the multiple-choice test. Significant differences atthe .01 level between two correlation coefficients involvinga common variable were investigated in reference to the clozehypotheses. An analysis of covariance was applied to ascertainthe effectiveness of rewritten social studies materials onlower readability levels as a means of improving comprehensi-bility of that material.
Conclusions: The following conclusions were drawn from thefindings..
1. The research hypothesis that cloze test scores pre-dict students' abilities to comprehend socialstudies materials better than do standardized read-ing test scores was not supported by the data.
2. The research hypothesis that cloze test scores pre-dict students' abilities to comprehend socialstudies materials better than do I.Q. scores wasnot supported by the data.
3. The research hypothesis that cloze test scores pre-dict students' abilities to comprehend socialstudies materials better than do previous socialstudies grades was not supported by the data.
4. The research hypothesis that students achieve high-er scores on the criterion measure, a multiple-choice test, when reading social studies materialswritten on a low readability level than when read-ing social studies material written on a more diffi-cult level was not supported by the data.
The following recommendation for practice is suggestedby the data from the study: Care should be taken to ascertainthat published materials which are rewritten on lower reada-bility levels are more readable for the intended student. Thedata of the study suggest that application of readability for-mulas may not predict the ability of students to comprehendthe materials adequately.
Several implications for research are suggested by thedata. First, although the cloze procedure was not found to he
107
the best predictor of ability to comprehend specific instruc-tional material, it was correlated highly enough with the cri-terion measure to warrant further investigation. The rationalefor the preceding statement lies in the significant differencesat the .05 level for hypotheses two and three. Secondly, afurther investigation of the effectiveness of combinations ofindependent variables as criteria for predicting academicsuccess in content areas appears to be warranted. Such aninvestigation is especially relevant because of evidence thatquantitative evaluation of instructional material may not givea true picture of readability.
The theory which is the basis for most readability formu-las involves objective, quantitative assessment of passages.Implications for theory include continued investigation ofcertain qualitative factors. Assessment of such qualitativefactors as concept load, abstractness, and organization, es-pecially through interaction involving the reader and instruc-tional material, is recommended.1
1Dissertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1970), Vol. 31, No. 5, p. 2002-A. (Order No. 70-22,549, 152 pages).
108
GREENE, FRANK PIERREPONT. "A MODIFIED CLOZE PROCEDURE FORASSESSING ADULT READING COMPREHENSION." UNPUBLISHED PH.D.DISSERTATION., THE UNIVERSITY OF MICHIGAN, 1964.
Purpose: This study reports 1) an investigation of the effecton concurrent validity of modifying the standard cloze proce-dure and 2) an attempt to demonstrate two components of readingcomprehension as measured by the cloze technique.
Principles, Concepts, or Procedures: The cloze procedure con-sists of deleting every nth word from a reading passage. Thedeleted words are replaced by numbered blanks, and the readeris asked to fill in the words missing from the passage. Thepercentage of words to be deleted is set and restrictions asto the class(es) of words eligible for deletion may be set.The deletion process is automatic: within the chosen deletionrate and eligible words, every nth word is removed.
The standard procedure used was the automatic deletion ofevery 12th word with no word class restrictions. The modifiedcloze procedure restricted eligible words to nouns, verbs, ad-verbs, and adjectives (content words) and each potential dele-tion was evaluated for possible effectiveness. For each worddeleted under the modified procedure, there was felt to besufficient redundancy remaining in the passage so that a su-perior reader could make positive identification of the missingword.
All testing was completed in one regularly scheduled classperiod. The subjects (N = 128) were all regularly enrolled inthe University-of Michigan and were divided in the proportions,80% female/ 20% male and 56% undergraduate/ 44% graduate stu-dents.
Conclusions: The following findings were reported: 1) No sig-nificant differences were found between the coefficients of thetwo cloze procedures and two standardized reading tests. Allcoefficients were significant.
2) Two cloze tests constructed by the standard procedurebut starting with different words did not differ in mean scoreor standard deviation.
'See also Frank P. Greene, "Modification of the Cloze Pro-cedure and Changes in Reading Test Performances," Appendix.
3) Item discrimination indices (r) demonstrated that themodified doze procedure produced items of higher "quality."Item difficulty indices (R) showed that the modified procedureeliminated items of extreme difficulty or ease.
4) The doze scores were not significantly raised byreading the original passage prior to taking the doze test ascontrasted with taking the doze test without prior reading.
5) Reading rate was found to be correlated at a low, butsignificant level with the performances on the doze tests.
6) Multiple correlation coefficients of the doze testsresults with the power and context vocabulary tests, while sig-nificant, accounted for only about one third of the doze testvariance.
It was concluded that 1) the modified doze procedure doesnot improve the effectiveness of the standard doze procedureas a measure of adult reading comprehension, and 2) the two vo-cabulary tests (power and context) do not, by themselves,account for the variance of the doze measure of reading com-prehension."-
'Dissertation Abstracts (Ann Arbor, Michigan: UniversityMicrofilms, 1965), Vol. 25, No. 10, p. 5734. (Order No. 65-5308, 130 pages).
110
GUTHRIE, JOHN T. "MOTIVATIONAL EFFECTS OF FEEDBACK IN READING."BALTIMORE, MARYLAND: CENTER FOR THE STUDY OF SOCIAL OR-GANIZATION OF SCHOOLS, JOHNS HOPKINS UNIVERSITY, 1971.
Purpose: The effects of feedback on motivation were investi-gated with a 3 x 3 factorial design.
Principles, Concepts, or Procedures: Adult subjects (72 malecollege students) read prose sentences and completed clozetest items. Feedback on each item was either immediate, de-layed, or omitted. A cloze retention test over the sentenceswas given either immediately, delayed, or was omitted. Toassess motivation, the subjects were given a continuation ofthe original passage which they read for as long as they wished.The time spent reading was recorded as a measure of persever-ance and motivation.
Conclusions: The results indicated that (1) delayed feedbackproduced significantly more learning on the original task thandid immediate feedback, (2) immediate feedback produced sig-nificantly more perseverance on the continuation passage thandid delayed feedback, and (3) perseverance on the continuationpassage was positively correlated (.46) with scores on a com-prehension test over the continuation passage. The results areexplained in terms of differential affective responses ac-quired to the reading task under different feedback conditions.Tables, references, and appendixes are included.1
1Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1971), Vol. 6, No. 1, p. 75.(ED 042 590, 35 pages).
111
HAFNER, LAWRENCE E.; GWALTNEY, WAYNE; AND ROBINSON, RICHARD."READING IN BOOKKEEPING: PREDICTIONS AND PERFORMANCE."JOURNAL OF READING, 14 (1971), 537-546.
Purpose: The study was undertaken to determine the relation-ships between intelligence, reading, and teachers' ratings ofstudents' characteristics.
Principles, Concepts: or Procedures: Subjects were 45 maleand female high school students enrolled in bookkeeping classes.Three cloze tests were administered to all subjects. Oneutilized every-fifth-word deletions in randomly selected pas-sages from one of students' texts. Another utilized every-fifth-noun or verb deletions in randomly selected passages ofthe same text. The third again used the same text but re-quired subjects to replace deleted punctuation marks.
Teachers rated the subjects on the following variables:intelligence, achievement motivation, reading ability, and un-derstanding of bookkeeping content in the text. Results frompreviously administered intelligence and achievement testswere also used.
Conclusions: Among the conclusions were that significant rela-tionships existed between reading- close and measures of intelli-gence and achievement.
112
HATER, SISTER MARY ANN. "THE CLOZE PROCEDURE AS A MEASURE OFTHE READING COMPREHENSIBILITY AND DIFFICULTY OF MATHE-MATICAL ENGLISH.", UNPUBLISHED PH.D. DISSERTATION, PURDUEUNIVERSITY, 1969.-L
Purpose_: The cloze procedure has been used by researchers asa measure of readability. Cloze tests are constructed by de-leting words or symbols from passages and replacing them withblanks. The subject attempts to complete the passages. Hisscore for each passage is the number if responses which matchthe deleted material.
Although the cloze procedure has been validated as a mea-sure of comprehensibility and difficulty for English passages,cloze tests cannot be used indiscriminately to measure theseconstructs without considering the content area of the passages.
In this research, the cloze procedure was applied to math-ematical English. Because of the concentrated symbolism andother factors which differentiate mathematical English from or-dinary English, the cloze procedure was adapted to be used inthis content area. In particular, the reliability and validityof the cloze procedure as a measure of the reading comprehensi-bility and difficulty of mathematical English passages werestudied. Reliability was measured by the K-R formula 20 andby rankings of means. The validity of the cloze procedure as ameasure of reading comprehensibility was assessed by correlatingscores on cloze tests with scores on comprehension tests writtenover the same passages. The validity of the cloze procedure asa measure e reading difficulty was assessed by correlating themeans on cloze tests with the means on comprehension tests writ-ten over the same passages.
Princi les Conce is or Procedures: Five mathematical Englishpassages, f ve comprehension tests on the passages, and twenty-five cloze tests, five for each passage, were the basic mate-rial administered to 1,717 subjects of Grades 7 through 10.
Conclusions: Results of this study supported the hypothesisthat cloze tests over mathematical English passages are highly
1See also Mary Ann Hater and Robert B. Kane, "The Cloze
Procedure as a Measure of the Reading Comprehensibility andDifficulty of Mathematical English," Appendix.
113
reliable measures for large or small groups, and across Grades7 through 10 or within each grade. Cloze tests over mathe-matical English passages are valid predictors of the readingcomprehensibility of mathematical English passages for thegrades tested. An average linear correlation of 0.69 (usinga 2 transformation) was found to represent the relationshipbetween cloze test and comprehension test scores. This cor-relation may underestimate the relationship between clozetests and comprehension tests since a quadratic model accountsfor more variance than a linear model. However, the addition-al variance may be due to artifacts of guessing and subjects'distributions on comprehension tests.
Double cross-validation techniques within the same popu-lation were used to arrive at these conclusions. Not only werethe correlations between cloze tests and comprehension testshigh, but correlations were also similar for the differentsamples tested over the same passages and for samples testedover different passages. Therefore, conclusions concerning theuse of cloze tests as predictors of comprehensibility werestrengthened.
Cloze test means over mathematical English passages wereranked the same by different samples from the same populationand were related highly for samples from different gradelevels. Thus, it was concluded that the ranking of cloze testsis a reliable measure.
Since the number of mathematical English passages used inthis study was only five, conclusions concerning the validityof cloze tests as measures of reading difficulty are tentative.Cloze test means were ranked the same as comprehension testmeans for four of the five passages. The difference in theranking for the fifth passage alone caused correlations to belower than desired. Consequently, there is enough evidence tosuggest the probable conclusion that cloze tests are valid pre- idictors of reading difficulty for mathematical English passages.'
1pissertation Abstracts Internationaland Social Sciences (Ann Arbor, Michigan:films, 1970), Vol. 30, No. 11, p. 4829-A.334 pages).
A: The HumanitiesUniversity Micro-(Order No. 70-8900,
HENRY, PEGGY ELAINE. "THE EFFECT OF INTEREST ON READING COM-PREHENSION AS MEASURED BY CLOZE AND MULTIPLE CHOICETESTS." UNPUBLISHED PH.D. DISSERTATION, THE UNIVERSITYOF IOWA, 1969.
Purpose: The central problem studied in this investigation washow interest affected reading comprehension. There were twomeasures of reading comprehension and two separate designs em-ployed to study this question.
Part I
Princi les Concepts, or Procedures: Part T of the study in-cluded 0 matched pairs of boys and 82 matched pairs of girlswho were tested in reading comprehension by means of tradi-tional multiple choice tests. First, each child took a pre-test over a selection he had not yet read. Seven days laterhe read the selection and took a post-test over the contentcontained in it. Two selections, comparable in readability,were involved in this design, one about a topic that appealedto boys and one about a topic that appealed to girls. Thetwo members of each pair read and took tests over differentselections.
Conclusions: Comparisons were made between boys and girls onthe same selection and between groups of the same sex on dif-ferent selections. The differences between the mean scoresof these four groups were tested simultaneously with analysisof covariance, then the adjusted means of the various groups,two at a time, were tested for significance with t-tests.
The following differences between the adjusted meanscores were significant at the .05 level of confidence: 1.The mean score of the girls was higher than the mean score ofthe boys on the test over the selection that was appealing tothe girls. 2. The mean score of the boys was higher than themean score of the girls on the test over the selection thatwas appealing to the boys. 3. The mean score of the girls onthe test over the selection that was appealing to the girlswas higher than the mean score of the girls on the test overthe selection that was appealing to the boys.
It was concluded that reading comprehension as measuredby the multiple choice tests in this study appeared to be posi-
115
tively and significantly affected by the readers' interest inthe selection over which the tests were constructed.
Part II
Principles, Concepts, or Procedures: Part II of the study wet;conducted with fourth and sixth graders who took two sets ofcloze tests over two selections comparable in readability.Both sets of selections consisted of pairs, one about a topicthat would appeal to boys and the other about a topic thatwould appeal to girls. After taking each set of cloze tests,the subjects indicated a preference for one of the selectionsin the set. One hundred thirty-six subjects took the firstset of cloze tests and 149 took the second set.
Conclusions: Comparisons were made between the two preferencegroups for each of the four cloze tests. That is, the subjectspreferring one selection and the subjects preferring the otherselection were compared on the same selection.
Analysis of covariance with ITBS reading comprehensionscores as the covariate was the statistical design used toanalyze the data. This served to equate the preference groupson reading ability so that initial differences in readingability were not reflected in the subjects' cloze test scores.The tests for each of the four preference groups indicatedthat there were no significant differences between any twoadjusted means. The conclusion drawn was that for these sub-jects, over these two sets of selections, preference for aselection did not appear to affect how it was comprehendedwhen comprehension was measured by cloze tests.'
1Dissertation Abstracts International A: The Humanitiesand Social Sciences lAnn Arbor, Michigan: University Micro-films, 1970), Vol. 30, No. 9, pp. 3857-A-3858-A. (Order No.70-4369, 171 pages).
116
KNIGHT, DAVID W. "THE READING PERFORMANCE OF STUDENTS WITHNINTH GRADE READING ACHIEVEMENT ON OCCUPATIONAL INFORMA-TION MATERIALS WRITTEN WITH VARIOUS LEVELS OF READABILITY."UNPUBLISHED PH.D. DISSERTATION, THE FLORIDA STATE UNIVER-SITY, 1966.
Purpose: The purpose of this study was to investigate theability of students whose reading achievement on the Metropoli-tan Achievement Test is at the 9th grade level, to comprehendoccupational information materials written at the 9th, 12th,and 15th grade levels as determined by the Flesch ReadabilityFormula.
Principles, Concepts, or Procedures: The study was designedto make a comparison between 9th grade reading achievement andreading passages of occupational information written at threelevels of readability. A treatment X subjects design was used.The treatments were one 50 item cloze test at each level ofreadability. The subjects were 150 students whose score on thereading section of the Florida State-Wide Ninth-Grade TestingProgram was t .5 S.D. of the mean for all students included inthe Florida based norms.
The Flesch Readability Formula was used to analyze occu-pational information briefs randomly selected from the S.R.A.Occupational Information Kit. Each level of readaoility was..represented by a 300 word passage. Cloze tests were made overeach passage. A cloze test score for each passage was derivedby averaging the percentage of words correctly restored to itscloze test forms. This passage cloze score became the basicOoze measure for the analyses. Three tests, one for eachlevel of difficulty, were bound into a booklet with a coversheet of instructions. The tests within the booklet were placedin random order. Only when a subject wrote in the blank theexact word deleted, was his response counted correct.
Conclusions: An analysis of variance with repeated measuresand a multiple analysis t-test were used to analyze the data.Significant differences at the .05 level were indicated in thesubjects performance on levels 9 and 15, also on levels 12 and15. No significant differences were found in the subjectsperformance on levels 9 and 12 at the .05 level.
The rate of comprehension of 3ccupational-information ma-terials decreased as the readability of these materials de-creased,
117
The Flesch Readability Formula ranked the reading pas-sages in the same order of readability as that indicated bythe performance of the subjects on the cloze tests.
The occupational information materials rated at the 9thand 12th grade levels by the Flesch formula appear to be writ-ten at the instructional level (75% to 90% comprehension) forsubjects with 9th grade reading achievement.
Occupational information materials rated at the 15thgrade level by the Flesch formula approached the frustrationallevel (less than 75% comprehension) for subjects with 9th gradereading achievement.
None of the occupational information used in this studyappears to be written at the independent level (90% comprehen-sion) for subjects with 9th grade reading achievement.
The degree of comprehension of occupational informationby 9th grade reading achievement is determined by the reada-bility level at which these materials are written.
The Flesch Readability Formula may be used to estimatethe readability of occupational information if the obtainedestimates are interpreted in broad limits or multiple gradelevel categories rather than as indicators of specific gradelevels.
Occupational information which is rated at the 9th gradelevel cannot be easily understood by students with 9th gradereading achievement.1
'Dissertation Abstracts A: The Humanities and SocialSciences Ann Arbor, Michigan: University Microfilms, 19.7),Vol. 27, No. 8, pp. 2425-A-2426-A. (Order No. 67-297, 71pages).
118
KOHLER, EMMETT THEOFORE. "AN INVESTIGATION OF CLOZE SCORES INTERMS OF SELECTED COGNITIVE VARIABLES." UNPUBLISHED ED.D.DISSERTATION, THE FLORIDA STATE UNIVERSITY, 1966.
Purpose: The major purpose of this investigation was to studythe relationships between cloze scores and selected cognitivevariables. The principal question asked was, What are the rela-41onships between scores obtained on the selected factor-pure
iitive ability tests and scores obtained on the cloze tests-te? Additional, but subordinate questions asked were 1. What
ate the relationships between the various scores derived fromthe cloze tests? 2. What are the relationships between thecloze scores and a set of factorially more complex achievementand ability measures?
Principles, Concepts, or Procedures: The subjects for thisstudy were 257 tenth-grade students from one Florida publichigh school. The data from these students were used for allthe major analyses with the exception that only the data from200 of these were used for the analyses of the relationshipsbetween the scores obtained on the Florida State Wide Ninth-grade Testing Program tests and those from the cloze tests.
The 250 word cloze tests were constructed from eight ran-domly selected passages by use of an every-fifth-word deletionprocess. There were two tests from four subject matter areas:(1) Biology, (2) Chemistry, (3) American Government, (4) WorldHistory. Three scores were obtained for each test: (1) Totalnumber of words correctly replaced, (2) number of Content wordscorrectly replaced, (3) number of Connective words correctlyreplaced.
A battery of eleven factor-pure tests described in theManual for Kit of Reference Tests for Cognitive Factors ande ght lorida State ide Ninth-grade Testing Program testswhich included the School and College Ability Test, and testsfrom the Metropolitan Achievement Tests battery were used asindependent variables.
The major statistical techniques used were multiplelinear regression analyses supplemented by canonical correla-tion analyses. The relationships between the various clozescores were investigated through the use of factor analysis.
Conclusions: Several conclusions were drawn.'
(1) The cloze tests appear to be relatively independent ofthe content covered by the tests and little additional informa-
119
tion was gained from the use of the Content and Connective wordscoring schemes.
(2) There was a general factor present in the doze scoreswhich was believed to be factorially complex and highly relatedto the School and College Ability Test, Verbal score.
(3) Although the Florida State Wide Ninth-grade TestingProgram battery generally did a better job of prediction thandid the factor-pure test battery, the analyses of the dozetests in terms of these factor-pure tests did provide informa-tion as to the abilities required in the successful completionof the doze task. The tests used to measure these abilitiesare presented below in order of their importance.
(a) Wide Range Vocabulary. This ability was thought tobe a reflection of the subject's knowledge of grammatical rulesand breadth of vocabulary.
(b) Logical Reasoning. This was believed to be a some-what formal reasoning ability which is related to the use ofrather rigid sets of rules to solve problems.
(c) Inference. This measure of reasoning was believedto reflect a less formal, more flexible approach to the prob-lem of the doze task.
(d) Addition. This test was believed to represent aspeed factor. Since the time allowed for each doze test wasfixed, this ability to rapidly cover the test could provide thestudent with more exposure to the contextual clues found withintie doze passage.
(e) Hidden Patterns. This measure of Flexibility ofClosure was thought to be important because there are timeswhen given solutions are technically correct but must be castaside in light of the contextual restraints imposed by the pas-sage.
(f) The abilities measured by the following tests appearedto be relatively less important in the successful completion ofthe doze task.
Planning Air Maneuvers Picture-NumberMap Planning Cube ComparisonObject Synthesis Figure Classification
1Dissertation Abstracts A: The Humanities and SocialSciences Ann Arbor, Michigan: Un versity Microfilms, 196 ),Vol. 27, No. 1, pp. 114-A, 115-A. (Order No. 66-5454, 105 pages).
120
LISMAN, LINDA C. "VOWEL DELETION AND CLOZE TESTS COMPARED WITHA READING ABILITY TEST." UNPUBLISHED MASTER'S THESIS,RUTGERS, THE STATE UNIVERSITY OF NEW JERSEY, 1971.
Purpose: The author does not state her purpose.
Principles, Concepts, or Procedures: Fifty-seven seventh and60 eighth graders were divided into three reading abilitygroups. All were given the Wechsler Intelligence Scale forChildren (WISC) prior to the study and the Gates-MacGinitieReading Test--Survey E for grades 7 to 9 immediately after thestudy. A practice sample was given before the tests on pre-pared vowel deletion and cloze passages.
Conclusions: Performance on both tests was correlated witheach other, with scores of the reading test, and with the WISCscores. Correlation between vowel deletion and cloze ecoreswas found to be .43 for the total group. Vowel deletion cor-related significantly with the reading test (speed, .51;accuracy, .63; vocabulary, .57; and comprehension, .51), andwith the IQ test (performance, .39; full scale, .45; and ver-bal, .37). The correlation was higher for eighth graders onspeed, accuracy, and vocabulary, but higher for the seventhgraders on comprehension. The cloze test correlated with thereading test as follows: speed, .29: accuracy, .41: vocabu-lary, .55; and comprehension, .60. It correlated with the 1Qtest as follows: performance, .28; full scale, .46: verbal,.50. Generally, for all measures except verbal IQ and read-ing comprehension, the vowel deletion correlations were higherthan or almost identical to the cloze correlations for thetotal popylation. Tables, a bibliography, and appendixes areincluded."-
1Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1971), Vol. 6, No. 12, p. 77.(ED 053 894, 64 pag9e).
121
MJSBERG, LUDWIG; POTTER, THOMAS C.; AND CORNELL, ROBERT K.THE RELATION BETWEEN CLOZE AND MULTIPLE-CHOICE TESTSCORES AS A FUNCTION OF RELATIVE PARAGRAPH DIFFICULTY
GRADt LEVEL. INGLEWOOD, CALIFORNIA: SOUTHWESTREGIONAL LABORATORY FOR EDUCATIONAL RESEARCH AND DEVEL-OPMENT, 1968.
Purpose: This study investigated the relationship betweencloze and multiple-choice test scores in grades five and eightwhen reading passage difficulties were two years below, twoyears above, and at subject's grade level.
Principles, Concepts, or Procedures: Materials were drawnfrom the Rate Builders section of the SRA Reading LaboratoryKits. Between three and five questions were added to thosefollowing the selections. A mean ratio of 3:8 was maintained'between multiple-choice items and cloze deletions.
Every-fifth-word deletions were made and five test formsconstructed. Final data were obtained for 708 fifth gradersand 708 eighth graders who had been assigned to either a clozeor a multiple-choice condition. At each grade level, matchedpairs were assigned to passage difficulty levels. Withinthese levels, subjects were randomly assigned to one of fifteenselections at that level.
All subjects were given 25 minutes to complete the twotests. California Achievement Test reading scores were ob-tained six months previously.
Conclusions: Among the findings were the following:
1. The overall performance of the eighth graders wassuperior to that of the fifth graders.
2. The cloze test was sensitive to grade level but lesssensitive to passage difficulty.
3. . . one should proceed cautiously when usingcloze test scores as measures of comprehension(or predictors of MC test score magnitudes) un-til a more detailed analysis of what the clozeprocedure is measuring and of grade and diffi-culty ranges at which the cloze procedure is anappropriate measure have been established. (21)
122
RANKIN, EARL FREDERICK, JR. "AN EVALUATION OF THE CLOZE PRO-CEDURE AS A TECHNIQUE FOR MEASURING READING COMPREHEN-SION." UNPUBLISHED PH.D. DISSERTATION, UNIVERSITY OFMICHIGAN, 1958.
Purpose: The cloze procedure, devised by Taylor, is a word-de-letion technique for constructing comprehension tests. Wordsto be deleted are selected mechanically and blank spaces aresubstituted. Subjects are instructed to predict from the re-maining context what word belongs in each space.
The study was an evaluation of the validity, performance,and utility of cloze tests based on "general science material"used by participants in a college reading-improvement program.It was postulated that comprehension tests emphasizing the mea-surement of either the substantive content of a message or theinterrelationships between ideas could be constructed by vary-ing the type of words deleted: the restriction of word dele-tion to nouns and verbs measures primarily the comprehensionof substantive content. The postulate was confirmed, and thisform of the cloze procedure was used in the investigation.
Principles, Concepts,_ or Procedures: Part I was a study ofthe validity of the cloze test as a measure of (1) generalreading skill, (2) pre-reading and post-reading knowledge ofthe content of a specific article, and (3) "specific readingcomprehension" (i.e., the gain between pre- and post-readingtests, attributable to learning through reading). The Diagnos-tic Reading Tests: Survey Section was used as a criterion ofgeneral reading skill, and a multiple-choice test constructedfor the purpose was used as a criterion of pre- and post-read-ing knowledge. Validity was ascertained both for the totalsample and for sub-groups defined by status on the followingorganismic variables: (1) reading rate, (2) intelligence, (3)previous knowledge, (4) level of permeability, (5) level ofanxiety, and (6) four personality types (defined by combina-tions of high and low status on the permeability and anxietydimensions). Personality measurements were made with the SA-SSenior Scales.
In Part II, mean performance on pre- and post-readingtests and the "gain score" was studied as a function of theabove organismic variables.
In Part III, the usefulness of the procedure for the con-
123
struction of multiple forms of a test for measuring improve-ment in reading was considered. The equivalence of fourcloze test forms was studied, and an attempt was made to de-termine the sensitivity of the test to change in reading com-prehension for the total group and for the aforementioned sub-groups.
Conclusions: Correlations ranging from .29 to .48 were ob-tained between cloze tests and the survey test. Validity co-efficients of .59 and .56 were obtained for the tests as mea-sures of pre- and post-reading knowledge of a specific article.Significant differences were obtained between pre- and post-tests. It was concluded that this form of the cloze procedureproduces tests which are not very accurate measures of generalreading skill but are sufficiently accurate for measuring pre-and post-reading knowledge and "specific reading comprehension."
Pre-test reliability, validity, and mean scores were sig-nificantly higher for low permeable than for high permeablesubjects. Such differences did not exist on the post-test.This outcome was interpreted as - reflection of the need ofpermeable individuals for structulJ in making cloze responsesand of the increased structure provided by reading an article.
No differences were found between "gain scores" as afunction of any organismic variable. Unlike other readingtests, this measure gives no advantage to individuals withhigh intelligence or pre-reading knowledge.
Apparently, anxiety reactions to one test form yieldedequivocal results in the improvement study. It was concludedthat the use of multiple forms to measure sub-group differencesin improvement is not desirable unless the equivalence of eachform for the specified sub-groups is determined in advance orunless procedures can be devised to reduce such lack of equiva-lence.1
'Dissertation Abstracts (Ann Arbor, Michigan: UniversityMicrofilms, 1958), Vol. 191 No. 4, pp. 733-734. (L. C. CardNo. Mic 58-3722, 234 pages).
124
RANKIN, EARL F., AND CULHANE, JOSEPH W. "COMPARABLE CLOZE ANDMULTIPLE-CHOICE COMPREHENSION TEST SCORES." JOURNAL OFREADING, 13 (1969), 193-198.
Purpose: The study was designed to replicate Bormuth's studyrelative to comparable scores on cloze and multiple-choice com-prehension tests.
Principles. Concepts, or Procedures: A pilot study preceded themain study which included as subjects 105 fifth grade pupils.Five 50-item cloze tests and five 31-item multiple choice testswere constructed over five World Book Encyclopedia articleswhich ranged in Fry readability from grade five to grade eight.The unmutilated passage and multiple-choice questions were ad-ministered four days after the cloze test. Only exact replace-ments were scored correct.
Conclusions: Bormuth's findings were corroborated. A 75% mul-tiple-choice score is roughly equal to a cloze score of 44%(Bormuth) or 41% (Rankin and Culhane) while a 90% _multiple-choice score is roughly equal to respective cloze scores of 57%and 61%.
125
RANKIN, EARL F., AND DALE, LOTHAR H. "CLOZE RESIDUAL GAIN -A TECHNIQUE FOR MEASURING, LEARNING THROUGH READING."THE PSYCHOLOGY OF READING BEHAVIOR. EDITED BY G. B.SCHICK AND M. M. MAY. EIGHTEENTH YEARBOOK OF THE NA-TIONAL READING CONFERENCE. MILWAUKEE, WISCONSIN:NATIONAL READING CONFERENCE, 1968, 17-26.
Purposes The authors investigated the cloze residual gaintechnique wherein the amount of knowledge learned by readingis separated from that known before reading.
Principles, Concepts, or Procedaress Subjects were 174 col-lege students. The materials consisted of an unmutilatedarticle concerning temperature, two cloze tests made over thepassage, and a multiple - choice comprehension test. A struc-tural cloze test deleted every fifth word while a lexicalcloze test deleted nouns and verbs.
Each class involved was divided evenly with half takinga cloze test followed by.the multiple-choice test and theother half taking these in reverse order. The unmutilatedpassage was not used until thirty days later after which theidentical tests were re-administered to each subject. TheDavis Reading Test was the standardized measure used.
Conclusions: Among the findings were the followings
1. Cloze is a sensitive measure of reading gain.
2. Structural cloze reading gains are related to testsof post-reading knowledge and standardized reading tests whilelexical gains are related only to post-reading knowledge.
126
RANKIN, EARL F., AND HESS, ALLEN K. "THE MEASUREMENT OF INTER-NAL (INTRA-ARTICLE) READING FLEXIBILITY." PAPER PRESENTEDAT THE NATIONAL READING CONFERENCE, ATLANTA, GEORGIA, DE-CEMBER 4-6, 1969.
Purpose: The effects of training on internal reading flexibil-ity (intra-article flexibility) and the relationships betweensuch flexibility and overall reading speed, reading comprehen-sion, anxiety level, and introversion-extraversion were inves-tigated.
Principles, Concepts, or Procedures: Flexibility coefficientswere computed before and after training for the total group of255 college subjects and for sub-groups of 127 subjects whofell in the top or bottom half of the distribution for readingspeed. Desensitizing exercises were employed to allow subjectsto adapt to the reading interruptions necessary to measureinternal flexibility. Subjects were enrolled in a college read-ing program. Pretest and post-test measures were made withalternate forms of the mjglyDiaosticReadiest:SurveSection.Test passage difficulty was determined by the cloze technique.Anxiety level and introversion-extraversion were measured bythe SA-S Senior Scales.
Conclusions: Results indicated that it was possible to desensi-tize subjects to allow for measuring internal flexibility. How-ever, the statistical procedures used and/or the restrictedrange of reading ability and reading difficulty of the selec-tions may have obscured between group differences of flexibilitycoefficients. References are included.1
1Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1970), Vol. 5, No. 8, p. 89.(ED 038 247, 14 pages).
127
RUDDELL, ROBERT BYRON. "AN INVESTIGATION OF THE EFFECT OF THESIMILARITY OF ORAL AND WRITTEN PATTERNS OF LANGUAGE STRUC-TURE ON READING COMPREHENSION," UNPUBLISHED ED.D. DISSER-TATION, INDIANA UNIVERSITY, 1963.1
Purpose: The purpose of this study was to investigate theeffect of the similarity of oral and written patterns of lan-guage structure on reading comprehension of fourth grade chil-dren.
Principles, Concepts, or Procedures: Two research hypotheseswere designed for the purpose of this investigation. Thesehypotheses were: (1) The degree of comprehension with whichwritten passages are read is a function of the similarity oforal patterns of language structure to written patterns oflanguage structure in reading passages. (2) The comprehensionscores on reading passages that utilize high frequency patternsof oral language structure will be significantly greater thanthe comprehension scores on reading passages that utilize lowfrequency patterns of oral language structure. Six exploratoryquestions were designed to investigate the effect of subjectbackground variables on comprehension scores over written ma-terial that utilizes high and low frequency patterns of orallanguage structure. These variables consisted of the occupa-tional status of the father, the educational background of theparents, and the intelligence, mental age, chronological age,and sex of the subjects.
A series of structurally controlled reading passages hav-ing a wide range of similarity indexes were designed to testthe hypotheses. These indexes were determined by assigningempirically determined frequency values of oral patterns oflanguage structure of fourth grade children to identical writ-ten patterns of language structure in the reading passages andtotaling the assigned frequency values. The empirically de-termined frequency values of the oral language patterns offourth grade children were obtained through calculations basedon data collected in a study by Strickland. Vocabulary diffi-culty, sentence length, specific subject-matter content andpassage length were equated in the reading passages. Clozecomprehension tests were constructed over the reading passages
1See also Robert B. Ruddell, "The Effect of Oral andWritten Patterns of Language Structure on Reading Comprehen-sion," Appendix.
128
and administered to 131 randomly sclected subjects on thefourth grade level in the Bloomington Metropolitan SchoolSystem, Bloomington, Indiana. Pertinent background data wereobtained from parents and teachers. A one-way and two-wayanalysis of variance using a repeated measurement design andthe statistic Epsilon were employed in the treatment of data.
Conclusions: Highly significant differences (beyond the .01level) were found between comprehension scores thus support-ing the two research hypotheses. Significant differenceswere found in comprehension scores over reading passagesutilizing high and low frequency patterns of oral languagestructure in relation to all subject background variableswith the exception of the sex variable. An interaction sig-nificant at the .05 level was found in relation to the sexvariable.
The conclusions related to the two research hypotheseswere: (1) Reading comprehension is a function of the similar-ity of oral patterns of language structure to written patternsof language structure used in reading materials. (2) Readingcomprehension scores over materials that utilize high fre-quency patterns of oral language structure are significantlygreater than reading comprehension scores over materials thatutilize low frequency patterns of oral language structure.The conclusions related to the exploratory questions statedthat the occupational status of the father, the educationalbackground of the parents, and the intelligence, mental age,and chronological age of the subjects are significantly re-lated (.01 level) to reading comprehension over reading mate-rials that utilize high and low frequency patterns of orallanguage structure. Sex differences are not significantly re-lated to reading comprehension though there is some evidenceto suggest that boys have disproportionally greater difficul-ty than girls in comprehending reading material written withlow frequency patterns of oral language structure.1
1Dissertati n Abstracts (Ann Arbor, Michigan: UniversityMicrofilms, 196u1, Vol. 24, No. 12, Part I, pp. 5207-5208.(Order No. 64-3826, 207 pages).
129
SAWYER, flITA, AND ERAAM, LEONARD S. "AN EXPERIMENT COMBININGOLOZE PROCEDURE WITH SPEED OF COMPREHENSION INSTRUCTION."PROCEEDINGS OF THE COLI3GE READING ASSOCIATION. EDITEDBY CLAY A. KETCHAM. SYRACUSE, NEW YORK: COLLEGE READINGASSOCIATION, 1964, 27-33.
Purpose: The authors report an investigation into the effectof instruction in speed of comprehension upon reading compre-hension developed from practice with cloze exercises.
Principles Concepts, or Procedures: A group of 134 collegefreshmen were divided into one control and two experimentalgroups. Speed was developed by instructing students to be"flexible" in their reading. The Power Builders of the SRAReading Laboratory were used.
For developing comprehension, one control group usedcollege type reading improvement material and answered theusual types of questions.
The experimental group used cloze materials which weregraded into nine levels. Each selection of 500 words had twocloze forms--unmutilated and every-tenth-word deletions. De-pending on progress, a subject could complete the materialsin a minimum of 27 or a maximum of 48 lessons. Subjectsscanned the Lnmutilated selection for one minute before com-pleting the mutilated form. Both synonyms and exact replace-ments were accepted. Students and instructor discussed re-sponses at variance with the expected replacements.
The second control group was given no special reading in-struction. All groups were administered the Cooperative Read-ing Test at the beginning and end of the semester.
Canclusions: The experimental group achieved higher gains inspeed of comprehension than did control group one, a compar-able group verbally, or than did control group two, a moreskilled group verbally according to ACE scores.
work on speed of comprehension is not antitheticalto work with cloze exercises and the two can be combinedto give greater reading power." (33)
130
SIKES, HELEN CRUSE. "A COMPARATIVE STUDY OF CLOZE PROCEDURESCORES ON ORIGINAL AND PUBLISHED MATERIALS." UNPUB-LISHED ED.D. DISSERTATION, UNIVERSITY OF SOUTHERN MISSIS-SIPPI, 1971.
Purpose: This study was designed to compare the readabilityof creative stories written by children with that of publishedmaterials. The following three aspects of readability wereconsidered: (1) vocabulary load, (2) interest, and (3) com-prehension.
The problem, or comparison is examined according to thefollowing questions: Does a significant difference exist be-tween original and published children's reading materials invocabulary usage and degree of personal references? Is therea significant difference between children's comprehensionscores on original and published materials as measured by theCloze procedure reading test? Do comprehension scores obtainedon a standardized reading test show a relationship to thoseresulting from the child's responses of cloze procedure tests?
PrincirleS,_ Concepts, or Procedures: Two groups of childrenwere involved in the study. Group 1 was composed of 222 fifthand sixth grade students at Hazel Park Elementary School, andGroup II was comprised of 224 fifth and sixth grade studentsat Bridgedale Elementary School. Both schools are in JeffersonParish, Louisiana.
In March, 1970, the comprehension section of the Gates-MacGinitie Reading Test was administered. A "t" test revealedno significant difference between means for the two groups.
At Hazel Park, eighty children scored between gradelevels 5.5 and 6.5. A table of random numbers was used to se-lect twenty-five of these students to write original stories.
At Bridgedale, ninety children scored between 5.5 and 6.5grade levels. Fifty of these students (five groups of teneach) were randomly selected to take cloze tests.
Five selections were used to construct cloze passages.The length of each passage was approximately 250 running wordsand contained fifty items in that every fifth word was deletedand substituted with a blank.
131
Materials came from three sources: (1) SRA, (2) originalstories, and (3) published materials. A practice test was con-structed from the SRA material. It was not scored or used inthe data analysis. Four tests, two original stories randomlyselected and two published materials of the same grade equiva-lent, were constructed into five forms each.
The twenty tests were stacked in random order with regardto group and passage. They were administered to the fifty stu-dents in Group II, one a day for five consecutive days. Thegroups were staggered with regard to the time of day that thetests were taken.
The tests were scored for exactness in replacement oforiginal word.
Conclusions: Hypothesis 1 stated that there would be no sig-nificant difference in the number of words not included in theDale List of 3,000 Familiar Words ,?_en children's originalstories were compared with published materials. To test thishypothesis, the "t" test for independent groups, or uncorre-lated measures, was used. This hypothesis was retained.
Hypothesis 2 specified that there would be a significantdifference in the number of personal inferences when chil-dren's original stories were compared with published materials.The obtained "t" of 4.1398 between personal inferences for pub-lished and original stories was greater than the 2.71 need tobe significant at the .01 level of confidence. Therefore, thenull hypothesis of no difference was rejected.
Hypothesis 3 stated that there would be a significantdifference in comprehension scores, as measured the cloze pro-cedure, comparing original and published children's readingmaterials. The obtained F of 22.48 for treatments was greaterthan the table value of 3.94 at the .b1 level of confidence,Therefore, the null hypothesis was rejected.
Hypothesis 4 was retained. There was no significant re-lationship between cloze procedure test scores.and Gates-MacGinitie comprehension scores at the .05 level of confidence.
132
The results of the study indicated that students compre-hend material written by their peers over published materialswith a greater degree of accuracy and understanding.'
1Dssertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1971), Vol. 32, No. 5, p. 2320-A. (Order No. 71-28,847,171 pages).
133
SMITH, WILLIAM LEWIS. "THE EFFECT OF SYNTAX ON READING." UN-PUBLISHED PH.D. DISSERTATION, THE FLORIDA STATE UNIVER-SITY, 1969.
Purpose: The purpose of the present research was to deter-mine whether more mature syntactic structure increases readingdifficulty, or whether all students, regardless of grade level,have the same syntactic skills and thus do read with equalfacility material written at different levels of syntacticmaturity, providing the vocabulary and content are held con-stant.
Principles. Concepts, or Procedures: Two sets of test instru-ments were constructed for this research, each consisting ofpassages written at four levels of syntactic maturity: 4thgrade, 8th grade, 12th grade, and Skilled Adult. The levelswere established using the findings from recent research onthe syntactic maturation of school children's writing.
The cloze technique was used to test reading comprehen-sion. For each set of test instruments, all combinations ofthe "every-fifth word" deletion pattern were employed.
Two experiments were conducted, each using a differentset of instruments. The same subjects were used in both ex-periments although there was a 6% attrition in the second ex-periment. From each of the grades 4 through 12, 4 groups of30 subjects were randomly selected, each of the groups beingadministered test instruments at one of the 4 levels of writ-ing.
It was hypothesized that, in each experiment, there wouldbe no significant differLnces: 1) in comprehensibility be-tween the 4 levels of writing at each grade level of reader,and 2) in comprehension between the 9 grade levels of readerson each level of writing.
Conclusions: Statistical analysis showed that only the 4thgrade subjects significantly (.05) distinguished between thefour levels of writing in both experiments. They read the 4thgrade writing better than either the 8th grade or SkilledAdult writing. However, in both experiments the older subjects(grades 10, 11, and 12) consistently read all levels of writingsignificantly (.01) better than younger subjects (grades 4, 5,and 6).
134
A critical analysis of the data revealed that, in both ex-periments, the subjects in grades 4, 5, and 6 read the 4thgrade writing best while the subjects in grades 8 through 12read the 8th grade writing best. Furthermore, the older sub-jects read the 4th grade writing less well than they read allother levels. The data implied, therefore, that older sub-jects read syntactically more complex material with greatercomprehension than they read less complex material. A tenta-tive hypothesis was proposed fox this phenomenon.
Four conclusions were drawn from this research:
1. For students in grades 4, 5, and 6, 4th grade writingappears to be easier to read than writing by more mature stu-dents. But for older students, 4th grade writing Is not theeasiest.
2. The more mature students (grades 8 through 12) find8th grade writing easier to comprehend than either the simpler4th grade writing or the more complex writing of the 12thgraders or Skilled Adults.
3. Since 4th grade writing is the most redundant, wordpredictability (which is what the cloze technique measures)is more than a function of grammatical redundancy for themiddle and upper grades. However, for the lower grade stu-dents, redundancy may be the only factor. This phenomenon isopen for speculation.
4. The high correlation (.82) between the results of thetwo experiments indicates that the controlled instrument tech-nique used in this study may provide an effective and simpletool for future research.'
'Dissertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1970), Vol. 30, No. 11, pp. 4970-A, 4971-A. (Order No.70-8575, 134 pages).
135
SWALM, JAMES E. "COMPARISON OF ORAL READING, SILENT READING,AND LISTENING COMPREHENSION ASSESSED BY CLOZE." UNPUB-LISHED ED.D. DISSERTATION, RUTGERS UNIVERSITY, THE STATEUNIVERSITY OF NEW JERSEY, 1971.
Purpose: The purpose of this study was to determine theeffect upon comprehension of oral reading, silent reading, andlistening when using material that had a readability levelequal to the grade level of the students being tested in sec-ond, third, and fourth grades.
Principles, Concepts, or Procedures: A separate story was usedfor each grade. It was anticipated that the results from thisinvestigation would have applicability to instructional proce-dures in regular classrooms for the grade span studied.
Comprehension was measured by means of the cloze technique.A 10 percent random deletion pattern, omitting only lexicalwords, was used in making the deletions from the stories. Stu-dents completed the test using the same method of response asthey had used in learning.
The sample population was composed of 324 students, 108from each grade. They were assigned to the three testingmethods by a process of stratified randomization. Thirty-sixstudents were included in each testing group within a grade:12 each from the above-average, average, and below-averagegroups, respectively.
Two null hypotheses were tested. The first was that therewould be no difference in the comprehension of material readorally, read silently, or listened to on tape in the threegrades. The second hypothesis was that there would be no dif-ference in the level of comprehension for the above-average,average, or below-average reading categories for the threemethods in the three grades. Statistical analysis of the datawas done through the analysis of variance formulas and TukeysHonestly Significant Difference Test. The significance levelfor the investigation was established at .05.
Conclusions: The results for the first hypothesis showedthat significant differences among the three methods existedonly at second grade. In this instance, oral reading was sig-nificantly better than either silent reading or listening. Atthird and fourth grades, no significant differences were noted
136
among the three methods.
The results for the second hypothesis showed that thereading level of the student was important for determiningcomprehension effectiveness with the three methods in eachgrade. Above-average students exhibited a strong tendencyto comprehend better when reading than they did when listen-ing, with significance being reached in second and fourthgrades. At second grade both reading approaches were equallyeffective, while at third and fourth grades silent readingwas slightly better than oral reading. The students withaverage reading ability comprehended equally well with thethree methods of learning regardless of grade.
For the below-average students, listening became the mosteffective method for comprehension, followed by oral readingand then silent reading. Listening was significantly higherthan silent reading at all three grades, but significantly dif-ferent from oral reading at only third grade. Hence, therewas a tendency for listening and oral reading to be about thesame in comprehension effectiveness for this group, especiallyat second and fourth grades.
The following conclusions were drawn from the results ofthis investigation. When the students' reading abilities wereabove the difficulty level of the article, they comprehendedbetter with some form of reading than they did with listening.When the students' reading abilities and the readability levelof the article were about the same, the three learning methodsresulted in about the same amount of comprehension. When thestudents' reading abilities were below the difficulty level ofthe article, listening was better for comprehension purposesthan silent reading. There was also a tendency exhibited withbelow-average readers for oral reading to follow listening incomprehension effectiveness at the second and fourth grades.l
1Dissert tion Abstracts International A: The HumanitiesAnd Social Sciences Ann Arbor, Michigan: University Micro-films, 19 "2), Vol. 32, No. 7, pp. 3578-A, 3579-A. (Order No.72-1102, 160 pages).
137
TAYLOR, WILSON L. "CLOZE READABILITY SCORES AS INDICES OF IN-DIVIDUAL DIFFERENCES IN COMPREHENSION AND APTITUDE."JOURNAL OF APPLIED PSYCHOLOGY, 41 (1957), 19-26.
Purpose:
This experiment specifically tested the hypothesisthat the cloze scores of individual subjects wouldcorrelate significantly with their performances on(a) carefully constructed preknowledge and immedi-ate-recall tests of the content of the material pre-sented, and (b) a standardized aptitude or "intelli-gence" test of supposed "ability to understand." (19)
Principles, Concepts, or Procedures: Three cloze forms--"any," "easy," and "hard"--were prepared from a 20% sample ofan Air Force technical article. "Before" versions of eachform and a pre-test of comprehension were administered to theapproximately 58 -'n in each of three sections. About a weeklater, the complete article was distributed for study, andthe recall test of, comprehension and the "after" cloze ver-sions of each cloze form were administered. Only exact-wordreplacements were counted in the scoring. All subjects hadArmed Forces Qualification Test scores on record as a measureof mental ability.
Conclusions:
For each of the three groups, ccrrelation coeffi-cients were computed between all 10 possible pair-ings of five distributions of scores (before- andafter-study cloze, pre- and post-tests of compre-hension, and AFQT). All 30 such coefficients werefound to be positive and significant to beyond the.001 level of confidence. (26)
138
WEAVER, WENDELL; KINGSTON, ALBERT J.; BICKLEY, A. C.; ANDWHITE, WILLIAM F. "INFORMATION-FLOW DIFFICULTY IN RELA-TION TO READING COMPREHENSION." JOURNAL OF READING BE-HAVIOR, 1 (1969), 41-49.
Purpose:
This study explores certain of the relations be-tween a reading co,. 'ehension measure, accuracy ofcommunication mek. , (the cloze tests), associa-tion and rote lea -__0; tasks, and personality andattitudinal varia _ s. (43)
Principles, Concepts, ?rocedures: Subjects were 53 collegesophomores who were adm_hiatered The Davis Reading Test, TheModern Language Antitude Test, the Cattell 16 PE, and theSemantic X Differential. Two cloze tests were made over areading passageon,. after and one before the passage was read.Two rating scales of bi-polar adjectives were also adminis-tered.
Conclusions:
The results of this study provide further evidencefor the interrelationships of rote learning pro-cesses and so-called higher level mental processes.The best predictors of reading comprehension andpost-cloze scores are sub-tests of the MLAT whichrequire the subject to learn by rote material withwhich he has little or no previous familiarity. .
(Al.?)
CHAPTER IV
THE CLOZE PROCEDURE AS A MEASURE
OF LANGUAGE FACILITY
The cloze procedure has permitted researchers to use
the language of individuals to illustrate characteristics of
certain personality and physical disorders.
In using the cloze procedure with deaf and hearing
people, Cohen studied a population of children and adults
to compare the redundancy of story paraphrases written by
deaf and hearing children, to determine their facility in
making use of this redundancy to reconstruct the paraphrases.
Marshall2investigated the effect of contextual
constraint on deaf and hearing fourth grade children, and
Moores3applied the cloze procedure to an assessment of
1Sandra Ruth Cohen, "Redundancy in the Written Lan-guage of the Deaf: Predictability of Deaf and HearingStory Paraphrases" (unpublished Ph.D. dissertation, Colum-bia University, 1964).
2William James Aloysius Marshall, "Investigating
the Effect of Contextual Constraint on Deaf and HearingChildren at the Fourth Grade Reading Level, Using the ClozeProcedure"(unpublished Ed.D. dissertation, University ofIllinois at Urbana-Champaign, 1970).
'Donald Frederick Moores, "Applications of 'Clone'Procedures to the Assessment of Psycholinguistic Abilitiesof the Deaf" (unpublished Ph.D. dissertation, Universityof Illinois, 1967).
139
140
the psycholinguistic abilities of deaf children.
Specific problems related to speech have been
studied by means of the doze procedure. Fillenbaum and1
Jones applied the technique to a study of aphasic speech.2
Honigfeld evaluated the effect of drugs on speech, and the
ability of schizophrenics to understand normal, psychotic,
and pseudo-psychotic speech.
Attitude and personality studies have been facili-
tated by applications of the doze procedure. Attitudes of
college students toward capital punishment were studied by
Manis and Dawes.3
Honigfeld, Platz, and Gillis4
investi-
gated the relationship between authoritarianism and verbal
style. The relationship between reading test reliability
and validity, and introversion-extroversion of college
1Samuel Fillenbaum and Lyle V. Jones, "An Applica-tion of 'Clone' Technique to the Study of Aphasic Speech,"Journal of Abnormal and Social Psychology, 65 (1962),183-189.
2Gilbert Honigfeld, "Effect of an HallucinogenicAgent on Verbal Behavior," Psychological Reports, 13 (1963),383-3851 Gilbert Honigfeld, "The Ability of Schizophrenicsto Understand Normal, Psychotic, and Pseudo-PsychoticSpeech," Diseases of the Nervous System, 24 (1963), 692-694.
1Melvin Manis and Robyn M. Dawes, "Cloze Scores asa Function of Attitude," Psychological Reports, 9 (1961),79-84.
4Gilbert honigfeld, Arthur Platz, and Roderic D.Gillis, "Verbal Style and Personality: Authoritarianism,"Journal of Communication, 14 (1964), 215-218.
1students has been investigated by Rankin.
141
Other studies have utilized the cloze procedure to
investigate the grammar and syrtax of the English language.2
Coleman and Blumenfeld compared nominalized sentences
with their grammatical transformations for comprehensibility.
Louthan3studied the effects on comprehension of various
systematic grammatical deletions.4
Clark and Clark, Williams, and Tannenbaum 5inves-
tigated, by means of the cloze procedure, the agreement
between encoders and decoders with respect to syntactic
and referential information. The ability of encoders and
decoders in predicting omissions was also investigated by
1Rankin, "Reading Test Reliability and Validity as
Function of Introversion-Extroversion," Journal of Develop-mental Reading, 6 (1963), 106-117.
2E. B. Coleman and J. P. Blumenfeld, "Cloze Scoresof Nominalizations and Their Grammatical TransformationsUsing Active Verbs," Psychological Reports, 13 (1963),69-654.
3Vincent Louthan, "Some Systematic Grammatical De-letions and Their Effects on Reading Comprehension,"English Journal, 54 (1965), 295-299.
4Ruth Anne Clark, "The Influence of Syntactic and
Referential Information on Word Predictability" (unpub-lished Ph.D. dissertation, The University of Wisconsin,1964).
5Ruth Anne Clark, Frederick Williams, and Percy H.
Tannenbaum, "Effects of Shared Referential Experience UponEncoder-Decoder Agreement," Language and Speech, 8 (1965),253-262.
142
1Weaver and Bickley.
To determine whether differences existed in the pre-
dictability of lexical and structural deletions, Weaver
applied the cloze procedure. Weaver and Kingston3
investi-
2
gated the relationship of cloze tests to a number of se-
lected measures of language ability.
Comparative differences between oral and written4
samples of prose were studied by DeVito.
Several studies have utilized the cloze procedure
to investigate language characteristics of educable mentally
retarded children. Brooks5hypothesized that non-intellec-
tual factors, such as personality variables, have a signifi-
1Wendell W. Weaver and A. C. Bickley, "Some Differ-
ences in Encoding and Decoding Messages," Journal of theReading Specialist, 7 (1967), 18-25.
2Wendell William Weaver, "An Examination of SomeDifferences in Oral and Written Language Using the ClozeProcedure" (unpublished Ed.D. dissertation, University ofGeorgia, 1961).
3Wendell W. Weaver and Albert J. Kingston, "A Fac-tor Analysis of the Cloze Procedure and Other Measures ofReading and Language Ability," Journal of Communication,13 (1963), 252-261.
Joseph A. DeVito, "Comprehension Factors in Oraland Written Discourse of Skilled Communicators," SpeechMonographs. 32 (1965), 124-128.
5Sadye Tune Brooks, "Effects of Locus of Controland Anxiety on the Ability of Mentally Retarded Childrento Use Context Clues in Reading" (unpublished Ph.D. dis-sertation, George Peabody College for Teachers, 1962).
143
cant effect on the reading behavior of mentally retarded
children.
The grammatical ability of these children was1
studied by Semmel, Babbitt, and Bennett, while their oral2
language proficiency was assessed by Goodstein. Ramanaus-
kas assessed the sensitivity of educable mentally retarded
children to contextual constraints beyond a sentence.
The cloze procedure has also found application in4
studies of listening comprehension. Carver et al. inves-
tigated sources of individual differences in listening
comprehension by means of speech samples and cloze testing,
'Melvyn I. Semmel, Loren S. Babbitt, and Stanley W.Bennett, "Performance of EMR and Nonretarded Children on aModified Cloze Task," American Journal of Mental Deficiency,74 (1970), 681-688.
2Henry Arthur Goodstein, "The Performance of Edu-
cable Mentally Handicapped and Average-IQ Children on TwoModified Cloze Tasks for Oral Language" (unpublished Ph.D.dissertation, The University of Connecticut, 1969).
3Sigita Ona Ramanauskas, "The Effect of ContextualConstraints Beyond a Sentence In Cloze Responses of Chil-dren in Special Classes for the Educable Mentally Retarded"(unpublished Ph.D. dissertation, The University of Connec-ticut, 1971).
4Ronald P. Carver, et al., "Factor Analysis of the
Ability to Comprehend Time-Compressed Speech" (paper pre-sented at the Annual Meeting of the American EducationalResearch Association, New York, N. Y., February 4-7, 1971.
144.
while Williams1applied the semantic differential and the
cloze procedure to a study of listening comprehension
when tape-recorded materials were used.
1Frederick Dowell Williams, "An Experimental Appli-
cation of the Semantic Differential and 'Cloze° Procedureas Measurement Techniques in Listening Comprehension" (un-published Ph.D. dissertation, Uriversity of Southern Cali-fornia, 1962).
145
BROOKS, SADYE TUNE. "EFFECTS OF LOCUS OF CONTROL AND ANXIETYON THE ABILITY OF MENTALLY RETARDED CHILDREN TO USE CON-TEXT CLUES IN READING." UNPUBLISHED PH.D. DISSERTATION,GEORGE PEABODY COLLEGE FOR TEACHERS, 1962.
Purpose: The primary purpose of this study was to determinethe effects of locus of control and anxiety-as-drive on theability of mentally retarded children to use context clues inreading. On the assumption that these variables were some-what different attributes of personality in the mentally re-tarded population, it was reasoned that combined effects ofthe variables would eventuate in specific predictable be-haviors in 'eading.
Principles, Concepts, or Procedures: Sixty educable mentallyretarded children of both sexes were selected from the elemen-tary grades of a State residential school for dependent chil-dren to serve as Ss. Four groups were equated on IQ, locus ofcontrol and manifest anxiety using scores from the PeabodyPicture Vocabulary Test, the Bialer-Cromwell Children's Locusof Control Scale, and the Children's Manifest Anxiety Scale.The CA range was 9-2 to 1.5-0 with a mean of 12-8; IQ, 55 to 85,with a mean of 73.45; and, MA 6-j to 11-4, with a mean of 8.6.The mean locus of control score for .411 Ss was 13.40 and themean anxiety score was 25.03. Four 'ubgroups of 15 Ss eachwere studied: e.g., internal locus of control with high anxiety,internal locus of control with low anxiety, external locus ofcontrol with high anxiety, and, external locus of control withlow anxiety.
All subjects were individually administered the followingtreatments: (a) A reading pretest was administered to estab-lish each S's basal (easy) and frustration (difficult) readinggrade levels. (b) A sequence of three cloze procedure passages(context clue tests) equivalent to the S's basal reading gradelevel was administered. This sequence was accompanied by in-creasing monetary rewards and was defined as a success experi-ence. (c) A sequence of three cloze procedure passages equiva-lent to the S's frustration reading grade level was adminis-tered. This sequence was accompanied by decreasing uonetaryrewards and was defined as a partial failure experience.
Conclusions: Analysis of varian.e was used to analyze clozeprocedure scores. A correct response was defined as havingsupplied the correct word in a reading passage from which every
3.46
fifth word had been replaced by a blank space. As predicted,the total group deluonstrated a significant increment in per-formance on Sequence I under conditions of easy tasks and in-creasing monetary reward. Contrary to predictions, on SequenceII, under conditions of difficult tasks and decreasing monetaryrewards the total group demonstrated a significant incrementin performance. The magnitude of the increment on Sequence IIwas even greater than the increment on Sequence I. An analysisof gain scores of the four subgroups revealed an interactioneffect between locus of control and anxiety. The gain scoresof the internal locus of control with high anxiety Ss were sig-nificantly lower than the gain scores of all other groups. Thecombined effects of internal locus of control and low anxietycontributed to the best performance. Within the internal locusof control group, anxiety apparently played a significant role,and within the high anxiety group, locus of control played asignificant role. However, within the external locus of con-trol group, anxiety did not influence results significantly,and within the low anxiety group, locus of control did not con-tribute to a significant degree. Contrary to predictions, theanalysis of shift scores .rrom easy sequence to difficult se-quence (Task 3-Task 4) revealed no significant differences be-tween groups. However, the shift scores within each subgrouprevealed a significant decrement in performance in shiftingfrom easy to difficult tasks with the exception of the internallocus of control with high anxiety group which showed a nonsig-nificant decrement in performance.
The results of the study supported the primary thesis ofthe study, i.e., nonintellectual factors, such as personalityvariables, do play a significant role in the reading behaviorof educable mentally retarded children. The data suggestedthat a factor analytic approach to identifying group factorsthat influence reading behavior seemed warranted.1
'Dissertation Abstracts (Ann Arbor, Michigan: Univer-sity Microfilms, 1962), Vol. 23, No. 6, pp. 2003-2004. (Or-der No. 62-5667, 141 pages).
147
CARVER, RONALD P. ET AL. "FACTOR ANALYSIS OF THE ABILITY TOCOMPREHEND TIME-COMPRESSED SPEECH." PAPER PRESENTED ATTHE ANNUAL MEETING OF THE AMERICAN EDUCATIONAL RESEARCHASSOCIATION, NEW YORK, N. Y., FEBRUARY 4-7, 1971.
Purpose: Sources of individual differences in the listeningcomprehension of speech presented at different rates were in-vestigated by factor analyzing 11 variables, including measuresof comprehension and field-independency.
Principles, Concepts, or Procedures: Twenty-two differenttests, two for each variable, were administered to 50 collegestudents.
Conclusions: It was concluded that (a) the cloze type oftest includes a large component of variance unrelated to com-prehension and quite specific to the technique itself, and(b) the comprehension of highly speeded speech probably in-volves a perceptual ability to be field-independent in additionto a separate ability involved in comprehending speech at nor-mal rates.'
1Research in Education (Washington, D.C.: United States
Government Printing Office, 1971), Vol. 6, No. 4, p. 163.(ED 045 702, 18 pages).
148
CLARK, RUTH ANNE. "THE INFLUENCE OF SYNTACTIC AND REFEREN-TIAL INFORMATION ON WORD PREDICTABILITY." UNPUBLISHEDPH.D. DISSERTATION, THE UNIVERSITY OF WISCONSIN, 1964.
Purpose: The primary purpose of the present study was to in-vestigate the effects of syntactic and referential informationupon the predictability of semantic and function words indescriptive and interpretative messages. Predictions of re-lationships among the independent variables were summarized inthe following two hypotheses:
Hypothesis I: Syntactic information enhances word pre-dictability, and does so to a greater extent for functionwords than for semantic words.
Hypothesis II: Referential information enhances word pre-dictability, with greater effect on semantic words than onfunction words, and to a greater extent in descriptive thanin interpretative messages.
Principles, Concepts, or Procedures: The four independentvariables -- syntactic and referential information and message andword type--were related in a 2x2x2x2 analysis of variance model.Eight groups of 20 Ss (college students) attempted to replacemissing words of both types. Each group reconstructed messagesin one experimental condition, the experimental conditions be-ing defined by all possible combinations of syntactic and re-ferential information and message type. In the primary analy-sis, measures of word predictability were verbatim replacementof the original word (V), correct form class replacements (FC),and the ratio of V/FC. Two additional analyses were performed:(1) the influence of the independent variables upon entropy ordispersion of decoders' responses, necessarily using wordsrather than Ss as replicates, was determined; (2) the effectsof syntactic and referential information and message type on V,PC, and 1622 were tested for words of each grammatical class.
Conclusions: Hypothesis I: Providing syntactic informationsignificantly increased V and FC, in addition to bringing de-coders' responses into closer agreement. Since the rise wasgreater for FC than V, V/FC decreased. V scores were signifi-cantly influenced, however, only in the case of function words.
Hypothesis II: Referential information caused a signifi-cant increase in V and V/FC and lessened dispersion among
149
decoders' responses. Since referential information had noeffect on FC, V/FC increased when the message stimults wassupplied. Referential information did increase the predicta-bility of semantic words more than function words, but didnot have a differential effect according ty message type.
Major implications of the present research were relatedto both theory and methodology relevant to investigation ofsemantic and syntactic constraints in decoding behavior.1
0
1Dissertation Abstracts (Ann Arbor, Michigan: Univer-sity Microfilms, 1964), Vol. 25, No 6, p. 3738. (Order No.64-13,863, 147 pages).
150
CLARK, RUTH ANNE; WILLIAMS, FREDERICK; AND TANNENBAUM, PERCY H."EFFECTS OF SHARED REFERENTIAL EXPERIENCE UPON ENCODER-DECCIDER AGREEMENT." LANGUAGE AND SPEECH, 8 (1965), 253-262.
Purpose: "The effects of referential experience upon encoder-decoder agreement were studied. . . ." (253)
Principles, Concepts, or Procedures: On-e hundred sixty under-graduate students were administered passages of approximately180 words in which every fifth word had been deleted. Subjectswere not told if the encoders had given descriptions or inter-pretations of the five paintings used in the study, and weresimply instructed to replace the words which they thought theencoder had used. Responses were scored on the basis of exact-word and form class replacements.
Conclusions:
1. ".availableencoder .
the decoder group having the paintings. . fihowee a higher agreement with the. . than the group without the paintings
" (256-257)
2. ". there was no significant difference betweenthese two conditions with respect to grammatical re-placement. . ." (257)
3. ". . the expected difference on the effect ofshared referential information as between descriptiveand interpretative messages failed to materialize."(257).
151
COHEN, SANDRA RUTH. "REDUNDANCY IN THE WRITTEN LANGUAGE OFTHE DEAF: PREDICTABILITY OF DEAF AND HEARING STORYPARAPHRASES." UNPUBLISHED PH.D. DISSERTATION, COLUMBIAUNIVERSITY, 1964.1
Purposes: Language functioning in deaf children differsularkedlv from that in hearing children. One aspect of languagein the deal Lhat 11PS not been studied previously is the extentto which its redundancy differs truw that of the language ofthe hearing.
Redundancy in language facilitates correct reception ofa message. In the deaf construct messages with less redun-dancy than is generally found in the language of their hearingpeers, they thereby decrease the likelihood that theirmessages will be correctly received. Alternatively, the deafmay share verbal habits with one another that make theirmessages more predictable to each other than they are to thehearing.
The present study used the cloze technique to study thecomparative redundancy of story paraphrases written by deafand hearing children and the ability of deaf and hearing groupsto use this redundancy.
Princi les Conce ts, or Procedures: The experimental groupswere composed of profoundly deaf and 46 hearing childrenmatched for reading ability. At the first of two testingsessions each subject read a brief story and then rewrote thatstory in his own words. Following that session, words wereperiodically deleted from each child's paraphrase and from theoriginal passages. At a second session, each subject restoreddeleted words to one original passage, one deaf paraphrase,and one hearing paraphrase, each involving a different and un-familiar story.
Ninety-two additional hearing children provided informa-tion about the redundancy of the original passages. On sepa-rate occasions, 175 hearing graduate students and 43 addition-al deaf children reconstructed the deleted materials.
1See also Sandra R. Cohen, "Predictability for Deaf andHearing Paraphrases," Appendix.
152
Information about the redundancy of materials was sup-plied by the restorations made by the experimental groups andby the additional deaf and hearing subjects. Responses werescored on three criteria: verbatim (identical to deletedword), form class (same part of speech as deleted word), andsense (same meaning as deleted word). Additional analyseswere carried out to locate the source of observed differencesbetween the predictability of deaf and hearing paraphrases.
Conclusions:1) Hearing subjects found deaf stories significantly lesspredictable than original or hearing stories.2) Deaf subjects found different types of material equallypredictable.3) Hearing subjects made significantly more correct re-
storations to hearing and original stories than did deaf sub-jects, but on deaf stories there was no difference between thegroups.
4) Even when the form class of a deleted word had beencorrectly identified, hearing subjects made significantly fewercorrect restorations to deaf than to hearing or originalstories.
5) Adult subjects found deaf stories significantly lesspredictable than hearing stories which were, in turn, signifi-cantly less predictable than original stories.
6) Average sentence length and total length of the para-phrases failed to differentiate between the groups.
7) Both deaf and hearing subjects had a slight tendencyto select the more redundant words in the original story foruse in their paraphrases of it.8) Deaf paraphrases contained a higher proportion of un-
grammatical sentences than did hearing paraphrases.
The difficulty of deaf paraphrases probably resulted fromsuch factors as idiosyncratic usage, random errors, and deaf-isms. It was hypothesized that differences between deaf andhearing performance were due to the fact that the hearing hada better grasp of English than the deaf and therefore showedgreater differentiation between good and poor language samples.
The failure of the deaf to learn the customary structureof English resulted in difficulty in receiving messages fromhearing individuals and in constructing messages which couldbe correctly received by the hearing. The problem of communi-
153
cation between deaf and hearing individuals seems especiallyserious in view of the fact that the deaf subjects in thisstudy were selected for their ability to perform well on ver-bal tasks.1-
1Dissertat on Abstracts (Ann Arbor, Michigan: Univer-sity Microfilms, 196 , Vol. 26, No. 8, p. 4792. (Order No.65-13,929, 127 pages).
154
COLEMAN, E. B., AND BLUMENFELD, J. P. "CLOZE SCORES OF NOMINAL-IZATIONS AND THEIR GRAMMATICAL TRANSFORMATIONS USINGACTIVE VERTS." PSYCHOLOGICAL REPORTS, 13 (1963), 651-654.
Purposes Nominalized sentences were compared for comprehensi-bility to their grammatical transformations using active verbs--"John's description of it ." versus "John described it."
princiceptsor Procedures: Subjects were 100 collegefreshmen each of whom was given a set of sentences of which halfwere the active-verb version and half were the nominalized ver-sion. Every sentence in the total design was written in thesetwo styles. For each set (nominalized and active verb), fivecloze tests were made by deleting every fifth word. "Then therewere 10 different cloze tests for each sentence, . . . Sswere divided into 10 groups of 10 each so that each cloze testwas filled in by 10 Ss." (652) Synonyms were not accepted.
Conclusions: "The word patterns of the passages written in ac-tive verbs are more predictable than those written in nominali-zations." (653)
155
DEVITO, JOSEPH A. "COMPREHENSION FACTORS IN ORAL AND WRITTENDISCOURSE OF SKILLED COMMUNICATORS." SPEECH MONOGRAPHS,32 (1965), 124-128.
Purpose: The purpose of the study was
to compare written and oral samples of thework of skilled comunicators for (1) over-allcomprehensibility as measured by cloze procedureand (2) significant differences in selected ele-ments, supposedly related to ease of comprehen-sion. (124)
Principles, Concepts, or Procedures: Ten university professorssubmitted written passages they considered the best examples oftheir writing styles. These were divided into three hundred-word segments from which three segments were drawn at random.
Oral discourse samples were obtained by asking generalquestions based on the submitted articles, recording the oralresponses, and transcriting the content.
One oral and one written sample from each of the ten pro-fessors was drawn at random. Every fifth word was deleted.Subjects were 85 students ranging from sixteen to eighteenyears of age. Each subject completed two oral and two writtenpassages and all from different sources.
Conclusions: Among the conclusions drawn by the author is:
Oral and written discourse, according to an appli-cation of cloze procedure, did not differ signifi-cantly in each of comprehension. Since the twosets of samples did not differ significantly onmany of the specific criteria, this result is puzzl-ing. One possible explanation is that the clozeprocedure is not a sensitive device when the samplesare at a relatively advanced level and the respon-dents are undergraduates. (126)
156
FILLENBAUM, SAMUEL, AND JONES, LYLE V. "AN APPLICATION OF'CLOZE' TECHNIQUE TO THE STUDY OF APHASIC SPEECH." JOUR-NAL OF ABNORMAL AND SOCIAL PSYCHOLOGY, 65 (1962), 183-189.
Purpose: The speech of aphasics was studied by means of thecloze procedure.
Principles, Concepts, or Procedures: Samples of approximately250 words were drawn from the free speech responses of nineaphasics and three normal subjects to selected cards of theThematic Apperception Test. Three sets of booklets were con-structed, each comprising, first, a normal subject's samplefollowed by three aphasic samples. Every-fifth-word dele-tions were used.
Subjects were college summer school students who, undertimed conditions, were required to complete at least three-quarters of the blanks on each page of their booklets. Foranalysis purposes, the deleted words were classified accord-ing to form class..
Conclusions:
The aphasic speech samples typically were less ac-curately completed than normal control samples, .;and given that the form class of an item had beencorrectly identified raters were still less likely tohit upon the exact missing word for the aphasic textsthan for the normal control texts. (189)
In general the cloze procedure was found to provide an in-dex of divergence from normal of a speaker's language.
157
GOODSTEIN, HENRY ARTHUR. "THE PERFORMANCE OF EDUCABLE MENTAL-LY HANDICAPPED AND AVERAGE-IQ CHILDREN ON TWO MODIFIEDCLOZE TASKS FOR ORAL LANGUAGE." UNPUBLISHED PH.D. DIS-SERTATION, THE UNIVERSITY OF CONNECTICUT, 1969.
Purpose: It was hypothesized that: (a) EMH children are sig-nificantly poorer than average-IQ children in cloze performance:(b) the recognition cloze task would be significantly easierthan the production cloze task; and (c) a significant interac-tion exists between IQ and task, with recognition exceedingproduction for the low-IQ sample to a greater extent than re-cognition exceeds production for the average-IQ children.
Principles, Concepts, or Procedures: A sample of public schooleducable mentally handicapped and a sample of average-IQ chil-dren were matched for mental age (approximately 9 years-5 months)and reading age (approximately 8 years-6 months). The sampleswere administered two modifications of cloze procedure that re-flect the two important processes in oral language, productionand recognition. The production task requires the child 'tocomplete appropriately a four-word sentence, in which one wordhas been randomly deleted. The recognition task differs inthat it requires the child to choose the appropriate responsefrom among four words to complete the sentence.
goclusions: The null forms of the first two hypotheses wererejected5 < .01). The hypothesized interaction was not sig-nificant (p > .01). No qualitative differences were found inthe performance of the two samples.
The results were interpreted as reflecting inefficienciesin the organization and retrieval of linguistic stimuli by low-IQ children. Implications for potential language trainingprograms utilizing cloze procedure were discussed.1
1Dissertation Abstracts International A: The HumanitiesAnd Social Sciences (Ann Arbor, Michigan: University Micro-films, 1970), Vol. 30, No. 8, p. 3320-A. (Order No. 70-126:),53 pages).
158
HONIGFELD, GILBERT. "EFFECT OF AN HALLUCINOGENIC AGENT ON VER-BAL BEHAVIOR." PSYCHOLOGICAL REPORTS, 13 (1963), 383-385.
Purpose: "This report is a first attempt to apply Cloze analy-sis to the study of the effect of drugs on 'understandability.'"(383)
Principles, Concepts, or Procedures: A drug was administeredto a 22 year old medical student whose spontaneous speechsamples were mutilated with every-fifth-word deletions whentranscripts were made. Twenty-two subjects enrolled in anevening psychology class were required to restore the deletedwords.
Conclusions:
. . . our S's speech appears to be somewhat lessunderstandable than a Life magazine article, butmore easily understood than excerpts from the worksof Francis Bacon or Benjamin Franklin. (384-385)
The author concludes that the cloze procedure is useful inevaluating drug effects.
159
HONIGFELD, GILBERT. "THE ABILITY OF SCHIZOPHRENICS TO UNDER-STAND NORMAL, PSYCHOTIC, AND PSEUDO-PSYCHOTIC SPEECH."DISEASES OF THE NERVOUS SYSTEM, 24 (1963), 692-694.
Purpose: The h7pothesis that the speech of schizophrenics ismore easily understood by other schizophrenics than other in-dividuals was tested.
Principles, Concepts. or Procedures: By means of the clozeprocedure, the understandability of six passages was deter-mined by a group of 32 college students. Three passages ofequal understandability were selected: a newspaper excerpt,an excerpt from an interview with a schizophrenic, and aspout. -eons speech excerpt from a normal volunteer under theinfluence of an hallucinatory drug. These passages were thenadministered to sixty schizophrenic patients.
Conclusions: "8 no evidence was found to support theclinical hunch that schizophrenics find it relatively easierto understand the speech of other schizophrenics." (693)
I.
160
HONIGFELD, GILBERT; PLATZ, ARTHUR; AND GILLIS, RODERIC D."VERBAL STYLE AND PERSONALITY: AUTHORITARIANISM."JOURNAL OF COMMUNICATION, 14 (1964), 215-218.
Purpose: This study sought to determine if the cautiousapproach of authoritarian persons finding themselves in novelproblem situations would manifest itself in the style adoptedby subjects in response to a cloze test.
Principles, Concepts, or Procedures: After authoritarianismscores were obtained for each of the 27 evening college stu-dents who were subjects, a series of six cloze tests was ad-ministered. Every-fifth-word deletions were used. The numberof-uncompleted items was computed as an estimate of cautious-ness of style.
Conclusions: Subjects rated as highly authoritarian left asignificantly greater number of blanks than did those receiv-ing a low rating.
161
LOUTHAN, VINCENT. "SOME SYSTEMATIC GRAMMATICAL DELETIONS ANDTHEIR EFFECTS ON READING COMPREHENSION." ENGLISH JOURNAL,54 (1965), 295-299.
Purpose: This study was undertaken to determine the effects ofdeleting certain grammatical classes of words.
Principles Concepts, or Procedures: Twenty-four prose passagesof 500 to 600 words were graded at a 5.0 readability indexaccording to the Yoakam formula. Seven cloze forms were pre-pared, utilizing ten percent deletions according to seven gram-matical classes. An eighth form was left intact as a control.All forms were followed by twelve comprehension questions:six factual and six inferential.
Each of the 236 seventh-grade subjects were assigned acloze form at random and completed six exercises in it. The ex-periment lasted three weeks.
4
Conclusions: Among the findings are the following:
1. The control group was significantly superior in some in-stances to groups using cloze materials.
2. Noun deletions, specific verb deletions, and modifierdeletions caused a marked loss in comprehension.
3. Deletions of prepositions, conjunctions, and pronounsubstantives do not cause a loss of comprehension.
4. The deletion of noun determiners produced a significantgain in comprehension.
162
MANIS, MELVIN, AND DAWES, ROBYN M. "CLOZE SCORES AS A FUNCTIONOF ATTITUDE." PSYCHOLOGICAL REPORTS. 9 (1961). 79-84.
Purposes The authors
1. tested the assumption that
recipients, who disagree with the contents ofa controversial statement, will be relatively in-sensitive to the redundancy of the communicator'swords. (79)
2. investigated the likelihood that a subject's viewswould influence his responses in situations where ambiguitywas increased by the omission of context clues.
Principles,_ Concepts, or Procedures:
Two 400-word passages, one favoring capital punish-ment and one opposing it, were chosen. . . Forthe "high-deletion" group (N = 41) every fourthword was removed; for the "low-deletion" group(N = 31) every eighth word was removed. (80)
Subjects were college students who, before replacing themissing words, rated a phrase concerning capital punishmenton six scales of the semantic differential.
Conclusions:
1. Subjects showed a general tendency to perform moreeffectively on the passage that echoed their own views.
2. Connective deletions- -verb auxiliaries, conjunctions,pronouns, and articles--were more highly related to attitudethan were content deletions--nouns, verbs, adverbs.
3. , these data support the proposition thatcognitive performance may be adversely affectedwhen the individual is presented with materialsthat contradict his beliefs. (84)
163
MARSHALL, WILLIAM JAMES ALOYSIUS. "INVESTIGATING THE EFFECTOF CONTEXTUAL CONSTRAINT ON DEAF AND HEARING CHILDREN ATTHE FOURTH GRADE READING LEVEL, USING THE CLOZE PROCE-DURE." UNPUBLISHED ED.D. DISSERTATION, UNIVERSITY OFILLINOIS AT URBANACHAMPAIGN, 1970.1
Purpose: An area of research interest having received minimalattention in the past is the manner in which redundancy affectsthe deaf child's perception of the syntactic and semantic cuesof written language.
Using cloze procedure to investigate linguistic compe-tence, the study examined how, through manipulation of theoverall language structure, a deaf child's psychological andlinguistic perception of contextual cues affected his abilityto make correct predictions on stimulus materials that hadbeen selectively mutilated with respect to their containedgrammatical construction.
Principles. Concepts, or Procedures: The experimental groupconsisted of 9 male and 15 female residential school deafchildren with a mean grade reading score of 4.4 and 4.5, re-spectively on the. Stanford Achievement Test. Their mean per-formance IQ's on the non-verbal Wechsler. Intelligence testswere 107.7 and 100.8 for the males and females, with the meanage being 16 years, 1 month and 16 years, 5 months, respective-ly. The comparison group of 9 male and 15 female fourth andfifth grade hearing students had a mean grade reading achieve-ment score of 4.3 and 4.4 respectively. Their mean IQ's onthe non-verbal California Short-Form Test of Mental Maturitywere 111.7 and 103.6 for the males and females with the meanage being 9 years, 6 months and 9 years, 5, months, respectively.
One 809 word passage was chosen as the stimulus materialfrom a graded reading text with a reported difficulty level of3.6 according to the Spathe Readability Formula. This passagewas then subjected to a total of 50 deletions occurring withingrammatical constructions specifically identifiable by the sur-rounding content. This passage thus manipulated comprised thefirst of three levels of contextual constraint, namely, theconnected discourse level.
1See also William A. Marshall, "Investigating the Effect
at Fourth Grade Reading Level Using the Cloze Procedure,"Appendix.
164
While maintaining the same deletions, the discrete sen-tence level of contextual constraint was formed. These sen-tences were presented in a randomized order so as to precludethe possibility of any connected ideas being gleaned from them.Lastly, with these same deleted items still being used, thefragment level of contextual constraint was constructed withonly a minimal number of contextual cues being retained so asnot to destroy the earmarks of any given grammatical construc-tion.
The criterion scores used for estimating the grammaticalcompetence of both groups of subjects were the Close scores ofcorrect Verbatim, correct Form Class and correct Verbatim givenForm Class response. The statistical design used to analyzeeach of these scores was a factorial 3 x 2 x 2 (Contextual Con-straint by Sex by Physical Status of Hearing) analysis of vari-ance. When a significant F occurred for the variable of Con-textual Constraint, the a posteriori "Tukey (a) Procedure"test of multiple comparisons was also used.
Conclusions: Three hypotheses were tested in the investigation.
Contextual Constraint. The hypothesis of no differencesamong the connected discourse, discrete sentence and fragmentlevels of contextual constraint was rejected at the one percent level for the Verbatim and Form Class cloze criterionscores only. Although there were differences among the Ver-batim given Form Class means, these differences were smallenough to be attributable to chance.
In testing further for differences between pairs of meansamong the three levels of contextual constraint with the"Tukey (a) Procedure," significant differences were found onlybetween the higher scores of the connected discourse level andthe lower scores at the fragment level for both the Verbatimand Form Class responses. Differences of a similar naturewere found for the Verbatim given Form Class responses, butthese did not reach significance.
Sex. The tenability of this exploratory hypothesis of nodifferences had to be accepted for all of the criterion scoreson all levels of contextual constraint and for all subjects.
Physical Status of Hearing. The hypothesis of no differ-ences between the mean criterion scores of the experimentaland comparison groups across all levels of context and sex wasrejected for the Verbatim and Forms Class scores only. It was
165
again found, however, that the Verbatim given Form Classscores failed to produce significant differences.
The results indicated that the relative redundancy of thegrammatical and semantic cues of the stimulus materials greatlyaffected those areas of grammatical competence as assessed bythe cloze scores of Verbatim and Form Class responses. It isquite difficult, however, to find a convincing interpretationfor the negative phenomenon produced by the Verbatim given FormClass scores. One possible explanation was that the Verbatimcompletions did not improve at any arithmetical rate propor-tionately faster than the Form Class completions. Another ex-planation was suggested by intimating the presence of a con-founding factor such as the relative redundancy of both thesyntactic and semantic cues inherent in the stimulus materials.Further controlled experimentation is-needed to test for theeffect of the relative redundancy of syntactic and semanticcues within various levels of contextual constraint.1
1Dissertation Abstracts International A: The Humanities
and Social Sciences Ann Arbor, Michigan: University Micro-films, 1970), Vol. 31, No. 5, p. 2218-A. .(Order No. 70-21,014,104 pages).
166
MOORES, DONALD FREDERICK. "APPLICATIONS OF 'CLOZE' PROCEDURESTO THE ASSESSMENT OF PSYCHOLINGUISTIC ABILITIES OF THEDEAF." UNPUBLISHED PH.D. DISSERTATION, UNIVERSITY OFILLINOIS, 1967.
Purpose: Traditional methods employed in the measurement ofgrammatical and semantic abilities of the deaf, typicallyanalysis of written language samples and the use of readingscores on standardized achievement tests, have been shown tosuffer from fundamental inadequacies. The study was designedto investigate the ability of "cloze" procedures to identifyand isolate morphological syntactic, and semantic differencesbetween a deaf gro, p and a hearing group matched on the basisof reading scores on a standardized achievement test. Basedon, the contention that standardized tests provide spuriouslyinflated estimates of the grammatical and semantic abilitiesof the deaf, it was hypothesized 'hat hearing students,matched to deaf subjects on the basis of reading scores, actu-ally possess superior psycholinguistic abilities in all areasand that their superiority would be exhibited through variousapplications of "cloze" techniques.
Principles, Concepts, or Procedures: The experimental groupconsisted of 37 residential school deaf children with a meangrade reading score of 4.77 on the Stanford Achievement Testand an average age of 16 years nine months. The control group,37 fourth and fifth grade hearing students, had a mean gradereading achievement score of 4.84 and an average age of nineyears, 10 months. A contrast group of 24 high school hearingstudents was used for purPosE:s of comparison but their scoreswere not used in the statistical analyses.
Passages of 250 words were chosen from a fourth, sixth,and eighth grade reading book. Every fifth word was deletedand subjects were instructed to fill in the most appropriatewords. Responses were scored for (a) verbatim reproduction,(b) fort. class (grammatical) reproduction, and (c) verbatimgiven form class reproduction--the percentage of correct ver-batim responses given correct form class reproduction.
Conclusions: Verbatim. The performance of the control groupwas superior (p. < .001) and the superiority existed for eachpassage, supporting the argument that standardized readingscores over-estimate psycholinguistic abilities of the deaf.
167
Form Class. The scores of the control group were superioron this measure (p. < .01) and the superiority existed for allpassages but did not reach significance for the eighth gradetest (.10 > p. > .05). The results suggest that at least partof the inferiority of the deaf may be explained in terms ofinadequately developed morphologico-syntactic integrations.
Verbatim Given Form Class. The scores of the controlgroup on this measure were significantly higher (p. < .001)and the superiority existed for each passage. It is concludedthat in addition to the poorly developed grammatical abilitiesidentified in the form class measure, the deaf are furtherhandicapped by semantic insufficiencies in the form of redun-dant, stereotyped modes of expression and limited vocabulary.
The results indicate that "cloze" procedures possess asensitivity to morphological, syntactic, and semantic abilitieswhich standardized tests are incapable of measuring. Furtheruses of " close" procedures could identify patterns of psycho-linguistic strengths and weaknesses and have implications forrevisions in the teaching of language to the deaf. The use of"close" procedures to develop individual assessment techniquesis also suggested. The technique could supplement or replacepresent methods of language measurement, which are classifica-tory in nature, and provide the basis for educational diagnosisleading to an individually proscribed program of remediation.1
1Dissertation Abstracts Al The Humanities and SocialSciences (Ann Arbor, Michigan: University Microfilms, 1968),Vol. 787 No. 8, p. 3032-A. (Order No. 66-1813, 142 pages).
168
RAMANAUS.CAS, SIGITA ONA. "THE EFFECT OF CONTEXTUAL CONSTRAINTSBEYOND A SENTENCE ON CLOZE RESPONSES OF CHILDREN INSPECIAL CLASSES FOR THE EDUCABLE MENTALLY RETARDED."UNPUBLISHED PH.D. DISSERTATION, THE UNIVERSITY OF CONNEC-TICUT, 1971.
Purpose: The author does not state a purpose.
Principles, Concepts, or Procedures: Subjects were 58 educablementally retarded pupils reading at a grade level equivalent of2.5 or above on the Wide Range Achievement Test and attendingjunior high special classes in suburban towns. They were admin-istered two cloze tasks constructed from basal reader selectionsat the 2-2 level. One task consisted of selections containingsentences in the natural order of discourse (NAT). The othertask involved materials wherein _sentence order was modified bybeing randomly rearranged (MOD). The two conditions differed inavailability of contextual cues beyond a sentence. A LatinSquare design was used with repeated measures over type of ma-terial (NAT and MOD) and order effect controlled through coun-terbalancing.
Conclusions: An analysis of variance confirmed Co < .0005) theexpectation that significantly more correct cloze responseswould be produced in the NAT condition. Order of treatment pre-sentation was not significant but a significant (p < .02) ordinalinteraction was observed between type of material and order.More correct cloze responses were produced when NAT was pre-sented second but the relative effect of NAT and MOD was undis-turbed. It was concluded that mentally retarded pupils havingthe characteristics of the subjects of this study are not wordcallers or even sentence specific readers at least as measuredby the cloze technique using basal series materials. They areresponsive to contextual constraints beyond the sentence. Pos-sibilities ,for future study and educational implications werediscussed.'"
1Dissertation Abstracts International At The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1971), Vol. 32, No. 5, p. 2515-A. (Order No. 71-29,900, 80 pages).
169
RANKIN, EARL F., JR. "READING TEST RELIABILITY AND VALIDITY ASFUNCTION OF INTROVERSION-EXTROVERSION." JOURNAL OF DEVEL-OPMENTAL READING, 6 (1963), 106-117.
Purpose: The author conducted a series of three studies todetermine the relationship between introversion-extroversionand reading test reliability and validity.
Principles. Concepts, or Procedures: Reading comprehensionand introversion-extroversion measures were computed for eachof the 161 college students participating. Some subjects tooka pre-reading cloze test while others took a post-readingcloze test. One article was used in the study.
Conclusions: The fact that pre-cloze tests demand more per-sistence and greater organizational skill than post-cloze testsmay account for the fact that extroverts performed less wellthan introverts on the pre-cloze test. Post-cloze scores didnot differ significantly.
The overall conclusion of the three studies was that arelationship exists between introversion-extroversion and read-ing test reliability and validity.
170
SEMMEL, MELVYN I.; BABBITT, LOREN S.; AND BENNETT, STANLEY W."PERFORMANCE OF EMR AND NONRETARDED CHILDREN ON A MODI-FIED CLOZE TASK." AMERICAN JOURNAL OF MENTAL DEFICIENCY,74 (1970), 681-688.
Purpose: The authors report a study designed to investigatethe grammatical ability of educable mentally retarded children.
Principles, Concepts, or Procedures: Two groups of EMR chil-dren attending an institution and a public school were matchedwith two groups of non-retarded children on the basis of in-telligence and chronological age.
Forty-four word sentences, utilizing different word orders,were placed on individual cards. One word was omitted fromeach sentence. A subject was required to replace it orallywhile the sentence was read by an experimenter and the subjectread the card.
Conclusions: EMR children did, in fact, have difficulty withthe cloze tasks due probably to weak grammatical decodingstructures. The results indicated, however, that languagedevelopment may be more closely related to chronological agethan to mental development.
171
WEAVER, WENDELL WILLIAM. "AN EXAMINATION OF SOME DIFFERENCESIN ORAL AND WRITTEN LANGUAGE USING THE CLOZE PROCEDURE."UNPUBLISHED ED.D. DISSERTATION, UNIVERSITY OF GEORGIA,1961.
Purpose: The problem of this study was to determine if differ-ences existed in the ability of subjects to predict words omittedfrom language materials intended for oral and written presenta-tion. Tests designed in this fashion have been named "cloze"tests.
Principles,, Concepts, or Procedures: Two groups of 80 subjectseach (Juniors and Seniors at the University of Georgia) wereused in this study. The subjects were further sectioned into 16groups of 10 each and administered 8 "cloze" tests in variousorders. The last test given in each instance was the experimen-tal test. The "cloze" tests used were constructed by deletingevery tenth word from language passages intended for oral andwritten presentations, and by deleting every tenth noun or mainverb in other language passages intended for oral and writtenpresentation.
One group received "cloze" reading tests and cloze auraltests in a multiple-pass situation. That is, they were allowedas many repetitions of the listening materials as they desired.The other group received "cloze" reading tests and "cloze" auraltests in a single-pass situation. They were allowed to hear theaural materials only once.
A 2 X 2 X 4 analysis of variance design was used for thestatistical analysis of the data. The multiple-pass and thesingle-pass situations were analyzed separately.
Conclusions: Although the multiple-pass aural situation in-creased the predictability of the missing words, the relation-ships within the two situations were essentially the same. Thecentral, finding is that structural meaning, as shown by the pre-dictability of "any-word" cloze, is conveyed significantlybetter by silent reading while lexical meaning, as shown by thepredictability of nouns and main verbs, is conveyed equally wellby listening or by silent reading. Whatever adve4itage allowedthe greater predictability of structural meaning in the silentreading situation is in some manner lost in the listening situa-tion.
172
There are indications that this loss !..3 due to the rela-tively small context necessary for supplying predictive infor-mation in the lexical case in most situations, compared to therelatively larger context needed to supply all the structuralmeaning available. The fact that large contextual elementscan be analyzed only with great difficulty in the listeningsituation may drive the organism to use the strategy of depend-ing on the small context in both structural and lexical situa-tions, thereby losiog the structural meaning which dependsupon large context.'
1Dissertation Abstracts (Ann Arbor, Michigan: UniversityMicrofilms, 1962), Vol. 22, No. 8, p. 2702. (Order No. 61-6587, 147 pages).
173
WEAVER, WENDELL W., AND BICKLEY, A. C. "SOME DIFFERENCES INENCODING AND DECODING MESSAGES." JOURNAL OF THE READINGSPECIALIST, 7 (1967), 18-25.
Purpose:
The purpose of this study is to determine the abil-ity of subjects, to predict the omissions from anatural language text which they had previouslyproduced themselves as contrasted with the abilityof other subjects to predict omissions from thesesame texts which they had read at the time of pro-duction. (18)
Principles, Concepts, or Procedures: Subjects were 64 random-ly selected college sophmores who ware randomly assigned toone of two experimental and two control groups. One experi-mental group produced written stories in response to twoThematic Apperception Test cards and the second group readthe stories. Two days later both groups were administereddoze tests which Utilized every-seventh-word deletions. Onecontrol group also produced stories but did not refer to themagain. Subjects read stories written by the experimentalgroup and were administered a doze test two days later. Thesecond control group was given stories to read followed twodays later by doze tests produced by the experimental group'sstories which had not before been seea.
Conclusions: Predictability of missing elements in passages-is increased when subjects are sources of the messages aswell as destinations.
l'(4
WEAVER, WENDELL W., AND KINGSTON, ALBERT J. "A FACTOR ANALYSISOF THE CLOZE PROCEDURE AND OTHER MEASURES OF READING ANDLANGUAGE ABILITY." JOURNAL OF COMMUNICATION, 13 (1963),252-261.
Purpose: "This study explores a number of measures of languageability and attempts to isolate a set of fundamental components."(252)
Principles, Concepts, or Procedures: A battery of eighteentests was administered to 160 college juniors. The battery com-prised:
1. The Davis Reading Test.2. The MLAT Number Learning Subtest.3. The MLAT Phonetic Script Subtest.4. The KLAT Spelling Clues Subtest.5. The MLAT Words in Sentences Subtest.6. The MLAT Paired Associates Subtest.7. STEP Listening Test.8. The Ohio State Psychological Examination-Vocabulary.9. The Ohio State Psychological Examination-Word Relations.10. The Ohio state Psychological Examination-Reading Com-
prehension.11. Cloze I - A Structural Deletion of Essay Material (Sub-
ject reads).12. Cloze II - A Lexical Deletion of Essay Material (Sub-
ject reads).13. Cloze III - A Structural Deletion of a Speech (Subject
reads).14. Cloze IV - A Lexical Deletion of a Speech. (Subject
reads).15. Cloze V - A Structural Deletion of Essay Material (Sub-
ject listens).16. Close VI - A Lexical Deletion of Essay Material (Sub-
ject listens).17. Cloze VII - A Structural Deletion of a Speech (Subject
listens .
18. Cloze VIII - A Lexical Deletion of a Speech (Subjectlistens).
Each cloze test contained 40 deletions and had a 30 minutetime limit. Exact word replacements were required.
175
Conclusions: Among the conclusions drawn from the analysis was
that cloze tests of varied kinds are more re-lated to each other than to the other two factorsisolated by this analysis, verbal comprehension and"rote memory, flexible retrieval." It is also im-portant to note that there is much specific varianceconnected with the cloze procedure which is unex-plained by this analysis. (261)
176
WILLIAMS, FREDERICK DOWELL. "AN EXPERIMENTAL APPLICATION OFTHE SEMANTIC DIFFERENTIAL AND 'CLOZE' PROCEDURE AS MEA-SUREMENT TECHNIQUES IN LISTENING COMPREHENSION." UNPUB-LISHED PH.D. DISSERTATION, UNIVERSITY OF SOUTHERN CALI-FORNIA, 1962.
Purpose: The purpose of this study was to apply two communica-tion research techniques--the semantic differential and clozeprocedure--to research in listening comprehension. The semanticdifferential, a scaling technique, was used as an operationalmeasurement of attitude-toward-topic and as an exploratory mea-surement Of receptivity. Cloze procedure, used as a comprehen-sion measurement, is a technique whereby respondents attempt toreplace words that have been systematically deleted from a lan-guage sample. All previous reported research with cloze proce-dure involved its application to written materials. In thisstudy a technique was devised for applying cloze procedure totape-recorded oral materials.
Principles, Concepts, or Procedures: The problem included (1)investigating relationships among variables in comprehension oftwo language Selections, (2) comparing two comprehension cri-teria on the basis of a matched group design, (3) seeking addi-tional relationships between attitude measurements and antici-pated effects of the persuasive selection, and (4) comparingsubjects' evaluations of the two comprehension criteria.
Subjects were used in a two-group (N = 126 and 127)._Group I used a multiple-choice test as a comprehension cri-terion, whereas Group II used cloze procedure. Both groupsheard tape-recordings of two selections taken from the Secn'en-tial Tests of Educational Progress (listening). Selection Awas a persuasive speech on the value of fear. 'Selection Bwas an expository lecture on word meanings. Relative to theselections, seven measurements were obtained from subjects ineach group: pre-exposure and postexposure (selection A) ,
attitude toward "fear," receptivity and comprehension of se-lection A, receptivity and comprehension of selection B, andevaluation of the over-all test materials. Language abilityscores (American College Testing Program) were obtained for105 subjects in each group.
Relationships among the foregoing measurements wereanalyzed by use of the Pearson product-moment statistic. Co-efficients were interpreted on the basis of minimum signifi-cant r t.05 level).
177
Conclusions. In listening to the persuasive selection,shift of attitude (difference between pre-exposure and post-exposure) measurements for both groups were substantially re-lated to favorability of postexposure attitude. There wasalso a low correlation between pre- and post-exposure favor-ability of attitude measurements. Receptivity measurementsshowed a definite but small relationship with both postexpo-sure attitude and shift of attitude. Both comprehension cri-teria were substantially related to subjects' ACT scores.But in neither group was the comprehension criterion relatedto attitude nor receptivity measurements.
In listening to the expository selection both groups'comprehension measurements were slightly correlated with re-ceptivity measurements. Both comprehension measurements weresubstantially related to subjects' ACT scores.
There was a definite but small relationship between sub-jects' (Group I) scores on the two multiple-choice tests.Subjects' (Group II) cloze scores on the two selections werehighly correlated.
In a matched group design (53 pairs on the bases of sex,ACT score, age, class year), the correlation coefficient be-tween multiple-choice test scores and cloze scores on the per-suasive selection was insignificant. On the expository selec-tion the comprehension criteria showed a marked relationship.
Additional relationships between semantic differentialmeasurements and anticipated effects of the persuasive selec-tion included (1) that both group.: showed a significant shiftin a favorable direction in over-all attitude measurements(sign test) and (2) that both groups' shifts on individualattitude scales were highly related (Spearman rho).
In evaluating the test materials, Group I rated the mul-tIple- choice test as significantly (median test) more favor-.
aole on bad-Rood and hard-easy scales than Group II had ratedcloze procedure. There was no significant difference onslow-fast, worthless-worthwhile, and passive-active scales.Neither group rated the test materials in an unfavorable
Tot' the scales.'direction on an
1Dissertation Abstracts (Ann Arbor, Michigan: Univer-sity MICTIITas, 1963) Vol. 23, No. 7, pp. 2628-2629. (Or-der No. 62-601, 219 pages).
CHAPTER V
THE CLOZE PROCEDURE AS AN
INSTRUCTIONAL TECHNIQUE
The cloze procedure has been variously applied in
the area of educational methodology.
Several studies have investigated the efficacy of
using the technique to determine reading levels. Alexander2
and Guszak assessed the independent, instruction, and
frustration reading levels of intermediate grade pupils.
Kirby3c:mpared cloze test scores with oral and
silent reading scores to determine whether cloze results
are accurate determinants of the instructional reading _Level
of eleme;rtary school children.
1Henry William Alexander, "An Investigation of the
Cloze Procedure as a Measuring Device Designed to Identifythe Independent, Instruction, and Frustration ReadingLevels of Pupils in the Intermediate Grades" (an unpublishedEd.D. dissertation, University of Illinois, 1968).
2Frank J. Guszak, A Com arative Studyof the Valid-
ity of the Cloze Test and Metropolitan Achievement TestDeadina Com rehension Subtest for Makin Jud ents of In-structional Levels Austin, Texas: University of Texas,199.
3Clara Lou Laughlin Kirby, "A Comparison of ScoresObtained on Standardized Oral and Silent Reading Tests anda Cloze Test" (unpublished Ed.D. dissertation, Ball StateUniversity, 1967).
178
179
The validity of using cloze test results as deter-
minants of the reading levels of first grade children who
had been taught beginning reading by four different1 2
approaches was investigated by Kirchhoff, while Sauer in-
vestigated the hypothesis that the cloze technique is a
valid determinant of the instructional reading level of
disabled fourth grade readers.
Wiechelman3compared the functional reading levels
identified by cloze tests and by informal reading inven-4
tories using eighth grade students as subjects. Ransom
also compared cloze test scores and informal reading in-
ventory scores but studied elementary school children.
1Leo Henry Kirchhoff, "A Study Utilizing the ClozeTest Procedure to Determine Reading Levels of First GradeChildren Who Have Been Taught Beginning Reading by FourDifferent Approaches" (unpublished Ed.D. dissertation,University of Kansas, 1968).
2Freda Mae Sauel, "The Determination of Reading
Instructional Level of Disabled Fourth Grade Readers Util-izing Cloze Testing Procedure" (unpublished Ed.D. disser-tation, Oklahoma State University, 1969).
3Duane Sylvester Wiechelman, "A Comparison of
Cloze Procedure Scores and Informal Reading Inventory Re-sults for Estimating Functional Reading Levels for Studentsat Eighth Grade Level" (unpublished Ed.D. dissertation,University of Northern Colorado, 1971).
4Peggy Elaine Hitchcock Ransom, "A Study to Deter-
mine Reading Levels of Elementary School Children by ClozeTesting" (unpublished Ed.D. dissertation, Ball State Uni-versity, 1965).
180
Schoelles1
investigated the cloze procedure as a,
predictor of reading group placement.
Criterion levels of performance for instructional2
tasks were developed by Bormuth,- and Rankin3assessed the
grade level interpretation of cloze scores.
Several studies have investigated the use of the4
cloze procedure as a teaching device. Bloomer employed
the cloze procedure as a remedial reading technique for
college students. Bloomer5
also studied the effects of
non-overt reinforcement of cloze responses of students at
selected grade levels.
1Iris Storey Schoelles, "Cloze as a Predictor of
Reading Group Placement" (paper presented at the AnnualMeeting of the International Reading Association, AtlanticCity, New Jersey, April 19-23, 1971).
2Bormuth, Development of Standards of Readability:
Toward s Rational Criterion of Passage Performances, FinalReport to the Office of Education, Department of Health,Education, and Welfare, Project No. 9-0237 (Chicago: Uni-versity of Chicago, June 1971).
3Rankin, "Grade level Interpretation of ClozeReadability Scores" (paper presented at the National Read-ing conference, St. Petersburg, Florida, December 3-5, 1970.
Richard H. Bloomer, "The Cloze Procedure as aRemedial Reading Exercise," Journal of Developmental Read-ing, 5 (1962). 173-181.
5Blaomer, "The Effects cF Non-Overt ReinforcedCloze Procedure Upon Reading Comprehension," New Frontiersin College-Adult Reading, ed. by G. B. Schick and M. M.May, Yearbook of the National Reading Conference (Milwau-kee, Wisconsin: National Reading Conference, 1965, pp.
181
After determining the krowledge of grammatical
usage of achieving students of English which enabled them
to learn material more efficiently than non-achieving stu-1
dents, Blumenfeld and Miller provided instruction to the
latter students by means of the cloze procedure.2
Guice, Hunter, Smith, and Stewart utilized the
cloze procedure as an instructional device to improve the
reading comprehension of college students, while Friedman
studied its effect on the reading comprehension of foreign
college students.
The differential effects of instruction in trans-
1Jacob P. Blumenfeld and Gerald R. Miller, "Improv-
ing Reading Through Teaching Grammatical Constraints,"Elementary English, 43 (1966), 752-755.
2Billy Milton Guice, "An Experiment Combining In-struction in Comprehension With Practice in the Cloze Pro-cedure in Reading" (unpublished Ed.D. dissertation, Uni- 2,--
versity of Southern Mississippi, 1966); Charles F. Hunter,"Clozing the Gap: An Approach to Solving Reading Prollemsat San Jose City College" (paper presented at the meetingof the Western College Reading Association, Los Angeles,California, April 1-3, 1971)1 Elmer L. Smith, "Use of theCloze Procedure in Improving Comprehension of Junior CollegeReaders" (paper presented at the National. Reading Confer-ence, Atlanta, Georgia, December 4-6, 1969); Elneita Wallace
. Stewart, "A Comparative Study of the Effectiveness of Clozeand Textbook Procedures in a College Reading Program" (un--published Ed.D. dissertation, University of Houston, 1967).
1Mildred McElhinney Friedman, "The Use of the ClozeProcedure for Improving the Reading Comprehension ofForeign Students at the University of Florida" (unpublishedM.D. dissertation, The University of Florida, 1964).
182
formational grammar and the completion of cloze exercises
on the reading, writing, and listening of college freshmen1
were investigated by Martin.2
Guscott presented exercises in the form of over-
head transparencies to sixth grade students and assessed the
effects on reading achievement, and Faubion3measured the
effect of the cloze procedure on the comprehension of fourth4
grade students. Hafner and Rankin and Overholser5 applied
the cloze procedure to teaching context aids to intermediate
grade children. The effects of cloze exercises on the
1Ruby Wheeler Martin, "Transformational Grammar,
Cloze, and Performance in College Freshmen" (unpublishedEd.D. dissertation, Syracuse University, 1968).-
2Charles Edgar Guscott, "The Effect of Cloze Proce-
dure Exercises on the Improvement of Reading Achievementand of Reading Comprehension of Selected Sixth-Grade Stu-dents" (unpublished Ph.D. dissertation, The University ofAkron, 1971).
Norma Nell Faubion, "The Effect of Training in theUse of Cloze on the Ability of Fourth Grade Pupils to GainInformation From Written Discourse" (unpublished Ph.D. dis-sertation, Texas A &M University, 1971).
4Lawrence Erhardt Hafner, "An Experimental Study of
the Effect on Various Reading Achievement Scores of Teach-ing Selected Context Aids to a Group of Fifth Grade Pupils"(unpublished Ed.D. dissertation, University of Missouri,1960).
5Earl F. Rankin and Betsy M. Overholser, "Reaction
of Intermediate Grade Children to Contextual Clues," Jour-nal of Reading, 1 (1969), 50-73.
183
reading comprehension of sixth grade students was studied1
by Schneyer.
Robinson' made use of the technique with function-
ally illiterate inmates of a federal penitentiary. The po-
tential uses of the cloze procedure in reading clinics have
been described by Rankin.3
Jongsma,4
in a review of the literature related to
the cloze procedure as a teaching device, offered sugges-
tions for future research.
-IJ. Wesley Schneyer, "Us,. of the Cloze Procedurefor Improving Reading Comprehens.,n," The Reading Teacher,19 (1965), 174-179.
2Richard David Robinson, "An Investigation Intothe Use of the Cloze Procedure With a Group of FunctionallyIlliterate Adults" (unpublished Ed.D. dissertation, Uni-versity of Georgia, 1971).
3Rankin, "Uses of the Cloze Procedure in the Read-ing Clinic," g3s21gReadinilaninSocietv, ed..by J. AllenFigurel, Proceedings of the International Readihg Associa-tion (New York: Scholastic Magazines, 1959), pp. 228-232.
kEugene Jongsma, The Cloze Procedure as a TeachingRogue (Newark, Delaware: International Reading Associa-tion, 1971).
184
ALEXANDER, HENRY WILLIAM. ."AN INVESTIGATION OF THE CLOZEPROCEDURE AS A MEASURING DEVICE DESIGNED TO IDENTIFYTHE INDEPENDENT, INSTRUCTION, AND FRUSTRATION READINGLEVELS OF PUPILS IN THE INTERMEDIATE GRADES." AN UN-PUBLISHED ED.D. DISSERTATION, UNIVERSITY OF ILLINOIS,1968.
Purpose: Identification of a pupil's independent, instruc-tional and frustration reading levels are essential if aca-demic progress, commensurate with aptitude, is to occur.Standardized reading tests have been found ineffective in pro-viding this basic information. The informal reading inventory,if properly constructed and administered, will provide an ex-aminer with this data, however, it must be given individuallythereby precluding extensive usage for purposes of readingevaluation.
The purpose of this study was to determine if a testpredicated on the Cloze procedure (a word deletion system)could provide basic information similar to that obtained withthe informal reading inventory. Because a cloze test samplesboth lexical and structural language factors, two componentsof reading comprehension, it appeared logical to propose suchhypothesis. Secondly, the study permitted consideration ofthe relationship of scores obtained with the two informal read-ing tests and those derived with the standardized reading testand the intelligence measure.
Principles, Concepts, or Procedures: A total of 365 studentssomewhat equally distributed among the three levels of the in-termediate grades of a suburban elementary school system nearthe City of Chicago were the subjects utilized in this study.The numerical division of sexes was almost equal althoughslightly favoring males. Each student was administered theLorne-Thorndike Intelligence Test Multi-Level edition, theGates-MacGinitie Reading Tests - Survey D, an informal readinginventory predicated on the Powell paradigm, and a cloze test.These were administered in the same sequence as just presented.The reading passages used in the construction of the informalreading inventory and the cloze test were identical. At leastone week elapsed between administration of the two informaltests. All examinations were completed during the month ofFebruary, 1968.
A total of seventeen passages extracted from the Scott
185
Foresman, The New Basic Readers: Curriculum Foundation Series(1965) provided the basis of constructing the informal readinginventory and the cloze test. The reading difficulty level ofthe passages ranged from grade 1.6 to 9.9 with approximately.6 grade difference between each passage. The readabilitylevel of those passages taken from basals below grade four wasdetermined through the application of the Spache Readabilityformula, and the excerpts taken from basals designed for useabove grade three were analyzed with the Dale-Chall Readabilityfc_mula.
Conclusions: Based on the statistical analysis it was possibleto conclude that a measuring instrument predicated on the Clozeprocedure can be expected to identify the independent, instruc-tional and frustration reading levels with a degree of accuracycomparable to that obtainable with a properly constructed andadministered informal reading inventory when the subjects areelementary school intermediate grade pupils. Cloze percentagescores below 47% were found to identify the frustration readinglevel; scores ranging between 47% and 61% demarcated the in-structional reaaing level with the maximum indicated by theratio identical with smallest of these polarities; and the inde-pendent reading stratum was identified when the percentage ex-ceeded 61.
Furthermore, cloze test scores correlated with outcomesof a standardized reading test and an intelligence scale aswell as the correlations of the informal reading test with thesame measurement instruments.
Recommendations:1) Cloze tests can be employed by teachers to identify the
independent, instructional and frustration reading levels ofpupils in the intermediate grades of the elementary school.
2) The cloze percentage criteria of 62% plus - independentlevel; 47% through 61% - instructional level; and less than47% - frustration level should be employed as only one set ofreference points. Other ratios might be suggested on the basisof classroom examination.1
'Dissertation Abstracts A: The Humanities and SociLlSciences (Ann Arbor, Michigan: University Microfilms, 1969),Vol. 29, No. 12, Part 1, pp. 4314-A, 4315-A. (Order No. 69-10,625, 253 pages).
186
BLOOMER, RICHARD H. "THE CLOZE PROCEDURE AS A REMEDIAL READ-ING EXERCISE." JOURNAL OF DEVELOPMENTAL READING, 5(1962), 173-181.
Purposes The use of the cloze procedure as a remedial read-ing technique for college students was investigated.
Princi les Conce is or Procedures: Subjects were two groupsof 4 students each, who were randomly selected from volun-teers.
Materials for the experimental group were developed fromelementary basal readers. Every-tenth-word deletions wereused. The materials were levelled and sequenced in a waywhich allowed students to complete the program in as few asten or as many as fifty class meetings. Both students andinstructor corrected the exercises, and both exact-word andsynonym replacements were accepted.
The control group used Gilbert's Power and Speed inReading for two one-hour sessions each week. In both groups,attempts were made to control the teacher variable by assign-ing the instructors invigilating roles.
Conclusions: The cloze procedure produced a significant in-crease in reading comprehension and was found to be highlymotivating.
187
BLOOMER, RICHARD H. "THE EFFECTS OF NON-OVERT REINFORCEDCLOZE PROCEDURE UPON READING COMPREHENSION." NEW FRON-TIERS IN COLLEGE-ADULT READING. EDITED BY G. B. SCHICKAND M. M. MAY. YEARBOOK OF THE NATIONAL READING CON-FERENCE. MILWAUKEE, WISCONSIN: NATIONAL READING CON-FERENCE, 1965, 31-40.
Purpose: The author describes the third in a series ofstudies which was designed to compare fifth, seventh, nin:,and eleventh grade boys and girls and the long-term recall vicloze materials made with various syntactical deletionsagainst straight reading material and random cloze.
Principles, Concepts, or Procedures: Subjects were 1411pupils from a regional school in upstate New York. Twenty-four 600-word passages were prepared and adjusted to thefifth grade reading level by the Yoakum Readability Formula.Twelve comprehension questions were prepared for each passage.Each passage was prepared in seven cloze forms deleting nowords, random words, nouns, verbs, adjectives, pronouns andconjunctions, and noun determiners.
Each student was required to read six of the selections,complete the deletions where appropriate, and answer thequestions. The subjects graded their own work. The studylasted three weeks.
Conclusions:
1. higher grade pupils confronted with mate-rial of extreme simplicity, . , could not de-rive sufficient self-reinforcement from readingthe material or completing the cloze exercises tocomprehend at a level better than that of theninth grade. (38)
2. , as the material gets relatively moresimple, the differences between the sexes seemto disappear, and the tendency for boys' perfor-mance to be superior over girls increases. (38)
188
BLUMENFELD, JACOB P., AND MILLER, GERALD R. "IMPROVING READINGTHROUGH TEACHING GRAMMATICAL CONSTRAINTS." ELEMENTARYENGLISH, 43 (1966), 752-755.
Purpose: The authors sought to determine
1. what :t was that good students of English did knowabout grammatical usage and poor students did not which en-abled the former to learn material more efficiently,
2. if the poor students would improve after instruction.
Principles, Concepts, or Procedures: A number of collegefreshmen were administered a series of 36 cloze passages.1Although exact replacement scores, synonym scores, and wordclass scores were completed from the results, only thelatter were used in this study. The Davis Reading Test wasadministered before and after the cloze exercises.
Conclusions:
1. Both good and poor students of English demonstrated agrammatical mastery of the language when the word class re-sults were analyzed.
2. No significant improvement in reading comprehensionwas in evidence after students had completed the cloze exer-cises.
3. When students were taught to transform verbs and ad-jectives into nominalizations, reading scores showed improve-ment significant at the .05 level.
1The materials are described in G. R. Miller and E. B.Coleman, "A Set of Thirty-Six Prose Passages Calibrated forComplexity," p. 61 of this thesis.
189
BORMUTH, JOHN R. DEVELOPMENT OF STANDARDS OF READABILITY:TOWARD A RATIONAL CRITERION OF PASSAGE PERFORMANCES.FINAL REPORT TO THE OFFICE OF EDUCATION, DEPARTMENT OFHEALTH, EDUCATION, AND WELFARE, PROJECT NO. 9-0237.CHICAGO: UNIVERSITY OF CHICAGO, JUNE 1971.
The author presents a composite of five studies designed
to develop and demonstrate a model for identi-fying criterion leliels of performance for a particu-lar insttuctional task. The specific objective wasto identify the score on a cloze test that representsthe most desirable level of performance on instruc-tional materials. (viii)
Studies I-IV will be reviewed; Study V deals with teach-ers' ratings rather than an application of the cloze procedure.
STUDY Il
Purpose: The purpose of this study was
. to determine how closely, if at all, the re-gressions between the information gain and clozemeasures obtained from different. passages resembledeach other, and also to examine the shapes of thoseregressions. (32)
Principles, Concepts, or Procedures: A 52-item cloze test wasmade over a 263-word passage from a psychology textbook andwas administered to 260 students enrolled in grades 3, 5, 7,and 11, as well as sophomore and graduate students for the pur-pose of matching the subjects by pairs for reading ability.
Five forms of a cloze test were made from each of twopassages from the textbook. Every-fifth-word deletions wereused, so that all words became items requiripg replacement.Two multiple-choice tests were prepared for these same pas-sages.
1See also John R. Bormuth, "Empirical Determination ofthe Instructional Reading Level," Appendix.
190
The tests were administered in three sessions over a two-week period3 There were no time limits.
Conclusions: Neither of the two regressions fell consistentlyabove the other in the range where comparisons were possible.The shape of the regression curve was approximately similar toa normal ogive curve.
The fact that the two curves were nearly identicalin their parameters indicated that the methods usedt, construct the comprehension tests were likely toproduce tests having at least a useful degree ofconformity to the instruction and to measure a fair-ly uniform set of behaviors across passages. (40)
STUDY II
Purpose: Because of a number of limitations inherent in thedesign of the first study, further exploratory work was deemednecessary and refinement undertaken.
Principles, Concepts, or Procedures:
. a design was used that permitted all data tobe gathered in a single testing session. This wasachieved by giving the student a cloze test overone passage and a preference rating scale and pre-and post-comprehension tests made from a secondpassage of matched difficulty. (47)
The passages were chosen from the Gray Oral Readirs: Para-graphs (1963 edition). Each test booklet contained cloze andcompletion tests made over two passages which had been matchedfor difficulty.
Subjects were 1320 students enrolled in grades 3 through12 in a suburban community.
Conclusions:
The really fundamental methodological problem ofdevising a direct test of the regression identityassumption was not solved and remained unsolved at
191
the time this report was being prepared. The sub-stantative problems consisted of determing whetherdifferent performance criteria must be derived forstudents at each grade level and materials at eachdifficulty level. . . . It was also necessary tomake a digression into the study of motivation theoryand then to reassess the methods used to measure pre-ferences for materials. (75)
The results clearly indicated that it is necessaryto calculate separate performance criterion scoresfor students at every grade level. (77)
It appeared unnecessary to calculate different cri-terion scores for different levels of passage diffi-culty. (77)
STUDY III
Purpose: "This study attempted to obtain regressions thatwould be useful for at least tentatively identifying perfor-mance criterion scores." (79)
Three specific objectives were:
1. . . . to analyze and model the concept of arationally derived performance criterion. (79)
2. . . . to further analyze the preference rat-ings of passages and to obtain the regressionsof those ratings and of information gain scoreson cloze scores. (79)
3. . . to further develop the procedures for con-structing comprehension tests. (79)
Principles, Concepts, or Procedures:
The testing design used in the preceding study provedto be highly efficient and so was retained for thisstudy, but with three major alterations. First, eachof the four types of tests taken by a student was madefrom a different passage. The second alterationwas to employ passages that could be said to be more
192
representative of instructional materials. The thirdalteration was to employ longer passages and compre-hension tests in order to obtain more reliable resultson the cloze and information gain tests. (81)
The study
involved 10 grade levels of students crossed by8 levels of passage difficulty, with four passagesnested within each difficulty level and five clozetest forms nested within each cloze passage. (86)
Each of the 1600 subjects therefore completed differentbooklets. The study lasted three weeks.
Conclusions: Before the obtained regressions can be appli-cable, items beyond those in the area of literal comprehensionwill have to be added.
The preference scales added to this study indicated thata student's willingness to study is greatly influenced by sub-ject matter, style, and passage difficulty as well as the usemade of the passage.
WUDY IV
Purpose: The relationship between rate of reading and clozescores was investigated.
Principles, Concepts, or Procedures: "The materials and pro-cedures followed in this study were nearly identical to thosein Study III." (119)
Twenty students at each of three grade levels - 4, 7, and10 - were subjects.
Conclusions: "The students at the higher grades appeared toshow more rapid rates of reading than the students at lowergrade levels." (120)
193
SUNIMARY
In assessing the validity of the thirty performance cri-terion scores produced by the model in its current stage ofdevelopment, the author underscored three points.
I. . it appears that the subject mattar, style,and willingness to study preference ratings arethe dominant variables in determining where thecriterion score is set by the model.
2. the model dictates that as students reachhigher grade levels they should be given moredifficult materials.
3. the model generally dictates that studentsbe given easier materials for use in textbookreading than for reference and voluntary reading.(140-141)
194
FAUBION, NORMA NELL. "THE EFFECT OF TRAINING IN THE USE OFCLOZE ON THE ABILITY OF FOURTH GRADE PUPILS TO GAIN IN-FORMATION FROM WRITTEN DISCOURSE." UNPUBLISHED PH.D.DISSERTATION, TEXAS AM UNIVERSITY, 1971.
Purpose: Two experimental groups and a control were employedfor this research which was designed to measure the effect ofcloze training upon the reading comprehension of fourth gradepupils when the training focused upon the use of structurewords and verbalization by students concerning responses tocloze exercises.
principles, Concepts, or Procedures: Three fourth grade class-rooms in an elementary school in South Texas were randomlyassigned to the two treatment groups and the control.
Stratified random samples of passages, approximately onehundred words in length, were taken from three social studiestextbooks. The passages were assigned to one of two forms ofa cloze test of ability to gain information. The two formswere administered to a group of fourth grade students for thepurpose of establishing reliability. Reliability scores of.909 and .933 were,obtained for Form A and Form B respectively.Stratified random samples from the three textbooks were usedto develop a series of ten cloze lessons, each containing twocloze exercises. Half of the exercises utilized the fifth-word deletion pattern and half utilized a structure word dele-tion pattern.
Pre-test measures of reading comprehension were taken byadministering the CTAGI and the Stanford Reading AchievementTest, Form W. The treatment phase covered a period of tendays. Subjects in the control group participated in a dailysilent reading lesson based on passages from which the clozeexercises were developed. In experimental A, subjects weregiven a cloze lesson each day. In addition, the corrected ex-ercises from the previous day were returned and the subjectswere requested to note the corrections. This procedure wasfollowed for ten consecutive school days. Subjecl3 in experi-mental B followed the same procedure as experimemal A withone exception. When the corrected cloze lesson was returned,the subjects were encouraged to verbalize abort the appropri-ateness or lack of appropriateness of their responses. Post-test measures of reading comprehension were taken by adminis-tering the CTAGI and the Stanford Reading, Achievement Test,Form X.
195
Conclusions: Three null hypotheses were generated as part ofthe original design of this study. No significant differencewas found between students who received training in the useof cloze and students who did not receive training in the useof cloze as measured by the CTAGI, the CTAGI: Structure Words,and by the Stanford Reading Achievement Test. Thus, the threenull hypotheses were not rejected. A fourth null hypothesiswas generated: that there would be no significant differencein performance between students who received training in theuse of cloze and those who did not receive training as mea-sured by the CTAGI: Lexical Words. A significant differencewas found at the .05 level of confidence. This resulted inthe rejection of the fourth hypothesis.
Although the researcher was unable to demonstrate thattraining in the use of cloze resulted in growth in comprehen-sion skills, the study seemed to indicate that there mighthave been certain variables which influenced the effectivenessof cloze training. It was, therefore, recommended that re-search efforts be directed toward the identification of theeffect of the interaction of such variables as reinforcement,the deletion pattern, and the types of materials utilized forthe development of cloze exercises.'
1glpsertation Abstracts International A: The Humanities
and Social Sciences (Ann Arbor, Michigan: University Micro-films, 1972 717;1732, No. 8, p. 4486-A. (Order No. 72-5655.164 pages).
196
FRIEDMAN, MILDRED MCELHINNEY. "THE USE OF THE CLOZE PROCE-DURE FOR IMPROVING THE READING COMPREHENSION OF FOREIGNSTUDENTS AT THE UNIVERSITY OF FLORIDA." UNPUBLISHEDED.D. DISSERTATION, THE UNIVERSITY OF FLORIDA, 1964.
Purpose: Although the need for foreign languages has been in-creasing vastly, there is as much confusion about the approachto language teaching as there was centuries ago. Throughoutits history, the methods and emphases of language teaching haverecurred in cycles. The advocates of each method have disre-garded the aims of their teaching or accepted the popularmethod as the panacea. As a result, language teachers aregroping for answers. Research is essential to clarify theissues and to improve the results of language teaching.
The purpose of this study was to find another successfulmethod for improving reading comprehension for foreign stu-dents. By using the doze procedure (a word deletion system),it was hoped that simple materials could be made interestingenough for mature students. Because doze samples both thelexical and structural factors of a language, certain insightsinto language learning were expected.
Principles, Concepts, or Procedures: Three classes of foreignstudents in English C-33 and C-34 at the University of Floridawere used as subjects one day a week for one trimester. Twoclasses were given a series of twenty deleted readings, calleddoze tests, from McCall Crabbs Standard Test Lessons in Read-ing, Books A to E (Grades 3 to 71. The other class was giventhe same readings undeleted. All groups were given a multiple-choice test after each reading.
Conclusions: The design of this study produced data from thepre- and post-test scores from the Metropolitan AchievementTest, Reading Sectic.r., Intermediate Level and a vocabulary-in-context test, based on the vocabulary subtest of the MAT, andfrom the errors on the doze and multiple-choice tests. Thefollowing conclusions were reached from these data.
1. The doze procedure is as effective in producing im-provement in comprehension and vocabulary-in-context as amethod using close reading without deletions. The t tests,applied to these changes, indicated that gains were signifi-cant for all groups but not between groups.
197
2. The validity of the cloze procedure as a measure ofcomprehension for foreign students has been established byits significant correlation with the MAT and the multiple-choice tests.
3. A compltte knowledge of the vocabulary is not essen-tial to the understanding of a reading if the subject matteris within the experience of the student. Cloze measuresfamiliarity with the subject as well as the difficulty of thearticle. This may confuse the readability scores.
The following implications and recommendations wereformulated:
1. Since one experimental group made significant gainsin vocabulary, the cloze procedure may be superior to the closereading method in improving vocabulary. Further studies shouldbe made in the use of cloze to improve vocabulary.
2. Students report that they are aware of the patternsof the language when they do cloze tests. It is possible thatthe cloze procedure might be a successful method of teachingand testing structure. Investigations in the relationship ofstructure to reading would prove most helpful to languageteachers.
3. Further study needs to be made on the permanency ofthe gains with the cloze procedure. It seems quite probablethat the need of close observance of language patterns Pndthe development of skill in using context and in findir.g re-lationships might establish a more permanent gain than othermethods.
4. The influence of personality rigidity upon the :4:-sults of the cloze procedure and the use of cloze to break thehabits of rigidity are factors observed in this study whichneed further research.1
'Dissertation Abstracts (Ann Arbor, Michigan: Univer-sity Microfilms, 1964), Vol. 25, No. 6, pp. 3420-3421.(Order No. 64-11,533, 124 pages).
198
GUICE, BILLY MILTON. "AN EXPERIMENT COMBINING INSTRUCTION INCOMPREHENSION WITH PRACTICE IN THE CLOZE PROCEDURE INREADING." UNPUBLISHED ED.D. DISSERTATION, UNIVERSITY OFSOUTHERN MISSISSIPPI, 1966.1
Purpose: The problem of this research was to experiment withthe cloze procedure as a technique for teaching reading compre-hension skills to college students enrolled in an improvementof study course at the University of Southern Mississippi dur-ing the winter quarter, 1965-1966.
The purpose bf this study was to ascertain whether a groupof college students, who received regular instruction in read-ing comprehension plus instruction in and practice with thecloze procedure, would make significant gains in comprehensionwhen compared with a group of college students who receivedonly regular instruction in comprehension. Some additionalinformation that this study sought to reveal was the relation-ship between comprehension and successful clozure, creativityand successful clozure, intelligence and successful clozure,and comprehension and creativity.
Principles, Concepts, or Procedures: The Improvement ofStudy Course was scheduled in two sections, one at 10:00 A.M.and the other met at 1:00 P.M. The lecture sessions for bothsections met on every Monday of the quarter. Each of the twolecture sections was broken into two laboratory groups. Forthe entire quarter, at its respective hour, one laboratorygroup met on Tuesday and Thursday and the other met on Wednes-day and Friday. Each period of work was fifty minutes induration, three days per week.
Through the use of the table of random numbers and stan-dard randomization procedure, the seventy-six students wererandomly assigned to one of the two scheduled lecture sections.The students enrolled in the morning lecture section were fur-ther randomly assigned to one of the two morning laboratorygroups. Likewise, the students enrolled in the afternoonlecture section were further randomly assigned to one of thetwo afternoon laboratory groups.
/See also Billy M. Guice, "The Use of the Cloze Procedure
for Improving Reading Comprehension of Cor,ege Students,"Appendix.
199
One morning laboratory group and one afternoon laboratorygroup were randomly selected as the experimental group. Theremaining morning and afternoon laboratory groups were the con-trol group. No student was allowed to change either his lec-ture section or his laboratory group assignment.
The Cooperative English Tests, Font u Form 1B, Reau-ing Compehension, The Guilford, Merrifi,_ and ChristensenTest of Creativity "Conseouences," and The Guilford, Merri-field, Christensen, and Wilson Test of Creativity "AlternateUses," Form A, and The Otis Quick-Scoring Test of Ment;1 Abili-ties wer3 used to collect the data.
The selection of materials used in teaching reading com-prehension was based upon the detailed analysis of both formsof The Cooperative English Tests, Reading Comprehension. Thecourse content emphasized the following reading comprehensionskills as a result of the analysis of the tests' main ideas,specific details, inferences, conclusions, interpretations.
The series of cloze exercises were based on college levelmaterials from recently published work texts. The cloze dele-tion technique was applied to each of the selected passages.Each deletion was replaced by an underlined blank space,twelve typewritten spaces wide. The deletion technique em-ployed in the cloze exercises was based on every nth cconceptword--noun, verb, adjective, or adverb as determined by theDepartment of English, The University of Southern Mississippi.
The cloze passages were given to the students during thelaboratory periods and after each exercise was completed, itwas scored by the instructor. The exercises were gradedallowing two points for a cloze unit and one point for anacceptable clozure synonym. The corrected passage was re-turned to the student at the following laboratory period.
Conclusions: The data were divided into three parts fortreatment. First, the data were entered into a LindquistType III analysis of variance. Secondly, Pearson Product-Moment correlation coefficients were computed between compre-hension and successful clozure, creativity and successfulclozure, intellgence and successful clozure, and comprehen-sion with creativity. Lastly, the data were subjected to amultiple correlation analysis.
200
The test of pre- post, experimental and control failedbe catistically significant at the .05 level and thus didupport the major hypothesis of this experiment.
An unexpected finding was the statistically significantpre-post-test, time interaction (P < .01). The change inpre- and post-test for all groups combined was significant atless than the .01 level. The highest correlation was betweencreativity and clozure .794 significant at the .01 level ofconfidence. The correlation between intelligence and crea-tivity was .640 significant at the .01 level. Comprehensionand intelligence correlated .636 significant at the .01 levelof significance. In assessing the relationship between com-prehension and creativity the correlation of .602 was likewisesignificant at the .01 level. The correlation between intelli-gence and clozure .400 was significant at the .05 level of sig-nificance. The lowest correlation was between comprehensionand clozure .203 and was not significant at the .05 level ofconfidence.1
1Dissertation Abstracts A: The Humanities and SocialSciences (Ann Arbor, Michigan: University Microfilms, 1967),Vol. 28, No. 1, p. 143-A. (Order No..67-8740, 195 pages).
GUSCOTT, CHARLES EDGAR. "THE EFFECT OF CLOZE PROCEDURE EXER-CISES ON THE IMPROVEMENT OF READING ACHIEVEMENT AND OFREADING COMPREHENSION OF SELECTED SIXTH-GRADE STUDENTS."UNPUBLISHED PH.D. DISSERTATION, THE UNIVERSITY OF AKRON,1971.
Purpose: The purpose of this investigation was to assess theeffect of cloze procedure exercises as defined in the studyon selected sixth-grade student's ability to comprehend theirassigned social studies text and their performance on a stan-dardized test of reading achievement.
Principles, Concepts, or Procedures: A readability study ofthe social studies text used by the selected students and acloze procedure comprehension test using every fifth word de-letions demonstrated the need for giving attention to the read-ing problem. The Dale-Chall formula was used to assess thedifficulty of the text. A stratified random sample was drawnfrom the six selected suburban schools. The sample was drawnon the basis of reading achievement stanine scores. N = 60.
Cloze procedure exercises were constructed using noun de-terminer deletions. Materials were taken from unfamiliar por-tions of the social studies text. Exercises were presented inthe form of overhead transparencies and numbered answer sheetsfor a period of eight school weeks.
Post - testing was of two kinds: (1) a cloze procedure testbased on unfamiliar social studies material and employing everyfifth word deletions, and (2) Iowa Tests of Basic Skills, Read-inR, Form I.
Conclusions: Employing a 2 x 3 factorial analysis of varianceit was found that the cloze exercises did produce significantlybetter scores on the standardized test of reaching achievementbut did not result in significantly better scores on the clozecomprehension test. An analysis of interactions indicated thateffect was not related to levels of achievement. It was con-cluded that cloze exercises as defined in the study had differ-ent effects on different levels of reading achievement. Al-though, on the average, effect on achievement was significant,it was not transferred in a significant way to the functionalreading situation by any level.1
1Dissertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1972), Vol. 32, No. 7, p. 3861-A. (Order No. 72-4051,93 pages).
202
GUSZAK, FRANK J. A COMPARATIVE STUDY OF THE VALIDITY OF THECLOZE TEST AND METROPOLITAN ACHIEVEMENT TEST (READING COM-PREHENSION SUBTEST) FOR MAKING JUDGMENTS OF INSTRUCTIONALLEVELS. AUSTIN, TEXAS: UNIVERSITY OF TEXAS, 1969.
Purpose: The use of the cloze procedure to determine instruc-tional reading levels for pupils in grades 4, 5, and 6 was in-vestigated.
Principles1 Concepts, or Procedures: For each grade, 50 sub-jects were selected who, on the basis of teacher judgment, werereading at their respective grade levels. The Botel ReadingInventory, Form A (Word Opposites) served as the criterion forinstructional level. Compared with the Botel Test scores werescores from three cloze' tests for each grade level (below gradelevel, at grade level, and above grade level) and the scores ofthe Reading Comprehension subtest of the Metropolitan Achieve-ment Tests.
Conclusions: Findings revealed relatively low correlations(.11 to .18) between the Botel and the cloze tests. The Boteland the Metropolitan tests correlated fairly high at grades 4and 5 (.49, .21), and showed a statistically significant differ-ence at grade 6. No significant relationships were noted be-tween the cloze test instructional levels and the MetropolitanTest scores. It was suggested that different means of assess-ing comprehension may have accounted for low correlations be-tween the cloze and the Botel tests. Tables and a bibliographyare included.1
1Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1970), Vol. 5, No. 9, p. 79.(ED 039 108, 36 pages).
203
LAWRENCE ERHARDT. "AN EXPERIMENTAL STUDY OF THEEFFECT ON VARIOUS READING ACHIEVEMENT SCORES OF TEACHINGSELECTED CONTEXT AIDS TO A GROUP OF FIFTH GRADE PUPILS."UNPUBLISHED ED.D. DISSERTATION, UNIVERSITY OF MISSOURI,1960.1
Purpose: To evaluate the effects on reading achievement scoresof an experimental group of fifth grade pupils of teaching thenature and use cf selected context aids. The achievement testsused were designed to measure reading comprehension, vocabulary-in-context, and context comprehension, and to yield a vocabu-lary-comprehension composite called average reading.
Principles, Concepts, or Procedures:
1. The reading achievement of the experimental group wascompared to the reading achievement of two control groups. Theexperimental group and the control-I group were two fifthgrade classes in the same school. The control-II group camefrom a school located in a more favorable environmental setting.
2. Characteristics of the groups in terms of intelligenceand reading achievement were obtained at the beginning of theexperiment. The California Test of Mental Maturity, the SRAReading Test, and the Comprehension "X" Test (constructed bythe writer) were used as pre-test instruments. Because of in-equalities existing among the groups in intellectual and read-ing achievement status, the analysis of covariance statisticaltechnique was utilized. A chi-square technique and "t"test of the significance of the difference between means werealso utilized.
3. The experimental variable was instruction in the use ofthese selected context aids: contrast, explanatory words andphrases, and two inferential aids. Instruction was given forthree thirty-five minute periods per week for a total of fourweeks.
4. At the end of four weeks the pupils of all the groupswere administered an alternate form of the reading achievementtests.
1See also Lawrence E. Hafner, "A One-Month Experiment in
Teaching Context Aids in Fifth Grade," Appendix.
204
Conclusions:
1. After short term instruction in the use of contextaids, none of the comparisons made in the reading achievementareas of comprehension, vocabulary-in-context, average reading,and context comprehension demonstrated unequivocally the influ-ence of the experimental method.
2. Though short term instruction in the use of contextaids did not differentiate between the experimental and thecontrol groups in a statistically significant manner, there isa suggestion that it is in the area of vocabulary-in-contextachievement that the teaching of context aids would be mostproductive.
3. In the situation where children were similar in intel-ligence and background of experience, more of the children whohad received instruction in the use of context aids made gainsin comprehension than was the case with children who had notreceived such instruction.1
1Dissertation Abstract's (Ann Arbor, Michigan: UniversityMicrofilms, 1961), Vol. 21, No. 12, P. 3714. (L. C. Card No.Mic 60-6806, 137 pages).
205
HUNTER, CHARLES F. "CLUING THE GAP: AN APPROACH TO SOLVINGREADING PROBLEMS AT SAN JOSE CITY COLLEGE." PAPER PRESEN-TED AT THE MEETING OF THE WESTERN COLLEGE READING ASSOCIA-TION, LOS ANGELES, CALIFORNIA, APRIL 1-3, 1971.
Purpose: The necessity to provide instruction in reading iswidespread in community colleges, due primarily to the liberal
. enrollment policies these colleges generally have.
Principles, Concepts, or Procedures: The program at San JoseCity College uses the Davis Eeasil.n&leEt and the Iowa SilentReading Test as part of its entrance battery and assigns stu-dents to either regular or one of two noncredite pass-fail,remedial English classes on the basis of scores on the speedsection of the Davis test. Regular English instructors,reading teachers, and graduate students in English acting asparaprofessionals teach the remedial reading and writing sec-tions. Emphasis is on skill development, beginning withisolated skill areas and working toward integration of skillsin successful critical reading. Machines such as tachisto-scopes and language masters supplement class instruction, andthe use of clozure activities to strengthen vocabulary andcomprehension has been particularly successful.
Conclusions: Evaluation of 310 students in remedial sectionsin the spring of 1970 showed an average gain of nearly 2 yearsas measured by the Iowa test. Future plans for the program in-clude extending instruction into various departments of thecollege. References are included.1
'Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1971), Vol. 6, No. 8, p. 70.(ED 049 895, 9 pages).
206
JONGSMA, EUGENE. THE CLOZE PROCEDURE AS A TEACHING TECHNIQUE.NEWARK, DELAWARE: INTERNATIONAL READING ASSOCIATION,1971.
Purpose: The author states three purposes:
1. to review the literature related to the use of clozeas a teaching technique.
2. to organize and synthesize the literature.
3. to offer suggestions as to future research.
Principles, Concepts, or Procedures: The work of Bloomer,Friedman, Schneyer, Blumenfeld and Miller, Heitzman and Bloomer,Martin, Guice, Roossinck, and Kingston and Weaver is reviewed.
Jongsma isolates several weaknesses which have been charac-teristic of many of the studies utilizing cloze as a teachingtechnique.
1. Lack of direct teaching and over-reliance on pupils'completion of exercises alone.
2. Lack of a lucid definition of the problem.
3. Weak experimental designs.
4. Measurement problems which interfered with true assess-ment of growth.
5. Inadequate descriptions of studies.
Because of the disparate efforts of the past, Jongsmamakes several recommendations for future studies:
1. Development of teaching strategies be they teacher-pupil,small group, or whole class in nature.
2. Investigation of presentation methods- complete cloze exercise
1These sources may be found elsewhere in this chapter, ex-
ilept the last two which are listed in Section II of the Appendix.
207
- complete cloze exercise " answer multiple-choice questions
- read intact passage complete cloze exer-cise
- complete pre-close --9 read passagecomplete post-cloze
3. Use of shifting scoring methods wherein grammaticalclass responses, synonyms, and later exact replacements arerequested.
4. More intense and longer studies.
5. Statement of specific, desired outcomes.
6. More attention to the choice of deletion system.
7. Use of cloze in areas such as context clues, vocabu-lary development, and content reading.
8. Less attention to experimental-control comparisons andmore to factorial designs.
9. More attention to randomization.
10. Better measurement procedures.
11. Adherence to guidelines when reporting research.
Conclusions: Because past studies utilizing cloze as ateaching device have failed to significantly improve readingproficiency should encourage future research if errors and in-adequacies of the past are ameliorated.
208
KIRBY, CLARA LOU LAUGHLIN. "A COMPARISON OF SCORES OBTAINEDON STANDARDIZED ORAL AND SILENT READING TESTS AND ACLOZE TEST." UNPUBLISHED ED.D. DISSERTATION, BALL STATEUNIVERSITY, 1967.1
Purpose: The purpose of the study was to determine the rela-tionship, if any, that exists between the reading levels of ele-mentary school children as indicated by scores received on acloze test and scores received on selected oral and silent stan-dardized reading tests.
Principles, Concepts_ or Procedures: Related hypotheses weretested to determine if the relationship would be similar forvarious sub-groups of the sample population. These sub-groupswere composed of boys, girls, outstanding readers, able readers,less able readers, high ability students, low ability students,and pupils at each grade level, one through six.
The population for the investigation consisted of 178 ele-mentary school children in grades one through six enrolled inone public school in a Midwestern city. Ninety-eight boys andeighty girls constituted the sample. The mean I.Q. for thegroup:was 101.7.
Instruments employed in the study for comparison purposesincluded the Gates Reading Tests (Primary, Advanced Primary andSurvey), the Gilmore Oral Reading Test, the Gray Oral ReadingTest and a cloze test. All were administered within a six-weekperiod beginning February 22, 1965.
Conclusions: Data were analyzed by simple analysis of variance,ratios, t tests, and inspection. Problems were stated in termsof null hypotheses and the .01 level of confidence was employedto determine significance of F ratios and t tests.
Analysis of data resulted in the following major findingsrelative to the sample employed in this study:
1. Statistically significant differences existed amongmeans of scores obtained on the cloze test, gateta.ReadirzLTests,Gilmore Oral Reading Test, and the Gray Oral Reading Test forelementary school children in grades one through six.
'See also Clara L. Kirby, "Using the Cloze Procedure as aTesting Technique," Appendix.
209
2. There were no statistically significant differencesamong the means of scores obtained on the four instruments atthe fifth and sixth grade levels, for high ability students,and for outstanding readers.
3. Mean scores on the cloze test did not differ signifi-cantly from those derived from the Gilmore Oral Reading Test(when the accuracy and comprehension scores were averaged) atany level or for any sub-group.
4. The Gates Reading Tests yielded consistently highermean scores than did the other three instruments.
5. The Gray Oral Reading Test yielded consistently lowermean scores than did the other three instruments.
Interpretation of findings led to the following major con-clusions:
1. The cloze test, Gates ReadinTests, Gilmore Oral Read-ing Test, and Gray Oral Reading Test do not appraise equiva-lently the reading attainment of elementary school children ingrades one through six.
2. The four instruments do not measure equivalently thereading achievement of pupils for grades one through four,boys, girls, low ability students, able readers, and less ablereaders.
3. Generally, the four instruments identify readinglevels comparably for pupils in the intermediate grades andfor superior readers.
4. Within the limits of the study, the cloze test yieldsreading level scores nearly comparable to those of the GilmoreOral Reading Test (when word accuracy and comprehension scoresare averaged on the Gilmore instrument).
5. As elementary school children develop increased read-ing skill, the four instruments tend to yield less dissimilarresults. The findings of this study indicate that the use ofa cloze test by classroom teachers for determining instruction-al reading levels of children is a promising technique.
121211KIELLULAWILDSLULLLIIAEUMAllitiPs and SocialSciences(Anniii'borlichigary Mic.eofilms, 1968),Vol. 28, No. 11, pp. 4512-4513. (Order No. 68-3243, 125pages).
KIRCHHOFF, LEO HENRY. "A STUDY UTILIZING THE CLOZE TEST PROCE-DURE TO DETERMINE READING LEVELS OF FIRST GRADE CHILDRENWHO HAVE BEEN TAUGHT BEGINNING READING BY FOUR DIFFERENTAPPROACHES." UNPUBLISHED ED.D. DISSERTATION, UNIVERSITYOF KANSAS, 1968.
Purpose: The investigation was designed to determine whetheror not a significant Lelationship existed between scores on acloze test and scores on an Informal Reading Inventory whenadministered to first grade children who have been taught be-ginning reading by four different approaches. The four differ-ent approaches employed in this study were: Initial TeachingAlphabet, Sullivan Programmed Reading, Linguistics, and Basal.
The 15 hypotheses involved four basic problems: (1) thecorrelation of reading levels for the entire population on acloze test and an informal reading inventory, (2) the correla-tion of reading levels obtained on the cloze tJst and informalreading inventory for first graders who had been taught by fourdifferent approaches to the teaching of beginning reading, (3)the correlations of levels identified on the cloze test and theinformal reading inventory for boys and for girls, (4) correla-tions were determined to find if differences existed among thefour different approaches to the teaching of beginning readingregarding eight variables when the measured mental capacity washeld constant.
Principles. Concepts, or Procedures: The population consistedof 120 first grade students who were selected at random fromfour school districts located in Johnson County in the state ofKansas. The four districts identified emphasized a differentapproach to the teaching of beginning reading. A cloze test,an informal reading inventory, a silent reading test, and anintelligence test were used in the investigation. All testingwas completed near the end of the first grade year during afive week period.
Coefficient correlations were used to compare three read-ing levels, independent, instructional, and frustration. Theselevels were determined from pupil responses on the cloze testand the informal reading inventory.
Conclusions: The reading levels derived from the use of thecloze test correlated with the informal reading inventory at
211
the .05 level of confidence for the total population in firstgrade.
The instructional and frustration reading level resultswere significantly correlated at the .05 level of confidencefor all first grade students who had been taught beginningreading by four different reading approaches. The indepen-dent reading level results were significantly correlated atthe .05 level of confidence for the first grade students whohad been taught beginning reading by the Sullivan ProgrammedReading, Linguistics, and Basal. The independent readinglevel results for the students who had been taught beginningreading by the Initial Teaching Alphabet approach were not sig-nificant at the .05 level of confidence. The cloze test andthe informal reading inventory, for boys and girls separately,provided statistically significant scores for all reading levels.
As a result of the statistical analysis, utilising theanalysis of covariance, it was concluded that there were nosignificant differences among the four different approaches tothe teaching of beginning reading regarding eight variables.
The correlations between the cloze test scores and the in-formal reading inventory scores at the instructional and frus-tration reading levels were statistically significant. Thisclose relationship would tend to indicate that a cloze testcould aid the fi?st grade teacher in determining the appropri-ate reading level for children nearing the end of their firstgrade experience. Cloze testing can be administered, simul-taneously, to an entire class and thereby save time in deter-mining the reading levels of children. The average length oftime required for administering the cloze test To first gra-ders, as a group, approached 30 minutes. The cloze procedureis easily administered and interpreted by classroom teacherswho have not had special training in test administration.1
'Dissertation Abstracts A: The Humanities and SocialSciencTFTWERTrbor, Michigan: University Microfilms, 1969),y21. 29, No. 10, p. 3329-A. (Order No. 68-17,408, 213 pages)."Gates-MacGinitie Reading Tests" - Form Al, and "The Lorge-
Thorndike Intelligence Tests," Level 1 and 2, Form A, follow-ing page 204 not microfilmed at request of author. Availablefor consultation at University of Kansas Library.
212
MARTIN, RUBY WHEELER. "TRANSFORMATIONAL GRAMMAR, CLOZE, ANDPERFORMANCE IN COLLEGE FRESHMEN." UNPUBLISHED ED.D. DIS-SERTATION, SYRACUSE UNIVERSITY, 1968.
Purpose: This study was designed to examine the relative effec-tiveness of two experimental programs in improving the languagearts skills of a group of Negro college freshmen. The two pro-grams were taught as a part of a freshman course in English.The two experimental treatments--instruction in transformation-al grammar and the completion of cloze exercises--were admin-istered to comparable groups of Negro college freshmen, andtheir pre-treatment and post-treatment scores on reading, writ-ing, and listening are compared to determine gains made andsignificance of difference between gains made in language artsskills.
Principles, Concepts, or Procedures: This study was conductedat South Carolina State College in Orangeburg, South Carolinaduring the 1966-67 school year. The one hundred students whichcomprised the treatment groups were selected from the four hun-dred fifty freshmen registered at State College during the ex-perimental year. In addition, forty-two freshman students pro-vided the population with which the basic competencies of thetreatment groups were compared on several measures.
During the experimental period, one treatment group de-voted eighteen sessions to transformational grammar. The othergroup devoted eighteen sessions to completion of cloze exercises.The nontreatment group followed the regular syllabus for Fresh-man English and Composition. Instructors were regular teachersof English in the Communications Center, South Carolina StateCollege. Criterion measures were the Iowa Silent Reading Tests,the STEP Writing Test, Writing Sample, the STEP Listening Test,and the Delancey Test of Linguistic Structures.
Conclusions: The two treatments produced significant differen-tial gains on pre- and post-test measures of reading, writing,listening, and recognition of linguistic structures. It wasfound that gains made in reading, writing, listening, and recog-nition of Mnguistic structures were not significantly different.Therefore, neither experimental treatment was superior to theother for these variables.
When treatment groups were compared on writing ability witha comparable nontreatment group, it was found that both treat-
213
ment groups were superior to the nontreatment group. No sig-nificant difference was found for the three groups on totalEnglish achievement as measured by the Cooperative EnglishTests. A comparison of the two treatment groups with the non-treatment group on listening revealed a significant F. Inthis analysis the cloze group was favored with the highestmean score.
'Dissertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 19691, Vol. 30, No. 1, p.,70-A. (Order No. 69-8659,103 pages). Appendix B: Measuring Instruments, pages 80-90,and Appendices D and E: Experimental Materials, pages 105-279, not microfilmed at request of author. Available for con-sultation at Syracuse University Library.
214
RANKIN, EARL F. "USES OF THE CLOZE PROCEDURE IN THE READINGCLINIC." READING IN A CHANGING SOCIETY. EDITED BY J.ALLEN FIGUREL. PROCEEDINGS OF THE INTERNATIONAL READINGASSOCIATION. NEW YORK: SCHOLASTIC MAGAZINES, 1959,228-232.
Purpose: The author describes the clinical utility of thecloze pr dure and suggests a way the procedure might be usedin a re' clinic.
Principles, Concepts, or Procedures: Rankin asserts that assess-ment of achievement is possible with cloze, for both contentand style would be constant when tests were made over instruc-tional materials.
Cloze exercises are easy to prepare and thus can be moresimply adapted to material of interest to the student than othermeasurement devices.
Close assesses readability and yields immediate reinforce-ment. Answer keys can be used in combination with discussionin this situation.
Vocabulary can be taught by deleting varying word classes.
Conclusions: In the author's view, the cloze procedure hastremendous potential as a diagnostic and remedial tool.
215
RANKIN, EARL F. "GRADE LEVEL INTERPRETATION OF CLOZE READABIL-ITY SCORES." PAPER PRESENTED AT THE NATIONAL READING CON-FERENCE, ST. PETERSBURG, FLORIDA, DECEMBER 3-5, 1970.
Purpose: Two studies were conducted to investigate the use ofa technique for providing a grade-level interpretation forcloze readability scores.
Principles, Concepts, or Procedures: In the first study, Bor-muth's cloze-test criteria were compared with the results ofthe Fry and Dale-Chall readability formulae. A total of 133students in grades 4 to 8 were required to read a 250-wordpassage with every fifth word deleted. The results indicateda fairly close correspondence between Bormuth's 44 percentcriterion for cloze readability analysis and the readabilityformulae. In the second study, a method devised by the writerwas used. The Informal Cloze Readability Inventory (ICRI)uses materials of unknown difficulty to be read by readers ofknown reading ability in order to evaluate the readability ofa book. The ICRI and the Paragraph Meaning subtest of theStanford Reading Achievement Test were given to groups of sixaverage readers in grades 4 to 6.
Conclusions: When cloze instructional levels were analyzed,they did not appear to 'be very different from the readabilitylevels as measured by the readability formulae. It was con-cluded 'hat valid readability assessments can be matte using theICRI based on a small number of subjects. References are in-cluded.'
'Research in Education (Washington, D.C.: United StatesGovernment Pr%nting Office, 1971), Vol. 6, No. 5, p. 110.(ED 046 657, 12 pages).
216
RANKIN, EARL F., AND OVERHOLSER, BETSY M. "REACTION OF INTER-MEDIATE GRADE CHILDREN TO CONTEXTUAL CLUES." JOURNAL OFREADING, 1 (1969), 50-73.
Purposes This study investigated ". the sensitivity ofintermediate grade pupils to contextual clues described byAmes' (1965) Classification Schema of Contextual Aids." (50)
Principles, Concerts, or Procedures: Two hundred eight inter-mediate grade pupils were administered The Context Test whichwas based on fourth grade basals not used by the pupils. Thetest utilized the cloze procedure by deleting nouns, adjectives,adverbs, and verbs from their contexts in accord with Ames'schema. Each subject completed a test in about one hour ofeach of three consecutive days. Synonyms were accepted iftense and grammatical structure were correct.
Conclusions: Among the findings were the following:
1. The difference in difficulty among the contextual clueswere significant for intermediate grade children.
2. The rank order of difficulty among clues was highlyconsistent among both grade levels and reading levels.
3. Reading ability is highly predictive of abi1fty to usethe context clues studied here.
4. Question-answer, comparison-contrast, main idea-suppert-ing details, and non-restrictive clauses (appositive phrases)were most difficult for the subjects.
217
RANSOM, PEGGY ELAINE HITCHCOCK. "A STUDY TO DETERMINE READINGLEVELS OF ELEMENTARY SCHOOL CHILDREN BY CLOZE TESTING."UNPUBLISHED ED.D. DISSERTATION, BALL STATE UNIVERSITY,1965.1
Purpose: The study was designed to determine whether or not asignificant relationship exists between scores on a Cloze Testand scores on an Informal Reading Inventory.
The nine hypotheses involved three basic problems: (1)the correlation of reading levels for the entire population ona Cloze Test and from an Informal Reading Inventory, (2) thecorrelation of reading levels obtained at each grade, onethrough six, on a Cloze Test and from an Informal Reading In-ventory, and (3) the correlation of levels identified on theCloze Test and the Informal Reading Inventory for boys andfor girls.,
Principles, Concepts, or Procedures: The population consistedof 178 students in grades one through six, in a Muncie, Indianaelementary school. An Informal Reading Inventory and a ClozeTest were constructed for use in the study. A silent readingtest and an intelligence test were administered to the group toascertain normalcy of the sample. All testing was completedin a six week period.
Two sets of criteria were used in identifying levels fromthe Cloze Test. These were the 50/30/20 criteria and the raw-score criteria. Coefficient correlations were used to comparethree reading levels, Independent, Instructional, and Frustra-tion. These levels were determined from pupil responses onthe Cloze Test and the Informal Reading Inventory.
Conclusions: The Independent, Instructional, and Frustrationreading levels derived from the use of the Cloze Test (usingboth sets of established criteria) correlated with the Infor-mal Reading Inventory at the .01 level of confidence for thetotal population in grades one through six.
For pupils in the first grade, the relationship between
1See also Peggy E. Ransom, "Determining Reading Levels
of Elementary School Children by Cloze Testing," Appendix.
218
the results from the Cloze Test and the Informal Reading Inven-tory was found to be not statistically significant.
The Instructional and Frustration reading levels werefound to be correlated at the .01 level of confidence in gradestwo through six, respectively.
Only at fourth grade was the Independent reading leveldetermined from the two test methods (using Cloze Test 50/30/20criteria) statistically significant.
For the boys scores at the Independent, Instructional, andFrustration reading levels derived from the Cloze Test (bothsets of criteria) displayed correlations to the .01 confidencelevel with results from the Informal Reading Inventory.
Reading level scores derived for the girls' at the Instruc-tional and Frustration reading levels, between the Cloze Test(both sets of criteria) and the Informal Reading Inventory,were correlated at the .01 level of significance. At the Inde-pendent reading level the .01 level of confidence was estab-lished with the Cloze Test 50/30/20 criteria, but not with theCloze Test raw-score criteria.
The correlations between the Cloze Test scores and theInformal Reading Inventory scores at the Instructional andFrustration reading levels, grades two through six, were sta-tistically significant. This relationship indicated that aCloze Test could aid the teacher in determining the appropriateInstructional reading level for children in a classroom forthese grades when administered at the beginning of the schoolyear. The Cloze Test also would indicate the level of materialwhich would be frustrating for children in these classes.
First grade performance on the two tests yielded no sta-tistically significant relationship. A higher correlationmight have occurred had the testing been administered nearerthe end of the school year.
The Cloze Test 50/30/20 criteria, based on percentages ofcorrect responses, proved to be a more accurate indicator thanthe raw-score criteria.
Cloze testing which can be administered to an entireclass saves time. The average length of time required for ad-ministering the test to the group in this study was forty
219
minutes. The instrument is easily applied and can be inter-preted by qualified teachers who have had no special trainingin test administration. It was concluded that the Cloze Testcan provide a more efficient procedure for identifying In-structiooal and Frustration reading levels in the elementaryschools.
1Dissertation Abstracts (Ann Arbor, Michigan: UniversityMicrofilms, 1966), Vol. 26, No. 7 pp. 3705-3706. (Order No.65-13,038; 133 pages).
220
ROBINSON, RICHARD DAVID. "AN INVESTIGATION INTO THE USE OF THECLOZE PROCEDURE WITH A GROUP OF FUNCTIONALLY ILLITERATEADULTS." UNPUBLISHED ED.D. DISSERTATION, UNIVERSITY OFGEORGIA, 1971.
Purpose: The purpose of this study was to investigate the useof the cloze procedure with a group of functionally illiterateadults. Several segments of this problem were investigated.First, the study sought to test three types of cloze construc-tion as predictors of a standardized reading test. Second,the cloze procedure was tested against several readabilityformulas in ranking seven reading passages. The third purposewas the gathering of information on the opinions of cloze heldby this population.
Principles, Concepts, or Procedures: The sample for the studyconsisted of 57 inmates of the United States Federal Peniten-tiary in Atlanta, Georgia. Test scores collected for use inthis research included results from the Paragraph Meaning andWord Meaning subtests of the Stanford Achievement Test. In-telligence was measured by the Revised Beta Examination. Onlysubjects who scored between grade levels 4.0 and 7.9 and had anintelligence score of 90 or above were included.
In addition to these standardized tests, three instrumentswere used to measure various aspects of the cloze procedure.The first test was developed to evaluate three different typesof cloze construction as predictors of a standardized readingtest. The various cloze deletion methods included: 1-5 dele-tion, noun-verb deletion and multiple-choice deletion. Thesecond test consisted of seven passages which were ranked bycloze and various readability formulas. The third instrumentwas a cloze opinion survey.
Data taken from these tests were treated statistically bythe use of simple correlation, stepwise regression analysis,and canonical correlation.
Conclusions: A positive and significant relationship was foundbetween cloze 1-5 deletion and the subtests, Paragraph Meaningand Word Meaning, taken from the Stanford Achievement Test. Inaddition, multiple regression analysis indicated that 1-5 dele-tion was also a significant predictor of the same subtests. Thecanonical correlation gave added weight to the use of the 1-5
223.
'cloze deletion technique. The implication of these findingsis that in this population, or in ones similar, the 1-5 clozedeletion may predict certain reading abilities as well asstandardized testing.
A rank order correlation between the ranking of seven pas-sages by cloze as compared with rankings using readability for-mulas was not significant. An important finding based on theseresults is the wide differences between the use of cloze andreadability formulas in measuring passage difficulty. Previousresearch has shown the limitations of selecting material forthe illiterate adult based on the use of readability formulas.The selection of the cloze procedure as an alternative seemsto have some basis for its use under these circumstances.
The results of the cloze opinion survey indicated a posi-tive feeling toward the cloze procedure. This result shouldhave some bearing on whether a teacher would want to use clozewith a group of functionally illitc.rate adults. Additional re-search is needed on this question.1
'Dissertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1972), Vol. 32, No. 7, P. 3572-A. (Order No. 72-2535,157 pages).
222
SAUER, FREDA MAE. "THE DETERMINATION OF READING INSTRUCTIONALLEVEL OF DISABLED FOURTH GRADE READERS UTILIZING CLOZETESTING PROCEDURE." UNPUBLISHED ED.D. DISSERTATION, OKLA-HOMA STATE UNIVERSITY, 1969.
Purpose: The primary objective of this study was to determinewhether or not Cloze Testing is a valid procedure to determinethe instructional grade level of fourth grade disabled readers.This study was one of four, all independent in nature, utiliz-ing the same population to obtain diagnostic information aboutthe reading skills of this particular population.
Principles, Concepts, or Procedures: Data were utilized froma total of 191 fourth grade children selected from the 34 par-ticipating schools, both rural and urban, in Kay County, Okla-homa. The Cloze Test was also administered to entire hetero-geneous classrooms from the same community, and included 454children.
A Cloze Test of original materials was prepared specifi-cally for this study. Three stories were composed, sequentialin nature, of approximately 200 words per page with the first,second, and third pages written respectively at the 2.0, 2.5,and 3.0 madability levels as measured by the Spache readabil-ity formula.
The Reading subtests of the Stanford Achievement Test,Primary II, Form W, were administered to those pupils whoseteachers' evaluation placed them in the lower one-third oftheir class in reading achievement. The Standard Reading. In-ventory, Form B, was administered to randomly selected pupilswith Stanford ReadinE Test scores from either test ranging be-tween 2.0 and 4.5. Those pupils whose Standard Readin Inven-tory, test scores (evaluated by three reading cliniciansplaced them in the instructional grade level range of 1.5 to3.9 were selected to comprise the population for this particu-lar study.
Statistical Procedure: The procedure for statisticalanalysis was antecedent probability and psychometric cuttingscores and signs. Two conditions for demonstration of prac-tical efficiency of the Cloze instrument were specified. Match-ing membership scores were the concern.
223
Conclusions: The statistical procedure consistently rejectedthe hypotheses that cloze technique is a valid procedure todetermine the instructional reading grade level of fourthgrade disabled readers. The one exception is to be viewedwith skepticim due to the small population.
Recommendations pointed toward cloze research that wouldanalyze the specific kinds of incorrect responses that aremade by disabled readers to give guidelines for reading com-prehension instruction. It was further recommended that testsbe designed that measure the absolutes of a specific readingskill.1
1Dissertation Abstracts International A: The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1971), Vol. 31, No. 8, p. 406-A. (Order No. 70-21,474, 153 pages).
224
SCHNEYER, J. WESLEY. "USE OF THE CLOZE PROCEDURE FOR IMPROV-ING READING COMPREHENSION." THE READING TEACHER, 19(1965), 174-179.
Purpose: The author reports a study undertaken to investi-gate the effects of the cloze procedure upon the reading com-prehension of sixth grade pupils.
Principles, Concepts or Procedures: A series of 200-wordpassages were chosen by the author from a basal readingseries. Two types of exercises were prepared. One utilizedevery-tenth-word deletions regardless of word class while theother deleted noun and verbs only. Twenty deletions weremade in each passage. Three exercises of each type were pre-pared at each reader levels a total of 57 exercises.
There were 32 subjects in the experimental group and 34in the control group. Measures of reading achievement andintelligence were given through the Gates Reading Survey andthe California Test of Mental Maturity.
Conclusions.'
1. The experimental group did not show significantlygreater improvement in reading comprehension.
2. Scores from both cloze test forms were significantlyrelated to reading comprehension, vocabulary, word recogni-tion, and verbal ability.
The author accounts for the lack of improvement in theexperimental group to the absence of discussion followingscoring.
225
SCHOELLES, IRIS STOREY. "CLOZE AS A PREDICTOR OF READING GROUPPLACEMENT." PAPER PRESENTED AT THE ANNUAL MEETING OF THEINTERNATIONAL READING ASSOCIATION, ATLANTIC CITY, NEW JER-SEY, APRIL 19-23, 1971.
Purpose: Four questions concerning the relationship between re-sults of cloze test scores and individual reading ability areinvestigated: (1) Is there a relationship between the cloze re-sults and individual reading ability? (2) Should cloze testsbe written at mean grade level? (3) Which provides higher cor-relations--giving credit for exact responses only or for exactresponses and synonyms? and (4) Can the cloze test be considereda valuable part of student reading assessment?
Principles, Concepts, or Procedures: Four hundred and seventystudents in grades 1 through 5 at a Snyder, New York, elemen-tary school were administered both the Stanford Achievement Testand cloze passages from basal readers. Correlation of thescores ranged from .64 to .76 using cloze material near thegrade mean. It appeared that greater dispersion.of the popula-tion is achieved through use of more difficult materials.Scoring synonyms and exact words was found desirable.
Conclusions: It was concluded that advantages of the cloze testare (1) that the student must comprehend what he is reading todo well, while random guessing can lead to high scores on theStanford test and (2) there is no time limit on the cloze, hencethe slow, careful reader scores well while the fast raidomguesser does not. Tables and references are included.1
1Research in Education (Washington, D.C.: "nited StatesGovernment Printing Office, 1971), Vol, 6, No. 12, p. 74.(ED 053 868, 6 pages).
226
SMITH, ELMER L. "USE OF THE CLOZE PROCEDURE IN IMPROVING COM-PREHENSION OF JUNIOR COLLEGE READERS." PAPER PRESENTEDAT THE NATIONAL READING CONFERENCE, ATLANTA, GEORGIA,DECEMBER 4-6, 1969.
Purpose: Uses of the cloze procedure for both diagnosis andfor practice exercises for improving reading comprehension ina junior college reading program are described.
Principles, Concepts, or Procedures: Every tenth conceptword--nouns, verbs, adjectives, and adverbs--was omitted fromexpository prose selections of about 350 words each, and thereader was asked to write the missing word in the blank pro-vided. Following completion of the writing, the teacher andstudents reviewed the responses orally. An additional proce-dure was added in which the students looked over the clozeexercise before doing it and noted unfamiliar words.
Conclusions: Use of the cloze procedure was found to be par-tilularly effective for (1) demonstrating the process of com-prehension--how words combine with words into wider units ofmeaning, (2) demonstrating the part grammatical knowledgeplays in comprehension, (3) pointing out to students theirown deficiencies in the comprehension process, and (4) pro-moting discussion about particJlar reading selections andabout the process of reaqoning which is reading comprehension.References are included.'
'Research in Education (Washington, D.C.: United StatesGovernment Printing Office, 1970), Vol. 5, No. 9, p. 77.(ED 039 091, 9 pages).
227
STEWART, ELNEITA WALLACE. "A COMPARATIVE STUDY OF THE EFFEC-TIVENESS OF CLOZE AND TEXTBOOK PROCEDURES IN A COLLEGEREADING PROGRAM." UNPUBLISHED ED.D. DISSERTATION, UNL..VERSITY OF HOUSTON, 1967,1
Purpose: This study was designed to answer the following ques-tions:
1. Do methods of teaching reading in which cloze proce-dures are used result in better total reading achievement, com-prehension and vocabulary than methods in which reading text-books are employed?
2. Is growth in total reading achievement, comprehensionor vocabulary affected by the age of the learner?
3. Is growth in total reading achievement, comprehensionor vocabulary affected by I.Q.?
Principles, Concepts, or Procedures: The experimental methodwas used to collect the data. Eighty-nine high school gradu-ates who were enrolled in an eight-week English-MathematicsInstitute which included a reading program were randomlyassigned to two comparison groups.
The 47 students in the experimental group used cloze read-ing exercises as their chief learning material. The cloze se-lections were prepared by deleting every fifth word excludingproper nouns, dates and numbers. The selections were takenfrom the same college reading textbook that the control groupused. One illustrated lecture on using one's knowledge ofsyntax as an aid to reading comprehension was given to the ex-perimental group. The 44 students in the control group used acollege reading textbook as their chief learning material.
Both cloze and textbook groups met twice per week inclass periods of 55 minutes in length and were taught by thesame instructor.
Con,:lusions: The findings were as follows:1. The hypothesis that the 47 students who used cloze
1See also Elneita Wallace Stewart, "Reading ImprovementProgram for College Students," Appendix.
228
reading materials would experience more growth in readingachievement than the 44 students who used the textbook as thechief learning material was rejected at the five per centlevel of confidence.
2. Reading achievement was not affected by age.3. Reading achievement was not significantly affected by
I.Q.
Based on the findings, it was concluded that the clozeprocedure was not significantly better than the textbook method.Neither did it prove to be less effective. The small differ-ences in gain in reading achievement favored the cloze group.
In view of the findings, as well as strong support fromlearning theory, it was recommended that further researchshould be done on the cloze procedure as a method of teachingreading to college students. It was further recommended thatthe hypotheses of this study should be experimentally retestedwith the following alterations in design and procedure:
1. Use matched pairs as subjects.2. Use cloze reading materials that are designed to en-
hance conceptual thinking.3. Use cloze reading materials over a longer period of
time.4. Give subjects in the cloze group more structured in-
struction on reading comprehension and structural linguistics.
1Dissertation Abstracts A: The Humanities and SocialSciences (Ann Arbor, Michigan: University Microfilms, 1968),Vol. 28, No.. 12, Part 1. (Order No. 68-7862, 117 pages).
229
WIECHELMAN, DUANE SYLVESTER. "A COMPARISON OF CLOZE PROCEDURESCORES AND INFORMAL READING INVENTORY RESULTS FOR ESTI-MATING FUNCTIONAL READING LEVELS FOR STUDENTS AT EIGHTHGRADE LEVEL." UNPUBLISHED ED.D. DISSERTATION, UNIVERSITYOF NORTHERN COLORADO, 1971.
Purpose: The purpose of this study was to compare the func-tional reading levels identified with a cloze test with thefunctional levels identified with an Informal Reading Inventoryfor eighth grade students.
Four major hypotheses were formulated and stated in thenull form for the purposes of this study. They were as follows:
Hypothesis 1: There will be no .dgnificant relationshipbetween the reading grade levels attained by eighth gradestudents on a cloze test and on an Informal Reading In-ventory for identifying the following:
a) independent reading levelsb) instructional reading levelsc) frustration reading levels
Hypothesis 2s There will be no significant differencesbetween the mean scores attained by eighth grade studentson a cloze test and on an Informal Reading Inventory foridentifying the following:
a) independent reading levelsb) instructional reading levelsc) frustration reading levels
Hypothesis 3: There will be no significant difference be-tween the mean instructional reading levels attained on acloze test and on an Informal Reading Inventory by eighthgrade students who are reading as follows:
a) at grade placement 7.5 or aboveb) below grade placement 7.5
Hypothesis 4: There will be no significant difference be-tween the mean instructional reading levels attained byeighth grade Spanish surnamed students on a cl-ze test andon an Informal Reading Inventory.
230
Princiles, Concepts, or Procedures: There were seventy-oneeighth grade students included in this study, thirteea of whomwere Spanish surnamed. A test utilizing the cloze techniquewas developed for the study. Cloze criteria for functionalreading levels were as follows: Independent Reading Level--57per cent correct responses; Instructional Reading Level--44per cent but less than 57 per cent correct responses; and,Frustration Reading Level--less than 35 per cent correct re-sponses. An IRI was individually administered to the subjects.Betts' criteria was used to identify functional reading levels.
Correlations were computed between the functional readinglevels obtained with the cloze test and the IRI. The PearsonProduct-moment measurement was used. Means were computed forthe functional reading levels obtained with the cloze test andthe IRI. The mean instructional reading levels obtained withthe cloze test and the IRI were computed for selected subgroups.The t test for related samples were used to determine the sig-nificance of the differences between the means. The .01 levelof confidence was used as the criterion for significance.
For descriptive purposes the individual students' func-tional reading levels identified with the cloze test and theIRI were compared. In addition the reading grade level scoresobtained with the Durrell Listening-Reading Series, a standard-ized test, were compared with the instructional reading levelsobtained with the cloze test and the IRI.
Conclusions: On the basis of the findings and the limitationsimposed by the study, the following conclusions were drawn:
1. There was a positive relationship between the functionalreading levels identified with the cloze test and the IRI.
2. Mean functional reading levels identified through theuse of the cloze test for the eighth grade students did approxi-mate their attained mean functional reading levels through theuse of the IRI.
3. Even though the variability was greater, the mean in-structional reading levels obtained using the cloze test forthe eighth grade students reading at grade level 7.5 or above,and below grade level 7.5 did approximate the mean instructionalreading levels obtained using the IRI for eighth grade studentsreading at grade level 7.5 or above, and below grade level 7.5.
231
4. The mean instructional reading level for the eighthgrade Spanish surnamed students which was obtained using thecloze test did approximate their mean instructional readinglevel obtained using the IRI.
An analysis of the functional reading levels attained byindividual students through the use of the cloze test and theIRI revealed that for all comparisons made, there were greatervariabilities among the cloze test results than among the IRIresults. When compared with the IRI, the cloze test identifiedindividual students' instructional reading levels more accu-rately than did the Durrell Listening-Reading, Test.1
'Dissertation Abstracts International As The Humanitiesand Social Sciences (Ann Arbor, Michigan: University Micro-films, 1972), Vol. 32, No. 7, pp. 3583-A, 3584-A. (Order No.72-3317, 205 pages).
APPENDIX
SECTION I
Suggested Readings Related to
"THE CLUE PROCEDURE: AN OVERVIEW"
233
1. Bickley, A. C.; Ellington, Billie J.; and Bickley, RachelT. "The Cloze Procedure: A Conspectus." Journalof Reading Behavior, 2 (1970), 232-249.
2. Green, Richard T. Comprehension in Reading: An Anno-tated Bibliography. Newark, Delaware: Internation-al Reading Association, 1971, 22 pages.
3. Greene, Frank P., et al. "Cloze Symposium." Multidis-ciplinary Aspects of College-Adult Reading. Editedby G. B. Schick and M. M. May. Seventeenth Year-book of the National Reading Conference. Milwaukee,Wisconsin: National Reading Conference, 1967, 110-131.
4. Hafner, Lawrence E. "Cloze Procedure." Journal of Read-ing, 9 (1966), 415-421.
5. Mork, Theodore A. "Clozing the Placement Gap: A NewTool for Administrators and Teachers." EducationalLeadership, 28 (April 1971), 763-767.
6. Potter, Thomas C. A Taxonomy of Cloze Research, Part I.Readability and Reading Comprehension. Inglewood,California: Southwest Regional Educational Labora-tory, 1968, 51 pages.
7. Rankin, Earl F., Jr. "The Cloze Procedure - A Survey ofResearch." The Philosophical and SociologicalBases of Reading. Edited by Eric L. Thurston andLawrence E. Hafner. Fourteenth Yearbook of theNational Reading Conference. Milwaukee, Wisconsin:National Reading Conference, 1964, 133-150.
234
8. Taylor, Wilson L., and Waldman, Ivan L. "Latency andFocus Methods of Cloze Quantification." Reading:Process and Pedagogy - Volume II. Edited by G. B.Schick and M. M. May. Nineteenth Yearbook of theNational Reading Conference. Milwaukee, Wisconsin:National Reading Conference, 1970, 241-262.
9. Weintraub, Samuel. "The Cloze Procedure." The ReadingTeacher, 21 (1968), 567-571, 607.
Suggested Readings Related to
"THE CLOZE PROCEDU'" 'S A MEASURE OF READABILITY"
1. Bormuth, John R. "Clr as a Measure of Readability."Reading as an In. ?ectual Activity. Edited by J.Allen Figurel. P- !eedings of the annual conventionof the Internati:, Reading Association, New York,N.Y.: Scholastic .magazines, 1963, 131-134.
2. . "Designs of Readability Research." Vistas inReading. Edited by J. Allen Figurel. Proceedingsof the annual convention of the International Read-ing Association. Newark, Delaware: InternationalReading Association, 1966, 485-489.
. The Implications and Use of the Cloze Procedurein the Evaluation of Instructional Programs. LosAngeles: Center for the Study of Evaluation of In-structional Programs, University of California, 1967,55 pages.
"Cloze Test Readability: Criterion ReferenceScores." Journal of Educational Measurement, 5(1968), 189-196.
5. Coleman, E. B. "Experimental Studies of Readability,Part II." Elementary English, 45 (1968), 316-324.
6. Gallant, Ruth. "Use of Close Tests as a Measure ofReadability in the Primary Grades." Reading andInauirY. Edi4;ed by J. Allen Figurel. Proceedingsof the annual convention of the International Read-ing Association. Newark, Delaware: InternationalReading Association, 1965, 286-287.
235
7. Jefferson, George L., Jr. "Lexical and Structural Itemsas Predictors of Readability for High and Low Abil-ity Readers." PIper presented at the National Read-ing Conference, Atlanta, Georgia, December 4-6,1969, 12 pages.
8. Kingston, Albert J., and Weaver, Wendell W. "Recent De-velopments in Readability Appraisal." Journal ofReading, 11 (1967), 44-47.
9. Klare, George R. The Measurement f ReadabilitY. Ames,Iowa: Iowa State University Press, 19 3, 84-85.
10. . "Comments on Bormuths Readability: A NewApproach." Reading Research Quarterly, 1 (1966),119-125.
11. MacGinitie, Walter H. "Contextual Constraint in EnglishProse Paragraphs." Journal of Psychology, 51 (1961),121-130.
12. , and Tretiak, Richard. "Sentence Depth Mea-sures as Predictors of Reading Difficulty." Read-ing Research Quarterly, 6 (1971), 364-377.
13. McLeod, John, and Anderson, Jonathan. "ReadabilityAssessment aril Word Redundancy of Printed English."Psychological Reports, 18 (1966), 35-38.
14. Rothkopf, Ernst Z. "Textual Constraint as Function ofRepeated Inspection." Journal of Educational Psy-chology.. 59 (1968), 20-25.
15. Ruddell, Robert B. "A Study of the Cloze Procedure Tech-nique in Relation to Structurally Controlled Read-ing Material." Improvement of Reading Through Class-room Practice. Edited by J. Allen Figurel. Pro-ceedings of the annual convention of the Internation-al Reading Association. Newark, Delaware: Inter-national Reading Association, 1964, 298-303.
236
Suggested Readings Related to
"THE GAZE PROCEDURE AS A MEASURE OF COMPREHENSION"
1. Aaron, Robert L., and White, William F. "Racial Effectin Reading Comprehension." Paper presented at theAmerican Educational Research Association Conference,Chicago, February 6-10, 1968, 5 pages.
2. Aquino, Milagros, et al. "Reading Comprehension Difficul-ty as a Function of Content Area and Linguistic Com-plexity." Journal of Experimental Education, 37(1969), 1-4.
3. Bormuth, John R. Relationships Between Selected LanguageVariables and Comprehension Ability and Difficulty.Washington, D. C.: United States Department ofHealth, Education; and Welfare, 1964, 131 pages.
. "The Effectiveness of Current Procedures forTeaching Reading Comprehension." Paper presentedat the annual convention of the National Council ofTeachers of English, Milwaukee, Wisconsin, November30, 1968.
5. Feldman, Margaret E. "Learning by Programmed TextFormat at Three Levels of Difficulty." Journal ofEducational Psycholoev, 56 (1965), 133-139.
6. Geyer, James R. Cloze Procedure as a Predictor of Com-prehension in Secondary Social Studies Materials.Olympia, Washington: Washirgton State Board forCommunity College Education, 1968, 10 pages.
7. Greene, frank P. "Modification of the Cloze Procedureand Changes in Reading Test Performances." Journalof Educational Measurement, 2 (1965), 213-217.
8. Hater, Mary Ann, and Kane, Robert B. The Cloze Proce-dure as a Measure of the Reading Comprehensibilityand Difficulty of Mathematical English. Lafayette,Indiana: Purdue University, 1970, 25 pages.
9. Jenkinson, Marion E. "Zelected Processes and Difficul-ties in Reading Comprehension." Unpublished doc-toral dissertation, University of Chicago, 1957.
237
10. Knight, David, and Alcorn, John D.' "Comparison of thePerformance of Educationally Disadvantaged Adultsand Elementary Children on Selected Measures ofReading Performance." Reading Process and Pedagogy.Volume II. Edited by G. B. Schick and M. M. May.Nineteenth Yearbook of the National Reading Con-ference. Milwaukee, Wisconsin: National ReadingConference, 1970, 113-117.
11. Musgrave, B. "The Effect of Who and What Context onCloze and Commonalty Scores." Journal of SocialPsychology, 59 (1963), 185-192.
12. Rankin, Earl F., and Culhane, Joseph W. "One PictureEquals 1000 Words?" Reading Improvement, 7 (1970),37-40.
13. Ruddell, Robert P. "The Effect of Oral and WrittenPatterns of Language Structure on Reading Compre-hension." The Reading Teacher, 18 (1965), 270-275.
14. Simons, Herbert D. "Reading Comprehension: The Needfor a New Perspective." Reading Research Quarterly,6 (1971), 338-363.
Suggested Readings Related to
"THE CLOZE PROCEDURE AS A MEASURE OF LANGUAGE FACILITY"
1. Carroll, J. B. "Vectors of Prose Style." Style in Lan-guage. Edited by Thomas A. Seboek. Cambridge,Mass.: John Wiley & Sons, 1960, 283-292.
2. Cohen, Sandra R. "Predictability for Deaf and HearingParaphrases." Journal of Verbal Learning and Ver-bal Behavior, 6 (1967), 916-921.
3. Darnell, Donald K. The Development of an English Lan-guage Proficiency Test of Foreign Students. Usinga Clozentrophy Procedure. Denver, Colorado: De-partment of Speech and Drama, University of Colo-rado, 1968, 73 pages.
238
4. Deutsch, Martin, et al. Communication of Informationin the Elementary Classroom. New York, N.Y.: De-partment of Psychiatry, New York Medical College,1964, 238 pages.
5. Dickens, Milton, and Williams, Frederick. "An Experi-mental Application of 'Cloze' Procedure and Atti-tude Measures to Listening Comprehension." SpeechMonographs, 31 (1964), 103-108.
6. Donahew, L. "Decoder Behavior on Incongruent PoliticalMaterial - A Pilot Study." Journal of Communica-tion, 16 (1966), 133-142.
7. Feldstein, S., and Jaffe, J. "Language Predictabilityas a Function of Psychotherapeutic Interaction."Journal of Consulting Psychology, 27 (1963), 123 -126.
8. Fillenbaum, Samuel; Jones, Lyle V.; and Rapaport, A."Predictability of Words and Their GrammaticalClasses as a Function of Rate of Deletion from aSpeech Transcript." Journal of Verbal Learninand Verbal Behavior, 2 19 3 , 18 -19 .
9. Goldman-Eisler, F. "The Predictability of Words inContext and the Length of Pauses in Speech."Journal of Communication, 11 (1961), 95-99.10. Jenkins, James J. "Commonality of Association as anIndicator of More General Patterns of Verbal Be-havior." Style in Language. Edited by Thomas A.Seboek. Cambridge, Mass.: John Wiley and Sons,1960, 307-329.
11. King, David J., and Cofer, Charles N. "ExploratoryStudies of Stories Varying in the Adjective-VerbQuotient." Journal of General Psychology, 62(1960), 199-221.
12. Marshall, William A. "Investigating the Effect atFourth Grade Reading Level Using the Cloze Proce-dure." American Annals of the Deaf, 115 (1970),682-689.
239
13. Odom, Penelope B.; Blanton, Richard L.; and Nunnally,Jura C. "Some Cloze Technique Studies of LanguageCapability of the Deaf." Journal of Speech andHearing Research, 10 (1967), 816-827.
14. Osgood: Charles E. "Some Effects of Motivation orStyle of Encoding." Style in Language. Edited byThomas A. Seboek. Cambridge, Mass.: John Wiley &Sons, 1960, 293-306.
15. , and Walker, Evelyn G. "Motivation and Lan-guage Behavior: A Content Analysis of SuicideNotes." Journal of Abnormal and Social Psychology.59 (1959), 58-67.
16. Pisoni, David B. Communicability of Verbal Behavior inSchizophrenic Patients and Its Relationship to Cur-rent Psycholineuistic Theory. Ann Arbor, Michigan:Center for Research on Language and Language Be-havior, University of Michigan, 1969, 15 pages.
17. Rosenberg, Sheldon. Associative Facilitation in theRecall of Connected Discourse. Ann Arbor, Michi-gan: Center for Research, Language, and Behavior,University of Michigan, 1967, 31 pages.
18. . Recall of Noun Pairs Embedded in ConnectedDiscourse as a Function of Association Strength.Ann Arbor, Michigan: Center for Research, Lan-guage, and Behavior, University of Michigan, 1967,9 pages.
19. Rothkopf, Ernst Z. "Learning from Written Sentences:Within Sentence Order the Acquisition of NameClause Equivalences." Journal of Verbal Learningand Verbal Behavior, 2 T19.3), 470-475.
20. Salzinger4 Kurt; Portnoy, Stephanie; and Feldman,Richard S. "The Effect of Order of Approximationto the Statistical Structure of English on theEmission of Verbal Responses." Journal of Experi-mental Psychology, 64 (1962), 52-57.
21. m "Verbal Behavior of Schizophrenic and NormalSubjects." Annals of the New York Academy ofScience, 105 (1964), 845-860.
240
22. Spolsky, Bernard. "What Does It Mean to Know a Language,or How Do You Get Someone to Perform His Compe-tence?" Paper prepared for the Second Conferenceon Problems in Foreign Language Testing, Universityof Southern California, November 7-9, 1968, 23pages.
23. Tannenbaum, Percy; Williams, Frederick; and Clark, RuthAnne. "Effects of Grammatical Information on WordPredictability." Journal of Communication, 19(1969), 41-48.
24. Weaver, Wendell W. "The Predictability of Omissions inReading and Listening." Problems Programs, andProjects in College-Adult Reading. Edited byEmery P. Bliesmer and Ralph Staiger. EleventhYearbook of the National Reading Conference. Mil-waukee, Wisconsin: National Reading Conference,1961, 148-153.
25. . "The Word as the Unit of Language." Journalof Reading, 10 (1967), 262-268.
26. g and Bickley, A. C. "The Retrieval of Learn-ing Sets by the External Display of Reading Ma-terial." Junior College and Adult Reading
Expanding Edited by G. B. Schickand M. M. May. Sixteenth.Yearbook of the NationalReading Conference. Milwaukee, Wisconsin: Nation-al Reading Conference, 1966, 240-261.
Suggested Readings Related to
"THE CLOZE PROCEDURE AS AN INSTRUCTIONAL TECHNIQUE"
1. Bloomer, Richard H., et al. Non-Overt Reinforced ClozeProcedure. Washington, D.C.: United States De-partment of Health, Education, and Welfare, 1966,152 pages.
2. Bormuth, John R. "Empirical Determination of the In-structional Reading Level." Reading and Realism.Edited by J. Allen Figurel. Proceedings of theannual convention of the International ReadingAssociation. Newark, Delaware: InternationalReading Association, 1968, 716-721.
241
3. Guice, Billy M. "The Use of the Cloze Procedure for Im-proving Reading Comprehension of College Students."Journal of Reading Behavior, 1 (1969), 81-92.
4. Hefner, Lawrence E. "A One-Month Experiment in TeachingContext Aids in Fifth Grade." Journal of Education-al Research, 58 (1965), 472-474.
5. Heitzman, Andrew J., and Bloomer, Richard H. "TheEffect of Non-Overt Reinforced Cloze Procedure UponReading Comprehension." Journal of Reading, 11(1967), 213-223.
6. Kirby, Clara L. "Using the Cloze Procedure as a TestingTechnique." Reading Diagnosis and Evaluation.Edited by Dorothy L. DeBoer. Proceedings of theThirteenth Annual Convention of the InternationalReading Association. Newark, Delaware: Inter-national Reading Association, 1970, 68-77.
7. McLeod, John, and Anderson, Jonathan. "An Approach toAssessment of Reading Ability Through InformationTransfer." Journal of Reading Behavior, 2 (1970),116-143.
8. Ransom, Peggy E. "Determining Reading Levels of Elemen-tary School Children by Cloze Teing." ForgingAhead in Reading. Edited by J. 4.1en Figurel.Proceedings of the annual convention of the Inter-national Reading Association. Newark, Delaware:International Reading Association, 1967, 477-482.
9. Stewart, Elneita Wallace. "Reading Improvement Programfor College Students." Reading: Process and Peda-imEC- Volume II. Edited by G. B. Schick and M. M.May. Nineteenth Yearbook of the National ReadingConference. Milwaukee, Wisconsin: National Read-ing Conference, 1970, 202-207.
242
SECTION II
SUPPLEMENTARY READINGS
1. Bennett, Stanley. "The Cloze Procedure: A Review ofthe Literature." Studies of Language and LanguageBehavior, 4 (1967), 414-428.
2. Carroll, J. B.; Carton, A. S.; and Wilds, C. P. An In-vestigation of ' Cloze' Items in the Measurement ofAchievement in Foreign Languages. Cambridge,Mass.: Laboratory for Research in Instruction,Graduate School of Education, Harvard University,1959, 138 pages.
3. Dulin, Kenneth L. "Measuring the Difficulty of ReadingMaterials." Reading Improvement, 8 (1971), 36.
4. Kingston, Albert J., and Weaver, Wendell W. Feasibilityof Cloze Technique for Teaching and Evaluating Cul-turally Disadvantaged Beginning Readers. Athens,Georgia: Research and Development Center, Univer-sity of Georgia, 1970.
5. Klare, George R.; Sinaiko, H. W.; and Stolurow, L. M.The Cloze Procedure: A Convenient Readability Testfor Training Materials and Translations." Arling-ton, Virginia: Institute for Defense Analyses,Science and Technology Division, 1971.
6. Roosinck, Pearl L. "A Method for Improving ReadingComprehension." Unpublished master's thesis, Uni-versity of Michigan, 1962.
7. Shopshire, T.. "An Investigation of the RelationshipBetween Some Properties of Closure and Reading Per-formance." Unpublished doctoral dissertation,Syracuse University, 1955.
8. Sukiori, S. "A Study of Readability Measurement Appli-cations of Cloze Procedure to Japanese Language."(English Abstract). Japanese Journal of Psynhol-2gy, 1957, 28.
243
9. Taylor, Wilson L. The Cloze Procedure: How It PredictsComprehension and Intelligence of Military Person-nel. Urbana, Illinois: Division of Communications,University of Illinois, 1953, 22 pages.
10. . "KM Readers Lend Hand to Science: ClozeMethod Works in Written Korean and May Serve as aTool for Korean Language Reform." Korean Messenger,3 (1954), 4-5.
11. Torrey, Jane. "The Learning of Grammatical Patterns."Journal of Verbal Learning and Verbal Behavior, 8(1969), 360 -368.
12. Weaver, Wendell W.; White, William F.; and Kingston,Albert J. "Affective Correlates of Reading Compre-hension." The Journal of Psychology, 58 (1968),87-95.
rya
245
BIBLIOGRAPHY
THE CLOZE PROCEDURE: AN OVERVIEW
1. Anderson, Jonathan. "A Report of Research on Comprehen-sion in Reading." Paper presented at the Third In-ternational Reading Association World Congress onReading, Sydney, Australia, August 7-9, 1970.
2. Bormuth, John R. "Experimental Applications of ClozeTests." Improvement of Reading Through ClassroomPractice. Edited by J. Allen Figurel. Proceedingsof the International Reading Association. Newark,Delaware: International Reading Association, 1964,303-306.
. "Validities of Grammatical and SemanticClassifications of Cloze Test Scores." Reading andInquiry. Edited by J. Allen Figurel. Proceedingsof the International Reading Association. Newark,Delaware: International Reading Association, 1965,283-286.
4. Culhane, Joseph W. "Cloze Procedures and Comprehension."The Reading Teacher, 23 (1970), 410-413, 464.
5. Hefner, Lawrence E. "Relationships of Various Measuresto the 'Cloze'." New Concepts in College-AdultReading. Edited by Eric L. Thurston and Lawrence E.Hafner. Thirteenth Yearbook of the National Read-ing Conference. Milwaukee, Wisconsin: NationalReading Conference, 1963, 135-145.
6. "Implications of Cloze." The Philosophicaland Sociological Bases of Reading. Edited by EricL. Thurston and Lawrence E. Hafner. Fourteenth Year-book of the National Reading Conference. Milwaukee,Wisconsin: National Reading Conference, 1965, 151-158.
7. Ohnmacht, Fred; Weaver, Wendell W.; and Kohler, Emmett T."Cloze and Closures A Factorial Study." The Jour-nal of Psychology, 74 (1970), 205-217.
246
8. Rankin, Earl F., Jr. "The Cloze Procedure - Its Valid-ity and Utility." Starting and Improving CollegeReading Programs. Edited by Oscar S. Causey andWilliam Eller. Eighth Yearbook of the NationalReading Conference. Milwaukee, Wisconsin: Nation-al Reading Conference, 1959, 131-144.
9. . "Closure and the Cloze Procedure." Collegeand Adult Reading. Edited by David M. Wark. Con-ference Proceedings of the North Central ReadingAssociation. Minneapolis, Minnesota: Universityof Minnesota, 1964, 54-84.
10. . "Research Design and the Cloze Procedure."Vistas in Reading. Edited by J. Allen Figurel.Proceedings of the International Reading Associa-tion. Newark, Delaware: International ReadingAssociation, 1966, 489-491.
11. Taylor, Wilson L. "Recent Developments in the Use of'Clone Procedure." Journalism Quarterly, 33(1956), 42-48.
12. "Relative Influence of Preceding, Following,and Surrounding Contexts on Cloze Performance."Paper presented at the Annual Meeting of the Na-tional Reading Conference, Tampa, Florida, Decem-ber 3, 1971.
13. Tuinman, J. Jaap, and Blanton, B. Elgit. The ClozeProcedure: Corollaries of a Search Hypothesis.Bloomington, Indiana: School of Education, Uni-versity of Indiana, 1971.
14. Weaver, Wendell W. "Theoretical Aspects of the ClozeProcedure." The Philosophical and SociologicalBases of Reading. Edited by Eric L. Thurston andLawrence E. Hafner. Fourteenth Yearbook of theNational Reading Conference. Milwaukee, Wisconsin:National Reading Conference, 1965, 115-132.
247
THE CLOZE PROCEDURE AS A MEASURE OF
READABILITY
15. Aborn, Murray; Rubenstein, Herbert; and Sterling, Theo-dor D. "Sources of Contextual Constraint UponWords in Sentences." Journal of Experimental Psy-chology, 57 (1959), 171-180.
16. Aquino, Milagros. "The Validity of the Miller-ColemanReadability Scale." Reading Research Quarterly, 4(1969), 342-357.
17. Beard, Jacob G. "Comprehensibility of High SchoolTextbooks: Association With Content Area." Jour-nal of Reading, 11 (1967), 229-234.
18. Blake, James Herbert. "Cloze Scores, Agreement Scores,and Approximated Equivalence Scores as Indices ofContextual Constraint." Unpublished Ph.D. disser-tation, Syracuse University, 1970
19. Bormuth, John R. "Optimum Sample Size and Cloze TestLength in Readability Measurement." Journal ofEducational Measurement, 2 (1965), 111-116.
20. . "Readability: A New Approach." Reading Re-search Quarterly, 1 (1966), 79-131.
21. "New Data on Readability." Forging. Aheadin Reading. Edited by J. Allen Figurel. Proceed-ings of the International Reading Association.Newark, Delaware: International Reading Associa-tion, 1967, 488-492.
22. . Cloze Readability Procedure. Los Angeles:Center for the Study of Evaluation of Instruction-al Programs, University of California, 1967.
23. and MacDonald, O. L. "Cloze Tests as aMeasure of Ability to Detect Literary Style."Reading and Inouirv. Edited by J. Allen Figurel.Proceedings of the International Reading Associa-tion. Newark, Delaware: International ReadingAssociation, 1965, 287-290.
214.8
24. Bormuth, John Robert. "Cloze Tests a', Measures ofReadability and Comprehension Ability." Unpub-lished Ed.D. dissertation, Indiana University,1962.
25. Covington, Richard John Lee. "An Analysis of Readabil-ity of Third and Fourth Grade Modern MathematicsTextbooks Using the Cloze Procedure." UnpublishedEd.D. dissertation, University of California, LosAngeles, 1966.
26. Elley, Warwick B. "The Assessment of Readability byNoun Frequency Counts." Reading Research_guarter-lx, 4 (1969), 411-427.
27. Froelich, Donald Max. "A Comparison of Two Methods ofAssessing Textbook Readability of Selected CollegeLevel Electronics Textbooks." Unpublished Ed.D.dissertation, University of Missouri - Columbia,1970.
28. Froese, Victor. "Cloze Readability Versus the Dale-Chall Formula." Paper presented at the AnnualMeeting of the International Reading Association,Atlantic City, New Jersey, April 19-23, 1971.
29. Fry, Edward B. The Readability Graph Validated atPrime Levels. Cambridge, Massachusetts: Compu-ter Assisted Instruction Laboratory, GraduateSchool of Education, Harvard University, 1969.
30. Gallant, Ruth Margaret Frances. "An Investigation ofthe Use of Cloze Tests as a Measure of Readabilityof Materials for the Primary Grades." UnpublishedEd.D. dissertation, Indiana University, 1964.
31. Houska, Joseph Thomas. "The Efficacy of the ClozeProcedure as a Readability Tool on Technical Con-tent Material as Used in Industrial Education atthe High School Level." Unpublished Ed.D. disser-tation, University of Illinois at Urbana - Champaign,1971.
32. Jefferson, George Lee, Jr. "Lexical and StructuralItems as Predictors of Readability for High andLow Ability Readers." Unpublished Ph.D. disser-tation, University of Georgia, 1969.
249
33. Kulm, Gerald. "Measuring the Readability of ElementaryAlgebra Using the Cloze Technique." Paper pre-sented at the Annual Meeting of the American Educa-tional Research Association, New York, N. Y., Feb-ruary 4-7, 1971.
34. Liske, Wilfred Wallace. "The Cloze Procedure for Deter-mining Readability of Encyclopedia Material forElementary School Pupils." Unpublished Ed.D. dis-sertation, University of Maryland, 1968.
35. MacGinitie, Walter Harold. "Contextual Constraint inEnglish Prose." Unpublished Ph.D. dissertation,Columbia University, 1960.
36. Miller, G. R., and Coleman, E. B. "A Set of Thirty-sixProse Passages Calibrated for Complexity." Journalof Verbal Learning and Verbal Behavior, 6 (1967),651-854.
37. Rentel, Victor Michael. "Form Class and Word Lengthas Indices of the Difficulty of Predicting ClozeEntries in a Sample of Continuous Prose." Unpub-lished Ph.D. dissertation, University of SouthCarolina, 1969.
38. Taylor, Wilson L. "' Cloze Procedure': A New Tool forMeasuring Readability." Journalism_guarterly, 30(1953). 414-438.
39. "Application of 'Clone' and Entropy Mea-sures to the Study of Contextual Constraint inSamples of Continuous Prose." Unpublished Ph.D.dissertation, University of Illinois, 1954.
THE CLOZE PROCEDURE AS A MEASURE OF
COMPREHENSION
40. Bloomer, Richard H,, and Heitzman, Andrew J. "Pre-testing and the Efficiency of Paragraph Reading."Journal of Reading, 8 (1965), 219-223.
250
41. Bormuth, John R. "Mean Word Depth as a Predictor ofComprehension Difficulty." California Journal ofEducational Research, 15 (1964), 226-231.
42. . "Comparable Cloze and Multiple-Choice Com-prehension Test Scores." Journal of Reading, 10(1967), 291-299.
43. . "New Measures of Grammatical Complexity."The Psycholinguistic Nature of the Reading Process.Edited by Kenneth S. floodman. Detroit: WayneState University Press, 1968.
44. . "Factor Validity of Cloze Tests as Measuresof Reading Comprehension Ability." Reading Re-search Quarterly, 4 (1969), 358-365.
45. Bradley, Michael. "Effects on Reading Tests of Dele-tions of Selected Grammatical Categories." Paperpresented at the National Reading Conference,Atlanta, Georgia, December 4-6, 1969.
46. Carver, Ronald P. "What Is Reading Comprehension andHow Should It Be Measured?" Paper presented at theNational Reading Conference, Atlanta, Georgia, De-cember 4-6, 1969.
47. Charles, Edward. "An Investigation of the Use ofCloze Tests to Compare Gain Scores of Students inScience Who Have Used Individualiied Science Ma-terials an0 Those Who Have Used Traditional Text-book Materials." Unpublished Ed.D. dissertation,Lehigh University, 1970.
48. Coleman, E. B. "Improving Comprehensibility by Short-ening Sentences." Journal of Applied Psychology,46 (1962), 131-134.
49. and Miller, G. R. "A Measure of InformationGained During Prose Learning." Reading ResearchQuarterly, 3 (1968), 369-386.
50. Cranney, Adelbert Garr, Jr. "A Comparison of a Free-Response and Multiple-Choice Cloze Reading Test."Unpublished Ph.D. dissertation, University ofMinnesota, 1968.
251
51. Crawford, Alan Neal. "The Cloze Procedure as a Measureof the Reading Comprehension of Elementary LevelMexican-American and Anglo-American Children." Un-published Ed.D. dissertation, University of Cali-fornia, Los Angeles, 1970.
52. Darnell, Donald K. "The Relation Between Sentence Or-der and Comprehension." Speech Monographs, 30(1963), 97-100.
53. Daugherty, Joan. "The Effect of the Cloze Procedureand Interspersed Questions as an Aid to Reading Com-prehension." Unpublished Ph.D. dissertation, theUniversity of Akron, 1971.
54. Feldman, Margaret Eichbauer. "The Effects of Learningby Programed and Text Format at Three Levels ofDifficulty." Unpublished Ph.D. dissertation,Cornell University, 1964.
55. Fletcher, Junior Eugene. "A Study of the RelationshipsBetween Ability to Use Context as an Aid in Readingand Other Verbal Abilities." Unpublished Ph.D.dissertation, University of Washington, 1959.
56. Geyer, James Rush. "The Cloze Procedure as a Predictorof Comprehension in Secondary Social Studies Ma-terial." Unpublished Ed.D. dissertation, Univer-sity of Maryland, 1970.
57. Greene, Frank Pierrepont. "A Modified Cloze Procedurefor Assessing Adult Reading Comprehension." Unpub-lished Ph.D. dissertation, The University of Michi-gan, 1964.
58. Guthrie, John T. Motivational Effects of Feedback inReading. Baltimore, Maryland: Center for the Studyof Social Organization of Schools, Johns HopkinsUniversity, 1971.
59. Hafner, Lawrence E.; Gwaltney, Wayne; and Robinson,Richard. "Reeding in Bookkeeping: Predictions andPerformance." Journal of Reading, 14 (1971), 537-546
60. Hater, Sister Mary Ann. "The Cloze Procedure as a Mea-sure of the Reading Comprehensibility and Difficultyof Mathematical English." Unpublished Ph.D. disser-tation, Purdue University, 1969.
252
61. Henry, Peggy Elaine. "The Effect of Interest on Read-ing Comprehension as Measured by Cloze and MultipleChoice Tests." Unpublished Ph.D. dissertation, TheUniversity of Iowa, 1969.
62. Knight, David W. "The Reading Performance of StudentsWith Ninth Grade Reading Achievement on OccupationalInformation Materials Written With Various Levels ofReadability." Unpublished Ph.D. dissertation, TheFlorida State University, 1966.
63. Kohler, Emmett Theodore. "An Investigation of ClozeScores in Terms of Selected Cognitive Variables."Unpublished Ed.D. dissertation, The Florida StateUniversity, 1966.
64. Lisman, Linda C. "Vowel Deletion and Cloze Tests Com-pared With a Reading Ability Test." UnpublishedMaster's thesis, Rutgers, The State University ofNew Jersey, 1971.
65. Mosberg, Ludwig; Potter, Thomas C.; and Cornell, RobertK. The Relation Between Cloze and Multiple- ChoiceTest Scores as a Function of Relative ParagraphDifficulty and Grade Level. Inglewood, California:Southwest Regional Laboratory for Educational Re-search and Development, 1968.
66. Rankin, Earl F., and Culhane, Joseph W. "ComparableCloze and Multiple-Choice Comprehension Test Scores.Journal of Reading, 13 (1969), 193-198.
67. , and Dale, Lothar H. "Cloze Residual Gain - ATechnique for Measuring Learning Through Reading."The Psychology of Reading Behavior. Edited byG. B. Schick and M. M. May. Eighteenth Yearbook ofthe National Reading Conference. Milwaukee, Wiscon-sin: National Reading Conference, 1968, 17-26.
and Hess, Allen K. "The Measurement of Inter-nal (Intra-Article) Reading Flexibility." Paperpresented at the National Reading Conference, Atlan-ta, Georgia, December 4-6, 1969.
69. . "An Evaluation of the Cloze Procedure as aTechnique for Measuring Reading Comprehension."Unpublished Ph.D. dissertation, University of Michi-gan; 1958.
68.
253
70. Ruddell, Robert Byron. "An Investigation of the Effectof the Similarity of Oral and Written Patterns ofLanguage Structure on Reading Comprehension." Un-published Ed.D. dissertation, Indiana University,1963.
71. Sawyer, Rita, and Braam, Leonard S. "An ExperimentCombining Cloze Procedure With Speed of Comprehen-sion Instruction." Proceedings of the College Read-ing Association. Edited by Clay A. Ketcham. Syra-cuse, New Yorks College Reading Association, 1964,27-33.
72. Sikes, Helen Cruse. "A Comparative Study of ClozeProcedure Scores on Original and Published Materials."Unpublished Ed.D. dissertation, University of South-ern.Mississippi, 1971.
73. Smith, William Lewis. "The Effect 6fIlttax on Read-ing." Unpublished Ph.D. dissertation, The FloridaState University, 1969.
74. Swaim, James E. "Comparison of Oral Reading, SilentReading, and Listening Comprehension Assessed byClose." Unpublished Ed.D. dissertation, RutgersUniversity, The State University of New Jersey, 1971.
75. Taylor, Wilson L. "Cloze Readability Scores as Indicesof Individual Differences in Comprehension and Ap-titude." Journal of Applied Psychology, 41 (1957),19-26.
76. Weaver, Wendell; Kingston, Albert J.; Bickley, A. C.;and White, William F. "Information-Flow Difficultyin Relation to Reading Comprehension." Journal ofReading Behavior, 1 (1969), 41-49.
r
254
THE CLOZE PROCEDURE AS A MEASURE OF
LANGUAGE FACILITY
77. Brooks, Sadye Tune. "Effects of Locus of Control andAnxiety on the Ability of Mentally Retarded Childrento Use Context Clues in Reading." Unpublished Ph.D.dissertation, George Peabody College for Teachers,1962.
78. Carver, Ronald P. et al. "Factor Analysis of the Abil-ity to Comprehend Time-Compressed Speech." Paperpresented at the Annual Meeting of the American Edu-cational Research Association, New York, N. Y.,February 4-7, 1971.
79. Clark, Ruth Anne. "The Influence of Syntactic and Re-ferential Information ou Word Predictability."Unpublished Ph.D. dissertation, The University ofWisconsin, 1964.
80. ; Williams, Frederick; and Tannenbaum, Percy H."Effects of Shared Referential Experience Upon En-coder-Decoder Agreement." Language Ind Speech, 8(1965), 253-262.
81. Cohen, Sandra Ruth. "Redundancy in the Written Lan-guage of the Deaf: Predictability of Deaf andHearing Story Paraphrases." Unpublished Ph.D. dis-sertation, Columbia University, 1964.
82. Coleman, E. B., and Blumenfeld, J. P. "Cloze Scoresof Nominalizations and Their Grammatical Transfor-mations Using Active Verbs." Psychological Re-ports, 13 (1963), 651-654.
83. DeVito, Joseph A. "Comprehension Factors in Oral andWritten Discourse of Skilled Communicators."Speech Monographs, 32 (1965), 124-128.
84. Fillenbaum, Samuel, and Jones, Lyle V. "An Applicationof 'Clone' Technique to the Study of Aphasic Speech."Journal of Abnormal and Social Psychology, 65 (1962),183-189.
255
85. Goodstein, Henry Arthur. "The Performance of EducableMentally Handicapped and Average-IQ Children onTwo Modified Cloze Tasks for Oral Language." Un-published Ph.D. dissertation, The University ofConnecticut, 1969.
86. Honigfeld, Gilbert. "Effect of an HallucinogenicAgent on Verbal Behavior." Psychological Reports,13 (1963), 383-385.
87. . "The Ability of Schizophrenics to Under-stand Normal, Psychotic, and Pseudo-PsychoticSpeech." Diseases of the Nervous_qystem, 24(1963), 642 =01..
88. ; Platz, Arthur; and Gillis, Roderic D."Verbal Style and Personality: Authoritarianism."Journal of Communication, 14 (1964), 215-218.
89. Louthi..;., Vincent. "Some Systematic Grammatical Dele-tions and Their Effects on Reading Comprehension."English Journal, 54 (1965), 295-299.
90. Manis, Melvin, and Dawes, Robyn,M. "Cloze Scores asa Function of Attitude." Psychological Reports,9 (1961), 79-84.
91. Marshall, William James Aloysius. "Investigating theEffect of Contextual Constraint on Deaf and Hear-ing Children at the Fourth Grade Reading Level,Using the Cloze Procedure." Unpublished Ed.D.dissertation, University of Illinois at UrbanarChampaign, 1970.
92. Moores, Donald Frederick. "Applications of 'ClozeProcedures to the Assessment of PsycholinguisticAbilities of the Deaf." Unpublished Ph.D. dis-sertation, University of Illinois, 1967.
;., Ramanauskas, Sigita Om. "The Effect of ContextualConstraints Beyond a Sentence on Clozo Responsesof Children in Special Classes for the EducableMentally Retarded." Unpublished Ph.D. disserta..tion, The University of Connecticut, 1971.
256
94. Rankin, Earl F., Jr. "Reading Test Reliability andValidity as Function of Introversion-Extroversion."Journal of Developmental Reading, 6 (1963). 106-117.
95. -eq, Melvyn I.; Babbitt, Loren S.; and Bennett,Stanley W. "Performance of EMR and NonretardedChildren on a Modified Cloze Task." American Jour-nal of Mental Deficiency, 74 (1970), 681-688.
96. Weaver, Wendell W., and Bickley, A. C. "Some Differ-ences in Encoding and Decoding Messages." Journalof the Reading Specialist, 7 (1967), 18-25.
97. , and Kingston, Albert J. "A Factor Analysisof the Cloze Procedure and Other Measures of Read-ing and Language Ability." Journal of Communica-tion, 13 (1963), 252-261.
98. Weaver, Wendell William. "An Examination of Some Dif-ferences in Oral and Written Language Using theCloze Procedure." Unpublished Ed.D. dissertation,University of Georgia, 1961.
99. Williams, Frederick Dowell. "An Experimental Applica-tion of.the Semantic Differential and 'Clone' Pro-cedure as Measurement Techniques in Listening Com-prehension." Unpublished Ph.D. dissertation,University of Southern California, 1962.
THE CLOZE PROCEDURE AS AN
INSTRUCTIONAL TECHNIQUE
100. Alexander, Henry William. "An Investigation of theCloze Procedure as a Measuring Device Designed toIdentify the Independent, Instruction, and Frus-tration Reading Levels of Pupils in the Intermedi-ate Grades." An unpublished Ed.D. dissertation,University of Illinois, 1968.
101. Bloomer, Richard H. "The Cloze Procedure as a RemedialReading Exercise." Journal of Develo mental Read-ing, 5 (1962), 173-1 1.
257
102. . 'The Effects of Non-Overt Reinforced ClozeProcedure Upon Reading Comprehension." New Fron-tiers in College-Adult Reading. Edited by G. B.Schick and M. M. May. Yearbook of the NationalReading Conference. Milwaukee, Wisconsin: Nation-al Reading Conference, 1965, 31-40.
103. Blumenfeld, Jacob P., and Miller, Gerald R. "Improv-ing Reading Through Teaching Grammatical Con-straints." Elementary English, 43 (1966), 752-755.
104. Bormuth, John R. Develoment of Standards of Readabil-ity: Toward a Rational Criterion of PassageEer-formances. Final Report to the Office of Educa-tion, Department of Health, Education, and Welfare,Project No. 9-0237. Chicago: University of Chicago.June 1971.
105. Faubion, Norma Nell. "The Effect of Training in the Useof Cloze on the Ability of Fourth Grade Pupils toGain Information From Written Discourse." Unpublished Ph.D. dissertation, Texas A&M University,1971.
106. Friedman, Mildred McElhinney. "The Use of the ClozeProcedure for Improving the Reading Comprehensionof Foreign Students at the University of Florida."Unpublished Ed.D. dissertation, The University ofFlorida, 1964.
107. Guice, Billy Milton. "An Experiment Combining Instruc-tion in Comprehension With Practice in the ClozeProcedure in Reading." Unpublished Ed.D. disser-tation, University of Southern Mississippi, 1966.
108. Guscott, Charles Edgar. "The Effect of Cloze Proce-dure Exercises on the Improvement of ReadingAchievement and of Reading Comprehension of Se-lected Sixth-Grade Students." Unpublished Ph.D.dissertation, The University of Akron, 1971.
109. Guszak, Frank J. A Comparative Study of_the Validityof the Cloze Test and Metropolitan AchievementasISileadinggssullansion Subtestl_for MakingJudgments of Instructional Levels. Austin, Texas:feversity of Texas, 1969.
258
110. Hafner,- Lawrence Erhardt. "An Experimental Study ofthe Effect on Various Reading Achievement Scoresof Teaching Selected Context Aids to a Group ofFifth Grade Pupils." Unpublished Ed.D. disserta-tion, University of Missouri, 1960.
111. Hunter, Charles F.Solving ReadingPaper presentedCollege Readingnia, April 1-3,
"Clozing the Gap: An Approach toProblems at San Jose City College."at the meeting of the WesternAssociation, Los Angeles, Califor-1971.
112. Jongsma, Eugene. The Cloze Procedure as a TeachingTechnique. Newark, Delaware: International Read-ing Association, 1971.
113. Kirby, Clara Lou Laughlin. "A Comparison of ScoresObtained on Standardized Oral and Silent ReadingTests and a Cloze Test." Unpublished Ed.D. dis-sertation, Ball State University, 1967.
114. Kirchhoff, Leo Henry. "A Study Utilizing the ClozeTest Procedure to Determine Reading Levels ofFirst Grade Children Who Have Been Taught Begin-ning Reading by Four Different Approaches." Un-published Ed,D. dissertation, University of Kansas,1968.
115. Martin, Ruby Wheeler. "Transformational Grammar, Cloze,and Performance in College Freshmen." UnpublishedEd.D. dissertation, Syracuse University, 1968.
116. Rankin, Earl F. "Uses of the Cloze Procedure in theReading Clinic." Reading in a Changing Society.Edited by J. Allen Figurel. Proceedings of theInternational Reading Association. New York:Scholastic Magazines, 1959, 228-232.
117. . "Grade Level Interpretation of Cloze Read-bility Scores." Paper presented at the NationalReading Conference, St. Petersburg, Florida, De-cember 3-5, 1970.
118. , and Overholser, Betsy M. "Reaction of Inter-mediate Grade Children to Contextual Clues." Jour-nal of Reading, 1 (1969), 50-73.
259
119. Ransom, Peggy Elaine Hitchock. "A Study to DetermineReading Levels of Elementary School Children byCloze Testing." Unpublished Ed.D. dissertation,Ball State University, 1965.
120. Robinson, Richard David. "An Investigation Into theUse of the Cloze Procedure With a Group of Func-tionally Illiterate Adults." Unpublished Ed.D.dissertation, University of Georgia, 1971.
121. Sauer, Freda Mae. "The Determination of Reading In-structional Level of Disabled Fourth Grade Read-ers Utilizing Cloze Testing Procedure." Unpub-lished Ed.D. dissertation, Oklahoma State Univer-sity, 1969.
122. Schneyer, J. Wesley. "Use of the Cloze Procedure forImproving Reading Comprehension." The ReadingTeacher, 19 (1965), 174-179.
123. Schoelles, Iris Storey. "Cloze as a Predictor ofReading Group Placement." Paper presented at theAnnual Meeting of the International Reading Asso-ciation, Atlantic City, New Jersey, April 19-23,1971. .
124. Smith, Elmer L. "Use of the C?oze Procedure in Im-proving Comprehension of Junior College Readers."Paper presented at the National Reading Confer-ence, Atlanta, Georgia, December 4-6, 1969.
125. Stewart, Elneita Wallace. "A Comparative Study of theEffectiveness of Cloze and Textbook Procedures ina College Reading Program." Unpublished Ed.D.dissertation, University of Houston, 1967.
126. Wiechelman, Duane Sylvester. "A Comparison of ClozeProcedure Scores and Informal Reading InventoryResults for Estimating Functional Reading Levelsfor Students at Eighth Grade Level." UnpublishedEd.D. dissertation, University of Northern Colo-rado, 1971.
1