ROBERT J. STERNBERG AND KASTOOR BHANA

Synthesis of Research on theEffectiveness of Intellectual Skills

Programs: Snake-Oil Remedies orMiracle Cures?

When properly administered, some programs appear todevelop thinking skills, but success dependson many implementation-specific variables.

The young girls are participants in the Odyssy projeca in uht Venezuelan teacers taubt tbinking skills to 450 7th grade students Thecourse, originally developed and evaluated by researchers at Bolt, Beranek, and Neuman and Harturd Uniersity, uas shown to result inI subsantal increases in scores on standardized aptiude and aodvement tests

60 EDUCATIONA. LEADERSHIP

u ppose you attend a conferenceon modern developments inhealth-related research at which

a pharmaceuticals salesman discussesa new drug his company manufacturesthat he claims greatly improves one'sgeneral health. The salesman is per-suasive but because he seems to beselling a drug at a research confer-ence, and because he works for thecompany peddling it, you decide to doa little research before buying thedrug, which is quite expensive andneeds to be used over a fairly longperiod of time.

The results of your research aredisconcerting. There are few studies ofthe drug's effects, and most of themhave been sponsored, supervised, ordone "in consultation with" the manu-facturer. The reports of the studies aresketchy, and many have inadequatecontrol groups or none at all. Someamount to little more than testimoni-als about the drug's effects; others useoutcome measures that seem to havebeen selected to maximize the favora-bility of the reports. Few of the studieshave been published in refereed jour-nals; many are reported in a company-sponsored magazine. Those studiesthat seem better controlled showmixed results. You are skeptical butperplexed. The drug may indeed doeverything it is supposed to do, but it'shard to tell from the evidence.

General ObservationsPotential consumers of programs toteach thinking skills are in a similarpredicament. We conducted a fairlyextensive review of the research onfive well-known, diverse, and, in mostcases, widely used thinking skills train-ing programs: Instrumental Enrich-ment, Philosophy for Children, SOI,Problem Solving and Comprehension:A Short Course in Analytical Reason-ing, and Odyssey. We chose these pro-grams because of their diversity, theirreputation and visibility in the think-ing skills training arena, and their hav-ing been subject to some evaluation.Our goal was to uncover whateverevidence existed to advocate or op-pose the use of such programs. Ourfindings paralleled those for the hypo-thetical drug.

1. Many of the studies were con-ducted or sponsored by the programdevelopers; others involved fairly ex-tensive consultation with them.

2. Reporting was usually sketchy nored. Our findings were disconcert-and often wholly inadequate. In a ma- ing, to say the least.jority of instances, detail was insuffi- In our communication with pro-cient for anything resembling a careful gram developers, a few odd experi-replication of what had been done. ences reinforced our concerns. One

3. Most studies involved inadequate program developer sent us certain ma-(untrained) control groups, ard some terials with the understanding that weentailed none at all. Only rarely was would not quote or cite them. Inthe efficacy of one program compared another instance, a program develop-with that of another program. er's assistant said that results would be

4. Some of the evidence offered in presented at a forthcoming confer-support of the various programs ence, but she was unwilling to shareamounted to little more than user any data in advance.testimonials. The means of selection of Some of our experiences with pro-such users was usually unspecified. gram developers were quite positive.

5. Outcome measures often over- One team of developers could notlapped program content, and thus have been more cooperative; they senttended to favor the program being us what was one of only a handful oftested. Inadequate attention was usual- scrupulously designed, carefully con-ly given both to transfer of training ducted, and fully reported studies ofand to durability of training over the an existing program. Another develop-long term. er readily admitted that the evidence

6. A very small proportion of the was spotty, but was willing to providestudies was published in refereed anv available information.journals. Such publication helps en- We do not wish to imply that thesure independent scrutiny of methods results of our review left us with noth-and results. ing but skepticism Several program

7. Many studies were unpublished, developers admitted that the existingpublished in media sponsored (andhence controlled) by the program de-velopers, or available only through Ph ph,, VI

the program developers. Availabilitythrough such limited channels, ofcourse, raises questions as to just howunbiased the sampling of availablestudies truly is.

8. Even given these caveats on re-porting, the results were generallymixed, some indicating significantgains and others not.

Many other findings emerged fromour review of the literature. For exam-ple, we were unable to find arn' evalu-ations that were useful in determiningwhich aspects of the programs workedand which did not. Such informationwould be useful not only in' furtherprogram development, but in consid-ering implementation of only thoseaspects of a given program that havebeen shown to work. None of theevaluations adequately assessed moti-vational effects (i.e., improvements inscores due to greater motivation to dowell on intellectual skills tests or toimpress examiners). Problems of sub-ject dropout, confounded variables,and sensitivity of the program to teach- The proof in Fe aised hand: cdilm as

er quality were rarely discussed. Is- L*pm-"s Pbioiff Cb')ipPrrgtrn

sues of which populations benefited Amuwr w naed atpblaosopwta/lr, and questnons 77my aremore and which benefited less from mowuwed to delo1p doe sblls needed to fonthe instruction were generally ig- deiroun rieus

O~roER 186 6

OC'oEeR 1986 61

evaluations are inadequate. At leastone was frustrated with the difficultyhe has experienced in obtaining fund-ing adequate to test his program.Indeed, such funding is difficult toobtain. Some of the program develop-ers had obviously tried to come byadequate evaluations but, confrontinglimited resources, had to use what wasavailable, which admittedly was notmuch.

Certainly, anyone who is seriouslyinterested in evaluating the effects of aprogram should call in a trained pro-gram evaluator who has no vestedinterest in the results of the assess-ment. Given the constraints of thetypical school environments in whichevaluations are conducted, even thebest of intentions can go awry whenschool schedules, resources, and ad-ministrative support (or the lack there-of) must be reckoned with. Clearly,districts need to conduct their ownevaluations, as the way in which anevaluation will proceed dependsheavily on the student population,how the course is taught, intendedcourse outcomes, and the extent towhich the program is taught as eitheran infused instructional unit or set ofunits (i.e., as part of the regular curric-ulum) or as a separate course.

Although the available evidence isso scant as to preclude any firm deci-sions, we were able to reach what webelieve are informed conclusionsabout the five programs we surveyed.

functions, (2) provision of conceptsnecessary for progress on the 14 "in-struments" (types of exercises) of theIE program, (3) development of intrin-sic as well as socially reinforced moti-vation, (4) production of insight, and(5) achievement of change in thelearner's perception from that of be-ing a passive organism to that of beingan acutive one.

The 14 instruments include (1-2)orientation in space 1-11 (which in-volves two instruments), (3) compari-sons, (4) categorization, (5) analyticperception, (6) family relations, (7)temporal relations, (8) numerical pro-gressions, (9) instructions, (10) illus-trations, (11) representational stencildesign (spatial relations through rec-ognition of embedded figures), (12)transitive relations, (13) syllogisms,and (14) orientation in space III.

The instruments tend purposely tobe very abstract; some might say theyare IQ test-like There is a great deal ofpaper-and-pencil activity, much of itnonverbal. The problems are chal-

lenging, even for teachers. A key ele-ment of instruction is "bridging,"which involves relating the intellectualskills taught in the program to real-world problem solving. However, theform and extent of the bridging ispretty much left to the instructor, al-lowing considerable variability in bothits quantity and quality.

We reviewed 38 studies of Instru-mental Enrichment, which were con-ducted by a wide variety of investiga-tors with diverse subject populationswith respect to country, social class,age, intellectual level, and educationallevel. The dosage of the program hasbeen quite variable across studies,with greater amounts of exposure gen-erally leading to greater gains. Thenumber of instruments actually usedin any particular implementation hasalso been variable. Assessments havegenerally been concerned with aca-demic types of gains. Very little infor-mation has been available on the con-structs of insight and motivation, bothof which the program is intended to

Instrumental Enrichment - iFeuerstein's (1980) Instrumental En-richment (IE) is an intervention pro-gram based on classroom and indi-vidual paper-and-pencil exercises pre-sented for three to five hours perweek as a separate curriculum Com-pletion of the entire set of instrumentscan easily take two years of instruction.Underlying the program is the beliefthat the human cognitive system isopen and modifiable both throughdirect exposure to information and,especially, exposure to informationthat is mediated by an agent whointerprets the information for the stu-dent Feuerstein refers to the lattertype of exposure as "mediated learn-ing experience." In Making Up Our Minds, a tuo-year case study of the use of Instrumental Endme infit

Mediated learning experience has Britt Local Educational Authorities sponsored by the Britsh Schools Council, erlluatorsassociated with it five pricmary has obseed that pupils uere 'interested, motuaed, and shoued ineased sef-confidence"as aassociated with it five primary goals: result of the Instrumr noal Enrichmnt intenntion Here students at the Blake School, Somerset,( ) correction of deficient intellectual England, uork on an instrument intended to develop organizational skills and abilities

62 EDUCAJno,-AL LEAI)ERSHIP

Spurred on by :eir reading qf Lpman's noel/for chden, Harry Stotdlemeier's Discover, theseclassmates debate the difen hesueen scienific desripton and scientic epanation

develop. In a large number of studies,we were concerned about teaching tothe test, especially because the prob-lems tend to be IQ test-like. We werealso concerned about the "justificationof effort" effects that can arise from theextensive commitment needed to theprogram.

Although the results are mixed, webelieve that when the full program isadministered by carefully trained, in-telligent, motivated, and conscientiousinstructors, gains can be attained onstandard kinds of IQ and aptitudemeasures. Greatest gains are likely tobe in the areas of abstract reasoningand spatial visualization, rather than inthe areas of verbal skills or "crystal-lized" abilities. Indeed, the emphasisof the program is nonverbal, and eventhe verbal items involve a fairly mini-mal reading load. The gains are notspectacular, but we have not found anyexisting program that is likely to resultin spectacular gains; we believe thatexpectations for such gains are unreal-istic. There appears to be transfer toschool work in some cases, althoughwe are less confident of the generalityof transfer, in part because the extentof transfer attained will be so much afunction of how well teachers are ableto conduct the required "bridging."We see no evidence of gains in insight-ful, creative, or synthetic thinking abil-ities, and a content analysis of thecourse materials leads us to believethat such gains are not likely to beattained, if only because they are notbuilt into the instruction contained inthe program.

OcroBER 1986

The program seems most appropri-ate for students of average or below-average ability at the junior highschool level. The skills taught arethose most likely already to be pos-sessed in generous amounts by above-average and gifted students. The pro-gram seems especially appropriate forstudents who do not grow up withstandard white, North American, mid-dle class experiences, and, indeed, apositive factor is that it has been sowidely used around the world. Thus,the program may be especially suit-able for special, including retardedand learning-disabled, as well as nor-mal populations. Potential program

users need to take into account itslong duration, its requirement of ex-tensive teacher training, the need forseparate instruction, and the kinds ofabstract-analytical skills that are em-phasized in the curriculum.

Philosophy for C aldrenLipman's Philosophy for Children hasat its core a series of six novels thatdescribe everyday experiences of chil-dren in which the characters applyphilosophical thinking to their dailylives. The program is taught as a sepa-rate course and is aimed toward devel-oping thinking and reasoning skillsthrough classroom discussion ofphilosophical topics. It is intended tobe used for three 40-minute periodsper week by teachers extensivelytrained in its use. The complete pro-gram is for children K-12.

Pixie involves reasoning about lan-guage and is for students K-5. Kio andGus is for the same grade levels, but itfocuses on reasoning about nature.Hary Stottlemeier's Discovery, the firstof the novels to be written and per-haps the core of the program, is in-tended to develop basic reasoningskills in children in grades 4 through7 Lisa concentrates on reasoning inethics for children in grades 7 through12, although it is probably most appro-priate at the junior high school level.Suki deals with reasoning in languagearts for grades 8 through 11, and Mar*

Using MeekL's SOi prognra, a for-year-od Naarjo boy sohes a p le in an Sol RerasoigReadin test dring a onerf thie Mind tite at Sol Ross Uniresitv, Alpine, Ters.

63

training are increased by having acomprehensive K-12 program. Where-as Instrumental Enrichment seems toresult in greatest gains on tests ofabstract and spatial reasoning, Philoso-phy for Children seems to produce thegreatest gains on verbal tests of criticalthinking abilities. This outcome makessense with respect to the heavy verbalemphasis of the program, both interms of the reading of the novels andof the philosophically based class-room discussions.

We believe Philosophy for Childrenis more motivating to children than

A.}~~~~~~~ _ Of -Ithe large majority of thinking skillstraining programs, but there seems tobe an underlying white, middle class

Using eebe-s SO prugra, ofiv-year-ol play a memory game with LOCANfigural orientation, even though not all thechildren in the novels are white ormiddle class. We view this program asappropriate for children with average,concentrates on reasoning in social develop the skills needed to formulate above-average, and gifted intellectual

studies for students in grades 9 their own viewpoints. For more infor- abilities, although below-average stu-through 12. mation on the program, see Lipman dents might be able to handle novels

Each children's text is accompanied (1976, 1980, 1985) and Thinking, the below their grade level. Potential us-by an extensive and detailed teacher's journal published by Lipman's Insti- ers should keep in mind that the pro-manual complete with instructions, tute for the Advancement of Philoso- gram is highly teachersensitive andphilosophical activities, and reasoning phy for Children. requires extensive teacher trainingskill exercises. Lipman has identified We reviewed 20 evaluation studies, The program is also costly in terms of33 thinking skills that the program is most of which pertained to Hary time and resources and requires aintended to develop, such as analyzing Stottlemeiers Discovey. (Little evi-value statements, discovering alterna- dence is available regarding the in-rives, making connections and distinc- structional effects to be had from the SOItions, relational logic, working with other novels.) None of the evaluations The SOI program was developed by

X analogies, and so on. Not all of the provided useful information on the Mary Meeker (1969) at the SOI (Struc-texts place equal emphasis on each development of component skills; ture-of-Intellea) Institute. It is basedskill. Underlying the entire program rather, they assessed the program as a on Guilford's (1967) structure-of-are the assumptions that children are whole. The research reports are very intellect theory, according to whichnaturally interested in philosophical sketchy: issues of subject dropout, intelligence can be understood inissues and questions and motivated to class selection, durability, transfer, terms of the intersections of five men-

subject population, and experimenter tal operations (cognition, memory,bias are generally not addressed. Sta- evaluation, convergent production,tistical analysis is often reported in and divergent production), four con-only the most minimal detail; in some tents (figural, symbolic, semantic, andcases, significance levels are present- behavioral), and six products (units,ed with no descriptive or inferential classes, relations, systems, transforma-statistics. tions, and implications). The crossing

!5 ~.- The results are generally favorable of the five operations, four contents,on a fairly wide variety of outcome and six products yields a total of 120measures. A disturbingly large propor- allegedly separable intellectual skills.

Wi C l B tion, however, are reported in Lip- Meeker claims that factor analysisman's own journal, creating obvious over a 40-year period has demonstrat-

:i. __ ~W interpretive problems. Some of these ed empirical support for 96 of the 120studies seem to involve problems of factors in Guilford's model, althoughvested interest. Despite these facts, we research by Horn and Knapp (1973),are quite favorably disposed toward among others, has seriously under-

Wih the SOl program at is center, e this program and the data reported on mined the credibility of this research.Frontiers of the Mind Intute proided it. Transfer is built into the program Note that the critical studies have notaciies for young mhwor*y/dda&nzaged through the use of novels involving necessarily undermined the theory,

and g#fed cb'draen m trwal Texas everyday problem solving and deci- but rather the empirical data basesion making. Chances for durability of claimed to support it.

64 EDUCATIONAL LEAIJERSHIP

I

SOl deals with 90 of the factors,namely, those that do not involve be-havioral content. Extensive materialshave been developed for 25 factorsregarded as being most relevant to thebroad areas of the school curriculum.These abilities are assessed by theStructure of Intellect-Learning Abilities(SOI-LA) test. The test is usually givenbefore the program, and deficientskills are identified through test-pro-file results. Students complete exer-cises emphasizing those areas inwhich they are deficient and then takea post-test Note that this form of im-plementation means that different stu-dents will receive a somewhat differ-ent program, reflecting initialindividual differences in SOI-LA testscore profiles.

The institute is a clearinghouse forresearch involving the SOl materials,although the studies are apparentlynot sponsored by the institute or itspersonnel. The institute does onlynorming studies and collection anddistribution of validity studies con-ducted by others. We reviewed 15studies of the SOI-LA measure and 21of the SOIl training program. Our fo-cus here will be upon the latter set ofstudies.

The studies we reviewed are poorlyreported and appear to have manydesign problems. A disproportionatenumber involve gifted children, al-though we have some difficulty imag-

The Wbimbe, Problem Sob ing and Comprebension pfgram e asizes "thinking-aloud pavdproblem sohlng. in uhcb paired students alternate as problem sot r and listener

ining supplementary instruction forgifted children involving little morethan training on IQ test items, regard-less of the theory by which such itemsare generated. Although the results aremixed, the training appears to result insome gains from pre- to post-test onthe SOI-LA. Of course, training is di-rected explicitly toward the model onwhich this test is based Thus. thetraining appears to yield some gain, tothe extent that one accepts the struc-ture of intellect model as representingintelligence or thinking skills, and tothe extent that one accepts the SOI-LA

Odyssev uas implemented in Venezuela during the earh 1980s as pan of tbe uor* of thatcountrn's .inistr, for the Detelopment of Intelligence The program s 100 45-minute lessonsdesignedfor 7th graders are not based on antr single theon, of cognitie functioning

as fully representing the model. Thereappears to be some transfer, but pri-marily to other tests that measureskills comparable to those measuredon the SOI-LA The program is evenmore testlike than Instrumental En-richment, and we doubt whethertransfer would be obtained to non-testlike situations. The SOl trainingprogram, unlike the IE one, does nothave bridging or its analogue as anintegral part of the program. We havereservations about the validity of theSOl model, and hence for the test andtraining based on it Because of themixed nature of the results. consum-ers should be aware that the maingains to be had may be in perform-ance on SOI-based abilitv test items.The program might sensibly be used,however, bv adherents of Guilford'smodel who are committed to trainingthe particular skills generated by it.

Potential program users should un-derstand that the exercises are verytestlike and perhaps not terribly inter-esting to many children. At the sametime, the program is much closer tobeing teacher-proof than any of theother programs we investigated because the training is prescribed by thescore pattern on the SOI-LA and alsobecause the training is relatively regi-mented. It does not involve the classdiscussion elements of the two pro-grams discussed above, which are veryextensive in Philosophy for Childrenand somewhat extensive in Instrumen-tal Enrichment.

(65OCTOBER 1986

N e # of W -* ta e

-Theh V p amis a w* del used, dive e thin skis train-id"ho beau sje to some eovalon. NAthouag the

... yr w. d n~ beedrtr e perds_ '_d , s thindng ddly ea. fmm gift _ 61110 pogr if dictik; implement;,., Wfilm pOwik saillsraint.

*pd't er hours p er as a separate curriculum, a.rvlar ndaeiwive teacher tranining;skilh; coplelion can take two yeas.

· _A! 1;~b so d g e or dbeow-avprage abUilty at thein-J__ssof level. r speiabl as well as normal populations.

*Gp·ou e at sandard IQ and aptitude measures, particularly inwi a valtef ritaonig andi abli vighuali tion.

Transfer to shool work in sme caes.

- -- Shol be used for three 40-minute periods per week, is highly teach-er-sesiie, requires extensive teacher training and a separate course,and is ca;y in terms of time and reources.

Apropiate for children K-12 with average, above-average, and gifted

* Greatest gains on verbal tests of critical thinking abilities; highly moti-atling to children.O Transfor btilt intn tfa nrnm

"Although all but afew of the availableevaluations leave agreat deal to bedesired, there areenough positiveresults to suggestthe potential forgains."

The program concentrates on whatSol the authors view as four components

0 More teacher-proof than the other four programs; exercises are very of problem solving: (1) decodingtest-lke ad perhaps not challenging to many children. skills, (2) vocabulary, (3) basic arith-Apprpriate for children of varyingages and abity levels. metic operations, and (4) preciseGains fom pre to post-test on the SOI-Learmin Abilities Test. thinking. Sources of failure in problemSane transfer evident, primarily to other tests that measure skills solving that the course attempts tocompable to those measured on the SOI4.eaming Abilities Test. remedy include failure to use all rele-

vant information, making leaps in logAR sWI-Cm'SNI'e *W jd idR on,, ic and inference that are too large,

ls both theonial rationale for academic problem solving and failure to identify appropriate relationektgenalizable instruction; is not as broad in its coverage as other ships, and failure to collect soundu prOums; is closest of the five to standard academic work and best information. The program seeks toused in wunctionj with another program. develop at least five attributes of good

Aippimprlfate for high school and college students. problem solving, namely, concern for-Gins primarily in analytical problem-solving strategies applicable to accuracy, positive attitude, problemthe prtemns pr'esent in the program. · decomposition skills, distance from* No b'aflsfer studies. guessing, and active problem solving.

While this program stresses aca-0*3157 demic problem solving, it lacks both* Has been evaluated only in Venezuela; consists of 100 45-minute les- theoretical rationale for academicsons, each of which if accompanied by a rationale, set of objectives andtat behaviors, and class procedure plan; is not based on any one the- piable instructions for how explicit generalatw of .citi e fu0nir izable instructions for how to do it. Itry cog ve unctioning.* Approriate for most 6th to 8th graders. is not as broad in its coverage as the* An inmalnative program that can probably contribute to gains in other programs considered in this arti: ffg sgalls. cle. The program, which is appropri-

No transfer studies. ate for high school and college stu-dents, emphasizes a teaching methodcalled thinking-aloud pair problemProblem Solving through practice, and that enables the solving (TAPS), in which paired stu-and Comprehension: learner actively to construct concepts dents alternate as problem solver andA Short Course in and relations. Whimbey and Lochhead listener. The problem solver reads and

Analytical Reasoning draw an analogy between learning thinks aloud while analyzing worked-physical and thinking skills: both in- out examples and solving new prob-The authors of Problem Solving and volve demonstration and use of the lems, and the listener assists him orComgrebension, Whimbey and Loch- skills, identification of the components her in reflecting upon problem-head (1979), view thinking as a cornm- of the skills, practice with feedback, solving procedures.

plex set of skills that can be acquired and learning over rime. Empirical data on this program are66 EDrJCATIONAL LEADERSHIP

very scant. Moreover, it is often usedin combination with other proce-dures, rendering problematical isola-tion of the specific contribution of theprogram to the results We locatedonly three evaluations and were un-able to draw any conclusions fromthem. Either the program was used inconjunction with other programs, re-sulting in confounded variables, or thereporting was too scanty to be useful.A major contribution of the programmay be the TAPS procedure, whichseems to provide a useful vehicle forlearning problem solving. The proce-dure may be problematical for low-ability students who have diflficultycommunicating with their partners, orfor students who are susceptible tofriction or competition with theirpaired classmates

This program is the closest of theones we have surveyed to standardacademic work, and shows how finethe line can be between teachingthinking skills and teaching standardacademic content. It is probably bestused in conjunction with another pro-gram, and, indeed, this is the primaryway in which it appears to have beenused Although no clear psychologicaltheory is behind the program, theauthors seem to base their ideas loose-Iv on information-processing theoryWhat is learned seems primarily to bea set of problem-solving strategies ap-plicable to the problems that happento be in the program These strategiesare probably useful in the analyticalproblem-solving domain, but we ques-tion whether transfer studies, whichremain to be done, would show muchgeneralization to problems with differ-ent surface structures but similar"deep" structures

OdysseyOdyssev is based on Harvard's ProjectIntelligence, which was implementedin Venezuela in the early 1980s duringthe days of the Ministry for the Devel-opment of Intelligence. The programinvolves six aspects of cognitive func-tioning, each comprising a separatestudent book and teacher's manual-foundations of reasoning, understand-ing language, verbal reasoning, prob-lem solving, decision making, andinventive thinking. The program con-tains 100 45-minute lessons, and eachis accompanied by a rationale, set ofobjectives, set of target behaviors, and

class procedure plan. The project wasaimed at 7th graders of all abilitylevels, but Odyssey could probably beused in the 6th to 8th grade levels inmost school districts.

The only evaluation (Hermstein etal. 1986) has been of the program'ssuccess in Venezuela, although it isprobably as good as or better than anyother single evaluation of any programthat we could find. The very thoroughand carefully designed evaluationleads us to conclude that Odvsse' canprobably contribute to some gains inthinking skills. The program is imagi-native and is not based on anv onetheory of cognitive functioning. LikeWhimbey and Lochhead's program, itseems to be based on information-processing principles. We see this as apromising program for future devel-opment and evaluation.

Positive Programs;Questionable EvaluationsSome thinking skills training pro-grams are probably not a whole lotbetter than snake oil, but the goodones, although not miracle cures, mavimprove thinking skills. A presenttrend is toward the development oftheoretically based programs (e.g., seeBransford and Stein 1984, Sternberg1986), and we believe that such pro-grams, if founded on sound psycho-logical and instructional theories,yield the greatest opportunities forgains. Although all but a few of theavailable evaluations leave a great dealto be desired, there are enough posi-tive results to suggest the potential forgains. Although any one study is likelyto be problematical, the sources ofproblems differ across studies, and sothe convergence of positive results isheartening about the better programs.if not about the quality of the evalua-tions of the programs. Ultimately. an-school district adopting a program willneed to conduct its own evaluation,since our analysis reveals that the suc-cess of a given program depends ona large number of implementation-specific factors, such as quality ofteaching, administrative support, ap-propriateness of the program for thestudent population, and the extent towhich the program is implemented inthe intended manner.

Although we are not optimisticabout the evaluations, we are confi-

dent about the possibility for thinkingskills instruction. The opportunitiesexist to increase students' thinkingskills, and the time to seize them isnow. What is needed to make suchinstruction succeed is cautious plan-ning, a 'sound program, effective im-plementation, strong commitment,and diligent evaluation. When theseingredients are present, instruction inthinking skills is both possible andfeasible.

References

Bransford .and B Stein The IDEAL Prob-ler Solver New York W H. Freeman,1984

Feuerstein. R Instrumental Enricment:An Intenrrt ion Program for CogniteAlodfiabilih' Baltimore: University ParkPress, 1980

Guilford,J. P The Nature of Human Intelli-gence New York: McGraw-Hill, 1967.

lermrnstein, R J. RR S Nickerson, M. San-chez, andJ A. Swets "Teaching ThinkingSkills ' Unpublished manuscript avail-able from the authors, 1986.

I tomrn, J L. and 1 R Knapp, "On theSubjective Character of the EmpiricalBase of Guilford's Structure-of-IntellectModel" Pslchbological Bulletin 80( 19-3): 33-43.

Lipman. M "Philosophy for Children" Me-taplilosophl - (19-6): 1--19.

Lipman. M 'Thinking Skills Fostered bsPhilosophs for Children." In 77inMingand Learning Skils. edited by J. W Segaland S F Chipman Hillsdale, NJ.: Erl-baum. 1985

Lipman, M. A M Sharp, and F S Oscan-van Philasopl, in tie Classoom. Phila-delphia: Temple Universit' Press, 1980

Meeker, M The Strcture of Intellect ItsInterpretation and i.ses Columbus,Ohio: Merrill, 1909

Sternberg. R. I Intelligence Applied nder-.standing and Increasing lour Intellec-tual Skilts San Diego: Harcourt Bracelovanovich. 1986

Trhimbev, A, and I Lochhead ProblemSolhing and Compnrelension: A ShortCowrse in Analytic Reasoning Philadel-phia Franklin Institute Press. 1979

Preparation of this article was supportedbe Contract N0001485K0589 from the Of-fice of Nasval Research. and ContractMDA90385K0305 from the Army- ResearchInstitute.

RobertJ. Sternberg is IBM professor ofps'chologk and education, Department ofPsychology. PO Box IIA, Yale Station,New Haven, CT 06520-744' KastoorBbha is a visiting scholar to Yale from the

niversin- of Durban 'estville. PO BoxX54001. Durban 4000. South Africa.

Oc-roBFii 1986 6

OCTRCHER 1986 6-

Copyright © 1986 by the Association for Supervision and Curriculum Development. All rights reserved.