17 2・3 E．G．Begle andJ．＼V．＼Vilsoll

2．3 EVALUATlON OF MATHEMATICS PROGRAMS＊

Edward G．Begle

and

James W．wilson

the ol・ganization or the matcrial・Book revie、VS arc orthis nature，Thev scrve a usefhlpurpose・ But as evaluations of mathematics programs， they areinadequate．Insightfhlanalysis of the content and organizatioll of a malhematics progl・am may，in・bct）bc a commonsense坑rst stepln prOgram eValuation・The NCTM gu；de－ 1ines fbrtextbookselectionprovideone丘amework fbr doing this type o［evaluation．1The criteria fbr evaluation within this n・ameWOrk pertain to Organizationorthematerial，qualityorthemathe－ matics，plLOPCr uSe Ofillustrations・COlor and typography，COVerOfthebook⊃Ordegreeorchange 丘om thelast cdition．The polnt Should not bc belaborcd，but evaluationin this senseis too limited and may not bc relevant to ho、V Wellthe materials can be adaptcd to a classrooln context inorderfbrstudentstousetheminlearnlngmathe－ matlCS．

The criteria fbr cvaluation which are stresscd

in this chapter are j・14，llol［icomeJ．Whatiden－ tifiablechangesinstudentscanbeobservedduring Or a丘er cxposure to the mathematics program？ By uslng the generalterm，pupilotltCOmCS， wide variety ofcriteria are possible．And thisis a kcy pointin this chapter：mathen：alic∫Programs ∫／乙0！JJd∂βビ〃αJJ′打′βJoぴどrαrα乃gβげcrブJgrfα．T、hcse

Criteria lnaY il－Cludc achievemcnt measuresl

rctention measurcs．att；tude measures．sclflconcept measul・es；they may（should）be measures ovcr as many difTbrcn＝ypes or achievementascanbe idcntifiedin the ot刃cctives or the mathematics pl、Ogl、anl・

Thc mathcmatics progl・am OperateS Within the totalcontexL or tlle SChool．Anylcgltlmate evaluation of the mathernatics program must takeinto account this Context．describcit、COnSidel，

itsinRuence、and account fbritin the analysis Orinfbrmation．ObviollSly、thc evaluation or mathemaLics r）rOgl・am Can becomc conlbundcd with other Lhctorsi11the schooIcontext．For

J′かt）（／肌・／∴川

Evaluation o［mathemaしics Progl・amSis colト Siderably moreimportant today thanlVaS the CaSe a decade ago・Radicalchangesin the Curl・iculum at allgradelcvels，COuPled with a Variety of ncw pcdagoglCaltechniques，have resultedin a much greater variety of textbooks and textbook sequences than we have ever had befbrcinthiscountry．Schoolo爪cialsresponsible fbr decisions on the choice of－textbooks and cduca＿

tional procedures are askinS for more and better infbrmation andfbrmorepo、Verfhlwayso［evalu－ at－ngalternative among whichtheymustchoose．

This chapter begins with a generaldiscussion Or eValuation，including thcimportant problcm Ofstating the objectives o［mathematics cducation inanorgar］izefhshion．Thisdiscussionisfbllowed by bTier reviews of a number or prq】eCtS Which have been concernedin one way or another with the cvaluation or mathematics pl，Ogl、amS・The 6nalportionis adetaileddiscussionofthevaricty Or eValuation procedures tlSed by the SchooI Mathematics Study Grou（SMSG））thclargest Or the mathematics currlCulum prq）CCtSin this COuntry・

The au†hors，througlltheir＼VOrk with the National Longitudinal Study of Mathematical Abilities（NLSMA）．a studycollducted by SMSG， have becn actively engagcdin manyl）hases of the evaluation or mathcmatics progranlS・A Substantialproportion of’this chapter draws upon this NLSMA experience．

都南り〆＆晶血晶

Thel・e are marly di仔ercnt kinds or activities that have been ca11ed“evaluation．” Thel・eis

no quarrellVith the“evaluation”of a mathe－

maticsprogramthatisbasedontheexpertopln10n of mathematicians and mathematics cducators

about the qualitv or matllematics presented and

＊ 69th Yearbook or thc NationalSocicty fbr thc Study or Education（NSSE）、Pl・atl、367q404； The permission has been granted to usc this PaPCr by the Secretary orNSSE．

1．（二o11川1itl（ユC川l八ids bl・Eヽ・alし1atOl・S OrTlextbooks，

NationalCouncil（）r Teachers or Mathcmatics， “Aidsfbr Eヽ′aluators orMathematicsTextbooks，” r厄ルね／カβ〃！〃／∫√∫ 乃〝Cカgr、LVllI（1965），467－73．

18 II・Papers Presented to the Seminar

PraCticc or fbl・mative evaluation・

SUMMATIVE EVALUATION

Contl・aSted with the fbrmative evaluationis that evaluation whichis done to dctermine theヽVOrth or cluality of a finislled product--a mathematics prog▲・am・This hasl〕een Called51”nmat［ue evallL－

alion・Itrepresentsthemoretraditionaluscofthe term“evaluatioll‥asit applies to co11ectlngin－ fbl・lnation tlPDn Which to make decisions al）Out theuse ora program・Thel・eis no cleardividing line between fbrmative and summative cvalu－

ation→nOris one necded．Inlもct，an eValuation

activltyCan besummativein the sense thatinfbr－ mationis used to make decisions about the use of the programland at the same time fbrmativein thalinfbrmationis uscd tulead to thcimprove－ mentofthepl，Ogram・

An evaluationisinherently comparativein nature．Thatis，thc program thatis being evalu－ atcdis being compared with something－an alternative progl、am，Critcria determined by a norm gl・OuP，Or Criteriain some absolutesense・ Ideally、matllematics educators should be able to takeany task which students are tolearn′and to to devclop criteria by which teachers can deter－ minelヽ・hen the student haslearncdit．Therc are

notverymanynontrivialexamples ofsuch criteria in the mathematics educationliterature．Usually， the criteria al・e what some nol・m grOup has done on a measure．or more typlCally，COVerage Of mathematicalmaterialin class without assessment

Ofstudent proficiency． Thereisnoreasontobeapologcticfb・’aneValu－

ation that compares altcrnative programs・Deci－ sions must be made about the sclection and oper－ ation ofa mathematics pI・Ogram．andinfbrmation On the coml）arative eflもctiveness of programsis useftll．There are some guidelines which wi11 help to maximize the tlSCfhlness of these com－ parisons・and setting fbrth thcse guidelincsis the prlmal・y COnCel，n Of this char）ter・

Comparison or alterl－ative pl・OgramS does havc limitatiolヽS．hov代Ver，and thoseヽ＼′ho utilize evalu－

ation results asIVellas those＼＼ナhoengagein evalu－

ation should bc carefhlto utili7e thc strengths of－ comparative cvaluation rather than be tral）PCd bv itslimitations．First．itis rardy possible to estab】ish expel・imentalconditions fbr a ＝pul・C”

experimerlt・Therefbre・SOし11、CeS Ol、i一－、・alidity、Vill

need to be rec叩nized and tl・eatcd by al叩rOPrlate means・For exaTnPle・ne、、▼PrOgramS tend to be used汽1・st With tIle best stuぐ】entsin thc best schooIs

witllthe best teachel・S．Thel、C arC Statisticaland

expel・；mentalmeans t（、deal“’ith these problems

and they should be tlSed．Second・COmparison

example，ifLtheevaluatiollirlVOi＼7eS thecomparison

Of two mathematics programs・a Suitable experi－ mentaldesign needs to be Lbllowed toinsure that

itis the altel・nativel）1・Og）・ams that are being com－

Pared rather than other fbcLors．Itis a ract or

SChooloperatio－1that eYpe上・imentalmaterials will

tend to be used by the better teachers with better

Students unless arleXr）1icit evaluation designis

accepted by the schooland董b1lowed．

FORMATIVE EVALUATION

The evaluation or a mathematics program that occurs in the process ol the development oT thel）1・ogl・am has come to be called jbrmal evalual／on・Formativeevaluationhasasits purpose the pl・Oduction oL●innLm－1ationlPerhaps diagnostic

in nalure. which can be used to ilnprovc the PrOdしICt．Itincludes、L）uLis notlimited to．the

preliminary tryout ofunits ormaterialin clas5eS． the g－athering o［comments fllOm teaChers．or the morelarge－SCale achieve＝一erlr teSting tofind how

Wellstudents handlelhematcrialinapl・eliminary VCrSiol一・The purpose ol、fbrmativc evaluationis to obtaininfbl、mationlbl・decision makingln the develuplnentO［aprogram・

Every culTiculum dt）Veloper・eVery teXtbook Writel∴ eVery prOgraml）1allnCr、Or CVen eVery teacher uses solnCli）1てn Or fbrmative evaluation

in his、、▼Ork・For thc n一（）Stl）art・thcse procedures

areinlbrmaland しInSyStemaLic．As such、they

maylose some ofthe potentialinherentin having

an cfrtctive plarlfhl・gatheringinLbrmation on Which to base decisions．Further，the pl－Oduction Ofcurlliculummaterials tcndslomovemorerapid－ 1vlhan do most schemeゝ fbr eval11ation．Or，

as sorrlC Critics chal・ge・a S〉▼Stematic sclleme Of’

fbrma霊i、re eValuation caninhibit thc developlnent Of alnathematics r）rOgram Or dampen creative Writing・For example，thc s【ating or ol刃ectives for a Llnit of materialis a highly relevantI）art Ofunit、～・riting・instrucしioll．al－d evaluaLion・But

if writinq of o旬ectivesin behavioralterms be－ COn－eS allCnd．the progl・alnitsclf mayl－eVer get writtell．A balanceis needr、（l．

Attentiol＝o procedures ol、ibrmativc e＼▼aluation

may、、■ellbe the mostimI）Ol・tant devclopmentin

the e、Taluatiot－ OrI－1athc一一1atics pl・OgramS dしIring

the ne＼t（1ccade・Tllel）OSSible payoffin ef7bctive

decision makingduri明〔hedcvelopmentormathe－

matics pl・OgramSis ve」・y gl℃at．Thereis alleed

fbr continued theorizltlg al）0しIL and al）Plication or

fbl・mative－C＼・aluatiorlteChni（luし・S．1tis truly an

intelldisciplinary l）rOblenl－しhe mathemati（：ian、

thel）STChometriciantlhe ct11・l・iculul－1SI）eCialist．

the ps），ChologlSt．the staLiゝtician．a11d the teacher

a］lhavesomethinq tocontl・ibute to fhe theol・〉▼and

2．3 E．G．Begle andJ．W．1＼7ilson 19

may be extremely costlv and time consumlng・ A fu11－blown curriculum comparison experiment

maybeoutsidethefiscalandproftssionalresources Of anindividualschooIsystem．

At the current stage orthe development o［the art and theory of mathematics－PrOgram eValu－

ation、COmParisons among alternative programs may be the most operational strategy. This PrOCedure mayinvoIve compromise with ex－ Perimentall・lgOr・The typlCalcomparisonis a status study whichis at best correlationalin nature・But progress can be madeinimproving mathematicsprograms，instruction，and cvaluation techniquesby bringlngreSOurCeS tObearo－1doing qualityevaluationstudies・

Any person who attempts an evaluation of mathematics progralnS Should，aS basic home－ work、makehimsclffamiliarwithsomeoftherecent

positionpaperswhichhavesharpenedsomeorthe COnCeptS O［program evaluation・Cronbach2has clearlystatedsomeofthepurposesandmethodsof PrOgram eValuation・Scriven、3 partially as a reaction to Cronbach，s position，has elabol・ated and expanded thepurT）OSeSandmeans ofcurricu－ 1um evaluation and preseIltCd some alternative views・Finally，areftrencewhichhasbeenwidely used－PerhapsnotalwaysinanapproprlateWay－ is the ㍑叩肋凧J画・1A taxollomy exists for both the cognitive domain and the afrtctive domain．but the fbrmeris the

ヽヽ one commonly callcd“the taxonolny・

」Jけ0鹿J♪γJl′九班帥抑症∫dc／～ぎβ乙－β椚ど乃f

The evaluation of．mathematics programs must，

in part，be based on measures of student pro一 見cicncyin mathematics．Therc should be a whole range or pupil－pCrfbrmance criteria・The task ofdescribing or speci吋ing these criteria can be made easierira schema to organize the task is available．For thisl，eaSOn，a mOdelor mathe－

matics achievementis presel－tCdin this section・5 The modelmay serve a number o［purposes

and uses・Ⅰ［may provide a unifying theme fbr COnSidering problems oF mathematics curriculum，

instruction，and evaluation．The mathematics achievement of students is at the heart of these

COnCernS．

The modcldescribed below assumes that mathe－

matics achievelnentis a many－COmPOnent r）heno－ menon．ThatlS，mathematics achievementis not a unitary trait，and thel・efbre a strategy needs tobeavailabletoinsuresamr）1il唱Ofa whole range or mcasures or mathematics a（、hievement．One

Strategyis to class坤 mLLthematjcs achievcment OutCOmeSin two ways－first by categories ofcon－ tent，and second bylevels of cognltive behavior assumed to be assaciated with the olltcome or its

meaSure．

The task or describilュg the content dimension is not difficult．Allthat has to be doneis to draw

on thc experLise of Lhe mathematicalcommunlty． Any gl・Oup Or perSOn Whois competentin mathe－ matics can easily dcscr・ibe this dimension and can modi吋the categories to serve anyparticular pur－ POSe．Forinstance，earlYin the operation or NLSMA，a Very detailed categorization ofmathe－ matics content was used because the task or test

development required it．G Later，a Simpler Classification of’content、、－aS uSed fbr organizlng

thc discussion ol、results．

The prlmary Value or describillg COngn；tive levelsis toinsure a more balanced coverage or Objective within a content area．ILisimportant thatthecognltivelevels are definedin termscom－ Prehensibleto the mathematics educationcommu－ nlty，areloglCa11y developed and jllternallY COn－ Sistent，andarepotentiallyrelatedtopsychologlCal， Or behavioral，1）henomena．Without consider－ 1ng SuCh a dimension、mathematics educators， teachers，and tcst constructors have［ended to emphasize some objectives and toユgnOre Others．

The particular modeldescribed here was de－ Veloped during T（LSMA andis the one usedin organizlng the discussion o「1・CSults．Romberg andlへ’ilson7haveprovided a thoroughdescrlptlOn Of．the evolution ofthis model．Theexplicitstate－ mcnt or the modelmust acknowledge the con－ tribution orallthe NLSMA repol・t、Vl・iters・8 The SpeCification or the cogrlltlVe－levcIs dime11Sionis 2．LeeJ．Cronbach、“EvaluationfbrCourscImpro－

Ⅴ。m。。t、”rβ。。カどrJc抽gg月g。∂rd．LXIV（1963）， 672－83．

3・MichaelScriven，＝The Methodology of Eval－ uation，＝in Robert Stake（ed．），Per5Pecliues qf－ CtLrriculzLm Evaluation（AERA Monograph Series oncurriculumEvaluation．no．1［Chicago：Rand McNa11y，1967］）・

4・BerJamin S・Bloom（ed・）、Tax？nOITリqf－Edu一 路巌朋Jα中頭加㌫C厄前抗献血潤正（New Yol’k‥

DavidMckey Co．，1964）；David R・Krathwohl， Bennjamin S，Bloom，an（1Bertram B・Masia， rJ、州＝り・イムh証・冊ノ（）恒‘イ‡こ〔・∫：・1耳ん圧∵ム恒畑

音読J）avidMcKayCo．，1964）． 飢玩（New

5．The word“model”is usedi－1the sense of’pro－ Viding an orgal－izationalfllame、VOl・k：itrepresents a categorization system with sol一一C Stated rules andrclationshipslbruslngthesystem・

6．Thomas A．Rombcrg andJames W・Ⅵ‾ilson， ‖The Developrrlent Of Testslbr the National Lol－gitudinalStudy of MathematicalAbilities，‖ r力g〟βJカβ∽d′f√∫ rどβCカどr LXI（1968），489－95・

7．Thomas A．Romberg and．James W．Wilson， れねエ血流中W扉 q／Tム嗅」㍑ふ軋4 月坤刑 nO・

7．Seelist or RFPorL∫ at end of－the chapter．

20 H・Papers Prcsented to the Seminar

Simi1ar to that glVen by13loom、リbut mod抗ed for CategOl・ies apl）IlOprlate tO mathematics nchieve－

ment．

Asmentionedl）1、CVious】y、the cssestialidea or the modelis that nleaSul・eS Ormathematics achieve＿

ment，Or tCStitems．01・objectives or mathcrnatics instruction，

．

bvleve】s or behavior．Thelevels or behavior re＿ flect the cognl【i＼・e COmplexity or a taslく．（7101

Simply the dirnculty o「a LaSk）．】rlthe model prcsentedin Figurel、the categories of－mathc－ maticalcontent are numl〕er SySLen－S・gCOmetl・y・ algcbra・Thelevels orbeha、’iol・arC COmPuiation． COmprehcnsion・aPPlicatiol一、and analysIS．

the studelltS preSulnably have】earned．EmphこiSisis upon perfbrmlng OPerations and not upon deciding Which operations arc approprlate・

Compre／leTL5ior7－Itcms designed to rcquire either recall OrCO一一CepLs and generalizationsL Or tranSfbrmation or PrOble111elements from one mo〔1e to another．Em－

Phasisis upon〔lcmonstratlng understanding of con－ CCPtS atld their relationshlpS、and not upol－ し1Slng CO11CeptS tO pl・Oduce a solし1tio11・

AMlicaI／0”－1tems designed to reuire（a）recallof rclevaヮt kno、VIcdge・（b）selectlOn OfappropTiate OPerat10nSland（c）pcrLbrmance orthe opeatlOnS・ 1tel一一S are Or a rOtltlnel－aturC・They requlre the

Stしl（lent to tlSe COl－CCPtSin a specific col－teXt andin

a way he has presし1－nabl）T PraCticcd・

A′′ゆ515－Iterns designed to requirc aI10nrOutil－C aP－

Plication or conccpts・Theitems may require the

detection or rclationshipslthe findil－g Of’patterns， and theorganization andしISCO［con〔・CPtS and opcra－

tio一一Sin a nonpracticed contcxt・

Thc seco11d dimension．1evels or bchavior．is

both hjcral・ChicaL an（lordered．Itis orderedin

the sense that analysISis mol・C COgnitivelY COm－ Plexthanapplication．、へ，hichisinturnlnOreCOgrl卜 tively complex than compl・ehension．and the com－ Putationlevelincludes thoseilems which are the least（二Ognitively complex．Itis／tierarchicalin that

anitem at the applicationlevelmay require both COmPl■CLlenSion－1cvelskills（selcc【ion ofapprol）riate

Ol）el’atiollS）and coITll）u－ation－1e、’elskills（Ⅰ一er－

Lbl，manCeOf－anopcratlOnl． Theuseol’fhemodelusuallyrcquilでSSOmCrules

or thlmb．1：orinstance、OutCOmCS are Classified

at the highest cognitiveleveleven thouglllo、、7er lcvelbchaviors aI．e required．じenera】1y．timc and pl．aCtice havelo be ex：つendedin orientation with thc modelandin adaptlngit to a specific

NumberS＼・Ste111S Geometry Algebra Computation Comprehension Application AnalysIS

Fig・1・A model［or mathematics achievement

The model・aSitis pr・eJ；ented、re〔Luires cxpliciL

SpeCification or the terms along each dimension． Thcse specincations a：lC aS Lbllol＼・S：

CATEGORIES OF MATHEMATICS CON＿ TENT

＾1L171ber砂∫Jem∫－Items concemcd with thc natttre an（1

properties of wl”1e numbers，integers、the mt；onal numbers，thc realnumbersland the complex num－ bers；the tect－nlqtleS and properties orthearitl＝一1（・tic

Operations． Geomeり－1tems concerned withline乙Ir and；Lngular

measurementIarealand、′01ume‥pOlntS，1ines・Planes： POlygons an（1circles：SOlids：COngruCnCe an（lsimi・ larlty：COnStruCtion＝graphsandcoordinategeo一一－etry；

formalproofゝ＝ an（1spatialvisualization．

A＠cbraLItcms concernc〔l、Vith open sentences：alge・ braic expressions；factorlng；SOlutio110r equations andincqualities；SyStemS Of equations：algebraic and transcendental丘1nCtions：graPhing o［rしInCtiDnS an（lsoltltion sets；theory oreqtlations：and＝－1gOnO－ 111etry・

LEVELS OF BEHAVIOR

CoTT4，ula；on－ItelnS desiglled to requirc straight（brward manlPulationorproblelnelementsaccordillgtOrules

8・The 二NLSMA reportlVriters arc E（1ward G． Begle・Stanfbrd Ul－iverslty：L・Ray Carry・Uni－ 、▼erSity：Or Texas；Jeremy Kilpatrick，Teachers College，Columbia Uユーi＼PerSlty；Gordon K．Mc・ Leo（llSta一一Ibrd Ul－i、′erSity；ThomasA・Rolnberg， しni、′■el、SiLyol、l＼’isc？nSi11り・l■、red＼＼「ea、’el－，しrl－i・ 、’el’Sity（－「＼＼－iscoI－Sl？；andJ；l－－－ぐSl＼丁・1＼’ilso－1， Universlty OrGeorgla・

9・：引001－－，坤・√～J．

1）r・Oblem．

rl’he modelis by no meal－S unlquC・ h

血hiC謡

n ≠ te d I

al

s 、ゝ fb］lowlng SeCtion、Val、ious other model

have been used by mathematics pllO）CCtS

cussed toi11ustrate some similaritics to

昆rcnces LIom the modelglVCn above． InemphasizlngmCaSureSO［●mathcmaticsachievc－

menL allcognitive outcomes．thercisllOintent to disregard afltctive outcomes such as attitude， appreciation．intclでSt・and anxiet）’．A model COuld bedevcloped rorsuch outcomcsin a mathe－ malics pllOgram also－Or tllC PreSent mOdelcould bc exl）anded．Thc first concern or mathematics PrOgram eVaJllation・ho、1′rCVel∴has been．and will COntinuc to be．with cogn吊＼′e OutCOmeS．Or achicvemclュt．rrhe afrtctive oulcomes arc sup－ POrtive andiInPOrtant but secolldary to．or at most or equalimportallCe tO，aChicvcment．

．ざ／〝～才J〟rJl加（おJ∫ こんgd／／～ 0ど／～ど7 ノ1JαJカβ7乃〟／／c．i・

ルーイ．、

21 2．3 E．G．Beglc andJ．＼＼′．Wilson

1e＼′elitcm．On thぐ Other ha11d．to ca11iL aIl

aI叩】icatior卜Ie、′e】iしCllli－叩】1eう alligl－Cl・CO糾ト

tivelevclthallCOml）1・ChellSio－1（undcl●Statlding・

けaI－S】atio叫 一1111is seenlS tlnI－CaSOnable・A＝ho岬11

Blooln，staxo＝Om）・hasbeelluSぐ（1initsg1、7enLbrn一，

it ゝeemS apPrOPr▲ate tOl・ede（ine thc cogllitivc levcIstobettercorlleSPCuld、、車hthestしIdvof、matlleT matics＼、・hen c＼′aluati】1g a mat！1ematicsl）rOgranl・

＼＼’00d】ヱ1、el）Orted thcllゝe OE：1Slightly di熊renし

cognltive behaviol・hiel◆al・CtlV rbr LheItem Danking

l）l・qleCt・His catcgo－・ies、、▼elte tll（、ibllo、、i【－g‥ 1・K■10、Vledgc an（li山br■－－atio－－＝ ▲・eCallnl、de－

nnitiol－S，ー－Otati（）11S，ぐO11CePIs

2・rllecl－rllquCS all（lskill：CO叫）しItation・1－1乙川1l州－

1atio－－（〕1’sγl－1boIs

3・（lompI・Cherlトio＝：CaP乙1Clt）710 し＝－〔l（甘Stall（1pro－

blems．to translate symbolk fbl・一一IS，1o rbllow a11d cxteIl（1reaso！－i－1g

4・Applicalio－－：01、apptoprlaLぐ COnCeptSi－－ し1－－－ h111i】iar IllとIthし‖1atiぐal sittlZltions

、）・lnve－－ti、▼州（－SS：汀aSO＝i＝g ぐl、〔・こIti、relvil－ ＝－atl－ト IllattCS

This diRersll・Om theIl10（lビIiilthelaゝt SeCLion

plt】l一一arilyin holdillq tO a Separate kl－0、、▲iedgc

le＼′elandin thc eml）hasis ollCl・Cativityin Lhc higIleSL categ（甘†′．

TheIIltenlationalStudy o「Achie、▼enlentin Mathematicsl＝le＼7OIve（l；lmOdelヽ＼・ith a cogn吊ve－

behavior dimensioll．＾n allalvsis or mathc－ maLics objeclivcsin ‖1（・しT11iled States used thc

hllo＼＼・i叩CategOries：

1．Ability tnl，ct－1川1ber りr reCalldぐnni【io－－S・1－0－ tation，Ol）erati（jtlS・an（lconccpts

2．Abi】ity tolllanlPulate an（1comptltel・aPidly乙川d accurately

3・Ability10interpl，et Symbolic data 4・Ability to put datail－tO S）・Ⅰ一一boIs

5．Abilitl▼ tO fbllo、V prOOrく 6．Ability to coI－Strtlぐt pl■00fミ

7．Ability Lo apply concepts tol一一athematical proble11－S

f3・Ability to apply concepts tol－O111一一atllematical pt’Oble】－1S

9・Ability to ．

lO・AbilitY tOinvent mathematicalgeneralizationsll

Evcntua11Y．the stし1dヽ・dcvelopeditemsibr a Very detailedlist or Tna†hcma［ics content aleach Ortrlefb1lowl昭1evels：

1・Kl10WIcdge andinEbrI一一ation：definitions、（一Ota－ tion・COnCePtS

2．Tc（・l－nlqueS and skills：SOlulions

Tlle mOdels which are dcscribed helle alle all

alikein that they use sonlClbrm or content bv COgnitiveprocesscsmatl・ixtodescl．ibcthe diLlbrent OutCOmeS Of mathematics achievemcnt，They each dillbr 丘om thc modelpl．eSentCd aboveL usually being construCLcd to rcnect the pl11Lposes Or tlle particulal・PrQ）eCL．I11SOme CaSeS a diト ItrelltpOmtOrviewisiIlhel、entin oneortheothcl・ Ohhe dimensions．IllmOSL cascs、0111y Tile COg－ 11itive processes dimensiollis discussed．

ThcStatewideMathemalicsA（1visoryCoTnmittee （SM＾C）in Calilbrnh has taken the nlOdeigiven aboveandadaptedi［totlletaSko［l）1Iel）arlr唱ItCmS Lbl、a proposed statevvidc mathe111atics assessment． The committec kepL the deIIIliLions orle＼・els o（ COgnitive bchavioralmost；tS dcLin（、dillthe model． but replaced the columllS by thc contcnt cate－ gol・iescontainedinitsl！）67－6E3’LSllla11dsliel）Ort∴1‖

These content categories、Vel・c、＼・holellumbers． rational numbers．l・eal11ulllbel・S．ntlmbcr line

andllumbcr planelTlalhenlaticalsentcnces．geom－ try．measurement．i）l・ot）ability and statiヽtics． runciionゝand gl・aPhs．andloglCalthinking．This COnten［breakdo、Vn 盲s moI・c a！lPl・Oprlate tO thc

l）url）OSeS Orthe assessment an（1ref］ecIs the policy

StaLement or the t：Om111iltビe aS g■1＼′elli11the

↓－Stl、ands ■Report・”TealnS Ol’wl・iters have T）川一

dしICed sets oL’items Lo measul▼e mathcmatics a－

chicvernent at cachlcve】0！、bし、havior Ebr each

COntent CategOry．This hug（－SeL oriter11Sis com－ I）rehcnsivein that jt covcl・S Lhc whole rallge Of、

obiectiveゝ aS defin亡d by the committce．The assessment し1SeSitem－SamPling r）rOCedures so that

SOme meaSure Orthe attainment ol、each objcctive is obtained for the state．

Ira behavioralcategolγdimensionis used hy a ptlqleCt，in most cases．somewhereillthe back一 票l・Ound、the work or Bloomlland his associates l、▼illhave contl，ibu†ed Loits ⊂1evelopment．The t〕rOad cogIlltive behavior caLcgories used by Bloom are（a）Knowled％e・

．

（f）E＼′aluation．These broad catcgories were dividedinto subcategorics，anCle7（a－11PIcs werc glVeIILl・OnlSeVeralsubJeCtS．入′1athematics，hoヽV－

ever．does notllt grace上t11l＼′irltO Lhescheme．ror CXamPle，Wherc would ol－e Put a COmPuLation itemlike 6三）3＋281？It does not seem to be

eitllCr a k110、Vledge－levelitぐm Or a COmPrehension一

12．R・1＼700d．“Ot刃cctivcsintheTcachingorMatlle－ t11atics，”gd‡JC〟′go”β／月ど∫ど♂rC月，X（1968），83－－

り8．

13・「1、・HtlSぐt－．J血r〝〟Jf彿プ～ 5’′〟少 q／血カfど〃どmビ〃f

ざ〃」lす“血7乃dJ～c∫（※ew l’ork＝Jollnl＼7iley ＆ SoI－S．1967）．chaptel■・1．

14．J占fd‥P．93．

10．Statcwide Mathcmatics A（lvisory CoI11mittce． “The Strands Rcpoll、Partl、”BzLllelin qr 上月g CαJげβr門fd〟βJカβmdJJr∫ Co〟円仁fJ1967，25（l）；

and“Reprint ortlle Stra－1ds Repot・t，PEIrt2．” 汗・．・・、′・、・J．l・・．い ．り・・・・・′ ・・、

1968．

11．Bloom，叫．仁子′．

22 ⅠⅠ・Papers Presented to thc Seminar

3. Translatioll oT data into symbols or schema or VICeVerSa

4・Comprehension：CaPaClty tO analyze problems・ tofbllo、VlteaSOnユng

5・Inventivcness‥1・easOl－illg Creativelyin mathe－ matics15

The simila＝ty O「theItem Bankir喝 PIIq）eCt CategOries and thel）reCeding categoriesis not accidental・WoodlG acknowledges that theItem Banking categories wel・e made ur）after a consid－ eration c．fthese categoriesin cor小InCtion、Vith thc Bloom taxonomy・The translation categol・y receivcd considel・able emphasisin tlle［nler71al［o7日l

∫わ（小，（13）．

The Committce ol一 Assessllュg the Progl、eSゝill

Education，the NationalAssessment PrqjecL uses thc fbllowlng CategOries o［bchavior fbrits maしll（：′－

matics objectives：17 1・Reca11芝and／or recognize definitiollS．facts，an〔l

SymboIs 2・Perform mathematicalmanlPulations 3・Understa11dlTlathematicalconcepts and pro－

CeSSeS

4・SoIve mathematicalproblemspsocial、teCllnical， and academic

う・Use malhematics and mathematicall、easOl一＝一g to analyze pl・Oblem situations，define problenlS， formulate hypothcscs，make decisions、an（1 Ver坤1・eStllts

6・Appreciate and use mathematics This scheme is palticulariy suitcd to tho

PurPOSeS Orllationalassessmcnt．Thc similarity to the previous schemesis obvious，but thel・C ar（：

importan［diFfbrences too．hlParticular、the lastit（チInlS，111pal・t、nOnCOgnitive．

A scheme＼Vhich has beelluSed to disctrl、S th（：

Objectives oi．mathematicsinstruClionin highe一・ educatiollis that use（1hy Lhe Ul－ivel、sily Gl・al再

Commissionin l11dia．1～’Thc objectives＼Ver（： Organized as fbllows：

1・Thestudentwillacquirckno、Vledgeor（lefinitio一一S， COl－CCptS，and theol・iesin mathematics・

2・The student will〔1evclop the ability to soIv problemsonthebasiso上’thethcoriesandmetl－Ods learned．

3・The student willdevelop the ability to thi一一k in terms of postulations andloglCalsLrtlCture・

4・．The student wi11acquire the ability to soIvc

PrOblems by（・Ombining di熊rcnt bl，al－Chcs of、

】Tlatllel11atics．

5．The studelltlVill become awarc of theintel、＿

relations bct＼VCCn 111athematics an（l other

branches or kno、Vledgc・ 6・Thestudentwilldevelop skilli11aPplylIlg matl一－

e）naticalmethodsto the solution orproblemsin Other fields of kI－0、V）edge・

7・Thestudentwilldcvelopabilitytoimpartmathe－ Cmaticalknowledge to others．1P

These categories are somewhat diLrtrc11t 丘・Om those reviewed previously．Ⅰ〔is doubtft］1tllat StrOng aSSumPt10nS On the hierarchicalllature Or thecategoriescanbcmade．

Thcsc various sclュemes indicate 几 COmmOIl po▲l－t：a mOdelwith a content dime11Sion and a levels－Or・behavior dimenごion can be adaptcd to the pul・pOSeS Of al－ eValuation prQ）eCt SO that a

range or criteria are delineated．

Eぴd／7′dJ～∂乃∫q／▲1′J〟′んど7乃α／／r∫ル呼αm∫

No comprehensive rcview of－evaltlation studies “・illbe attenTPted．Rathel∴a Ltw sttldies willbe di3CLISSed toi11ustrate some or thel）rOblemsin evaltlatiolュ．

HungcrmanヱO L：OntraSIcd ten classes who had used SMS（う textsiLIGl・adesIV、V．and VIwith

ten classes who had used a conventiolュaltextin

GradesIV．V、and Vl．A strongl）Oint orthis Study was an atten－pt tO Obtain severalmeasul・cs or convcntiollalmathematics achievement，SeVCral

measures or contcmE）0一，ary mathematics achieve－ me－1t，alld a meastlre Or a†tittlde．771e Calihr71i（l

」〟諭犯錯那≠f 二n畑＝ d7－狛閃蝕k al－d the 仇擁呵加叩

MalhemHl汀5 Tbjl（Expel、imentaifbrm 丘om the Califbl、1ュia Testliul・Cau）＼Vere uSed．Subscores

Were Calculated on each Lがt aS determined by itcms ol’s言milar col－tent．No attemp†was made to discuss theiternゝin tcnrlS Of－1eve】s or behavior．

Mostite汀1S On either tes［pl・Obably would be at the computat；onlevcl．

On the conventionaltest，引唱nif3cant diffbrences wcre fbund fbr twりOrthL：Seヽ・en Subscol・es，bothin

tもvor or the conventionalg上・OuP．011the con－ temporary tests，the SMSG groupl〕Crfbl・Ined Sigllihcantly higher onlbur o［seven content sub－ scales and on both terminology subscales．No di抒erencF：S Were fburld on Lhe attitude measul・eS．

The】・Flationships or socioeconomiclevel，Iq、 and sex with acl‘上icvcmcntl、－eJ・e exPloredin the

S（udy．Secondary a71alyses on theitems ヽVel・C l）Cl・Lbrmed to aidinintcrpreting theI・eSults．

Thes［udv by HtlngCrmal＝VaS doneヽVithintact groups．＾nalysIS Ol’cov■ariance was used to ad－ justlbi・T（ユalld socioeconomiclevel．The unit ofanalysISWaSthesしud（Lnt、～▼hereastheclasswould

15．J占gd．，P．94．

16．1＼700d、∂♪．cf～‥P・89・

17．Committee on Assessll－g the Progressin Edu－ Cation，“0句cctiveslbr NationalAsscssmcnt，” C．4P且〃ど∽∫Jg‘′どr，】T，l－0．4（1969）、6．

】8．Universlty Grants（」omTllission．Eual2／alioT7in H；gher EdtLCatioTl（Ne、V Delhi：U・G・C・，1961）．

19．J占fd．，P．247．

20・AnnD・Hungell－1al一．“AchievcmentandAttitu（1e oft Sixth－Grade P＝Pilsin Conventionaland ConteI11pOrary ゝ1こ1tl－eI－1atics Progran－S，”dr言－ 上長mど′言c rgβCカどr，LX（1960），30－39．

23 2．3 E．G．Begle andJ．W．Wilson

and the norms on these tests．

An evaluation conducted by thclミducatiollal Testlng Sei・Vice Lbr SMSG during1960－61、、・aS reported by Payeしte・ユ3 The study v、′aS desigl－ed

fbr rep】icationoverfivegradelevels－Grades VII， IX，Ⅹ、XI，andXII－－andhadasitslt）ndamental purposethecomparisol－Ofstudentachievementin SMSG courses with student achievementin non－ S丸iSG mathematics courses．

Evidence pertinent to the thrce fbllowing ques－ tiollS WaS SeCurCd：

l．Does theSMSG curriculum detract fl・0111Student achicvement with respect to traditionalmathe－ maticalski11s？

2．1）oestheSMSGcurriculumresultinallleaSurable extension of developed mathematicalability beyond that of conventionalmathematics lnStruCtlOn？

3．How e拝もctivclylS the SMSG curric11］um colll－ municated to students at variouslevels of scholastic ability？

The design ofthese studies called fbl・the anaT Ivsis of data丘om classrooms of teachers selected randomlyh・Omtl－OSeWi11illgtOparticlpate・There we▲・e aPprOXimately thirty teachersin each group at each gradelevel）and the unit ordata fbr the analysis was the class（i．e．．the mean fbr all studentsin a class）．The dependent variables were scores on standard traditional mathematics

tests on the one handand thoseon testsprepared bv S九1SG on the other．

The results taken co11ectively，1ed to the CnClusionsthat（√l）studentsexposedtoconven‾ t10nalmathematics have neither a pl・0Ibund nor a consistcnt advantage over students exposed to SMSC mathematics with respect to learninS of traditionalmathcmaticalskills；（b）students ex－ r）OSed to SMSGinstruction acqulre Pl・OnOunCed and consistent extensions ofdeveloped mathema－ tical ability beyond that developed by students exposed to conventionalmathematicsinst▲・uCtiol一，

and（c）scholastic aptitudc（as meastlred by SCAT）isnotsufBcientfbrpredictingachievelnent； students at allSCATlevels canlearn considerable

Segmel－tS OrSMSG materials・ Maierニュ reported speci丘c resし11ts or the ninth－

grade replication orthis evaluation andI）reSented results consistent with the conclusions stated above．

Bock2持 uscd this data as anillustrative example and did not arrive at the same conclusion when

have bcen the most approl）111ate unit．21The Significalュt di能rences may not translaLe to any practicaldi鮎rel－CeSdueしO theincreasedl〕0、、′erOr

the sta†isticaltests resulting filOm thelarge nump ber ofdata cases．

WilliamsandShu耶2reportedstudicsconductビd during196O・・－－61in the Roseville（Minnesota） SchooIsin G上・adcs VII，VIII，IX，and X．The SMSG text materials were usedin the experi－ mentalclasses．and traditionaltext materials were

usedirlthe controIclasses．Thc expcriment was Carefhl】y designed．studentsl・andomly asslgnCd to classes，and care taken to controlfbr teacher

e鮎cts．The dependent variables were the meas－ urcsJbtained on the Sequenlia17blj QrEdLLCa Progre∫∫（STEP）：JIIalhe77）aticsandtheappropriate

grade and course test from the Co4er（～liue Tb∫ls （COOP）．Hoth of these sets of tests＼1′el・e based

On COnVelltinnalma†hcmatics programs． Williams and Shufffbund no advantage．as fhr

as measured b、’these tests、Lbr the SMSG group． Theyrecognizcd thatlhetestswere notmeasurlng important obiectivesin the SMSG program，but abdicated the responsibilirv as evaluators to ob－ tain．or develop，apPrOprlatC meaSureS aS dc－ pendent variables・Hcre agaln，the approprlate unitof’analysISapl）earS tOhavebeen thcclass、but student data was uscd instead．

The Minnesota NationalLaboratory conducted an extendcd progl◆aTn Or eValuation＼Vhich was Summarized by Rosenbloom．：u The studies of lVilliams and Shu抒’＼Vere relatcd to thc＼＼，01・k or

the Minnesota National Laboratory．Rosen－ bloom described scveralexpeltiments which fbll the most part supported the results o［alleValu－ ation bytheEducationalTestingService．discussed below．For most of’these studies，the results ob， taincd on the STEP tests wel，e uSed こIS thc de－

Pendent variable．ThercisIlO discussion o［how Wellthese tests match the goals o［mathematics instruCtion・One suspects that the choice o「the STEPtestswasoneo［expediency andavailability・ In some of the studies，the”compal・ison‖ was bet、Veen the perLbrmallCC Of the SMSG classes

21・The“approprlate”data unitis thc sma11est group that could beindependently asslgned to and administered o11e Orthe experimcntaltreat－ mentsin a study・In this study，Classrooms were selected rather than students．

22・Emmcnt D．Williams and Robert V．Shuf㌔ “Comparative Study of SMSG andrrraditional Text Material，”The Mathemalics Teacher，Ⅰ．VI （1963），495－504．

23．PaulRosenbloom，“MillneSOta NationalLabora・ tory Evaluation ofL SMSG，Gradcs 7～12，” SMSG jWzL）∫leuer，nO・10（Novembcr．1961）， 12－26．

24・RolandF・Payette，＝EducationalTestil－gService Summary Report of the SchooIMathematics Study Group Curl・iculum Evaluation，‖ SMS ∧存抄∫Jgg′どr，IlO．10（Novellュber、1961），5－11・

25．Milton Maier．＝Evaluation or a New Mathe－

maticsCurriculum：’American PiyChL・Iogi∫i、XVII （1962），336（Abstract）．

24 1Ⅰ．Papcrs Prese11ted to the Seminar

thc data、、′aS Subjected Lo a multivariate analysIS． The evaluation produced verylitt！eil－fbrmaしion

useihlfbr fhl・thcl・Cu汀iculum revision or Lbr deci＿

Sion maklngin the schooIs・Obviously・the一℃ 1Vere many queStionsleit unallSWered．It underT

SCOl・ed the rleedlbl・a more comprehensive study O［ma†hcmatics achievcment・Although not rca－

1ized by the cul・1・icuium buildcrs or evaluators；lt

thattime・itundel・SCOredtheneedlbraconcept11al tl・ame、VOl・k・StlCh as Lhc modcl・tbr analyzlng and SpeCi年ing mathemaLics achicvemellt OしttCOmeS．

Weavel・2T analyzed the perfbrmance ol－1burtll－

the（1atal、▼Cl・C Only normative．

In animportantsel－SC、thiゝStudyshouldbc con－ sidet・ed an cxellCiscin fbrmative evaluation．since

anilTIPOrtant Objcctive was to obtaininfbrmation u、ヾellullbrthel、evisionoi、theexperimentalmaterials duril唱the summer ol、1962．The studyis note－ 、VOrthv forits attempts to measure attitudes to－ ward mathematics as one or the out（、omes．

SMSG ol・1glnally prepared tc7（tbooks fbr Grades VII．VIII、andIXJbI・thecollege－Capablestude11t． Later、reVisions oL－ thc teltbooks were prepared ibl，1he a、′Crage Studcnt．and thc mathematicsin the orlglnaitexLbclOks was mainLailュed．but pre－ Se11t（！din a d汗krent style．FlemlngOり demon－ Strated tha＝hc changein style did not resultin tllelossoI、anyo［themathematicswhenl）01hwere uscdlVith high－ability students．Classroom try， OutS a＝d subsc（！uClュt uSe Of－these textbooks con－

fi＝nC（ltheir approI）riatcncss Lbl・aVerage Students・

It＼＼aS fhrtherI）rOl）OSed that the textbooksin the‥入1’’sel・ies would be suitablc fbrloIVer abilitY

Students（i．e．．il＝he2うth to50th percentile range Onarilhmetiraptitudc）ifasIowerpacewastlSed－ Perllal）S takin昌一 tWO yeal・S tO’cover the matel・ial

rathel・1han one yeal・．

1ielli・iot3u reported a ratherlarge－SCalc stLldv o［

the mathcmatics achievement or sevcralhundred

junio！・high sc！1001students believed by their counselors and teachers to be “ゝlow－！eal・nerS”

i】】 mathematics．rThc slow－1earner classes in

se＼▼erlthgradeandninthgrade、faliof1963．studicd SC、†Cnth－gradematheITlaticsorninth－gl、ade algebra Lbrい〟O yCal・S．uSlng the SMSG’－M’■series text－

books・Comparisongroups uslngthesameSMSG texLbooks were selccted 丘om the same schoo；sin

the L封lof、】96LL to cover the courscin one）・ear．

Standardized aptitude and ach盲evemcnt tests wcl・e use（l：SMSG achievemenL tests and mcasures of atLit11de Lo＼Vard mathematics were alFO uSed．The

ar）Liしude tcsLsillClllded SCAT－a・SCAT－Vt DaE：

C加直ピ／Jβ〃、わ仇 al－d 上）∠′～′J∫一郎融・In the an∈巾rsis・

al・andotn orle．halfo「thc datawas used Lbr“data

s；100l〕】11g Or hypothcsIs genel・ation・The second

l▲alr was used上br hypothcsis－teStlrlg tO Lb1low up

the hypotheses gcl－eraledin thcfirst part or the ana】ysIS．

Regresゝion analysISICOrrelationaianaユysIS・al－d

analvsisol、covariancelVerCuSedtoexplort‘・thcdatc

and fifth－g一・ade students・uS＝1g

SMSG textbooks〔lurlng1961L62・

SamPleorapproximateiy200students

level丘om the elght tllyout centers

ScolleSl＼・ere obtaine仁Iin thelゝ1lon

ノ眺′血Jd∂JJ／玩、・in fも11and sl〕1・＝－g

e：叩el、imcntal

Ara11（loIn

atcacllgl、∂dc

叫・aS Selecte（l．

・ヾ〃・・l八八′J彗l，

0Ilt】1e．9尺d Ar［lhmel；c Achleuemenl二持止il－Spl，lng On Sl）CCia－1y

COnStruCted SMSG mathematics tests．aIldin fhll

and spI・1ng Olla meaSし11・e Or aLtitudc constl・uCtCd

bv an SMSG－SPOnSOred prQ）eCt・ 仙reavel・fbulld the sample to be abovc＋aVerag・e

inIq and arithmctic aptitude．ThisI）laced SOme reStriction ol－the genel・ali－y or the resulLs．

Either the tryout centers werein schooIslへ′ith

aboveTaVerage－apti［ude students40r tl・yOut Class（モS

Within the centers tendcd to be above averagc・ The analysIS Or thc SfL4 Arilhmetic Ach；e乙㈲Je）7t

乃∫t SCOrCS fbuI－d the di恥rencein npl〕rOXimale grade－Cquivalent scol・cs fl・Om fhllto splllllg Lo bc

be仙een・8andl・4・Thc＝reasonlngJand‥com－ r）しItation”scores were analyzed‥the‥cot－Cel）tS

SCOl●es were dismissed by Weaver asinvalid meas－

ures since theitems tcnded to measttl・e onlv read＿

ing ornumel†alq and vocabulary・281へrea＼▼er COn－

Cluded that the usc or SMSG tcxtbooks（1id not

inhibit thF：aChievemcnt or studentsin Gl、a（1（！S

IV and V as measしIred by the test adnlillis（el・ed

anditsllOrn15．

The data on thc SMSC; malhematics lcsts wrrr

CXamined・but they、VCre Strictly normativc s；IICe this、VaS the fhlSt time thc tests had becn adnlini＿

stel、ed．

The attitude daLa、′Yere Sし直icctcd しO a fbctor ana】yJISinol・dcrtoidenti付sしIbscales．Asa、へllOIc，

VCl・ylittlechangeinaltiLudewasibundLbuLagal－l・

26・R・Darre11Bock，りCo11tributions orMulti、7之Iriatc Experime11tall〕esign to EducationalResearch，＝ Chap・28in 肋乃d∂∂0たイ 〃〟兢βrf〟Jg gゆr卜

mg〝′♂JP叩CカウノogJ′・ぐ（1・RayInOl－d －う■〔てatlcl－

（Chicago：Ran〔lMcNally，1966）． 27・J．Frcdl＼rCaVer，“Stu（1ent AchiぐVe．一－。rlti．一

S14SG Classes・Gradcs4and5，＝SMSG New∫－ Jビ′叫no・15（Apl・il，1963）、3－8．

28．JみJdリP．ヰ．

29・Walter FlelTlillg，‖C（”nParison Experimcnt of SMSG 7M and 9M Texts with SMSG 7th and 9th Grade Teコく〔s：’SMSG NeLL）51etter、

no．15（ノIpI、il、1963ト11－16・ 30．Sarah T．Herrjot．7’カβ ∫Jo㍑ エビβ，胴r PrβJgcf：

mほ ∫∽加吋γ ふ血涙」弘拙・山肌研一 玩 A加減一 777月JfrJ（∫〟∫G 月ど♪仰′∫・Ⅰ－0・5［Stallfbrd：Sc王1001

Mathematics St‖dy Gr（・uP．19671）・

2二I 2．3 E．G．Begle andJ・＼＼’・＼＼●ilson

”7al／Cj：；ヱi”lSuall）▼nOt COllゝidered a！L CValuation or

malhcmatics pl、OgramS．Albwl・eSults fl、Om the Studydohavcancvaluationna、rOrtOthcm・though・ Fol・t－XamPle・1earning・Celltel・ed or・＝di～COVery”

approachest as conLrasted todrillorrote methods・ ＼VCre Lbulldlo bc corrclated with mathemalics

Pel・formanceandintercstinmathematics：13 Therc is the curioしIS StatCment that’’studentsl＼rho had

had coursesill－new mathematicslachieved higher scores than othel・Sludents olヽitemsilltraditional

mathcmatics＝．こ1iIn fhct．the data ollCOurSeSin

nc＼～▼ malhematicsい・aS Lost alld new mathcmatics

cxposur（二WaS L・dctermincd‥bv response to a set o上’itcms．Ncヽ′erthcless、there arc a numbcr or

conclusionsil－ thc rcI）01・L that are ‥cvaltlation”

innature・ltisapacc－Seui）1gStudyandonev；hich needs to be bcuel・understood by Lhc mathematics cducatiollCOmlT川mty．

This bパer revie≠・Or a reⅥ′ elJaluation sLudics

indicates that a conccl・n fbr cl：aluation has existcd

alongwiththerapidchangesinschoolmathematics duril一書thel）aStdccade・YctfもwsystematicassessT ments or the outconleS Or these pl’Ogl－amSNpar－ ticu］arly thelollg－range ent・CtS that a good many

innovators think al、C Or pl，lmeimpol・tanCC－have beenattemptcd．

The乙IbselュCe OL’long－tel、】11Studiesis no surpl●ise・

Atremelldousamountorresourcesmustbebrought to bcal・Lo organize astudy ovel・a Period ortimc Ior唱er tha110ne SChoolγear・

Thestudicsrevie＼Vedlackanc7（Plicitrl・alneWOrk withinlVhich to discuss the outcomes．＼JVhere

a modelhas befm uSed．ortenitems across several

cognltivelevels・01・aCrOSSCOntentlevels・ha、▼ebeen

combined．Thf？inclusion o［items spanning cognltive or co11tCntlcvels can mask di鮎renccs・ This polnt Wi11beillustratedirlthe ＝NI・SMA Results‥sectiol11Vhich n）1lo、～▼S．Suppose a gl’Oup

usingtextbookAscores5うonacomputationmcas－ ure and 35 0n a COmp一，ehension measure・and suppose a gl・OuP uSing textbook B scoresil＝he oppositc way－350n COlnputatio11and550n Thc COmprehe11Sion．Ir separatc variablcs measure these tヽ＼0】evels，then the colltraSt Ofthe textboolくS

Can bc meanlngn】iand－hc decision as to which textbookis betterdependson ajudgment（e・g・ the schooldistl，ict policy）as to which outcomelS mol・eimportant・：けon the other hand・a Single criterionis obtaincd byignorhlglevels．bothlext， book gl・oupS Obtain tlle Same SCOrC、9O・and one could beled to the erl・OneOuS decision that the

textbooks are equa11y good． This rathcl，involヽ・ed discussio11does have a

in thcい＼・0－Phaゝe prOCCduL・e．111itiall11eaSul，t：S O上、

qualltitative abilily an（lormathenlatics achievc－ men＝、rerC StrOngly related to criteriollmeaSureS：

the relatiollStlips of allaptiLude、′ariables to the

Cl、iterion me年SureS Were CSSCntially thc same fbr both grouI〕S．These were Lhc mainl・esulLs．

An analysIS O［covariancclt）u11d the a鵡us〔ed SCOrCS Or theゝlow－1earner、t、～rO－year grOuPS Slgl－1－

flCantly higher than しhe adjusted scoresibr the abo、’e・aVel．age・One・year grOuPS．Thc dii危renccs

ininilialaptitudes make tt－eanalysISOrCOVal・iance Pl，OCedure extremcly tenuous as a statisticaltech－ nlque．ItisqultepOSSiblethatthecriter；onscores fbr both groups were adjustcdinto a一・ange、Vhcrc the erroIS O［estimatcIVerelargc．

The stしtdyis al）1・OVOCative and h・uit［し11ex－

Ploratory evaluation．BcIow－aVerage Students St－1d）TiIlg marhematics ovcl・a L、VO・yeal・l）eT，iod

Can Perfbrm at aletTelsimilal，tO Lhat ofabovc－ average st－1dents studying tlle Same materialrol・ Onlvo】1eYeal・．

Biggs31has rcported al．athel●lal、ge・SCale status StudvinthethirdyearofthcEnglishjuniol・sChooIs in1959．In part、the study was an cvaluation or three killds or teaching mcthods：

1．TradiLional－teaChing children to calctllatc witll SPeed and accuracy：Lbrmalmethod

2・SlrtLCltLral－developlng unde▲・Stal－ding：uSC Orl・ods． blocks，and concrete111aterials

3．Moti乙，a［ional－infbrmal；emPhasiso＝real－Lilt）PrOb・ 1emsituations

Thcinfbrmation ol－SCtlOOIs．i＝0■、derLoclass坤 them by teachi叩method．was obtained by ques－ tiollnaires．Test baLteries wcre glVen tO Ol）tain measures orverbalreasoning．computationalskill、 l）rOl）1em－SOIving．and undel・Standit唱 Or COnCeptご．

An attitude scale ar）d a student quenio11‖airc ＼1・CI・einc】uded．

Thc results were as a＼Vholei11COnClusive．T！lCre

scemed fro be一一O SupPOrt rbr ullilnOdelsLrucLure materials such as Cuisenaire or Stenlmaterialゝ．

入Iultimodelstructuralmaterials ヽし1Ch as l）leneS

materialsヽVCre aSSOCiatedl＼′ith highel・】neChanical

and concepts scol・eS and cspeCiallv Lht／’Ored the

Slo＼＼・leal・1ヽerin mathcmatics．Tl－Cl’e wel、e a＼▼er）’

ft＼ヽ′Classes tlSlng multimodelstructuralmatcl・iais．

hn＼＼－eヽ・er．and this resultis very tentaしive．

A strong polnt O［this s［udvis thc thollOughrleSS

and care＼1・ith whichits resul【s are sLated．1t

sufrtl・S．however、ftom alilnited range一）r meaS－ ures n・om which to do an cvaluatioIl．

T’ヱ1eJ／ょねr／～〟Jオ0〃d′馳ゆげdc／／／ど乙′gmβ′7JJ／～ ▲lJdJムβ－

31．J．B．Biggs．止す〟血7”βJ～c∫ 〃”♂ 血 α′′JJJノ0′～∫ げ

LearllinLざ（SIough：NationalFoし111dation Lbr EducatlOnalResearchin Eng）an（1an（l＼＼’aLes． 1967）．

32．HllSel－．坤．√7J．

33．Jろfd．、p．298． 34．〃f♂・，P・196・

26 II．Papers Presented to the Seminar

POlntin the review．Many standal・dizcd tcsts are of a type whichincludesitems丘om several COgnitjve or severalcontentlevels．Thel・eliancc Ollrelativelylong standardized testsillthe evalu－ ation or mathematics programs may repl・eSent；1 serious weakness．None of the sl，udies reviewc（1

used a multivariate arlalysIS Or the data．But mathematics achievementis a nlultival・iate phcno－ menon・A modelis useLtllfbridenti付ing．select－ 1ng，Or devcloplng the severalmeasuresibr arl evaluation；itis approprlate that multivariate methods be uscdin the analysIS．

The NLSMA effbrt has conftonted a number

Ofthese cvaluationissues．Itisinstl・uCtivc，there－ fbre，tOeXaminethisprqJeCtinconsiderabledetail・

－＼1～／J仙～／ム”～g／血晶‘7／∫／〃小（ゾ．1J（血州／／川／

．如才J7JJg∫

The SchooIMathcmatics Study GI・OtLl）・begil▲l－

nlnglrl1958，undertook alarge curricul＝rn d（：－ Velopment pl・qeCt and the prepal・at10110L．sample mathematics textbooks．This developmcnt started With textbooks at the high schoollcvelalldl）r（）－

Ceededdownthro”ghthegradelevelstl＝Lilsample textbooks had becI▲ preparedlbr kindergarten through t、、7e的h grade．These textbooks＼、▼ere Subjected to tヽVO killds or evaluatio11．

The first evaluationil－VOlved thosc procedul・es andinfbrmation usedin the develor）ment alLd revisiorlOf theI】1aterial．On the onc hand．each

text was preparcd bv a writlng team Of mathe・ maticians and mathematics leachers．As thc

materialwas preparedJbr tryoutin classI・00mS． there was considerable comment，judgment．a11d selection exercised bv the various membcrs ol’thc team・Ciassroom tryouts were then at・rangCd to find out whether thc text could bc used com＿

fbl・tably by reasonablv we11－trained teachel・S．Fol・

mostorthewritln♀tCamS，OnCOrmOrCOrtheteam members conducted a tryout classroom fbl・ex－ tended uFe O［the material・During these claゝS－ rr）Om tryOutS，eOnSidel・ableinfbrmation was ob－

tainedlilOm the［eaChers、Vhich could be used by the wrltlng teamSin revislng the text・Thc pro－ Cedures fbr this rormative evaluation variedlilOm

One Writlng team10 thc next・The procedures Wereinfbrmal，andlhe amount and quality or infbrmation gained 丘om the class】・00m tl・yOutS Val●icd considerably．Yet this evaluatio11WaS extremely valuable35 to the wl、itiI一号 teamS a・nd

COntributed much to the qualitv orthe textbooks． These pl・OCedures have also been desc▲・ibed by

1へToot（さ11．36

011Ce the classroom tryouts and the reLVisions oL－ the tぐXtlladbeen completed、and thepILeSCntation deened ftasiblc 丘om the experience，thel’e WaS nlrther evaluation to find out＼Vhether・thc text－

booksⅥ′ereaCtuallydoinganything’；tOSee、、▼hether

the studenlS Were aCtuallylearning mathematics l＼′hile uslng them．

The wave or mathematics ctulriulumlteIbrm， represcntedbySMSG and by severalother pro－ JeCtS，had resultedin two diamctrically oppoバed polntSOrViewil－thescientificcommunlty・Thel・e were ゝOme Who Ltlt that thc new curriculum materialswere an unfbrtunate departure ftom the tI・ied and true curriculum of、the past and that their use would resultin drastically infもrior s【tldeIlt aChievement of ski11s and kllOWledge． ゝ′1anyothersfbltthatthenewcurriculummaterials ヽ1・erC a VaStimprovement ovcr what had been availablein the past and that thcir use would resultin greatimproveme11tSin student achicve－ 111ellt．

Suchlt†elings，in the mathema【icalcommunlty atleast、WCreSOStrOng that an earlv e＼′aluation or lhe ne＼V materials and a comparison orthemlVith conventionalcul・ricula seemed necessary．Ac－ COrdingly・aChievcment over one academic ycar by students using the secondary schooltexts pI，0－ dtlCed by the SchooIMathematics StudY Groul） wascomparedwithachievementbysLudentsし1Sing conventionaltext．This studv was carl・ied out in thc academic year1960－－61．37

Thel・eSults showed that both thc fbars and tlle

hope～llad been exaggerated．1∵hile there、、■ere SOme d旧もrencesillaChievementin the twogroups oL－s‡udents．the di打もrences wel、e nOt SPeCtaCularly

lal’ge．

Unfbrtunately．this reduction oLlfbal・S and hol）eS

ヽVa3 the on】y outcome of－this evaluation．The instruments usedin the evaluation wcre so bl，Oad

and covered such a■い▼ide variety or toI）lCS aI－d

problcrn tyl）eS that no conclusionscould bedra、Vn as to which aspeCtS Orthe curl・icula wel・e reSPOn－

Sible rorゝuCh di駄rences as did appear・Col－Se→ qucntly、nO guidance、VaSPrOVid亡dfbrfhrthel・Cur－ rict11um developl－1ent．

Ful・thcrmore、＼＼，hile［he cvaluation did denlOn－

Stl・atCthatstudentsuslngtheSMSGtextsdidlearn CertaintopICSnOtillCludedilltJleCOnVentionalcur－ riculum．noevidencewasp】・uVidedaboutthevalue of these toplCS either fbr r）1・Oblem solving ol◆fbr

Lhrtherlearning of mathematics．Conscquently、

36．＼＼√illiaml＼700tOn，▲ゞ．1イ∫G，mβ．1す〝た∫J7g q／．1

CILrriculum（Ncw Haven： Yale Uni＼・el・Sity Press，1965）．

37．Payette，坤．riJ．

35・SchooIMathematics Study Group、Philospph dJ？dPrβビg血rg∫げメカβ∫Jl∬Gl作晶曙丁払那（SlanJ

fbrd：SMSG，1965）．

27 2．3 E．G．Begle andJ．W．1＼’ilson

no guidance、VaS prOvided to those schooladmini－ StratOl．S reSpOnSible fbr selection or textboolくS．

Realizlng the shortcomlngS Of this evaluation and thc continuing need fbr curl・iculum revision・ the SMSG Advisory Board asked that alargc－ SCale e仔brt be undertaken［00btain quantitative infbrmation which would he usefulin rurther cur－ riculum development and also would be usefhl toschooladministratorsin makingdecisions about CulTiculum problems．It was also noted at this timethatthefieldormathematicseducation needed

a greaL dealofinfbrmation on how studentslearn mathematics and on the variables that are related

to mathematics achievement．1Vith these conT

Sideraしions，and theinterestin evaluation or the new mathematics ctlrriculavoiced by theNatiorlal Science Foundation，SChoolpersonnel．and the Public・theNationalLongitudinalStudyofMathc・ maticalAbilities（NLSMA）wasrgani7ed・The SMSG］）anclon Tests3R was apl）01ntedin Lhe La1l Of1961and assigned thc responsibility oforganiz－ ingal－ddirectingtlleStudy・Thispanelormathe－ maticianst psychologlStS，and psYChometricians adol）ted three obiectives orthe study．correspond－ 1ng tO the above types or neededinfbrmation． Thesc wereこ（（L）to obtain infbrmation which COt11d be usedin LtlturC Curriculum developmcl－L （b）to obLaininfbrma†ion which could be uscd in tllC Sぐhooユs fbr decision making on culTiculum problemミ・and（c）to obtainlaS Lもr as possible、

infbl・nlat10n uSerulfbl・basic rescarchin mathe－

matics education．

AtitsfirstftwmeetlngSノthePanelonTestsmadc a number orbasic decisions about thefi1・St Ster）S to be taken．The mostimporLant ofthesc wel、e：

1，Information gathering rather than hypothcsis testlng Should be stresse（l．It was Lblt that not allimportant hypotheses could be statedil－ advance・Therefore，alarge poolor measure〔l

Subjectsshouldbeestablishcd丘omwhicllSamPles COuld be drawn to test ヽ▼arious hypotheses as they weredevelopcd・（01－theother han（l，many

Or the tests developed during the sttldy、、で1・e

Ob、▼iouslybasedon hypotheses，eVen though they may not have been col一叩1etelylbrmし11atecl．）

2・Studel－t aChievement．both cognltive and aト fecti＼▼e，Should bc measure（llt，r Studentsin a

Wide、・arietyorcurricula．

38・ThcrllembersofthisPanelwereRichard AIpert， Harvar（l；Max Beberman，UniversILy・）fJllil一川S；

E．G．Begle，Stanfbrd：Robertl）ilwortll，Cali－

1brniaI11Stitute o［Tecl11－Ology；．Jel，Ome Kagall．

Har、’ard：M・Albert Linton．］一・．，1＼Jillia】11Penn Charter School；1＼’illiamI」isterlSUNヽ’－Stony Brook：SamuelMessick，Educatiol－alTestillg Service；J・Fred Weaver，Boston Uni、▼erS恒・ and Ul－i＼でrSltY Of＼＼’ilscons；n：＾rthur P．Cola－

darci，Stanfbrd：and LeeJ．Cronbacll、Stanlt）l・（l．

3．Lol－g－term e打盲cts were of maJOrinterest，SO Students should be tested periodica11v over seve－ ralyears・

4・Alarge number ofvariables could beidentified Which mightinflucnce mathcmatics achievement・ Asmanyoftheseaspossibleshouldbemeasured・

5．Sinceit was unlikelv that allrelevant variables could beidentifiedin advance and measured， the poptllation to be studied should belarge so that the eflもcts orunmeasured variables could

hopefhllyberandomizedout・ 6・The type of mathematics tests then available

wot11d not provide the detailedinformation desired，SO ne＼V teStS，reStricted to fairly narrow specified toplCS．Should bedeveloped・

7・Since modern curricula fbr the prlmary grades hadlat the time・Only a restricted use，thestudy Shouldbegin、VithGradeIV・

♪gjな／～す〃エ∫▲リd

Figure2illustrates the design or NLSMA・A large population o［students at each ofthe three gradelevcIs was testedin the fhlland sprlrlg Or CaChyear、beginn；ngwiththefburth，SeVCnth．and

qqq≡喜口喜喜。 X－P叩UIQtion

Y－PopリーoIion

Z－Popul8一丁on

Fig．2．Disign of’NLSMA

tcnth gradesin thefa11，1962・The X－Population andY－Population were testedibr6、TeyearS－ The Z－Population was tested fbr threc years and then fb1lowedbymeansofaquestionnaireaftergraduat－ 1ng丘orn high schoo】．The desigllStreSSed three itatures：（1）thelong－tel・mStudyorgroupso［stu－ dents－uptOLうve）7earS・（2）sLudyorthesamegrade levelat diL7trent times－forinstance．the seventh

and eighth grade3i111962－64 fbr the Y－Pol〕u－ 1ation and agalnin1965T66 fbr the X－Popu－ 1aLioIl、a，nd（3）extensive data on mathematics achievcmelュt fbr thelburth thro明h the t＼へ7elrth gltades．

In Lhe fhl］of－196二三、th〔・re＼＼・ere OVer110．000

Sludents hlOm］500schooIsin fbrty states partici－ PatingiIlthe stud、′，There werL・about 35・000 Students who ヽ＼－erC StarLing fburth grade．about 4j，000students、、心0、、rere Startlng SeVenth srade， and30．000studentswhowerestartlngtenthgrade． SchooIs wel・e aSked tol）artlCIPatCIVith the un－ deI、standing that theylヾOuld not be required to makeallya4justme11tゝintheircurriculaorin their PrOCedures because oI－this studv．The study＼17aS

28 1I．Papcrs PT・eSented to the Seminar

such as analysis，nlnCtions，logari－hmL arld cxT ponentials、fbr＼、′hichmcasし11・eSdonotfit、Ve11ullder

thesemaJOrhcadil－gS，but thebulkol’thc mathe－ matics achievement meaさureS do．Under any Particular maJOr hcading there are many smaller OneS．Forinstancc，Within the number systcms CategOry thercis‘・whole numbers”and、VithiT－ thatthercarethefburmaJOrOperations，thestruC－ ture or the whole number sYStem，and special toplCS SuCh as systcms of numeration・

Morcimportant than to make the finer con－ tent distirlCtion，it was desired to test thelnathe－

maticalabilitics orstudents at dilitrent cognitive levels・For examl）le・it was ccrLainlydesirableto knowabout theabilityofthcstudcnttocarryout routinc computation・It may be alittle hard to thinkor、Vhatcomputation wotlld bcin geometry， but＼ViLhin thc derinition orthc computationlevcl glVCnWiththc model，thercarea rewthingssuch as thc routine constructioninvoIvedin bisectlng allanglc・Carrylng On the examr）le with whole 】lumbers fblt Othcr cognltivclevels・it wasim－ POrtant tOknow how wellstudcnts understood a glVCnlnathemat；caltoplC・Orhowwellthcstudents undel・SrOOd what the addition or whole numbers rcallyl－一eanS．And bcyond that・i＝vas desirable toassesshowwellstudents couldsoIve bothroutine

andnonl、OutillCPrOblems・

Duringthefiveyears，alargentLmberoflnathe－ matics achievement tests were developed・It、VaS notl）OSSible to have allCntryin cvcry cellof the matrix，buta sample丘om year to year and 丘om cach ofthethreepopulations、VaSattemPted・ For each studentwhopal・ticIPatedinallfiveyears or the study，there were approximatel†・100difl ferentmatllCmaticalscalcs，Samplingallorthece11s ofthe matrix approprlate fbrthestude11t’smathe－ maticsprogram・At tlle end or頁ve ycars・therc weredatafbraverysubstantialnumberorstudents andfbreachstudentavery）onguec10r Ofmeasures， including mathcmatics achievement scores，PSy－ chologlCalvariable scores，a SOCioeconomicindex， dclnOgraphic data．teachers’scores・and school－ CC）mmunitydescrlPtlOnVariables・

Fl・Om thisimmense bank ordata manydinもrenし kinds orinfbrmation can bc cxtractcd．Some

vcry substalltialanalyses havc been undertakel－ byl・11e SMSG Research and Analysiざ Section・ Tlle data bank wi11evcntually be made available 上br ma［hematics educators who have qucstions oL、 their own vtTh；ch can bc answercd 丘om the NLS

入4Adata．

Ti・＼・／わ（－（）んC州ソ刷イ∫釧．」′J（′（…、－一高一＼’上∫．1J・▲I

ThcfirstsetorallalvsescompletedbvtheSMSG Research alld Analysis Section；s a series ortext－

to be conductedin actualevclyday situations． Care was taken to obtain schooIs uslng a Val・iety ormathematics curricula．

In anylong－term Studv Lbllowlng the same set Of－sLudents，attrition willbe a problem．Apq proximatelyljl）erCent Of’the rcmainlr唱1）OPt卜 1at言on each year was】ostin this study．ThibヽVaS due to a number of causes：families moved and

the studentswere nolongerilュtheschooIs partici・ Patingin thestudy；aJをwschooIsdccided thatthc time commitnlCnt Of、thrcc hours fbl・tCSting cach fhlland o［three hoursJbr testlng eaCh sprlng WaS too much；and somc students werelost h・Om thc Stud〉Tin movユng fl－Oln an elementary schoolto a Juniorhighschool，Orn▼Om aJuniorhighschoolto a senior high school．

A tremendous mass oi、ilュformation was collecte（l

OVer the five years．Mathematics achievemellt WaSmeaSurCdineachorthcsprlngteStSCSSionsan（1 in three of the f封1sessions．Alargc number o［ psychologlCalvariables believed to be potentially relevanttothelearnlngOrmathematics、VeremeaS－ ured．MeasuringinsLrumentsincludcd cogIlltive tests such as those ofIq，SpaLialvisualization， numericalrcasoning，loglCaF thinking、CtC．，and also a number of afTt：CLive tests which provided measures of－attitude toward mathcmatics．attitude

toward schooIs、SelrJcollCCPL and anxiety． A good dealofinfbrmation was obtained about

the teachers－howmuchexperience theyhadhad， Whaltheir traiIllng had becnlilくe，and how they fblt about teaching and about mathematics， （む1eStionnairc data were obtained to dcscribe the SChooIs and thc communitiesinlヽ′hich they were located．

AfhlldescrlpLlOnOl、eachot’1hevariablesisglVen in the NLSMA Rqporls，3”numberslh7 and 9．

．1J‘7Jlぐ賄7／∫〔†れIJ／J／〃 ▲＼‾J．∫．UJ

ThemathematicstestsusedinNLSMArcqulrea little more discussion．The NLSMA stafr and

its consultantS，丘om thc very beginnlng O［the study，had a firm conviction that mathcmatics achicvementsis not a singlc thir唱．Itis a very lnu］tivariate quantity：there arc many aspects or mathematics achievcment．Thc filamCWOrk which

wasfiIlallyusedfbrspccifyingmathematicsachicve－ ment measurev is the model prcsellted earlier in this chapter．The mathcmatics content、VaS dividedinto alarge number o［compartmcnts； butin generalthere wcre threc m往lOr OneSM number systems，geOmetry，and algebra．There arealもwtopICStOWardlhccndorthehighschoo】，

39．The NLSMA RqporIs are detail（－（1backgrou一一d and refbrence（locuments．A nll11ist of them is glVen at the cnd o［this char）ter・

29 2．3 E．（∴Begle andJ．＼V．Wilson

andlhcrc！brc the analyses al・e based on tllC mCal－

Lbr each school．

The textbook groul－S Were nOtlbrmedjust Lbr this study・Rather，thcy、、▼erC eXistlng gl・Oul）S Of

schooIs，anditwasknown that Lhegl・OulつSdiL7trcd prior to thc study．AllalysIS O［covariance was judgcd most approprlaしeibl・analyzlng the pos－ sib】c textbook＿related eflbcts＼Vhen measures such

as Lo7二ぐe－ThrndikeIntellkenc［Thts scores andpre－

vious rnathematics＿aChievement test scorcs l＼■ere

used as tlle COVariates．The use o［analysIS Or covariancedoesnotmakeanevaluationstudysuch as thisinto an experiment・It remains correla－ tionaland dinbrcnces betヽVeellthc grour）S are at most textbook related．Thatis．the di汀erellCCS

that ca11be fbund betwecLlteXtbook groups may inlhctnotbecausedbythetcxtbook，butbvother ibctors which also determine te：くtbook selecLion．

Thel，esLllLs of stiCh analysi・S al、e j少少Othe∫e5about

textbook－rClated phcllOmCna・raLher than con－ clusiollS．

▲・1′′J〟〃、、／用／J■汀」…巨■♪刷．▲＼‾ムi－1J・・1

Thc analysIS Of mathemaしics achievementin Gl・adcsIV．V，and VIis the topIC Of NllSMA R車r／numberlO．41Itis a comi）arison of six texthook gl・OuPS OVer a pCriod ortl－reeyearS【use・

Tlle aChicvement data ヽVere Obtained 丘om test

batteriesadministcl・edillthesl）rlng Of1963・1964・

and1965．and丘om thclも1lo上、196二）aS tlle Students

began GradeVTI・Six textbook series wel、eiden－ tiiied as being usedin enougllXLSMA schooIs LO beincludcdinl．he analvsis．Data Lbr those studcllls who used the Yame tes:book series in the samc schoolover thc thl・ee gl・ades、、・Cre Selected

行om the NLSMA files．Schooldata ollCaCh

variable werc obtaiiled by takil唱the meal－fbr allbovs and the meanibr allgillsin the school・ The school●SeX meanS Were then combincd to fbrm the scllOOlullits．

Tlle SlX teXtbook groups wcrc contrasted on38 variables．The analysIS WaS COnlPletedin fbur parts－OnCPart上bl・eaChofthetestsessions・Signl－ ficant（p＜．001）multivariate F・Statistics wcre lbund fbl・CaCh parLindicatlng Variatior上 aCrOSS thc vcctors of mean scoresilllhc groul）S after adjustmentfbrCOreSOllVerbalIq・nOnVerbalIql and mathcmatlCS aChievcment variables fl・Om the begin111】一g OrGradeIV・These multivariate ana－ 1yses were fbllowed by univariate anal）－SIS Of、γari－

ance on the adjusted scores tolocate the source of variationin each of thc vectors．Significant

book coml）arison analyses．ThatisLitis a com－ pariso110「grOupS deterIllined b、’the textbook which the students usedin their mathemat主cs Classes．Thcse analysesinvestlgate the eflbcts on student achievement that†3eem aSCribable to the

textbookswhichthey areuslng． Thedesigrlfbr these analysesinvo行ed allelabo－

rate sequence or statisticalproccdurcs・The details orthe procedures and the rcstllts arcl，1’0－

videdin NLS九日 Rqport∫，numberslO－18・A brier descrlptlOn and summarizaLion willbe aL－ temptedhere． For eachstudent．in eachyear ofthe study，the

m叫Or teXtbook usedin his）Tlathematicsinstl、しIC－

tion lVaSidentified．ThisinfbrmatiollWaS eX－

amined and textbook groups wcl・edeしCrminedLbr the analvses on thc basis ol、the textbook used；n a

Particular gradc・Or a Seque11Ce OrteXtbooks tlSed OVer Lwo or three grades・For thc X－PoI）ulation t、VO ana】yscs、Vere POSSible・DataたomTtuderltS uslng SPeCified textbook seriesin Gl・adesIV，Vl and VIw・erC uSedibr thc fhlSt analysIS、alld thc SeCOnd analysIS＼VaS Or data rl・01Tlthose sLudents using speciBed textbook seriesin Grades VlIa－1d VIII．Four analyses、Vere made fbrlhc Y－PoT）u－ 1ation．Thefirstwasacomparisonofgroupsusing SpeCificd textbook seriesin Grades VIIandヽ′■lII・

The secoIld was a comparison or f！rOupS u＝n白 SpeCihed algebra textbookゝin GradcIX・The third、VaS a COmPal・isoJ10［groups uslng SPCCincd geometry textbooksin Grade X，and the fburth lVaS a COmparison ofgroups uslng SpeCiGed al誓C－ bra 2 0rintermediate rnathematics textbooksin

Grade XI．For the Z－Population，three anal）’SeS Were done．one fbl・eaCh orthe thl、CeyearS：Gl●adc X，Grade XI．and Gl・ade XII．The Grade X

and Grade XIanalyses parallelthose done 上br the Y－Populatio11in Gl・ades X nnd Grade XI except that they wel、e at an Callier YCar Or tlle StudY．rrhe Grade XIlanalysISis a comparison Ofgroups deしCrmined bv the typc ormathcmatics COurSe taken rathcr than or groups detel・mined by a specific textbook．These analyses arel、e－ portedin NLSMA R4）OrtS，numbcrslO－18・

The unit ofL data fbr thesc analyses＼VaS thc SCh〇Ol．Actually．an elaboraLe design．f－ullv de－ SCribedin NLSMA Rqporls noted above，WaS uscd to examine the dataibr bovs and Lbr gil・1s SePal・ately andjointlyln eaCllSChool・Almost no interaction orsex and textbook e拝もcts＼VaS Lbund40

40．This does not mean that dintrences bctlVeen

boys’and girls’perfbrl－1anCe、Vere nOL fbund、 but rather that the e打ect or textbook＼～・aS the

Same both fbr boys an〔1fbr glrls．Difftrel－CeS betwecn boys’perfbrmancc and girls’perror－ 1TlanCe Were Ofmil－Orintcl・eSLin thcsc analyieS．

41．L．Ray Carry andJ．Fre〔11＼’ea、▼Cr，Pa／ter花∫q／ 〃〟∠加mαJJc∫ dr力JβぴβmβJi′ ～ノ～ Cr〝壷∫ 4，5．βJ～d 6：

Ⅹ－♪車㍑Jβめ′？，〃エ∫〟▲1月中ロrJ∫t －10・10・Seelist

orR中ロrl5at the end oftl－C Cllapter・

30 II．Papers Presented to the Seminar

Fig・3・Profi1c for Textbook Group T．（X－Population：Gra（1es4，5，6；ヽ’earsl，2、3）

（p＜．Oj）variationwas fbundon370rthc38vari－ ables・Thel－e Were aPPrOXimatelv 300 schooIs in theallalvsis．

Thc、●al’iables on which signi6cant＼・ariation

occur7・ed included eleven mcasul・es Ofllumber

SyStemS－COmPし1tation．sixteen measures of－number SyStemS－CO11叩rChension，tWO meaSureS OrgeOmetry COmprehenゝion，One meaSure Or a1gcbrとICOm－ Prehension．three measures of－ mcmber systems－ applications．two mcasulIeS Of numbel・systems－ analvleS・al－d twomeasurcs orgeomctr〉・analysis．42

0neoftllCteXtbookgroups，T．J、′aSdetermined by the use orthc SMSG textbDOksin（iradesIV． V，alld VI．For each of－thc variablesl、，ith slgnl－ ficant ヽ・al・iation．contrasts or the SMSG textbook

group meal－Witheachoftheothertextbookgroup means、、’el’e e7（amincd and a t－Statislic computcd．

These contl、astS Were discussed to analyze the PalterrLS Oi’mathematics achievemellt aSSOCistcd With cach of■the textbook gl・oups．

Befbre discussmg the additionalmeans or ex－ amin‖lg aIld describing7）attCI・ns Or aChicvemcrlt．

it shou）d be noted that three or the six tcxLbook

groups、＼’el、C dctermined by rllCir use ortextbooks

judged to be modern，and three groups wel・e dc－ termined b〉・their usc or textbooks judgcd to be COnVentionalin point of、vieヽヽ′andpresentatiollOr

42・Theselabels or groups or variables correspond to thc ce11sorthemodelin Figurel・

matcILial．’rhlSidcntificatiol10f’teT（tS aS mOdern

or conventionallVaS Of cotlrSe aSSOCiated ヽVith a

number oI－exl－eCtations－■eVen though hYpOthesis

testing was rlOt a gOalol’the anal）・S】S・Rather，

the goalu；aS hYl）Othcsis jbrmalio7Z・

Patterns of mathematics achievenlent ヽVere analyzed al－d ←lisplaYedin threc other、VayS・

First、Ⅰ一一しIhlPIc di5Cl・imillant anal、▼SeS43“・ere Per－

fbrmed on the vector or a（臣IStCd scoIIes．Ⅵ7hile

43・MultlPle（liscril－1ina一一t analysISis a statistical proce（1urein whicllthelinear：combination of variablesisdetermiIled＼1・llich maxima11yspreads the gz・（）し1PS・The welgl－tSin thelincar combi－ 11ation，the discrilllinanlfunctio‖ COCLficients， Ca一＝hen bc exal一一inしLd tol－yPOthesize or charac－ terizc thc丘1nCtion or（1epcndcnt variables・In a sensc，Lhis“describes”how the groups differ ollth（・Set Ofヽ▼ariables．The first discrinlinant

fし1nCLiozlis thenlbllolヽ・ed bY a SeCOndlVhich I－1aXi＝一a‖y spreads111e grOupSil－a Way Ortho－ gonalto（tlnCOrrelat（－（ll＼′ith）the first・1＼′hen scores rroll＝hesc two dimensions are plotted on ＝1COOrdi11ate SySLenl．＼－isuali－1SpeCtion can some－ 1il－1eSide址埠groups which clustcr・ Sぐ（、R．1）arrelBock，‖Contl・ibutions orMulti－ 、・ariaLe ExperimcntalDesigns to Educational RぐSearCh，”chap．28in HnlTdbook QFMulliuari E．tj，erime′血IP．リ・chologJ，ed．Raymond B．Cattell （Cl－icago：Ra！－（1McNf111）・，1966），ibr a dis－ cussion ol、the tlSe Or discrirninant analysISin tllisぐO11t（、Xし．

31 2．3 E．G．Begle andJ．W．Wilson

andsimilarlyfbrtheitwapplicationsandanalysis scales．Onthe37scales、Tlhadthehighestmean on19．The＼Veaknessin computation scalesis notsevere；nO meanis more than a standard deviation a、、′ay丘om the grand mean or50・On comprehensionschles・SeVeralaremorethanastan－ dard deviation above the grand mean・ One oftheconventionaltextbook groupsin the

analysISl〟aS T2・45 The achicl・ement prOfi1e fbr textbookT2isglVeninFigure4・Tosomedegree， thisprofi1eisalmosta mirrorimage of that fbr the SMSGgrouplalthough Ebrtheentire set or variables thisgroupISprObablvbestcharacterized asavcrage・Whcre the Sヽ′TSG textbook group tended to be below the grand mcan on comput－ ation．theT2teXtbook group mean’is aboveit， but not far abovc． And whelでaS the SMSG textbookgroupwasabovethc即and meanon all comprehension，aPPlication・and analYS－S SCales， textbookgroupT2tends to bebelowornearthe grand mean．

Unfbrtunatelylthesepattcl・nSdol－Otgeneralize・ Thatis．the modern tcxtboolく gl・OuPS do not all have an achievcmentprofilelike thatin Figure 3，and thc conventionaltextbook groups do not allhaveanachievementprofjlelike thatin Figure 4．On the contral・y・thereappeal・StObeasmuch variation among tlle mOderl‥extbook gl・OupS aS thereisbetwccnthemodernand the conventional・ AIso thercis considerable ヽ・al、；alion an－Ong the convenlional．

Oneadditionalprofi1e行om this analysIS Wi11 su托ce、Lbr this example oraNLSMAanalysIS・tO showal・atherdi飢汀ent pattt汀n fbra conventional textbook group、T6・This profileis glVen jn Figure5・The groupis very strong on COm－ putation mcast11・eS，Sho、、・Sgreat Variabilitv across comprehensionmcasures・genCl・all、・improvmgfrom fburthgradctosixth grade・al－diゝrail・l、・StrOngln application arld analvsis・

Gg′′どrβJOムJg化αJJo乃∫0／7／んg∧’んゞノ1J」月か〃如

TllCl）reCCdi叩SeCti0日ga、▼e al－CXampleorone or the．YLSMA textbook compal・iso＝ analYSeSL Genel・altrendsin†he results or these analyses、 insofゝras they ha、でbeenidentified・are nOtCdin thissectioIl．

First、in gellCral・thc highel，1he宰1・adelevel・ theless seems t。hc thc eLTect associatedヽ、′；th thc

tcxtbook．Thc grca†est Variation among text－ book group means・aI－dlhe greatest numbel・Or variables on whiぐh signihcant variationis fbund・ occul・in thelower grades・As o一一e eXamines the analyses up to eleventh and t、、・elrth grades・itis

45・Sec Carrr andl＼▼eaver・loc・CiE・，fora fulliden－ tification an〔1comparisonofthetextbookgroups・

noぐIaim can be made that the discriminant hlnC－

tions、Vere thc L－ame fbr each or thc parts or the analysIS・there was some consistency．In most CaSeS、t、VOClusterso上■tcxtbookgroul）5didapr）ear－－

the three modern textbook groupsil－One Cluster and thc three convenlionalonesin the other．The

firstdimension（thefirst、mOStimpol・ta址discrimi－ nant fhnction）usually＼VaS SOme COntraSt O「COm－ Prehension・1evelsca］es on onc end and compu－ tation－Je＼・elscales on thc other：thclnOdern clustel・

tcnded to be to、Vard the comI）reh川Sion side and the convcntionalcluste▲・Lo、、′ard thc c…一叩utation

Side．A！so．in most cases，th（・SeCOlld dimension 、VaS dctermined hy an unusualspread ol、the text－ book groupsollOneSCale．Whe一・ぐdistinct clustel・s

werelTOt fbrmcd．there secm〔・d しO b（、an unuSual

Spread or the textbook means on o11e SCalcin the ＼・eCtOl，OrSCOl・es．

Second．the adjusted scores＼ヽ▼ercIlOl・ITlalized to

a mean or 50 and a standard dcviation ol－10．

Then．1br each scale、the meanslbl・the textbook

groups、、，el’e T）lotted and a90pelICCnlconfidence intervaldisplaYed around each mean．

Third，aprOFileorachievementscoresIVaSPlotted fbr each textbook．011these profi7eて・the means fbr a11textbook groups wcre plott（－din ordel・tO Show the spl・ead orgroups on cach variable and thcposition ora particular textbook ollCaChvari－ ab】eil－dicated within tlle Plot．rorinstance，the SMS（ユtextbook group profilcis glVCninl：igure 3・The darke一一Cd marks are thc SM湖コ group means・S〔ale codes fbr the37scales、Vith signl－ ficant variation areglVen aCrOSSlhe bottom orthe figure・These scales are grouped by cognltive leveland then by content asindicated atlhe top Orthefigul・e．

These profilcs use standardizぐd scoL・eS：they COmmullicate the position or a textbook group relat；乙・e fo the other groupsin the analysIS．Thcy

are usefhlfbl、ident坤1ng al・eas of●strel－glh and

Weakness among the scales fbr each textbook．44 It can be seen ftom Figure 3 that the SMSG textbook group exhibited rclative、Veakness fbr COmPutation scales・It waslowest、however、On Only tlVO Ofthe eleven computation scales．The groupwaslowoncomputation．butl一nt eXtremely loll■．On the comprehension scales．the SMSG grouplSquitehighrelativetotheothertextgroup5，

44．The profiles．however．are subjcct to misinter－ Pretation．The reader shoul（1notinrer that the SMSG textbook group does bettel－On SCale XlO5 thanOnSCaleXlO4（seeFigure3）for the between group variation on Xl,lis less than the betweenn groupvariationonXl。5．ToinferthattheSMSG group does better on XlO5thanonXlO4does not takeinto account the characteristics of the me． aSureS．

ⅠⅠ・Papers Presented to the Seminar 32

＝ ○ A 醐用冊描

亭 ■ － － ● ● ■王 聖 ● 雪ぎ苫旨看ぎ蔓魯さ壱貰壱嘗篭署長貫首董 ぎさ旨 扁扁貫首 H；ベトく；H

Fig．4．Profi1eforTextbook T2（X－Population；Grades4，5，6；Yearsl，2，3）

Fig．5．Pro抗1efor TcxtbookGroupT6（X－Population；Grades4，5，6；Ycarsl，2，3）

33 2．3 E．G．13egle andJ．W．＼Vilso11

ventionaltextbookgroupwasavel・age・Thelong－ term curnulative effects for development of a division algorithmseemcd tosupportthesequence andemphasisinthcmoderntext・Unfbrtunately、 these NLSMA textbook comparison analyses do not deal wilhlong－term）Cumulative eLrtcts・ Textbook sequences are notidentiEiablein the upper grades・Hence，the GradeIX algebra textbookcomparison doesnot build on thecしImu－ 1ative e仔bcts ora sequence through Grades VIT， VIII，andIX．Instead，these efrtcts areadjusted out of the Grade IX analysis by using end of Grade VlIIachievement measures as covariates．

1n general，the results fbr the SMSG textbooks were favorablc．13ut not a11modern teXtbooks

produced the kind or results that were cxpected fbrthem．Someorthemin hctdid ratherpoorlv on al11evelsr丘om computation to analysis・ Those tcxtbooks which did not do vcry well、Vere fbr the most part considcrably morelbrmaland morc rlgOrOuS than the SMSG tcxtbooks・In the upper grades・mOre attention、、▼aS Paid to axiomatics alld strict deduction than is paid by SMSG，andless，apparently，tO undersLanding the basic concepts・This remark on the greater fbrmalismis cor11eCtul・al；itis an oplnion as to a possible gcncralexr｝1anation ofthel）00r Showlng of’some modern textbooks．Itis aninterestlng leadir］tOl～′hat could be hclprulfbr nlrther rc－ SearCh．Parenthetically，nO Similar seemlngly consistent observation has bec11made across

conventional textbooks showlng pOOr reSults・ Theresultso「thcNLSMA textbookcomparison

arc cxccedingly complex．There are not very many clear gcncralizations that can be rnade・ Thcl・eis variability among modcrn textbook groups and variability among conventionaltext－ book groups、aS、、，ellas thc variability bet、Veen lnodern and conventional lextbook groupY. The IJLSMA results dcmonstratelhat the simplistic question or＝Which textbookis bctter？．’is not approprlate and they emphasize the multivariate natureormathcmatics achicvemcnt．

The NLSMA results havc demonstrated wc！1

theimportancc or a range of critel・ia・There are clear cxamplesiI－mOSt eVery ana】ysis of†he reversaliIlthe rank order orlhcIT】CanS Of’two

textbook groupsin golng ftom mcasures at onc levelto measurcs at another．The tlSe Of．a slnglc criterion，SuCh as most standardized tests provide． willfbrce one or two situations：Eithel．（a）the Criterion willcomc entirely h■Om OnClevel．and hence otherlevels ヽ、・i11beignol、Cd、Or（b）thc criteria willbe sumlned acrosslevcIs．and di汀ヒr－

encessuchasmanyorthosc fbundiI＝helLSMA analysISヽVi11bc hidden．

observcd that the difrtrences between textbook groupsarequlteSmall・Itis clear that the trend isin thedata；1tis alsoclear thatthereisnosim－ ple explanation ofit・The trendis probably the result of severalfactors－amOng Which m｛Ly be included the more adequate background of teachcrs，narrOWer range OL、aptitudein the stu－ dents，increasedcapacityofstudentsfbrindividual Study，anddifhcultyin buildingsatisfactorymeas－ ures or achievement at the upper grades・The NLSMA data does not provide much evidel－Ce fbroragainstanyorthesepossibilities（OrOthers）； the NLSMA data does indicate quite clearly that the trend is there．

The difrtrence betwecn the SMSG textbook

group and most conventionaltextbook groups at any gl・adelevelislargely a contrast or compu－ tation－1evelscales on the one hand and under－

Standing or mathematicalideas asindicated by COmprehension－，application－，and analYSIS－1evel SCales on the other・There are exceptlOnS tO this Or COurSe．In the example analysISin thelast SeCtion，teXtbook group T6did as wellor better than the SMSG textbook group on most or the application－ and analysis－1cvelscales・

The scquencein which mathematicaltoplCS are taught，and the emphasis glVen them，Varies 丘om one textbook sequence to thc next．This sequence must be kept in mind carefully in inter- pretlng the NLSMA textbook comparison analysIS results．For example，it was rbund at thc end of GradeIVthatoneofthemoderntcxtbookgl・oupS was doing quite poorlv on a mcasure of division o［whole numbers－mOre than a standard devi－ ation below the grand mean．One of－the con－ Ventionaltextbook groups wasJuSt the opposite； itappeared tobedoingverywell．ArlOthermeasT ure of division of whole numbers was given at the endoi●GradeVandbothorthesetextbookgroups 、Vereright at the grand mean． the textbooks showed that the

putalotofcmphasisondivisioll then verylittle morein Gradc text barely introduced division putalotoftimeonitinGrade of mathematical topics must interpretatjon o［thedata．

Examination or

conventional one

in GradeIV and

V． The modern

in GradeIV and

V．Thesequencc

be consideredin

The example above can be extended a bit to underscore theimportanceofcxamlnlng thelong－ term development or mathematics achievcment． At the end ofGradeIVthe conventionaltext was

Clearly superior to the modern text on divisicn． At the end orGrade V they、Vere equally good． But o11a Similar measure at the cnd of－Grade VI

fbr division of－ decimals，the modern textbook group was Lもr above Lhe average and the con－

II・Papers Presented to the Seminar 34

or this chapter，itis an admonishment to use a modeltoanalyzetheoutcomesofthemathematics program to be evaluated，一O guide selection and preparation or measurlnginstruments，and to analyzetheresults．

Formative evaluation has received very little emphasis・Thisdocsnotrelegateit toaplaceor lessimportance・On the contrary，the potential fbr systematic fbl・mative evaluationsis extremely good・Yet、thereis only alimited body ofliter－ ature on fbrmative evaluation of－ mathematics programs. The absence of a thorough discuvsion and review or fbrmative evaluationisin part a subtleprotestthat the topicdeservesthcliterature lbr a rullchapter o［．its own・

Another topic，hardlYdiscussedin this chapter， is the nonco卿Itive outcomes or mathematics progl・amS SuCh as attitudeslmOtivation・and ap－ preciation・Future mathematics program evalu－ ations should glVe mOl・e COnSideration to these outcomes in addition to mathematics achievement

measul・CS・Verylittleis understood about these outcomes and their interaction with cognitive outcomes．Much workis neededin theory build－ 1ng，eXperimentation，and the development o［ satisLbctoryinstrumentation fbr assesslng nOn－ COgnitive outcomes・

。＼■上∫．1J．」砧か机

Edited by：．Jamesl＼’．Wilson・Ⅰ・eOnard S・Cahen， and Ed“▼ard G・Beglc

Publishe（1by：SchooIMathe－rlatics Studv Group， StanLbrd Univcrslty

Ntlmber

l．ズーP坤乙∠ん～／J抑 乃∫gJi抽r／β・r・Partト ∧ al－d B・ 1968．

2．ユ∵P呼〟J〟／／0′～ 乃∫Jβ抽rナビ∫・Parls A and B・ 1968．

3．Z－Pゆ〟J〟／JクJ！乃∫Jβ〝〃gr∫ど∫・1968・

4．」加∫m少雨乃‘mゴ 5祝抗証跡！fh小机払F q′・ふ P〃ク〟Jd‘∫0乃 ∫cβJg∫・1968・

5．ββ∫√γ妙言0′ld柁d ∫／αょぎ∫′JcαJPro函㌻／7ど∫ イ1’－ P坤〟JdJfo花∫c〟Jど∫・1968・

6．加血神前‘血■ 乱流血最 戸叫刷血げJ一 夕坤‡JJβJ；0乃∫cdJど∫・196臥

7．れ毎 工血旬坤血 げ m血・勘’TlhonlaS A・ Romberg andJamesl＼’▲1＼’ilson・1969・

8．∫Jd′言Jfg√dノ アrロビど血「ど∫ d〝d Co刀坤‡一Jどr Prログα′J～∫・B）「

▲James W・＼＼’ilson and L Ra）’Carr），・（In Preparation）

9．〃川＿れ扉」仇由し】968．

10．ね冊那 り＝肋加咄血 ノ銅品肋而 わ＝如血 “．d乃d 6：ズー印画加加．B）－ L・R叩 Ca叩・ and、J．Fredl＼’eaver・1969・

1l．靴招m∫ qr〟助血Ⅷ危∫ A混拙鼎面L ㍍（れ壷 7a71d8：X－PopulaLioTl．By L・Ray Carry・1969・

12．れ加Ⅶq／朗加如痛払 一勅崗脚酬ぃ玩 Cねぬ 7and8：－Y－PoPtLlalion．By Gordon K・McLeod andJeremyKilpatrick・1969・

Co〃抽／JJ〃．ギ〟βJ〃Jげた∫

The rev（）1uLionin schoolmathematics of－ the

PaSt decadc orso has consisted mainlyorthepro－ POSal、development．and dissemination oi’alterna－ tive curriculum programs．Theiiame fbrces that spawnedthisrevolutionhavebroughtabouta modest amount orworlくin as5eSSing thc outcomes Of these programs・I3ut progressin mathematics PrOgram eValuation has been slow．Thisis partly because the obiect of an evaluation－ a mathematicsl）rOgram－；s a very complex phe・ nomenon・Progl■eSSin mathematics program eva－ luation has also been dclayert because of a lack or powerfulevalua霊ion tcchniques and a theorctical丘amework fbr organlZlng SuCh evalua－ tion：and because orthc11eCeSSlty tOdevelop ade－ quateinstruments fbrlhe assessment or mathe－ matlCSlnCOmeS．

Thischapterhascrnphasizedsomeofthegcneral guidclines fbr mathematics program evalualion rather than attemptll－g tOI）rOvide specific advice Onhowtodoanevaluatiol一．Suchspeci6cadvice JuSt does notexistin anYSatisfhctory fbl・m－agaln because the outcomes of mathematics instruction

are very complex a一一d because a wide varicty of’ evaluation activitiesis nceded．Grobman11；has

reviewed some evaluaLion activitics or ct11TicuLum

prq）eCtS and oLrt：rS SOme advicc on how to do an evaluatiol一・］臥1t．aS hcr titlc sし1ggeStS．1Vhat has

been doneandwhatis availablerepresentsonLy StartlIlgpOLnt・

Itis to be hoped thaHhe dcvclopment orpra－ Cticala一一d e［rbctive evaluation procedures willbe

Carried out without dc・1ay so that schooladnli11i－

StratOl、s and othel・intcrestcd parties can obtain the infbrmation needed Lbr bcttcl・ぐurriculum decision making・

NLSMA has dcmonsけatcd Llle Varylngl）auCrnS ofl・eSultsthatcanoccurwhenal10rganizedmulti－ p］e－OutCOme Vicw or malhematica11eal’nln3‡is taken・The evaluation or a mathematics pro－ gram thatis based on a slllgle criterionlVi11at best producelimitedinlbllmation；at WOl・Stit Can bemisleading by Lbil；Ilg tO Showimportant characteristics．

A modelor mathematics achievcment caIlbe

extl、emely uscLhl【btlan eValuation pt．q］eCt． Within thc basic ftamcl～▼Ork or the modcl．the

dimensions canbestatediIlgl・eatel・detaillbl・finer distinctions to suit the purposes of a particular evaluation．Ir thel・eis a”hol、▼tO doit”aspect

46．Hulda GrobmELn，Eualua（lollAcL；uiLle∫ Qf’cbrr；－ √〟J！J∽PrむどrJJ：d5’加／f／ig P∂f乃f（AERA ～【ono・

grapllSeries on Currictllum Evaltlatiol一，nO，2 ［Chicago：Ran（lMcr（a11y、1968］）・

2．3 E．G．Begle andJ．W．Wilson 35

】3・Pl】JJ〝〃∫（イ．け（－Jん一肌浴L－J．」‘ノ～J（1こ川川／～′～ C晶ノ（・

9：－Y・Population．ByJcrcmy Kilpatrick and Gordon K．McLeod．1969．

14・タβ′Jgr月∫ 扉■ 九k痛m血克∫ ノ毎払mⅧ∽J査〃G加滋

10：－Y－Populalion．By Gordon K・McLeod and Jeremy Kilpatrick．1969．

15．Pβ‘′βr〃∫ q／Aオα才力β椚β′f√∫ dcカ言どぴβmど柁′ さ乃 Crαdβ

Jl：－Y－PppulalioTl・ByJeremy Kilpatrick and

Gordon K．1｛IcLeod．1969．

16‥靴鮎椚び（ゾ ー灯油転m需打．加而跡佃酬イ よ〃（れ痛

ノ0：一乙月掛品最血．ByJamesl＼7・1Vilson・1969・ 17．P▲JJJ（・mJ（イ．1JLJ／／汀耽浴‘・∫．1（1∧∫（・ご川‘リニ／f′J（；＝ム

ノノ：－Z一八郎止浴川．ByJanleSl17．＼Vilson．1969． 18．鳥肌椚持 り′ ノ仇沼閻M抗了 ノ＝巌路冊両＋玩の皿

12：－Z・Population．By Thomas A・Romberg andJamesl＼’．1＼’ilson．1969・