DOCUMENT RESUME

ED 055 103 TM 000 830AUTHOR Prediger, Dale J.TITLE Converting Test Data to Counseling information.INSTITUTION American Coll. Testing Program, Iowa City, Iowa.

Research and Development Div.PUB DATE Jun 71NOTE 21p.

EDRS PRICEDESCRIPTORS

ABSTRACT

MF-$0.65 HC-$3.29Aptitude Tests; Computer Oriented Programs; DecisionMaking; Discriminant Analysis; Feedback; FieldStudies; High School Students; *Interest Tests;Longitudinal Studies; *Profile Evaluation; Scores;*Test Interpretation; *Test Results; Test Validity;*Vocational Counseling

Intormation from tests, when considered in thecontext of decision theory, can play an important role in vocationaldevelopment. This role is primarily one of stimulating andfacilitating exploratory behavior. Objective procedures forconverting test data into counseling information are discussed andillustrated with special attention being given to similarity scoresand similarity score profiles. The similarity score profiles, whenused in conjunction with success estimates, avoid many of thepitfalls inherent in Parsonian approaches to test interpretation. Acomputer-based system incorporating the above data-informationconversion procedures was developed and field tested with potentialvocational school students. Longitudinal validation analysesinvolving more than 1,500 students and 20 interest and aptitudemeasures provided the empirical base for the data-informationconversion procedures. Similarity scores and success estimates weredeveloped for each of the 12 vocational program areas in which thestudents enrolled. These reporting procedures were then field ÷--7tedwith an additional group of 900 students enrolled in 1? ols.Illustrations of the similarity score reporting procedu.. mustudent reactions to them are provided. (Author/CK)

U S DEPARTMENT OF HEALTH,EDUCATION & WELFAREOFFICE OF EDUCATION

THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROMTHE PERSON OR ORGANIZATION ORIG-INATING IT POINTS OF VIEW 04 OPINIONS STATED DO NOT NECESSARILYREPRESENT OFFICIAL

OFFICE Of EDU-CATION POSITION OR POLICY

zpNvEIFFTING-TkitYpaTAi-ci.,...0.90NsiONG NFORWATION

tika Pordi

TEStINGFROGRAJ

ABSTRACT

Part 1: Rationale

Information from tests, when considered in the context of decision theory, can play animportant role in vocational development. This role is primarily one of stimulating andfacilitating exploratory behavior. Objective procedures for converting test data into counselinginformation are discussed and illustrated with special attention being given '.o similarity(centour) scores and similarity score profiles. Alternative names emphasizing the guidance useof these procedures are suggested, i.e., exploration indices and exploration maps. Thesimilarity score profiles, when used in conjunction with success estimates, avoid many of thppitfalls inherent in ?arsonian approaches to test interpretation.

Part 2: Field Trial with Feedback

A computer-based system incorporating the above data-information conversion procedureswas developed and field tested with potential vocational school students. Longitudinalvalidation analyses involving more than 1,500 students and 20 interest and aptitude measuresprovided the empirical base for the data-information conversion procedures. Similarity scoresand success estimates were developed for each of the 12 vocational program areas in which thestudents enrolled. These reporting procedures were then field tested with an additional groupof 900 students enrolled in 12 high schools. Illustrations of the similarity score reportingprocedures and student reactions to them are provided.

2

CONVERTING TEST DATA TO COUNSELING INFORMATION'

Dale J. Prediger

PART 1: RATIONALE

This report is concerned with objective pro-cedures for converting test scores and other datainto information which is relevant to a counselee'seducational and vocational plans, decisions, orproblems. Validity studies are crucial to data-information conversion procedures. However, thesestudies as usually reported, are of little direct helpto the counselor who has a student and a s -t of testscores before him. While it is true that an accumu-lation of validity studies performed within a

theoretical framework may support various uses ofa particular test, the task of converting a

counselee's test scores into usable information isleft undone. Typ;cally, the counselor can find thestanding of a counselee in some norm group; afterthat, he is on his own. Professional knowledge,clinical juolciment, crid personal sensitivity alwayswill play crucial roles in test interpretation.However, objective data-information conversionprocedures can make the counselor's job mucheasier. Just what does a percentile rank of 63 onthe XYZ Mechanical Aptitude Test say to thestudent and his counselor?

Local Validity Data and Decision-Making

Almost 15 years have passed since Dyer (1957)made a convincing case for local studies of testvalidity. Dyer was only one of a host of measure-ment specialists who cautioned test users aboutaccepting tests on the basis of face validity orassuming that one o two validity studies con-ducted in some other setting provide sufficientevidence that a test wc u Id be useful in their settingand for their purposes. Recent reviews ofvalidation research (Gh'zelli, 1966; Prediger, Waple,& Nusbaum, 1968) have reinforced this caution.

An excellent discussion of the role of localvalidity data in vocational guidance and decision-making has been provided by Clarke, Gelatt, and

Levine (1965). In their discussion, attention wasfocused on the process of decision-making, with

'With minor modifications, Part 1 of this report will appear as anarticle in Volume 18 of the Journal of Counseling Psychology underthe title "Data-Information Conversion Procedures in Test Inter-pretation." Part 2, also with minor modifications, will appear as anarticle in the Journal of Educational Measurement under the title"Converting Test Oata to Counselinc Information: System Trialwith Feedback,"

The studv reported in Part 2 of this reJort was partially supportedby a contract with the Office of Education, U.S. Department ofHealth, Education, and Welfare. Supr,ort was also provided by thePenta-County Vocational High SchcJI (Perrysburg, Ohio) and theUniversity of Toledo.

)rmation on the possible outcomes of variousrses of action being seen as a necessary if noticient condition for wise decisions. Examplesocal validity studies were given to illustrate theelopment of objective probabilities useful incational planning. As with Dyer, use oferience (expectancy) tables was emphasized.)sequent articles (Gelatt & Clarke, 1967; Katz,i6, 1969; Thoresen & Mehrens, 1967) elabo-K1 on the role of objective probabilities in;ision-making, the influence of objectivebabilities on subjective probabilities, and thewactions among subjective probabilities, choiceion utilities, and personal values.

Katz (1963, 1966), in particular, was careful toshow how the decision-making process is related tothe broader process of vocational development.Results from the massive Project TALENT valida-tion studies also have been placed firmly within thecontext of vocational development theory anddecision theory (Cooley & Lohnes, 1968). We havepassed the era in which the Parsonian concept oftest interpretation could be viewed as the epitomeof educational and vocational guidance. However,the above studies and formulations leave littledoubt about the continued importance of testinformation, when properly validated, in thevocational-development process.

Bridges between Data and Information

Soldrnan (1961) described three objectivedges between test scores and their meaning for

counselee: the norm bridge, the regressiondge, and the discriminant bridge. Most of our-rent data-information conversion proceduresisist of some form of the norm bridge. AsIdman noted, the norm bridge is an incornpletedge since test norms simply permit one toimate standing in some group and do not, per se,licate the implications of this st:Iression bridge, however, is a complete bridge)m test scores to their implications and, as such,Wily lends itself to data-information conversion.ually,, the implicatio ls are h the form of successadictlions obtained v a e.41aerience tables orjressiion equations.The third bridge ncril'ed by Goldman, the dis-

crirninant bridge, provides an objective measure ofa counselee's similarity to various criterion groups.Discriminant analysis techniques, when combinedwith the similarity score procedures dedeloped byTiedeman, Bryan, and Rulon (1951), permit thecomparison of a counselee's test results with thoseof various criterion groups along the major dimen-sions of test data that differenti;-- "e the groups. Thecomplen nt., natur_ .idrity id suessestimates was first discussed 20 years ago (Rulon,1951; Tiedeman et al., 1951). Few ,counselors,however, are familiar with the characteristics ofsimilarity (centour) scores or their pozItial role intest interpretation. These topics have TGeved littleattention in testing texts or test 1- tel Jretationmanuals. For this reason, an illustraticm is providedin the following section.

Data-Information Conversion via Similarity Scores

Consider the infoirmat-:on nc-3eds of Fred, ;:) highhoof student thinking about enrofling in a

)cational-technical education program. Sim; larityore procedures applieo: to red's high schoolade record and test scores cc9,:ild result in a report

24

indicating Fred's similarity to succes.7U1 and satis-fied students in various programs. Ir the examplethat follows, the similarity scores sl.Fown in paren-theses after each of the vocational p=gram.: are ona scale ranging from 0 to 100 with 00 represen-

:he higilest degree of similarity. The closer5 scores on the relevant tests are to the test; of the typical successful and satisfied stu-in a vocational program, the higher hisrity score will be for that program. Fred'sirity score report might look like this: voca-1 horticulture (87), carpentry (41), com-a! art (28), auto body (26), distributivertion (25), auto mechanics (14), radio-TV(3), and data processing (1).

us, on the basis of the measures used, Fred'sides and interests are most similar to studentscadonal horticulture. Carpentry ranks second,hree other programs are in an approximate tienird. Fred is least similar to students in data!ssing and radio-TV repair.this exampie, test data have been transformedinformation that is directly relevant to one of

the major functions of tests in educationsl andvo c at ion a I gu idancefacilitating exploratorybehavior. Fred's counselor might use the similarityscores quite advantageously in stimulating Fred toexplore the program options available to him. Forthis reason, similarity scores might be moreappropriately called exploration indices; that is,indices which suggest choice options a counseleemay want to explore.

Of course, the similarity scores should not beused alone in facilitating exploratory behavior.Their potential value in this example lies insuggesting vocational possibilities that might nothave been recognized otherwise. The degree towhich Fred explores these possibilities will be afunction of his value system and the opportunitiesprovided him.

Complementary R ole of Success Estimates

Iccess estimates obtained from regressionfses or expectancy tables also can be used totate ep tory behavior. For example, Fredt be encouraged to explore the vocationalrams for which he is predicted to receive the1st grades. However, estimates of success mightlore appropriately incorporated into the actual)ration process where they can be consideredonjunction with a host of other relevantors. After all, Fred may not place much valuenaking high grades. His similarity scores couldtify vocational programs in which he is similar"successful and satisfied" enrollees. Hisable level of success could then be determined

further exploration. Thus, a two-stageegy is suggested with similarity scores being

to stimulate and facilitate exploration and.9ss estimates being an important considerationig the process of exploration.ther considerations also indicate the need forion in the use of success estimates as theary basis for facilitating exploration. Consider,example, an experience table showing theionship between the scores on some test andes in carpentry. Could this table be used

35

appropriately with Sally, Fred's sister? Can Sallybe considered similar to the group from which theexperience table data were obtained? To whatdegree would the trends shown in the results applyto her? Similarly, to what degree would successpredictions in radio-TV repair apply to Fred(similarity score = 3)? Can we legitimately useFred's test scores to predict lab grades in cos-metology? Or, in another context, are we justifiedin comparing a high school senior's college fresh-man GPA predictions in engineering, humanities,education, physical science, or business? Thesequestions, recently discussed by Rulon, Tiedeman,Tatsuoka, and Langmuir f1967), need furtherinvestigation. A "reasonable" degree of similaritybetween counsolees and the validation samplemight well be an appropriate prerequisite for theuse of success estimates in counseling.

A second difficulty with success predictionsresults from the well-known "criterion problem."Obtaining a suitable criterion of success in educa-tion and training is difficult enough, but when onemoves into the world of work, the definition andmeasurement of success become infinitely morecomplex (Thorndike, 1963; Thorndike & Hagen,

969). Members of various occupations or occupa-onal clusters can be readily identified, however.3ross criteria for identifying "successful andatisfied" occupational ,group members can also be

applied.) This is all that is needed to permit use ofsimilarity score procedures. Thus, data-informationconversion would be possible.

Overcoming the Profile Problem

Judgment of a coumelee's similarity to variousriterion groups is not new in counseling. Certainommercially available inventories, e.g., the Strong(ocatiorial !nterest Blank and the Kuder Occupa-ional Interest Survey, provide scores giving directid in the process. However, special test construc-ion and criterion group requirements presentormidable barriers to the development of suchrwentories. In addition, existing inventories coveronly one area of measurement, usually interests.limultaneous coverage of other areas (e.g.,Iptitudes, aspirations, biographical data) is not

'ctical throuc"-i use of these special test construcn chniques.Profiles showing the typical performance of

,arious criterion groups often have been providedor other tests and inventories. Counselors are:xpected to use these profiles by noting the;imilarity of a counselee's profile to the profiles of:he criterion groups. Anyone who has engaged in.his process does not need a description of whatlas long been known as the profile problem:Tiedeman, 1954).

Similarity scores take the guesswork out ofProfile comparison; but unless they are used in;onjunction with discriminant analysis procedures,they fail to deal directly with important aspects ofthe profile problem, e.g., do the criterion groupsactually score differently on the measuresinvolved? If so, what are the important measuresand how should they be weighted? Unfortunately,a set of similarity scores can be obtained frompurely irrelevant variables.

For the above reasons, discriminant analysisprocedures are often used in conjunction withsimilarity scores. These procedures enable one todetermine whether the criterion groups are, in fact,differentiated by the measures. If so, the measurescan be weighted and combined into independent

factors (discriminant functions) that maximizecriterion group differentiation. Furthermore, thenumber of factors needed for criterion groupdifferentiation can be determined. Thus, one may,and usually will, find that two factors account formost of the discriminatory power of a set ofmeasures when applied to criterion groups relevantto educational and vocational counseling. Finally,the nature of the factors can be identified bynoting the measures that correlate highly withthem.

Equations resulting from discriminant analysiscan be used to calculate criterion group positionson the discriminating factors. These positions, inturn, can be plotted on a coordinate grid with thevertical and hori7ontal aXPS representing the twomajor factors. In the same manner, the factorscores of a given student may be calculated andplotted. The student's position on the coordinategrid then may be compared visually with thecriterion group positions.

This technique for graphically depicting a stu-dent's similarities and dissimilarities was implicit inearly work on the profile problem (Tiedeman,1954) and was specifically suggested more than 10years ago (Whitla, 1957; Dunn, 1959). However, ithas been mentioned only occasionally in theprofessional literature since then (e.g., Baggaley &Campbell, 1967; Cooley & Lohnes, 1968) and hasreceived little attention in testing texts. Rulon andhis colleagues (1967) have provided a detaileddiscussion of the rationale underlying discriminantanalysis and similarity score procedures along withample illustrations of the resulting graphicalsolution to the profile problem. However, inpresenting the illustrations, emphasis was placed onrepresenting the geometry of the statisticalprocedures. Little attention was given to testinterpretation applications of the illustrations.

Data-Information Conversion via "Similarity Score Profiles"

le graphical procedures described above canseveral imp ortant functions in data-

mation conversion. Application of these.dures to the problem faced by Fred and hisselorconsideration of vocational programmsis illustrated by the similarity scorele presented in Figure 1. Note that the twonsions or factors that best differentiate thetional programs are represented by the axes of;oordinate grid. Each factor is scaled with a

of 50 and a standard deviation of 10 formts in all vocational programs combined. Of10 aptitude and interest measures used in thefsis on which the profile is based, those havingir correlations with an absolute value of .40 or

are listed as factor anchors. (The following)ols are used to indicate level of correlation.

r > .59; * = r of .50 - .59; no * = r of .40 -

ed's factor scores of 56 and 36 for Factors 12, respectively, have been plotted on theJinate grid along with the mean scores of the

vocational programs. (In several instances,auto body and welding, related vocational

rams have been combined into one group.)ellipse surrounding the vocational horticultureip encloses the factor scores of approximatelyof the students in the group and is analogous

ne scatter diagrams often seen in discussions ofelation. Similar, but not identical, ellipsesd be used to represent the scatter of factores for the other vocational programs. Onese should be sufficient, however, to obtain aI estimate of program overlap.he size of Fred's similarity scores is reflected indistances between Fred and the various voca-al programs on the profile. Although thetionship of similarity score size to distance isexact, it is sufficiently high to give a useful

al picture of degree of similarity. For example,i's position is quite close to that of the3tional horticulture group, the group for whicheceived his highest similarity score, i.e., 87. Onother hand, programs for which he has lowlarity scores are much farther away.y comparing Fred's position on the profile

with the positions of the various vocational pro-grams, one can obtain valuable insights into thereasons underlying Fred's similarity scores. Fieldtrials with potential vocational school students(Prediger, 1970) have shown this is important,since use of similarity scores by themselves oftenleaves the counselee with the question, "But whydid the scores come out like that?" In Fred's case,we see that both he and the typical horticulturestudent score only slightly above average on Factor1, while Factor 2 scores suggest relatively strongartistic interests and rAlatively low mechanicalreasonina aptitude and scientific interests.

Fred may wonder why his similarity score forarea H was so low. From the profiie, we see thatthe Factor 2 scores for Fred and area H studentsare relatively far apart. In addition, the position ofarea H students on Factor 1 suggests that they havestronger clerical skills and interests than Fred.Similar reasoning can be followed in comparisonswith the other groups represented on the profile.

In a sense, similarity score profiles provide thestudent with a map of an important domain ofchoice options. In helping Fred understand thepossible reasons underlying his simiiarities anddissimilarities, one is helping him to project certainaspects of his self into the choice domain. Hisposition in that domain is shown, and he isencouraged to relate his position and character-istics to those of students who have elected variouschoice options. This might be viewed as a type ofvicarious exploration within the dimensionsmapped by the similarity score profiles. Perhaps amore appropriate, guidance-oriented term for theseprofiles would be "exploration maps."

Variation in the characteristics of students whohave made a specific choice will be clearly evidentfrom the ellipse shown on the profile (or explora-tion map). If ellipses are shown for all of thevocational programs, overlap among the variousgroups also will be evident. Criticisms of counselingapplications of trait and factor research sometimesare based on the fact that divergencies withincriterion groups and similarities among criteriongroups are concealed. Similarity score profilesreveal these facts of life. At the same time,

hanical Int.door Int.

75

ical Int.PC GPA 70iputational Int.ical Parc. Apt.

65

60

55

50

45

Fred Cartesian Carpentry

Auto Body,Welding

Auto & Ag. Mech.Machine Trades

Voc. Horticulture

Commercial Art,35 Printing, Drafting

30

lea! int.PC GPA

nputational Int. 25ical Pere. Apt.

:hanical Int.:door Int.

SI)=-10for both factors

Radio & T.V.Electronics

DistributiveEducation

Data Processing,Account Clerk

25 30 35 40 45 50 55 60 65 70

High* Artistic Int.

Low* Mechanical Reas. Apt.

Scientific Int.

Factor 2

Fig. 1. Similarity Score Profile for Fred Cartesian.

6 8

High* Mechanical Reas. Apt.

Scientific Int.Low* Artistic hit.

nces among criterion groups, if present inrta, are presented in a manner that permits)unselee to "try the various groups on for

;hould be noted that an extreme score onne of the variable shown as factor anchorsby itself, have a major influence on Fred'sscores. Hence, Fred's test score reportsbe consulted while discussing his similarity

profile. (Fred's counselor would probably

want to do this anyway in order to facilitate Fred'sparticipation in the test interpretation process.)The factor positions of the vocational pi-ograms, onthe other hand, are unlikely to be subject to theinfluence of extreme scores on one or two vari-ables. These positions are, after all, based r',1 groupmeans rather than the scores of a singieIt is well-known that the spread of group means ona variable will be much less their the spread ofindividual scores.

Similarity and the Status Quo

ce the reasons underlying Fred's similarity; have been determined, Fred and his coun-may be able to develop a program of activitiestudy that will increase his similarity score forn criterion group. The feasibility of doing this

of course, depend on the variables involved.wer, the suggested strategy represents anple of a counseling application of test datafacilitates change in the status quo rather thany a representation of it.'other strategy can be used in combating the

quo, in this case, a strategy that reflectsutional goals. Suppose a college is unhappythe characteristics of members of certain

:War groups. For example, its enginaeringnts need more math, or its business students: know how ty spell. As French (1956) hasi, the similarity score technique, when usedunselected criterion groups, might result in

Jraging students with similar deficiencies tothese areas. Use of regression-based success

otes in conjunction with similarity scoresd guard against this. In fact, Tatsuoka (1956)Jeveloped an index which combines successrimilarity estimates into one, overall probabili-Tie index has considerable promise for place-. applications. However, guidance applicationsar limited because the joint index makes itissibie for a counselee to place separate valuesle success and similarity estimates. Instead, he

is confronted with a single number somewhatanalogous to a joint index of height and weight.

Two additional procedures can be employed bythe college to combat possible perpetuation ofcurricular group deficiencies through use ofsimilarity scores. Both involve the strategy ofredefining the criterion groups. The first, anempirical procedure, involves the selection ofcriterion group members according to predeter-mined definitions of satisfactoriness. Thus,engineering students with poor math backgroundsand business students who flunk the freshmanEnglish course could be eliminated from thecriterion groups. The second, ad hoc procedure forredefining criterion groups involves the arbitraryassignment of group position on the test variablesor discriminant factors that are thought to beimportant. For example, current engineering stu-dents could be assigned a much higher mean on amath proficienc%: exam than they actually have.Or, the position of the mechanics and machinetrade group (area 8) shown in Figure 1, might bemoved to the right on Factor 2, thus requiringcounselees to have a higher level of mechanicalreasoning aptitude and scientific interest to appearqimilar. In each case, parallel adjustments could bemade in the calculation of similarRy scores.Further discussion of this strategy for combatingthe status quo has been provided by Rulon et al.(1967).

Some Technical Considerations

)ne obvious limitation of similarity score,files is the difficulty in representing more than) tests or factor dimensions at one time.criminant analysis, fortunately, results in a

uction in the number of dimensions needed toresent criterion group differentiation. Usually) dimensions are sufficient. Nevertheless, theilarity score profile technique can be used withae factors by developing a series of profilesresenting group positions on the first twotors for successive values of the third (Prediger,'0). Instances in which more than three factorsuld be required to represent criterion groupAmination are rare, judging from the results ofriminant analyses reported in the literature.;o far, the discussion has involved only one or) general approaches which have been used toelop estimates of similarity to various groups.

similarity score approach illustrated above)s. an independent estimate of a counselee'silarity to each of the criterion groups undersideration. The second approach, which is3d on the maximum likelihood principle,vides probabilities that take into account thetive degree of a counselee's similarity to each ofcriterion groups (Cooley & Lohnes, 1962). Thething scores are given as decimal probabilitiesch total 1.00. Thus, if five criterion groups ofal size were involved and a counselee wereally similar to all five, his scores via the second

approach would be .20 for each group. This wouldbe true whether his- similarity scores were all 99 orall 1. That is, relative degree of similarity ratherthan absolute amount is determined.

As another example, suppose that a counseleehad factor scores of 70 and 25 on Factors 1 and 2,respectively, in Figure 1. Through use of themaximum likelihood approach, he would receive avery high score for area E, possibly even higherthan Fred Cartesian's score. This would occurbecause the counselee's relative similarity to stu-dents in area E is much greater than his similarityto students in the other areas. However, his actualdcgree of similarity is quit:: low; as can be seen byplotting his factor scores on Figure 1. Since acounselor (and counselee) would undoubtedlywant to know this, the educational-vocationalcounseling applications of the maximum likelihoodapproach are limited.

If, on the other hand, one is interested inallocating individuals to treatment groups so as toachieve some overall goal of efficiency or correctplacement, the maximum likelihood procedurewould be an appropriate technique to use. Whendifferences in criterion group size and dispersionare taken into account, the predictions based onmaximum likelihood procedures maximize theclassification accuracy achievable by the predictorvariables (Cooley & Lohnes, 1962).

Recapituhtion

he following seven points have been empha-in the foregoing discussion of objective

,niques for data-information conversion:

. Information from tests, when viewed in thetext of decision theory, can play an importantin vocational development.. This role is primarily one of stimulating and

facilitating exploratory behavior.3. Tw o d a ta i n f ormat ion conversion pro-

cedu ressim ilarity scores and success estimatesare crucial to this role.

4. On the basis of both logical and technicalconsiderations, similarity scores are more appropri-ate than success estimates in stimulating andfacilitating exploratory behavior. Success estimates

represent an important consideration during theprocess of exploration.

5. Similarity scores eliminate much of the guess-work inherent in test profile interpretation.

6. Graphical procedures are available to providehelp in understanding the reasons underlying agiven counselee's similarity scores, thus avoiding

the take-it-or-leave-it aspects of test interpretationbased on similarity scores alone.

7. These procedures can also facilitate use of testdata to initiate changes in counselee characteristicsand/or the characteristics of groups representingvarious choice options, rather than merely torepresent the status quo.

Implications

All of this is likely to be of little comfort to theconscientious counselor or personnel worker whohas neither the time, training, nor inclination tobecome involved in data-related duties as versuspeople-related responsibilities. Few test userswould argue about the need to strengthen thebridges between test scores and their meaning forthe counselee; but how is the job to be done?

Two major stumbling blocks to data-informationconversion and the prerequisite validity studies aredata collection and analysis, fields in which greatstrides have been made in the last 10 years throughthe use of computers. In addition to providing helpwith record-keeping functions, computers havemade time-consuming and/or highly sophisticateddata analyses economically and psychologicaHyfeasible. Approaches to data-information con-version that have been available for some time arenow possible on a large scale. This is nowherebetter illustrated than in the work of ProjectTALENT staff members, in particular, Cooley andLohnes (1968). However, practical applications ofProject TALENT findings are limited because ofthe multitude of measures involved and theirunavailability to practitioners. Unless the test

equating studies proposed by Cooley and Lohnes(1968) eventually are undertaken, counseling useof Project TALENT data will be restricted tospecial programs such as Project PLAN (Flanagan,1969).

The computer-based system illustrated by theCooley-Lohnes studies is generalizable to othersettings, however. Development of such systems,either by educational institutions or private enter-prise, and provision for wide access to them areessential to any major improvements in test inter-pretation procedures. If systems for data-informa-tion conversion were readily available to the localpractitioner, his only tasks would be to askimportant questions of his dataand then help hiscounselees use the resulting information. Much ofthe work required in data preparation could becompleted by clerical help or one of the severaltypes of guidance technicians proposed by Hoyt(1970).

Data-information conversion systems will neverreplace professional knowledge, judgment, andexperience. However, they can go a long waytoward moving test interpretation beyond the eraof squint and tell.

PART 2: FIELD TRIAL WITH FEEDBACK

loted in Part 1 of this report, the vocationalHor who looks to empirical data for aid interpretation faces several problems. One ofost serious is the paucity of validity dataling many uses of tests in vocational!ling. Equally serious, however, is the limited:al value of the validation studies available.nmonly reported, these studies provide the!lor with no direct help in bridging the gapri the test score and its meaning for thedee. Indeed, the counseling use of info ma-btained from validation anaPyses is seldom

reported in the literature. Test manuals do some-times present experience tables, regression equa-tions, or group profiles based on restricted samples.However, the counselor usually has only a set ofnorms, the suggestions in the test manual, and hisown experience to guide him. Empirically basedprocedures for converting test data to readilyusable counseling information, while available intheory, are largely lacking in practice. Such proce-dures will not remove all of the guesswork from-zalt interpretation. Howevel , they can place testirca-erpretation on E more objective foundation.

Problem

primary purpose of this study was top and demonstrate a system for convertingita to locally validated counseling informa-leactions of counselors and students to thethis information in 12 field-test high schoolsimmarized. In the development of theJ t e r- b a sed data-information conversion, heavy reliance was placed on the decision-d paradigm for local guidance researched by Clarke, Gelatt, and Levine (1965),le multivariate descriptive strategies repre-by the work of Cooley and Lohnes (1968,Longitudinal validation studies involving

tion and discriminant analyses were con-to provide the empirical base needed for

formation conversion. Two conversion pro-s were used: experience (expectancy) tablesiilarity (centour) scores.ough similarity score procedures were devel-?0 years ago (Tiedeman, Bryan, & Rulon,their use in an ongoing guidance program isare. In Figure 2, an illustration of similarityis presented in conjunction with the studentform used (with slight modifications) inling potential vocational education students

tr,

/ 211

in the field-test high schools. The similarity scoresfor Fred Cartesian, a fictitious counselee, appear onthe "computer-printed label" to the right center ofthe form. As explained on the form, these scoresreflect the similarity of Fred's aptitudes andinterests to the aptitudes and interests of successfuland satisfied students enrolled in eight vocationalprogram areas.

Previous research has shown that the relation-ships between antecedent and outcome variablesrequired for data-information conversion are likelyto vary from one setting to another (Bennett,Seashore, & Wesman, 1966; Ghiselli, 1966; Pass-more, 1968; Prediger, Waple, & Nusbaum, 1968).Hence, the nature of these relationships must bedetermined for the setting in which data-information conversion procedures are to be used.For this reason, answers to the following researchquestions were sought:

Similarity Score

1.1s it possible to differentiate successful andsatisfied students enrolled in 12 vocational educa-tional programs through use of aptitude and

EXPLORING PENTA-COUNTY VOCATIONAL PROGRAMS

If you are thinking about going to Penta-County, you probably face a difficult decisionthe choice of whichvocational program you wish to enter. This report won't tell you what to do. But it will provide some informationthat can help you explore what Penta-County has to offer. The vocational programs at Penta have been groupedinto the 12 areas listed to the left of the box below. Your counselor will give you a label that fits over the box. Thislabel contains scores giving a rough estimate of your similarity to students in the different programs. These"similarity scores" are based on aptitude and interest tests you have taken in the last year.

THE KEY POINT IS THIS: The higher your score for an area, the more similar you are to students in thatvocational area. The highest score you can get is 100. The lowest score r.c.). A zero score for are.. E would mean"that your test scores do not look like the scores made by students in c horticulture. It's still O.K. toconsider horticulture, however. Test results, after all, don't give the whol oitucu. You must coAsider them alongwith all the other things you know about yourself and Penta-County prograrns.

THE BEST WAY TO USE THIS REPORT is to find the vclational program- !7.1 vvrich mu score the highest. Theseare programs you might want to explorefind out more about. Perhaps you tn,ould nc-t have thought of themotherwise. You certainly don't want to overlook a good possibility. There's too 1-77 _:ch at z7-3ke.

VOCATEONAL AREAS

Mostly boys enrollA. CarpentryE. Auto & Ag. Mechanics,

Machine TradesC. Radio & TV, ElectronicsD. Auto Body, Welding

Both boys & girls enroll\/ E. Vocational Horticulture

F. Distributive Education\/ G. Commercial Art,

Printing, DraftingH. Data Processing,

Account Clerk

Mostly girls enrollI. Child Care Aide or

Ass't., Community &Home Service,Dietary Aide

J. Cosmetology, DentalAssistant

K. Co-op Office Education,Office Machines

L. High Skill Steno

SAMPLE LA0EL

032154 FRED E CART=_Th 10/14/69

STUDENT SIMILARITY SCORES -OR P-C VOC. PROGRAMS

AREA= ABCDEF GHIJK LSCORES= 41 14 03 26 87 25 28 01

RANK= aAREA= ABCDEF GHIJK LPROFILE FACTOR SCORES: 96, 36

SO HOW DO YOU USE 'f H E SCORES ON YOUR LABEL?

First, paste your label on the box shown above. Next, find and rank your top 3 or 4scores. Give the highest score a rank of 1, etc., and write the ranks on the line belowyour scores. Finally, put a check mark beside the names of the 3 or 4 areas rankingthe highest. These are the areas that your test results suggest you might want to findout more about. Some students receive low scores in all of the areas. This simplymeans that the test results aren't of much help in suggesting areas to explore.Whether your scores are "high" or "low," your counselor can help you figure outwhy they came out the way they did.

In order to judge how successful you might be in a program, you must also consider if you have the course work,aptitudes, and personal desire _hat is needed. This report does not tell you -72-at. However, with the help of your.counselor and your parents, you can use it along with other information al-s you exolore the programs available at

Pente-County.Fig. 2. Similarity Score Report 'Form.

12 13

interest measures obtained prior to entry intoprograms? If so,

2. Which measures are most effective and what isthe nature of the group differentiation that isachieved?

Experience rabies

3. Within each of the vocational program areas,considered separately, which of the aptitudemeasures has the highest correlation with success?

4. Is the use in vocational guidance of the best,two-variable combination of predictors warrantedon the basis of the level of correlation achieved andthe contribution made by each predictor?

Although the answers to the previously statedquestions are specific to the sat_ j in which thestudy was conducted, the techn'ques are directlytransferable to other settings. For example, thesame questions could be asked f data obtainedfrom students prior to entry into arious occupa-tional groups or colleye majors.

Procedures

A detailed description of the research design andstatistical analyses required to answer the fourquestions on antecedent-outcome variable relation-ships has been presented by Prediger (1970). Onlya summary is given here.

Subjects and Variables

The subjects were enrolled at the Penta-CountyVocational High School, an area vocational schoolserving 14 feeder high schools surrounding Toledo,Ohio. Students entering Penta-County as juniors orseniors in the fall of 1966, 1967, and 1968 formedthe sample (N 1,684) from which various analysisgroups were identified. Although the same schoolswere involved, the analysis sample did not overlapwith the field-test sample.

Scores from the following tests and inventorieswere used as antecedent variables: GeneralAptitude Test Battery, Mechanical Reasoningsubtest of the Differential Aptitude Tests, and theKuder Prefer-ence RecordVocational. The 20measures involved were generally obtained duringthe fall of the year preceding a student's entranceinto the vocational school, although the actual timeof testing was left to the discretion of feederschool counselors. Since several Kuder scales aresubject to substantial sex differences, normalizedstandard scores based on separate percentile normsfor males and females were used in the analyses.

Score reports for the above measures wereprovided to feeder school counselors for use in

helping students explore the vocational programsavailable at Penta-County. The scores were notused by Penta-County personnel for placement ofstudents in programs. Instead, students applied forthe programs of their choice.

One additional antecedent variable, student GPAprior to entering Penta-County (PRE-GPA), wasalso available. Typically, PRE-GPA was based onall feeder school grades received during the fresh-man year and the first semester of the sophomoreyear.

Design

The 24 programs offered at the vocationalschool were grouped into the 12 areas listed atthe left of Figure 2. Regression analyses requiredto answer Research Questions 3 and 4 wereperformed, separately, for each of the 12 areas.However, answers to Research Questions 1 and 2required that discriminant analyses be conductedacross the various areas. If the analyses wereperformed on all 12 areas simultaneously, sexdifferences from area to area might obscure otherdifferences among program areas. One might findthat the interest and aptitude measures chieflydifferentiate programs enrolling girls from thoseenrolling boys, which would be of little valuesince there are better ways to differentiate girlsfrom boys. Clearly, a design was needed thatcontrolled for the effect of sex differences ondifferences among programs.

13 14

3 minimize the problem noted previously and at;arne tir provide for comprehensive analyses,12 vocational areas were organized into thewing groups: areas enrolling primarily males

503), areas enrolling primarily females (N =, and areas having a substantial enrollment ofmales and females (N = 483). As shown in

re 2, the male, female, and mixed groups eachided four areas. Separate discriminant analyses

run on the eight male and mixec2 groupsbined (the M-MF analysis group) and the eightile and mixed groups combined (the F-MFysis group). Similarity scores were also basedthis grouping. Each analysis group had about10 members, with the mixed group appearing in

analysis groups.he vocational areas included in the dis-

criminant analyses required by Research Questions1 and 2 dio not include students who (a) droppedout of school or returned to the feeder high school,(b) expressed dissalisfaction with program choice,or (c) failed to achieve a GPA of 1.8 (on a 4-pointscale) in their voc3tional courses. In this sense,students in the discriminant analysis criteriongroups can be ctescribed as "successful andsatisfied."

The regression analyses required by ResearchQuestions 3 and 4 excluded only those studentswho left school before they had established a gi:aderecord. Vocational course GPA at the time ofgraduation or dropout was used dF the criterion ofsuccess. The median number of students in the 12vocational areas involved in the regression analyseswas about 110.

Results and Conclusions

lultivariate analyses of variance conducted onaptitude and interest measures showed that

-1 types of measures, used &one or in combine-!, differentiated the vocational areas far beyond.01 level of significance. Thus, an affirmative

N3r- was obtained for Research Question 1.

ximinant analyses were used to identify theensions (factors) of test data that best differ-eted the areas. Since a large proportion of theAminating power of the measures was concen-ed in the first two factors, attention is focusedthese two factors in the results following.-he nature of the factors that best differentiatedvocational areas in the M-MF analysis group

I the manner in which these areas were differ-iated can be seen from the "similarity scorefile" presented as Figure 1 in Part 1 of thisort. (A profile was also developed for the F-MFlysis group.) The axes of the coordinate grid,ich are scaled with a mean of 50 and a standardiation of 10 for the 8 vocational programsnbined, represent the two most effective dis-ninant factors. The position of each vocationala mean (centroid) on the factors has been

plotted as a single point, as have the factor scoresof our fictitious counselee, Fred Cartesian. (Thesescores appear at the bottom of the label in Figure2.) Perspective on amount of overlap among theareas is provided by the ellipse enclosing the scoresof about 50% of the members of Group E,vocational horticulture.

Similarity score profiles provide a conciseanswer to Research Question 2. As noted in Part 1

of this report, aptitude and interest measureshaving a substantial relationship with the factorsare shown as anchors on the horizontal and verticalaxes. In Figure 1, the masculine-mechanical areas(auto body, welding, carpentry, auto mechanics,etc.) are all relatively dose together and toward themechanical-outdoor interest end of Factor 1. Voca-tional horticulture differs from these groupschiefly in its position toward the artistic interestend of Factor 2. The data processing and accountclerk groups tend to be characterized by highclerical and computational interest, PRE-GPA, andclerical perception aptitude. The radio, TV, andelectronics groups are not exceptional in theserespects, but they do have relatively high mechan-

114 5

reasoning aptitude, scientific interests, and;ibly low artistic interests. The positions ofal- areas can be analyzer+ .1 a simiiar fashion..s.orrelation analyses u...,ng the aptitude measurespredictors and vocational area GPA as theerion of success were run for each of the 12is. In answer to Research Question 3, therelations showed that PRE-GPA was, withouteption, the best predictor across the areas.)ending on vocational area, a variety of other]sures ranked second in order of effectiveness.! correlations obtained with PRE-GPA ranged infrom .30 for carpentry to .70 for high skill

lography, with the median being about .45.-he most effective combination of two pre-.ors was determined for each vocational pro-n group by means of multiple correlationiyses. In answer to Research Question 4, practi-application of two-variable predictor combine-is was warranted in all but 2 (carpentry, and

auto body and we(ding) of the 12 vocatior areas.The multiple correlations ranged in size im .36for carpentry to .73 for hiigh skill stenogr, Themedian was about .49.. There was s,_4-9,antialvariation among the P--)tiitude measures impa-nying PRE-GPA in the two-variable combif ytions.For example, mechan:,..J1 reasoning w2:s the :ecomdvariable for the auto mechanics aref., an fingerdexterity for the cosmetology area. In gene, il, verylittle predictive ability was lost by Lsing thie bestcombination of two predictors rather than r. e bestthree.

Cross-validation analyses based or fall o 1 969entrants will be conducted when these studentshave had a chance to graduate. A follow-up studyhas been completed for 1966 and 1967 eta-Frits.This study, and the studies that are project-7A, willprovide post-high school criteria of voci tionaldevelopment, satisfaction, and success.

Data-Information Conversion

The results described above provide the poten-area vocational school student with little help

the exploration of vocational program choice.tical payoff from the statistical analyses doesoccur until the results are used to convert

lent data into counseling information.n the case of success estimates, data-informa-i conversion was readily accomplished viale-entry and double-entry experience tables.31e-entry experience tables showing the per-tage of students obtaining various vocationalgram GPAs at various levels of PRE-GPA wereeloped for each of the 12 vocational areas.able-entry tables based on the best combinationtwo predictors were developed for the 10 areasvhich this was warranted. Through use of theseles, counselors and students could see theure of the relationship between the mostctive aptitude measures and vocational programaess.

iimilarity scores, the second data-informationversion pi ocedure, were based on a combine-

of 10 of the most effective aptitude and

irterest measures. Separate sets of similarity scoreswere developed for boys and girls. Each setcontained scores for the eight vocational pTogramareas appropriate to the student's sex. Reportswere in the form of a computer-printed labelattached to a preprinted interpretation sheetsimilar to the one shown in Figure 2. Emphasis inthe use of these reports was in stimulating studentsto explore those programs for which they receivedthe highest scores. Counselors also received similar-ity score profiles for use in representing, visually, astudent's similarities and analyzing their basis. Thiswas accomplished by plotting the factor scoresreported for each student on the appropriatesimilarity score profile, as illustrated for FredCartesian in Figure 1. Both the amount anddirection of difference between a student'sposition and that of the various vocational areasprovide information r.slevant to vocational counsel-ing. A detailed description of this test interpre-tation procedure is provided in Part 1 of thisreport.

1516

Field Tests

Initial fie:d tests of the similarity score reportsnvolved 160 potential vocational school enrolleesmending four feeder high schools during the 1968-39 school year. Field-test counselors felt that the;imilarity scores were much more helpful thanocal norms or regular test score reports. However,:hey sometimes founc that the similarity scoreswere difficult to interpret, especially when artudent's scores were all low or when a studentluestioned why his scores came out as they did. As

result of these reactions, similarity score profiles:e.g., Figure 1) were developed to facilitate similar-ty score interpretation.

Field testing during the 1969-70 school yearInvolved approximately 900 potential vocationalschool enrollees attending 12 of the 14 feeder highschools. In addition to the commercially availabletest score reports, counselors received similarityscore reports, a set of experience tables, a set ofsimilarity score profiles, and an interpretivemanual. Suggestions for improvement resultingfrom the second field test chiefly involved modifi-:..ations of the similarity score profiles to facilitatetheir introduction to students. The profiles initiallyprovided to the counselors contained ellipses show-ing the distribution of students in each of the eightvocational areas appropriate to the student's sex.To reduce the visual confusion experienced bysome students and counselors, two series (one forgirls and one for boys) of three similarity scoreprofiles were developed for field testing during the1970-71 school year. Figure 1 represents theintermediate stage in the series for boys. The firstprofile in the series contained no ellipses while thethird profile contained ellipses for all eight areas.Depending on student need and understanding, anyor all of the three profiles can be used in testinterpretation.

An info7mal survey of student reaction toreporting procedures was obtained after the

1969-70 field tests. Nine of the 12 schoolsidentified a "reasonably representative" sample ofstudents who had received the reports. When asked"Did you find the Similarity Scores ... to behelpful as you thought about programs that youmight enter at Penta-County?", only 13 of the 166students in the survey chose the response option:"They really weren't of any help to me." Anotherkey item on the student survey was worded asfollows: "What was the main way in whichSimilarity Scores were helpful to you? (Pleasecheck only one response.)" Student response per-centages are shown in parentheses after each ofthe following response options:

1. They weren't of any help.(8%)2. They told me which program I should enter.

(6%)

3. They suggested programs that I hadn'tthought about before. As a result, I looked intosome of these programs. (38%)

4. They backed up the program choices I hadalready made. ,(34%)

5. They suggested that some programs I hadbeen thinking about might not be as "right" for meas some other programs. (13%)

6. They told me that I shouldn't go to Penta-County. (1%)

It is encouraging to note that few studentsviewed the similarity scores as telling them what todo and that most students saw the scores playingeither an exploratory or a confirmatory role.Student comments on the reporting procedureswere especially refreshing. Who could feel dis-appointed by a test interpretation that told alittle more about me that I didn't quite know"!

16 17

Discussion

ocational counseling has much to gain fromavailability of validated test information. The-information conversion procedures illustratediis paper have been available for a number ofs. Existing computer programs (Cooley &nes, 1962, 1971) make them feasible for anyiseling agency or guidance department having

access to a computer of moderate size. Regionalservices analogous to test scoring services couldalso be established.

Local norms are now in widespread use whenonce they existed mostly as illustrations in text-books. Perhaps locally validated test informationwill also become commonplace.

1718

References

ggaley, A. R., & Campbell, J. P. Multiple-discriminant analysis of academic curricula byinterest and aptitude variables. Journal of Edu-aational Measurement, 1967, 4, 143-150.

nnett, G. K., Seashore, H. G., & Wesman, A. G.Differential Aptitude Tests: Fourth editionmanual. New York: Psychological Corporation,1966.

arke, P, Gelatt, H. B., & Levine, L. A decision-making paradigm for local guidance research.Personnel and Guidance Journal, 1965, 44,40-51.

loley, W. W., & Lohnes, P. R. Multivariateprocedures for the behavioral sciences. NewYork: Wiley, 1962.

loley, W. W., & Lohnes, P. R. Predicting develop-ment of young adults. Pittsburgh, Pa.: AmericanInstitutes for Research and School of Education,University of Pittsburgh, 1968.

aoley, W. W., & Lohnes, P. R. Multivariate dataanalysis. New York: Wiley, 1971.

ann, F. E. Two methods for predicting theselection of a college major. Journal ofCounseling Psychology, 1959, 16, 15-26.

fer, H. S. The need for do-it-yourself predictionresearch in high school guidance. Personnel andGuidance Journal, 1957, 36, 162-167.

anagan, J. C. The implications of ProjectTALENT and related research for guidance.Measurement and Evaluation in Guidance, 1969,2, 116-123.

'ench, J. W. The logic of and assumptionsunderlying differential testing. Proceedings,1955 Invitational Conference on TestingProblems. Princeton, N.J.: Educational TestingService, 1956, 40-48.

Gelatt, H. B., & Clarke, R. B. Role of subjectiveprobabilities in the decision process. Journal ofCounseling Psychology, 1967, 14, 332-341.

Ghiselli, E. E. Validity of occupational aptitudetests. New York: Wiley, 1966.

Goldman, L. Using tests in counseling. New York:Appleton-Century-Crofts, 1961.

Hoyt, K. B. Vocational guidance for all: New kindsof personnel needed. American VocationalJournal, 1970, 45(5), 62-65.

Katz, M. R. Decisions and values: A rationale forsecondary school guidance. Princeton, N.J.:College Entrance Examination Board, 1963.

Katz, M. R. A model of guidance for careerdecision-making. Vocational Guidance Quar-terly, 1966, 15, 2-10.

Katz, M. R. Can computers make guidance deci-sions for students? College Board Review, 1969,72(Summer), 13-17.

Passmore, J. L. Validation of a discriminant analy-sis of eight vocational-technical curriculargroups. (Doctoral disssertation, University ofMissouri) Ann Arbor, Mich.: University Micro-films, 1968. No. 69-3271.

Prediger, D. J. Validation of counseling-selectiondata for vocational school students. Toledo:University of Toledo, 1970. (Grant No.0EG-3-6-551169-0379, Bureau of Research,USOE.)

Prediger, D. J., Waple, C. C., & Nusbaum, G. R.Predictors of success in high school level voca-tional education programs: A review, 1954-67.Personnel and Guidance Journal, 1968, 47,137-145.

18 19

lulon. P. J. Distinctions between discriminant andregression analysis and a geometric interpreta-tion of the discriminant function. HarvardEducational Review, 1951, 21, 80-90.

iulon, P. J., Tiedeman, D. V., Tatsuoka, M. M., &Langmuir, C. R. Multivariate statistics for per-sonnel classification. New York: Wiley, 1967.

"atsuoka, M. M. Joint probability of membershipin a group and success therein: An index whichcombines the information from discriminant andregression analysis. Unpublished doctoral disser-tation, Harvard University, 1956.

"horesen, C. E., & Mehrens, W. A. Decision theoryand vocational counseling: Important conceptsand questions. Personnel and Guicance Journal,1967, 46, 165-172.

`horndike, R. L. Prediction of vocational success.Vocational Guidance Quarterly, 1963, 11,179-187.

Thorndike, R. L., & Hagen, E. Measurement andevaluation in psychology and education. (3rded.) New York: Wiley, 1969.

Tiedem2n, D. V. A model for the profile problem.Proceedings, 1953 Invitational Conference onTesting Problems. Princeton, N.J.: EducationalTesting Service, 1954, 54-75.

Tiedeman, D. V., Bryan, J. G., & Rulon, P. J. Theutility of the Airman Classification Battery forassignment of airmen to eight Air Force special-ties. CambrAge, Mass.: Educational ResearchCorporation, 1951.

Whitla, D. K. An evaluation of differential predic-tion for counseling and guidance. (Doctoraldissertation, University of Nebraska) Ann Arbor,Mich.: University Microfilms, 1957. No. 20-991.

ACT Research Reports

This report is Number 44 in a series published by the Research and Development Division of The American College TestingProgram. Tn.! first 26 research reports have been deposited with the American Documentation Institute, AD I AuxiliaryPublications Project, Photoduplicabon Service, Library of Congress, Washington, D.C. 20540. Photocopies and 35 mm.microfilms are available at cost from ADI; order by ADI Document number. Advance payment is required. Make checks ormoney orders payable to: Chief, Photoduplication Service, Library of Congress. Beginning with Research Report No. 27, thereports have been deposited with the N3tional Auxiliary Publications Service of the American Society for Information Science(NAPS), c/c CCM Information Sciences, Inc., 22 West 34th Street, New York, New York 10001. Photocopies and 35 mm.microfilms re available at cost from NAPS. Order by NAPS Document number. Advance payment is required. Printed copiesmay be obtained, if available, from the Research and Development Division, The American College Testing Program.

The reports since January 1969 in this series are listed below. A complete list of the reports can be obtained by writing to theResearch and Development Division, The American Col leye Testing Program, P. 0. Box 168, Iowa City, Iowa 52240.

No. 28 A Description of Graduates of Two-Year Colleges, by L. L. Baird, J. M. Richards, Jr., & L. R. Shevel (NAPS No. 00306;photo, $3.00; microfilm, $1.00)

No. 29 An Empirical Occupational Classification Derived from a Theory of Personality and Intended for Practice and Research,by J. L. Holland, D. R. Whitney, N. S. Cole, & J. M. Richards, Jr. (NAPS No. 00505; photo, $3.00; microfilm, $1.00)

No. 30 Differential Validity in the ACT Tests, by N. S. Cole (NAPS No. 00722; photo, $3.00; microfilm, $1.00)

No. 31 Who Is Talented? An Analysis of Achievement, by C. F. Elton, & L. R. Shevel (NAPS No. 00723; photo, $3.00;microfilm, $1.00)

No. 32 Patterns of Educational Aspiration, by L. L. Baird (NAPS No. 00920; photo, $3.00; microfilm, $1.00)

No. 33 Can Financial Need Analysis Be Simplified? by M. D. Onnrig, & P. K. Jones (NAPS No. 01210; phoco, $5.00; microfilm,$3.00)

No. 34 Research Strategies in Studying College Impact, by K. A. Feldman (NAPS No. 01211; photo, $5.00; microfilm, $2.00)

No. 35 An Analysis of Spatial Configurntion and Its Application to Research in I-11;qher Education, by N. S. Cole, & J. W. L.Cole (NAPS No. 01212; photo, $5.00; microfilm, $2.00)

No. 36 Influence of Financial Need on the Vocational Development of College Students, by A. R. Vander Well (NAPS No.01440; photo, $5.20; microfilm, $2.00)

No. 37 Practices and Outcomes of.Vocational-Technical Education in Technical and Community Colleges, by T. G. Gartland, &J. F. Carmody'(N APS No. 01441; photo, $6.80; microfilm, $2.00)

No. 38 Bayesian Considerations in Educational Information Systems, by M. R. Novick (NAPS No. 01442; photo, $5.00;microfilm, $2.00)

No. 39 Interactive Effects of Achievement Orientation and Teaching Style on Academic Achievement, by G. Domino (NAPSNo. 01443; photo, $5.00; microfilm, $2.00)

No. 40 An Analysis of the Structure of Vocational Interests, by N. S. Cole, & G. R. Hanson (NAPS No. 01444; photo, $5.00;microfilm, $2.00)

No. 41 How Do Community College Transfer and Occupational Students Differ? by E. J. Brue, H. B. Engen, & E. J. Maxey(NAPS No. 01445; photo, $5.50; microfilm, $2.00)

No. 42 Applications of Bayesian Methods to the Prediction ot Educational Performance.. by M. R. Novick, P. H. Jackson, D. T.Thayer, & N. S. Cole (NAPS No. not available at this time.)

No. 43 Toward More Equitable Distribution of College Student Aid Funds: Problems in Assessing Student Financial Neeo, byM. D. Orwig (NAPS No. not available at this time.)

2121