2 BROWN, HUDSON, CLARK

rather that for the handbook to be useful, it must address con-cerns of real programs while recognizing their constraints.Discovering the placement concerns of a number of real pro-grams would obviously be very helpful in planning the man-ual.

Methods

Participants

Sampling. To distribute the paper-based survey, a list ofnames of language department chairs was rented from theModern Language Association. The entire list consisted of2589 names. From the outset, we were not planning to havea truly representative survey. If we were, we would ideallyhave to sample from the entire universe of colleges in theUS. Instead, we were mainly interested in getting an idea ofthe types of placement procedures used in various schoolsaround the country. For this more limited purpose, we de-cided to shoot for approximately 200 total responses (eitherpaper or electronic) as a goal. Initially assuming a 20% re-sponse rate, this meant that we would need to send question-naires to approximately 1,000 programs in total. To conserveresources should the response rate prove higher than antici-pated, we decided perform a smaller mailing first with theoption of a second mailing should we not reach our goal of200 responses. This meant that two lists of 500 names eachwould be needed.

Choosing the 500 names for each mailing was donethrough a semi-random process in which every 5th namefrom the MLA list was chosen starting from a randomly gen-erated starting point. This procedure was repeated to gener-ate the second list as well. It should be noted that the listitself is not a random sample in that non-MLA members areexcluded and because MLA membership is not necessarilystrictly proportional to various geographical populations orprogram types. We intentionally did not keep track of towhich programs questionnaires would be sent because thatwould create a great administrative burden with no real ben-efit. The reader is referred to Table 2 for information onwhich geographical locations are represented by survey re-spondents.

Characteristics of the responding institutions. Becauseour survey deals with language programs, it is a little un-clear how much of the information is respondent specific(i.e., the opinions of the individual actually filling out thesurvey) and how much is institution specific. Of the 169 re-sponses, 109 were from public schools while 58 were fromprivate schools. Not surprisingly, Spanish, French, and Ger-man were the three most common languages. Table 1 showsall of the languages reported and Table 2 shows the responsedistribution by state.

Although the surveys were initially mailed to departmentchairs, a cover letter instructed the chairs to pass on the ques-tionnaire to the person in the department most involved inplacement testing. The self-reported job description of therespondents can be seen in Table 3, with teacher being themost common answer.

Table 1Languages taught in various programsLanguage ProgramsSpanish 120French 90German 70Italian 35Japanese 32Russian 32Chinese 31Spanish for Spanish speakers 18Portuguese 13Hebrew 9Arabic 8Korean 6Vietnamese 5Swahili 3Thai 3Dutch 2ESL 2Filipino 2Irish 2ASL 1Burmese 1Farsi 1Modern Greek 1Hindi 1Indonesian 1Norwegian 1Ojibwe 1Polish 1Swedish 1Tagalog 1Urdu 1

Materials

The questionnaire used in this study was developed over aperiod of several months through discussions among the au-thors. Several earlier surveys of language programs in gen-eral (Rhodes & Branaman, 1999; Walker & Li, 2003) andplacement testing in particular (Wherritt & Cleary, 1990)provided initial models for the development of the currentquestionnaire. During the first few discussions, it becameapparent that the survey would only be able to provide a gen-eral window into the placement process. To truly understandand evaluate the placement process, knowledge of the con-text in which placement takes place is important. This meansthat ideally one would have information about the program’scurriculum, goals, students, etc. Unfortunately, there was nofeasible way to collect this type of information on any kind ofnational level. Because so much of the placement process isprogram specific (i.e., geared towards placing a certain popu-lation of students into a certain sequence of courses), a majorchallenge was to develop the questionnaire in such a waythat it was general enough to cover many different types of

PLACEMENT 3

Table 2Geographical location of programsState ResponsesCalifornia 22New York 14Illinois 12Minnesota 8Pennsylvania 8Iowa 7Massachusetts 7Michigan 7Ohio 6Wisconsin 6Tennessee 5Hawaii 4Texas 4Virginia 4Connecticut 3Florida 3Georgia 3Indiana 3Kentucky 3Louisiana 3Missouri 3New Jersey 3Oregon 3Washington, D.C. 2Kansas 2Nevada 2Oklahoma 2Rhode Island 2Vermont 2Arizona 1Arkansas 1Colorado 1Delaware 1Mississippi 1New Hampshire 1New Mexico 1North Carolina 1Puerto Rico 1South Carolina 1Washington 1

Table 3Self-reported job descriptionPosition NLanguage teacher 76Foreign language department chair 57School foreign language coordinator 16Language resource center director 6Testing coordinator 4State foreign language coordinator 1District foreign language coordinator 1

language programs while at the same time providing detailedinformation about the placement testing process.

Basic considerations. After several rounds of discussion,it was decided that the questionnaire would focus on six gen-eral areas with varying degrees of detail:

1. Basic program information. This is information aboutthe location of the program, the program type, languagestaught, number of students, and the types of courses offered.This information was collected to help with the categorizingand generalizing of results.

2. Language ability assessment. This section is designedto get information about the types of language ability assess-ment that occur in the program other than placement test-ing. This information was collected to give an idea of thecontinuity between placement procedures and other types ofassessment in the program.

3. Placement test administration. This section asked ques-tions about the details of placement test administration in-cluding how often the test is administered, who proctors thetest, who scores the test, and how scores are transmitted tostudents. It was hoped that one use of this section would beto get an idea of how ”hands-on” teachers are in the place-ment testing process.

4. Placement test content. This section dealt with the ac-tual content of the placement test in terms of skills tested andthe origin of the test materials.

5. Course placement procedures. Because placement testscores are not always the only part of the placement process,this section was designed to elicit information on how thefinal course placements were actually made.

6. Comments and contact information. Comments onwhat respondents would like to see in a placement manualwas solicited. In order to facilitate potential follow-up ses-sions, contact information was collected from those partici-pants willing to provide it.

For each of the basic sections, several questions were de-veloped that would elicit the type of information we needed.

Determining the response format. While developing thespecific questions for each section of the questionnaire, it be-came clear that a decision had to be made about how muchleeway to allow the respondents. Survey questions can rangefrom very tightly controlled to very open-ended, and thereare advantages and disadvantages to each (Brown, 2001).There were essentially two competing factors, in our view.We wanted respondents to be able to give an information thatthey deemed necessary while at the same time constrainingthe types of answers that they gave to facilitate future analy-sis. We were also aware of the fact that as our target audiencewere people who tend to have many administrative things tojuggle, we would need to cover a lot of ground in a veryefficient fashion.

In the end, we opted for a very controlled set of questions(yes/no, multiple-choice) with an option of ”other” for virtu-ally every question. We also provided a space for commentsat the end of each of the major sections of the survey to giverespondents a chance to amplify their answers. This proved

4 BROWN, HUDSON, CLARK

Table 4Survey responses by monthType March April May June JulyOnline 16 7 5 52 -Maila 21 56 8 2 3Total 37 63 13 54 3a1st mailing March 17; 2nd mailing April 6

to be a decision not without some regret, however.

Developing the electronic version. Early on in the devel-opment process, we decided that the survey should also beavailable in electronic format on the World Wide Web. Oncethe paper-based version was ready, creating the web-basedversion was mainly a matter of translating the survey formatinto machine-readable format. Though we had investigatedthe use of commercially-available form generation software,we decided that a simple HTML page would be sufficient.By combining a simple HTML form hosted on a computerwith a FileMaker Pro database, we were able to have thesurvey results be submitted directly into the database, thussimplifying the data collection process.

Procedures

The mailing. The mailing was conducted by one of theauthors during the period of March to April 2004 throughthe University of Hawai‘i post office. Each mailing piececonsisted of a cover letter, a consent form, the questionnaireitself, and a business reply envelope. Assuming (perhaps op-timistically) a 20% response rate, we decided to send 1,000pieces split into two 500 piece mailings approximately twoweeks apart. This way, if the response rate were much higherthan anticipated, we could postpone or eliminate the secondmailing. In the end, both 500 piece mailings were sent. Theonly difference between the two mailings was that the duedate on the cover later was changed to reflect a slightly laterdate for the second mailing. Of the 1,000 pieces sent, approx-imately 10 surveys were returned as undeliverable becausethe addressee was no longer at that address or was deceased.Five surveys were returned as undeliverable as the addresslabel had either fallen off or not been affixed initially.

At approximately the same time as the surveys were beingmailed, we also placed announcements on several electronicLISTSERVs announcing the availability of an electronic ver-sion. The URL for the electronic version was also includedin the cover letter of the mailing. The electronic version wasaccessible for approximately four months after which it wastaken off line.

Return rate. Because the survey was available in both pa-per and electronic format and because even those people re-ceiving a paper questionnaire had the option of completing iton-line, it is very difficult to judge the overall return rate. Inall, 90 people returned the survey in the business reply enve-lope while 80 people completed the survey on-line. Table 4shows the responses by month.

Table 5Language ability assessment activities used in programsAssessment Activity Number of programsConstructed-response tests 145Oral proficiency interviews 131Student presentations 131Selected-response tests 120Authentic activities 112Translation exercises 80Student portfolios 65Student self-assessment 44Extended writing or report* 5Poems* 1Debate* 1Dictation* 1*Write-in response

Results

Language Ability Assessment

Table 5 shows the breakdown of programs that reportedusing various types of assessment procedures to assess lan-guage ability in contexts other than placement situations. Ascan be seen from the table, programs reported using a ratherwide range of assessment tools. As with all of the tablesin this section, participants often marked more than one re-sponse while others left responses blank, thus the totals maybe considerably more or less than the total number of pro-grams responding.

Almost three-quarters of the respondents indicated thatthe teachers in their program were aware of the Standardsfor Foreign Language Learning when asked (Yes = 121, No= 42), but considerable fewer indicated that assessment prac-tices in their program had changed because of this awareness(Yes = 77, No = 79). Use of proficiency scales was reportedby a majority of programs (Yes = 105, No = 58) with themajority of scales being developed within the program ei-ther originally through discussions among program faculty(n = 34) or with reference to scales used in similar programs(n = 53). Other programs reported developing proficiencyscales jointly with outside organizations (n = 15) or usingcompletely externally developed scales (n = 13).

Placement test administration. A majority of programsreported using language placement tests when asked (Yes =122, No = 42); of those programs, 87 reported using an lo-cally developed test while 54 reported using a commercial orother externally developed test. (Note that some programsreported using both kinds). Details about test administrationare shown in Table 6 through Table 11. Again note that manyprograms selected multiple answers on the questionnaire sothe totals may exceed the number of programs reporting testuse.

PLACEMENT 5

Table 6Test frequencyFrequency ResponsesOnly when needed 36Several times a semester 30Once a semester 20Once a year to incoming students* 16On demand/available on-line* 14As needed* 5Weekly* 1*Write-in response

Table 7Test registration processProcess ResponsesMandatory participation 32Department office 22First-come, first-served 19Admissions office 19Testing office 13Multimedia lab staff 9Through appointment with advisor* 9On-line* 7Taken at orientation* 6Test given during class time 5Regionally delivered* 2*Write-in response

Table 8Test proctorProctor ResponsesTeachers 64Testing office staff 21Multimedia lab staff 16Test delivered automatically/on-line* 13Admissions office staff 11Student help 11Dept. Chair or staff* 5Language coordinator* 1LRC Director* 1Placement director* 1*Write-in response

Table 9Test scoringScoring method ResponsesTeachers 66Machine-scored 42Scored automatically by computer* 14Admissions office staff 9Testing office staff 8Multimedia lab staff 3Student help 2Dept. Chair or staff* 2*Write-in response

Table 10Communicating scores to studentsCommunication method ResponsesTeachers 52Academic advisor 20Scores are posted 20Automatically generated/on-line* 14Testing office staff 13Admissions office staff 11Dept. office staff* 81st year studies office* 4Language coordinator* 2*Write-in response

Table 11Availability of retestsTimes students can take test ResponsesOnly once 69Unlimited 27Restest with special permission only 12Not more than twice 9Once per year* 2Once per semester* 1Once every three years* 1Retests for suspicious scores* 1*Write-in response

Placement test content

Tables 12 through 21 give some details about the contentsand validation procedures used for both internal and externaltests. Perhaps unsurprisingly, the oral skills and real-worldskills were less often assessed than the traditional skills ofreading, writing, and grammatical knowledge (see Table 12).One program also reported assessing cultural knowledge inaddition to more linguistic skills.

Internally developed tests. Tables 13 through 18 refer tothose tests developed locally by individual programs. Better

Table 12Skills assessed for placement purposesSkill ResponsesReading 115Grammar 109Vocabulary 102Writing 80Listening 56Speaking 42Ability to complete a real world task 15Cultural knowledge* 1Personal interview* 1*Write-in response

6 BROWN, HUDSON, CLARK

Table 13Reasons for locally developing placement testsReason ResponsesTo better control test contents 38To better reflect ability rangeof students in program 31Commercial tests unsuitable 31No commercial test exists for language 20Availability of funding/resourcesfor development 19As temporary measure only 8To supplement external test 3No cost/less expensive* 3External assessments not well-known* 1Create state-wide instrument for articulation* 1Complement oral interview* 1*Write-in response

Table 14Responsibility for test creationTest developer ResponsesCurrent/past teachers 77Current/past administrator 5Special committee 5Language resource center 5Individual test developer 4Foreign language office* 1Testing division* 1*Write-in response

control over test content (including maximizing the matchbetween student ability and test difficulty) and the unsuit-ability of commercially produced tests were the most oftencited reasons for developing a test locally (Table 13). Threeprograms noted that internally developed tests incurred fewercosts than commercial tests.

As can be seen in Table 14, teachers bear most of the bur-den in developing tests at the program level, with test con-tent coming from textbooks, course objectives, authentic andoriginal material (Table 15).Two programs reported havingthe tests developed through the testing division or foreignlanguage office. Perhaps because of their role in the test cre-ation process, teachers’ involvement in locally produced testsextends through all phases of the placement process, as seenin Tables 8 through 10.

Because teachers have other duties in addition to test cre-ation, test revision seems to be a function of course contentrather than natural test development per se (see Table 16 and17). One program reported revising the cut scores each year,but not necessarily the test contents.

The validation process for internally developed tests (Ta-ble 18) included having current students take the test, match-ing the contents to course objectives, or review by languagespecialists. Some programs reported performing an itemanalysis, but those were in the minority. Some programs

Table 15Primary source of test contentSource ResponsesCourse objectives 40Original materials 31Course textbooks 28Authentic materials 25State/local content standards 11MLA* 1Proficiency guidelines* 1Combination including national exams* 1Past exams* 1*Write-in response

Table 16Test revision frequencyFrequency of test revision ResponsesWhen necessary 37Once every several years 16When conditions (personnel, financial) permit 11Once or twice year 10Never revised 10With each new intake* 1Only cut scores are revised* 1*Write-in response

also reported that the review by language specialists was per-formed only when the test was first developed. One pro-gram reported that the test was not validated in the traditionalsense, but that it seemed to be working fine.

Externally developed tests. Table 19 shows the numberof respondents using various commercially produced tests.The Brigham Young University CAPE (Computer AdaptivePlacement Exam) and the Advanced Placement (AP) subjecttests were the most common commercially produced testsused by the respondents. Part of this is no doubt due to the

Table 17Reason for revising testReason ResponsesTo better reflect course contents 42To address a deficiency in current test 28To assess a skill not previously assessed 15To make the test contents more timely 13To lengthen or shorten the test 8To prevent cheating 7Demography of students* 1More contextualization* 1Improve validity, delivery, student-friendliness* 1Upgrade/develop* 1Prepare test for web delivery* 1*Write-in response

PLACEMENT 7

Table 18Validation method for internally developed testsValidation method ResponsesPiloting on current students 57Matching contents to course objectives 43Review by language specialists 33Item analysis 18Not validated but seems to work* 1*Write-in response

Table 19Externally produced tests used by respondentsTest ProgramsAP subject test 38CAPE (BYU) 21Wisconsin Test 8ACTFL OPI 7CLEP 5SAT II subject test 2Michigan Test 1SOPI 1Japanese Proficiency Test 1MLPA 1AATG Test* 1University of Oregon NFLRC STAMP* 1College Board* 1sras* 1*Write-in response

availability of these tests for the most commonly taught lan-guages. Widespread use in other programs and the consis-tency of the results were the most cited reasons for choosingthe tests (Table 20), though some programs commented thatthe decision to use the test was out of their hands (i.e., a deci-sion by the Dean) or that practical considerations, such as thelack of available staff for testing duties during the summer,were also considerations.

As with the internally developed tests, content review and

Table 20Reason for choosing commercial testReason RespondentsWidely used in other programs 21Gives consistent results 18Have always used it 8Familiar to teachers 7No other option 4Ease of administration* 1Cost* 1Convenience* 1Faculty not available in summer* 1Dean’s decision* 1*Write-in response

Table 21Validation method for externally developed testValidation method ResponsesReview by instructors/department head 37Piloting on current students 31Matching contents to course objectives 7Item analysis 2Compared favorably to previous test* 1*Write-in response

Table 22Final course placement determinationPlacement decision ResponsesTeacher’s recommendation 64Previous semesters of study 44Advisor’s recommendation 44Student self-selection 40Placement test scores 13High school transcripts* 2Combination of factors* 1*Write-in response

piloting on current students were the most common ways ofvalidating externally produced tests (Table 21). One pro-gram cited a high correlation obtained between a previouslyused test. Perhaps because of the inherent mismatch betweena standardized test and an individual program’s curriculum,matching the test content to course objectives was not usedto the extent it was for locally produced tests.

Course placement

Regardless of whether or not a placement test is used, pro-grams still must place their students into classes. Tables 22through 24 highlight various parts of the placement process.Note that most respondents indicated a combination of pro-cedures for any given facet of the process. It is interesting tonote in Table 22 that very few programs placed students ex-clusively on the basis of test scores. Of course, the extent towhich advisor and teacher recommendations are influencedby test scores is not seen. It is also important to note, ascan be seen in Table 24, that only in a very small number ofprograms were course changes not allowed once instructionhas begun. Some programs commented that changes of levelwere not allowed, but students were free to switch languagesor drop language courses.

Comments and additional information

Of the 169 surveys received, 137 included contact infor-mation for the person who had filled out the survey while 32people chose to remain anonymous. Additionally, 68 respon-dents were kind enough to include their thoughts and com-ments about both the placement process in general as wellgive some thoughts on what the placement handbook should

8 BROWN, HUDSON, CLARK

Table 23Determination of cut scoresCut score setting method ResponsesPiloting on current students 59Based on proficiency level 48Test-maker’s recommendation 31No specific cut-scores* 5Past experience* 3Language faculty* 3Colleague’s recommendation* 2Combination* 2Students make final choicebased on recommendation* 1Tracking of placed students* 1*Write-in response

Table 24Class changes allowed after instruction commencesCourse change policy ResponsesChanges allowed during add/drop period 112Changes allowed only with special permission 37Course changes not allowed 2Change to other language only* 1Changes allowed up to week six* 1Class change only, no level change* 1Change allowed if prerequisites met* 1*Write-in response

cover. In general, comments could be classified into one ofseven categories (with illustrative comments):

1. The philosophy of placement testing or other overarch-ing concerns (”How passive knowledge or ability (reading,grammar rules, recognition of word meanings) sometimesgained over a period of years can be balanced with actualproduction in speaking and writing when determining place-ment level.”)

2. The testing of a particular language skill (”Testing writ-ing skills”)

3. Appropriately defining levels of proficiency (”How todistinguish between students at ’high beginner’ and interme-diate level”)

4. Administrative or practical issues (”How to find a reli-able placement test that assesses all four skills and can be ad-ministered efficiently to a fairly large number of students.”)

5. The placement of heritage students (”How to place her-itage speakers with limited reading and writing proficiency”)

6. On-line or computerized testing (”Specific informationon web-based and/or other on-line testing, especially for lev-els beyond beginners”)

7. General comments or encouragement about the survey(”I look forward to seeing that manual”)

Table 25 shows the number of comments for each of theseven categories. Although some answers had elements ofmore than one category, the perceived main concern was usedfor classification purposes.

Table 25Suggestions for the placement handbookCategory Number of commentsPlacement philosophy 15Testing specific skills 14Computerized placement 10Practical issues 9Heritage students 7General comments 7Defining levels 6

Discussion

Because individual departments are often charged withteaching multiple languages, the survey did not constrain re-spondents to describing the process for any individual lan-guage and, in fact, most respondents checked several boxesin response to the question of for which language the sur-vey is valid. This multiplicity of languages coupled with theformat of the survey which allowed for answers of ”Other”for almost every question lead to a situation in which it be-came difficult to determine what weight to give any singleanswer. In other words, if a person had indicated that thesurvey covered French, Italian, and German, and that read-ing, writing, grammar, and speaking were assessed for place-ment purposes, it could be the case German placement wasbased primarily on reading, while French and Italian werebased on the other skills. Though some respondents includedmarginal notes to the effect that a particular answer onlycovered a particular language, most of the surveys did notinclude such notation. Since there was no principled waydisambiguate the responses, it was impossible to get detailedlanguage specific information (e.g. Japanese placement mostoften includes reading whereas French placement most of-ten includes grammar). For this reason, no differentiationbetween placement procedures and languages can be made.

What skills are most often assessed for placementpurposes?

Given the focus on the communicative use of the languagein the pedagogical literature, one would expect programs tobe especially interested in assessing their students’ commu-nicative ability. Certainly, the wealth of assessment proce-dures used in the programs in general (cf. Table 5) wouldindicate that programs have, for the most part, moved beyonda single-minded focus on the raw materials of the language,grammar, vocabulary, and so forth, and have begun to explorethe students’ ability to use the language.

Despite this trend in the instructional component of theprogram, placement tests are still fairly restricted in terms ofthe types of skills assessed. Reading, grammar, and vocabu-lary were the three most often tested skills and although therewere a good number of programs that assessed speaking aswell, the number was only half that of the more traditionalskills. The lack of speaking assessment can be explained toa large extent by the difficulty of assessing speaking in any

PLACEMENT 9

type of large scale testing procedure. Some respondents in-dicated that although they would like to assess speaking, theconstraints of the placement testing process effectively ruledout anything other than selected-response testing. In fact,many of the comments about the promise of on-line testingwere specifically interested in the testing of oral proficiency.Though the lack of oral ability testing can be attributed toconstraints rather than avoidance, it is a little harder to justifythe widespread assessment of grammar for placement pur-poses in a communicative context.

Do programs tend to use commercially-producedor self-produced materials?

Despite the availability of commercial placement test, es-pecially in the more commonly taught languages, a largernumber of respondents reported using a locally produced testfor placement. This could be partially a type of responsebias in which programs using locally produced materials feltmore inclined to respond to a survey on placement practices.Despite the use of locally produced test and the predomi-nance of constructed-response test use in general, relativelyfew programs reported the use of item analysis in the de-velopment process. If teachers are mainly responsible forplacement test construction, some basic knowledge of com-mon test development practices should be beneficial.

How do language programs validate their place-ment instruments?

For the majority of the programs responding, test valida-tion was essentially a process of giving the test to currentstudents and looking at the results. Since piloting was themost common form of cut score setting, and since relativelyfew programs reported performing item analyses, it is likelythe case that most programs considered a test valid if currentstudents performed more or less as expected in terms of totalscores. Although this is certainly an important quality of aplacement test, a more thorough validation procedure wouldgive the potential for a better placement process.

How involved are teachers in the placement pro-cess?

At almost all stages of the placement process, teachers dothe bulk of the work. Except for registering students for thetest, which seems to be the province of administrative offices,teachers are most often called upon for test administration,test scoring, and score reporting.In addition, in the case ofinternally produced tests, teachers are also the major play-ers in test development. Of course, involvement does notnecessarily mean investment, and it could very well be thecase that many teachers see this aspect of their job as a chorerather than an academic endeavor on par with pursuing theirown research interests.

What issues would language programs like to seeaddressed in the handbook?

In response to solicitations for handbook suggestions,many of the comments dealt with the distinction betweentesting to assess language ability in general and testing toensure appropriate placement. Several comments addressedthe need for institution specific placement procedures:

Definition – placement does not refer to achieve-ment, rather to ”placement”. It is a matchinginstrument to match students’ current ability tocourses. Placement results are relevant to theparticular institution and course selection avail-able. (Private University)

–ending the practice of borrowing some place-ment instrument from other sites – stressingplacement into a curriculum that is local – build-ing a valid one based on changing curriculargoals using alternative ways to place studentsand monitor these (Public University)

Other respondents stressed the notion of fairness with re-spect to the types of courses offered and the placement pro-cedures used:

Fair assessment of students based on proficien-cies which a program espouses. (Private Univer-sity)

I would like to see learner perspectives on place-ment testing –both in terms of satisfaction, andin terms of how they perceive the testing? Dothey think the tests and their outcomes were agood predictor of the kind of language and lan-guage tasks that were included in the course thatthey move into.(Public University)

In addition to concerns with the overall philosophy of test-ing, several responsdents indicated that they would like to seethe question of oral ability testing addressed. The difficultiesof testing the spoken language and the potential for comput-erized tests to ease the burden seemed to be fairly commonconcerns:

If the placement exam includes speaking, howis the issue of compensation dealt with for thefaculty who must evaluate the speaking profi-ciency? (Public University)

Quick and easy to administer oral placement?(Private University)

Options for computerized placement testing.Are there options to test production skills? (Pri-vate University)

I would like to know about online tests that cando a relatively quick evaluation of speaking abil-ity. (Public University)

10 BROWN, HUDSON, CLARK

Other respondents raised concerns about the placement ofheritage speakers as well as difficulties placing students inprograms where there might be an incentive to do poorly onthe test.

Distinguishing heritage language placement andestablishing place-out procedures. (Public Uni-versity)

How do departments control some students’ de-liberate poor scoring on the placement test in or-der to place into a level lower than their abil-ity so that they may have a (perceived) easiercourse? (Private University)

Conclusions

The placement survey, though not perfect by any means,did seem to provide enough information about what is hap-pening in various language programs around the country togive some direction as to the types of issues that should becovered in the placement handbook. Because teachers seemto be the people mainly responsible for the creation, admin-istration, and evaluation of placement tests, it would makesense for the handbook to target that population rather thantesting professionals (though the two are by no means mu-tually exclusive). This means that the handbook should pro-vide basic information about general testing issues, such asreliability and validity, while also giving nuts and bolts in-struction in item writing and item analysis. It is also pos-sible to also have one or more sections of the manual de-voted to slightly more technical issues for those teachers oradministrators who have measurement experience. Becauseprograms tend to test multiple skills, issues specific to thetesting of those skills, such as the use of raters, need to beaddressed. Also, as many programs expressed interest incomputer based testing, issues pertaining to different test for-mats will need to be discussed, though the technical detailsof computerized assessment is probably beyond the scope ofthis manual.

Given those general parameters, a potential outline ofhandbook sections or chapters is presented below.

1. What is placement, articulation, curriculum?2. How do programs tend to approach placement?3. General testing issues: Validity, reliability, practicality4. Determining what to test5. Determining how to test6. Developing selected-response tests7. Interpreting test scores from selected-response tests8. Developing constructed-response tests9. Scoring and interpreting scores of selected-response

tests10. Setting cut-scores or guidelines11. Maintaining test quality12. Issues in testing specific skills13. Issues in testing specific populations (e.g. heritage stu-

dents)14. Issues in using commercially available tests15. Issues in test fairness

16. Introduction to computer-based testing17. Issues in computer-based testing18. Advanced statistical analysis

As the manual begins to take shape, it might be useful tofollow-up with some of the programs that responded to thesurvey to get detailed information on how actual programsare dealing with some of the issues raised in the handbook.

References

Brown, J. D. (2001). Using surveys in language programs. Cam-bridge University Press.

Rhodes, N. C., & Branaman, L. E. (1999). Foreign language in-struction in the United States: A national survey of elementaryand secondary schools. McHenry, IL: Center for Applied Lin-guistics and Delta Systems Co., Inc.

Walker, G., & Li, M. (2003). Report on the Survery of East AsianLanguage Programs (No. 11/30).

Wherritt, I., & Cleary, T. A. (1990). A national survey of Spanishlanguage testing for placement or outcome assessment at B.A.-granting institutions in the United States. Foreign Language An-nals, 23(2), 157-165.

AppendixIssues in placement survey

PLACEMENT 11

Figure A1. Survey Page 1

12 BROWN, HUDSON, CLARK

Figure A2. Survey Page 2

PLACEMENT 13

Figure A3. Survey Page 3

14 BROWN, HUDSON, CLARK

Figure A4. Survey Page 4