the glastonbury foreign language assessment initiative: flap 2008-2013
DESCRIPTION
Presenters:Rita Oleksak, Director of Foreign Languages, GPSBarbara Lindsey, University of Connecticut Dan Conrad, ACTFLTRANSCRIPT
THE GLASTONBURY FOREIGN LANGUAGE ASSESSMENT
INITIATIVE: FLAP 2008-2013
Presenters:
Rita Oleksak, Director of Foreign Languages, GPSBarbara Lindsey, University of Connecticut
Dan Conrad, ACTFLNECTFL CONFERENCE
BALTIMORE, MARYLANDApril 2, 2011
Receipt of Foreign Language Assessment Grant 2008-2013
Ambitious Objective 1:
Development of national assessment tools to improve the articulation of curriculum and instruction over time and from schools to colleges and universities
Receipt of Foreign Language Assessment Grant 2008-2013
Ambitious Objective 2:
Develop a process that is replicable across the nation and across levels for the collection of student work samples in an electronic portfolio system
Historical Significance of the Glastonbury Foreign Language
Program Visionary Leadership - 1956
Expert Administrators - 1957
Collaboration of Higher Education -1957
Funding from the National Defense Education Act - 1957
Historical Significance of the Glastonbury Foreign Language
Program Creation of Elementary Foreign
Language Program - 1957
Creation of Glastonbury Foreign Language Materials - 1958
Implementation of Russian Language Program - 1958
Construction of 1st Foreign Language Laboratory - 1959
Historical Significance of the Glastonbury Foreign Language
Program Continued Funding from the Sale of
Audio-Lingual Materials (Harcourt Brace/Ivanovich) 1959-1968
Training of All Staff in ACTFL Oral Proficiency Interviews - 1983
100% Local Funding for the Continuation of Foreign Language Program - 1983-2000
Grants Received for Standards-Based Articulation of Curriculum, Assessments, and Teacher Training - 2000-2009
OUR RUSSIAN STUDENTS ARE…
successful in their first(other) foreign language(s)
love learning about another culture enthusiastic learners aware that they have been afforded a
unique opportunity sometimes native speakers with or
without formal language training standouts in the college application
process
THE COLLEGE EXPERIENCE: 6 years of Russian…now what??
Colleges ponder where to place a student who has already had an extended sequence of Russian
Dilemma solved: start over!!
American Council of Teachers of Russian (ACTR) to the rescue
Pilot AP Russian Program initiated in 2004
OUR STUDENTS SPEAK QUIA survey to upper level students and
Glastonbury graduates to probe:Metacognitive skills/recognitionSuccessful study habits/skillsEfficacy of foreign language assessments Influence on their lives and advanced studySuccess in other courses due to knowledge
of another language/cultureEffect on cultural awareness and
acceptance of cultural diversity
MLA High School to CollegeArticulation Project
Glastonbury Public SchoolsUniversity of Connecticut 1998-1999
Measuring Student Preparedness
• Student attitudinal surveyPurpose: To determine if current articulation
effortsbetween Glastonbury and the University of
Connecticut are effectively meeting the needs of these
students• Traditional placement exam
Purpose: To establish a baseline cutoff score for incoming
students wishing to place in the first two years of language study at the university and to compare performance levels of Glastonbury students (high
school levels 3-6 in predominately high achievement classes) with University of Connecticut students
Two Vehicles Used
Findings from survey
• In both groups, only 19 percent continued to study a foreign language at UCONN
• Nearly two-thirds (60 percent) in each group were studying a foreign language to fulfill a requirement
• Lack of awareness (almost 75 percent) of facilities which support language learning such as the multimedia language center, foreign language residence floor, Linkage Through Language Program
Similarities
Findings from survey
• Students who began learning a foreign language prior to sixth grade were more likely to continue studying a language in college (Numbers in the survey not large enough to draw a firm conclusion)
• Number of students who began the study of a foreign language because of a requirement but continued due to personal interest was nearly three times higher among Glastonbury graduates (eighteen versus seven percent)
• None of the general population surveyed started a new language at college, two percent of Glastonbury students did
• A slightly higher percentage of Glastonbury graduates are majoring in languages (five percent) than those from the general population (two percent)
Differences
Placement exam findings• Less than twenty percent of the variance among
test scores for UCONN students could be explained by current enrollment, total number of years studied and outside language experience.
• Additional testing and continued collection of background information should explain more of the variance between test scores
• Lack of inclusion of background information during scoring for Glastonbury students precludes a comparison between the two groups of students
DOE National FL Survey Grant• 4 years: 2000-2004• Students• Teachers• Administrators• Self-reported perceptions
on various aspects of language study in the U.S.
College Administrators: Percentage of student population enrolled in foreign language courses(2003: n = 309)
0%
20%
40%
60%
80%
100%
0% - 25% 26% - 50% 51% - 75% 76% - 100% Don't know
College Administrators: Course offerings(2003: n = 309)
0%
20%
40%
60%
80%
100%
Standard Literature Native speakers Specific purposes Methods
Literature in translation
Conversation/Composition
Content-based Linguistics Other
College FL Students: Type of foreign language course in fall of 2002 (n = 309)
0%
20%
40%
60%
80%
100%
Beginner Intermediate Advanced Don't know
College FL Students: Type of foreign language course in spring of 2003 (n = 309)
0%
20%
40%
60%
80%
100%
Beginner Intermediate Advanced Don't know
College FL Students: Took placement exam to determine which foreign language class to take in college (2003: n = 895)
College Administrators: Placement of students who do not use foreign language placement exams(2003: n = 94)
0%
20%
40%
60%
80%
100%
HS grades # of years studied Student meets withFL department
Student meets withfreshman advisor
All No procedure Other
College Instructors: Perceptions of which aspects should be measured by a college foreign language placement exam* (2003: n = 304)
0%
20%
40%
60%
80%
100%
Speaking Reading Writing Listeningcomprehension
Cultural knowledge Grammar Vocabulary
* multiple responses accepted
2003: College Administrators: Foreign language skills measured on placement test (n = 215)
0%
20%
40%
60%
80%
100%
Speaking Reading Writing Listening comprehension Cultural knowledge
College Administrators: Satisfaction with placement procedure (2003: n = 309)
0%
20%
40%
60%
80%
100%
Very satisfied Somewhat satisfied Not too satisfied Not at all satisfied Don't know
College Instructors: High school foreign language classes have adequately prepared students for college-level classes (2003: n = 304)
0%
20%
40%
60%
80%
100%
Strongly agree Somewhat agree Somewhat disagree Strongly disagree Don't know
College FL Students: College foreign language class repeated material already mastered in high school(2003: n = 309)
0%
20%
40%
60%
80%
100%
Yes No
College FL Students: High school foreign language classes prepared them well for college level classes (2003: n = 309)
0%
20%
40%
60%
80%
100%
Strongly agree Somewhat agree Somewhat disagree Strongly disagree
College FL Students: College foreign language class required material not learned in high school to be prepared for class (2003: n = 309)
0%
20%
40%
60%
80%
100%
Yes No
High School Teachers: Main cause for some students to perform less well in college foreign language classes than in high school (2002: n = 189)
0%
20%
40%
60%
80%
100%
Teachingmethods of HS
Teachingmethods of
college
Placementmethod in
college
Assessmentsused in college
Assessmentsused in HS
IndividualStudent
Other Faster pace incollege (vol.)
College Instructors: Main cause for some students to perform less well in college foreign language classes than in high school (2003: n = 92)
0%
20%
40%
60%
80%
100%
Teachingmethods of HS
Teachingmethods of
college
Placementmethod in
college
Assessmentsused in college
Assessmentsused in HS
IndividualStudent
Other Faster pace incollege (vol.)
High School Teachers: Importance of high school to college foreign language articulation initiatives(2002: n = 401)
0%
20%
40%
60%
80%
100%
Very important Somewhat important Not too important Not important at all Don't know
College Instructors: Importance of high school to college foreign language articulation initiatives(2003: n = 304)
0%
20%
40%
60%
80%
100%
Very important Somewhat important Not too important Not important at all Don't know
High School Teachers: Articulation efforts could support a student’s transition from high school to college(2002: n = 401)
0%
20%
40%
60%
80%
100%
Strongly agree Somewhat agree Neither agree/disagree Somewhat disagree Strongly disagree Don't know
College Instructors: Articulation efforts could support a student’s transition from high school to college(2003: n = 304)
0%
20%
40%
60%
80%
100%
Strongly agree Somewhat agree Neither agree/disagree Somewhat disagree Strongly disagree Don't know
High School Teachers: Familiarity with any high school to college foreign language articulation efforts(2002: n = 401)
0%
20%
40%
60%
80%
100%
Very familiar Somewhat familiar Not too familiar Not familiar at all Don't know
College Instructors: Familiarity with any high school to college foreign language articulation efforts(2003: n = 304)
0%
20%
40%
60%
80%
100%
Very familiar Somewhat familiar Not too familiar Not familiar at all Don't know
High School Teachers: Involvement in any program to address high school to college foreign language articulation (2002: n = 401)
0%
20%
40%
60%
80%
100%
Yes No
College Instructors: Involvement in any program to address high school to college foreign language articulation(2003: n = 304)
0%
20%
40%
60%
80%
100%
Yes No
High School Teachers: If not already involved, interest in participation in high school to college foreign language articulation program(2002: n = 374)
0%
20%
40%
60%
80%
100%
Very intersted Somewhat interested Not too interested Not at all interested Don't know
College Instructors: If not already involved, interest in participation in high school to college foreign language articulation program (2003: n = 221)
0%
20%
40%
60%
80%
100%
Very intersted Somewhat interested Not too interested Not at all interested Don't know
High School Teachers: Frequency of sharing information about foreign language programs between high school teachers and college instructors (2002: n = 399)
0%
20%
40%
60%
80%
100%
Never Once Twice More than twice Don't know
HS teacher shared
College instructorshared
College Instructors: Frequency of sharing information about foreign language programs between high school teachers and college instructors (2003: n = 304)
0%
20%
40%
60%
80%
100%
Never Once Twice More than twice Don't know
HS teacher shared
College instructorshared
High School Teachers: Important components of high school to college foreign language articulation initiatives* (2002: n = 401)
0%
20%
40%
60%
80%
100%
Reciprocal visits Dialogue b/w HS &college
Collaboration on learningsequences
Collaboration onassessment & placement
Other
* multiple responses accepted
College Instructors: Important components of high school to college foreign language articulation initiatives* (2003: n = 304)
0%
20%
40%
60%
80%
100%
Reciprocal visits Dialogue b/w HS &college
Collaboration on learningsequences
Collaboration onassessment & placement
Other
* multiple responses accepted
College Administrators: Areas of collaboration with school/department of education for pre-service teachers (2003: n = 309)
0%
20%
40%
60%
80%
100%
Curriculum Proficiency Pre-serviceteaching
Study abroad Methodology All areas Other Nocollaboration
Don't know
Task Force MembersMarty Abbott, Director of Education, American Council on
the Teaching of Foreign LanguagesKelly Aceto, Associate Director of the Center for International
Business Education and Research (CIBER), University of Connecticut
Gilbert Andrada, Psychometrician, Connecticut State Department of Education
Carol Any, Ph.D., Associate Professor of Language and Culture Studies (Russian) Trinity College
Dana Bourgerie, Ph.D., Associate Professor of Chinese, Brigham Young University
Christine Brown, Assistant Superintendent for Curriculum and Instruction, Glastonbury Public Schools
Lynne Campbell, Teacher of Russian, Glastonbury Public Schools
Carol Chen-Lin, Ph.D., Teacher of Chinese, Choate Rosemary Hall, Wallingford, CT
Dorie Conlon, Teacher of Spanish, Glastonbury Public Schools
Task Force MembersDaniel Conrad, Principal Assessment Specialist, American
Council on the Teaching of Foreign Languages Hanadi Dayyeh, Arabic Language and Curriculum Specialist,
American School of BeirutMuhammad Eissa, Ph.D., Consulting in Arabic/Islamic
Studies, Lecturer of Arabic, University of ChicagoCarine Feyten, Ph.D., Professor, Foreign Language
Education, Miami University OhioWafa Hassan, Ph.D., Outreach Coordinator for the Arabic
Language Instruction Program, Michigan State UniversityBonnie Hoskins, Teacher of Spanish, Glastonbury Public
SchoolsMei Ju Hwang, Lead Chinese Teacher, Springfield Public
SchoolsHong Gang Jin, Ph.D., Director of Chinese, Hamilton
CollegeCatharine Keatley, Ph.D., National Capital Language
Resource Center, George Washington UniversityBarbara Lindsey, Director, Multimedia Language Center,
University of Connecticut
Task Force Members
Kevin McKenna, Ph.D., Professor of Russian Language, Literature and Culture in the German and Russian Department University of Vermont
Priscilla Meyer, Ph.D., Professor of Russian Language & Literature Russian, Wesleyan University
Rita Oleksak, Director of Foreign Languages, ELL, Glastonbury Public Schools
Mark Pearsall, Teacher of Latin and Ancient Greek, Glastonbury Public Schools
Ken Peterson, Internet Administrator, American Council of Teachers of Russian
Tony Smith, Technology Specialist Consultant, American Council on the Teaching of Foreign Languages
Roger Travis, Ph.D., Associate Professor of Classics, Department of Modern and Classical Languages, University of Connecticut
Manuela Wagner, Ph.D., Assistant Professor of Foreign Language Education, Department of Modern and Classical Languages, University of Connecticut
Kathleen Wang, Principal, Chinese Charter School, Springfield Public Schools
Tina Wu, Chinese Professor, Central Connecticut State University
Activities Accomplished to Date
Collaboration with ACTFL on the development of the AAPPL Test item
Collaboration with the University of Connecticut
Meetings of the Portfolio Task Force, November 2009, May 2010 and October 2010
Evidence efforts in place in Glastonbury to date:
• Common monthly performance checks, midterm and final exams
• Digital speaking samples• Video and audio blogs• Digital evidence from study
abroad• Evidence from distance learning• Cross-grade level student service
learning projects• Capstone projects
How the AAPPL became part of this project:• Worked from the AAPPL
Framework• Met to discuss curricula
within GPS• Created prototype• Created proof of concept• Finalized content and met
with GPS• Conducted a very small
scale tryout in early November
• Preparing to conduct small pilot in GPS and elsewhere in December
AAPPL: A history
• Indefinite postponement of NAEP FL led to formation of task force, grant
• Grant focused on development of a Framework describing a new, standards-based national assessment
• Result was AAPPL and an AAPPL prototype
• On-line, on-demand, standards-based, curriculum linked test of four skills across three modes
AAPPL: A history
• AAPPL became a component of the Glastonbury FLAP grant
• Started with Chinese• Piloted Interpersonal
Listening/Speaking 2008• Piloted all skills/modes in 2009• Began building the delivery
infrastructure in 2009/2010• Greatly refined the concept in
2010• Continued to pilot in
Glastonbury throughout 2010• Began development on Russian
and Arabic in 2010• Expect to be operational in
2011
AAPPL: The experience• Task-based role
play in which student is him- or herself (a student of a FL in best-practice classroom)
• Sits at the computer with headphone and microphone
• Is greeted by the FL teacher who provides (in English) an overview of tasks.
AAPPL: Response modes
• Create • Compose• Converse
AAPPL: Examples
• Russian• Arabic• Chinese