Sakai in Language Courses: Present Uses and Future Possibilities

Ken Romeo, Ph.D.Academic Technology Specialist

http://kenro.web.stanford.edu :: kenro@stanford.edu

Outline• Background• The Present: Sakai in the Stanford Language Center• Objectives and Results• Summative Assessment• Formative Assessment

• The future: What this could be• How students study• How teachers (would like to) teach• A framework for curriculum

• Heads up! This is not just about language teaching.

July 9, 2009 Stanford Language Center / Academic Computing 2

Background• Me

• ESL instructor (20 years)• Academic Technology Specialist (2006)• CourseWork (Sakai) team meeting observer / participant (2008)

• Stanford Language Center• Language requirement• 1995 – new director: Prof. Elizabeth Bernhardt• Emphasis on assessment and professional development

• This presentation• American Council on the Teaching of Foreign Languages (ACTFL

2007) Bernhardt, Molitoris, Miano, Gelmetti, Tsethlikai, Romeo• Sum of experience

3July 9, 2009 Stanford Language Center / Academic Computing

SAKAI AND THE STANFORD LANGUAGE CENTER

The Present:

July 9, 2009 Stanford Language Center / Academic Computing 4

Assessment Program

• Objectives• Improve student performance• Enhance credibility (students and the public)• Programmatic consistency

• Methods• ACTFL Oral Proficiency Standards (ACTFL, 1999)• Oral Placement and Exit exams• More face-time for instructors put diagnostic

assessment online (CourseWork)

5July 9, 2009 Stanford Language Center / Academic Computing

Overview: Results

• More highly trained staff• 95% go through ACTFL interview training• Over 1/3 certified

• Professional conversation• 20% first year and 24% second year increase• Highly positive student reaction• Use Sakai to:• Deliver formative assessments (SAMigo)• Connect to exit assessment (enrollment & archiving)

6July 9, 2009 Stanford Language Center / Academic Computing

Foreign Language Placement

• 800 unregistered students: not in Sakai

• Online tests during the summer

• Oral assessment on campus: 1 hour, cassette tapes

7July 9, 2009 Stanford Language Center / Academic Computing

Exit Assessment: SOPI Definition

• Oral Proficiency Interview (OPI): Live• Nationally standard format – used everywhere

• Simulated Oral Proficiency Interview (SOPI): Recorded• 1st year: ~10 items, 2nd year: ~20 items: • English instructions + line drawing• Thinking time• Native prompt• Response recording• Go to next item (NO USER CONTROL)

• From 2008: Add a writing assessment

8July 9, 2009 Stanford Language Center / Academic Computing

SOPI Delivery

• Scheduling, content creation – human hours• Software requirements• High stakes: content security and reliability• Enrollment from existing courses (Sakai)• Non-standard roles: Coordinator, instructor• No student control = no web delivery• Playback or recording• Test progression

• Securely archive audio recordings (Sakai)

9July 9, 2009 Stanford Language Center / Academic Computing

SOPI Software (Project Manager: Makoto Tsuchitani)• Application (Developer: Casey Dunn)• Desktop Java application• Quicktime for Java – play and record• WebServices to communicate with Sakai

• Sakai Monitor Tool (Developer: Zach Thomas)• Realtime progress of each student• UI depends on role• Packaging for further dissemination

10July 9, 2009 Stanford Language Center / Academic Computing

SOPI Application

July 9, 2009 11Stanford Language Center / Academic Computing

SOPI Monitor Tool

July 9, 2009 12Stanford Language Center / Academic Computing

Formative Assessments in Sakai• 1st year: Oral diagnostic

assessments• Collaboratively developed

content (audio / video / text)• SAMigo: Audio Recording applet• Resource (course) sites

• 2nd year: Writing Diagnostic assessments• Collaboratively developed

content (images / text)• SAMigo: Timed, short answer• Resource (course) sites

13July 9, 2009 Stanford Language Center / Academic Computing

Technology Integration: Key Points

• Each part has a pedagogically valid purpose and is not focused on technology.

• Uses a standalone application with connection to Sakai to do what the web can’t.

• All instructors create material based on the same standards-based framework.

• Implementation takes a huge number of human hours and coordination across many different groups.

14July 9, 2009 Stanford Language Center / Academic Computing

WHAT THIS COULD BEThe Future:

July 9, 2009 Stanford Language Center / Academic Computing 15

Prologue: Why use an LMS at all?

• Privacy and authentication• Scalability: • class department university (?)

• Modularity: • Centrally stored, clone-able units

• One portal to existing technology (connection, organization, computation, audio, video, telephony, etc.)

16July 9, 2009 Stanford Language Center / Academic Computing

HOW STUDENTS STUDY

July 9, 2009 Stanford Language Center / Academic Computing 17

Increased levels of connection

• Cell phones: voice and text • Sharing – small groups, whole class, future

classes (Submission? Grading? Feedback?)• Online office hours• Social networking – managing multiple

identities, authentication to protect privacy

18July 9, 2009 Stanford Language Center / Academic Computing

Familiarity with digital environment

• New formats for work (video, etc.)• Don’t always carry laptops – often use clusters,

do carry cell phones• Use multiple resources – search / self-study /

scaffolding

• Note: Find the least common denominator

19July 9, 2009 Stanford Language Center / Academic Computing

HOW TEACHERS (WOULD LIKE TO) TEACH

My Wishlist

July 9, 2009 Stanford Language Center / Academic Computing 20

Different tools for different tasks

• In class teaching• Homework• Self-study

21July 9, 2009 Stanford Language Center / Academic Computing

No-brainer: Video, everywhere

• Upload ingest to streaming• Clip creation, indexing, delivery, annotation,

collaboration• Control playback – just once, twice• Why streaming? Too large to download.• Why not YouTube? No privacy / authentication.

(which is what Sakai does very, very well!)

22July 9, 2009 Stanford Language Center / Academic Computing

Organization (schedule, groups)

• Course structure: Linear by definition• Tracking students• Integration with University registrars• Arbitrary groups of students• In/across courses/programs• Requirements, milestones, electives

23July 9, 2009 Stanford Language Center / Academic Computing

Identities (roles)

• Need a departmental or other arbitrary level• Social networking (or not) …• Multiple identities• Retaining pedagogical control: Assessment• In/out groups: Fairness? Motivation?

24July 9, 2009 Stanford Language Center / Academic Computing

Assessment (SAMigo)

• Control response format: • Limit and time chances to view/listen• Limit and time chances to respond

• Enable large scale assessment• Get rid of all those blue books• Work with infrastructure groups: machines, space

• Telephone delivery of Sakai is a killer app: • Accessibility• Very controlled linear assessment • Anyone want to be partners in a grant?

25July 9, 2009 Stanford Language Center / Academic Computing

Content creation (SAMigo)

• Authoring: basic desktop tools, conversion, definitions, spell checking

• Break up into modules: Re-usable, sharable, organize-able

• Changing order of delivery• Changing details of content: Randomization of

items / variables• Downstream control of shared content

26July 9, 2009 Stanford Language Center / Academic Computing

Reports (gradebook, etc.)

• Grading on an arbitrary curve• Item analysis: Stats, test theory• Assessments / items across departments / years

(aggregate-able)• Log files: Reportable numbers to stakeholders

27July 9, 2009 Stanford Language Center / Academic Computing

A MODULAR CURRICULUMA Framework for the Future

July 9, 2009 Stanford Language Center / Academic Computing 28

How?

• Homework: collaboration, or not – pedagogical control

• Self-study:• Students who need it• Students who want to do self-study • Publicly available “open-source” set of online

activities

• Classroom: more in a minute …

29July 9, 2009 Stanford Language Center / Academic Computing

Why?• Textbooks = information + activities + order• Textbooks unbound • Teacher must decide order• Information • Multiple sources• Multiple formats• Students can independently supplement

• Basic unit = activity• Requires / allows creativity

30July 9, 2009 Stanford Language Center / Academic Computing

Basic curricular unit: the Activity

• Re-conceptualize interactive lecture / seminar• Control information in the classroom• Elicit search / curiosity outside of the classroom• Facilitate all departments’ products • Linguists’ tree diagrams• Engineering simulations• Video• Etc.

31July 9, 2009 Stanford Language Center / Academic Computing

In the Classroom

• Physical: personal interactions, hardware (or not)• Virtual: distance learning• Small group activities:• Quick, arbitrary, but airtight membership• Posting materials for each group separately• Posting product of each group separately• Quick, but controlled access to multimedia• Easy creation of multimedia product• An example: DIY fill in the blanks

32July 9, 2009 Stanford Language Center / Academic Computing

Classroom Example: DIY Fill in the Blanks

• 2 groups• 2 short (30 sec) audio clips• 2 paper transcripts• 2 pairs of scissors• Each group cuts out 15 words to make a fill in

the blanks problem• Exchange transcripts• Play clips (x2-3)• Group with most correct answers wins

July 9, 2009 Stanford Language Center / Academic Computing 33

Digitize DIY Fill in the Blanks

• Arbitrary groups• Secure content: audio and text• Modify text• Exchange text – without revealing original• Play audio• Check answers• Group collaboration

July 9, 2009 Stanford Language Center / Academic Computing 34

Epilogue: Take home messages• Focus on the users: how can we change

education? (ref: introduction of textbooks)• Understand what the important characteristics of

pedagogy are (and are not).• Don’t just facilitate pedagogy as it is, find out

where pedagogy is going by talking to expert teachers.

• Focus on pedagogically valid activities – not on the tools.

• Motivated students / creative teachers are NOT the problem: We need to reach everyone.

35July 9, 2009 Stanford Language Center / Academic Computing

THANK YOU VERY MUCH!Ken Romeo :: http://kenro.web.stanford.edu :: kenro@stanford.edu

July 9, 2009 Stanford Language Center / Academic Computing 36