Takahide Ishii s1170047 Supervised by Prof. Debopriyo Roy

1. Introduction

2. The Background this research

3. The Objective this research

4. Research Questions

5. Sample and context

6. Research Design ・ In-class Website Analysis Assignment

・ Actual Experiment

・ Data Analysis – Use of Coders

7. Findings

8. Conclusions

Literature on computer assisted language learning is mostly silent on how web-based design analysis could be effectively used as a tool and framework for developing critical thinking skills and language proficiency in an EFL classroom.

This article reported on how EFL learners perform with English website analysis

tasks in a language reception and product context.

There is substantial research in language studies and cognition that establishes cognition and language development to be closely related

Educators have identified multiple features and elements of reading and writing to have always influenced thinking skills to a large extent.

There has always been a strong appeal to promote

higher order thinking in ESL and EFL classrooms, and research has clearly focused on the need to foster critical thinking in a foreign language classroom

However, unfortunately language learning and thinking skills were almost always treated as independent processes

This study focused on extensive and sustained content analysis using information technology resources.

This study establishes that such attempt helps with both linguistic and cognitive information processing ability.

This study is also influenced by the fact that we have shifted from Web 1.0 towards Web 2.0 where there is an increased emergence of computer-mediated communication, social networking and active interaction between the user and the web environment.

How d id the EFL readers per form wi th var ious des ign quest ions as asked dur ing webs i te ana lys i s ?

I s there any s ign i f i cant d i f fe rence be tween coders who graded responses to the des ign quest ions suggest ing s ign i f i cant d i f fe rence between responses to a des ign quest ion and/or suggest ing that

one or more coder ( s ) have not unders tood the quest ions and responses cor rec t l y and resu l tant l y cou ld not use the assessment rubr i c cor rec t l y for grad ing the des ign responses?

Participants (N=17) are junior level students (age group: 18-20 years) in their third year undergraduate program specializing in computer science in a Japanese technical university

With this specific elective course named Writing and Design for World Wide Web, students mostly focused on the process of online writing, designing and analyzing

websites based on design principles, besides designing concept maps on websites they analyzed.

As part of the website analysis assignment, students were asked to study a specific website in a chosen domain (e.g., education, entertainment, government, tourism, sports etc)

Students provide open-ended responses to 8 standard questions asked of them.

1. Explain whether the organization of information in the site is user-friendly or not?

2. Explain whether the presentation of content is appealing or not?

3. Explain whether the effective use of technology is demonstrated?

4. Who is the target audience? Is the website appropriate for the projected audience?

5. Explain the quality of the text content.

6. Is the information accessible?

7. Explain whether the resources use real-world situations.

8. Here are some common reasons for building this website. Rank them in order of importance to you. Do you have a reason that is not listed?

Open-ended Design Questions asked of the Participants

The experiment was conducted in a controlled environment as an in-class activity, and over two weeks.

students analyzed the Belize tourism website based on the 8 open-ended questions asked during the same assignment that happened over the previous weeks

Students had one-week to complete the analysis, besides the 90 minutes of class time where they could consult their friends.

To encourage writing and proper explanation, the minimum word limit for the assignment was set at 500 words.

Three undergraduate students (not part of the class with the sample) who took the same class at an earlier semester were appointed as coders with the task of grading the first week assignment where readers participated in an open-ended evaluation of the Belize tourism website.

The coders were given a set of criteria on the basis of which they graded each open-ended response, for all the 8 questions assigned.

Each of the 8 open-ended responses for each of the 17 participants was rated thrice, once by each of the three coders.

The group (including the project supervisor and the three coders) then discussed each grade for each question and criterion.

The Coding Schema

N Minimum Maximum Sum of Mean Scores Std. Deviation

Question1 18 9 16 14.56 2.479

Question2 18 3 16 12.56 3.110

Question3 18 0 14 11.56 3.585

Question4 18 3 15 11.50 4.148

Question5 18 3 15 11.33 3.804

Question6 18 2 15 11.28 4.254

Question7 18 0 15 10.33 4.366

Question8 18 6 15 11.67 2.990

Valid N (list

wise) 18

Mean and SD Values for Each Question on 6 Criteria and Each Criteria Scored Thrice by Three Different Coders

Descriptive Statistics for Student Score Calculated over Six Criteria by Coder A

N

Minimum Mean

Score

Maximum

Mean Score

Mean of Total

Mean Score Std. Deviation

S1 6 3 7 5.33 1.366

S2 6 3 7 5.67 1.506

S3 6 2 7 4.00 2.098

S4 6 2 8 4.17 2.639

S5 6 1 6 3.50 2.258

S6 6 5 8 7.00 1.549

S7 6 1 6 3.33 2.582

S8 6 1 1 1.00 .000

S9 6 3 8 5.00 2.098

S10 6 1 8 4.00 3.098

S11 6 3 8 5.83 2.317

S12 6 3 7 4.17 1.602

S13 6 3 8 7.00 2.000

S14 6 2 8 6.33 2.422

S15 6 1 8 3.17 2.639

S16 6 3 8 4.83 1.941

Valid N

(list wise) 6

Descriptive Statistics for Student Score Calculated over Six Criteria by Coder B

N

Minimum Mean

score

Maximum Mean

Score

Mean of Total Mean

Score Std. Deviation

S1 6 3 8 6.00 2.098

S2 6 4 8 6.50 1.643

S3 6 4 7 5.67 1.033

S4 6 3 8 5.83 2.137

S5 6 5 8 6.83 1.472

S6 6 6 8 7.50 .837

S7 6 3 8 5.67 2.338

S8 6 1 2 1.50 .548

S9 6 3 8 6.17 2.483

S10 6 2 8 6.00 2.449

S11 6 3 8 6.50 1.871

S12 6 4 8 6.50 1.761

S13 6 6 8 7.67 .816

S14 6 3 8 6.17 1.941

S15 6 4 8 6.33 1.633

S16 6 5 8 6.83 1.169

Valid N (list

wise) 6

Descriptive Statistics for Student Score Calculated over Six Criteria by Coder C

N

Minimum Mean

Score

Maximum Mean

Score

Mean of Total Mean

Score Std. Deviation

S1 6 6 8 6.83 .753

S2 6 7 8 7.67 .516

S3 6 5 7 6.67 .816

S4 6 6 8 7.50 .837

S5 6 6 8 7.50 .837

S6 6 5 8 7.50 1.225

S7 6 6 8 7.50 .837

S8 6 1 1 1.00 .000

S9 6 5 8 6.33 1.033

S10 6 7 8 7.83 .408

S11 6 8 8 8.00 .000

S12 6 6 8 7.67 .816

S13 6 6 8 7.50 .837

S14 6 6 8 7.50 .837

S15 6 5 8 7.00 1.549

S16 6 6 8 7.67 .816

Valid N (list

wise) 6

This study, Coders were a test of not only understanding what constitutes valid information, and good organization of response, but it also required ability to read through the criteria rubric used in the study

demonstrate at least moderate levels of language proficiency.