Teaching and Learning Development Grant (2009-2010) report

Project title: Self-guided workbook to facilitate an effective learning experience for international students

Names, schools and campus locations of the Project leaders and group members

Dr Sugumar Mariappanadar School of Business MelbourneDr Bulend Terzioglu School of Business MelbourneMs Nancy Reid ACU Melbourne-Academic Skills AdvisorMs Elizabeth Powell ACU Melbourne- Academic Skills AdvisorMs Kathryn Duncan ACU Melbourne- Liaison Librarian for Business School

Purpose of the project

The purpose of the project was to enhance the learning experience and improve the

academic success of the International MBA students enrolled at the Melbourne

Campus of the ACU using a self-guided workbook. To achieve the purpose:

It was proposed to examine if the self-guided workbook (2006)

complemented the subject lecturer’s learning expectations of the students

and improved their academic skills

To revise and include new activities in an updated version of the 2006

self-guided workbook.

Background

In 2006, as a pilot project aiming to enhance the learning experiences of

International students in the Master of Management and MBA(Acc) courses, Dr

Sugumar Mariappanadar and staff in the Academic Skills Unit developed a self-

guided workbook entitled 'Writing your way to Academic Success' (see Appendix 5).

The workbook contains guidelines in synopsis writing, note-taking, paraphrasing,

academic writing, referencing and essay writing. It also provides step-by-step

activities to learn how to complete business and management unit tasks (e.g., critical

synopsis, essay and case report writing) and help students become independent adult

1

learners. The purpose of the self-guided workbook is to help international students to

understand academic expectations, and to work on aspects of the assessment tasks that

are unfamiliar or complex. This work helps to strengthen their searching, researching,

reading, writing and referencing skills. The workbook was developed with the

additional aim of serving as a vehicle to facilitate International students’ transition to

the Australian postgraduate system.

Analysis of effectiveness of self-guided workbook as a learning tool

It is imperative to ascertain if the designed workbook is performing its initially

intended function and to determine how it can be improved to better serve the needs

of International students. Therefore, in this project an attempt is made to understand

the effectiveness of the self-guided work among MBA students (both domestic and

international) enrolled in the 2009 mid-year intake.

A questionnaire (see Appendix 1) was designed by the project team to explore if

the introduction of the self-guided workbook has helped MBA students understand

the course learning expectations as well as develop academic skills. The questionnaire

has two sections: learning expectations and self-perception of level of academic skills.

The majority of items in the learning expectations and academic skills sections of the

questionnaire were developed by the project team but a few were drawn from the

Samuelowicz (1987) study. The questionnaire was administered in two stages – pre

and post-introduction of the self-guided workbook to participants enrolled in the

MBA unit, Organisational Behaviour, in which the workbook is embedded. The

participants were seventeen MBA students enrolled in the Organisational Behaviour

unit (2009) at Melbourne Campus.

The pre-stage questionnaire was administered to MBA students in Week 1 of 2nd

semester (2009). Students were requested to enter an allocated code number on the

2

completed questionnaire. This code number was used to match the pre-stage with the

post-stage questionnaire. Subsequent to completing the pre-stage questionnaire, the

LIC provided an essay topic for assessment to the students, which was due for

submission in Week 3 of the semester. In Week 4, the LIC invited the Academic

Skills Adviser to his class to provide a copy of the workbook and to explain to the

students the rationale of the different activities included in the workbook.

In the post-stage, students are expected to submit a second essay in Week 11 of the

semester. To prepare a quality essay in this unit, students need to know the learning

expectations as well as having developed academic skills to complete the assessment.

Therefore, the activities in the workbook are used as a learning tool to understand the

learning expectations as well as to develop academic skills. Also in Week 4, the

Academic Skills Adviser introduced relevant activities such as targeted ‘research

database searches’ with assistance from ACU library staff, regarding APA

referencing, paraphrasing, and critical questioning skills to enable the students to

complete the ‘critical synopsis writing’ assessment, which is one of the unit

assessments. Critical synopsis writing on a chosen research article is used as the

foundation for essay writing. The students were asked to complete the above

mentioned activities in their own time. Subsequent to completing relevant activities in

the workbook, the students were advised to start preparing the critical synopsis

writing assignment on a provided journal article relevant to the Organisational

Behaviour unit.

The students were requested to submit the critical synopsis writing assignment in

Week 5 of the semester. The LIC marked this assignment and returned it to the

students in Week 6 with feedback. Students who did not do well in this assignment

were requested to contact the Academic Skills Adviser with the feedback provided by

3

the LIC to find ways to improve their writing skills. A research article for the second

critical synopsis writing assignment was given to the students in Week 8 and

submitted in Week 9. The purpose of the second critical synopsis writing assignment

is to reinforce the academic writing skills developed using the activities in the

workbook. The marked second critical synopsis writing assignment was returned to

students with feedback in Week 10. By embedding the workbook activities in the

Organisational Behaviour unit, students were able to develop academic skills to

complete the second essay, which was due in Week 11. The Academic Skills Advisor

was invited to the class in Week 6 to explain the rationale of other activities in the

workbook to develop students’ academic skills with respect to case study report and

essay writing. The purpose of this second visit by the Academic Skills Adviser to the

class was to help the students integrate the learning they have acquired by completing

different activities in the workbook and use it to prepare a quality case report and

essay.

An independent academic with more than twenty years of experience in teaching

management subjects was used to assess both the first and second essays. Both the

essays were marked out of 100 using a set of assessment criteria developed by the

LIC, which was communicated to the students at the beginning of the semester

through the unit outline. In Week 12 of the semester the questionnaire was again used

to collect post-stage data.

4

Findings

The data collected on learning expectations and level of ability students possess

were analysed using a T-test. The results in Table 1 indicate that except for one

learning expectation, which was to write clearly, the awareness level in all other five

learning expectations improved positively after the introduction of the workbook to

the students. This implies that the self-guided workbook is an effective learning tool

for students in raising their awareness level of learning expectations of the course.

Table 1. T test for pre- and post-stage learning expectations

Paired Samples Test

Paired Differences

t dfSig. (2-tailed)M SD

SE Mean

95% Confidence Interval of the

Difference

Learning expectations Lower Upper

Pair 1 Learn independently -.412 .712 .173 -.778 -.046 -2.384 16 .030*

Pair 2 Critical & analytical approach to learning

-.412 .870 .211 -.859 .036 -1.951 16 .051*

Pair 3 Research skills -.706 .920 .223 -1.179 -.233 -3.165 16 .006**

Pair 4 Formulate complex ideas

-.412 .939 .228 -.895 .071 -1.807 16 .052*

Pair 5 Targeted search of articles

-.824 .951 .231 -1.312 -.335 -3.570 16 .003**

Pair 6 Write clearly .176 1.380 .335 -.533 .886 .527 16 .605* p < .05; ** p < .01

The results in Table 2 show that there was a significant increase in the perceived

level of ability to complete the six academic tasks after introducing the workbook

activities to the students. The six academic tasks that improved significantly after the

introduction of workbook were: APA referencing, critical synopsis writing, critical

questioning, targeted research database search, targeted search of articles and

reflective journal writing. In summary, the results reveal that the workbook is a

5

valuable learning tool to communicate to students the learning expectations of a

course and also to develop the required academic skills to achieve those learning

expectations to become independent learners.

Table 2. T-test for pre- and post-stage level of perceived abilities to complete academic tasks

Paired Samples Test

Paired Differences

t dfSig. (2-tailed)

95% CI of the Difference

Academic tasksMean SD

SE Mean

Lower Upper

Pair 1 Paraphrase -.235 1.300 .315 -.90 .433 -.746 16 .466

Pair 2 APA referencing -.765 1.678 .407 -1.62 .098 -1.879 16 .053*

Pair 3 plagiarism -.353 1.693 .411 -1.22 .518 -.859 16 .403

Pair 4 Critical synopsis -.588 1.004 .243 -1.10 -.072 -2.416 16 .028*

Pair 5 Critical essay -.412 1.278 .310 -1.06 .245 -1.329 16 .203

Pair 6 Analytical essay -.412 1.372 .333 -1.11 .294 -1.237 16 .234

Pair 7 Critical questioning -.529 1.179 .286 -1.13 .077 -1.852 16 .043*

Pair 8 Critical thinking skills

-.412 1.228 .298 -1.04 .219 -1.383 16 .186

Pair 9 Search research database

-1.000 .935 .227 -1.48 -.519 -4.408 16 .000***

Pair 10 Targeted search of articles

-.824 .951 .231 -1.31 -.335 -3.570 16 .003**

Pair 11 Reflective journal writing

-.941 1.345 .326 -1.63 -.250 -2.885 16 .011**

Pair 12 Engage in debating -.412 1.121 .272 -.98 .165 -1.514 16 .150

Pair 13 prepare case report -.353 1.169 .284 -.954 .248 -1.24 16 .231

* p < .05; ** p < .01; *** p < .001

The results in Table 3 indicate that after the introduction of workbook activities to

the students in the Organisational Behaviour unit, assignment quality (i.e. essay

writing) improved. This is evident from analysing the group of students with a ‘fail’

grade in the pre-stage assessment of the unit. That is, 35% of students among the

6

group of ‘fail’ grade students received a ‘pass’ grade after the introduction of the

activities in the workbook. This suggests that the self-guided workbook is an effective

learning tool to improve the quality of assignments of the students by helping them to

understand the assignment requirements ahead of time and to prepare their

assignments in line with the course learning expectations.

Table 3. Students’ change in grade from pre-stage to post-stage assignment

Variations in grades

Frequency Percent Valid PercentCumulative

Percent

Valid Fail to fail 5 29.4 29.4 29.4

Pass to fail 1 5.9 5.9 35.3

Fail to pass 6 35.3 35.3 70.6

Pass to pass 5 29.4 29.4 100.0

Total 17 100.0 100.0

Contrary to our expectations that embedding the workbook activities in an

academic unit would reduce the one-to-one consultation time of students with the

Academic Skills Adviser, an analysis of the data from the ‘Student appointments’

system used by the Academic Study Skills unit suggest that more students from the

targeted MBA group requested one-to-one or small group consultations with the

Academic Skills Adviser. There is need for further understanding of this outcome;

however, some explanation can be posited. When students became aware that the

workbook activities can help them to develop academic skills, they were keen to

develop these skills by requesting more consultation time with the Academic Skills

Adviser. It also made them more aware of the services the Unit provides.

Furthermore, it was observed that the request for consultation time increased from

students who received a ‘fail’ grade in their first essay, or first and second critical

7

synopsis. Anecdotal evidence from the Academic Skills Adviser suggests that the

request for individual consultations dramatically declines once the student has

developed the academic skills required to complete quality assessments. That is, over

the duration of the course student requests for individual consultations decline.

Workbook revision

It is evident from the findings that the self-guided workbook is an effective

learning tool to help students develop an early understanding of the learning

expectations of the course and also to develop the required academic skills to

complete quality assessments. Therefore, this project has revised the activities in the

earlier version of the workbook (2006). Initially, focus groups were organised for

MBA students enrolled on the Melbourne and Sydney Campuses. The purpose of

these focus groups was to understand the extent to which the self-guided workbook

helped first and second semester International students with their learning experiences

in the MBA course and changed their perceptions about the importance of specific

workbook activities.

Focus groups with first and second year MBA students in Melbourne were

conducted. In Sydney, only one MBA student volunteered to participate in the focus

group and hence a telephone interview with the student was organised. A list of

questions (see Appendix 2) was prepared and used. The transcribed responses of the

focus group discussion with MBA students from Melbourne and telephone interview

with one Sydney student are provided in Appendix 3 and 4 respectively.

8

Elizabeth Powell, Nancy Reid and Kathryn Duncan in consultation with Sugumar

Mariappanadar and Bulend Terzioglu revised the following elements in the workbook

based on the feedback received from the focus group discussions:

All activities in the workbook were revised.

The learning expectations and provision of a rationale including graduate

attributes were listed.

The library skills section was rewritten including the addition of library

tutorials and exercises associated with database searches.

Sample models were added.

More up to date articles were included for synopsis writing exercise.

The paraphrasing section was rewritten.

The whole book was reformatted to make it more attractive, user friendly and

in line with the ACU Study Guide (for example colour scheme).

APA referencing system was updated to bring it in line with the latest APA 6th

edition.

A section on strategies for effective group work including problem solving

tasks was added.

Conclusion

The purpose of the project is to enhance MBA International students’ learning and

also improve academic success among students enrolled at the Melbourne Campus of

ACU using a self-guided workbook. Initially, 17 MBA students enrolled in the MBA

unit, Organisational Behaviour, were requested to participate and complete the

questionnaire to examine whether the self-guided workbook (2006) has

complemented the LIC's learning expectations of the students and improved academic

skills. The analysis of data collected with the questionnaire clearly indicates that the

9

workbook is an effective learning tool to help students understand the learning

expectations of the course. The workbook activities have also facilitated an

improvement in students’ academic skills and enhanced the quality of academic

assignments. Further, it was found that there was an increase in ‘pass’ grade among

students who received a ‘fail’ grade in their first assignment, once they had completed

the workbook activities.

A copy of the revised self-guided workbook (2010) entitled “Writing your way to

academic success” is attached to this report (Appendix 5). It is evident from the

project that the self-guided workbook improves teaching and learning outcomes for

the targeted students. The academic staff involved in this project found that the

quality of the assignments submitted by the students improved considerably and the

submitted assignments aligned well with the learning expectations for the unit.

Furthermore, the Academic Skills Advisers found in the post-stage of the project,

students approached them requesting help to improve ‘targeted or specific’ academic

skills to prepare a quality assignment compared to the ‘non-specific or vague’

questions, which MBA students had previously asked. It is important to note that this

project did not attempt to collate data separately for the domestic and international

students for analysis, and hence it was a limitation of this study. However, the

Academic Skills Advisers provided anecdotal evidence that the domestic students

enrolled in the unit informed the Advisers that by actively completing the activities in

the workbook they benefited immensely in developing academic skills.

Recommendations

The following recommendations are made to ACU based on the teaching and

learning outcomes achieved by the project:

10

The self-guided workbook can be now expanded to MBA students at the

Sydney and Brisbane Campuses.

To enhance the effectiveness of the workbook, it should be embedded in one

of the core units in the 1st semester of study (i.e. regular and mid-year intake).

Explore the opportunity to embed the workbook into a first year undergraduate

core unit to support students in their transition into university education.

Members of the initial Project team to be used in the training of staff for

project delivery on other campuses to ensure appropriate quality knowledge

transfer of the self-guided workbook.

Members of the Project team to assist lecturers to customise the workbook into

modules which support their learning expectations and assessment tasks.

Future Project Development

Considering the initial effectiveness in enhancing student learning, it would be

beneficial to add additional modules on self-reflective writing, debating and

presentations to the workbook.

Anticipating a future need for customising the workbook, cluster activities into

core and applied tasks to develop modules to support the specific learning

expectations and assessment tasks needs of individual units.

Developing this workbook as an online learning tool with more interactivity

for students including a self grading system to improve their academic skills

for preparing quality assignments.

11

References

Samuelowicz, K. (1987). Learning problems of overseas students: Two sides of a

story, Higher Education Research and Development, 6(2), 121-133.

12

Appendix 1

QUESTIONNAIRE

This questionnaire has been designed to determine to what extent you have understood the learning expectations of the MBA course after attending the introduction session conducted by the lecturer-in-charge and by using respective unit outlines for the first semester units.

Section A

Please indicate extent to which you are aware of the following learning expectations using the five-point rating scales. Choose only one of the five given rating scales.

Extent of awarenessLearning

expectationsNo

awareness0

Someawareness

1

Moderateawareness

2

Good awareness

3

Excellentawareness

4

1 Demonstrate a critical and analytical approach to learning

2 Learn independently

3 Formulate complex ideas in English

4 Apply research skills to construct keyword searches and evaluate results

5 Understand the building blocks of assignment writing

6 Write clearly

13

Section B

Indicate your belief in the level of ability you possess in completing the following academic skills to prepare quality assessments in the MBA course. Choose and circle only one of the six given rating scales.

Academic tasks Level of skills you haveLess than the level

expected in the courseSatisfactory Moderate Good Excellent Unsure

1 Ability to paraphrase 1 2 3 4 5 62 Ability to use APA referencing 1 2 3 4 5 63 Ability to avoid plagiarism 1 2 3 4 5 64 Ability to write a critical synopsis 1 2 3 4 5 65 Ability to write a critical essay 1 2 3 4 5 66 Ability to write an analytical essay 1 2 3 4 5 67 Ability to use probing and critical

questioning skills1 2 3 4 5 6

8 Ability to use critical thinking skills 1 2 3 4 5 69 Ability to search research database 1 2 3 4 5 610 Ability to do a targeted search for

relevant and credible research articles from research database

1 2 3 4 5 6

11 Ability to write a reflective journal 1 2 3 4 5 612 Ability to engage in debating 1 2 3 4 5 613 Ability to prepare a case study report 1 2 3 4 5 6

14

Appendix 2

Focus Group questions for North Sydney and Melbourne MBA students

1. Do you believe that the instructors made their expectations clear to you at the beginning of the course?

2. What did the University do to make your learning experience a positive one?

3. Can you recall any specific academic support or activities provided in your first semester at ACU which helped you complete quality assignments?

4. Did meeting past students help you to understand the learning requirements of the university?

5. Do you feel that staff are approachable and are taking an interest in your personal well being?

6. How did you learn to perform keyword searches of the library’s on-line catalogues and databases?

7. What more could the university do to ensure that your learning experience is a positive one?

15

Appendix 3

Transcribed Focus group discussion(Melbourne - 2 groups – 31st July 2009)

General Comments Workbook gave an example and all the steps to complete an assignment Workbook gave inspiration to write Expectations were made clear by the instructors but students felt they needed

them to be reiterated at different points in the semester as they didn’t always understand/remember the information from week 1 due to overload

Participating in lectures is a way of learning about the expectations Students felt the workbook combined with the academic skills sessions enabled

them to complete their assignments They felt the interactions between students and teachers were good They preferred the classroom to be arranged in a way to facilitate group work One student mentioned that Melbourne Uni has a brochure/booklet on what a

good essay looks like but not how to write one, so the student loaned her copy of the workbook to her fellow student at Melbourne Uni. Another student mentioned using the workbook to help a fellow student at another institution who was having referencing difficulties.

The academic skills workshops reinforce the workbook and it’s not just reading and writing – they also benefit from listening

Several students mentioned that failing an assignment was not necessarily a bad thing but often acted as a milestone

The workbook bridges the gap between lecturer’s expectations and students Group work is the most difficult especially semester 1 as the lecturers expect

them to make their own groups and they don’t know anyone yet. This means they usually make groups with students from their home country

They felt the workbook gave them confidence They felt ACU gave them more personalised attention than some of their friends

at other institutions and that the workbook contributed to this positive feeling. They also felt the staff made themselves very available to see them

They liked having the fob room for study and felt they were well resourcedDifficulties cited Getting the main ideas from the readings though they said this is where

completion of the critical synopsis helped them Finding keywords necessary to prepare their research assignments and

performing keyword searches that give them relevant informationSuggestions More work to help students locate keywords from their lectures and notes and

better enable them to perform relevant searches They also wanted examples of poor assignments to allow them some contrast More interaction with seniors who can share their own experience A section on strategies to prepare for open book exams

Students were asked to write down the five most important advice items that they would provide to junior students related to improving assignment grades.These are their suggestions: (transcribed not edited)(Student A)

16

Follow the subject outline (required topics need to be covered in an assignment)

Follow the study guide for Referencing Listen to the lecturer Do not copy Ask for help from academic skills Try to use the workbook in semester 1 at least three times

(Student B) Start your preparation from the first week Finish your assignment at least 1 week before your submission date and show

it to an academic skills adviser Marking scheme is very important and should be strictly followed Referencing and format should be according to the ACU Study Guide (Please

keep it after the first semester as well)

(Student C) Keep in mind the mark criteria and make sure that it is complete understood

and met Try to understand the lecturer’s point of view about main topics Share understandings with other classmates Use appropriate referencing Seek for another perspective from skill advisor and advice

(Student D) Read carefully the unit outline regarding the assignment and understand the

structure of assignment and the marks allocated to each point Understand the topic well and try to research some good articles relating to

topic Organise time well Group work

(Student E) Workbook really essential Get Library training Academic Skills – ask for help Group member adjustment

(Student F) Refer/back to workbook for assignment preparation Accessing the library – database Go to academic skills (to talk) Ask the seniors Ask for the lecture

Appendix 4

TELEPHONE INTERVIEW NOTES

17

Tuesday 4 August 2009 16.30

Interviewers:Sugumar Mariappanadar MelbourneBulend Terzioglu Melbourne

IntervieweeMBA student North Sydney

What has been your experience in the MBA course so far?My experience to-date is generally good.One of the problems I have is ACU is offering a limited number of subjects. Accounting is the most popular course. As far as I know 40-50 students are currently doing accounting.There are no marketing or HR courses.There is no project management course.

In your opinion, what are the primary contributors to student’s success:Academic skills unitUnderstanding the appropriate referencing requiredUnderstanding what the plagiarism is and how to avoid itLecturers’ comments on assignmentsReading books and other related materials

Do you think if you had a workbook that explains the rules for essay writing, referencing etc. would have helped your academic performance?Workbook. It’s hard to say whether I’d like to have it or not. I want to do research on my own and I prefer freedom.

How do you identify business journals and relevant articles?Generally, I use Factiva

Do you employ any strategy to reduce the time spent for research?I browse articles

Have you spoken to past MBA students about the requirements of the course?No. I haven’t met senior students and haven’t sought their opinion.

What can ACU do to make learning experience a positive one?All assessments are usually due in week 4. Due dates should be pushed 1-2 weeks further.Students come in late.I didn’t do well at first but my grades improved later.

Do you need any support to do your assignments well?No I don’t. Also, assignments are not taken by Indian students too seriously.Expectations vary from subject to subject.

Have you experienced difficulties in writing essays?Initially, I wasn’t very confident. But, in time my skills have improved.

18

What kind of learning tools do you think would be helpful in achieving better grades?Academic skills advisers explained how to write essay which has been very helpful.In Sydney, students were given a handbook.

In what ways, could ACU help you?Lecturers should explain more about the assignments. Early identification, say before week 4/5 of issues is also important.As for assignments, students do not often know at the beginning the magnitude of work required to complete assignments. Therefore, it would be helpful if requirements can be communicated to students early and comprehensively.

19