Faculty of Information Technology,

Monash University

Teaching ICT

The ICT-Ed Project

The report on learning outcomes and curriculum development in major university disciplines in

Information and Communication Technology

Refe rence GroupRefe rence Group

John Hurst (Chair), Angela Carbone, Malcolm Eley, Ainslie Ellis,

Dianne Hagan, Selby Markham, Judithe Sheard, Juhani Tuovinen

Commis s ioned Resea rche r s Commis s ioned Resea rche r s

Julianne Lynch, Francesca Collins

Computing Education Research Group

Faculty of Information Technology, Monash University

00/ December 2001

Higher Education Division

Department of Education, Training and Youth Affairs

© Commonwealth of Australia 2001

ISBN 0-9580945-0-0

ISBN 0-9580945-0-0 (Internet copy)

DETYA No. XXX

This work is copyright. It may be reproduced in whole or in part for study or training purposes subject to the inclusion of the source and no commercial usage or sale. Reproduction for purposes other than those indicated above, require the written permission from the Commonwealth available through AusInfo. Requests and inquiries concerning reproduction and rights should be addressed to the Manager, Legislative Services, AusInfo, GPO Box 1920, Canberra ACT 2601.

This report is funded under the Evaluations and Investigations Programme of the Department of Education, Training and Youth Affairs, through the Australian Universities Teaching Committee. Support from the Monash Faculty of Information Technology and School of Computer Science and Software Engineering is also acknowledged.

The views expressed in this report do not necessarily reflect the views of the Department of Education, Training and Youth Affairs.

National Library of Australia Cataloguing-in-Publication data

Teaching ICT : the ICT-ed project : the report on learning outcomes and curriculum development in major university disciplines in information and communication technology. Includes index. ISBN 0 9580945 0 0. 1. Curriculum planning - Australia. 2. Information technology - Study and teaching (Higher) - Australia. 3. Universities and colleges - Australia - Curricula. I. Hurst, A. J. (Angas John). II. Lynch, Julianne. III. Collins, Francesca. IV. Monash University. Faculty of Information Technology. V. Australia. Dept. of Education, Training and Youth Affairs. Higher Education Division. 004.071194

i

Contents

Contents ........................................................................................................................................................................i

Tables ..................................................................................................................................................................... iii

Abbreviations and acronyms................................................................................................................................... v

Acknowledgments....................................................................................................................................................vii

Executive Summary..................................................................................................................................................ix

Conclusions............................................................................................................................................................. x

Recommendations .............................................................................................................................................. xiii

1 Introduction.......................................................................................................................................................1

1.1 Background and project specifications ................................................................................................1

1.2 Setting the scene: University ICT education .......................................................................................4

1.3 Structure of the report.............................................................................................................................6

2 Employers’ views of ICT graduates ..............................................................................................................7

2.1 Introduction................................................................................................................................................7

2.2 Research on needs and satisfaction of employers...............................................................................8

2.3 The market context of employer expectations .....................................................................................9

2.4 Employer Data Collection......................................................................................................................11

2.5 University responses to stated needs from industry.........................................................................26

2.6 Conclusions .............................................................................................................................................27

2.7 Recommendations arising from this section.....................................................................................28

3 Graduates in the workforce...........................................................................................................................29

3.1 Introduction..............................................................................................................................................29

3.2 Graduate interviews ................................................................................................................................31

3.3 Conclusions .............................................................................................................................................41

3.4 Recommendations arising from this section.....................................................................................41

4 ICT Educators’ views on innovation..........................................................................................................43

4.1 Introduction..............................................................................................................................................43

4.2 Context of university teaching in Australia.......................................................................................43

4.3 Methods.....................................................................................................................................................45

4.4 Findings: Factors driving innovation in ICT education ...................................................................47

4.5 Needs of students and employers .........................................................................................................59

4.6 Conclusions .............................................................................................................................................61

4.7 Recommendations arising from this section.....................................................................................61

5 Teaching and learning initiatives ...............................................................................................................63

5.1 Introduction..............................................................................................................................................63

5.2 Teaching and learning initiatives ........................................................................................................63

ii

5.3 Innovation and best practice ................................................................................................................. 74

5.4 Conclusions............................................................................................................................................. 85

5.5 Recommendations arising from this section .................................................................................... 87

6 ICT educators’ views on dissemination of innovation........................................................................... 89

6.1 Introduction............................................................................................................................................. 89

6.2 Methods .................................................................................................................................................... 89

6.3 Findings – dissemination of innovation.............................................................................................. 90

6.4 Conclusions............................................................................................................................................. 98

6.5 Recommendation arising from this section ...................................................................................... 99

7 Educational evaluation..............................................................................................................................101

7.1 Contextual factors ................................................................................................................................101

7.2 Criticism of dominant student feedback approaches .....................................................................105

7.3 Alternative approaches: Using student feedback to model student behaviour ..........................108

7.4 Conclusions...........................................................................................................................................110

7.5 Recommendation arising from this section ....................................................................................111

8 Conclusion.....................................................................................................................................................113

8.1 Reception of project.............................................................................................................................113

8.2 Summary of finding .............................................................................................................................115

8.3 Challenges for the future ...................................................................................................................120

References...............................................................................................................................................................123

APPENDIX A. Reference group membership .................................................................................................131

APPENDIX B. Mini-conference participants .................................................................................................133

APPENDIX C. Intensive workshop participants............................................................................................137

APPENDIX D. Methodology – Mini-conference program...........................................................................139

APPENDIX E. Mini-conference workbook.....................................................................................................151

APPENDIX F. Statistical tables for industry survey data............................................................................167

APPENDIX G. Methodology – Graduate interviews.....................................................................................185

APPENDIX H. Broad context of university teaching in Australia.............................................................189

APPENDIX I. Movements in university teaching and learning ..................................................................195

APPENDIX J. Teaching and learning initiatives – brief accounts ............................................................205

APPENDIX K. Industry questionnaire .............................................................................................................231

APPENDIX L. Graduate cases ...........................................................................................................................239

iii

Tables Table 1. Project aims and outcomes........................................................................................................................2 Table 2. Counts by course employed ....................................................................................................................13 Table 3. Contact with Universities (Q.1.7) (Percentages) ...............................................................................14 Table 4. Descriptives for satisfaction measures..................................................................................................15 Table 5. Meets needs ratings..................................................................................................................................15 Table 6. Descriptives for 'gap' between Importance and Meet needs.............................................................17 Table 7. Correlations between outcome variables ............................................................................................19 Table 8 Anticipated future recruitment of graduates - Do Not Currently employ .......................................25 Table 9. Summary of interviewees by graduate destination and graduate type combinations.................33 Table 10. Initiatives identified as innovation and/or best practice ...............................................................77 Table 11. Channels identified by participants for the dissemination of teaching and learning innovations................................................................................................................................................................95 Table F1. Respondents’ business size................................................................................................................ 167 Table F2. Respondents’ industry type ............................................................................................................... 167 Table F3. Respondents’ state ............................................................................................................................. 168 Table F4. Respondents’ region........................................................................................................................... 168 Table F5. Respondents’ level of education...................................................................................................... 168 Table F6. Organisation’s contact with universities through industrial experience................................ 168 Table F7. Organisation’s contact with universities through Committees.................................................. 169 Table F8. Organisation’s contact with universities through Research & Development......................... 169 Table F9. Organisation’s contact with universities through Consulting .................................................. 169 Table F10. Importance of skills in ICT graduates........................................................................................... 170 Table F11. Extent to which recent ICT graduates meet organisation’s needs.......................................... 171 Table F12. Satisfaction with graduates ............................................................................................................ 171 Table F13. Outcome variables............................................................................................................................ 172 Table F14. Other variables.................................................................................................................................. 172 Table F15. Important Skills - Difference from assumed mean 3.0 ............................................................... 173 Table F16. Meet company’s needs - Difference from assumed mean 3.0 ..................................................... 174 Table F17. Satisfaction - Difference from assumed mean 3.0 ....................................................................... 174 Table F18. Outcome measures - Difference from assumed mean 3.0 ............................................................ 175 Table F19. Other variables - Difference from assumed mean 3.0 ................................................................ 175 Table F20. SATISFACTION and NEEDS Coefficients.................................................................................... 176 Table F21. Skills in particular computer languages ..................................................................................... 177 Table F22. Skills in particular software applications................................................................................... 177 Table F23. Foundation in theoretical principles ........................................................................................... 178 Table F24. Ability to communicate with clients ............................................................................................. 178 Table F25. Written communication skills ......................................................................................................... 178 Table F26. Ability to work in teams................................................................................................................... 179 Table F27. Project management skills .............................................................................................................. 179 Table F28. Problem solving skills...................................................................................................................... 179 Table F29. Understanding of business processes............................................................................................ 180 Table F30. Preparedness for quick entry into job tasks ............................................................................... 180 Table F31. Respondent’s business size............................................................................................................. 180 Table F32. Respondent’s industry type............................................................................................................. 181 Table F33. Respondent’s state........................................................................................................................... 181 Table F34. Respondent’s region......................................................................................................................... 181 Table F35. Respondent’s level of education .................................................................................................... 182 Table F36. Respondent’s contact with universities through Industrial Experience................................ 182 Table F37. Respondent’s contact with universities through Committees .................................................. 182 Table F38. Respondent’s contact with universities through Research & Development ......................... 182 Table F39. Respondent’s contact with universities through Consulting................................................... 183 Table F40. Anticipated future employment of ICT graduates....................................................................... 183 Table G1. Summary of interviewees by graduate destination and graduate type combinations........... 186

v

Abbreviations and acronyms

ACS Australian Computer Society

AIIA Australian Information Industry Association

AIMIA Australian Interactive Multimedia Industry Association

ALIA Australian Library and Information Association

ASP active server page

AUTC Australian Universities Teaching Committee

CAUT Committee for the Advancement of University Teaching

CEQ Course Experience Questionnaire

CERG Computing Education Research Group

CSCW computer supported collaborative work

CSSE computer science and software engineering

DETYA Department of Education, Training and Youth Affairs

FIT Faculty of Information Technology

GCCA Graduate Careers Council of Australia.

GDQ Graduate Destinations Questionnaire

ICT information and communication technology

IE industry experience

IEEE Institute of Electrical and Electronics Engineers

IM information management

IP intellectual property

IS information systems

IT&T information and technology and telecommunications

LOTE language other than English

NOIE National Office for the Information Economy

QA quality assurance

SET student evaluation of teaching

SOS student opinion survey

TAFE Technical and Further Education

UKQAA UK Quality Assurance Agency

UNS Unified National System

VET Vocational Education and Training

vii

Acknowledgments

This document reports the findings of an Australian Universities Teaching Committee (AUTC) research project on the major discipline area of Information and Communication Technology (ICT). The research project, known as the ICT-Ed Project, was undertaken by the Computing Education Research Group (CERG) in the Faculty of Information Technology, Monash University. It was funded by the Department of Education, Training and Youth Affairs (DETYA), through the AUTC. Support from the Monash Faculty of Information Technology and School of Computer Science and Software Engineering is also acknowledged.

A research team, comprising Julianne Lynch, Francesca Collins and Selby Markham, under the direction of a reference group, conducted the research. The reference group was chaired by John Hurst. Reference group members were Angela Carbone, Malcolm Eley, Ainslie Ellis, Dianne Hagan, Selby Markham, Judithe Sheard and Juhani Tuovinen. Administrative support was provided by Pauline Bond.

The bulk of this report was written by Julianne Lynch, Francesca Collins and Selby Markham. All members of the reference group contributed to the revision and preparation of the final draft.

ix

Executive Summary

This national study was undertaken as the result of a tendering process and grant from the Department of Education, Training and Youth Affairs, through the Australian Universities Teaching Committee (AUTC). A review process was undertaken with the aim of investigating the ways that teaching and learning are being approached in the major discipline of Information and Communication Technology (ICT) in Australia’s universities.

A variety of methods were used to review research and, where necessary, to collect data, from and about stakeholder groups in ICT education.

• A combination of a literature survey and the administration of a largely quantitative questionnaire was used to investigate employers’ views on their needs in, and satisfaction with, university ICT graduates.

• Interviews were used to investigate ICT graduates’ views on their courses.

• A literature survey provided information about methods for assessing students’ satisfaction with their courses and for the evaluation of teaching. This information was complemented by the reported experiences of ICT educators.

• Consultative methods were used to collect qualitative data on ICT educators’ views of educational innovation and the dissemination of good practice, and on current teaching and learning initiatives.

Conclusions can be made about

• employers’ views of the preparedness of ICT graduates for the workforce

• future research needed to assess ICT graduates’ views on the value of their courses

• methods for assessing student satisfaction and for evaluating educational innovations and ICT educators’ skills in these evaluation methods

• ICT educators’ views on innovation in ICT education and the dissemination and diffusion of innovation.

• the types of teaching and learning initiatives currently being undertaken by ICT educators and the issues they respond to

Recommendations are made for the improvement of interactions between ICT departments and the outside world (specifically employers and graduates), the promotion, dissemination and diffusion of educational innovation, and the promotion of the evaluation of educational innovations. The major conclusions are outlined below, and then recommendations are listed.

x

Conclusions

Employers’ views Employers were found to be generally satisfied with their ICT graduates. Industry sees graduates as having a number of obvious deficiencies but these are irritants, or squeaky wheels, rather than being significant decision-making factors. These irritants have been reported in different papers over a 10-year period and are not isolated to graduates from ICT disciplines.

A review of the literature and data collected in this project suggest that ICT skills shortages are not uniform. More labour market analysis is needed if ICT departments are to have a clear understanding of the career environment of their graduates. There is a need for further research to develop a clearer picture of the structure of the ICT industry, of what is critical to the graduate needs of employers, and of employers' commitment to career structures for ICT professionals. Due to the changing nature of the ICT industry, continuing research is required to provide up-to-date information to educators for the purpose of curriculum development.

The results from the analysis of the employer questionnaire, combined with suggested shortcomings of previous satisfaction studies, point to the importance of building research models that are sufficiently sophisticated to analyse employer behaviour in terms of graduate recruitment and satisfaction.

In terms of curriculum development, the ICT sector is subject to the vagaries of changing technologies, ideas and, to some extent, fashions. ICT departments should have depth in their degree structures, which will allow them to readily shift direction as new waves pass through. Focusing on skills training would lead to medium term staffing problems. It would also lead to broad human resource problems as changing technology makes redundant those who have not been educated for adaptation and change.

Graduates’ views The primary conclusion to be drawn from the review of literature on graduate satisfaction and graduate behaviour is that very little is known about ICT graduates and their attitudes towards their courses. Universities routinely monitor initial graduate destinations, but this exercise is not intended to inform curriculum development and educational innovation.

Our preliminary exploration of graduates’ career stories identified a number of themes that possibly characterise a particular type of ICT graduate, that is, the type who are exposed to, and develop an interest in, computing technology during their childhoods, are motivated to pursue the intrinsic rewards of using computing technology, and teach themselves how to program before entering any formal ICT education. This type predominated in our small sample, but may not be typical due to the methods used to recruit graduates to the study.

Differences were observed between male and female interviewees. Unlike the male graduates who had long pursued interests in computer technology and saw their university degrees as a natural continuation of this pursuit, the two female graduates were more pragmatic in their views about the value of their degrees and

xi

had made conscious decisions as young adults to pursue a career in ICT. Due to the small number of graduates interviewed, observed gender differences cannot be generalised. However, this does raise questions about the needs, attitudes and behaviours of graduates with different backgrounds and interests and should be addressed if students who do not fit popular stereotypes are to be attracted to ICT degrees.

Interviewees’ attitudes towards the non-programming subjects in university ICT degrees raise questions about their expectations of their courses and their expectations of the work of ICT professionals. Many of the graduates interviewed had done very little research into the nature of careers in the industry prior to enrolling in their degrees.

The themes and issues identified in the analysis of these exploratory interviews warrant further inquiry. Further research would have implications for the recruitment of students to ICT degrees, the promotion of ICT degrees and the ICT professions to students, the provision of career counselling to ICT students, and the development of programs to better prepare graduates for work as ICT professionals.

Student satisfaction and educational evaluation Methods currently used in university departments to assess students’ satisfaction with their courses are inadequate for the purpose of continuous improvement.

On the whole, ICT educators are unfamiliar with the principles and techniques of educational research and evaluation methods. This restricts their ability to conduct formative evaluations that provide useful information for the improvement of their teaching. It also inhibits their ability to conduct the summative evaluations that would support the dissemination of educational innovations. A related problem is their lack of skills to assess the innovations of others as reported in the wider educational literature. A lack of resources and a lack of reward for the pursuit of educational evaluations further compound these problems. Evaluation and dissemination activities are seen to be a very low priority in ICT departments. Consequently, ICT educators and their departments have difficulty demonstrating that their teaching activities are innovative and have real effects on student learning outcomes.

ICT educators’ views Consultation with ICT educators revealed that the learning needs of students are the primary driving force in innovation in ICT education. Innovative teaching staff are motivated by a desire to be better teachers. ICT educators perceive extrinsic motivation and reward for educational innovations to be minimal. In fact, many associate being an innovative teacher with personal and professional risks. Institutional agendas and reward systems were seen by some to discourage educationally motivated innovation.

ICT educators reported a tension between some of the demands of students (for particular skills and knowledge) and the skills and knowledge they believe are necessary for students to be ICT professionals and life-long learners. They report that students often have misconceptions about their future working roles and

xii

careers. This perception is supported by observations made during the graduate interview phase of the research.

Students were generally seen as conservative in terms of the types of teaching and learning activities they expected and desired in their courses, with ICT educators reporting that students were often resistant to innovative, non-traditional teaching.

A range of factors was identified as inhibiting ICT educators’ ability to generate and disseminate educational innovations. These included the relatively lowly status of teaching when compared with research, ICT educators’ general lack of training in educational evaluation, a lack of resources and a lack of recognition and reward for innovative teaching.

Teaching and learning initiatives The impetus for, and aims of, the majority of the teaching and learning initiatives presented to the Project are student-focused, with many responding to needs or problems encountered by students and a desire to improve the teaching of ICT. Other initiatives were motivated by the changing context of university teaching such as reduced resources, increased student numbers and more diverse student population, or the changing needs of employers.

Most initiatives had been formally evaluated, however, for a large proportion of these initiatives, standard student feedback questionnaires were the only source of evaluation. Participants recognised the limitations of relying solely on student feedback questionnaires for the evaluation of teaching and learning initiatives. However, given the lack of encouragement of, and reward for, evaluation activities, and their lack of skills, many ICT educators chose not to pursue formal evaluations beyond the standard student evaluation of teaching questionnaires required by their departments.

It was also observed that many ICT educators do not recognise the significance of their teaching practice as innovative, so they use informal means to evaluate and improve their teaching, rather than pursuing evaluation methods that better qualify their work for publication as educational research. This tendency also inhibits their ability to demonstrate the scholarship in their teaching for the purpose of promotion.

Forty-one of the initiatives presented to the project team at the mini-conferences specifically targeted ICT education. Eleven of these were identified by members of the Monash Computing Education Research Group (CERG) as being innovative, where innovation is taken to involve something that is new or not practised by the majority, that adds value to learner-based outcomes, that has been implemented, that is practicable in educational and infrastructure terms and that has been evaluated. Areas addressed by those initiatives identified as innovative included student retention and student engagement, enhancing learning, catering for groups of students with diverse skills, accommodating large class sizes, curriculum integration and teaching skills associated with professional practice.

Fifteen of the forty-one initiatives were specifically identified by CERG as being best practice, where best practice is taken to involve something that has been formally evaluated in terms of educational outcomes or teaching outcomes, and that is integrated into a wider educational program. Areas addressed by those

xiii

initiatives identified as best practice included student retention, attracting and retaining female students, enhancing learning, catering with groups of students with diverse skills, accommodating large class sizes, the teaching of skills associated with professional practice and the teaching of learning skills.

Recommendations

Improving interactions with the outside world

• Recommendation 1 That universities’ ICT departments be given support in establishing links with industry. This will enable them to explore real-world options on how the university-industry interface can be made permeable enough to help meet industry needs without compromising the broader educational needs of graduates. In conjunction with this, that exploration be carried out on the development of post-graduate, skills-based courses which reflect changes in technology and the industry.

• Recommendation 2 That universities be given support in implementing graduate follow-up and employer satisfaction studies which focus on local issues and local needs, and which use a diagnostic approach. This data would complement that routinely collected by the Graduate Careers Council of Australia.

• Recommendation 3 That the research and development currently being undertaken by CERG members on student satisfaction form the basis of broader work on the development of user-oriented tools and methods for interpreting and diagnosing graduate and employer satisfaction.

• Recommendation 4 That research funds be allocated to longitudinal and/or retrospective studies of graduates to obtain an understanding of the career paths through the ICT industries, both within Australia and overseas.

• Recommendation 5 That university ICT departments, in collaboration with industry, devise programs for the better management of students’ expectations of professional life.

• Recommendation 6 That further ongoing research be initiated, within the context of labour market models, to determine the structural differences between those organisations who are likely to employ ICT graduates and those who are unlikely to employ them. This research would ideally be based on collaboration between universities, government and industry and aim to provide up-to-date information to university curriculum developers about industry needs. In conjunction with this, research is needed into the influence of company size and the consequences of this for the size of the market and the skills and knowledge needed.

xiv

Educational innovation and dissemination • Recommendation 7 That the management of university ICT departments

value teaching by

§ developing acceptable and practicable systems for the assessment and reward of educational innovation and good practice

§ supporting and encouraging the pursuit of educational research and publication in appropriate journals and conference proceedings

§ formally supporting, recognising and rewarding scholarship in teaching in the ICT disciplines. Examples of scholarship in teaching include the development of new tools, techniques and materials, and related evaluation activities.

§ encouraging staff to undertake educationally focussed professional development activities

§ allocating resources to these activities

• Recommendation 8 That the management of ICT departments support the development of innovative solutions to the challenges posed by the changing context of ICT education, such as increasing class sizes, increasingly diverse student populations and the increasing incidence of plagiarism.

• Recommendation 9 That universities continue to give ICT educators the freedom to choose the particular teaching tools that they use, including the development of their own tools. While it is expected that each university will provide support for a limited range of courseware platforms, care should be taken to ensure that further innovation is not stifled by the promotion and support of specific tools.

• Recommendation 10 That effective means be developed for the dissemination and diffusion of educational innovations to the ICT education community. Diffusion efforts should

§ acknowledge and accommodate ICT educators’ need to ‘make innovations their own’

§ facilitate two-way interaction between those who generate the innovation and those who plan to adopt it

§ allow for the dissemination of work-in-progress and the exchange of peer feedback

§ take advantage of existing relationships between individuals and between institutions

§ take advantage of discipline-based loyalties and shared concerns to promote inter-university collaboration and dissemination across the disciplines

§ acknowledge issues such as inter-university competition and intellectual property

xv

Educational evaluation

• Recommendation 11 That management encourage ICT educators to evaluate their teaching and learning initiatives through

§ the use of evaluations, both formative and summative, that focus on investigating links between the particular objectives of initiatives and educational outcomes.

§ the provision of professional development and support programs that focus on educational evaluation as well as on teaching tools and methodologies. Programs should be developed in consultation with available experts, for example, educators from university Education departments and academic development units.

§ the user-based evaluation of the services and programs described above to ensure that they meet the needs of ICT educators.

§ the allocation of resources to support these activities

• Recommendation 12 That continuous improvement approaches be adopted in ICT departments so that ICT educators can more rapidly respond to change and input from professional, organisational and industry sources.

1

1 Introduction

This national study was undertaken as a result of a grant from the Department of Education, Training and Youth Affairs, through the Australian University Teaching Committee (AUTC). The aim was to investigate the ways that teaching and learning are being approached in the major discipline of Information and Communication Technology (ICT) in Australian universities. The purpose of the project was to provide information to the AUTC that would assist the Committee in its promotion and support of excellence in university teaching.

The study was designed in response to a Project Brief. A Research Team, comprising a senior research fellow, a research assistant and an administrative assistant, was commissioned to administer and manage the implementation of the research. A Reference Group (see Appendix A for membership), comprising members of the Monash University Computing Education Research Group, was formed to oversee and manage the overall project. A National Steering Committee, comprising representatives from the funding body, was appointed to provide general advice and support for the project and to ensure that the project meets its aims within the proposed timeline and budget, in accordance with the brief and requirements of the grant agreement.

The Project aimed to maximise the cooperative input from ICT departments and faculties across Australia as well as involving employers and graduates in a constructive way. Eighty-three ICT educators from 29 universities contributed to the data collected by the Project (see Appendix B for participant list). Ten senior ICT academics and one international ICT educator (see Appendix C for participant list) contributed to the Project outcomes by reviewing the recommendations made and identifying implementation issues. This high level of consultation and collaboration with those with interests in the ICT discipline has resulted in research outcomes and recommendations that are both relevant and credible.

The background to this report, the project specifications and the structure of the report are described below.

1.1 Background and project specifications

1.1.1 Australian University Teaching Committee In January 2000, the minister for Education, Training and Youth Affairs established the Australian University Teaching Committee, a national body aimed at improving Australian university teaching and learning. During 2000 the AUTC commissioned a small number of projects to focus on issues of teaching and learning within major disciplinary areas, with a view to learning more about the ways in which teaching and learning have been approached.

2

This report is the final product of Stage 1 of a research project focusing on the major interdisciplinary area of Information and Communication Technology (ICT) education. As described below, Stage 1 of the project comprises a review whose purpose is to identify the extent to which innovations in teaching and learning and processes of curriculum development and review have been deployed in response to the needs of students and employers.

1.1.2 Research specifications The Brief for this project (Australian University Teaching Committee 2000) is wide-ranging and, while its focus is on issues of teaching and learning, the role of graduate and employer satisfaction, academic staff shortages and professional development, the assessment of quality teaching and the dissemination of innovation are also important elements. Crouch and McColl (2001) noted that in commissioned studies involving a high level of collaboration among diverse stakeholders, with multiple policy and practice expectations, it is useful to “distinguish between the aims of the broader project, in which the research [sits] and for which particular audiences could be identified, and the specific aims and objectives of the research itself” (p.3). The tender document (Computing Education Research Group 2000) for the project specified research activities that would be used to meet various elements of the Brief. These activities, including the specific aims of each activity, are summarised in Table 1.

Table 1. Project aims and outcomes

Aims Outcomes

Establishing parameters

• To make contact with ICT departments in Australian universities

• Database containing contact details for all department heads

• Written communication with all department heads, announcing project

• Written communication with all department heads, inviting participation in the consultation phase.

• To develop a Web site for disseminating information and collecting data

• Web site developed at http://cerg.infotech.monash.edu.au/icted including general project information, contact information and an inquiry form

• Web site development continued over life of project

• To review literature that would inform subsequent phases

• Literature reviewed on

• Educational innovation and good practice in ICT education

• Industry needs and employer satisfaction

• Graduate behaviour and graduate satisfaction

3

Aims Outcomes

Consultation with ICT educators

• To investigate current innovations in ICT education

• Mini-conferences held in each capital city

• Data collected via mini-conference presentations provided an overview of ICT educators’ innovative activities

• To investigate ICT educators’ views on developments in teaching and learning and the dissemination of innovation

• Data collected via mini-conference open discussions provided a picture of ICT educators’ views about the directions of innovation and the dissemination of innovation

• To create a positive feel for the project among ICT educators

• Mini-conferences involved quality engagement with ICT educators and received positive feedback. A broad range of academic levels attended, from a large number of departments.

Survey of employers • To investigate employers’ attitudes towards ICT education

Questionnaire administered to employer groups provided a picture of employers’ needs, their satisfaction with graduates and their attitude towards ICT education

Graduate satisfaction • To gain an overview of graduate outcomes in ICT education

• Review of literature on graduate outcomes, including graduate behaviour and graduate satisfaction

• To investigate graduates’ satisfaction with their courses

• To investigate views on the role their courses played in their careers paths

• Graduate interviews focusing on graduate employment behaviour and graduates’ attitudes towards ICT education

International trends • To identify the parameters being used to define innovation and good practice in ICT education in other countries

• Contact made with innovative ICT educators in other countries through CERGI

• Development of a Web form for submission of information about teaching and learning initiatives in other countries; little information was forthcoming

Follow-up interviews • To clarify and validate information gathered in previous phases

• Follow-up interviews conduced with selection graduate interview and industry survey participants

Intensive workshop • To gain input and support from senior ICT educators for recommendations

• Ten senior Australian ICT academics and one International expert participated in a day-long workshop. Participants reviewed recommendations. Their suggestions then informed the revision of the report.

4

1.2 Setting the scene: University ICT education Information and Communications Technology covers a very diverse range of courses and areas within the University education environment. This study would not be complete without setting the scene regarding the structure and location of the various aspects of ICT within the university sector.

While there are a plethora of degrees and subjects related to this area, the ICT field can be divided up into six broad categories:

• Communications technology

• Computing technology at the electronic and circuitry development level

• Computer science and software engineering

• Information systems within business commercial computing

• Multimedia and Multimedia Systems

• Information management

1.2.1 Communications Technology This category involves the exploration of telecommunications areas. This category focuses on such areas as digital logic, signal processing and transmission, TV and radio telecommunications, satellite systems and ATM and ISDN networks. This category is firmly situated with Engineering departments, schools or faculties.

Courses are often labelled Telecommunications Engineering or Communications Engineering.

1.2.2 Computing technology at the electronic and circuitry development level This involves the exploration, design and development of microchip technology and the building of computing circuitry. It also includes the control and the development of electronic systems using computer technology to control mechanical processes and establish process control systems.

Some courses place the emphasis on the electronics discipline and tend to be situated in Engineering departments, schools or faculties, while other place the emphasis on the computing discipline and the process control aspects and tend to be situated within Information Technology departments, schools or faculties.

Courses in these areas are often labelled as Computronics, Electronic Computing, Electrical and Computer Systems, Mechatronics or Digital Systems.

5

1.2.3 Computer Science and Software Engineering This category concentrates on information technology at the computer design level and its application in the software engineering discipline. It covers such areas as operating systems, systems architecture and software management and the development.

In the early establishment of separate computing departments within the university sector, the first information technology department was often a Department of Computer Science established within the Science Faculty of a University. This is still the case, although these departments often incorporate far more computing courses than just those that concentrate on Computer Sciences. Computer Science is now also part of Information Technology departments, schools or faculties. Software Engineering will be situated in Engineering and / or Information Technology faculties.

Courses in these areas are usually labelled Computer Science, Software Engineering, or computing or engineering with a defining tag (e.g. Bachelor of Computing (Computer Science), Bachelor of Engineering (Software Engineering)).

1.2.4 Information Systems within Business Commercial Computing This category of Information Technology includes the design and development of computing application systems for the commercial environment. It covers a wide range of areas that involve the development of information systems and the management of information systems within the commercial area. Since the extensive development of the Internet, more specialised areas, particularly in relation to electronic commerce and network systems, are emerging.

Courses in this area, particularly those that concentrate on information systems or business management will often be located within a Business department or school which resides either within a Business Faculty or an Information Technology Faculty. Those that concentrate on commercial computing or networking will usually be located in a Computing department, school or faculty, although some reside within Science faculties.

Courses in this category are often labelled Computing, Information Technology, Networking or Network Systems, Information Systems, Management Information Systems, Business Systems and Electronic Commerce.

1.2.5 Multimedia and Multimedia Systems With the rapid expansion of the use of the World Wide Web and the increasing use of computer technology within the arts and entertainment environment, there have been a growing number of courses developing within information technology under the heading of multimedia. This category includes the exploration of varying forms of media for advertising, information dissemination and interactive environments using computer technology.

Courses in this category often reside within, or have links with, departments, schools or faculties of Graphic Art or Art and Design.

Courses in this category are often labelled Multimedia, Multimedia Technology and Multimedia Systems.

6

1.2.6 Information Management With most libraries placing a strong emphasis on electronic resources and services, information management courses focus on knowledge and skills essential to the management of information content. These may be in intranet environments or within the broader context of global networks. Information Management courses focuses on principles, processes and tools for the creation, organisation, categorisation, storage, retrieval and dissemination of a range of information types to meet user requirements in a variety of different organisational and social contexts.

1.3 Structure of the report The research findings for the various phases of the report are presented in five sections:

• Section 2 presents findings relating to the views of employers. Conclusions are based on findings of a literature review and an industry survey.

• Section 3 presents findings relating to the views of graduates. Conclusions are based on findings of a literature review and exploratory graduate interviews.

• Sections 4, 5 and 6 present findings relating to the views and activities of ICT educators. Conclusions are based on findings of the mini-conference program.

Section 7 draws on findings from Sections 4, 5 and 6, to discuss the role of evaluation in the improvement of ICT education. Finally, conclusions made in each section are presented and discussed in Section 8.

The body of the report contains summaries of the methods used. These can be found in each respective section. Detailed descriptions of methods are given as appendices.

7

2 Employers’ views of ICT graduates

One factor that just came up to my mind, the fact that the students are going to be life long learners. I mean they’re not going to finish their schooling here, they’re going to go into industry, five years from now whatever … It’s much more important to learn how to learn than to actually learn a particular language which we are teaching now which is going to be obsolete anyway. (Mini-conference Participant)

2.1 Introduction The interaction between employers and universities is a key component in any attempt to understand the way in which university courses are responding to the wider needs of ICT education. Elements within the interface between universities and industry which will be dealt with in this section of the report include:

• Research into employers’ expectations of graduates

• The relationship between expectations and satisfaction and the implications for educational programs

• Effective means of measuring employer satisfaction with graduates

• The perceived response of universities to industry needs

2.1.1 Rationale for approach taken There are varying levels of information about the interface between universities and industry. Within this there are myths about what is wanted and what is needed, and these myths are generated by both the university sector and the industry sectors. Unfortunately, many of these myths permeate down to the school level and become a part of the informal information system on careers. For example, one of the pervasive myths in the ICT sector is that graduates earn a great deal of money. Since the e-bubble was burst, this is an even more mythical entity. Another set of myths has been the skills and leading-edge myth. Computing has been seen as an area where everyone is doing leading-edge work applying the highest level of cognitive skill. The reality is that many computing graduates do what most other graduates do - fairly routine work which has a modicum of challenge.

By first building a picture of what we know from local and overseas research and practice, and then looking at local data on the university-industry interface, it was assumed that the project would be able to isolate important issues in the push-pull debates about university and industry in the ICT sector. Further, the project would be able to provide data that would clarify significance of the innovation and best practice information Section 5. Finally, the data on employers’ views

8

would also provide cross-comparisons with data collected from graduates in the workforce (see Section 3).

2.1.2 The components to be considered The project brief asks about the way in which universities have responded to industry needs and looks towards the exploration of means of measuring industry satisfaction with ICT graduates.

The obvious component to be dealt with is the ways in which satisfaction might be measured and how this could relate to curriculum planning and development in universities.

The second component, which has taxed the TAFE sector for some years and is effecting higher education overseas, is the definition of skills and competencies which are needed for effective development of undergraduates.

Within this is the question of the effect that these issues have upon university educators, particularly through quality management thinking and the vocationalisation of education.

2.2 Research on needs and satisfaction of employers

2.2.1 Skills and competencies perceptions Perhaps the most important issue when researching employer response to graduates (and any group of employees coming out of education) is the employers’ expectations. The Vocational Education and Training (VET) sector has focussed upon identifying competencies and skills that allow for a close analysis of what a VET graduate can be expected to bring to a job. The university sector in Australia has not embraced this approach although there is some current pressure to do so.

Individual employers have their own views of what a graduate should be able to do. These range from students coming in with good problem solving skills through to students being able to do specific, practical tasks.

The problem of defining the relative balance between practical skills and academic knowledge is by no means new. Educational researchers in the United Kingdom debated, during the 1960’s and 70’s, the question of the differences between what were then called Sandwich Courses against standard courses. Research that was carried out at Brunel University during the late 1960’s indicated that students from practical Sandwich Courses quickly fitted into organisations, but it was the students from the standard academic course who were of most value in the long term when research and development activities were needed. Nothing in the subsequent years has contradicted these basic findings.

Consequently, any attempt to research employer satisfaction with graduates has to be approached with caution.

9

2.2.2 Recent research on employer satisfaction This caution is reinforced by the relative paucity of published research on graduates entering employment, particularly in the ICT sector. The interim report on DETYA’s employer satisfaction project (ACNielsen Research Services 1998) has only 13 references. A search of education bibliographic databases suggests that little work has been published in the English-speaking world on employer satisfaction with university graduates. None were found on ICT graduates.

The most recent Australian report into employer satisfaction with graduates was commissioned by DETYA and carried out by ACNeilsen (2000). The methodology used generated a very general picture of graduates entering the workforce. This picture is of limited use to universities in their attempts to explore ways of better preparing students and it offers nothing to guide ICT course leaders.

The key data collection for the report was based on the employers rating the importance of certain characteristics and their satisfaction with graduates for each of these characteristics. Using a gap analysis approach, the difference between importance and satisfaction is analysed by looking at mean trends.

Most of the graphs in the report show similar patterns across most areas of study.

The data as presented in the ACNeilsen report does not allow us to determine the relative importance of any given components. There are, for example, no outcome measures such as overall satisfaction with performance or an indication of the future likelihood to employ graduates of this type or from the same institution. Without this type of data it is difficult for the universities to make sense of the consequences of the reported deficiencies.

2.3 The market context of employer expectations Various studies point to skills shortages in the ICT industry and forecast a continuation of these shortages (Poole 1997; Australian Information Industry Association 1996; Australian Information Industry Association 1999; Denning 1999; IT&T Skills Task Force 2000; National Office for the Information Economy 1998). The question of skills shortages in the ICT industry has been partially aligned to the role of Universities in preparing students for entry into the workforce. There is a clear pragmatic and pedagogical gap between what market place projections say about given job environments and what is being done in educational environments. Currently there is no educational policy that says that the university sector should respond in a way similar to that taken by the VET sector, that is to focus upon a skills/competencies approach.

2.3.1 ICT Graduate Skills and The Employer

2.3.1.1 History

There have been a series of reviews that, beginning in the late 1980’s and early 1990’s, look at the skills and attributes needed by the computing industry. The future of discipline assessments (Higher Education Council 1991) prompted the Commonwealth Government to commission a number of Discipline Reviews to

10

assess the nature and extent of current provision of education in major disciplines in Australian including Computing Studies and Information Sciences Education.

The review of computing studies (Department of Employment Education and Training, Department of Industry Technology and Commerce & Information Industries Education and Training Foundation 1992) identified a number of employers’ preferences, and a number of weaknesses, in terms of graduates’ skills and attributes. Employers were found to prefer graduates who, further to subject competence, had transferable skills and graduates who had completed an industrial placement. There was also a noted deficit in communication skills. As we shall see, the title of the subject area may have changed along with the technology, but the employer perceptions have remained quite constant.

2.3.1.2 Current perspectives

The IT&T Skills Taskforce survey (2000) identified broad business skills as one of the two most commonly cited deficiencies in ICT recruits, the other being insufficient experience. According to the 2000 Employer Satisfaction with Graduate Skills report (ACNeilsen Research Services 2000), ICT graduates have been described by employers as being deficient in three key areas: communication skills, team work and general business skills. However, this problem is not confined to the ICT industry, nor is it clear against what standards deficiency is being assessed. In the 2000 Employer Satisfaction with Graduate Skills report (ACNeilsen Research Services 2000), the most common skill deficiencies across all professions in terms of importance to employers were in the area of creativity and flair, oral business communications, and problem solving skills. So, the complaints made against ICT graduates seem to be the complaints made against graduates in general. While this survey provides a breakdown of major disciplines, it does not treat ICT graduates as a group.

Overseas studies into employer expectations of ICT graduates generally confirm the picture generated by those in this country (Dawson & Newsham 1997).

According to The good universities guide to universities, TAFE and private colleges in 2000 “by 1998, despite a lot of hype ever since about the industry crying out for staff, around one in seven new [ICT] graduates who look for a full time job fail to find one” (Ashenden & Milligan 1999, p. 270). That is, 14.3 % of graduates are unable to secure employment. The National Office for the Information Economy (NOIE) reports that in 1997, the unemployment rates for new Computer Science and Electronic/Computer Engineering graduates were 16.9% and 18.2% respectively. In contrast, in late 1998, the unemployment rate for the ICT profession as a whole (new graduates plus experienced ICT professionals) was 2.7% (National Office for the Information Economy 1999, January). These findings offer no simple interpretation. On the one hand they have to be related to other industry unemployment figures while, on the other, they could be suggesting that either ICT courses have not provided graduates with the required skills or the jobs just are not there for new graduates.

The graduate employment numbers are further complicated by the findings from the IT&T Skills Taskforce (2000) survey where employers anticipated demand for ICT professionals with between one and three years experience (more than three years for software engineers), rather than for newly graduated recruits. In a survey of ICT recruits who had graduated from ICT courses at the University of New South Wales and the University of Technology, Sydney, the Australian

11

Information Industry Association (1999) found that industry experience in the course of their degrees was rated highly among recruits as satisfactory preparation for their careers.

In the 10 years from the review of computing studies to the IT&T Skills Taskforce and DETYA studies, communication skills continue to be an issue. What this means is not resolved because, as noted above, graduates from most disciplines appear to suffer from this weakness. The employer may be expecting something which cannot be added to the skills base of the graduate if that graduate is to have the skills needed to function as computer professionals rather than as general business professionals with some computer skills. An important paragraph in the IT&T Skills Taskforce report (IT&T Skills Task Force 2000, p.4 para 1) points to the need for IT&T employees to have advanced level skills in the following areas:

• Technical knowledge

• Commercial understanding

• Developing or commercially applying technology

The report then points to a level of intermediate skills where the list is probably even harder to attain.

The expectations referred to in the IT&T Skills Taskforce report raise significant questions for the universities. A basic degree course covers three years and most ICT degrees are seen to be on the edge of being overloaded through the range of options which are being added to meet varying needs. In what way can the universities look to add commercial skills, plus the other skills identified in the Employer Satisfaction study from DETYA, and still provide graduates who meet the broad needs of the ICT industry, as well as providing the people who can provide research teaching and applied research and development?

2.3.2 The project and employer expectations The paucity of information on employer satisfaction with ICT graduates pointed to the need to implement a data collection strategy and the size of the task suggested that a quantitative survey would be a realistic starting point.

There was little point in repeating the type of data collection carried out for Employer Satisfaction with Graduate Skills (ACNeilsen Research Services 2000). As noted above, this report gives a set of response percentages. Even if some form of analysis is carried out between what is important to employers and the things they are satisfied with, we are unable to obtain an understanding of the impact these things have upon likely hiring practices and upon the likely communication needed between industry and universities. Consequently, an approach to exploring satisfaction and potential hiring was adopted which mirrors the regression-based modelling used in wider satisfaction studies.

2.4 Employer Data Collection The content of the survey ranged over most of the issues identified in the literature review. A major challenge was to keep the survey as short as possible,

12

knowing that long surveys reduce response rates. The final version was a compromise between content and size. The survey can be found in Appendix K.

The format of the content was designed to maximise the possibility of extracting structure from the responses. As was noted in Section 2.2, simple frequency counts have failed to produce data that would influence educational decision-making. The format of the survey used here assumed that links could be made between employers’ reported experiences and their anticipated future actions.

Sections of the ICT industry are complex and potentially unstable and it is important to obtain, not just a picture of what those who are currently employing ICT graduates say, but also we need to get information from those who do not currently employ graduates. The survey form was, therefore, designed to collect base data on all respondents and then specific data for employers and for non-employers. In this way, it was a modular survey form that was effectively two surveys – one targeting those businesses that currently employ recent ICT graduates and the other targeting those that do not – with a common section asking for demographical information.

While the survey was primarily quantitative, both sections of the survey asked for a limited amount of qualitative data. Respondents were also asked to indicate if they were willing to take part in follow up interviews. This data is reported in Section 2.4.6.

2.4.1 Survey respondents The task of a nation wide sampling of industries that employ ICT graduates was quite difficult. On the one hand, we could assume that a general mailing list would cover a reasonable percentage of graduates because most larger organisations appear to employ graduates from the computing courses for their IT sections. But, on the other hand, this would exclude many specialist IT organisations, not to mention Libraries and Multimedia companies who definitely employ graduates from the broader ICT area.

Two quite different strategies were adopted:

• Membership mailing lists were used, particularly from the Australia Computer Society, the Australian Interactive Multimedia Industry Association and the Australian Library and Information Association.

• The Australia-wide Yellow Pages database was searched under various keywords and a list of companies and addresses was created. The keywords were:

§ Business Systems Consultants

§ Multimedia Services

§ Internet Access Providers

§ Internet Web Services

§ Computers--Technical Support

§ Computer Systems Consultants

§ Tele-communications Consultants

§ Management Information Services

13

Through the combined approaches, 3673 surveys were distributed to valid addresses. The overall number of useable responses was 523, giving a response rate of 14%. This is below the anticipated level of around 25% but current experience on other survey-based projects suggests that a response rate of around 15% is what can be expected with an unsolicited survey that has no payoff (eg. a survey response prize).

A small percentage (8%) of the data collection came back via e-mail - both survey forms were made available on the project Web site. It is interesting that ICT staff seemed to be more comfortable filling out and returning via snail-mail than going to a Web site and posting an electronic return. This supports a general impression from other research projects that Web-based surveys are yet to be seen as the preferred response mode.

2.4.1.1 Graduates employed

Many of the organisations that responded to the survey currently employed recent graduates from a number of sub-disciplines within the major discipline of ICT. Table 2 shows the potential complexity of this breakdown. Consequently, most of the analysis uses the overall ICT discipline as its base for describing graduates.

Table 2. Counts by course employed

Graduate type employed* Number of respondents employing

only one graduate

type

each graduate

type

CSSE 29 148

DS 0 29

BIS 17 125

Multimedia 5 55

LIS 18 48

*CSSE = Computer Science and Software Engineering; DS = Distributed Systems; BIS = Business Information Systems, LIS = Library and Information Systems

2.4.2 Analysis and results In the following sub-sections, the analysis of the data will be reported as non-technically as is possible. The data was subjected to extensive analyses; Appendix F contains tabular output covering most of the items on the survey as well as inter-correlations, analyses of variance and other appropriate analyses.

For the sake of simplicity, we will call the group that employs recent graduates Do Employ (N=213) and the other Don’t employ (N=308).

2.4.2.1 Demographics

2.4.2.1.1 Do Employ

14

The respondents who do employ recent graduates can be characterised in the following way

• Half were from organisation with 100 or more employees but only eight percent were from micro organisations (6 or less employees)

• One third classified their business in the Other industry category which, from the written in specification, generally meant they were in ICT. Education and Government each had about half of the remaining two-thirds, while the rest were spread across major industry categories.

• 85% of respondents had a degree or higher degree.

• Their geographic spread was fairly consistent with Australian demographics with 83% in capital cities.

2.4.2.1.2 Don’t Employ

The characteristics for this group were:

• Just over half were micro-organisations with a further quarter in the 6-20 staff category

• Very similar to the Do Employ respondents on industry type

• Only 62% had a degree but a higher proportion had Certificate/diploma level qualifications

• Their geographic distribution was reasonably good although there was a disproportionate number from Western Australia and the regional areas were probably over-represented.

2.4.2.2 Hiring Practices

The Don’t Employ group is much less likely to hire any graduates from any type of training.

To clarify this pattern we removed all of the micro-business from the Don’t Employ analysis. The results showed that the larger organisation Don’t employ group is still unlikely to hire from TAFE and Private Providers but has a pattern more like the Do employ for employing University graduates. This effect is more pronounced when we look at the two largest organisation sizes for the Don’t employ.

2.4.2.3 Contact with University Departments

Table 3. Contact with Universities (Q.1.7) (Percentages) Purpose of contact

Do employ

Do not employ

Industrial Experience 51 8

Committees 25 6

R&D 25 7

Consulting 28 10

15

The data in Table 3 shows that the Do Employ group has a markedly higher level of contact with universities than does the Do Not Employ group. The meaning relative level of the Do Employ contact is hard to determine, as there is no comparative data from other discipline areas.

2.4.3 Comment on Do and Don’t The information in section 2.4.2.2 raises an interesting point in relation to questions about the employment market for ICT graduates. Australian industry is dominated by small business and there is no reason to doubt that this is true in the ICT area, including Information Management. The availability of jobs in this environment will be restricted and very susceptible to economic and market fluctuations. This makes it difficult to make adequate projections of job vacancies.

2.4.4 The Do Employ Data The general satisfaction with graduates (see Appendix K Q2.4) was above the mid-point on the five-point scale, and this difference from the mid-point was statistically significant. Employers are satisfied with their ICT graduates.

Table 4. Descriptives for satisfaction measures

Satisfaction measure

N Mean SD

Overall Satisfaction 196 3.87 .80

Compared Satisfaction 100 3.68 .96

If we look at the block of tasks and skills in Q2.3 (each rated in terms of meeting organisations needs), we see that only Understanding of business processes and LOTE (Skills in Language other than English) are not significantly different from the mid-point on the scale. The important deviation from the general trend is that Project management skills has a statistically significant difference below the mid-point indicating that the mean trend for the sample is to say that graduates do not meet their needs in this skill area.

Table 5. Meets needs ratings

Skill/knowledge

N Mean SD t

Skills in particular computer languages 171 3.71 .84 10.9*

Skills in particular software applications 179 3.55 .91 8.0*

Foundation in theoretical principles 160 3.73 .85 10.8*

Ability to communicate with clients 192 3.42 .95 6.1*

Written communication skills 194 3.28 .95 4.1*

16

Skill/knowledge

N Mean SD t

Ability to work in teams 194 3.75 .88 11.8*

Project management skills 182 2.86 .97 -2.0*

Problem solving skills 195 3.70 .85 11.4*

Understanding of business processes 182 2.90 .94 -1.4

Preparedness for quick entry into job tasks 190 3.41 1.03 5.4*

Skills in languages other than English 96 3.22 1.35 1.6

Note: The one sample t-test is measured against the mid-scale value of 3. * p<=0.05

This, quite clearly, gives a very limited view of the nature of the industry response to this set of skills. The mean pattern is that most are above the centre point of the scale or at the centre point. Only two are below.

Is Project management skills the sole source of skill deficit from ICT courses?

To answer this we looked at regression modelling using both satisfaction and the outcome variables as dependent variables.

2.4.4.1 Satisfaction and Meets Needs

The regression of the 11 skills on Overall Satisfaction produced quite a clear result with an R2 of 0.56 and an overall fit, which was statistically significant. There were four skills with statistically significant Beta Coefficients:

• Skills in particular software applications

• Ability to communicate with clients

• Problem solving skills

• Preparedness for quick entry into job tasks

It is interesting to note that Project management skills did not produce a significant impact on satisfaction even though it appeared to stand out in the mean trends (Table 5). If we were interpreting the descriptive statistics in Table 5, we would be inclined to say that Project management skills and Understanding of business processes would be clear deficiencies in the graduates being recruited. The regression modelling says that, if these are deficiencies then they have no relationship to the perceived satisfaction with the performance of graduates.

If we also look at the relationship between the needs being met skills and Continue to recruit, we find that there is no statistically acceptable relationship between the low rating skills and the outcome variable (Continue to recruit). That is, the ratings of the skills in terms of needs being met do not have any impact on the rating of the likelihood of continuing to recruit graduates.

It is informative to note that 64% of the respondents said that it is highly likely that they will continue to recruit graduates.

2.4.4.2 Satisfaction and Importance versus Needs met

As was pointed out in the initial discussion about previous research, studies on satisfaction often utilise a measure of the gap between the importance of a skill or

17

attribute and the extent to which that skills or attribute is satisfactorily performed. This information is seen to be important because it tells us what is inadequate when the gap is large. The data collected here allowed for this type of analysis.

In order to carry out the analysis the Meets Need Rating was subtracted from the Importance rating. If the result was negative (an unimportant skill was meeting organisational needs) it was coded as zero. Consequently, the data ranged from zero (no gap) through to four (maximum gap).

From Table 6 it can be seen that the proportion of cases where there was a gap varied considerable over items, with Ability to communicate with clients, Written communication skills and Problem solving skills having the highest proportion.

Table 6. Descriptives for 'gap' between Importance and Meet needs

Skill/knowledge N Min Max Mean

Gap SD

Skills in particular computer languages 69 1 3 1.34 0.60

Skills in particular software applications

63 1 3 1.38 0.63

Foundation in theoretical principles 68 1 3 1.36 0.54

Ability to communicate with clients 141 1 3 1.68 0.74

Written communication skills 134 1 3 1.69 0.77

Ability to work in teams 120 1 3 1.40 0.64

Project management skills 108 1 3 1.57 0.70

Problem solving skills 135 1 3 1.47 0.66

Understanding of business processes 108 1 3 1.66 0.70

Preparedness for quick entry into job tasks

93 1 3 1.48 0.70

Skills in languages other than English 10 1 2 1.10 0.32

The gap data was not consistent across items within individual respondents. Consequently, continued analysis of the data was difficult but it was decided that there was value in trying to establish relationships between the gap data and the outcome variables. When the items with the largest gap proportions (Ability to communicate with clients down to Preparedness for quick entry into job tasks) in Table 6 were regressed on Overall satisfaction and Continue to Recruit a viable regression model was not produced.

The means in Table 6 suggest that the two communication skills questions’ proportions (Ability to communicate with clients and Written communication skills) produce clear gap. If we focus on those gaps we would have to say that Universities are failing to provide students with effective communication skills and this is important to industry. Within human resource thinking, satisfaction with employee performance is at the core of future employment thinking and, in this case, satisfaction has no relationship to the communications questions.

18

2.4.4.3 Satisfaction and outcome variables

If we are looking at the way in which the current state of university courses impact upon industry then we have to assume that if industry is not happy it will look elsewhere. The five variables that can give us some indicators about this are:

Overall satisfaction Recommend that people seeking a career in ICT do an ICT degree Recommend your ICT graduates to other employers The likelihood that your organisation will continue to employ ICT graduates Responsiveness of Universities

The intercorrelations between these variables are given in Table 6. It can be seen that all are statistically significantly intercorrelated, although the value of the Recommend Uni by Overall Sat is too small to be of practical significance.

19

Table 7. Correlations between outcome variables

Outcome variables

Cont. using

Rec. Grads

Rec. Uni

Overall Sat

Resp.

Continue using 1.00

Recommend Grads .379* 1.00

Recommend Uni .457* .432* 1.00

Overall Sat .409* .431* .153* 1.00

Responsiveness .285* .319* .318* .453* 1.00

Of interest is the Relatively low level of relationship between Continuing to employ and Overall sat. Intuitively, these should be more strongly related because if you are satisfied then you would be expected to continue recruiting graduates. The reality will be that market forces are determining the rating given for likelihood to be continuing to recruit, whereas Satisfaction has no such contaminating effect.

Responsiveness has a reasonably strong relationship with Overall Satisfaction. This is not unexpected and it is of interest to note the level of participation with universities as shown in Table 3.

2.4.5 Results from Graduate Uptake Groupings The survey asked the respondents to indicate which types of graduates they had recently recruited. This question allowed multiple responses.

What is of importance is that most organisations were taking graduates from more than one area. The range of graduates being employed was such that it was not possible to unambiguously analyse industry by type of graduate uptake.

The larger organisations, the organisations that appear to be taking ICT graduates, clearly have a range of needs that ICT graduates can satisfy. The medium sized organisations that responded are in a similar position. This suggests that, if ICT faculties, schools and departments are to understand the scope of organisational needs for graduates more data is needed.

2.4.6 The open-ended questions

2.4.6.1 ‘Why doesn’t your organisation currently employ ICT graduates?

Respondents whose businesses do not currently employ recent ICT graduates were asked to explain why. The majority of respondents (305 out of the 313 that did not employ) provided an explanation. Reasons for not currently employing ICT graduates were coded into seven broad categories:

• Not required

• Prefer employees with other qualifications

20

• Business too small

• Need experienced people

• Outsource ICT work

• Cannot afford university graduates

• Cannot access ICT graduates

These response categories are described below.

2.4.6.1.1 Not required

One hundred and seven respondents (35% of those who commented) who do not currently employ ICT graduates wrote that it was because they did not require them. Many of the responses in this category simply said there was “No need.” This is not the primary reason why respondents’ businesses do not employ graduates, with such responses indicate other reasons why they do not require graduates. Some respondents gave more detailed responses that explained why they did not require graduates. Two common explanations were that they had not had recent vacancies, or that they had no need for ICT staff with the skill level provided by a university degree.

There was significant overlap between this and other categories. For example, many said their businesses did not require ICT graduates because they were small companies or because they preferred employees with other qualifications that were more suited to the skill base required.

2.4.6.1.2 Prefer other qualifications

Eighteen respondents (6% of those who commented) said they do not currently employ university ICT graduates because they prefer people with other types of qualifications. This appeared to relate to their need for particular skills and for employees with ready-to-apply skills.

Comments were also made about skills learnt at university becoming quickly outdated. This again indicates that there is a segment of the ICT industry that is interested in up-to-the-minute skills rather than the broad grounding and professional skills that universities seek to provide their students with.

University ICT educators’ views are discussed in Section 4 and indicate that universities would respond to these types of comments by saying that they ‘miss the point’, and that universities are not targeting the types of vacancies that require specific up-to-the-minute technical skills.

2.4.6.1.3 Business too small

Just over half of the respondents who do not currently employ ICT graduates were from micro organisations (fewer than 6 employees). Seventy-six respondents (25% of those who commented) who do not currently employ ICT graduates wrote that it was because their businesses were too small. Again, the common response of “Too small” is not a reason for not employing graduates, but points to reasons related to being small. Those respondents who gave more detailed answers explained that, due to the size of their businesses, they only employed experienced staff, they outsourced their ICT work, they could not afford to pay for a university

21

graduate, or they simply were not planning for growth. Examples of these types of responses are given below under the appropriate category.

2.4.6.1.4 Need experienced people

Forty-four respondents (14% of those who commented) who do not currently employ ICT graduates wrote that it was because they needed or preferred to employ people with experience. This was often related to the size of the business. Micro businesses do not have the resources to train graduates, they do not have the ability to provide a new graduate with a career path and they are often not willing to take on the risk associated with an inexperienced employee.

2.4.6.1.5 Other categories

Twenty-three respondents (8% of those who commented) said they do not currently employ ICT graduates because they outsource their ICT work either to consulting or service businesses or, in the case of large businesses, to other departments or divisions.

Twenty-two respondents (7% of those who commented) said that their businesses did not employ ICT graduates because they could not afford them. In some cases, this related to the size of the business: as mentioned above, respondents from micro businesses indicated that they were not able to meet the time and resource investment required to support a graduate. Other comments referred to an inability to pay the salaries required by university ICT graduates.

A small group of respondents (4% of those who commented) said their businesses did not currently employ ICT graduates because they could not access or attract them. Respondents from rural or remote areas saw their location as a barrier to attracting graduates.

2.4.6.2 ‘What do you think Australian universities could do to improve the preparation of graduates for work in your industry?’

Respondents whose businesses currently employ ICT graduates were asked what universities could do to improve the preparation of graduates for work in their industry. The majority of respondents (147 out of the 213 that employed ICT graduates) provided a suggestion or a number of suggestions. The analysis of these suggestions is presented in two sections. First, common themes in the advice given are described. Then, two issues around which a diverse range of responses were focused are described.

2.4.6.2.1 Advice to ICT departments

Suggestions were coded into six broad categories:

• Work experience

• Industry consultation

• Industry awareness

• Generic skills

• Business skills/knowledge

22

• Technical skills

Each category is described below.

2.4.6.2.1.1 Work experience

Forty-four respondents (30% of those who commented) who employ ICT graduates said that universities should provide their ICT graduates with more work experience. Comments included:

“Work experience is the best training.”

“More double degrees, more sandwich courses, more industry

“Placed (managed) work experience of at least 1 semester allows the student to contribute to the organisation and gives the student more than just a superficial insight into the industry.”

Related to experience in real workplaces is experience of authentic projects:

“Continue efforts for work place placements & ‘real world’

“Ensure that students do at least 1 in-depth real world study which goes from start to finish.”

“Real life projects. Broad non-computing electives.”

2.4.6.2.1.2 Industry consultation

Twenty-five respondents (17% of those who commented) who employ ICT graduates recommended university ICT departments consult more with employers. They suggested that communication channels be established so that employers can communicate their needs, that partnerships be developed for the provision of work experience and project work to students, that lecturers become more familiar with what is going on in industry, that industry play a greater role in informing course structure and curriculum and that universities use industry lecturers. Typical comments included:

“More consultation with industry. Better knowledge of

“Greater involvement of industry in university courses and curriculum committees.”

2.4.6.2.1.3 Industry awareness

Twenty-one respondents (14% of those who commented) suggested that universities focus on developing students’ understanding and expectations of the ICT industry. Many implied that students have misconceptions about the nature of entry-level graduate work and the career opportunities available.

Section 3 reports the career paths of graduates and their views on their courses. The comments made by employers about graduates’ expectations of the industry are consistent with observations made in this section.

2.4.6.2.1.4 Generic skills

Twenty-one respondents (14% of those who commented) in this group suggested that universities pay more attention to the development of generic skills in graduates. Skills mentioned included interpersonal communication, team work,

23

customer skills, verbal and written communication and problem solving. Comments included:

“Give a stronger emphasis to key competencies, such as interpersonal communication & teamwork, etc.”

“Communication skills – being able to articulate & also listen. Customer skills – how to deal with difficult customers & keep them informed and happy.”

“A greater emphasis on generic skills such as problem solving and

“Train them in people skills and ensure they have a good grounding

This category is reflected in the response pattern in the quantitative section where communications were seen as an area of deficit in graduates. As was pointed out, this does not actually impact on satisfaction with performance, raising questions about its actual importance in the workplace. But more than this, we are still faced with the problem of what the ICT curriculum should contain and how far universities can expand course content to meet needs being expressed by industry.

2.4.6.2.1.5 Business knowledge/skills

Eighteen respondents (7% of those who commented wrote that ICT graduates need to be better versed in business processes.

“One cannot stress too highly the importance of understanding business processes and the requirement for excellent communication skills.”

The comments made on the previous category also apply here.

2.4.6.2.1.6 Technical skills

Thirteen respondents (9% of those who commented) who currently employ ICT graduates suggested that ICT courses give more emphasis to technical skills, with some naming particular skill areas.

2.4.6.2.2 Issues

2.4.6.2.2.1 Theory vs. application vs. principles

Sixteen respondents who currently employ ICT graduates mentioned the relative value of theory versus applications versus principles. A range of different views was evident. Some suggested that university ICT courses contain too much theory:

“Let them know that theory doesn’t equal reality.”

“Less theory; its people that count not technology.”

“Less theory: more pragmatic.”

Others stressed the importance of “the basics” and of theoretical principles:

“Don’t neglect the basics. A surprising number of graduates interviewed weren’t familiar with the systems development life

24

“Understand the fundamentals that can be applied to any language & not just learn how to code in a particular language.”

Still others suggested that course content be more practical:

“Focus on practicalities – Groundings in computer theory are good from a historical context but doesn’t help me if they can’t configure

“More practical.”

“Courses can sometimes be too theoretical & not concentrate on the

Many comments alluded to an appropriate mix of theory and its practical application. As suggested in earlier sections, this difference of opinion may be directly linked to the different needs of different types of businesses.

2.4.6.2.2.2 Old vs. new vs. emerging technologies

A similar difference of opinion was observed among comments concerning the types of languages and applications that should be taught or used to teach principles. Fourteen respondents variously advocated the teaching of old established technologies, current technologies or new emerging technologies. Comments included:

“There tends to be a negative attitude towards COBOL and Legacy systems. The fact is we still depend on these. COBOL will be around for a long time yet.”

“Train graduates for existing platforms, not future platforms. Although html and java are exciting & interesting, the majority of enterprise systems and business are in SAP or similar products.”

“Train them in the latest programming languages & software.”

One respondent more philosophically commented:

“This will always be difficult – find the right balance between current practical skills, and future methods and thinking.”

2.4.7 Research Conclusions The relatively small sample size from across such a diverse industry area could create some difficulties in generalising from the results. In fact, the response patterns are reasonably tight (see the standard deviations in Table 5), allowing us to assume that we have some consistency in the response sample and to be able to suggest directions.

2.4.7.1 Job Market and Job Skills

The differences between the Do employ and Don't employ groups raises interesting questions about the state of the job market in ICT. The preponderance of small to micro businesses in the sector may be a key issue in questions about the over-supply-under-supply saga evident in the literature. These businesses are least likely to employ graduates. Projecting the job market in ICT would require an understanding of factors such as the job life span of staff in such organisations as well as the likely expansion rates in job numbers.

25

In addition, it is worthwhile noting that the current Do Not employ group do not show a promising pattern (Table 8) of responses to the question on future likelihood to employ. The pattern is towards uncertainty.

Table 8 Anticipated future recruitment of graduates - Do Not Currently employ

Anticipated future recruitment

Frequency Percent Cumulative Percent

Definitely will not 44 14.4 15.6 2 53 17.4 34.4 3 83 27.2 63.8 4 48 15.7 80.9 Definitely will 22 7.2 88.7 DK 32 10.5 100.0 Total 282 92.5 Missing 23 7.5

There appears to be good reason for more labour market analysis if Universities, and TAFE, are to have a clearer understanding of the career environment they are feeding into.

2.4.7.2 Are we dealing with an inevitable outcome?

We have noted that, underlying all the analysis and interpretation, there is the basic fact that 64% of Do employ respondents said it was highly likely that they would continue to employ graduates.

There seems to be an inevitable market for ICT graduates, which is partially independent of what industry thinks Universities should be doing to produce graduates. Exactly how many graduates will be employed in the future is another question.

2.4.7.3 Squeaky Wheels and significant issues

A conclusion that is clearly demonstrated by the data presented here is that employers are generally satisfied with the graduates they recruit. They see graduates as having a number of obvious deficiencies but that these are irritants, or Squeaky Wheels, rather than being significant decision-making factors. And these irritants have been reported in different papers over the last 10-years.

An apparently key issue, lack of communication skills, does not appear to impact upon satisfaction or outcome variables. It has one of the largest discrepancy scores between Importance of skills and Needs being Met, but it did not appear as a significant contributor to satisfaction under these circumstances.

General customer satisfaction research has shown the importance of distinguishing between what is important, either to be maintained and to be changed, versus what is talked about and is an easy topic to grumble about.

We have isolated some possible significant skill-based factors which influence the perception industry has of the university sector's capacity to produce job-ready graduates.

26

2.4.7.4 Data Collection issues

The results from the comparison of the regression modelling and the mean trends for the skill data, combined with the suggested shortcomings of the DETYA employer satisfaction study (ACNeilsen Research Services 2000), point to the importance of building research models which are sufficiently sophisticated to allow analyses of the structure of behaviour. If the industry data had been based upon interviews or focus groups, it is unlikely we would have been able to see the distinct ways in which the skills interact, or to make a distinction between those that are important (in satisfaction terms) and those that are not.

2.4.7.5 Flexible careers versus job-ready

The ICT sector is subject to the vagaries of changing technologies, ideas and, to some extent, fashion. The speed at which the e-bubble was inflated and then deflated strongly suggests that university ICT departments should have depth in their degree structures, which will allow them to readily shift direction as new waves pass through. Focusing on skills training would lead to medium term staffing problems for industry. It would also lead to broad human resource problems as changing technology makes redundant those who have not been educated for adaptation and change.

2.5 University responses to stated needs from industry There are various examples of how the higher education systems in other countries have tried to respond to stated needs from industry.

The most extensive example is the development of core competencies for computing within the Quality Assurance system in the United Kingdom. From the lack of published reports both in the professional literature and from the Quality Assurance Agency for Higher Education, little appears to have been done with the ICT component of this system to date.

Using skills and competencies thinking, the University of Luton developed a set of transferable skills that all graduates could be assumed to have achieved by the end of their course (Atlay & Harris 2000). The project was begun in 1996 and students have graduated from the program but no substantial conclusions have yet been drawn.

The reported activities in the US have tended to be about post-university training programs such as the University of Oregon Masters of Software Engineering (Faulk 2000) that was designed in close consultation with industry in that state. Carnegie Mellon University offers a one-year Master of Software Engineering program focusing on broad-based problem solving skills and the development of leadership potential (Garlan, Gluch & Tomayko 1997).

Successful corporate-style collaborations have been reported by Beckman and her colleagues (Beckman et al. 2000; Beckman et al. 1997) who surveyed 14 collaborating universities and their industry partners. When asked to identify the most important reason for collaborating, half of the industry partners cited fulfilment of their organisation’s education mission.

27

Skinner and Cartwright stated that the most successful collaborations between higher education and the ICT industry “tend to be sustained, multiple-year programs of cooperation, not short-term projects” (Skinner & Cartwright 1998, p.55).

Much of what has been found in the literature appears to assume that university degrees should be responding to what industry says it needs rather than there being a realistic, future-oriented approach to developing the university-industry interface.

At some level we seem to have lost some of the understanding of the issues surrounding lifelong learning and career change. During the last two decades of the 20th century, emphasis was placed on the need to prepare people for change. The content and structure of careers was seen to be changing at such a rapid rate that the average white-collar employee would have at least five significant career changes or shifts in his/her lifetime.

The VET sector, in most states, is driving various initiatives to support lifelong learning and socially relevant learning environments, and this is supported by industry groups. The University sector can argue that this has been the underlying motivation for university education for centuries. The continuing pressure to narrow the focus of higher education seems an inappropriate pressure within an commercial and industrial environment where flexibility and adaptation are key issues.

2.6 Conclusions The relationship between ICT schools and the ICT industry is not defined by clear, general parameters. Universities have their view of what they are doing and believe that they are quite responsive to the changes in knowledge and technology. Industry believes that students are not well prepared for jobs and that universities are not giving students the essential skills to fit them for rapid job entry.

It almost seems reasonable to argue that while there is a debate going on between industry and the universities then there will be awareness of the essential questions about preparing and training students versus giving them a broader than needed education to make them, in the longer term, more adaptable.

Some aspects of this discussion are made more difficult by the apparent confusion over the state of the ICT job market, particularly on the computing/multimedia side rather than Information Systems. The common wisdom is that there is a shortage of graduates, yet about 14% of graduates are not getting jobs.

Various surveys on first year undergraduates in the Computer Science and Software Engineering courses at Monash included a question on the reasons the students entered the course. The responses indicated a lack of career direction. This is not unusual in many degree programs but it does add an additional motivational question to the others we have generated.

If there is a clearly definable skills deficiency and if this impacts upon graduate employability, then there is a clear need for some a dialogue between industry, universities and the TAFE sector to determine an appropriate model for the career development of ICT professionals. The IT&T Skills Hub is assumed to be working on this but it is becoming difficult to find out what is happening due to a

28

combination of a closed Web site and expensive reports1. The need is not for more surveys of so-called industry needs, rather it is for:

• a clearer picture of the nature of the ICT industry structure

• an understanding of what is critical to industries' graduate needs

• an understanding of industries' commitment to career structures for ICT professionals

• and the willingness of industry to support post-graduate specialist training which will reflect the changing nature of the industry and its technology

2.7 Recommendations arising from this section

• Recommendation 1 That universities’ ICT departments be given support in establishing links with industry. This will enable them to explore real-world options on how the university-industry interface can be made permeable enough to help meet industry needs without compromising the broader educational needs of graduates. In conjunction with this, that exploration be carried out on the development of post-graduate, skills-based courses which reflect changes in technology and the industry.

• Recommendation 3 That the research and development currently being undertaken by CERG members on student satisfaction form the basis of broader work on the development of user-oriented tools and methods for interpreting and diagnosing graduate and employer satisfaction.

• Recommendation 5 That university ICT departments, in collaboration with industry, devise programs for the better management of students’ expectations of professional life.

• Recommendation 6 That further ongoing research be initiated, within the context of labour market models, to determine the structural differences between those organisations who are likely to employ ICT graduates and those who are unlikely to employ them. This research would ideally be based on collaboration between universities, government and industry and aim to provide up-to-date information to university curriculum developers about industry needs. In conjunction with this, research is needed into the influence of company size and the consequences of this for the size of the market and the skills and knowledge needed.

1 The project does thank the IT&T Skills Hub for forwarding, gratis, a copy of the Executive Summary and major data tables.

29

3 Graduates in the workforce

But it’s even more fundamental than that because a lot of the students don’t appreciate that some of the fundamentals are essential to the fluffy things that they want to do, like databases – “Who wants to study data modelling?” You know. “I just want to learn how to write a web page”. Well I’ve got news for you pal, take a job at the Age, work on their breaking news site, which is run by one of our ex honours students, and you wont write a bit of HTML, it’s all done by back end database and ASP and things like that. … a lot of them wanting to get up the you beaut latest and brightest stuff don’t know what they mean. (Mini-conference participant)

3.1 Introduction This section takes an alternative look at the way in which university ICT courses prepare students for the workforce with special reference to the question of entrepreneurial behaviour. The impetus for this part of the project came from members of the National Steering Committee who pointed to the promotion of entrepreneurial behaviour as a continuing concern in many courses. The Committee also suggested that graduate outcomes might most usefully be studied through an in-depth investigation into a small number of ICT graduates. The Reference Group recognised that these two aspects of graduate behaviour – entrepreneurial behaviour and graduate outcomes – could be combined in an investigation of graduates’ perceptions of the influence of their courses on their work and career paths. Interviews were conducted, targeting graduates working in specific types of graduate destinations, was used to investigate the complex of factors surrounding graduate behaviour.

3.1.1 Literature on Graduate evaluation of course There is a paucity of material in the educational and management literature on graduate evaluation of the course against their occupational outcomes. A search of the relevant databases produced only marginal references.

The Australian system has produced the Course Experience Questionnaire (CEQ) that has generated local information. The CEQ is discussed below.

3.1.2 Student evaluation of university education: Subjects, teaching and courses. As the key consumers or, depending on your view, products of the Australian university system, students provide perhaps an important source of information on the evaluation of institutional performance. Retrospective evaluations by

30

graduates have been initiated through the Course Experience Questionnaire (CEQ).

A large body of literature exists regarding the development and use of the Course Experience Questionnaire (CEQ; see McInnis et al. 2001). A brief overview of issues pertaining to the present project will be presented here.

Delivered in conjunction with the Graduate Destinations Questionnaire (GDQ), the CEQ is an annual survey administered, under the auspices of the Graduate Careers Council of Australia, to all graduates of Australian university programs. It represents “a national performance indicator of the quality of teaching and is the major source of comparative data on student satisfaction with the overall course experience” (McInnis et al. 2001, p. 4).

The original purpose of the instrument was to provide individual institutions with information concerning those areas in which their teaching and learning could be improved, at the course level, based up recent graduates’ responses to a 25-item survey. The questionnaire focuses on aspects of teaching and learning that are common across a wide variety of courses and areas of study. Necessarily it excludes important factors that will impact upon graduate perceptions of and satisfaction with their course, most notably, those factors specific to particular fields of study. For example, factors associated with the completion of practical and industry experience periods which are core components of specialist programs such as medicine, nursing, engineering, tourism and hospitality and social work and optional in a wide range of degree programs including law and ICT. To some extent, the CEQ taps areas such as the availability of teaching and learning support services but does not inquire about the delivery and the quality of pastoral care. Furthermore, the CEQ does not take into account differences that may impact upon the experience of tertiary education, for example, reasons for participating in higher education, disability, ethnicity and cultural factors. These factors may also influence whether and how graduates respond to the instrument.

The CEQ makes no claims of being a definitive measure of student course experiences nor does it claim to be a perfect instrument. However, the data collected via this questionnaire is often used as if it were. This indicates that the CEQ is being overused and being extended upon the behavioural boundaries for which it was intended.

Within the context of graduate satisfaction, a survey instrument that is administered within 6 months of graduation must be interpreted with caution. For example, within this period graduates can be roughly categorised into three groups:

• I got the job I wanted

• I got a job

• I haven't got a job

Career destination is not determined by being in the first category within the first 6 months of leaving university. Career counselling and placement practice tells us that graduates may require more time than this to get the job they want and that during the period they are trying they can become very frustrated and dissatisfied.

31

The issue here has nothing to do with graduate satisfaction as a viable indicator2. Rather it is about finding ways of assessing satisfaction that are realistic and practicable. Clearly, the CEQ may have significant problems in doing this.

3.1.3 Alternative approaches to evaluation Elliot and Hughes (1999) utilised an ‘integrated’ approach to student satisfaction in their work with the AIIA Career Tracking Research project. The authors measured graduates’ satisfaction with their ICT courses in terms of how well they prepared them for industry. Individuals who graduated from Bachelor of IT courses at two universities in the years 1992-1997 were asked whether, in relation to their first industry position, they thought that their degree was relevant preparation. The majority of these individuals rated their degree as mostly or completely relevant. This approach makes sense because satisfaction with a product cannot be gauged until the product (in this case, a tertiary degree) has been used. Unlike the CEQ, which takes a purely retrospective approach to student satisfaction with university education, this study went some way toward linking, or integrating, course experience with post-university life and employment preparedness.

A more highly integrated approach is suggested by Malley (1998) in his model for investigating student satisfaction in the Australian vocational education and training (VET) sector. Malley sees student satisfaction as an analogue of customer satisfaction. He proposes a model of satisfaction comprising three domains: the antecedent domain, the transactional domain and the outcomes domain. The antecedent domain includes the needs, wants expectations and prior experience that the student brings with them to the education experience. The transactional domain encompasses factors related to the transactional relationship between the student and their education experience including the match (or mismatch) between the student needs and expectations and the ‘product’ delivered by the institution. Finally, the outcomes domain represents the integration of factors from the antecedent and transactional domains with the post-university ‘out-side’ world. The value of this and other approaches to educational evaluation is discussed in Section 7.

3.2 Graduate interviews As discussed above, there is a dearth of research into ICT graduate outcomes. The data routinely collected in Australia on graduate destinations is limited and is not designed to investigate issues of particular interest ICT departments. As a first step to remedying this gap, interviews were used to identify issues salient to recent ICT graduates. These interviews could be targeted by further research. The methods used are summarised below and the main themes arising from the analysis are described. Accounts of each graduate’s story as relayed in interviews can be found in Appendix L.

2 In looking at the CEQ, we have to distinguish between those like Koder who believe that satisfaction measures are intrinsically worthless (in Hand & Trembath 1999, p. 12) and the failure of the instrument to do what it is supposed to do. It is very poor logic to suggest that a bad measurement tool eliminates a behavioural approach.

32

3.2.1 Methods The approach taken here was intended to identify those factors that ICT graduates view as having affected their ability to find employment and their preparedness for work. Semi-structured conversational interviews (Patton 1990) were used to collect graduates’ accounts of their career histories. Open-ended prompts were used to elicit participants’ stories about background factors, such as family, schooling, university education and work history, and to encourage them to reflect on the role that these factors played in their careers. Prompts included instructions such as, “So you are a Software Developer in a small business, tell me the story of how you got to this point”. Follow-up telephone calls or emails were used to clarify details or to fill in any gaps where necessary. The methods used in this section of the research are described in more detail in Appendix G.

3.2.1.1 Participants

Graduates were identified using the Project team’s own networks. Efforts were made to maximise variation among graduates: the criteria for variation were graduate destination and graduate type. Graduate destination varied according to the size of the business for which the graduate worked, ranging from micro businesses (with six or fewer employees, including sole traders) to large businesses (with one-hundred or more people employees). Graduate status varied according to discipline of study (computer science and software engineering, information systems, or electrical engineering). Comparative data were also collected from a small number of non-graduates and a small number of graduates from non-ICT disciplines. These individuals were employed in the graduate destinations identified above.

Graduate type Participants were eleven men and two women aged between 21 and 28 who work in the ICT industry. Of the thirteen, eight had studied Computer Science and/or Software Engineering (CSSE), two had studied Information Systems (IS), one had studied Electrical Engineering, one had studied Biological Science and one had no tertiary qualification. Four of the male participants had commenced, but not completed their degrees in Software Engineering, Information Systems and Electrical Engineering. Three of these participants do not intend completing their degrees in the near future. Of the CSSE graduates, four had commenced degrees in other disciplines, for example, engineering and digital systems, before transferring to computing degrees.

Graduate Destination Seven of the participants are employed by large organisations (100+ employees) as analysts, sales professionals, programmers and systems engineers. One works as in a small business as a software developer. The remaining three work for micro organisations (< 5 employers) or as sole-traders contracting to larger companies. Table 9 maps those graduates interviewed according to graduate destination and graduate type.

While the participants represent a convenience sample, their training covers a broad range of levels and specialisations. For example, levels of training

33

represented include: no formal training, industry certificates, diplomas, pass degrees and honours degrees. Areas of specialisation represented include: software testing and quality assurance, programming, payroll systems administration, web development, software development, pre-sales engineering, database administration and financial/business analyst. Although Monash networks were used to identify graduates, interviewees had studied ICT-based degrees in five Melbourne-based universities representing a range of university types.

Table 9. Summary of interviewees by graduate destination and graduate type combinations

Graduate type

Micro (<6) Small-Medium

(6-20 /

21-99)

Large (100+)

CS/SE 2 1 5

IS/BS - - 2

Elec Eng 1 - -

Non-ICT grad 1 - -

Non grad 1 - -

CS/SE = Computer Science and/or Software Engineering; IS/BS = Information Systems/Business Systems; Elect Eng = Electrical or Electronic Engineering; Non-ICT grad = a degree from a non-ICT discipline; Non grad = no formal tertiary qualification

3.2.1.2 Analysis

Narrative accounts of each participant’s career history were constructed. These accounts were then returned to participants to be approved. Participants were free to made corrections and to add or delete sections. Common themes were identified across graduates’ stories using the constant comparative method (Silverman 2000; Strauss & Corbin 1997). The themes that emerged are described below.

3.2.2 Findings Common themes that emerged across graduates’ stories included

• Early exposure to computers

• Pursuit of intrinsic rewards

• Self-teaching

• Entering industry prior to completion of degree

• Finding employment

• Life-style preferences

34

When asked to reflect on the value of university ICT degrees, four main topics were covered:

• Value of non-programming subjects

• Value of industry experience

• What I got out of uni

• Advice for ICT departments

3.2.2.1 Themes

3.2.2.1.1 Early exposure to computers

Most of the males interviewed reported early exposure to computers, through home computers (particularly computer games), through their parents and through friends. Many interviewees recounted manipulating configuration files and learning DOS in their pre-teen years, so they could get computer games running. Neither of the female interviewees had exposure to computers prior to secondary school. The most common route for being introduced to computers was via a home computer introduced to the family by the interviewees’ fathers. In some cases, interviewees’ fathers actively encouraged their sons to become interested in computing. Five interviewees have fathers who work in computing or related fields. In other cases, interviewees are the only computing-minded members of their families.

For example, Grant, now working for a Web-development business, first learned a programming language from his father at age eleven. Similarly, Brendan, now a programmer for his old university, was first exposed to electronic technology through his father’s work. When he was seven, his family bought their first home computer and Brendan taught himself to program. Tim, who now works for a large telecommunications firm, has been around computers since he was eight because his father owned a computing business. He first began experimenting with programming at age twelve.

Some interviewees described a social side to their early computing interest, talking about “computer friends” with whom they played games and about the social side of Electronic Bulletin Boards. For example, as a teenager, Jarrod, now a games programmer, wrote ‘primitive games’ with his friends on weekends. Others stated that their interest in computers was a means of escaping from family problems and/or avoiding interactions with others.

3.2.2.1.2 Intrinsic rewards

Most of those interviewed reported intrinsic rewards, such as “fun” and sense of achievement, as the motivation behind their interests in computers, both in the early years of their exposure and later in their pursuit of ICT-related degrees and careers. For example, Ben, who now runs his own Web-development business, said that he enjoyed his successes when he “conquered the machinewas motivated by the challenges and the “fun”. Dave, a Business Analyst for a large consulting firm, explained how his choice of career is motivated by the challenge it poses. And Steve, a Software Developer in a small business, described the satisfaction derived from devising elegant solutions and sharing solutions with peers who can understand and appreciate good work. The pursuit of intrinsic

35

rewards is a theme that continued through to interviewees’ careers aspirations and life-style choices.

3.2.2.1.3 Self-teaching

Most of those interviewed reported having taught themselves how to program or how to build or configure computers many years prior to their university courses. The most common motivation for this self-teaching was to play games, with some seeing the computer itself as a puzzle to be solved or a challenge to be conquered. Common areas of learning included using DOS commands, manipulating himem and experimenting with autoexec.bat and config.sys files. Others reported learning about the hardware so they could repair, upgrade and configure their own machines and those of their friends. James, a database administrator/IT consultant for a large consulting firm, said that by Year 10 he was repairing,

-developer for a micro business, said that as a child he would dismantle his toys and other household gadgets and then reassemble them. Similarly, Tim recalls always having a curiosity for technology and, as a child, pulling things apart to see how they worked.

Many of the interviewees discussed the relative value of self-teaching versus formal education in terms of preparation for work. Most believed formal university education was not necessary for a successful career in ICT. ICT professionals with different backgrounds were seen to bring different strengths and weaknesses to their work.

A common view was that much of the technical content covered in university ICT courses is obsolete by the time graduates enter the workforce. For this reason, self-teaching was seen as important, both for graduates and non-graduates. Some interviewees saw a university ICT course as laying the foundation for ongoing training and self-teaching. James, who completed a double degree in Business and Computing and now works for a large consulting firm, said that, although he was trained on the job, he realises that you cannot be trained in new skills if you do not know the fundamentals. He now realises how much theory he has learnt and retained from his Computing degree and has seen other new recruits, who do not possess the same knowledge, struggle during company training. Grant, a Computing graduate who now works for a micro business, believes he possesses more and better abstract knowledge than his non-graduate peers but thinks that this is not always considered valuable by employers, especially non-technical management who are more interested in speed and cost of delivery. Tim, who did not complete the final year of his Information Systems degree and now works in a large telecommunications company, sees university as teaching you how to acquire new knowledge. He sees university degrees as helpful but not as important as hard work and networking. He also believes that courses provided by software and hardware vendors are more up-to-date and more highly valued by some employers than a university degree.

Steve, a software developer in a small business, has only one year left of his five-year software engineering degree and does not think he will complete it. He thinks that by studying at home, he will pick up more useful skills and then be able to put them into practice at work. Steve said that only one of the programmers at his work is formally trained. He believes that self-taught people make better programmers because they naturally enjoy programming, but that they are limited

36

within a team environment because they have not learnt standard programming styles, documentation and how to work with other people.

Learning on-the-job was seen as an important part of becoming an ICT professional. Two interviewees expressly argued that preparation for work is not the role of university education. Others recognised that universities are limited in their ability to prepare students for specific workplaces. Rita explained that while you need a basic understanding of theory, employers have their own practices and standards. Similarly, Steve believes that the best place to develop the skills required for employment is the workplace itself, with university education providing a framework on which to build further skills.

3.2.2.1.4 Entering industry prior to completion of degree

Many of those interviewed reported taking up ICT-related employment prior to completing their degrees. In fact, in some cases, obtaining employment in the computing industry reduced their motivation for completing their degrees. Two interviewees have “deferred” their studies with no intention of returning to complete their degrees, despite being in the final stages of their courses. Particular note should be taken of the case of Steve whose pursuit of a formal education in software engineering is intrinsically motivated. Steve is highly motivated to learn for learning’s sake, yet he accepted work as a programmer in the fourth year of his five-year course because he was no longer enjoying his studies. The temptation to exit courses and work in the industry must be even more tempting for those students whose motivation for studying is to get a job.

3.2.2.1.5 Finding employment

The majority of interviewees attained their first positions in industry through their family and social networks. The most common means of being “put forward” for a job was via the recommendations of friends and family. Others were recommended by their lecturers, or made their own contacts with firms and recruitment agencies through student societies.

Two interviewees had been involved in discipline-based student societies while at universities. Both felt strongly about the positive influence that this involvement had on their career planning and networking. James believes that if he had earned straight HDs but had not socialised and become involved in his university’s Accounting and Computing Society, he would not have been successful in securing his current position. James believes that course administrators should do more for students in terms of career planning and advice. Similarly, Rita believes that her involvement in her universities Information Systems Society, and subsequent contact with ‘big five’ sponsors, improved her employment prospects in two ways: Her involvement in the Society exposed herself to recruitment practices and it also demonstrated her leadership potential. She believes that socialising with like-minded peers helped her crystallise her career goals.

3.2.2.1.6 Life-style preferences

Two of the interviewees are involved in what might be called entrepreneurial behaviour. Ben, a non-graduate, has been running his own business since 1997. He worked as a “one-man show” until 2000, when he brought a friend into the

37

business. As an employee in previous positions, Ben was always unhappy with the management and thought he could do better.

Tim, who has been working for the same telecommunications company since Year 12, is currently creating a “side business” which he expects to be up and running within months and possibly supporting him and his two business partners within 12 months. Tim is inspired in his entrepreneurial pursuits by three men: his father who founded his own company, a businessman who he worked for while holidaying in the United States and the manager who offered him his current position.

Another interviewee, Mark, aspires to have his own company, but does not have any concrete plans yet. His motivation is his dissatisfaction with his current employer.

A number of interviewees said they were not interested in starting their own businesses. Reasons included the stress that would be involved and the financial risk. A significant number of interviewees stressed that they aspired to a quality life-style rather than financial gain. For example, Brendan said that when he was younger, he had aspired to be “rich and powerful”, but he has “mellowed with age” and now aspires to be happy and to have an easier and more relaxed life than he had earlier imagined he would want. Similarly, Dave said that he identifies with “Generation Xers” in that he does not care what he does for a living; he just wants to be happy and have nice people around. Grant explained how he has seen his parents work very long hours and does not relish that lifestyle: “I would rather not have heart disease at 50.” These types of comments were repeated by most of those interviewees who do not aspire to run their own businesses.

3.2.2.2 Issues

When asked to reflect on the value of university ICT degrees, two main areas were discussed: the value of non-programming subjects and the value of industry experience. Some interviewees also identified what it was that they saw they “got out of” their degrees. Others made specific suggestions as to how university ICT

3.2.2.2.1 Value of non-programming subjects

Most interviewees who had studied for ICT degrees reported not enjoying subjects on project management and documentation. Steve reported that, after two years of working diligently in his software engineering course, he began to lose interest. He explained that the course was moving away from programming and towards project management and “paperwork.” He said that the non-programming components were “too much like English and Humanities and things I don’t want to touch.” This sentiment was common among the graduates interviewed.

The commonly reported preference for the programming subjects involved in Computing and Software Engineering degrees over those focussing on documentation was seen as a source of problems for students working on group projects. As Steve explained, “everyone wants to do the programming and no one wants to do the documentation.” James recounted an industrial experience subject that he completed in the final year of his Computing degree. He recalls being unmotivated from the start, particularly because he was allotted the administrative and documentation tasks rather than the “hands-on work”.

38

Some graduates, despite their dislike for the non-programming elements of their degrees, realise their importance. Rita, a Computing graduate who works for a big five consulting firm, recorded only one fail in her degree. She failed an accounting-oriented project management subject, which she now realises was the most relevant of all her subjects to her current work. Brendan, who now works as a programmer for his old university, also failed a project management subject. Although he is not interested in this area of computing, he realises that it involves important knowledge and skills. He sees documentation and testing as “necessary evils”, appreciating why they are necessary, but finding them boring.

Steve says, despite seeing the importance of documentation, he avoids it in his work. He has not fully documented the work he has done in his current job. He sees lack of documentation skills as one of the weaknesses of self-taught programmers.

3.2.2.2.2 Value of industry experience

A number of interviewees had completed extended project-based industrial experience subjects in the final year of their degrees, involving working in groups with an industry-based advisor to solve particular problems for real clients. Responses to these subjects were varied. A number of graduates reported problems due to the group work involved. Those that had positive experiences saw the group work as an essential part of their learning.

Brendan, who was the project leader for his group, said he learnt about the importance of group dynamics and conflict management. Rita regards the industrial experience subject she completed to be one of the more valuable things about her degree because it involved dealing with an actual client. However, she thinks a block industry placement would be more valuable. Similarly, Grant, who believes that his industrial experience group was poorly organised, poorly guided and produced what he considers to be a worthless solution, also believes that an industry placement would be more valuable than a few hours a week while continuing other studies.

Only one of the interviewees, Matt, had completed an extended industry placement as part of his degree. Matt selected his degree (Bachelor of Business/Information Systems) because it offered a ‘co-op’ or ‘sandwich’ industry year at third year. Matt now works on a permanent basis as a Systems Engineer with the business that he had his industry placement with. When asked what universities could be doing to better prepare students for graduate employment, Matt stated that there should be a greater emphasis placed on industry placements: “work experience is the only way of knowing what it’s all about.”

3.2.2.2.3 “What I got out of uni”

Many of the interviewees found it difficult to articulate what they had “got out of” their ICT degrees. However, three things were repeatedly identified. University ICT degrees provided

• an overview of ICT disciplines and the ICT industry

• learning skills

• networking opportunities

39

Brendan said he appreciated the exposure to different sides of computing. He already knew most of the programming through his own self-teaching, but he says that the course gave him a formal structure to “bind his knowledge together”, giving him an overall view of how different parts of computing relate to each other. Similarly, Michael believes his university education gave him a broader perspective on the ICT industry.

Tim sees university as teaching you how to acquire new knowledge. Similarly, Michael said he acquired the knowledge required to stay in tune with this rapidly changing industry as well as learning that there is more than one way to solve a problem and that even incorrect solutions are a source of learning and progress. Jarrod saw the analytical skills he gained as the main benefit of his university degree.

The skills content covered by university degrees was taken for assumed by many of the interviewees, with very few remarking on it. Many had already taught themselves much of the programming. Some saw the content covered as not relevant to work in the ICT industry or as redundant before or soon after they completed their degrees, but this was seen as unavoidable.

3.2.2.2.4 Advice for ICT departments

Few complaints were made against interviewees’ university degrees or departments. Three sources of complaint were that

• some content covered is out-of-date or not relevant to industry

• degrees lack integration across subjects

• departments do not provide adequate career advice

Shona believes that some of the content of her degree lacked industry relevance because it is emerging and not yet applicable in industry. Furthermore, she believes that some content, which is fundamental and highly applicable in industry, was taught too late in her degree. Jarrod also thought that some of the content covered in later years, for example programming in C, should have been introduced earlier because of its relevance to industry. These types of comments raise the question of the purpose of university education, a question raised in the mini-conferences with ICT educator. As described below in Section 4, ICT educators perceived a tension between the expectations of students and their own understandings about what a university education should provide. For example, while students demand particular languages that they see as needed by employers, ICT educators are generally of the opinion that the particular language learnt is of little consequence. ICT educators are more concerned that students learn the principle involved in programming, along with skills to continue to learn new languages as they are developed and go in and out of fashion.

Shona also thought there was a lot of duplication in her degree due to individual lecturers and departments not communicating with one another: “it’s like the departments don’t talk”. She feels subjects are poorly integrated and recalls being taught similar material at each year level. Jarrod also made comments about course structure, suggesting that students be given more power to choose subjects and build up degrees that are more tailored to their interests. Course design and curriculum integration is an ongoing concern in all disciplines. Problems of duplication most likely vary between departments and courses. Curriculum integration was the subject of three of the teaching and learning initiatives

40

reported by ICT educators (see section 5). Specifically, the integration of the teaching of professional skills or graduate attributes has motivated some departments to rethink the design of courses and the integration of content.

James believes his degree has made him highly employable but feels that course administrators should do more for students in terms of career planning and advice: “no-one gives you a kick up the bum”. The amount and quality of career counselling that is provided to ICT students would vary between departments and between universities. A lack of direction was observed in the majority of graduates interviewed. Many were not pursuing university educations primarily for career-oriented reasons. If graduates outcomes such as employment and satisfaction with employment are to be seen as measures of quality of university education, then perhaps more concerted efforts need to be made at the departmental level.

3.2.2.3 Entrepreneurial behaviour

3.2.2.3.1 Sampling issues

The intention in carrying out this exercise was to obtain a sample of recent (past 2 or 3 years) graduates and to take a fairly open-ended approach to exploring what their occupational and career behaviour looked like. At the background was the intention to try to understand the way in which entrepreneurial behaviour in the ICT industry might relate to the content of university courses.

The main problem we faced was that it was difficult to find recent ICT graduates who were working in, or were running small to micro businesses. Extensive use of staff and student networks, mainly within Monash University, produced a limited number of possibilities and only a small proportion of these were accessible or were willing to participate.

This raises some questions about the nature of small and micro businesses in the ICT sector. The data from the industry survey shows that those who do not employ graduates are much more likely to be in the small to micro group - 78% of respondents versus 29% of the respondents who did employ graduates. Those who did not employ graduates and who were in small to micro business were much less likely to have a degree than the similar sub-group of respondents who did employ graduates.

We do not know what proportion of these industries would fit within the entrepreneurial rubric.

3.2.2.3.2 Vocational behaviour, course and career

The thirteen individuals interviewees can also be looked at in terms of vocational behaviour and its expression in terms of course choice and career choice. None of the cases would fit into a conventional picture of the individual who makes a clear choice of career and then works in a fairly linear way to achieve career goals. Many of them would be seen by a careers counsellor as being quite confused in their career orientation.

There is little definitive work which tries to understand the behavioural domain of the entrepreneurial individual in relationship to overall career behaviour. We do not know what distinguishes entrepreneurial individuals from their non-

41

entrepreneurial fellows in the way they make careers decisions. Basically, we find it difficult to decide whether the entrepreneurial person makes a career of being an entrepreneur or simply works in an entrepreneurial way.

It is very difficult to determine how a university course can provide support to this type of individual when there is a problem defining the type and the associated behaviour.

3.3 Conclusions The primary conclusion to be drawn from the review of literature on graduate satisfaction and graduate behaviour is that very little is known about ICT graduates and their attitudes towards their courses. This is an international phenomenon - little published research was found anywhere in the English-speaking countries.

Our preliminary exploration of graduates’ career stories identified a number of themes that possibly characterise a particular type of ICT graduate, that is, the type who have exposure to and develop an interest in computing technology during their childhoods, are motivated to pursue the intrinsic rewards of using computing technology, and teach themselves how to program before entering any formal ICT education.

Interviewees’ attitudes towards the non-programming subjects in university ICT degrees raise questions about their expectations of their courses and their expectations of the work of ICT professionals.

The themes and issues identified here warrant further inquiry. Further research would have implications for the recruitment of students to ICT degrees, the promotion of ICT degrees and the ICT professions to prospective students and to current students, and the development of programs to better prepare graduates for work as ICT professionals.

Differences were observed between male and female interviewees. Unlike the male graduates who had long pursued interests in computer technology and saw their university degrees as a natural continuation of this pursuit, the two female graduates were more pragmatic in their views about the value of their degrees. Due to the small numbers of graduates interviewed, observed gender differences cannot be generalised. However, this does raise questions about the different needs, attitudes and behaviours of male and female graduates generally, and warrants further investigation.

3.4 Recommendations arising from this section • Recommendation 2 That universities be given support in implementing

graduate follow-up and employer satisfaction studies which focus on local issues and local needs, and which use a diagnostic approach. This data would complement that routinely collected by the Graduate Careers Council of Australia.

42

• Recommendation 4 That research funds be allocated to longitudinal and/or retrospective studies of graduates to obtain an understanding of the career paths through the ICT industries, both within Australia and overseas.

43

4 ICT Educators’ views on innovation

4.1 Introduction This section focuses on the identification of factors to which innovation in ICT education respond, taking an open, qualitative approach to the stated overall purpose of the research project: “to identify the extent to which innovations in teaching and learning and processes of curriculum development and review have been deployed in response to the needs of students and employers” (Australian University Teaching Committee 2000).

First, background factors in the broad context of university teaching in Australia are outlined. Then a summary of the methods used in this part of the research is given. These methods correspond to the phase identified in section 1 as Consultation with ICT educators (see Table 1). Finally, the research findings for this part of the research are presented and links are made between these findings and the Project Brief.

4.2 Context of university teaching in Australia Australian universities have undergone significant change over the past fifteen years. The impact on university teaching is unprecedented in this country. Much of this change has been in response to government policy and governments’ repositioning of the role of universities. To respond to policy changes and subsequent changes to the environments within universities, institutions have sought to take advantage of new and developing computer- and network-based technologies. In teaching and learning in particular, significant and controversial changes have been proposed and continue to be implemented. Before presenting the findings for this section, the key changes that have taken place are identified, so as to paint a picture of the broad context of university teaching. It is important to consider these contextual factors when interpreting the data collected on academics’ views. However, it is perhaps more important when formulating recommendations to be implemented within this context, both those that are based on the findings reported in this section and those made in other sections.

This section on the context of university teaching has two main parts. First, the major changes impacting on university teaching are outlined. Second, significant movements in university teaching are described.

4.2.1 Broad changes to the university environment Over the last fifteen years, Australia’s universities have been subject to extensive and multi-facetted change. These changes parallel those experienced by the university sectors in most developed countries around the world. They include

44

• Increasing mass education and increasing government focus on vocational education

• Changing policy environment for the funding and governance of higher education

• Internationalisation and the advance of information and communication technologies

The pressures leading to these changes in Australia are evidenced in and acknowledged by a series of Government reviews and reports (eg. Dawkins 1988; Dawkins 1989; Hoare 1995; Vanstone 1996; West 1998). They have been widely described and discussed in academic literature, the popular press and government reports. They are described in detail in Appendix H.

Information and Communication Technology-based university departments and faculties in part owe their growth to the massification and the vocationalisation of university education and the advance of information and communication technologies. With the diffusion of computer-based technologies across society, graduates skilled in ICT are required for growth in the economy. Skill shortages in this area continue to be reported and forecast (see discussion in Section 2). This opportunity is also one for employment growth. With the rapid expansion of the number of students taking ICT-based courses, ICT has developed from a specialty area located in Science and Engineering faculties to a discipline in its own right. The implications of these factors for ICT education are described and discussed in the findings section below.

As new technologies are integrated into workplaces, the skills and knowledge required by graduates change and the number of graduates requiring skills in ICT increases. In ICT departments, developments in technology usually translate into curriculum changes, particularly in terms of content. And skill shortages that are answered by a growth in the number of students mean not only a larger student body, but also a more diverse student body. Both of these implications, specific to ICT education, are described and discussed in the findings section below.

4.2.2 Movements in university teaching and learning The broader contextual factors listed above and described in Appendix H are now largely taken for granted and seen as part of the environment in which universities and university teaching staff have to operate. Further to these broad contextual factors, university teachers are subject to pressures fro a number of movements that specifically target university teaching. These movements include:

• that towards flexible delivery, which in some manifestations includes a move towards student-centredness

• that towards quality assurance of higher education and the particular role of student evaluation of teaching questionnaires

• that towards the promotion of a scholarship of teaching and evaluation of educational innovations, such as practitioner-run and targeted forms of evaluation

These movements are widely debated and their outcomes are still largely uncertain. They are described in detail in Appendix I.

45

4.3 Methods

4.3.1 Rationale for approach taken A mini-conference format was developed to collect two three types of data:

• ICT educators’ reported perceptions about factors to which educational innovations respond

• ICT educators’ reported perceptions about the dissemination of innovation

• ICT educators’ reported accounts of specific teaching and learning initiatives in which they are involved

This section focuses on the first type. Data on dissemination is reported in Section 6, while specific teaching and learning initiatives are described and discussed in Section 5.

Mini-conferences were held in each capital city. Two mini-conferences were held in Melbourne and in Sydney. One mini-conference was held in each of Hobart, Brisbane, Adelaide, Perth and Canberra. A video-link was used to conduct an abridged version of the mini-conference with educators in the Northern Territory.

An open approach to collecting data on ICT educators’ views was used to increase the salience of factors identified. A qualitative approach ensured that the data collected on complex and interrelated factors and issues were rich, enabling detailed descriptions and ensuring a degree of transferability of conclusions. Data collection for this phase was highly participative, increasing the quality of engagement with, and the credibility of conclusions about, one of the key stakeholder groups (ICT educators) in ICT education. The methods used to collect, analyse and report data from the mini-conference open discussions are described in detail in Appendix D, along with limitations. Below, the mini-conference format and participant information is summarised, as is the method of analysis.

4.3.2 Mini-conference format The mini-conferences ran from 10am to 4pm on designated days in each capital city. Between four and fourteen people participated in each mini-conference. These small numbers allowed for unstructured discussions. A workbook was used to structure the proceedings of each mini-conference. Two researchers directed the proceedings.

4.3.2.1 The workbooks

Workbooks were provided to each participant on their arrival at a mini-conference. The workbooks served a number of purposes, providing

• a structure for the mini-conferences

• somewhere for participants to make notes in preparation for their contributions to the mini-conferences and about the contributions of others

• participants with a record for participants of their participation and of notes taken

46

• the researchers with a record for the researchers of notes made by participants

• researchers with outline information for the researchers about the teaching and learning initiatives reported

The workbooks contained a description of the study, an informed consent form, spaces for participants to make notes and five stimuli items:

• What are the factors driving educational initiatives in ICT education?

• What are the factors inhibiting educational initiatives in ICT education?

• Please describe a teaching and learning initiative in which you or your department are currently involved. (included sub-prompts)

• Deciding whether your teaching initiative is effective. (included sub-prompts)

• What issues would be involved in disseminating your initiative to other ICT educators?

A sample of a workbook is provided in Appendix E.

Questions 1, 2 and 5 were used to stimulate discussion among participants. For each question, participants were given five minutes to think about and make notes about the question before they were prompted to discuss the issues involved. As issues were raised and discussed, one of the researchers recorded the issues on a whiteboard. This often involved negotiation between participants. At times, the researchers asked for issues to be clarified. Questions 3 and 4, along with accompanying sub-prompts, were used to stimulate participants to prepare for presentations to the group. For each of these questions, participants were given short periods of time to prepare by making notes in their workbooks. After each presentation, questions were taken from the group.

Feedback from participants regarding the format and conduct of the mini-conferences was generally positive and constructive. Two participants offered unsolicited written feedback. A participant at the first Sydney mini-conference wrote:

• I felt the time was well spent

• Well run and facilitated

• fun/ enjoyable/ challenging

• Met interesting colleagues

• Felt that I learnt more than I gave

Boosted my enthusiasm and encouraged me to keep on innovating.

Similarly, an Adelaide participant wrote:

• Well organised/ well run

• Open/frank exchange of views

• Free flowing format

• Useful to hear others’ views/ experience

Participants at the Hobart and Darwin mini-conferences regretted not being able to interact with participant from other universities. Similar comments were made by some participants of the Adelaide mini-conference, where the majority of participants were from the one institution.

47

4.3.3 Participants To recruit ICT educators to the mini-conference program, invitations were sent to all heads of Australian university departments, schools and divisions where ICT-related courses are offered. These included departments of information systems, software engineering and computer science, for example. Invitations described the target group as staff teaching in ICT-related areas who are involved in teaching and learning initiatives that aim to improve ICT education. Department heads were encouraged to circulate invitations to relevant staff members.

In total 83 ICT educators from 51 administrative units (46 departments/schools, 1 faculty, 4 university level education units) in 29 universities (27 public, 2 private) participated in the mini-conference program. Participants included staff members from a range of academic levels, including all levels of lecturer, department heads, professors, associate and faculty deans, one pro-vice chancellor and a small number of instructional designers. A full list of mini-conference participants is given in Appendix B.

4.3.4 Analysis A coding process, using NUD*IST VIVO (Qualitative Solutions & Research 1999), was used to reduce the data collected. The aim the coding process was to reduce the data, so an intelligible account of what is a very large body of qualitative data could be produced (Dey 1993). For questions 1, 2 and 5, notes made in participants' workbooks and whiteboard records of discussions were coded into a large number of descriptive categories. These categories were then grouped, reducing the data further. They were then tested against transcriptions of audio-recordings of discussions, using the constant comparative method (Silverman 2000; Strauss & Corbin 1997). As data were compared categories were merged and revised. The resulting small number of categories allowed for the complexity of particular issues/factors to be captured in each category and examined. Particularly rich excerpts were coded into multiple categories. To increase the reliability of this coding process, a co-researcher inspected reports of each code for consistency. The limitations of this method of analysis are described in Appendix D.

4.4 Findings: Factors driving innovation in ICT education Findings of the analysis of the open discussions stimulated by Questions 1 and 2 are presented here. Factors described by participants to be driving and inhibiting innovation in ICT education are identified and related to the purpose of the project as stated in the Project Brief.

Analysis of the mini-conference open discussions on factors driving and inhibiting innovation in ICT education resulted in nine interrelated categories. These categories are

• personal initiative

• changing scale of teaching

• changing student population

48

• push for flexible delivery

• development of new content

• availability of new tools

• support from management

• limited academic freedom

• student demand

These categories are described in detail below, with reference to illustrative quotes. This section paints a picture of the complex web of pressures under which ICT educators find themselves.

4.4.1 Individual initiative Individual initiative refers to factors that, although linked to broad contextual factors, were described by participants as originating within individual ICT educators themselves. This category includes references to individual educators’ desire to be better teachers and their desire to be seen to be at the forefront in their fields. One participant said, “There’s also enthusiasm and a desire to be a better teacher and help your students to learn better” (mini-conference (M-C) transcript). This comment is typical of those comments referring to the initiative of individual ICT educators.

The desire to be a better teacher was described as driving responses to observed problems. Some problems discussed in this context were first year transition problems, student dis/satisfaction, plagiarism, equity and access issues and unsatisfactory learning outcomes. Unsatisfactory learning outcomes included the failure of students’ skills to meet the needs of industry. Both student dissatisfaction and first year transition problems were discussed in terms of the changing student population and the changing scale of teaching. As one participant explained,

… there’s a sort of positive and negative reasons for [teaching and learning initiatives], so you know, you get enthusiasm in an individual saying, ‘alright I want to do this because it’s a good idea and it will promote better teaching blah, blah, blah’, versus ‘well here’s a way of coping with this appalling problem we’ve got, let’s

-C transcript)

The individual initiative category also includes contributions about individual ICT educators’ decisions not to pursue an innovation due to the professional and personal risks involved in trying something new. Discussions about these risks included references to lack of support from management, and lack of reward and recognitions for efforts. Concerns about intellectual property were also discussed as potentially contributing to an individual educator’s decision not to pursue an educational initiative. The following mini-conference excerpts illustrate the types of concerns that participants had when describing individual initiative as an inhibiting factor:

Participant: Another factor is the amount of risk that an individual might be prepared to take, I think to show initiative you have to be a risk taker and I think a lot of academics might not be risk takers and there might be a few reasons for that one, they're just, that's not the

49

type of individual they are and a measure of performance I was told to be promoted just based on teaching can't just be a good teacher you have to be an outstanding teacher, I don't see those requirements placed on the research emphasis, so you know, if you want to put under risk, you have to be a risk taker and not all academics are risk takers. [Moderator:] So riskiness is an inhibiting factor. The riskiness of actually trying something new when it just may not be valued? Participant: Yeah that's right. [Moderator:] You're trying something that hasn't been? Participant: And it will affect your reputation for life so if it goes badly you have got this black star against your name and nobody will want to do anything with you but [if] it goes well, well you could be an international superstar, who knows. (M-C transcript excerpt)

Participant: I think one inhibiting factor is uncertainty about the educational efficacy of this stuff and a lack of research findings so people say ... Participant two: Saying it works. Participant one: Well you can’t prove it works so I’d rather you

-C transcript excerpt)

Participants generally agreed that, despite often being in an environment that was not supportive of innovation, individual educators continued to develop teaching and learning initiatives both as mechanisms for surviving changing conditions and as a means to satisfy their own desires to “be a better teacher”.

4.4.2 Changing scale of teaching Participants consistently cited changes in the scale of teaching both as a driving and an inhibiting factor, referring to increasingly larger class sizes and fewer contact hours. Larger scale teaching was described as causing particular problems, thus driving particular innovations, while the teaching and learning environment of larger scale teaching was described as inhibiting some types of teaching and learning activities. The following excerpt describes the changing scale of teaching in negative terms, as a problem that needs to be survived:

I think number one is the size of our student body and how are we going to survive with, you know, we used to have classes of one hundred, big ones were two hundred and now our big ones are a thousand and we don’t seem to have any more money, we don’t have teaching assistants for PhD students ‘cause nobody does a PhD in Computing any more and so we have unskilled tutors and we just need strategies to survive. (M-C transcript)

Changes in the scale of teaching were linked to Government and university level policies and pushes to save money. They were discussed in relation to increasing staff workloads, first year transition problems and problems arising from the increasing diversity of students. Although, generally critical of pushes for flexible delivery through the use of Internet-based technologies, new modes of delivery were acknowledged as the most obvious way to deal with the larger student numbers. However, many participants had reservations about more flexible modes

50

because, although they are an obvious response to larger student numbers, they did not see them as solutions to problems caused by these large numbers.

4.4.3 Changing student population Changes noted in the student population include the more diverse language and educational background of students, the more diverse experience and skills of students and more diverse student circumstances. The increase in diversity of language and educational background of students was linked to the growing number of International Students and to Government and university level policies on enterprise funding and fee paying. Increased diversity in the range of students’ experience and skills was linked to an increase in the number of enrolments and discussed in terms of Government policies for more vocational education and for greater access to higher education. That is, the greater diversity in students was seen as a result of greater student numbers. The following mini-conference excerpts illustrate the types of concerns that participants have about the diverse range of educational backgrounds, skills and experience of students.

Participant: I think linked to that is a more diverse student population than the universities have had in the past. <general agreement> … Participant: Yes and accommodating a wider range of abilities in the one class. (M-C transcription excerpt)

Participant: Yeah we’ll get secondary school students doing tertiary level units. Participant two: Yeah. Yeah. Participant one: But you still get people coming into computing degrees who haven’t used computers. Participant two: That’s right. Participant three: Yeah that’s right. Participant one: You’ve got someone who’s [a] Cisco qualified engineer and the person who’s never used a computer in the same class. Participant two: That’s right. Participant one: And you have to -- Participant two: Yeah first year is a nightmare, there’s no two ways about it. Participant one: And you have to keep them happy. (M-C transcription excerpt)

The fact that computers are so ubiquitous now means that some of our first year students have been doing computer programming for ten years and they’re good at it indeed, whereas others don’t know where the enter key is and that’s, that is a real problem. How do we amuse the experts or give them, add some value to their education in first year? (M-C transcription excerpt)

While increasing student numbers is a feature of other discipline areas, participants argued that it was a particularly acute problem in ICT education because of both the increasing demand for ICT graduates and the increasing demand from employers for ICT literate graduates and the resulting growth in the need for graduates from a wide range of disciplines to have some ICT education.

51

Students were also described as having more diverse circumstances in terms of time fraction, workforce commitments, family commitments and faculty of enrolment. For example, in the following excerpt, a participant explains that the amount of time students spend in paid work has increased.

Participant: One thing we’ve really noticed with our students is the need for them to work more, so they spend a lot more time working as supposedly part time and trying to do a full load and therefore we have to be more flexible in letting them pick what subjects will fit in. Moderator: You mean work for money not academic work? Participant: Yes, yes absolutely. Second participant: Yes. First participant: And that’s really changed over the last ten or fifteen years. (M-C transcript)

Participants then described the implications of the diversity among students’ circumstances. In particular they referred to these students “wanting to operate in that flexible manner” and a growing demand for “time/place independent

-C transcript excerpt).

As with the scale of teaching, the growing diversity of the student population was noted both to inhibit and to drive innovation, particularly diversity in ability and skills at first year.

4.4.4 Push for flexible delivery Participants reported being under pressure to move towards more flexible delivery of courses. This factor was linked to other factors in complex ways. It was discussed in relation to the increasing scale of teaching, money saving, the availability of new tools, the “Internationalisation” of higher education, growing competition between universities, student needs/demands and fashion. In the following excerpt, a participant describes the relationship between increasing diversity in students’ circumstances, student demand and flexible delivery of education.

I think another thing is, with regard to the student base, I can’t remember the exact numbers but I know [my university’s] intake of students that are sort of part-time or are working is getting higher and higher <general agreement> And the students do want to operate in that flexible manner and that is sort of pushing the university to go for this flexible learning avenue, so we’re having, you’ve got to pull out the figures but I don’t think our classroom numbers are going down, but our numbers where students want to study in their own time and their own place at their own convenience is going up and that might include yeah, new mothers, it might include people who are working part-time, working full-time and want to do the courses outside of hours and so I think there’s that you know population mix has changed. (Mini-conference transcript, Melbourne 1)

Participants were aware of both Government and university level encouragement of the flexible delivery of programs. Despite recognising potential advantages of flexible modes, participants were generally critical and even cynical when

52

discussing the policy environment surrounding pushes for flexible delivery. This cynicism seems to arise from participants’ perceptions of poorly justified university level initiatives, particularly when initiatives are seen to be motivated by money saving and revenue raising, rather than by educational aims. Participants also described misunderstandings at the university administration level about the educational advantages and disadvantages of flexible delivery and about effects on teaching staff members’ workload and work practices. Comments containing implicit criticism of university level pushes for flexible delivery include:

… our university would see something that’s effective as something that fits into their teaching and learning plan which is ‘lets make learning flexible’. ‘Oh look, this makes learning flexible: that’s effective. (M-C transcript excerpt)

Participant: What was the one that came after changing student populations? Participant two: Larger classes and -- Participant three: Workloads. Participant one: Yeah higher workloads. Participant three: That inhibits it doesn’t it? Participant one: Well both. Participant four: No that’s why I say, “Oh we’ll just put it all on the web,” and you don’t have a class. (M-C transcript excerpt)

You got to admit, I mean half the people in management they just, you know, every, “You can do anything on the web now.” “We must

-C transcript excerpt)

Participant: Now separately but aligned with this question about not understanding what is possible this almost totally entrenched belief in some people that you can do absolutely anything on the world wide web if there’s a problem all you have to do is put it on the web and it will be solved. Participant two: Internet boosting. Participant one: Yeah Internet hype. (M-C transcription excerpt)

At the high levels I think their beliefs are not necessarily founded [in] research about costs and about learning and about the quality of learning. (M-C transcription excerpt)

Participants also discussed risks related to flexible delivery. They expressed concerns about lack of evaluation or pedagogical framework, the increased likelihood of problems such as plagiarism, investing time in initiatives that are “faddish” or that suit marketing purposes rather than educational purposes, about student access and reliance on inadequate or unreliable technology.

4.4.5 Development of new content Participants described a constant need to develop new course material. Participants supposed that this need is more acute in ICT education than in many other disciplines because of the rapid development of information and communication technologies. They discussed this need in terms of demands from students to learn about the latest technologies and the changing needs of

53

employers and of students as future employees. The following excerpts illustrate participants’ perceptions of the role of employer needs in content development.

I think the work place and corporation determine to a large extent what sort of student population we have to strive to produce. (M-C transcription excerpt)

Participant: I think [employers’ needs have] a bigger impact on our area than it might have on Philosophy or History or any of those areas Participant two: Yeah a five-year old text book is pretty useless. (M-C transcription excerpt)

Participant: One of our drivers is the fact that we have a one year Grad Dip Program and those people come in with no background in computing whatsoever, perhaps no mathematics, and they’re going to be employed a year later and we haven’t got time. We’re very much industry driven. Participant two: I think we have to resist -- Participant one: Well they’re our customers and our clients and that’s the service we’re providing, a crash education, crash training. (M-C transcription excerpt)

One factor that just came up to my mind, the fact that the students are going to be life long learners. I mean they’re not going to finish their schooling here, they’re going to go into industry, five years from now they’re going to retrain or whatever … It’s much more important to learn how to learn than to actually learn a particular language which we are teaching now which is going to be obsolete anyway. (M-C transcription excerpt)

Participants described some employer and student demands for content as a demand for the latest fad. They also discussed a conflict between their own desire to teach general “the fundamentals” and students’ desires to learn the latest programming languages, describing students’ misconception about what degrees in ICT are all about. Specific skill requirements in newspaper job advertisements, such as requirements that applicants have experience in particular versions of particular programming languages, were seen to add to these misconceptions.

There’s a bit of a conflict between teaching basic principles and teaching the latest stuff and the latest skills, and this is to do with student motivation and employer demands, and it’s do to with a certain degree of ignorance about what’s important, and I think there are real issues here and I, as I say, I don’t know whether they belong in [question] one or two but it sort of applies to a lot of what we’re saying. Would the students be better off if they had less of learning the latest skills of the, that are very specific, that are going to change next year, and learning more general principles or are there no general principles any more? Are things changing so fundamentally? I think this is a problem some people have and it distracts them a bit from building up wonderful ways of teaching cause they’re still thinking what the hell should I be teaching. (M-C transcription excerpt)

There was not consensus among participants about the role of university education and the relative importance of the latest skills versus general principles.

54

Many preferred to differentiate between education and training, seeing up-to-the-minute training as the role of the Vocational Education and Training sector or of employers themselves, and arguing that university education was about the production of “IT professionals”.

Linked to discussions of content and debates about current skills vs. general principles was discussions of graduate attributes, such as effective communication skills. An increasing focus at the university level on graduate attributes was seen as a response to employers’ needs for more well-rounded graduates.

One of them is the one you’ve just mentioned about the diversity of students. Another one is, in particular in the software industry, it is the fact that in the software industry people are working in teams and talking about software development. The days in which somebody was sitting down, down in the garage and developing something is really over, and I think these two are actually pushing us to teach in teams because teaching in teams, or learning in teams actually is one way of solving, partially at least, the diversity. … [moderator:] Okay so that’s put into the category of the needs of industry? The changing needs of industry? Both industry and of students. … And allied to that as well is the kind of area that we’re in there’s a predominant push for the sort of the vocational side of education rather than what I would call the academic side.” (M-C transcription excerpt)

Graduate attributes were described as a new area of content and were also seen as encouraging new ways of teaching. For example, it was seen as appropriate to use team-based learning to teach skills in team work.

The need to develop new content was generally seen as driven by a combination of changing industry needs and the development of new technology. The development of new content was characterised by the tension between the short-term and long-term needs of students. Discussions about this tension often raised questions about the role of universities and the purpose of university education.

4.4.6 Availability of new tools As with the development of new content, the availability of new tools was linked to the rapid development of new technologies, particularly the Internet and related tools. These tools were described as “enabling technology” and many participants described how the increasing accessibility of electronic information and communication technologies has allowed them to develop more interactive and more responsive learning environments.

However, not all discussions of these new technologies were positive. Educational fashions, commercial suppliers of tools and university policies were also described as driving educators up-take of new tools. In particular, participants discussed the role of commercial courseware, such as WebCT™, and university level decisions to promote and support particular courseware products. In discussions, these forces were often described as distracting from the development of effective teaching practices and learning materials. These discussions echoed and often overlapped those specifically dealing with “the push for flexible delivery”, with this push being seen as a particular instance of education’s infatuation with the latest educational tools. The following excerpt is an

55

example of participants’ concern about universities’ responses to the development of new tools.

I think technology also has a lot to do with education initiatives and quite often when a new technology arises it’s grabbed by the university as being an educational innovation without any considered thought as to the genuine educational value of the technology, so that often the technology itself becomes a marvellous PR exercise for government, for universities and actually has little to do with genuine education. (M-C transcription excerpt)

Participants were also concerned about students’ access to technology and about the risks involved in using unreliable or inadequate technologies. The following excerpts illustrate participants’ frustration due to the rapid development of new tools.

Participant: On the technology side I’d say the Internet bandwidth is still an issue … Internet bandwidth, it’s not so much what in theory is there, it’s what’s in practice. Participant two: Yeah. Participant one: It’s about trying to down load videos on it. Participant three: And access too. (M-C transcription excerpt)

Even more basic than that is when you have electronic submission of assignments and you get some kid who’s still got Microsoft Works on their computer and you look at this thing and say what am I going to do with it. Or the ones who send in the ones that are even more advanced than what you’ve got. Yes. <laughter> Yes and you’re using the university’s computers. (M-C transcription excerpt)

Participant: I mean all of us are computer-conferencing, I teach an MBA unit and I never see any of the students, um not at all, it’s all done by e-mail, there’s no lectures, there’s two assignments and

Moderator: You never see the students? Participant: No. Participant two: So who are you teaching? Don’t know? Who gets the qualification at the end? They guy with the name or the guy who paid him to do it?

4.4.7 Support from management Support from management was described as enabling or driving innovation. Lack of support from management was seen to inhibit innovation. Support was described in terms of resources (funding, time, human resources), reward and recognition for efforts and the presence of senior champions. The following excerpts are representative of the comments participants made about how they experienced lack of support from management.

You really need to have management support, I mean as an educator you would like to see that there is a plan at the university, at a high level, with some goals and some strategy or whatever and some

56

really, some public recognition. … you would like to know that in the promotion process … education, or whatever, is equivalent to any other avenues that you have for promotion forces, and I know everybody’s smiling because sometimes you have this in writing, or even in, but it doesn’t work. (M-C transcription excerpt)

They have to put some kind of support, what I mean by support for example, is a lab to experiment or with time, that is if I would like to explore some kind of new technique in teaching I would like to teach less for that particular semester because I want to have time to do that, that’s management support. (M-C transcription excerpt)

Participant: No bloody money. Participant two: Reduced funding. Participant three: Not enough funds. Participant ?: Mm. Participant ?: Along with no support to learn all these new technologies, you’ve got to do it all yourself. Participant ?: Yes that’s, that was resourcing. Participant ?:But the resources are equipment and sort of development time aren’t they? Participant ?: Resources, yeah there’s no need and time for training. (M-C transcription excerpt)

Lack of champions of change too, put it under uni attitudes, champions cause you need those, this sort of stuff, you need someone that’s going to stick their neck out and say no it will take a few years to sort it out. (M-C transcription excerpt)

ICT educators’ lack of pedagogical knowledge was also described as evidence of lack of support from university management. The pursuit of educational training was generally not seen to be encouraged and participants’ workloads did not allow for professional development in this area. Participants consistently described management as not valuing teaching as highly as research. Participants complained that, although management said that excellence in teaching would be rewarded, this was not the case.

Participant: Traditional university attitudes to teaching, I mean this is, we’ve merged from a CAE with a traditional university … and it’s like there are several people, lots of people they’re not sort of from [the original university campus] who I mean, teaching you know, you get up you give your lecture and then you go and that’s it. Participant two: Teaching is not seen as important. Participant three: Well that’s sort of belongs with conservatism really. Participant two: Funding and promotion for research but not for teaching. (M-C transcription excerpt)

This perceived promotion of the value of research over that of teaching must be understood in the context of the nature of the participants who were attracted to the mini-conference program. As noted earlier, the majority of participants are employed by universities or on campuses that, prior to 1988, were teaching-only institutions. These institutions merged with universities with a traditional focus on the value of research and that have only begun to consider explicitly the quality of teaching in recent years. It is arguable that lecturers who worked in the

57

pre-Dawkins universities may be in a better position to perceive a growing valuing of teaching, than those who previously worked in teaching-only institutions. While a small number of participants argued that “things are changing” and that teaching is becoming more valued and better rewarded, all agreed that excellence in teaching is not rewarded as greatly as excellence in research, and that to advance their careers, time would be better spent on research and publications. It was also noted that research into teaching and learning was perceived as a lowly pursuit compared to research and development in ICT-related areas.

4.4.8 Limited academic freedom Limited academic freedom includes comments about controls or guidelines for production of materials, decisions about particular platforms or courseware tools and issues about intellectual property. Many participants observed that they have less flexibility in and less control over the subjects they teach in terms of how and when they are taught. They described an increasingly centralised organisation of teaching, such as timetabling:

Another inhibiting factor that I've found is that we have centralised timetabling now that's things are locked in a year ahead, what the number of contact hours and what kind of classroom you've got for your subject and for example, I didn't want two hour blocks, but I'm stuck with it for second semester this year. I'll swap, I'd like two hour blocks. I wanted single one-hour blocks. Replicated six times. But that is important because we very rarely get to have the choice of what kind of blocks we want and that's typical as well. (M-C transcript excerpt)

This centralised organisation was seen as less flexible and less able to quickly respond to the needs of teachers and, therefore, as inhibiting innovation. Increasingly bureaucratic systems were also seen as inhibiting innovation. In particular, participants described the development of online or flexible materials to be more highly scrutinised than other types of materials and modes of delivery.

I thought we might have touched on the … over regulation. [What] I’m getting at is this um notion that you can employ a lecturer, you can give them some ideas of what you want in the course and they’ll go into a classroom and teach. If you want to do the same thing online, you’ve got to go through umpteen committees, approval from the, you know where and all sorts of other -- over regulations from outside interests before you get even onto the web site. You know that sort of thing has changed and not many people are aware that it’s shifting [in] that direction. Centralised, corporatised teaching

-C transcription excerpt)

This increased scrutiny of materials was seen as a result of university marketing and branding. It was also seen as part of a movement to standardise educational products and services that are delivered via different modes to different types of student. Many participants described the development, in their universities, of multimedia development units who “packaged” content so that it had the professional and consistent “look” that the university marketers were aiming for. This trend was seen more as a marketing ploy than as motivated by educational

58

aims. Many participants discussed the growing push for educators to provide content as a challenge to their academic freedom and to their personal approaches to teaching:

Participant: I've got one more inhibiting factor, in terms of the university policy towards initiatives, in many cases there's a removal of control from the academics like at [my university] we have [a resource unit], which, from the university perspective are responsible for the flexible delivery, but what happens is that academics end up working for them rather, rather than them working for us, so that we're the providers Moderator: You're the content provider? Participant: Yeah. Moderator: And they wrap it up? Participant: And they wrap it up and in many cases they wrap it up in a non-innovative way. Moderator: So how would you … Participant: Well I think I would just describe it as a removal of control if you like. Moderator: Of your material? Participant: Of any form of initiative in some cases. (M-C transcription excerpt)

Another source of loss of control that was described was Universities’ purchase and support of particular courseware programs. Participants generally agreed that this would be enabling for many educators, but they supposed that for ICT educators it was more of an imposition because of their own knowledge of software development and skills in web site production and HTML.

4.4.9 Student demand Student demand includes comments about students’ demands for particular content and for particular modes and also to student resistance to the teaching of “the fundamentals” and to innovative teaching methods.

Student resistance was expressed in terms of students’ expectations and resistance to new ways of teaching and learning. Participants describe conservatism in students who expected traditional modes of delivery. They described students who “just want the answers” and who resent being asked to participate in innovative activities, such as group work or problem solving.

Participants described students’ responses and demands as being increasingly important because of the increasing client-focus of education. This was discussed in terms of increasing fees and in an increasing focus on student evaluation of teaching.

I think our fee paying overseas students in particular are being much more vocal and demanding more quality and that’s certainly a driving factor. (M-C transcription excerpt)

Well they’re our customers and our clients and that’s the service we’re providing, a crash education, crash training. (M-C transcription excerpt)

59

The positioning of students as clients was seen as problematic by many participants who argued that the satisfaction of students’ short-term goals was sometimes at odds with their long-term outlooks. Students were generally described as not being good judges of what was good for themselves; they were seen as misinformed and to have misconceptions about what their courses were all about.

But it’s even more fundamental than that because a lot of the students don’t appreciate that some of the fundamentals are essential to the fluffy things that they want to do, like data bases – “Who wants to study data modelling?” You know. “I just want to learn how to write a web page”. Well I’ve got news for you pal, take a job at the Age, work on their breaking news site, which is run by one of our ex honours students, and you wont write a bit of HTML, it’s all done by back end database and ASP and things like that. … a lot of them wanting to get up the you beaut latest and brightest stuff don’t know what they mean. (M-C transcription excerpt)

Factors influencing student demand included their perceptions of employers’ needs and their perceptions about what was valuable in ICT education. Preferences for traditional modes of teaching and learning was discussed in terms of what the students were used to, that is, their expectations, and also to students’ changing circumstances and their lack of time for approaches that foster a deeper understanding of content.

4.5 Needs of students and employers The project brief requests specifically that the extent to which innovations in teaching and learning have been deployed in response to the needs of students and employers be identified. The categories described above represent those driving and inhibiting factors that are most salient to ICT educators. The following discusses the extent that the needs of students and employers featured in these categories.

4.5.1.1 Innovation and the needs of students

Students' needs were of primary concern to ICT educators and central to innovations. Innovations were most often deployed in direct response to the needs of students (motivations behind particular teaching and learning initiatives are described in Section 5). Students' needs were discussed in relation to the following factors: individual initiative, the changing scale of teaching, the changing nature of the student population, the push for flexible delivery, the development of new content, the availability of new tools and student demand.

When ICT educators described individual initiative as a factor driving educational innovation, referring to "trying to do the best job" and the "desire to be a better teachers", they almost always linked improvement to student needs. Common motivations were to improve pass rates, to motivate students, to improve learning, and to increase the participation and engagement of students. Often individual initiative was described in terms of teachers' iterative responses to observed problems associated with the scale of teaching and the diversity of the student population. However, the goals of individual teachers were couched in

60

the context of other institutional goals that sometimes ran parallel and at other times ran counter to the motivation behind individuals' aims to improve their teaching. One participant noted, "I identified a number of levels to that: there's your own personal goals in terms of quality, but there's also faculty goals, institutional goals, government goals and so on".

Both the changing scale of teaching and the changing nature of the student population were seen by ICT educators to influence the needs of students and the ways in which these needs can be met. Large-scale teaching drives initiatives that aim meet students' needs for interactivity and feedback. The increasing diversity in students' educational backgrounds, academic abilities, preferred learning styles and circumstances drive teachers to develop ways to accommodate this diversity. For example, students' diverse circumstances drive teachers to develop more flexible modes of delivering subjects. As with other trends, the move towards flexible modes was seen as being driven by a network of interrelated factors of which students' needs was only one.

The development of new content and the availability of new tools are often connected in ICT education because the tools often comprise part of the content. Participants described both of these driving forces in terms of students' need for employability and life-long learning. Teachers were aware of the need to develop new content so that graduates could meet employers' expectations in terms of skills, knowledge and attributes. However, they also described students as being misinformed and as having misconceptions about the type of skills and knowledge that was required. In this way, teachers were mindful of the need to resist students' demands to learn in faddish areas, and instead, to attend to what they saw as the skills, knowledge and attributes that would not only help graduates to find employment, but that would help them to perform well in work environments and develop careers as ICT professionals. The types of skills, knowledge and attributes that participants identified as needed by industry reflected those described in the literature (see Section 2) and in the popular press, including for example problem solving skills, communication skills and team work skills. Many also stressed that a foundation in theoretical principles was crucial for students to continue to meet the changing needs of employers.

Student demand was seen to play three main roles in educational innovations: First, student demand for flexible access to materials drove educational innovations. Second, students' misconceptions about the syllabus, about learning methods and about the workplace led to tensions that both drove and inhibited educational innovations. Finally, teachers reported that student were conservative and resisted educational innovations. In particular, student evaluation of teaching (feedback questionnaires) and the need for teachers to perform well on these evaluations were seen as inhibiting educational innovation. Participants’ views on dominant methods for student evaluation of teaching are described briefly below. This issue is taken up in more detail in Section 5.

4.5.1.2 Innovation and the needs of employers

While participants did not report that the needs of employers applied direct pressure on them to innovate, these needs did drive educational innovation in that they were mediated through the availability of new tools, the development of new content and student demand.

61

The development of ICT has provided teachers with a wide range of new tools that can be put to use in the delivery of higher education. In ICT education, the development of new technologies results in the development of new content, as knowledge about and skills in the latest ICT becomes part of the ICT curriculum. Participants were mindful of the changing needs of employers in terms of the knowledge and skills of ICT graduates. Many reported that students demanded to use the latest applications and to learn the latest languages. Reported sources of students' knowledge about the latest ICT developments included the Internet and advertisements in the employment sections of newspapers. Participants were aware of the fashions in ICT, as promoted through the popular media, but they were also aware of literature on graduate attributes and the needs of employers. Many spoke of a tension between satisfying the short-term desires of students to pursue current fashions and satisfying the long-term needs of industry for graduates with a foundation in theoretical principles and attributes required to participate in the workplace. Universities graduate attribute profiles were a reference point for many participants who hoped to produce graduates with long-term career prospects as ICT professionals.

4.6 Conclusions

• educational aims are the primary driving force in innovation in ICT education, particularly those answering the learning needs of students. Innovative teaching staff are motivated by a desire to be better teachers. Extrinsic motivation and reward for educational innovations are perceived by ICT educators as minimal. In fact, many associate being an innovative teacher with personal and professional risks. Institutional agendas and reward systems were seen by some to discourage educationally motivated innovation

• ICT educators perceive a tension between some of the demands of students (for particular skills and knowledge) and the skills and knowledge they believe are necessary for students to be ICT professionals and life-long learners. They report that students are often misinformed or have misconceptions about their future working roles and careers.

• ICT educators perceived students as generally conservative in terms of the types of teaching and learning activities they expected and desired in their courses. They reported that students were often resistant to innovative, non-traditional teaching.

• ICT educators perceive a tension between the short-term needs of employers and the skills and knowledge that they believe are necessary for students to become ICT professionals and life-long learners.

4.7 Recommendations arising from this section • Recommendation 7 That the management of university ICT departments

value teaching by

§ developing acceptable and practicable systems for the assessment and reward of educational innovation and good practice

62

§ supporting and encouraging the pursuit of educational research and publication in educational journals and conference proceedings

§ formally supporting, recognising and rewarding scholarship in teaching in the ICT disciplines. Examples of scholarship in teaching include the development of new tools, techniques and materials, and related evaluation activities.

§ encouraging staff to undertake educationally focussed professional development activities

§ allocating resources to these activities

• Recommendation 8 That the management of ICT departments support the development of innovative solutions to the challenges posed by the changing context of ICT education, such as increasing class sizes, increasingly diverse student populations and the increasing incidence of plagiarism.

Recommendation 9 That universities continue to give ICT educators the freedom to choose the particular teaching tools that they use, including the development of their own tools. While it is expected that each university will provide support for a limited range of courseware platforms, care should be taken to ensure that further innovation is not stifled by the promotion and support of specific tools.

63

5 Teaching and learning initiatives

5.1 Introduction This section responds to sections of the brief referring to innovations and national developments in ICT education. The data reported here was collected during the Consultation with ICT educators phase of the research (see Table 1 in Section 1). It provides an overview of the type of teaching and learning initiatives that are currently being undertaken by Australian ICT educators in terms of the motivation behind the initiatives and their aims or foci. It also discusses what was identified as a source of major weakness in many of the initiatives reported: evaluation.

5.2 Teaching and learning initiatives

5.2.1 Methods The data reported in this section was collected during the mini-conference program described in Section 4 and in Appendix D. The participants in this section of the research are the same as those described in Section 4 and listed in Appendix B. The section of the mini-conference proceedings that are relevant here are described briefly below, as is the approach taken to analysing the data collected on teaching and learning initiatives. The participants, data collection methods and analysis are described in more detail in Appendix D.

5.2.1.1 Data collection

At the mini-conferences, participants were asked to prepare short presentations on teaching and learning initiatives in which they were involved. Items 3 and 4 from the workbooks (see Appendix E) were used as stimuli for participants’ preparation of presentations. These prompts encouraged participants to describe the background, goals and lifecycle of the initiatives, the strengths and weaknesses of their initiatives and any planned, current or completed evaluation. As with other sessions in the mini-conferences, participants first made notes in their workbooks, then they presented to the group. Participants were given fifteen minutes3 to prepare responses to item 3, then ten minutes to present to the group. They were given ten minutes to prepare responses to item 4, then five minutes to present this section to the group. Data collected included notes made in participants’ workbooks and audio-recordings of presentations.

3 Times given here are approximate and varied depending on the size of the group and the total time available.

64

5.2.1.2 Analysis

The review process undertaken focussed on identifying and describing types and trends, rather than the tailored evaluation of specific innovations.

As a first response to the data collected at the mini-conference on particular teaching and learning initiatives, brief descriptive accounts were developed for each initiative. These accounts were then emailed to respective participants for approval. This approval process increased the validity of accounts, enabling clarification of details and the collection of additional information. It also enabled the acknowledgement of participants’ contributions and the appropriate attribution of initiatives. Approved accounts of teaching and learning initiatives were posted on the Project’s Web-site (http://cerg.infotech.monash.edu.au/icted/) and appear in Appendix J. In total, seventy-nine initiatives were described by the eighty-three ICT educators who participated in the mini-conference program. Of these, the accounts of fifty were later expressly approved. No accounts were expressly not approved.

Initiatives (both those that were approved and those that were not) were then categorised according to the reported motivation behind the initiative and the reported aim or focus of the initiative. Then the evaluation of the initiative, and any reported scholarship involved in the initiative, was reviewed. Initiatives were categorised according to the reported presence or absence of a formal evaluation, and the type of evaluation methods reported. It was also noted whether academic papers had been produced on the initiatives and whether the initiative was subject to an investigation linked to formal study, such as doctoral research.

5.2.2 Findings – Teaching and learning initiatives The findings of this part of the research are presented in two sections. First, the motivations and aims behind the teaching and learning initiatives are described. Many of the initiatives were multi-focal and intended to meet more than one purpose and, therefore, appear in more than one category.

Second, the evaluation of these initiatives, including evaluation methods, is summarised. Limitations of the reported evaluations are then discussed with reference to relevant literature.

5.2.2.1 Motivation and foci of initiatives

Both the motivation behind, and the foci of, the teaching and learning initiatives varied widely. This section provides an overview of these motivations and foci. The aim is not to highlight any one initiative, but to provide an overview of the types of initiatives that are being implemented. Initiatives given as examples have been numbered (see square brackets). Descriptions of many of these initiatives, along with the names and institutions of the innovators, can be found in Appendix J.

5.2.2.1.1 Motivation

The motivation behind the initiatives were grouped into four broad categories:

• Student needs

65

• Student employability

• Pedagogical Concerns

• Resources

Each category is described below.

5.2.2.1.1.1 Student needs

Specific student demands provided the impetus for a number of initiatives. Demands responded to by presenters included access to lecture materials prior to lectures [7] and industry certification courses [19, 71, 73]. Student feedback regarding satisfaction with course structure has also been responded to [79]. Other initiatives focussing on student needs related to problems attributed to large class sizes and diverse student skill levels, and to problems with student engagement and retention.

Student numbers

Recognition of the problems associated with increasing student numbers drove the implementation of many initiatives. Problems identified included the diminished quality of learning and of the 'university experience' in crowded classrooms [68, 72, 70] and the logistical difficulties associated with running subjects where student numbers exceeded 600 [41, 59]. While some initiatives have as their goal the management of increasing student numbers, others were undertaken to increase the number of students participating in ICT education [44].

Skill diversity

With increasing numbers of students comes increased diversity in student skill levels. A number of presenters were keen to ensure that the highly skilled students were challenged while middle to low-end students remained engaged in their subjects. Initiatives included the use of tailored, adaptable course materials and assessment processes [76, 35, 2, 26].

Student engagement and retention

Student engagement was the goal of a number of initiatives. Presenters recognised the link between engagement and performance. Difficulties with engagement were seen to be most prevalent in very large first year subjects, a number of initiatives targeting this area [36, 8]. Another area where engagement was seen as important was in online subjects where there is little opportunity for student interaction. One initiative was motivated by the need to engage students enrolled in online subjects [31]. Engagement problems in more senior students were reported in relation to complex course content [45].

A problem related to engagement was student retention. The problem of identifying those students at risk of failing or leaving their course is being addressed via well-monitored assessment tasks [34] and 'early warning systems' [77]. Increased opportunities for peer interaction were seen by some presenters as playing a pivotal role in student retention and was the focus of initiatives that made use of online interaction [58], mentoring/support frameworks [24] and peer assisted learning [70].

5.2.2.1.1.2 Student employability

Maximising the employability of graduates was the goal of many of initiatives involving both ICT and non-ICT students. A range of undergraduate activities were viewed as increasing ICT students' employability including industry

66

experience via placements and group project work with 'real' clients, exposure to professional standards and practices and certification in areas such as networking and information system administration. Generic skills identified as increasing employability included problem solving, group work, communication and life-long learning skills.

In some cases, initiatives involving these skills were driven by institutional 'graduate attributes' policies [82] and the institutional implementation of systems whereby students track their achievement of these attributes [67]. IT literacy was seen as increasing the employability of non-ICT students [35, 36, 41 80].

5.2.2.1.1.3 Pedagogical Concerns

A desire to improve the teaching of ICT motivated many presenters in their initiatives. Factors related to teaching improvement included curriculum integration, efficient assessment processes and the adoption of student-centred approaches.

Curriculum integration

Curriculum integration was seen as a method for optimising student learning opportunities and is being addressed at the program level through the application of professional standards and practices to coursework [11,15], the use of common teaching and learning tools across subjects within a degree [62] and the establishment of studio based learning environments [18, 30, 21, 6]. At the subject level, reviews of individual subject content are being conducted to maximise student learning opportunities by eliminating duplication of content across subjects [78, 69, 16].

Assessment

The objectivity and efficiency of assessment processes are being addressed through the application of professional standards to the assessment process [81]. The problem of plagiarism is being addressed through an investigation of students' understanding of cheating and the factors influencing plagiarism [29, 72] while the efficiency of assessment is being improved through the development of automated assessment systems [3]. Non-traditional assessment, for example journaling [27], is being used to increase student engagement and participation.

Student-centred learning

The goal of a number of initiatives is the introduction of student-centred learning principles to ICT teaching [32, 21, 6, 18, 30]. Others were driven by a desire to understand the effect of student learning style on learning outcomes [25, 38, 5]. Issues of ‘deep’ versus ‘surface’ learning were the impetus for a number of initiatives [45, 75,78] as was the desire to promote student responsibility for learning [17, 70].

5.2.2.1.1.4 Resources

Diminishing resources were often cited as the motivation for innovation. The cost-effective use of existing resources was seen as important for some [3, 4, 62] while the development of novel, cost efficient technologies that maximise flexibility, and consequently student numbers, was important to others [18, 28, 41, 44, 49, 66, 83].

5.2.2.1.2 Foci

67

The foci of initiatives were grouped into ten categories:

• Skills

• Student-centred teaching

• Group work

• Assessment

Certification

• Delivery modes

• Teaching tools

• Computer-mediated communication

• Collaboration

• Investigation

Each category is described below.

5.2.2.1.2.1 Skills

Teaching and learning initiatives focusing on developing students' skills [6, 8, 12, 14, 15, 18, 21, 25, 50] were of three main types: those focusing on problem solving and critical thinking, professional standards and practices, and learning skills.

Problem solving and critical thinking

A number of initiatives address the development of problem-solving skills. These cover general areas as well as discipline specific areas such as solving information system design or project management dilemmas. Initiatives focussing on problem solving skills are being implemented at both the subject and course level.

At the subject level, problem solving is addressed by a variety of means, including problem-based learning approaches [14], group work and real-life simulation [18]. Critical thinking skills have also been targeted through problem-solving activities, with students required to analyse their own and others' work in the completion of assignments [11, 15].

At the course level, problem solving and critical thinking are being encouraged in students through 'studio' learning environments. Based upon a social constructivist conception of learning, studio environments are intended to simulate the professional workplace and facilitate teamwork [6, 18, 21, 30].

Related to problem-solving skills is creativity. Creativity skills are being investigated and developed among engineering students, many who have preferred learning styles that do not lend themself to creativity and innovation. These students may also perceive creativity as 'being different', and not valued in the engineering world [25]. Creativity is also being developed in information systems students through the construction and solving of lateral thinking puzzles [8].

Professional standards and practices

Traditionally, students have been exposed to professional standards and practices through industry experience subjects and capstone projects. Newer approaches involve designing projects that simulate problems found in the real world [20]. Other initiatives have focused on teaching practices designed to instil students with a sense of personal responsibility for their actions and to develop students'

68

self-awareness in addition to their IT skills [78]. Still others, having identified professional challenges specific to female ICT graduates, have increased emphasis in their courses on interpersonal, communication and team-work skills [24].

Initiatives aimed at building professional skills included work done with software engineering students where students are required to conduct a year-long project within the professional framework described in the IEEE Software Engineering Standards document [81]. In doing this, students are exposed to the practical application of the professional standards and develop a shared professional language.

Another approach to exposing students to professional standards involves having fourth year students apply software engineering standards to the work of third year students as part of an assessable exercise [15]. Here, again, the students learn how to apply professional standards, not only to their own work, but also to the work of others and to identify areas of strength and weakness.

Real-life simulation in coursework has been extended from traditional group project work to include professional activity simulations such as student conferences (complete with paper submission processes, peer-reviewing and presentations) [45].

Professional practice subjects have been designed to introduce students to social, ethical and legal issues as they relate to the computing profession and to develop students' communication and teamwork skills via presentations and debates [47].

Learning skills

Initiatives aimed at developing learning skills are varied, with programs being developed to promote deep learning [45, 75], peer learning [73], effective study habits [70], self-paced/directed learning [73] and to introduce first year students to university-style learning (as contrasted with highly-directed secondary-school learning) [16]. Learning-focused initiatives include the implementation of experiential learning approaches, peer assisted study schemes and industry certification programs. Other initiatives have involved investigations into the preferred learning styles of ICT students with a view to developing more effective learning and teaching practices [25, 38].

5.2.2.1.2.2 Student-centred learning

Many of the initiatives presented reflect a shift from teacher- to student-centred learning with the teacher taking a facilitation role of the 'guide on the side' as opposed to the ‘sage on the stage’ deliverer of information.

An important aspect of university student-centred learning is student choice regarding educational goals, level and mode of participation. Furthermore, students are required to take responsibility and control of their own learning, attributes expected in the modern graduate.

Student-centred learning initiatives include an introductory computing subject in which compulsory practical classes are replaced by optional 'help sessions' where an instructor is present to answer questions relating to any computing topics - not just those specific to a particular subject [17]. A similar initiative involves a peer-assisted study where weekly lab-based sessions are conducted by paid undergraduate students of high academic standard [70]. Rather than focusing on subject specific material, these voluntary sessions focus on how to study for success in a subject.

69

Student choice is the focus of an initiative where the number of lectures is halved and problem-based learning opportunities increased [59]. In this initiative, lectures are delivered in the first half of semester, with students able to sit a pass-only exam at the end of this period. Students wishing to gain higher marks can choose to participate in problem-based learning for the remainder of the semester.

5.2.2.1.2.3 Group work

Group work activities, while a traditional feature of most ICT courses, have been extended to incorporate 'real-life' project group processes and the establishment of online work groups.

'Real-life' processes have been introduced in a software engineering initiative where students self-select into various project group roles according to job specifications, and, as a group, make formalised bids for projects offered by external and staff clients [20]. Studio-based initiatives also encourage professional group work practices by providing a physical environment that facilitates teamwork and simulates the professional work place [6, 18, 21, 30]. Peer oriented processes have been introduced into a computer design subject where project groups attend weekly project seminars and meetings at which they present their work in progress, share problems and receive feedback from their peers [10].

Computer supported collaborative work

A number of initiatives take a computer supported collaborative work (CSCW) approach to communication among project group members and among students in general. CSCW involves the formation of online work groups or learning communities, the members of which can engage in synchronous (asynchronous?) discussion, virtual conferences and secure meetings. CSCW initiatives make use of a range of online communication tools including instant messaging services, chat rooms, Multi-User Dungeons (MUDs) and groupware. In addition to facilitating communication, CSCW introduces students to professional work group practices [22, 37].

5.2.2.1.2.4 Assessment

Initiatives focussing on assessment issues include innovative assessment tasks and practices.

Innovative assessment tasks have been designed to aid in the identification of students at risk of falling behind. This is of particular importance where classes are large and students are from a range of educational backgrounds and of diverse skill level. In a cross-discipline information resource subject offered to both under- and post-graduate students, students are required to keep a journal which is submitted weekly allowing teachers to monitor each individual student’s understanding of the materials and progress through the course materials [36].

The time it takes a student to complete an individual laboratory-based software engineering task has been found to be related to the student's likelihood of successfully completing the whole subject. An initiative is underway to identify students at risk by way of Teacher Learner Collaborascope (TLC) technology developed by the University of Glasgow [77]. Using this technology, teachers can track the length of time it takes individual students to complete lab-based tasks and allocate teaching resources accordingly.

Initiatives are also underway to automate the process for assessing programming assignments [3, 79]. Systems are being developed for the online submission and marking of students' programs where students upload their programs to a

70

UNIX/PERL host. The system automatically generates marks reflecting the success of the program and emails feedback directly to the student.

Related to assessment is the problem of plagiarism among ICT students. A research initiative is currently exploring students' understanding of plagiarism and factors that may lead students to plagiarise [29].

5.2.2.1.2.5 Certification

Student and industry demand for specific industry certification has led some universities to incorporate certification courses into their undergraduate degree programs [19, 71, 73]. Certification courses are an optional component of these undergraduate courses, are delivered online and are offered as full-fee paying programs. Certification is offered in information systems and networking.

5.2.2.1.2.6 Delivery modes

Many of the initiatives reported reflect a move from the traditional lecture-tutorial mode of course delivery to modes that utilise the latest in ICT, in particular, the Internet, electronic media and video-conferencing. Cutting edge technology, such as Wireless Application Protocol (WAP) and Personal Digital Assistants (PDAs), is also being utilised in the flexible delivery of courses.

A number of courses across the spectrum of ICT are now being offered partially or totally online [4, 5, 13, 23, 46, 49, 55, 64, 66, 68,74]. Online delivery aims for equitable access to students whether they are local, distance or off-shore as it is time and place independent. Using commercial web-based courseware, online delivery allows for the presentation of subject material, assignment submission, student-staff and student-student communication, and subject management and monitoring. Other features of online subjects include video-streaming, instant-messaging and course-related chat rooms.

Other media employed in the flexible delivery of courses include CD-ROMs containing all course materials and digital video recordings of live lectures [41]. Video-conferencing technology is also being employed for synchronous delivery of lectures to students across campuses [61].

Many initiatives involve the delivery of courses by multiple- or mixed- modes [26, 32, 39, 42,44, 57] in an effort to maximise accessibility and student choice. This may include the presentation of a course in more than one form (eg., on-campus and online) or as a combination of face-to-face lectures and tutorials, web-based delivery and synchronous and asynchronous electronic interaction.

5.2.2.1.2.7 Teaching tools

A range of teaching tools have been developed to facilitate the teaching of complex material and maintain the interest of students [1, 8, 28, 33, 34, 35, 38, 62, 63, 65, 69 76].

Integrative teaching tools are being introduced across subjects as a means of simplifying manual tasks [69] and providing a consistent learning environment, thus integrating learning so that students can focus on the content of their subjects rather than on mastering a number of different tools [28, 62]. Teaching tools employed across subjects and courses include embedded system learning environments, network protocol simulators and process-oriented educational tools that allow teachers to build educational packages.

Other tools aim at maintaining student interest by presenting material in non-traditional forms [76]. For example, puzzles are being used in knowledge-based

71

information systems lectures to increase student participation and to encourage lateral thinking and problem solving [8], while in networking subjects students work with open-source software and assess each others' work [34].

5.2.2.1.2.8 Computer-mediated communication

Methods of communication that are both time and place independent have flourished along with the movement toward flexible delivery. The use of both synchronous and asynchronous computer-mediated communication (CMC) methods feature in many of the initiatives reported. CMC is employed to enable interaction among geographically dispersed workgroups [22, 37, 53] and to encourage communication among isolated, distance and off-shore students and their teachers [1, 58]. At a more local level, regular cross-campus interactive chat sessions and subject bulletin-boards can be used to discuss course content and post questions [46].

5.2.2.1.2.9 Collaboration

Educational collaborations included those between teachers and those between the university and industry.

Teacher-teacher collaborations included team teaching initiatives where teachers share the administrative and teaching responsibility for a given subject [40, 60]. Other teacher-teacher collaborations included the establishment of faculty wide teaching interest groups with the purpose of sharing and developing teaching innovations [54].

A university-industry collaboration which is in the planning stage involves the development of a professional speech and language technology course [51] and the establishment of a commercial company where industry partners will benefit from access to the expertise of university staff, university staff will benefit from access to industry resources and opportunities and students will gain experience of working on authentic projects [52].

5.2.2.1.2.10 Investigation

For some ICT educators, their initiative formed part of a formal investigation into a particular teaching and learning innovation and for others, their initiative formed part of their higher degree research program. Initiatives involving formal investigative procedures included:

• an investigation of the impact of class project meetings and class project seminars on student learning and product design quality [10].

• an investigation of students' understanding of plagiarism and the factors influencing cheating behaviour [29].

• an investigation of students' preferred learning style and how this impacts upon their navigation of complex software packages [38].

• an evaluation of the effectiveness of a problem-based learning approach to first-year programming [43].

• an exploration of how the culture surrounding IT capstone subjects can be changed to promote peer learning and 'learning organisation' modes [56].

• an investigation of the educative effects of making lecture material available to students prior to lectures [7].

72

• an evaluation of the effectiveness of an 'early warning system' to aid in the identification of students at risk of failing or falling behind academically [77].

• an exploration of methods for structuring effective co-operation between online learners [9].

• an evaluation of electronic delivery modes for urban, rural and remote north Australia [48]

5.2.2.2 Evaluation

This section summarises the approaches taken by ICT educators to the evaluation of their teaching and learning initiatives. The evaluation of initiatives was identified as a major source of weakness in many of the initiatives described. This limitation is discussed with reference to the literature on evaluation of teaching.

5.2.2.2.1 Evaluations reported at the mini-conferences

Of the seventy-nine initiatives described, seventy-one were reported to be the subject of planned, current or completed evaluations. Fifty of these evaluations involved the use of standard student evaluation of teaching (SET) questionnaires. Of these fifty evaluations, forty-two relied on SET questionnaires alone. Thirty-seven reported the involvement of other evaluation methods, including the content analysis of online communications, experimental design, focus group research, interviews, questionnaires targeting the objectives of the initiative, peer review, the analysis of statistics (eg. enrolment, completion), monitoring student exams and student journals. Twenty-seven of the eighty-three participants reported having published or presented academic papers on their initiatives. Seven were currently enrolled or planned to enrol in an award course (PhD or Graduate Diploma in Higher Education) in which the initiative would be the subject of an investigation. In total, nine participants explicitly identified their initiatives as the subject of a formal investigation4.

5.2.2.2.1.1 Dependence on SET type questionnaires

While the dependence of the evaluation of many of the reported initiatives relying on SET questionnaires alone is not surprising, as this reflects the findings of other reviews, it is cause for some concern. It was evident in the discussions that took place at the mini-conferences that many participants equated evaluation with this type of student feedback exercise. One participant wrote, in response the question about planned, current or completed evaluations: "There can be problems with too much evaluation, students can get fed up with them (e.g. questionnaires on every subject) and therefore results may be of little use" (Workbook excerpt). Similarly, another participant wrote, "[Our university] has a standard program for this - the SET/SEU scheme" (Workbook excerpt).

While SET type questionnaires are a valuable source of feedback for teachers and are recognised in the literature to be a valid indication of student learning (Moses & Trigwell 1993), the limitations of student feedback in terms of the evaluation of teaching and learning initiatives are well recognized (see discussion in Appendix I on Movements in university teaching and learning), and the reliance on student feedback alone has been widely criticized. In their review of CAUT

4 The figures given here are not representative of ICT educators in general but are based on a group of self-selected ICT educators with a particular interest in educational innovation.

73

funded ICT-based projects, Alexander and McKenzie concluded that "the inadequacy of evaluation methods appears to have resulted in project leaders being unable to report on the outcomes of projects other than those related to student reactions to projects" (Alexander & McKenzie 1998, p.253). The trends observed in the evaluation of initiatives reported during the mini-conference program are consistent with those observed by Alexander and McKenzie in that many evaluations were not designed to determine whether the initiatives had produced their intended outcomes. Similarly, Cosser was critical of the role student evaluations of teaching has played in what he calls "the disjunction of formative and summative evaluation", observing that "student evaluation of teaching has come almost to preclude other forms of assessment" (Cosser 1998, p.143).

The limitations of SET type questionnaires were discussed by mini-conference participants, with one participant concluding, "I don't think it is a tool that gathers impressions of what's going on in your classroom. I don't think it is enough" (MC Transcript excerpt). Another participant reflected,

It's really sad. We've really had 10 years of pretty regular class surveys now in Australia. It's become a fairly standard thing and we've stopped there. If we get a minimal level of satisfaction, we think 'phew, that's good' and we stopped there. We need to do more than that. (MC transcript)

Criticisms made by mini-conference participants of SET type questionnaires were consistent with those found in academic commentary on the subject. In particular, participants recognised that, as Koder put it at the 1998 Course Experience Questionnaire Symposium, "satisfaction and mediocrity are comfortable companions [and] those who accept that 'challenge' is part of the learning environment are less comfortable with satisfaction" (Koder 1999, Online document). Mini-conference participants repeatedly described the "conservatism" of students who were resistant to non-traditional teaching methods. Student resistance to more challenging teaching and learning activities was identified as one of the factors that drive/inhibit innovation in ICT education (see Section 4). Comments made included the following:

If the innovation is genuinely pedagogically challenging and the students don't appear to like it, then the institutional reaction is to revert to delivery because that's safe. So you don't necessarily get proper institutional support for innovation I don't think. Because the students are often conservative. ... I mean of course you can do some little innovation if it's like put your material on the web say. That's not very threatening, because students love that because it's all about the delivery rather than the teaching and learning, but [not] if you do something that's a bit way out, even if though it's pedagogically really well known and understood. (MC excerpt)

Well sometimes if you want to do something but it's very radical, there'll be such an outcry either from the heavies who think that the students are all going to yell and scream or there's students who don't want to be part of this radicalism. (MC excerpt)

Participants generally recognised the need to evaluate their initiatives, for formative evaluation of the initiative, the evaluation in terms of the learning outcomes and for dissemination purposes, and they recognised the limitations of

74

relying solely on SET type questionnaires. However, this type of evaluation still predominated. Possible explanations for this predominance, such as lack of training, lack of reward and lack of resources for the conduct of more tailored evaluations, are discussed in the next section on the dissemination of innovation.

5.3 Innovation and best practice Seventy-nine teaching and learning initiatives were presented at the min-conferences. No evaluation has been made of their possible level of innovation or their role in defining best educational practice. Rogers (1983) points out that judgements about whether a particular idea is innovative are subjective. Definitions of innovation usually require the idea to be new but “it matters little, so far as human behavior is concerned, whether or not an idea is 'objectively' new … If the idea seems new to an individual, it is an innovation” (Rogers 1983, p.11). It is more feasible, however, to determine whether an idea is seen as innovative within a particular community because individuals within a community have a degree of shared interests, shared experiences and familiarity with activities within that community.

On the suggestion of Dianne Hagan, a small sub-project was initiated to identify which of the teaching and learning initiatives would be viewed by ICT educators as innovative and/or as best practice . Mini-conference participants reported initiatives that reflected developments in their departments. The sub-project aimed to obtain the view of a group of ICT educators on the degree to which they believed the initiatives were examples of innovation and best practice. The methods used are outlined below.

5.3.1 Methods First, ICT educators were consulted as to the criteria that they would use to define innovation and best practice . All members of Monash’s Computing Education Research Group (CERG) were emailed with the following request:

We are asking you, as a teacher and educator in Information Technology, to give your ideas on all or some of the following 3 areas:

1. What criteria you might use to define innovative educational activities?

2. What criteria you might use to define best educational practice?

Responses were collected via email. Twelve of the twenty active members of CERG replied. The set of criteria that was extracted from the responses are given below:

Popular criteria for the definition of an innovation were

3. Has not been done before

4. - or has not been done by majority

5. Valuable/important to helping learner-based outcomes

75

6. Has been developed to a point where it is not simply an idea. There have been participants who have experienced the initiative

7. Is practicable in educational and infrastructure terms – it can be done without having vast/specific resources beyond the norm

8. Evaluation may have been carried

Other criteria, which were mentioned by respondents, were

• Innovative relative to its educational context

• Engenders comment and discussion

• Publishable

• Reaching to new markets

Popular criteria for the definition of best practice were

1. Has been formally evaluated in terms of educational outcomes and teaching outcomes.

2. Program’s relevance to the student/learner population

- ability to involve them

- oriented to student-based learning

3. Educational quality of the initiative, including integration into a wider educational program

Other criteria, which were mentioned by respondents, were

• Positive impact on student performance and pass rates

• Best teaching practice, including that it can be implemented on time and fully prepared

• Aligned to external needs, including industry

• Enhances equity and access issues

5.3.1.1 Selecting Initiatives

Forty-one of the seventy-nine initiatives were then presented to CERG members. Initiatives were omitted from this process if they were

• not specifically targeting ICT education

• were still in the planning stage

• had been discontinued

Initiatives that were retained included:

• 3 implemented in electronic engineering contexts

• 15 in information systems contexts

• 23 in computer science and software engineering contexts

76

The thirty-eight initiatives not included in this process included

• those initiatives being run at the university level, for example, a program targeting graduate skills [67]

• many initiatives that were still under development, for example, the planned restructuring of a database subject [59]

• initiatives that had been discontinued, for example, mixed-mode teaching that was discontinued based on an evaluation [44].

5.3.1.2 Staff Ratings

CERG members from the appropriate disciplines were sent up to six of the forty-one initiatives and asked to indicate whether they saw each initiative as an innovation or as best practice. Ratings were

For innovation:

• Is an innovation

• Not sure/Maybe/Indifferent

• Is not an innovation

For best practice : Is best practice Not sure/Maybe/Indifferent Is not best practice

Initiatives that were categorised by a majority of staff members as innovation or as best practice are described below.

5.3.2 Findings – Innovation and best practice

5.3.2.1 Initiatives identified as innovation and/or best practice

Eleven of the initiatives were identified by ICT educators as innovation. Fifteen were identified as best practice . Five of these initiatives were identified as both innovation and best practice.

Areas addressed by those initiatives identified as innovations included student retention and student engagement, enhancing learning, catering for groups of students with diverse skills, accommodating large class sizes, curriculum integration and teaching skills associated with professional practice.

Areas addressed by those initiatives identified as best practice included student retention, attracting and retaining female students, enhancing learning, catering with groups of students with diverse skills, accommodating large class sizes, teaching skills associated with professional practice and teaching learning skills.

Many of the foci of these initiatives reflect those factors reported by ICT educators as driving innovation, such as increasing class sizes and the increasing diversity of students (see Section 4).

The initiatives identified are described briefly in Table 10. They are grouped according to the ratings they received: Innovation, Innovation and best practice , and best practice.

77

Table 10. Initiatives identified as innovation and/or best practice

Rating Discipline context

Brief description of initiative

Innovation and best practice

CSSE Keywords: Teaching Tools - Learning Styles

Description: In an effort to increase understanding of student learning styles and to improve teaching practices, student learning of large, complex software packages is being investigated. The initial research uses Rational Rose®, a professional Software Engineering CASE tool. As part of the first year foundation unit, learning style inventories are administrated to all students, who then identify their self-reported learning styles. An online tracking tool has been developed to monitor usage and movement through the CASE tool software. The goal of this initiative is to examine the relationship between learning styles and the actual methods of usage of the software by students, as revealed by the tracking process. In this way, it is hoped that a learning methodology can be developed by which both students and lecturers become aware of the impact of learning styles on the learning process.

Evaluation:

• quantitative research focussed questionnaires

• statistical records

• peer review during workshops

Publications: Yes

Innovation and best practice

CSSE Keywords: Teaching Tools - Simulations

Description: A network protocol simulator has been developed for use in undergraduate computer networking courses. In development since 1992, this tool comprises simulation tools that allow students to experiment with various aspects of networking within a laboratory environment. It can currently be run on UNIX and Linux platforms.

Evaluation:

• student feedback questionnaires

• peer review

Publications: Yes

78

Rating Discipline context

Brief description of initiative

Innovation and best practice

CSSE Keywords: Assessment - Student Retention

Description: Research findings suggest that the time it takes a first-year student to complete a lab-based software engineering task is related to their likelihood of successfully completing the whole subject. To test this hypothesis, an 'early warning system' has been developed. With this system, the length of time it takes individual students to complete lab-based tasks can be tracked. It is anticipated that this information will aid in the identification of those students at risk of failing the subject and the allocation of teaching resources to those students most in need.

Evaluation:

• student questionnaire

• peer review

Publications: Yes

Innovation and best practice

IS/IM Keywords: Flexible Delivery - Self-Paced Learning

Description: In an effort to overcome the problems of variable student skill level and learning style, self-paced learning materials have been introduced into an introductory computing subject. Students' take control of their own learning and, instead of attending regular weekly practical classes, may attend optional 'help sessions' where an instructor is present to answer questions relating to any computing topics. Replacing regular practical classes with help sessions also reduces the costs of holding classes that may be poorly attended and eliminates the problem of students who miss classes, missing important material.

Evaluation:

• student feedback questionnaires

Publications: Yes

Innovation and best practice

IS/IM Keywords: Problem-based Learning - Assessment

Description: A large (approx 600 students) database subject has been redesigned. The subject is currently taught in a traditional lecture/tutorial/assignment mode and draws heavily on the Department's resources. The redesign involves reducing the number of lecturers by half and incorporating student-centred, problem-based learning. Lectures will be delivered in the first half of semester, with students able to sit a pass-only exam at the end of this period. Students wishing to gain higher marks will participate in problem-based learning for the remainder of the semester.

Evaluation:

• student feedback questionnaires

• student interviews

Publications: No

79

Rating Discipline context

Brief description of initiative

Innovation CSSE Keywords: Course materials - Teaching tools

Description: In an effort to increase student engagement with their courses, a range of interactive techniques have been integrated into the teaching of a networking subject. The subject's website is incorporated into lectures by browsing the relevant pages 'live' during lectures. Students are encouraged and rewarded for asking questions in the lectures. Quizzes are conducted during lectures where students mark each other's work. In laboratory classes, open source software is used so students complete assignments based on 'real' code. All software used to teach labs is also available to students at home via the subject's password protected website.

Evaluation:

• student feedback questionnaire

Publications: No

Innovation CSSE Keywords: Communication skills - Experiential learning

Description: A first year core subject on professional computing has been developed to meet industry's demand for graduates with well developed professional and communication skills. Students taking this subject examine social, ethical and legal issues as they relate to the computing profession and develop communication and teamwork skills via presentations and debates. In addition, students are required to dismantle and reassemble a computer under supervision and are exposed to a range of programming languages as a way of introducing them to the reality of the profession.

Evaluation:

• student feedback questionnaire

Publications: Yes

80

Rating Discipline context

Brief description of initiative

Innovation CSSE Keywords: Teaching Tools - Curriculum Integration

Description: In the past, there has been inconsistency in the type of development tools employed across a software engineering program. As a result, students' time was taken up with mastering the various tools when it could have been better spent concentrating on subject content. Three years ago, the development of an embedded system learning environment began. The aim was to develop an environment that could be used effectively across the whole engineering program. A common tool has now been implemented across the program for both on- and off-shore students.

Evaluation:

• student feedback questionnaire

• peer review

Publications: No

Innovation IS/IM Keywords: Computer-mediated Communication - Teaching Tools

Description: A computer supported collaborative work (CSCW) approach has been adopted for teaching OO programming concepts and virtual world building. The goal of the initiative is to train students in professional workgroup practices. Using as its infrastructure, LC_MOO, an object oriented MUD learning community, students form online workgroup communities where they can engage in synchronous discussion, virtual conferences and secure meetings.

Evaluation:

• student feedback questionnaires

• peer review of resources

Publications: Yes

Innovation IS/IM Keywords: Experiential Learning - Small Group Teaching

Description: Dissatisfied with the lecture/tutorial teaching format, a small group teaching approach has been adopted for teaching a second year information systems subject. The aim of the approach is to encourage students to take a deep approach to their learning and promote conceptual understanding through learning activities designed to teach both subject-specific content and general learning skills. Students attend workshops and tutorials in which they undertake experiential learning tasks, working in groups of 3 or 4.

Evaluation:

• student feedback questionnaires, with both quantitative and qualitative items

Publications: Yes

81

Rating Discipline context

Brief description of initiative

Innovation ElecEng Keywords: Peer Learning - Assessment

Description: It was observed that students tended to be passive in learning situations in a conceptually difficult third-year subject on electro-motion. Student Conferences are now used as a means to develop students' initiative and to encourage deep learning in the subject. Early in the semester, a "call for papers" is put out and students are provided with a list of possible topics. The students' task is to write a literature review one of the topics. They submit abstracts, have their papers peer reviewed and present their papers at the Student Conference. Papers are then published in the conference proceedings. The students run the editorial and review processes and organise the conference.

Evaluation:

• student feedback questionnaire

• peer review

Publications: Yes

Best practice CSSE Keywords: Capstone Subjects - Experiential Learning

Description: To prepare final-year students for the reality of the IT workplace, a capstone subject has been developed in which students work in groups of 10 or more on a software development project. Project groups are self-selected to meet the job specifications for various roles, and make formalised bids for the projects offered by external and staff clients. Group members appoint a project manager who is responsible for assigning tasks and the overall running of the group. Professional communication skills are developed through weekly formal project meetings and end of project presentations and product displays.

Evaluation: Not yet evaluated

Publications: No

82

Rating Discipline context

Brief description of initiative

Best practice CSSE Keywords: Teaching Tools - Maths Education

Description: It has been observed that student interest and skill level in compulsory subjects on discrete mathematics and statistics vary considerably. To ensure that the top students are challenged and the middle to low-end students remain engaged in the subject, spreadsheet software is used to teach topics such as calculus, combinatorics, induction, and recurrences. Concepts can be usefully demonstrated using tables, graphs, patterns and colours. An investigative approach is encouraged, and it is possible to illustrate a wide range of fundamental math principles with minimal algebra.

Evaluation:

• student feedback questionnaire

Publications: Yes

Best practice CSSE Keywords: Academic skills - Student needs

Description: In response to the variability of academic skill levels observed in students, the teaching of writing skills has been incorporated into a second year subject as part of an information technology degree. The subject focuses on developing students' skills in thinking, research and report and essay writing. It draws on expertise from other divisions in the University, including the library and the learning support units.

Evaluation:

• student feedback questionnaire

Publications: Yes

Best practice CSSE Keywords: Curriculum Integration - Peer Learning

Description: Accredition of software engineering courses by the Institute of Engineers requires that graduates be familiar with the major parts of the software development life-cycle. To this end, third year students are required, individually and in groups, to analyse, design and implement a software system. Fourth-year students then inspect the third year students' work. In this way, students gain experience in both the development of project specifications and the quality assurance issues.

Evaluation:

• student feedback questionnaire

• industry feedback

Publications: No

83

Rating Discipline context

Brief description of initiative

Best practice CSSE Keywords: Industry experience - Mentoring

Description: Students' learning during a semester length industry experience program is monitored in regular meetings between each student, his/her mentor (teacher) and a member of the industry organisation. The program allows mentors, students and organisations to tailor each student's experience according to the possible opportunities in the organisation.

Evaluation:

• formative via student-mentor-industry consultation

Publications: No

Best practice CSSE Keywords: First Year Students - Teaching Tools

Description: A first year programming subject has been redesigned in response to low student satisfaction and performance. The programming language taught has changed from C to VB, resulting in students producing higher quality programs and taking a greater interest in conceptual issues rather than procedural problems. Concurrent with the change in language was a shift to a more practical teaching approach; in addition to automated marking and the online delivery of subject materials, collaborative learning amongst students has been encouraged through the use of online discussion and chats and in-class desk-checking.

Evaluation:

• student feedback questionnaire

• exam results records

Publications: No

Best practice IS/IM Key words: Experiential Learning - Problem-Based Learning

Description: It was observed that final year students did not have good project management skills and the ability to apply/integrate prior knowledge in undertaking a real project. In response, a problem-based learning approach was developed in an introductory IS analysis and design subject. This subject is delivered primarily through students' group work as they develop solutions to given problems. Problems are designed to simulate those found in the professional world, each taking from one to five weeks to complete.

Evaluation:

• student interviews

Publications: Yes

84

Rating Discipline context

Brief description of initiative

Best practice IS/IM Keywords: Access and Equity - Females in Computing

Description: A number of initiatives have been implemented to increase the enrolments and retention of females in computing courses. They include Girls into Computing days, the development of promotional videos and a mentoring/support scheme. To better prepare female computing students for professional life courses contain an increased emphasis on interpersonal, communication and team-work skills.

Evaluation:

• student interviews

• statistical records

Publications: Yes

Best practice IS/IM Keywords: Student Journals - Graduate Attributes

Description: Within an information resources subject, a learning environment has been developed to help students broaden their knowledge base, and yet allow a certain flexibility to develop their own independent interests in the field of IT. In the unit, diaries and team projects are used to foster information literacy, particularly the development of communication and practical problem solving skills. Students learn to identify, retrieve and evaluate print and electronic business information resources that are relevant to a variety of problems. The aim of this subject is to encourage students to become information literate, Internet literate and resources literate, to develop good communication skills, the ability to work in teams and skills in the selection and evaluation of information. The primary method used to help students to develop these skills is a reflective journal that underpins the students' work throughout the semester.

Evaluation:

• student feedback questionnaires

• student journals

Publications: Yes

85

Rating Discipline context

Brief description of initiative

Best practice IS/IM Key words: Communication skills - Computer-Mediated Communication

Description: A team-work approach has been taken to a third year project management unit. Students work in teams of approximately ten to manage authentic projects. The teams use computer-mediated communication (via First Class®) to manage their work, conducting virtual meetings. This approach is designed to emphasise the importance of communication in project management. It takes the emphasis away from the product, putting it on the project management process, communication between team members and communication with the client.

Evaluation:

• student feedback questionnaires

Publications: Yes

5.3.2.2 Discussion

The process that was implemented here was aimed at creating some level of external, practitioner evaluation of the status of initiatives presented as the mini-conferences. The task was limited by the amount of data that the evaluators had available to them. This issue was raised by the evaluators:

“One would need to see the materials to determine how student-centred they were to determine if this was best practice.”

The significance of the task is the identification of the type and focus of initiatives that they viewed as innovative or as best practice. In this way, the task says more about the views of ICT educators than about the initiatives. Evaluators’ comments indicated that they were focussed upon the teaching/learning value of initiatives:

“This sounds like just putting stuff on the Web. No focus upon teaching/learning outcomes.”

Others made wider pedagogical judgments, given their background and knowledge:

It shouldn’t be innovative – it should be standard practice to improve quality.

The conclusion that can be drawn from this process is that it has helped gain a better picture of the wider context of what might be seen as innovation and best practice. That is, local educators may be instituting innovation and best practice within their particular environments but that this may not reflect the perspectives of practitioners in other institutions with different needs and local educational contexts.

5.4 Conclusions

86

• The impetus for, and aims of, the majority of the teaching and learning initiatives reported are student-focused, with many responding to needs or problems encountered by students and a desire to improve the teaching of ICT.

• Other initiatives were motivated from the changing context of university teaching that reduced resources available, increased student numbers and required support of a more diverse student population, or changing needs of employers.

• Most initiatives were formally evaluated, however, for a large proportion of these initiatives, standard student feedback questionnaires were the only source of evaluation

• Participants recognised the limitations of relying on student feedback questionnaires alone for the evaluation of teaching and learning initiatives

• Given the lack of reward and the lack of skills, many ICT educators chose not to pursue formal evaluations further than the standard SET type questionnaires required by their departments

• Many ICT educators underestimate the significance of their teaching practice as innovation and they use informal means to evaluate and improve their teaching, rather than pursuing evaluation methods that better qualify their work for publication as educational research and better demonstrate the scholarship in their teaching

87

5.5 Recommendations arising from this section

• Recommendation 11 That management encourage ICT educators to evaluate their teaching and learning initiatives through

§ the use of evaluations, both formative and summative, that focus on investigating links between the particular objectives of initiatives and educational outcomes.

§ the provision of professional development and support programs that focus on educational evaluation as well as on teaching tools and methodologies. Programs should be developed in consultation with available experts, for example, educators from university Education departments and academic development units.

§ the user-based evaluation of the services and programs described above to ensure that they meet the needs of ICT educators.

§ the allocation of resources to support these activities

89

6 ICT educators’ views on dissemination of innovation

6.1 Introduction This section presents further data collected as part of the phase of the research referred to in the Project Specifications as Consultation with ICT educators (see Table 1, Section 1). It responds to sections of the Brief requiring that recommendations be made for the effective dissemination and take-up of educational innovations. The data described in this section were collected via the mini-conference program (described in Appendix D) during open discussions on the dissemination of innovation. Before the data is described the methods used are outlined.

6.2 Methods The data reported in this section was collected during the mini-conference program described in Section 3 and in Appendix D. The participants in this section of the research are the same as those described in Section 3 and listed in Appendix B. The section of the mini-conference proceedings that are relevant here are described briefly below, as is the approach taken to analysing the data collected ICT educators’ views of the dissemination of educational innovations.

6.2.1 Data collection As with that reported in Section 3, the data reported here was collected in the context of open discussions between ICT educators who participated in the mini-conference program. The stimulus for these discussions was Item 5: “What issues would be involved in disseminating your initiative to other ICT educators?” Participants were prompted to consider both issues involved in the dissemination of particular initiatives that they had earlier presented and of educational innovations generally. They were given five minutes to think about the issues and to make notes in their workbooks. The topic was then opened for discussion. Data sources included the audio-recordings of discussions, whiteboard records of discussions and participants’ workbook entries.

6.2.2 Analysis The analysis method used for the data on dissemination was the same described for the analysis of other open-discussion data. It involved the forming of categories from whiteboard records and workbook entries, then the testing of these categories against transcriptions of the audio-recordings using a constant

90

comparative method to refine categories. This process is described in more detail in Appendix D.

6.3 Findings – dissemination of innovation The mini-conference participants reported a range of issues thought to impact on the dissemination of educational innovation among ICT educators. They are presented below in two sections: The first section describes those characteristics of ICT education seen to impact on dissemination. The second section concerns effective channels of dissemination and qualities identified by ICT educators as contributing to effective dissemination.

6.3.1 Contextual issues impacting on the dissemination of educational innovation Issues identified by ICT educators as influencing the dissemination of innovation among university educators included:

• ICT educators’ level of training in educational evaluation

• the status of teaching compared with research

• intellectual property issues

• inter-university competition

• professional risks

• resources

6.3.1.1 Training in evaluation, learning theory and educational terminology

Many mini-conference participants reported not possessing adequate training in research design and educational evaluation methods. This was seen as inhibiting their ability to communicate the value of an innovation and to identify the effective innovations of others.

Daniels et al. (1998) point out that, in the dissemination of educational innovations, it is “especially important to distinguish truth from assumption, to have practice that is well-founded. Evolving teaching practice is normal to good teaching, but evaluation reliant on anecdote is not good enough” (p.204).

While most mini-conference participants have, or were, engaged in some form of educational initiative, many were unsure about how they should go about evaluating their initiative. This problem is not isolated to ICT education, but appears to be a wide-spread problem in the reporting of educational innovation in the university sector. Particularly when innovations incorporate the use of new technologies, there is a tendency to report description rather than evaluation. In addition, there is trend toward the reporting of innovations that are ‘good’, that ‘worked’ or that the ‘students seemed to like’, with no account of educational outcomes. Participants understood this problem as being due to university educators’ general lack of training in learning theory and educational evaluation. Although a number of participants were originally trained as secondary school teachers or had completed graduate certificates in higher education teaching,

91

many felt that they lacked the research skills to adequately evaluate their own innovations and to critically assess the reporting of others. As one participant stated, ICT educators should be “reporting rigour, not rosiness”.

Weaknesses in the evaluation and reporting of educational innovation are discussed in more detail in Section 6.

Participants also discussed the ‘discipline-bound’ language used in ICT education; the language is often technical. This phenomenon is true for most disciplines which, over time, have developed their own style of discourse and accompanying technical terms. The translation of ideas from a discipline-bound language to one that can be understood across disciplines and institutions was seen by participants as a challenge for innovative ICT educators. Issues of educational jargon were also raised in relation to the identification of innovations originating outside of ICT education. In particular, educational journals were seen to contain much language that was not accessible to ICT educators without formal training in educational theory. This inhibited both the accessibility of ideas reported in these forums and the dissemination of innovation, generated by ICT educators, through these channels to the wider educational community.

A related issue is that of naivety. Ignorance was brought up a number of times during the mini-conference program. Many participants stated that their engagement in dissemination activities was inhibited by their not recognising themselves as innovative. This is supported by research that identifies ignorance as a key barrier to knowledge transfer. Szulanski (1994) found that, for effective dissemination to occur, individuals need to recognise that they possess knowledge that would be of interest or use to others. Related to this is the issue of recognising your audience. Participants reported that identifying yourself as innovative was tied in with belonging to a community in which your initiatives would be seen as innovative; the idea of dissemination is meaningless where there is no identified audience or interested community. The identification of an idea as an innovation is subjective, depending on the perspective of the community: what is innovative for one community may not be so for another.

6.3.1.2 Teaching vs. research

The most pervasive factor reported by mini-conference participants as influencing the dissemination of educational innovation is the lowly status of teaching, compared to research, in most university ICT departments. On the whole, participants saw innovative teaching as not being highly valued in a higher education environment where a) research is seen as attracting funding and teaching is seen as not attracting funding, even though the majority of the funding received by universities relates to their teaching rather than to their research, and b) promotion is based, for the most part, on an individual’s research publication record. It was generally agreed by participants that such an environment discouraged, rather than promoted, the dissemination of teaching and learning innovations. In this context, activities involved in the generation, adoption or dissemination of educational innovations take up precious time that could be more profitably spent engaged in better rewarded activities, such as research. The following quote encapsulates many participants’ perceptions of the value placed on the dissemination of teaching innovations in Australia’s universities.

How many faculty members [do you] actually see going to an educational conference or … reading an educational journal as a

92

high priority? We’re getting back to this cultural thing in universities that the research is important and I’ll go and read 75 journals … and I won’t open one on education because teaching’s only what I do so that I can do research. (M-C transcript)

There is a second layer to this problem. Mini-conference participants saw research into technical, discipline-specific issues as being valued over research into teaching and learning. That is, they perceived research into educational innovations and pedagogical issues as not being seen as ‘important’, or attracting the same level of recognition and reward as research into technological innovations. Without reward for the generation or adoption of educational innovations, only those driven by intrinsic motivations are likely to pursue the improvement of teaching and learning. Dissemination is less likely to occur where the individual sees no professional reason for sharing their ideas (Szulanski 1994).

6.3.1.3 Intellectual property

Issues surrounding intellectual property (IP) are complex. IP issues were consistently reported as restricting dissemination. The constraints of IP policies were most often described in the context of the production of online teaching materials.

Recent years have seen an increase in the amount of course content delivered online. Online material often includes lecture notes, tutorial activities and administrative material such as subject guides and information regarding assessment, university policies and contact information. Where courses are developed for off-shore delivery, it is becoming more common for students to access all course material online and to use electronic media as their sole means of contact with teaching staff and other students.

Participants expressed concern about the lack of intellectual ownership of course materials that they have developed and which have then been packaged by their university into electronic form. The impact this has on dissemination is two-fold. First, the material becomes the outright property of the developer’s university whereas, when the material is delivered ‘live’, the developer is relatively free to alter the content or its delivery to suit the needs of students. Second, the developers of the material lose control of the distribution and use and potential misuse of their materials when delivered online. This is especially so where access to their material is not restricted to students taking the course.

However, password protected access to online course materials may present other problems for dissemination. If access to online course material is restricted to students and teaching staff, it becomes difficult to obtain feedback on the material from peers outside of the course or institution.

Of particular concern to participants was the question of who owns the IP rights of an educational innovation. With the reduction in government funding for higher education, universities are increasingly required to seek funding from commercial organisations. ICT teaching tools and online course content are increasingly being developed in collaboration with commercial information technology organisations who may provide support financially or in terms of equipment. One mini-conference participant stated the problem as follows:

There’s an increasing amount of corporate sponsorship of online [content] … and with it goes either explicit or implicit the

93

requirement that part of the material is theirs (the sponsor’s) and only theirs, not for general use. So you end up having a light and a heavy version of what ever subject matter you’re distributing and you can’t talk about it until after the heavy version, the corporate version, is well and truly out in the marketplace, so it prevents the usual research ethos from allowing us to contribute freely. … It’s like waiting for a patent. You wait for the patent and then you can talk … it’s changed the way we do it. (M-C transcript)

While industry sponsorship is much sought after by information technology departments, the spirit of discipline-based inter-university collegiality may be threatened by the reliance of universities on industry collaboration.

6.3.1.4 Inter-university competition

Related to the issue of IP is that of the corporatisation of Australian higher education.

We’re no longer an academic community as seen and funded by the Federal government; we are a collection of universities competing for market share, which is a load of bullshit. (M-C transcript)

As the above, somewhat disgruntled, mini-conference participant sees it, Australian universities are no longer centres for academia so much as corporate entities. If this is so, the natural consequence is that the role of the university educator is now that of producer of products and deliverer of services. Inter-university competition was seen by participants as directly restricting the dissemination of innovations because innovations are the ‘products’ that give a university the ‘market edge’. For the entrepreneurial or media savvy researcher, this does not pose such a problem. For the university staff member whose talents and interests lie chiefly in the domain of teaching and the development of course content, corporatisation can be frustrating. Where educators perceive the discussion of ‘products in development’ as being prohibited, they can feel isolated and miss out on valuable feedback from peers.

A further problem arising from inter-university competitiveness is duplication. A number of participants noted that re-invention of the wheel is a common phenomenon within a corporatised higher education environment; where inter-university collaboration is prohibited, duplication is bound to occur. And duplication is not restricted to good ideas; limited collaboration across institutions is also likely to result in the re-invention of the square wheel.

It cannot be said definitively whether the reporting of ICT innovations in the mini-conference program was inhibited by IP and competition considerations. However, on the whole, participants reported that they felt free to speak about their initiatives and appreciated the opportunity to discuss their opinions and teaching practices within the collegial atmosphere generated by participants of the mini-conferences.

6.3.1.5 Professional risks

ICT educators associated dissemination activities with a degree of professional risk. Specifically, participants felt that sharing work in progress with peers from other institutions may be seen by their own institutions as a conflict of interest,

94

that their ideas might be ‘pinched’, consequently weakening their institution’s competitive edge. One participant said,

The tricky thing is [the problem of dissemination] operates on an institutional level and . . . also a personal level . . . I mean, your institution has an interest in what you do but also we’ve got to

ch more competitive between academics now and if you say the great stuff you’re doing when you haven’t actually yet had your article published in a famous journal, then that’s really tricky. (M-C transcript)

Many felt that prioritising teaching (both practice and research) ahead of mainstream or technological research would be a backwards step in terms of their careers because of the relatively lowly status of teaching and educational research compared with discipline specific research. Almost all felt that sharing successful innovations may be viewed by peers as confronting or ‘being told what to do’ as one participant noted, “no-one likes a smarty-pants” (M-C transcript).

6.3.1.6 Resources

The availability of resources was also seen as affecting ICT educators’ ability to disseminate innovations. Although a number of resource constraints were identified by participants as inhibiting the dissemination processes, the most frequently cited was time. The term “time poverty” was put forward by one participant to describe her perception that a standard academic workload leaves little time during working hours for anything other than what is strictly necessary. In regard to dissemination activities, time is required to develop ideas, to put them into a form that can be understood and utilised by others, to access and absorb the ideas of others and to incorporate new ideas into teaching practice.

In addition to having little time available for dissemination activities, these activities were seen by participants as, at times, being more trouble than they are worth.

If we want to talk about [our innovations] with other people, it needs to be in some kind of forum. If it’s a seminar, other people are teaching at that time, you’ve got to get to a campus, you don’t have the audience because everybody’s too busy. If you take it to a conference then it’s got to be a fully refereed paper …, it’s got to be a high standing conference, they’ve got to have proceedings … otherwise, you won’t get funding. (M-C transcript)

In short, participants perceived dissemination activities as being curtailed by workload, information overload and the likelihood that such activities will “create extra work . . . without the extra resources”. (M-C transcript excerpt). Inadequate resources have been identified by diffusion research as one of the key barriers to the transfer and implementation of new ideas. Szulanski (1994) found that individuals need to be capable of communicating and making use of ideas and that their capacity to do so may be limited by lack of money, time or management support. Each of these was described by ICT educators as lacking in their situations.

The six issues described above are interrelated. They act together to contribute to an atmosphere where dissemination activities are viewed as limited and often ineffective. Issues surrounding IP and inter-university competition merely

95

compound a situation where the pursuit of educational innovation is seen as professionally risky and unrewarded. These issues do not bode well for the generation or effective dissemination of educational innovations in ICT education, particularly, when research on the diffusion of innovation suggests that successful knowledge sharing is dependent upon an organisational culture that promotes communication (Nahapiet & Ghoshal 1998).

6.3.2 Effective communication channels In spite of the inhibiting factors described above, mini-conference participants unfailingly reported a desire to share their ideas and optimise opportunities for their dissemination. The reason most frequently given by ICT educators for participating in a mini-conference was “to find out what other people are doing”. Mini-conference participants identified a range of dissemination channels that were considered to vary in their effectiveness. Table 11 shows the communication channels identified.

Table 11. Channels identified by participants for the dissemination of teaching and learning innovations

Online Internal External

• web sites

• email

• listservers

• discussion boards

• mailing lists

• networking

• online professional development courses

• tours of university departments

• teaching interest groups

• staffroom discussions

• cross faculty seminars

• journal clubs

• professional development activities

• bringing ideas from conferences back to your department/ school

• working papers

• institutional education units

• Graduate Certificate in Higher Education

collaborative teaching

• professional associations

• conferences

• journals

• workshops

• teaching and learning symposia

• professional development activities

• informal gatherings

• road shows

• collaborative teaching

In discussions of dissemination, ICT educators identified qualities that add to the effectiveness of communication channels and to diffusion efforts. The more effective channels were seen to be those that were interactive and informal. In terms of diffusion efforts, mini-conference participants believed they were more likely to be effective if they allowed adopters to personalise the innovation, if the

96

attempt to diffuse an innovation was seen as educationally motivated. Each of these factors is described briefly below.

6.3.2.1 Interactive channels

Participants generally agreed that more interactive communication channels are more likely to lead to the diffusion of innovation. This is consistent with the findings of diffusion studies. Nahapiet and Ghoshal (1998) argued that passive activities are useful for delivering large amounts of material to large numbers of people, but they are less likely than interactive activities to change the behaviour of recipients. Similarly, Rogers (1983) stressed that interpersonal communication channels that facilitate the two-way exchange of information are more likely to influence strongly held attitudes and lead to a decision to adopt an innovation.

Interactive dissemination activities reported by participants included collaborative teaching, special interest groups, professional development activities and informal activities such as staffroom discussions.

6.3.2.2 Informal forums

Mini-conference participants also perceived informal forums to be more useful than formal forums. There was general agreement that there was more opportunity to learn, and more likelihood of resultant change, when communicating with people with whom they have a prior relationship or with whom they have common interests. The staffroom in particular, was seen as a rich source of information and a useful forum for exploring new ideas. A number of participants observed that forums like the mini-conferences, where participants have an opportunity to connect with their peers in both informal and semi-structured settings over an extended period, were often most effective as a source of new ideas that can be tried out in their teaching.

Rogers (1983) emphasised the importance of interpersonal communication in the persuasion stage of the innovation-decision process. In particular, he highlighted the role of face-to-face communication between peers of subjective innovation-evaluation information:

The heart of the diffusion process is the modeling and imitation by potential adopters of their near-peers who have previously adopted a new idea. In deciding whether or not to adopt an innovation, we all depend mainly on the communicated experience of others much like ourselves who have already adopted. These subjective evaluations of an innovation mainly flow through interpersonal networks. (Rogers 1983, p.293)

Similarly, Szulanski (1994) identified relationships as one of the four key factors in effective dissemination, stressing that individuals are more likely to take up the ideas of those they know and trust over ideas coming from anonymous or distant sources.

6.3.2.3 “Making it your own”

Mini-conference participants repeatedly discussed the importance of customising an innovation. They generally agreed that, for an innovation to be adopted by another educator, it needs to be possible to personalise it, so that adopters have a

97

sense of ownership and can integrate the innovation into their own personal teaching styles and established work practices. As one mini-conference participant said, “academics like to think things through themselves and are reluctant to take other models off the shelf without a lot of personal input” (M-C transcript). This comment highlights participants’ desire to absorb new material, adapt it to their own needs and ‘make it their own’. This reinvention is more likely to occur using interactive channels of communication. Participants also recognised that the personal style of the person who generated the innovation is often integral to its success. As one participant wrote in his workbook: “Teaching methods are very dependent on personality of the teacher – what works for one may be a disaster for others.”

Participants described what they saw as an increasing trend towards the “disintegration” of teaching, that is, particularly the de-integration of the production of subject materials and the delivery of subjects. Participants considered that there was more to teaching than the sum of its parts, expressing concern about the future of their roles as teachers if educational innovation continued to head in this direction. Linked to this concern were perceived threats to academic freedom and other issues related to a packaging approach to subjects.

Participants were generally uncomfortable with the idea of producing materials that others would deliver. One participant explained with some disbelief that, “At one point it looked like I was going to be lecturer-in-charge without teaching it at all.” Conversely, similar discomfort was expressed at the idea of teaching with materials produced by someone else. Participants found it difficult to articulate why this was undesirable, with many referring broadly to teaching as a personal activity.

Teaching is a very personalised activity and it’s highly individualistic … I just think it’s a very personalised activity well it is at the moment; I take [another participant’s] point about the de-layering or the disintegration of teaching that’s occurring but traditional teaching is in the sense that the academic both creates the curriculum, researches it, creates the learning materials and teaches it, and it’s seen as an integrated whole. And it’s only when, for example, you’re given other peoples’ lecture notes to try to teach from, I just can’t do it. It is amazingly hard, you know, when occasionally you’ve been asked to teach a course and “it’s all there”,

you haven’t worked your way through it. I really am not quite sure why, but I would find it impossible to teach that way. Having said that, increasingly it’s a model that is being used in higher education, where people – private providers in the US are given material. Not only do they have to teach it, but they’re not allowed to change it. They have to teach it as given. I could not teach that way. (M-C transcript excerpt)

Resistance to packaging approaches to teaching have been observed by Koppi et al who explained this resistance in terms of academic culture:

Tertiary teachers are generally creative individuals and experts in a particular field. They are generally also critical thinkers and maintain an objective distance from the work of their peers. This combination inclines the teacher to design unique courses commensurate with personal experience and knowledge. … An academic will not

98

normally adopt another academic's course without personalising it. (Koppi et al. 1998, p.425)

Similarly, Taylor et al. found that,

academics feel threatened and are reluctant to teach from packages because this requires that they teach other people’s work – ‘a very unrewarding teaching experience’. (Taylor, Lopez & Quadrelli 1996, p.56).

Participants were concerned that trends towards the packaging of subjects would lead to loss of ownership of material and loss of control over its delivery. They saw the delivery as integral to the aims they had for their material.

6.3.2.4 Motives behind diffusion efforts

Mini-conference participants describe the suspicion of non-educational motivations behind diffusion attempts as a main source of criticism of and resistance to the adoption of innovations. The management-driven push for Web-based delivery of subjects is an example that was repeatedly given as a diffusion attempt that served non-educational agendas. Non-educational agendas were a source of cynicism and resistance even when innovations were seen to have educational potential; that is, ICT educators resisted the diffusion attempt rather than the innovation itself. There was a general perception that teaching staff would resist diffusion attempts that were not primarily motivated by educational goals.

6.4 Conclusions

6.4.1 Contextual factors affecting dissemination of educational research • Inadequate training in research design and educational evaluation methods

was seen as a major inhibitor of dissemination of educational innovations, both in terms of promoting one’s own work and making assessments about that of others.

• The relatively lowly status of teaching compared with research was viewed as a major inhibitor of dissemination activities that focus on educational innovation. Many ICT educators believed that putting significant time and effort into the improvement and evaluation of teaching would not be rewarded in their departments or recognised as valuable by the larger ICT community

• Some ICT departments are seen as only playing lip service to the value of teaching. The links between scholarship in teaching and staff reward and promotion structures that exist in these institutions are not credible in the eyes of ICT educators

• Issues surrounding intellectual property and inter-university competition were viewed by many participants as potentially restricting the dissemination of their teaching and learning innovations and their freedom to discuss and share their ‘products in development’

99

• Dissemination activities that focused on teaching were generally viewed as unrewarded in terms of career progression

• While a range of factors at government, institutional and individual levels was reported as threatening collegiality in Australia’s universities, participants were able to identify these threats and are, at an individual level, actively seeking ways around them.

6.4.2 Effective dissemination channels • While many channels for dissemination exist, most are seen as ineffective.

Those channels that allow interaction with the innovators are seen as the most effective.

• Diffusion attempts that accommodate ICT educators’ need to personalise innovations and tailor them to meet their own needs and existing work practices are more likely to succeed.

• Diffusion attempts that focus on the educational value of innovations are more likely to elicit positive responses from ICT educators than those that are promoted primarily for non-educational agendas.

6.5 Recommendation arising from this section • Recommendation 10 That effective means be developed for the

dissemination and diffusion of educational innovations to the ICT education community. Diffusion efforts should

§ acknowledge and accommodate ICT educators’ need to ‘make innovations their own’

§ facilitate two-way interaction between those who generate the innovation and those who plan to adopt it

§ allow for the dissemination of work-in-progress and the exchange of peer feedback

§ take advantage of existing relationships between individuals and between institutions

§ take advantage of discipline loyalties and shared concerns to promote inter-university collaboration and dissemination across the disciplines

§ acknowledge issues such as inter-university competition and intellectual property

101

7 Educational evaluation

Sometimes if you get journals written by people from education, I’ve got a PhD and I’m well read, but I cannot understand a word that they are saying and … well I don’t know if other people find this but I find it in some so called education publication … I just did not have a clue what they were talking about, not a clue. (Mini-conference Participant)

This chapter draws together conclusions made across the various phases of the Project, highlighting the importance of educational evaluation in the dissemination of innovation and good practice in ICT education. First pertinent contextual factors are identified, including the policy environment of educational evaluation in the university sector, the organisational milieu in ICT departments and the teaching and learning environment being evaluated. The use of standard student feedback questionnaires and criticism of this approach to educational evaluation are discussed. Alternative methods of evaluation that use student feedback to model student behaviour are then described.

7.1 Contextual factors There are three main layers to the context of educational evaluation in ICT education: the wider policy environment, the organisational milieu of ICT departments and the characteristics of particular learning environments. Each is outlined briefly below. A factor in the evaluation of ICT education that was identified in previous sections (particularly Section 5 and Section 6) is ICT educators’ skills in, and knowledge about, educational evaluation. The three contextual layers described all have implications for this important factor.

7.1.1 Policy environment – Quality Assurance and evaluation of teaching Brennan observed that, “the growth of external assessment of higher education quality has been one of the most marked international trends in higher education in recent years” (Brennan 1997, p.23). Quality became an issue in Australian universities in response to the Dawkins’ reforms of the late 1980s. Massaro (1997)

-chancellors and other commentators began to express concern that the reforms and the reductions in funding had led to a reduction in quality” (p.53). In 1991, Peter Baldwin, the Minister for Higher Education and Employment Services, issued a policy statement, Higher Education: Quality and Diversity in the 1990’s (Baldwin 1991), announcing that it was necessary to assure the community that the quality of higher education was of a high standard and that $70m would be allocated to those institutions which could demonstrate better than adequate quality. In 1992, an independent Committee for Quality Assurance in Higher

102

Education was established to undertake quality assessments of higher education institutions.

The evaluation of university course content and delivery has become an important task in an environment of inter-institutional competition characterised by pressure from government and other stakeholders to demonstrate the quality of teaching and learning activities (Trembath, 1999). As the key consumers of the Australian university system, students are seen as an important source of information on the evaluation of institutional performance.

However, the difficulty for higher education is that there are no educational models for quality assurance. There are derivative models that raise credibility issues. Quality Assurance originally developed out of engineering practice and the manufacturing industry. The focus of manufacturing quality assurance was on maximising the efficiency of processes through the use of approaches such as just in time (eg. rather than maintaining a large parts inventory, having an ordering system which operates at a critical inventory level). Quality standards that have been developed from this production- or product-orientation are about the process of getting a job done. They fail to take into account the consumer or customer and this has, in part, led to the interest in customer satisfaction and reuse and repurchase approaches. Any model that represents higher education students in production terms, or as products, fails to reflect the real world of higher education.

A skills and competencies approach was taken by the Quality Assurance Agency for Higher Education in the UK. The student outcomes specified are very broad. A further limitation of a skills and competencies approach is that there is no simple and economical mechanism for updating the student outcomes, so, in areas such as Computing, the framework quickly becomes outdated.

The educational research literature contains very little on the application of quality assurance systems and the evaluation of the impact of these systems; although, Billing (1998) gives a useful summary of theories and possibilities. A paper by O’Neil, Bensimon, Diamond and Moore (1999) is one of the few found in professional journals that work through how a department explored the options, chose an approach and is in the process of assessing how it is working. They used the Balanced Score Card approach (Kaplan & Norton 1996), which has the advantage of focussing on continuous improvement rather than simple benchmarking. It also has the advantage that, with a little creative thinking, it can be customised to fit educational environments.

The educational literature has emphasised the division between formative and summative approaches to evaluation of programs where summative evaluation is primarily conducted for the purpose of accountability, while formative evaluation focuses on evaluation for improvement. The educational literature tends to place more importance on formative evaluation. There is a wider literature on education and training evaluation that does not use this type of terminology but has its own models of how evaluation can be carried out (see Phillips 1997). It is widely recognised that quality assurance has two main purposes: accountability and improvement. However, in Brennan’s (1997) international review, he concluded that almost all quality assessment bodies emphasise value for money and greater accountability for the use of public funds over improvement in their statements of purpose.

103

An approach that is not prescriptive against standards, skills or artificial production models, is the continuous improvement approach. In this approach staff use evaluation and feedback methods that can demonstrate that teaching and academic content are being reviewed and restructured to make them more effective in the teaching-learning environment.

7.1.2 Organisational milieu As identified in Chapters 5 and 6, ICT educators report that current organisational structures and value systems discourage the implementation of effective improvement-oriented evaluation procedures. ICT educators perceive both a lack of reward and a lack of resources. In fact, many see pursuing research in ICT education as a career inhibitor rather than a career developer. It is not surprising that, in terms of educational evaluation, the organisational level reflects the larger policy environment, with a narrow emphasis on accountability rather than on improvement. Not only do ICT educators have little time for developing their teaching, after their research and administration duties had been met, but educational evaluation is seen as a low priority because of the lack of reward for this type of evaluation. This phenomenon is not isolated to ICT departments, but has been generally observed across the university sector. Taylor has commented on the situation in Australia: “where academic career advancement is strongly linked to achievement in both teaching and research, much has been written about the need to recognise and reward teaching, but little progress made - research rules!" (Taylor 1999, Online document). Only the most dedicated ICT educators will prioritise, and put time into, improving and evaluating their teaching practice in an organisational milieu where teaching is not seen to be highly valued. Many commentators recognize the need for change at the level of organizational culture and the status teaching if innovation and good practice in teaching is to be promoted.

Brennan wrote that, “quality judgements which lack legitimacy in the eyes of those on the receiving end of them are not likely to be acted upon if action can be avoided” (Brennan 1997, p.13). In regard to recognition, Cosser observed that

rewards for good teaching are grossly inadequate”, concluding that “unless teaching is seen to be valued in and by the institution, all attempts to introduce new systems for evaluation its effectiveness will fail” (1998, p.159). Authors such as Boyer (1990) and Rice (1991) introduced the idea of the scholarship of teaching as a means for elevating the status of university teaching. A considerable literature has developed, discussing what the scholarship of teaching might look like, and how it might be encouraged and demonstrated. In Australia, those interested in the scholarly use of new electronic technologies to improve learning have done the most recent work on the role of educational evaluation in the demonstration of scholarship. Alexander wrote,

Evaluation … must become as much a part of professional practice as project development. Academics have a professional responsibility to design, develop, implement and evaluate [information and communication technology] for learning in a scholarly way, as would be expected in their research and in all other aspects of their academic lives. (Alexander 1999, pp.181-2)

Taylor and Richardson, in their report on a national scheme for external peer review of ICT-based teaching, make the distinction between what they call

104

scholarship-about-teaching and scholarship-in-teaching, where scholarship-in-teaching focuses on the design process and accounting for outcomes, rather than on understanding the teaching and learning process:

Our sense is that most examples of scholarship of teaching are really discussions about teaching and learning processes, mostly in the form of conference and journal papers. We refer to this publication-focused aspect of the scholarship of teaching as 'scholarship-about-teaching'. Our interest in this work is with those aspects of the scholarship of teaching that directly inform the decisions teachers make. That is, we are interested in those aspects of the scholarship of teaching that both inform the design of pedagogical practices and are evident in teaching itself. We refer to these aspects as 'scholarship-in-teaching'. (Taylor & Richardson 2001, p.32)

In this way, the processes of design, implementation and evaluation are seen as sources of scholarship. This is consistent with Alexander’s (1999) recommendation that evaluation be undertaken at all stages of development. The goal of this type of recommendation is not only to advance knowledge about teaching and learning, but also to recognize and reward quality teaching and to elevate the status of teaching in university departments.

Elton (1993) argued that the promotion of quality in university teaching requires the following changes:

• a rapid increase towards the professionalization of university teaching

• the establishment of TQM practices at all levels of each university

• increasing recognition and resourcing of teaching and rewards for excellence in it (p.145)

7.1.3 Learning environment Perhaps the most important consideration in educational evaluation is the particular environment that is being evaluated. Evaluation processes are applied to particular initiatives, teaching episodes, teachers, subjects or courses. If evaluation processes are to be tailored to the educational aims of a particular initiative, the specific characteristics of the object being evaluated need to be considered. Similarly, individual educators have specific strengths and weaknesses that lead to specific evaluation needs and should therefore inform the design of any evaluation. As discussed in Section 5, the most common complaint against educational evaluations is that they often do not provide an indication of the extent to which the object being evaluated meets its intended aims.

7.1.4 Skill level of ICT educators Participants at the mini-conferences discussed lack of familiarity with educational research and evaluation methods as a barrier to the dissemination of their teaching and learning initiatives (See Chapter 6). They noted difficulties accessing educational literature because of their lack of familiarity with educational terms and theories, and reported a lack of knowledge about methods of educational

105

evaluation. Some also described difficulties getting papers published in educational journals because of their lack of training in educational research.

Although it was clear from presentations and discussions at the mini-conferences that evaluation was a weak point in many of the teaching and learning initiatives described, it was also clear that the ICT educators who participated were dedicated to the improvement of their teaching. However, as described above, the organisational milieu of ICT departments is generally not one that encourages teaching staff to give priority to educationally-focussed professional development. In a recent study McInnis (1999) found that only 34% of Australian academics had any training for teaching at the beginning of their careers and that only 24% had experienced any professional development in teaching methods during the previous two years. It can be expected that even smaller percentages of academics have training in educational research methods and evaluation.

As the higher education system has implemented a quality management policy, there seems to be a fundamental issue about who, within the average university department, will have the skills to explore, customise and then implement a realistic system which carries out evaluations of educational practice.

7.2 Criticism of dominant student feedback approaches Discussion of the educational performance of tertiary institutions tends to focus on

• teaching performance

• graduates’ ability to find employment

• graduates’ retrospectively views of their course

• industry’s views of courses

There appears to be a lack of research, outside of education faculties, into the way subjects or courses actually affect student learning. Most research has been carried out on delivery issues involving student feedback, course evaluation and student satisfaction (Alexander 1999; Alexander & McKenzie 1998). This was evident in the presentations of ICT educators discussed in Chapter 5 where the most commonly used means of evaluating teaching and learning initiatives was standard student feedback questionnaires administered by university learning units.

This approach to evaluating teaching currently predominates in Australian universities. Student feedback questionnaires have been used in Australian universities since the 1960s but gained greater popularity in the late 1980s with the advent of the Unified National System and the attendant move toward quality assurance approaches to management (Bedggood & Pollard 1999). Three main purposes can be identified for student feedback measures:

• To provide teachers of a subject with information that can be used to improve the subject’s content and delivery

• To satisfy institutional and governmental requirements concerning the monitoring and reviewing of teaching quality

• To provide staff with a source of information regarding their teaching performance for the purposes of promotion

106

One of the advantages of the student-based feedback approach is that it is a low-cost, easily administered approach. Stringer (1993) identified features of teaching for which student feedback is valuable. They include course organisation and structure, workload and course difficulty, marking and exams, course impact on students, breadth of coverage and global or overall ratings. A distinction is made between feedback on these features of subjects and courses and the evaluation of learning outcomes, to which they are generally less suited.

Data gathered via feedback and evaluation surveys for the purposes of teaching and learning improvement should, however, be approached with caution. A range of problems associated with the use of student opinion surveys (SOSs) have been identified (see Bedggood & Pollard 1999; McInnis et al. 2001) as undermining the educational objectives of the surveying of students and resulting in an imbalanced and potentially damaging picture of students’ perceptions. These problems relate to the construction of SOSs and characteristics of respondents. In particular:

• As SOS items are drawn from a large pool of potential items, the reliability and validity of the constructed instrument can be questionable. These may lead to difficulties in creating relevant devices and in being able to interpret output.

• Respondent characteristics can influence the quality and nature of data obtained via SOSs.

• Where an SOSs is for the production of data to be included in a teaching portfolio as evidence of teaching performance for promotion purposes, it is possible to construct an instrument that showcases a teacher’s strengths while down-playing their weaknesses. (Bedggood & Pollard 1999).

Where teachers are unfamiliar with test construction principles, data may be produced that is repetitive, incomplete or misleading. The intended purpose of the surveying activity may also lead to the production of misleading or questionable data. Often, a teacher is interested in students’ perceptions of specific areas of their teaching or a subject. The interconnectedness of various aspects of the classroom experience should not be ignored and an imbalanced picture of student perceptions may arise where questionnaire items focus on a circumscribed aspect of the classroom experience (eg., specific teaching and learning resources) to the neglect of the total experience (including teaching performance, subject difficulty and assessment processes).

As shown in Chapter 5, many ICT educators express frustration with the inadequacy of this approach to evaluation. This finding is also supported by information generated during a workshop in the Monash Faculty of Information Technology in 2000, where it was agreed that staff members experience difficulties applying the conventional, centralised system of evaluation to specific teaching and learning environments. The item pools from which staff can create a survey often lack items that reflect what the teacher is doing. The basic design problem is exacerbated by the quality and structure of feedback, particularly when a staff member needs more structured information. At another level, the outcomes can be very limited when staff members do not have access to data that they might want to analyse further.

Respondent characteristics can also influence the quality and nature of data obtained via standard student feedback questionnaires. Unless administered in a supervised environment (ie., in class) the response rates for surveys can vary widely and lack the representativeness required for validity. Furthermore, they are

107

often administered at the end of semester when students are most anxious and perhaps least interested in completing questionnaires, especially if they have already been surveyed in other subjects. Other respondent characteristics that influence the quality of data obtained include their relationship with the teacher, perceptions of the teacher and specific grievances regarding the teacher, subject, course or its administration. These factors may bias a student’s responses positively or negatively but, as questionnaires are completed anonymously, the presence and sources of bias cannot be determined.

The use of student feedback questionnaires to produce data for teaching portfolios as evidence of teaching performance for promotion purposes is also questionable. Given their highly flexible nature, it is possible to construct an instrument that showcases a teacher’s strengths while down-playing their weaknesses. However, because of their inherent invalidity and unreliability, even a well-considered instrument may “expose teachers to unreliable, invalid opinions, that influence teacher career advancement and job security” (Bedggood & Pollard 1999, p.129). This problem arises from the accountability focus of current systems and might be alleviated if portfolios focused on evidence of teachers’ efforts to improve their teaching, rather than on summative evaluations.

A side-effect of the dominance of standard student feedback questionnaires in quality assurance systems is that evaluation of teaching methods and materials more generally are also often limited to this method; that is, student feedback questionnaires are not only seen as an activity required by quality assurance systems, but they become seen as a sufficient means of evaluating teaching and learning initiatives. This was observed in the research reported in Chapter 5 and is discussed in the literature. Bain noted that reports of innovative teaching practice

… often limit their evidence on the influence of the innovation to students' and peers' reactions during development, and their impressions of learning achievements after implementation. Sometimes data from standard course assessments also are reported, but often without regard to the relationship between the learning encouraged by the innovation and the learning assessed by standard methods. (Bain 1999, p.166)

Similarly, in a review of 104 teaching and learning initiatives that made use of information and communication technologies, Alexander concluded that,

There continues to be a heavy reliance on student reaction surveys, and in some cases there is an apparent confusion between student reactions and student learning. Positive student attitudes and increased motivation may encourage better learning outcomes, but they are not in themselves evidence of improved learning. While student reaction surveys are a useful component of any evaluation, they should not be the only component. (Alexander 1999, p.181)

Although standard form student feedback questionnaires continue to dominate in the evaluation of university teaching in Australia, educators increasingly recognise the limitations of relying on this method and there is a move towards evaluations that are targeted to the objectives of particular teaching methods and materials and implemented for the purpose of improvement.

108

7.3 Alternative approaches: Using student feedback to model student behaviour

Bedggood and Pollard (1999) are sceptical about the contribution made by SOSs to the improvement of higher education. The authors state that any effect that the results of SOSs may have on teachers’ performance is minimal and that there is little evidence to suggest that performance appraisal actually leads to performance improvement in Australian higher education. Furthermore, the authors claim that improvement seen during periods of formative evaluation would probably have occurred anyway.

Similar observations have been made about the routinely administered Course Experience Questionnaires (CEQ). McInnis et al argued that, while useful in its “function of sifting the best from the worst courses within a field of study” (McInnis et al. 2001, p. 5), student satisfaction data arising from the CEQ is less useful at the coalface. The impact of CEQ data at the course or subject level is minimal and tends to be viewed with some cynicism by teachers (Hicks, 1999).

Alternative approaches to the evaluation of university teaching should be considered in order to improve the validity of claims made based on student evaluation data. Most student feedback research is carried out without any account for student behaviours. This makes it difficult to build comparative measures that are sensitive to the particular issues being faced by teachers. Appropriate modelling of satisfaction can provide the basis for a continuous improvement model for quality assurance.

One way of helping build a continuous improvement approach is to apply work done on customer satisfaction (Wittingslow & Markham 1999) to student, graduate and employer satisfaction. Specifically, this work argues that satisfaction should be treated as an outcome variable rather than as an attitudinal variable. This approach produces diagnostic information for helping develop a continuous improvement approach. If evaluation is conceptualised within both a pedagogical and behavioural milieu, and if it is oriented to both the educational process and the student’s perceptions of outcomes, then it is possible for evaluation to help improve teaching. This must be done within a context where the educational managers give rewards for effective evaluation, change and development.

Such an approach will deepen our understanding of student evaluations of teaching and learning and improve the validity of claims made based on student evaluation data. In part, it will help us understand student satisfaction as a whole life experience and one which must include the role of the ‘extra-curricular’ factors (eg., motivations, pressures, expectations) in the students life.

Some approaches to student satisfaction can be criticised for their dubious relevance to teaching and learning. For example, Elliot and Hughes (1999) utilised what they saw as an ‘integrated’ approach to student satisfaction in their work with the AIIA E&T Career Tracking Research project where they measured graduates’ satisfaction with their ICT courses in terms of how well they prepared them for industry. This research tells us about one aspect of possible satisfaction. Malley (1998) argues that student satisfaction and its assessment must be seen as a complex set of interacting factor which belong in a causal chain, beginning with the expectations and attitudes the students brings into the university experience and ending with the students assessment of the whole university experience.

109

He describes an approach to looking at student satisfaction, advocating a three-part model comprising the antecedent domain, the transactional domain and the outcomes domain. The antecedent domain includes the needs, wants expectations and prior experience that the student brings with them to the education experience. The transactional domain encompasses factors related to the transactional relationship between the student and their education experience including the match (or mismatch) between the student’s needs and expectations and the ‘product’ delivered by the institution. Finally, the outcomes domain represents the integration of factors from the antecedent and transactional domains with the post-university ‘out-side’ world.

Approaches to using student feedback to evaluate and improve teaching should:

• Be oriented towards students as evaluators

• Be tailored to the particular teaching initiatives being used by staff

• Be relatively easy to implement in any teaching-learning environment

• Not require special skills in the teacher over and above basic educational research skills

If we explore some of the educational behaviour questions that determine the way students respond in an educational environment, we can use expectancy-value and/or social learning theory (Bandura 1977) as a starting point for a coherent approach. These models of behaviour have the advantage that they are person-centred in that they start with the perceptions the individual has of the world.

Social learning models, derived from the attribution model of Heider (1958), are premised on the person as ‘the actor on life’s stage’ world view. One of the key features of Heider’s model of behaviour is that we become involved in a task when we have a challenge. We are less motivated when the tasks are not challenging. Csikszentmihalyi (1975; 1996) develops this idea by arguing that we become more involved in a task when we have a challenge and, where there is a reasonable balance between challenge and our belief in the abilities we have; then we get a sense of satisfaction on completing a task.

From this we conceptualise satisfaction broadly and look at how this might be measured for students. It is important to understand that student satisfaction measurement is not an end in itself. Rather it is a tool to aid in the process of educational evaluation and improvement of the teaching and learning environment.

7.3.1 Research support for a student satisfaction approach Guolla (1999) has carried out a research project using a customer satisfaction approach from the University of Michigan. He established that he could use a traditional feedback survey and generate a useful model of student satisfaction and its antecedent conditions.

Postema and Markham took an approach similar to the one used by Guolla to investigate students’ responses to a second-year software engineering subject (Postema & Markham 2001). Possible sources of student satisfaction were investigated in terms of how they contribute to students’ overall satisfaction with the subject. This investigation isolated both strengths and weaknesses in delivery

110

that then informed the further development and rethinking of specific teaching activities.

Postema and Markham (2001) also looked at the relationship between rated enjoyment and rated satisfaction. They found that when enjoyment was used as the dependent variable in a regression with the various course content and delivery variables, enjoyment behaved somewhat differently to satisfaction. Students did not equate satisfaction with enjoyment, raising doubts about the criticisms of satisfaction research as being simply about students enjoying classes.

Within Australian higher educational circles, there is an increasing recognition of the importance of context-specific evaluations of teaching that are more action-oriented than summative. The conduct of this type of evaluation by university teaching staff is promoted as both a means of improving teaching and a means of elevating the status of teaching.

In a discussion of trends in the evaluation of university teaching, Henkel (1998, p.289-290) noted trend towards more context-specific conclusions, a greater emphasis on the need for evaluation to be more practical, a greater emphasis on organisations taking charge of evaluations themselves and incorporating them into change strategies, and a shift from summative evaluation for the purpose of accountability towards formative evaluation for the purpose of improvement. Similarly, Oliver (2000, Online Document) observed a “shift of power away from the evaluator as an arbiter acting for the commissioning authority and towards evaluation as a collaborative process of building mutual understanding.” He argued that,

This approach advocates a continuous process of evaluation by practitioners (as opposed to evaluators) as a strategy for dealing with organisational change and meeting the need for responsive, adaptive organisational structures. ... Moreover, this style of evaluation represents a shift in focus away from self-contained programmes or projects to the ongoing evaluation of processes and systems. It requires the creation of a culture of reflective practice similar to that implied by action research. (Oliver 2000, Online Document).

7.4 Conclusions • Methods of educational evaluation that are commonly used in ICT

departments are inadequate for the purpose of improvement. Nor do ICT educators usually have the skills to implement evaluation designs that can be used to improve their teaching.

• The organisational milieu of most ICT departments generally does not encourage ICT educators to prioritise, and develop skills in, their teaching and the evaluation of their teaching. ICT educators perceive a lack of resources and a lack of reward for the pursuit of educationally-focused activities. Consequently, ICT educators and their departments have difficulty demonstrating that their teaching activities represent innovative or good educational practice.

• The quality assurance policy environment of Australian universities, particularly the focus on accountability over improvement, contributes to environments at the university and departmental levels that promote simplistic

111

educational evaluation methods that are generally not useful for the purpose of improvement.

• There are ways ahead for evaluating the extent to which educational initiatives and programs meet their intended aims. They are varied and need exploring.

7.5 Recommendation arising from this section • Recommendation 12 That continuous improvement approaches be adopted

in ICT departments so that ICT educators can more rapidly respond to change and input from professional, organisational and industry sources.

113

8 Conclusion

The broad aim of the research reported here was to investigate the ways teaching and learning are being approached in the major discipline area of ICT in Australia’s universities. More specifically, the Project Brief asked for an investigation of the extent to which innovations in teaching and learning have been deployed in response to the needs of students and employers, and that recommendations be made for the dissemination of good practice. The research comprised a number of review processes and complementary data collection phases. Traditional review processes were used to identify key findings and trends in the literature in a number of areas identified as relevant to the Project Brief. They included

• Employers’ needs and satisfaction with ICT graduates

• Student and graduate satisfaction with ICT courses

• The broad context of university teaching

§ Massification and vocationalisation of higher education

§ Changing policy environment for the funding and governance of higher education

§ Internationalisation and the advance of information and communication technologies

• Current movements in university teaching

§ flexible delivery and student-centredness

§ quality assurance of higher education

§ student evaluation of teaching

§ scholarship of teaching and practitioner-run, targeted evaluation of teaching

• Innovation and dissemination

A more novel approach was taken to reviewing current teaching and learning initiatives in ICT education and the factors to which innovations respond. A mini-conference format was used to collect data on teaching and learning initiatives, and to facilitate open discussion of the factors driving and inhibiting innovation and issues involved in disseminating innovation.

The review of current teaching initiatives, the factors driving them and the issues involved in their dissemination was complemented by data collection phases that focused on the views of employers and the views of ICT graduates.

8.1 Reception of project The Project aimed to maximise the cooperative input from ICT academics, employers and graduates. The mini-conference program was the first point of contact between the Project and members of the ICT education community and

114

an important phase in terms of setting the collaborative tone of the overall project. In total, eighty-three ICT educators from twenty-nine of Australia’s universities participated in the program. The number of participants at each mini-conference ranged from four to fourteen, with fourteen being the maximum number of participants that could be accommodated on any one day. Only small numbers were attracted to the Hobart and Darwin mini-conferences. This was expected as Tasmania and the Northern Territory have only one university each. In total, fifty-one university administrative units were represented, with representatives ranging across all academic levels. The Project team was satisfied that the participants represented a range of view points from a range of different university departments. In terms of the data collected, saturation point was reached both for the general issues raised and the type and focus of initiatives presented.

The atmosphere of the mini-conferences was open and congenial. Informal feedback indicated that participants saw the activities as worthwhile in terms of the Project and also in terms of their own time investment. Two participants provided unsolicited written feedback at the end of the mini-conferences in which they participated:

A participant in the second Sydney mini-conference wrote:

• I felt the time was well spent

• Well run and facilitated

• fun/ enjoyable/ challenging

• Met interesting colleagues

• Felt that I learnt more than I gave

• Boosted my enthusiasm and encouraged me to keep on innovating.

A participant in the Adelaide mini-conference wrote:

• well organised/ well run

• open/frank exchange of views

• free flowing format

• useful to hear others’ views/ experience.

Contact with employers included a quantitative survey about needs in, and satisfaction with, ICT graduates, and follow-up telephone interviews. Contact with graduates included face-to-face interviews about their courses and the role of education in their careers, and follow-up telephone and email questions.

In October, senior ICT academics were invited to attend an Intensive Workshop to review draft recommendations arising from this research. Ten senior ICT academics from non-Monash universities, or their representatives, participated in the Workshop. In general participants were supportive of the Project. The conduct of discipline-based national studies of this kind was seen as a positive development. Participants made specific suggestions for the fine-tuning of particular recommendations and many of these were responded to in subsequent drafts of the final report.

It is hoped that the high level of consultation and collaboration that characterised the Project has resulted in research outcomes and recommendations that are both relevant and credibled.

115

8.2 Summary of finding As a result of the review processes and data collection that was undertaken by the Project team, conclusions can be made about

• employers’ views of the preparedness of ICT graduates for the workforce

• future research needed to assess ICT graduates’ views on the value of their courses

• methods for assessing student satisfaction, for evaluating educational innovations and ICT educators’ skills in these evaluation methods

• ICT educators’ views on innovation in ICT education and the dissemination and diffusion of innovation.

• the types of teaching and learning initiatives currently being undertaken by ICT educators and the issues they respond to

Conclusions relating to each of these areas are outlined below.

8.2.1 Employers’ views The differences between the attitudes of those employers who do employ recent graduates and those who do not raises interesting questions about the state of the job market in ICT. The large number of micro to small businesses in the ICT industry may be a key issue in the question of skills shortages evident in the literature. These businesses are least likely to employ graduates. Projecting the job market in ICT would require an understanding of factors such as the job life span of staff in such organisations as well as the likely expansion rates in job numbers.

There appears to be good reason for more labour market analysis if universities, and TAFEs are have a clearer understanding of the career environment they are feeding into. We have noted that, underlying all the analysis and interpretation, there is the basic fact that 64% of those who do employ recent graduates said it was highly likely that they would continue to employ graduates. There seems to be an inevitable market for ICT graduates, which is partially independent of what industry thinks universities should be doing to produce graduates. Exactly how many graduates will be employed in the future is another question.

The employer survey data clearly indicated that those businesses employing recent ICT graduates are satisfied with them. It is also clear that industry sees graduates as having a number of obvious deficiencies but that these are irritants, or Squeaky Wheels, rather than being significant factors in the decision to recruit. These irritants have been reported in different papers over the last 10-years.

The results from the comparison of the regression modelling and the mean trends for the skill data, combined with the suggested shortcomings of the DETYA employer satisfaction study (ACNeilsen Research Services 2000), point to the importance of building research models which are sufficiently sophisticated to allow analyses of the structure of behaviour. If the industry data had been based upon interviews or focus groups, it is unlikely we would have been able to see the distinct ways in which the skills interact, or to make a distinction between those that are important (in satisfaction terms) and those that are not.

The ICT sector is subject to the vagaries of changing technologies, ideas and, to some extent, fashion. The speed at which the e-bubble was inflated and then

116

deflated strongly suggests that university ICT departments should have depth in their degree structures, which will allow them to readily shift direction as new waves pass through. Focusing on skills training would lead to medium term staffing problems for industry. It would also lead to broad human resource problems as changing technology makes redundant those who have not been educated for adaptation and change.

8.2.2 Graduates’ views Little research has been published on ICT graduates’ experiences and attitudes after they have entered the workforce. Universities routinely monitor initial graduate destinations through the Graduate Destination Survey, but this exercise is not intended to inform curriculum development and educational innovation.

Exploratory interviews with a small number of Melbourne-based ICT graduates identified a number of common themes in ICT graduates’ experiences and a number of salient issues.

The themes identified possibly characterise a particular type of ICT graduate, who had exposure to and developed an interest in computing technology during their childhood, taught themselves how to program before entering any formal ICT education, and are motivated to pursue the intrinsic rewards of using computing technology. This type of graduate fits the popular stereotype of a computing student and eight of the thirteen interviewees fitted this type. This type of student poses particular challenges for ICT educators because of their prior knowledge and the specificity of their prior interests. If this type of student does predominate in ICT courses5, the particular needs, challenges and strengths of this type may mask those of other types. Popular perceptions of ICT students’ resistance to learning generic skills, such as communication and problem-solving skills, may be due to the predominance or the high visibility of this type.

Five of those interviewed did not fit this type, having not acquired extensive knowledge about computers prior to their university degrees. Two of these were the two female graduates interviewed. Both women were pragmatic in their views about the value of their degrees, and both had made conscious decisions to pursue careers ICT – one after consulting a career advisor, the other after having completed a tertiary diploma that she deemed was not suited to her interests. This contrasts with those interviewees who had long pursued interests in computer technology, saw their university degrees as a natural continuation of this pursuit and appear not to have sought advise on their careers.

Due to the small numbers of graduates interviewed, observed gender differences cannot be generalised. However, this does raise questions about the needs, attitudes and behaviours of graduates with different backgrounds, and warrants further investigation.

Interviewees’ attitudes towards the non-programming subjects in university ICT degrees raise questions about their expectations of their courses and their expectations of the work of ICT professionals. Most of those interviewees who raised the issue of programming versus non-programming subject expressed a

5 Note that predominance has not been identified statistically and may only be a perceived predominance.

117

dislike for subjects on project management and documentation despite recognising the importance of these subjects to their future work.

The themes and issues identified in the exploratory interviews warrant further inquiry. Further research would have implications for the recruitment of students to ICT degrees, the promotion of ICT degrees and the ICT professions to prospective students and to current students, the provision of career guidance to prospective and current students, and the development of programs to better prepare graduates for work as ICT professionals.

8.2.3 ICT educators’ views Consultation with ICT educators revealed that educational aims are the primary driving force in innovation in ICT education, particularly those answering the learning needs of students. Innovative teaching staff are motivated by a desire to be better teachers. However, the need to improve learning outcomes for students often stems from perceived problems whose origin is the changing context of university teaching. Contextual factors such as large class sizes and increasingly diverse student bodies are seen to cause problems that then need to be responded to by teaching staff in innovative ways. While these factors apply pressure to university teaching generally, they manifest more acutely in ICT departments where student numbers have grown rapidly over the last decade due to the expansion of the ICT industry and the growth of needs in the general community for ICT professionals. Because of the diffusion of computing and networking technologies into the broader community, ICT educators, more than in any other discipline, face challenges due to the diversity of skill levels in first-year populations. Yet another factor that, although affecting teaching across all disciplines, is more acutely experienced in ICT education, is the development of content. The rapid development of new technologies and related techniques has implications for the curriculum in ICT courses. For educators in ICT disciplines, the question of what to teach is perhaps more problematic than that of how to teach. This question is subject to a number of tensions. First, there are the skill and knowledge needs of employers. The market place for ICT graduates is a diverse one and little is known about the different requirements of the various segments of this employment market. There is a tension between the demands for particular skills, demands for more generic skills and demands for principles and areas of theoretical knowledge. Employers’ needs for particular skills are perhaps most directly communicated to ICT educators through the expectations and demands of their students who are familiar with fashions in new technologies and with trends in job vacancies. However, the demands of students, for example to learn specific programming languages, are often at odds with ICT educators’ beliefs about the skills and knowledge necessary for graduates to be professionals and lifelong learners in a rapidly changing industry. ICT educators report that students are often misinformed or have misconceptions about their future working roles and careers.

Students were generally seen as conservative in terms of the types of teaching and learning activities they expected and desired in their courses, with ICT educators reporting that students were often resistant to innovative, non-traditional teaching. Providing an innovative learning environment was perceived as likely to result in negative SET responses by the predominantly conservative students.

118

ICT educators perceive extrinsic motivation and reward for educational innovations to be minimal. In fact, many associate being an innovative teacher with personal and professional risks. Institutional agendas and reward systems are seen by some to discourage educationally motivated innovation. The source of the risks associated with innovative teaching was the low status of teaching compared with research. When academics reputations are built on their research profiles, there is little incentive for spending precious time on planning, implementing and evaluating new teaching strategies. This point is raised again, below, in reference to educational evaluation and dissemination of innovation.

8.2.4 Teaching and learning initiatives The impetus for, and aims of, the majority of the teaching and learning initiatives reported here are student-focused, with many responding to needs or problems encountered by students and a desire to improve the teaching of ICT. Other initiatives were motivated by the changing context of university teaching (including reduced resources, increased student numbers and more diverse student populations), or by changing needs of employers.

Although the broad focus of initiatives was student learning, they addressed a wide-range of particular issues and concerns. Some, responding directly to student demands, involved flexible access to course materials, or the certifications of courses. Others responded to observed problems, such as the logistical challenges posed by large class sizes or the need to engage students with vastly different prior learning. First-year subjects were the target of a range of initiatives that focus on student engagement and retention. At higher year levels, initiatives aimed to develop students’ generic skills, particularly skills associated with ICT professionals, such as problem solving, team-work and communication. Initiatives involved a range of strategies, from the creation of online learning environments to the use of small groups and project-based curricula. Many initiatives that might be seen as best practice within educational circles were presented as noteworthy and novel within ICT education. Initiatives presented to the Project team variously targeted classroom and online teaching modes and environments, assessment, subject and course structure and content, partnerships with employers, collaboration between teaching staff, a range of teaching tools, and research projects into student learning and related issues.

When brief accounts of the initiative presented to the Project team were reviewed by Monash CERG members, those identified as particularly noteworthy initiatives included addressed student retention and student engagement, attracting and retaining female students, enhancing learning, catering for groups of students with diverse skills, accommodating large class sizes, curriculum integration, teaching skills associated with professional practice and teaching learning skills.

Most initiatives reported had been formally evaluated, however, for a large proportion of these initiatives, standard student feedback questionnaires were the only source of evaluation. Participants recognised the limitations of relying on student feedback questionnaires alone for the evaluation of teaching and learning initiatives. However, given the lack of reward for such activities and ICT educators’ general lack of skills in educational evaluation, many ICT educators chose not to pursue formal evaluations further than the standard student evaluation of teaching questionnaires required by their departments.

119

It was also observed that many ICT educators underestimate the significance of their teaching practice as innovation and that they use informal means to evaluate and improve their teaching, rather than pursuing evaluation methods that better qualify their work for publication as educational research and better demonstrate the scholarship in their teaching.

8.2.5 Dissemination ICT educators’ general lack of training in research design and educational evaluation methods was seen as a major inhibitor in the dissemination of educational innovations, both for the purpose of promoting one’s own work and for making assessments about that of others.

The relatively lowly status of teaching compared with research was viewed as a major inhibitor of dissemination activities that focus on educational innovation. Many ICT educators believed that putting significant time and effort into the improvement and evaluation of teaching would not be rewarded in their departments or recognised as valuable by the larger ICT community. Some ICT departments are seen as only playing lip service to the value of teaching. The links between scholarship in teaching and the staff reward and promotion structures that exist in these institutions are not credible in the eyes of ICT educators. Dissemination activities that focused on teaching were generally viewed as unrewarded in terms of career progression

Issues surrounding intellectual property and inter-university competition were viewed by many participants as potentially restricting the dissemination of their teaching and learning innovations and their freedom to discuss and share their ‘products in development’

While a range of factors at government, institutional and individual levels was reported as threatening collegiality in Australia’s universities, participants were able to identify these threats and are, at an individual level, actively seeking ways around them.

8.2.6 Student satisfaction and educational evaluation Methods currently used in university departments to assess students’ satisfaction with their courses are inadequate for the purpose of continuous improvement.

On the whole, ICT educators are unfamiliar with the principles and techniques of educational research and evaluation methods. This impairs their ability to conduct formative evaluations that provide useful information for the improvement of their teaching. It also inhibits their ability to conduct the summative evaluations that would support the dissemination of educational innovations. A related problem is their lack of skills to assess the innovations of others as reported in the wider educational literature.

A lack of resources and a lack of reward for the pursuit of educational evaluations further compound these problems. Evaluation and dissemination activities are seen to be a very low priority in ICT departments. Consequently, ICT educators and their departments have difficulty demonstrating that their teaching activities are innovative.

120

8.3 Challenges for the future As a consequence of the conclusion made above, the Project team makes twelve recommendations pertaining variously to improving the interactions between ICT educators and industry and graduates, promoting educational innovation and dissemination in ICT education, and promoting the evaluation of teaching and learning initiatives in ICT education. These recommendations can be found at the end of the sections from which they arise (sections 2, 3, 4, 5, 6 and 7). They are also listed in the Executive Summary.

This report has highlighted a number of challenges for the future of ICT education. They include challenges for teaching and learning, challenges for the profession of ICT education and challenges for future research that targets this diverse, and in some areas, emerging discipline area.

8.3.1 Challenges for teaching University teachers currently face a raft of challenges due to recent and ongoing changes in the higher education system. For ICT educators, challenges of particular and ongoing concern include those posed by large class sizes and a diverse student population, those posed by the rapid development of new technologies and related content areas, and those posed by the increasing pressure to respond to the needs of employers and students.

8.3.2 Challenges for the profession of ICT education The promotion of teaching as a profession has been a concern in primary and secondary sectors for a number of decades. In contrast, university academics have long been considered professionals. However, since the establishment of Australia’s first universities, teaching activities have been of a relatively low status, with academics’ reputations and advancement being based on their contributions to research, and their adequacy as teachers being based on their expert status in particular fields of research. This situation is changing. At least in terms of policy documents, university teaching is becoming more valued. However, regardless of new and emerging policies, teaching is still well down the list in terms of the professional activities of university academics. The scholarship of teaching movement seeks to remedy this situation, as does the increasingly active role taken by university academic development and teaching units. However, ICT educators generally still view teaching as undervalued by their departments and, therefore, as a low priority for the majority.

8.3.3 Challenges for future research across the discipline Specific challenges for future research into ICT education have been discussed in previous chapters, particularly for future research in to employers’ and graduates’ views about ICT education. Broad challenges facing future research into ICT education include the diversity among ICT departments, the diversity among employers of ICT graduates and the difficulty of identifying and following-up graduates in the workforce.

121

The major discipline area of ICT contains a wide range of disciplines with disparate histories and a wide range of university departments that vary in history, size and structure. In fact, members of some departments may believe they have little in common with other departments in this broad category. Similarly, the businesses that employ ICT graduates vary considerably in size, industry and type and, therefore, in their needs and views of ICT education.

Informal feedback on the research reported here from those with interests in ICT education suggests that national research across this broad discipline area is valuable. However, national studies should be complemented with site-specific investigations.

123

References

ACNeilsen Research Services 2000, Employer satisfaction with graduate skills: Research report, Commonwealth of Australia, Canberra.

ACNielsen Research Services 1998, Research on employer satisfaction with graduate skills: Interim report, Commonwealth of Australia, Canberra [Available: http://www.detya.gov.au/archive/higherd/eippubs/eip98-8/execsum.htm].

Alexander, S. 1999, 'An evaluation of innovative projects involving communication and information technology in higher education', Higher Education Research & Development, vol. 18, no. 2, pp. 173-183.

Alexander, S. & McKenzie, J. 1998, An evaluation of information technology projects for university learning, Australian Government Publishing Service, Canberra.

Anderson, J. R., Reder, L. M. & Simon, H. A., Applications and misapplications of cognitive psychology to mathematics education, (patent).

Ashenden, D. & Milligan, S. 1999, The good universities guide to universities, TAFE and private colleges in 2000, Hobsons Australia, Subiaco.

Ashman, H. & Simpson, R. M. 1999, 'Editorial', Computing Surveys' Electronic Symposium on Hypertext and Hypermedia, ACM Computing Surveys, vol. 31, no. 4, pp. 325-336.

Atlay, M. & Harris, R. 2000, 'An institutional approach to developing students' 'transferable' skills', Innovations in Education and Training International, vol. 37, no. 1, pp. 76-84.

Australian Bureau of Statistics 1997, Participation in higher education, Author, Canberra.

Australian Bureau of Statistics 1999, Education and training in Australia, Australian Government Publishing Service, Canberra.

Australian Bureau of Statistics 2000, Participation in education, Australia, 1999, Australian Government Publishing Service, Canberra.

Australian Information Industry Association 1996, Demand for skilled people in the IT&I industry: Towards the year 2000, Australian Information Industry Association, Deakin West, ACT.

Australian Information Industry Association 1999, Career tracking research project: Final report, Author, Sydney.

Australian University Teaching Committee 2000, Project brief: Learning outcomes and curriculum development in major disciplines in Information and Communication Technology, [Available: http://www.autc.gov.au/pr/ict.htm].

Australian Vice-Chancellors' Committee 2000, Key statistics: Internationalisation, Author, Canberra.

Australian Vice-Chancellors' Committee 2001a, Higher education Visa statistics , Author, Canberra.

124

Australian Vice-Chancellors' Committee 2001b, International links of Australian universities: Formal links in place or being negotiated between universities in Australia and higher education institutions overseas, Author, Canberra.

Australian Vice-Chancellors' Committee 2001c, Offshore programs of Australian universities: Offshore programs conducted under formal agreements between Australian Universities and overseas higher education institutions or organisations, Author, Canberra.

Bain, J., McNaught, C., Lueckenhausen, G. & Mills, C. 1997, 'Relationship between academics' educational beliefs and conceptions and their design and use of computer software in higher education', in Paper presented at the Annual Conference of the Australian Association for Research in Education, Brisbane.

Bain, J. D. 1999, 'Introduction', Higher Education Research & Development, Special issue: Learning centred evaluation of innovation in higher education, vol. 18, no. 2, pp. 165-172.

Balcytiene, A. 1999, 'Exploring individual processes of knowledge construction with hypertext', Instructional Science, vol. 27, p. 303.

Baldwin, P. 1991, Higher Education Quality and Diversity, Australian Government Publishing Services, Canberra.

Bandura, A. 1977, Social learning theory, Prentice Hall, Englewood Cliffs, New Jersey.

Barett, R. 1992, Improving higher education: Total quality care, Society for Research into Higher Education, Buckingham, Pennsylvania.

Bassey, M. 1999, Case study research in educational settings, Open University Press, Buckingham.

Bates, A. W. 1995, Technology, open learning and distance education, Routledge, London.

Beckman, K., Khajenoori, S., Coulter, N. & Mead, N. 1997, 'Collaborations: Closing the industry-academia gap', IEEE Software, vol. 14, no. 6, pp. 49-57.

Beckman, K., Lawrence, J., Mead, N., O'Mary, G., Parish, C., Unpingco, P. & Walker, H. 2000, Industry/university collaborations: Different perspectives heighten mutual opportunities, [Available: http://www. sei.cmu.edu/collaborating/ed/indust-univ-collabs.html] (December 12).

Bedggood, R. R. & Pollard, R. J. 1999, 'Uses and misuses of student opinion surveys in eight Australian Universities', Australian Journal of Education, vol. 43, no. 2, pp. 129-141.

Bessant, B. 1996, 'Higher education in Australia: The Unified National System', Education Research and Perspectives, vol. 23, no. 1, [Available: http://www.ecel.uwa.edu.au/gse/erp/vol23no1/bessant.html].

Billing, D. 1998, 'Quality management and organisational structure in higher education', Journal of Higher Education Policy Management, vol. 20, no. 2, p. 139.

Boyer, E. L. 1990, Scholarship reconsidered: Priorities of the professoriate, Carnegie Endowment for the Advancement of Teaching, Princeton, New Jersey.

Brennan, J. 1997, 'Authority, legitimacy and change: The rise of quality assessment in higher education', Higher Education Management, vol. 9, no. 1, pp. 7-25.

125

Brown, J. S., Collins, A. & Duguid, P. 1989, 'Situated cognition and the culture of learning', Educational Researcher, vol. 18, no. 1, pp. 32-42.

Bruck, D., Hallett, R., Hood, B., Macdonald, I. & More, S. 2001, 'Enhancing student satisfaction in higher education: The creation of staff teaching communities', Australian Educational Researcher, vol. 28, no. 2, pp. 79-98.

Bruner, J. S. 1967, Toward a theory of instruction, Belknap Press, Cambridge, Massechusetts.

Bruner, J. S. 1977, The process of education, Harvard University Press, Cambridge, Massachusetts.

Caulley, D. N. & Lindsay, J. 2001, 'A job well done: Markers of rigorous qualitative research', in Paper presented at the 2001 Conference of the Association for Qualitative Research , Melbourne, July 5-7.

Coaldrake, P. 2000, 'Rethinking academic and university work', Higher Education Management, vol. 12, no. 3, pp. 7-30.

Computing Education Research Group 2000, AUTC project specification: Learning outcomes and curriculum development in major disciplines in ICT [Unpublished document], Faculty of Information Technology, Monash University, Melbourne.

Cosser, M. 1998, 'Towards the design of a system of peer review of teaching for the advancement of the individual within the university', Higher Education, vol. 35, pp. 143-162.

Couch, M. & McColl, M. 2001, 'Rigour and collaboration in a qualitative inquiry: the strategic use of research methods in a national study of computer-based technology for students with disabilities', in Paper presented at the 2001 Conference of the Association of Qualitative Research , Melbourne, July 5-7.

Crittenden, B. 1999, 'The proper work of educating', Education Research and Perspectives, vol. 26, no. 1 [Available: http://www.ecel.uwa.edu.au/gse/erp/vol26no1/26-1.2.crittenden.html].

Csikszentmihaly, M. 1975, Beyond boredom and anxiety, Jossey-Bass Publishers, San Francisco.

Csikszentmihalyi, M. 1996, Creativity: Flow and the psychology of discovery and invention, Harper Collins Publishers, New York.

Daniels, M., Petre, M. & Berglund, A. 1998, 'Building rigorous research agenda into changes to teaching', The Proceedings of the 3rd Australasian Conference on Computer Science Education pp. 203-209.

Dawkins, J. 1988, Higher education: A discussion paper, Australian Government Publishing Service, Canberra.

Dawkins, J. 1989, Higher education: A policy statement, Australian Government Publishing Service, Canberra.

Dawson, R. & Newsham, R. 1997, 'Introducing software engineers to the real world', IEEE Software, vol. 14, no. 6, pp. 37-42.

Denning, P. J. 1999, 'Computing the profession', The proceedings of the 30th Technical Symposium on Computer Science Education, March 24-28, New Orleans, LA, USA. pp. 1-2.

Department of Employment Education and Training, Department of Industry Technology and Commerce & Information Industries Education and Training

126

Foundation 1992, Report of the Discipline Review of Computing Studies and Information Science education, Australian Government Publishing Service, Canberra.

Dewey, J. 1961, Democracy and education, The Macmillan Company, New York.

Dey, I. 1993, Qualitative data analysis: A user-friendly guide for social scientists, Routledge, London.

Dillon, A. & Gabbard, R. 1998, 'Hypermedia as educational technology: A review of the quantitative research literature on learner comprehension, control, and style', Review of Educational Research, vol. 68, no. 3, pp. 322-349.

Elliot, S. & Hughes, J. 1999, Career Tracking Research Project Final Report, Australian Information Industry Association, Deakin, ACT.

Elton, L. 1993, 'University teaching: A professional model for quality', in Quality assurance for university teaching, ed. Ellis, R., Open University Press, Buckingham, Great Britain, pp. 133-148.

Evans, T. 1997, '(En)countering globalisation: Issues for open and distance education', in Shifting borders: Globalisation, localisation and open and distance education, eds. Rowan, L., Bartlett, L. & Evans, T., Deakin University Press, Geelong, Victoria, pp. 11-22.

Everingham, P. 1999, Education participation in Australia, 1997, Department of Education, Training and Youth Affairs, Canberra [Available: http://www.detya.gov.au/iae/research/edpart.htm].

Faulk, S. R. 2000, 'Achieving industrial relevance with academic excellence: Lessons form the Oregon Master of Software Engineering.', Proceedings of the 22nd International Conference on Software Engineering, June 4-11, 2000, Limerick, Ireland. pp. 293-302.

Fox, R. 2001, ' Constructivism examined', Oxford Review of Education, vol. 27, no. 1, pp. 23-35.

Garlan, D., Gluch, D. P. & Tomayko, J. E. 1997, 'Agents of change: Educating software engineering leaders', Computer, vol. 30, no. 11, pp. 59-654.

Greeno, J. G., Collins, A. M. & Resnick, L. B. 1996, 'Cognition and learning', in Handbook of educational psychology, eds. Berliner, D. C. & Calfee, R. C., Macmillan, New York, pp. 15-46.

Guolla, M. 1999, 'Assessing the teaching quality to student satisfaction relationship: Applied customer satisfaction research in the classroom', Journal of marketing theory and practice, vol. 7, no. 3, pp. 87-97.

Hand, T. & Trembath, K. 1999, Report of the Course Experience Questionnaire Symposium 1998, Commonwealth of Australia, Canberra [Available: http://www.detya.gov.au/archive/highered/eippubs/99-2/ceq.pdf].

Heider, F. 1958, The psychology of the interpersonal relations, Wiley, New York.

Henkel, M. 1998, 'Evaluation in higher education: Conceptual and epistemological foundations', European Journal of Education, vol. 33, no. 3, pp. 285-297.

Hicks, O. 1999, 'Ensuring teaching quality at UWA - Best practice pathways', in Course Experience Questionnaire Symposium 1998, eds. Hand, T. & Trembath, K., Commonwealth of Australia, Canberra, pp. 81-92 [Available: http://www.detya.gov.au/archive/highered/eippubs/99-2/ceq.pdf].

127

Higher Education Council 1991, The future of discipline assessments, Australian Government Publishing Service, Canberra.

Hoang, K. E., Tran, V. H. E., Luu, T. H. E., Phan, V. H. E., Owens, T. E., Nguyen, S. T. E., Vuong, S. T. E., Dong Thi, B. T. E. & Phan Thi, T. E. 1998, 'Proceedings of Selected Research and Development Presentations', in National Convention of the Association for Educational Communications and Technology (AECT), St. Louis, Missouri.

Hoare, D. 1995, Report of the committee of inquiry: Higher education management review, Australian Government Publishing Services, Canberra.

Hutching, P. 1994, 'Peer review of teaching: 'From idea to prototype'', American Association for Higher Education Bulletin, vol. 47, no. 3, pp. 3-7.

IT&T Skills Task Force 2000, Future demand for IT&T Skills in Australia, 1999 - 2004, IT&T Skills Task Force, Melbourne.

Jackson, D. L., Teal, C. R., Raines, S. J., Nansel, T. R., Force, R. C. & Burdsal, C. A. 1999, 'The dimensions of students' perceptions of teaching effectiveness', Educational and Psychological Measurement, vol. 59, no. 4, pp. 580-596.

Jacobs, G. 1992, 'Hypermedia and discovery-learning: An historical perspective', British Journal of Educational Technology, vol. 23, no. 2, pp. 113-121.

Kaplan, R. S. & Norton, D. P. 1996, The Balanced Scorecard: Translating strategy into action, Harvard Business School Press, Harvard.

Kemp, D. 1998, Strategic developments in higher education, Department of Education, Training and Youth Affairs, Sydney [Available: http://www.detya.gov.au/archive/ministers/kemp/ks210498.htm].

Kemp, D. 2001, Higher education report for the 1999 to 2001 triennium, Australian Government Publishing Service, Canberra.

Kemp, D. & Alston, R. 2001, May 17, 'The hub of the IT Skills Exchange', Department of Education, Training and Youth Affairs, Canberra.

Koder, M. 1999, 'Introduction to the 1998 CEQ Symposium', in Course Experience Questionnaire Symposium 1998, eds. Hand, T. & Trembath, K., Commonwealth of Australia, Canberra, pp. 3-4 [Available: http://www.detya.gov.au/archive/highered/eippubs/99-2/ceq.pdf].

Koppi, A. J., Chaloupka, M. J., Llewellyn, R., Cheney, G., Clark, S. & Fenton-Kerr, T. 1998, 'Academic culture, flexibility and the national teaching and learning database', in Presented at the Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, Wollongong [Available: http://www.ascilite.org.au/conferences/wollongong98/ascpapers98.html].

Kreber, C. & Cranton, P. A. 2000, 'Exploring the scholarship of teaching', Journal of Higher Education, vol. 71, no. 4, pp. 476-495.

Lave, J. 1988, Cognition in practice: Mind, mathematics and culture in everyday life, Cambridge University Press, Cambridge.

Lefoe, G. 1998, 'Creating constructivist learning environments on the Web: The challenge in higher education', in Presented at the Annual Conference for the Australasian Society for Computers in Learning in Tertiary Education, Wollongong [Available: http://www.ascilite.org.au/conferences/wollongong98/asc98-pdf/lefoe00162.pdf].

128

Malley, J. 1998, 'The measurement and meaning of student satisfaction in the VET Sector: A review', in Workplace and institute accredited training: Costs and satisfaction, eds. Symmonds, H., Burke, G., Harvey-Beavis, A. & Malley, J., National Centre for Vocational Education Research, Adelaide.

Marginson, S. & Considine, M. 2000, Enterprise University: Power, governance and reinvention in Australia, Cambridge University Press, Cambridge.

Martin, L. H. 1964, Tertiary education in Australia: Report of the Committee on the Future of Tertiary Education in Australia to the Australian Universities Council, Commonwealth Government Printer, Canberra.

Martin, R. 1992, 'Education: Present and two futures', in Education in the 1990s: Competencies, credentialism, competitiveness? ed. Economic Planning Advisory Council, Australian Government Publishing Services, Canberra, pp. 31-39.

Massaro, V. 1997, 'Institutional responses to quality assessment in Australia', in Standards and quality in higher education, eds. Brennan, J., Vries, P. d. & Williams, R., Jessica Kingsley Publishers, Bristol, Pennsylvania, pp. 44-60.

Mayer, R. E. 1992, 'Cognition and instruction: Their historic meeting within educational psychology', Journal of Educational Psychology, vol. 84, no. 4, p. Cognition and instruction: their historic meeting within educational psychology.

McCaslin, M. & Hickey, D. T. 2001, 'Educational psychology, social constructivism and educational practice: A case of emergent identity', Educational Psychologist, vol. 36, no. 2, pp. 133-140.

McInnis, C. 1999, The work roles of academics in Australian universities, Australian Government Publishing Service, Canberra [Available: http://www.detya.gov.au/archive/highered/eippubs/eip00_5/fullcopy.pdf].

McInnis, C., Griffin, P., James, R. & Coates, H. 2001, Development of the course experiences Questionnaire (CEQ), DETYA, Canberra.

Moses, I. & Trigwell, K. 1993, Teaching quality and quality of learning in professional courses, Australian Government Publishing Services, Canberra.

Nahapiet, J. & Ghoshal, S. 1998, 'Social capital, intellectual capital, and the organizational advantage', Academy of Management Review, vol. 23, no. 2, pp. 242-266.

National Office for the Information Economy 1998, Skill shortages in Australia's IT&T industries: Discussion paper, National Office for the Information Economy, Canberra.

National Office for the Information Economy 1999, January, 'A Strategic Framework for the Information Economy', Commonwealth of Australia, Canberra.

Nicaise, M. & Crane, M. 1999, 'Knowledge constructing through HyperMedia authoring', Educational Technology Research and Development, vol. 47, no. 1, pp. 29-50.

Oliver, M. 2000, 'An introduction to the evaluation of learning technology', Educational Technology & Society, vol. 3, no. 4 [Available: http://ifets.gmd.de/periodical/vol_4_2000/intro.html].

O'Neil, H. F., Bensimon, E. M., Diamond, M. A. & Moore, M. R. 1999, 'Designing and implementing an academic scorecard', Change, vol. 31, no. 6, pp. 32-40.

129

Patton, M. Q. 1990, Qualitative evaluation and research methods (Second Edition), Sage Publications, Newbury Park, California.

Peters, M. & Roberts, P. 1999, 'Globalisation and the crisis in the concept of the modern university', Australian Universities Review, vol. 42, no. 1, pp. 47-55.

Phillips, J. J. 1997, Handbook of training evaluation and measurement methods, Gulf Publishing, Houston.

Poole, P. C. 1997, Information technology: Sink or swim?, Academy of Technological Sciences and Engineering,, Gold Coast, Queensland.

Postema, M. & Markham, S. 2001, 'Student satisfaction: A method for exploring quality factors within computing education', in Presented at the Annual Conference of the National Advisory Committee on Computing Qualifications, Napier, New Zealand.

Qualitative Solutions & Research 1999, QSR NUD*IST VIVO [software], Qualitative Solutions & Research, Bundoora, Victoria.

Rice, R. E. 1991, 'A new American scholar: Scholarship and the purpose of the university', Metropolitan University, vol. 1, pp. 7-18.

Rogers, E. M. 1983, Diffusion of innovations, Free Press, New York.

Rumble, G. 1989, ''Open learning', 'distance learning', and the misuse of language', Open Learning, vol. 4, no. 2, pp. 28-36.

Samuelowicz, K. & Bain, J. 2001, 'Revisiting academics' beliefs about teaching and learning', Higher Education, vol. 41, pp. 299-325.

Shattock, M. L. 1995, 'The university of the future', Higher Education Management, vol. 7, no. 2, pp. 157-163.

Shuell, T. J. 1986, 'Cognitive conceptions of learning', Review of Educational Research, vol. 56, no. 4, pp. 411-436.

Silverman, D. 2000, Doing qualitative research: A practical handbook, Sage Publications, London.

Skinner, R. A. & Cartwright, G. P. 1998, 'Higher education and the technology workforce shortage', Change, vol. 30, no. 3, pp. 52-55.

Strauss, A. L. & Corbin, J. M. 1997, Grounded theory in practice , Sage, Thousand Oaks, California.

Stringer, M. & Finlay, C. 1993, 'Assuring quality through student evaluation', in Quality assurance for university teaching, ed. Ellis, R., Open University Press, Buckingham, Great Britain, pp. 92-113.

Symes, C. 1999, ''Working for your future': The rise of the vocationalised university', Australian Journal of Education, vol. 43, no. 3, pp. 241-256.

Szulanski, G. 1994, Intra-firm transfer of best practices project: Executive summary of findings, American Productivity & Quality Center, Houston, Texas.

Taylor, P. G. 1999, 'Linking developmental-cycles to career progress: Recognising and rewarding ICT-enriched teaching', in Paper presented at the 6th International ALT-C Conference , University of Bristol.

Taylor, P. G., Lopez, L. & Quadrelli, C. 1996, Flexibility, technology and academics' practices: Tantalising tales and muddy maps, Australian Government Publishing Service, Canberra, ACT.

130

Taylor, P. G. & Richardson, A. S. 2001, Validating scholarship in university teaching: Constructing a national scheme for external peer review of ICT-based teaching, Commonwealth of Australia, Canberra [Available: http://www.detya.gov.au/highered/eippubs/eip01_3/01_3.pdf].

Trembath, R. 1999, 'The Institutional Arrangements for Student Feedback project', in Course Experience Questionnaire Symposium 1998, Commonwealth of Australia, Canberra, pp. 27-29 [Available: http://www.detya.gov.au/archive/highered/eippubs/99-2/ceq.pdf].

Tuovinen, J. E. 2000, 'Multimedia distance education interactions', Educational Media International, vol. 37, no. 1, pp. 16-24.

Vanstone, A. 1996, Higher Education Budget Statement 1996, Department for Employment, Education, Training and Youth Affairs, Canberra [Available: http://www.detya.gov.au/archive/publications/budget/budget96/budget_statement/hedst.htm].

von_Glaserfeld, E. 1998, 'Why constructivism must be radical', in Constructivism and education, eds. Larochelle, M., Bednarz, N. & Garrison, J., Cambridge University Press, Cambridge, pp. 23-28.

Webberley & Haffenden 1990, Open Learning, NBEET, 4.

West, R. 1998, Learning for life: Final report of the review of higher education financing and policy, Australian Government Publishing Services, Canberra.

Willmot, M. & McLean, M. 1994, 'Evaluating flexible learning: A case study', Journal of Further and Higher Education, vol. 18, no. 3, pp. 99-108.

Wittingslow, G. W. & Markham, S. J. 1999, ' Customer-driven model of satisfaction behaviour', Australasian Journal of Market Research, vol. 7, no. 2, pp. 29-38.

Wittrock, M. C. 1974, 'Learning as a generative process', Educational Psychologist, vol. 11, no. 2, pp. 87-95.

Wittrock, M. C. 1978, 'The cognitive movement in instruction', Educational Psychologist, vol. 13, pp. 15-29.

131

APPENDIX A. Reference group membership

Angela Carbone Senior Lecturer School of Info Management Systems

Monash University

Ainslie Ellis Senior Lecturer School of Comp Sci & Software Eng

Monash University

Assoc. Prof. Malcolm Eley Associate Professor Centre for Learning & Teaching Support

Monash University

Dianne Hagan Senior Lecturer School of Comp Sci & Software Eng

Monash University

Assoc. Prof. John Hurst Assoc. Dean (Teaching) Faculty of Info Tech

Monash University

Dr Selby Markham Research Fellow Faculty of Info Tech

Monash University

Dr Juhani Tuovinen Senior Research Fellow Centre for Multimedia & Hypermedia Research

Monash University

Judithe Sheard Lecturer School of Comp Sci & Software Eng

Monash University

133

APPENDIX B. Mini-conference participants

William Adlong Educational Designer

CELT Charles Sturt University

Carole Alcock Lecturer School of Info Tech & Comp Sci

University of Wollongong

Dr Rob Allen Senior Lecturer School of Info Tech

Swinburne University of Technology

Robert Andrews Lecturer School of Info Systems

Queensland University of Technology

Jocelyn Armarego Lecturer School of Eng

Murdoch University

Martin Atchison Lecturer School of Info Management Systems

Monash University

John Atkinson Lecturer School of Environmental & Info Sciences

Charles Sturt University

Kevin Austin Assoc Lecturer School of Management

University of Western Sydney

Dr Reuven Aviv Senior Lecturer School of Comp Sci & Software Eng

Monash University

Dr Mahfuz Aziz Senior Lecturer School of Electrical & Info Eng

University of South Australia

Dr Clem Baker-Finch Senior Lecturer

Department of Comp Sci

Australian National University

Maria Blanco Barajas Assoc Lecturer Department of Computer Systems

University of Technology, Sydney

Monica Bayer Administrator International & Offshore Programs School of Computer & Info Science

University of South Australia

John Bentley Senior Lecturer School of Info Systems

Victoria University

Clive Boughton Lecturer

Department of Comp Sci

Australian National University

Ilona Box Lecturer School of Management

University of Western Sydney

Julie Brear Lecturer School of Informatics

Northern Territory University

Allison Brown Head Learning Resources Development Unit

University of Queensland

Prof Bill Caelli Head School of Data Communications

Queensland University of Technology

Malcolm Campbell Senior Lecturer School of Computing & Mathematics

Deakin University

Angela Carbone Lecturer School of Info Management Systems

Monash University

Penny Collings Senior Lecturer School of Computing

University of Canberra

Lorraine Connell Lecturer School of Informatics

Northern Territory University

Annemieke Craig Lecturer School of Info Systems

Victoria University

Dr David Cropley Senior Lecturer School of Electrical & Info Eng

University of South Australia

134

Dr Paul Crowther Lecturer School of Computing

University of Tasmania

Gordana Culjak Lecturer Department of Info Systems

University of Technology, Sydney

Assoc Prof John Dearn Director CELTS

University of Canberra

Martin Dick Lecturer School of Comp Sci & Software Eng

Monash University

Michael Docherty Lecturer School of Comp Sci & Electrical Eng

University of Queensland

Peter Donnan Instructional Designer CELTS

University of Canberra

Assoc Prof Carolyn Dowling Head School of Arts & Sciences

Australian Catholic University

David Edmond Senior Lecturer School of Info Systems

Queensland University of Technology

Bob Edwards Lecturer Department of Comp Sci

Australian National University

Assoc Prof Jenny Edwards Assoc Prof Department of Software Eng

University of Technology, Sydney

Sylvia Edwards Lecturer School of Information Systems

Queensland University of Technology

Ken Eustace Lecturer School of Info Studies

Charles Sturt University

Lynne Fowler Lecturer School of Eng

Murdoch University

Royce Glemi Learning Systems Administrator School of Electronic Commerce

Notre Dame, Australia

Rohan Genrich Assoc Lecturer Department of Info Systems

University of Southern Queensland

Jennifer Goddard Head and Assoc Dean (Teaching) School of Management, Tech & Environment

La Trobe University (Bendigo)

Assoc Prof Ozdemir Gol Assoc Prof School of Electrical & Info Eng

University of South Australia

Dr Robert Goodwin Lecturer School of Informatics & Eng

Flinders University

Dr Edward Gould Senior Lecturer Department of Info Systems

University of Wollongong

Tony Greening Senior Lecturer Basser Department of Comp Sci

University of Sydney

Jackie Hartnett Lecturer School of Computing

University of Tasmania

Neville Holmes Hon Research Associate School of Electrical & Computer Eng

University of Tasmania

Wanda Jackson Educational Designer ILD - Multimedia

Northern Territory University

Anita Jawary Lecturer School of Computing & Mathematics

Deakin University

Alanah Kazlauskas Lecturer School of Business & Informatics

Australian Catholic University

Prof Robin King Pro Vice Chancellor and Dean Division of Info Tech, Eng & the Environment

University of South Australia

Anjan Kundu Assoc Dean Faculty of Tech & Industrial Education

Northern Territory University

John Lamp Lecturer School of Management Info Systems

Deakin University

135

Dr Richard Langman Senior Lecturer School of Eng

University of Tasmania

Michael Lawrence-Slater Lecturer School of InfoTech & Comp Sci

University of Wollongong

Rod Learmonth Senior Lecturer School of Info Tech

Griffith University

Brian Lederer Lecturer Department of Software Eng

University of Technology, Sydney

Kam Li Lecturer School of Computer & Info Science

University of South Australia

Dr Raymond Lister Senior Lecturer Department of Software Eng

University of Technology, Sydney

Ee Kuan Low Assoc Lecturer Department of Info Systems

University of Southern Queensland

Jo Luck Senior Lecturer Faculty of Informatics & Communications

Central Queensland University

Clive Maynard Head Department of Computer Eng

Curtin University

Dr Chris McDonald Senior Lecturer School of Comp Science & Software Eng

University of Western Australian

Penny McFarlane Lecturer School of Info.Tech. & Comp Sci

University of Wollongong

Joseph Meloche Lecturer School of Info Studies

Charles Sturt University

Ed Morris Senior Lecturer Centre for Intelligent Info Processing Systems

RMIT University

Prof Andrew Nafalski Head School of Electrical & Info Eng

University of South Australia

Jon Pearce Senior Lecturer Department of Info Systems

University of Melbourne

Margot Postema Lecturer School of Comp Sci & Software Eng

Monash University

Michael Roggenkamp Assistant Dean School of Computer Sci & Software Eng

Queensland University of Technology

Amit Rudra Lecturer School of Info Systems

Curtin University

Tony Sahama Assoc Lecturer School of Info Systems

Queensland University of Technology

Dr Andy Simmonds Senior Lecturer Department of Computer Systems

University of Technology, Sydney

Dr Jill Slay Director of Overseas Programs School of Computer & Info Science

University of South Australia

Lorraine Staehr Lecturer School of Management, Tech & Environment

La Trobe University (Bendigo)

Asst Prof Steve Sugden Assistant Prof School of Info Tech

Bond University

Assoc Prof Richard Thomas Assoc Prof Department of Comp Sci

University of Western Australia

Michael Turk Lecturer School of Computing & Info Tech

University of Western Sydney

Jodi Tutty Lecturer School of Informatics

Northern Territory University

Sue Tyerman Lecturer School of Computer & Info Science

University of South Australia

Richard Walker Assoc Lecturer and Honours Convenor Department of Comp Sci

Australian National University

136

Prof Warren Yates Assoc Dean (Teaching & Learning) Telecommunications Eng

University of Technology, Sydney

Catherine Zuluaga Lecturer Department of Comp Sci

RMIT University

137

APPENDIX C. Intensive workshop participants

Penny Collings Senior Lecturer School of Computing

University of Canberra

Assoc Prof Kit Dampney Associate Professor School of Information Systems

University of Newcastle

Jennifer Goddard Associate Dean (Teaching) Dept of IT, Faculty at Bendigo

La Trobe University

Dr Chris Johnson Head Department of Computer Science

Australian National University

Catherine Lang

Lecturer School of IT

Swinburne University

Dr Chee-Kit Looi President Asia-Pacific Chapter Association for the Advancement of Computing in Education (AACE)

National University of Singapore

Prof Graham Low Head School of Info Systems, Tech and Management

University of New South Wales

Prof Leon Sterling

Head Department of Comp Sci and Soft Eng

University of Melbourne

Prof Geoff West

Head School of Computing

Curtin University

Assoc Prof David Wilson Associate Dean (Education) Faculty of IT

University of Technology, Sydney

139

APPENDIX D. Methodology – Mini-conference program

This section describes the aims, data collection methods, participants, analyses and reporting techniques for the part of the study that used mini-conferences to elicit data from ICT educators. It also discusses the strengths and limitations of the approaches taken in terms of Caulley and Lindsay’s (Caulley & Lindsay 2001) framework for markers of rigour in qualitative research. The methods described here correspond with the phase referred to in the Project Specification document as Consultation with ICT educators (see Table 1 in Section 1). They were implemented in response to those sections of the Brief that refer to the ways in which issues of teaching and learning have been approached in ICT departments, the factors to which innovations have responded, and the directions that have been taken. They also provided information on current teaching and learning initiatives and on dissemination of innovation.

Aims and objectives The broad aim of the mini-conference program was to investigate the ways in which ICT educators are approaching issues of teaching and learning. Two types of data were sought: (1) data on educators’ perceptions of the issues currently surrounding educational innovation and the dissemination of good practice and (2) data on particular teaching and learning initiatives that are currently being planned, implemented and/or evaluated.

This data was collected to meet the three aims of the Consultation with ICT educators phase of the research (see Table 1 in Section 1). Those aims were

• To investigate current innovations in ICT education

• To investigate ICT educators’ views on developments in teaching and learning and the dissemination of innovation

• To create a positive feel for the project among ICT educators

Methods associated with each of these aims are described in separate sections below, headed Current innovations in ICT education, ICT educators’ views and Reception of the project respectively, but first the mini-conference program and the participants are described.

Mini-conference program The data collected for this part of the study was qualitative. A mini-conference format was devised to facilitate the discussion of issues and the description of initiatives. The research project is, in the most part, a national review study. The national mini-conference program was intended to maximise the opportunity for participation of ICT departments from around Australia. Mini-conferences were held in each capital city. Two mini-conferences were held in Melbourne and in

140

Sydney. One mini-conference was held in each of Hobart, Brisbane, Adelaide, Perth and Canberra. A video-link was used to conduct an abridged version of the mini-conference with educators in the Northern Territory. The mini-conference format, materials and data sources are described below.

Mini-conference format

The mini-conferences ran from 10am to 4pm on designated days in each capital city during February, March, April and May of 2001. Between four and fourteen people participated in each mini-conference. Small numbers allowed for authentic discussions and also allowed time for each participant to present information about teaching and learning initiatives underway in their institutions. A workbook was used to structure the proceedings of each mini-conference, breaking the day up into five main sessions. Two researchers directed proceedings.

The workbooks

Workbooks were provided to each participant on their arrival at a mini-conference. They were collected by the researchers at the end of the day. Photocopies were provided to participants. The workbooks served a number of purposes:

• To provide a structure for the mini-conferences

• To provide participants with somewhere to make notes in preparation for their contributions to the mini-conferences and about the contributions of others

• To provide participants with a record of their participation and of notes taken

• To provide the researchers with a record of notes made by participants

• To provide researchers with outline information about the teaching and learning initiatives reported

The workbooks contained a description of the study, an informed consent form, space for participants to make notes and five stimuli items:

1. What are the factors driving educational initiatives in ICT education?

2. What are the factors inhibiting educational initiatives in ICT education?

3. Please describe a teaching and learning initiative in which you or your department are currently involved. (including sub-prompts)

4. Deciding whether your teaching initiative is effective. (including sub-prompts)

5. What issues would be involved in disseminating your initiative to other ICT educators?

A sample workbook is given in Appendix E.

Items 1, 2 and 5 were used to stimulate open discussion among participants. For each question, participants were given five minutes to think about and make notes in response to the item before they were prompted to discuss the issues involved. As issues were raised and discussed, one of the researchers recorded the issues on a whiteboard. This often involved negotiation between participants. At times, the researchers asked for issues to be clarified. The researchers retained

141

whiteboard printouts as a data source. Photocopies of these were supplied to participants for their records.

Items 3 and 4, along with accompanying sub-prompts, were used to help participants prepare for presentations to the group about specific teaching and learning initiatives. Sub-prompts for item 3 were the following headings:

• Background of your initiative

• Goals of your initiative

• Life-cycle of your initiative (including figure)

• Sub-prompts for item 4 included the following:

• Strengths of your initiative

• Limitations of your initiative

• Please give details of any planned, current or completed evaluation of your teaching and learning initiative

Participants were first given fifteen minutes6 to prepare a ten-minute presentation on their initiative in response to item 3. A blank page was provided for notes under “Please describe a teaching and learning initiative in which you or your department are currently involved”. Another blank page was provided for responses to the remaining prompts for item 3. The life-cycle prompt was accompanied by an unmarked timeline showing “planning”, “implementing” and “continuing” from left to right. Participants were encouraged to mark-up the time line, or to draw other diagrams, as they saw appropriate. Each participant at the mini-conference then gave his or her presentation. A few minutes were provided after each presentation for questions from the group.

For item 4 and its sub-prompts, participants were given seven minutes to prepare and five minutes to present to the group. Again, after each presentation, presenters responded to questions from the group. At mini-conferences where the attendance exceeded seven participants, the group was split into two for these sessions, with each group moderated by a researcher. The researchers made notes during these presentations, then summarised the presentations of the other group once participants had reassembled.

Data sources

Data sources at the mini-conferences included the following:

• Transcribed audio-recordings of discussions (approximately 900 minutes)

• Transcribed workbook entries for each participant (83 in total)

• Whiteboard printouts (or, in some cases, transcriptions) (30 boards in total)

Participants To recruit ICT educators to the mini-conference program, invitations were sent to all heads of Australian university departments, schools and divisions where ICT-related courses are offered. This contact with ICT departments was consistent

6 Times given here are approximations. Actual times varied according to the numbers of participants.

142

with the requirements of the phase of the research referred to in the project specifications as Establishing parameters. Heads contacted included those of departments of information systems, software engineering and computer science, for example. The invitations that were sent to department heads described the target group of staff teaching in ICT-related areas who are involved in teaching and learning initiatives that aim to improve ICT education. Department heads were encouraged to circulate invitations to relevant staff members.

In total 83 ICT educators from 51 administrative units (46 departments/schools, 1 faculty, 4 university level education units) in 29 universities (27 public, 2 private) participated in the mini-conference program. Participants included staff members from a range of academic levels, including all levels of lecturer, department heads, professors, associate and faculty deans, one pro-vice chancellor and a small number of instructional designers.

For a number of reasons, participants can be described as a group of ICT educators who are specifically interested in teaching and learning. First, the invitation to participate described the target group as staff members who are involved in teaching and learning initiatives that aim to improve ICT education. Second, participants self-selected into the study knowing that it focused on teaching and learning. Finally, the nature of participants’ contributions indicated that they were not only interested in teaching and learning, but were enthusiastic about and committed to improving teaching and learning in their disciplines. Therefore, participants are not a representative sample of ICT educators, but rather those ICT educators who were both interested in participating in a study with an education focus and able to attend on one of the days offered.

When informally asked why they attended, participants generally gave reasons such as, “to find out what other people were doing.” When asked what they had gained from participating, two benefits were frequently given: an opportunity to hear about what was going on in other institutions and an opportunity to reflect on and receive feedback about their own practice. An interesting observation about the participants is that the majority were from institutions that, prior to the introduction of the unified national system, were predominantly colleges or institutes of advanced education, that is, institutions where teaching was the primary activity.

Of the eighty-two participants, eighty are from government-funded universities. Of these, fifty are from universities that, prior to the reforms, were predominantly colleges or institutes of advanced education, while of the thirty participants from pre-Dawkins universities, at least ten are employed at old CAE campuses. This bias helps us to describe the participants and it has implications for the findings.

Current innovations in ICT education This section describes data collection, analysis and reporting methods associated with the aim to investigate current innovations in ICT education. It should be remembered that this aim was pursued in the context of a review study.

Data collection

As described above, each mini-conference participant was asked to give a brief, impromptu presentation on a teaching and learning initiative in which he or she is

143

involved. Each of the eighty-three participants gave presentations; however, in some cases, more than one participants presented about the same initiative. In total, 79 teaching and learning initiatives were presented to the researchers and other participants.

Data collected on particular teaching and learning initiatives included the written records from participants’ workbooks and the audio-recordings of participants’ presentations and ensuing discussions. In some cases, further information was collected from participants after the mini-conferences via email.

Analysis

The review process undertaken focussed on identifying and describing types and trends, rather than the tailored evaluation of specific innovations. Processes were also undertaken to ensure that each participant’s contribution to the research was acknowledged and that particular initiatives were appropriately attributed.

Brief accounts

As a first response to the data collected at the mini-conference on particular teaching and learning initiatives, brief descriptive accounts were developed for each initiative. These accounts were based primarily on the information supplied in participants’ workbooks. In some cases, audio-recordings were also referred to. The brief accounts were then emailed to respective participants for approval. This approval processes increased the validity of accounts, enabling clarification of details and the collection of additional information. Participants were asked to supply details of the people, institutions or funding who should be acknowledged and of Web sites where more information about the initiatives could be found. When initiatives did not have Web sites, those containing the contact details of the participants were collected. Approved accounts of teaching and learning initiatives were posted on the Project’s Web-site (http://cerg.infotech.monash.edu.au/icted/) and appear in Appendix J. Of the seventy-nine initiatives described, the accounts of fifty were later expressly approved. No accounts were expressly not approved.

Foci and motivation

Initiatives (both those for which accounts were approved and those that were not) were then coded according to the reported motivation behind the initiative and the reported aim or focus of the initiative. Both workbook and audio-recorded data were used to code initiatives.

Evaluation

Initiatives were then coded according to information supplied about their evaluation. This involved reviewing the information supplied to determine if a formal evaluation had been undertaken. For those initiatives that had been evaluated, evaluation methods were coded. It was also noted whether academic papers had been produced on the initiatives and whether the initiative was subject to an investigation linked to formal study, such as doctoral research.

Markers of rigour

144

Caulley and Lindsay (2001) identify from the qualitative research literature three key interrelated areas that need to be examined as sources of rigour in qualitative research: research relationships, data handling and research results. Each of these areas, as they pertain to the methods described above, is discussed below. Caulley and Lindsay’s work is useful here because, although it draws on a wide range of complex literature on qualitative research, it provides a simple framework for interrogating research methods. This framework will be used to discuss the rigour of the research described in each section of this appendix.

Research relationships

In terms of relationships with participants, qualitative research should be open, honest and fair (Caulley & Lindsay 2001). This requires researchers to critically examine the research context, the role they play, and the power-relationships involved. Caulley and Lindsay (2001) suggest considering the relationship between the researchers and the participants, the interests the research serves, and the audience of the research. These contextual factors play an important role in the credibility and trustworthiness of the research.

This Project has many characteristics that led to layers of power differentiation between the people involved. First, the Project is a national, government-funded study, so it is of high status and high profile. Second, the Faculty of Information Technology at Monash University is conducting the research. This Faculty is not only the largest IT faculty in Australia but one of the largest in the world. Third, recruitment of participants was conducted via department heads. These characteristics of the project have implications for the type of ICT educator that chose to participate, for their motivations for participation and for the quality of their participation. They bring an element of performance to participation that would have been less likely to occur in a study of lower status or profile. In fact, the mini-conferences were sold to potential participants as venues for self-promotion. Invitations to participate named one of the benefits to participants as “the opportunity to promote their own teaching and learning initiatives”.

Relationships between participants also involved power differentials. Not all participants were of equal seniority, with participants varying in rank from Level A lecturers and tutors to Pro-Vice Chancellor and other administrative levels. This power differential has implications for the truthfulness of contributions, as does the fact that participants were drawn from competing university departments. Possibilities include participants’ overemphasising the positive attributes of particular initiatives or withholding information.

The relationship between participants and researchers was probably the least fraught of the research context. The researchers were not peers of the participants, with neither of them having a background in ICT education. Nor were they permanent Monash staff; their recruitment for the purpose of implementing this research project allowed some distancing from the Faculty of Information Technology at Monash University. Every effort was made to ensure that the atmosphere at mini-conferences was informal, including the use of “ice-breaking” warm-up activities. As a result, the mini-conferences generally had a collegial and collaborative atmosphere.

An important feature of credible and trustworthy qualitative research is quality engagement in the field. The part of the mini-conferences that focused on teaching and learning initiatives were not prolonged. Time constraints were such

145

that no more than a basic understanding of the concept of each initiative was possible. Pursuing alternate accounts of initiatives or other sources of data was outside of the scope of the study.

Data handling

The methods used to investigate specific teaching and learning initiatives did not allow for in-depth analysis. They were conducted in the context of a review study and were constrained by limitations on time and resources. The validity of accounts of particular initiatives was increased by the use of member checking. Each participant was asked to approve the account of his or her initiative and was encouraged to make any changes thought necessary. The validity of coding of teaching and learning initiatives was increased by the use of multiple analysts.

Research results

Qualitative research usually involves in-depth analysis. However, this section of the mini-conferences did not. Instead, snapshots of particular teaching and learning initiatives were collected. In the presentation of research results, the phenomena being described are not specific ICT teaching and learning initiatives, but instead an overview of educational innovations in ICT disciplines. When considering issues of transferability, the questions asked should focus on this intention of the research. That is, rather than asking “how do the conclusions apply to this other instance of innovation” one should ask “where does this instance of innovation fit within the description given of the types of innovations being undertaken generally. Caulley and Lindsay (2001) pointed out that transference is the responsibility of the reader. It is done after the research has been reported. In terms of rigour, we should ask, “how well does the reporting of the research results allow ICT educators to position, or even to better describe, their own teaching and learning initiatives.”

In terms of fairness of reporting, all participants were acknowledged as innovators regardless of the relative size or value of the teaching and learning initiatives described. All initiatives were attributed to the participants who presented them, with other attributions and acknowledgements made were appropriate. Member checking was used to clarify details of initiatives and to attribute them appropriately.

This part of the mini-conference had both educative and catalytic authenticity in that participants were given the opportunity to learn about the innovative practices of their peers, to network with peers from other institutions with common interests, to receive feedback on their own teaching and learning initiatives and some participants indicated that they would “try out” what they had heard.

Summary of limitations

The main limitation of the innovations section of the mini-conference research was the review design. Engagement with the initiatives was limited and only sketches of each initiative were developed. While conclusions can be made about evaluation of educational initiatives generally, the data does not provide for a tailored evaluation of particular teaching and learning initiatives. The data did provide for a description of the realm of innovation in ICT education generally,

146

particularly the types of initiatives being undertaken and the motivation behind them.

Another limitation was the reliance on participants’ self-reports of initiatives. The usual limitations of self-report data are aggravated here because of the performance element introduced by the research context and the study design.

ICT educators’ views This section describes data collection, analysis and reporting methods associated with the aim to investigate ICT educators’ views on developments in teaching and learning and the dissemination of innovation.

Data collection

An open approach to collecting data on ICT educators’ views was used to increase the salience of factors identified. A qualitative approach ensured that the data collected on complex and interrelated factors and issues were rich, enabling detailed descriptions and ensuring a degree of transferability of conclusions. Data collection for this phase was highly participative, increasing the quality of engagement with, and the credibility of conclusions about, one of the key stakeholder groups (ICT educators) in ICT education.

As described above, participants responded to items 1, 2 and 5 by making notes in their workbooks and then by engaging in open discussions. During discussion, participants raised and discussed the factors and issues they had noted in their workbooks. As factors and issues were discussed a moderator from the research team listed them on an electronic whiteboard. At times this required negotiation between participants. Whiteboard printouts were collected by the researchers; copies were provided to participants for their records.

Analysis

A coding process, using NUD*IST VIVO (Qualitative Solutions & Research 1999), was used to reduce the data collected. The aim the coding process was to reduce the data, so an intelligible account could be produced (Dey 1993). This objective is distinct from the development of theory: the aim here is communication. To begin with, a data-up, rather than theory-down, approach was taken to the analysis. For questions 1, 2 and 5, notes made in participants’ workbooks and whiteboard records of discussions were coded into a large number of descriptive categories. These categories were then grouped, reducing the data further. They were then tested against transcriptions of audio-recordings of discussions, using the constant comparative method (Silverman 2000; Strauss & Corbin 1997). As data were compared categories were merged and revised. The resulting small number of categories allowed for the complexity of particular issues/factors to be captured in each category and examined. Particularly rich excerpts were coded into multiple categories. To increase the reliability of this coding process, a co-researcher inspected reports of each code for consistency.

147

Markers of rigour

The discussion of markers of rigour for this section of the study uses Caulley and Lindsay’s (Caulley & Lindsay 2001) framework as described earlier.

Research relationships

The key relationships involved in the research context were described above under Research relationships in the section on Current innovations in ICT education. The presence of different academic levels in the open discussion sections of the mini-conferences has possible implications for the candidness of contributions. However, participants did not appear to be inhibited even when discussing controversial issues. This is evidenced by participants’ willingness to criticise management even when their own department heads were present. The implications of the various power differentials present in the Project are not as great for these sections of the mini-conferences as they are for those sections dealing with particular teaching and learning initiatives. The open-discussion sections involved discussing general issues, whereas the innovations sections focused on specific initiatives owned by specific participants. In this way, participants were less vulnerable and the data less likely to be distorted by participant performance for the open discussion sessions.

Over the period of the mini-conference program, saturation point was reached in terms of the issues discussed, with the same issues being raised at each mini-conference.

Data handling

QSR NVIVO (Qualitative Solutions & Research 1999) was used to manage discussion data. Data included workbook transcripts, whiteboard printouts and transcripts of the audio recordings of discussions. The whiteboard printouts were used as a starting point for coding because they contained dot-point records of each discussion that had been negotiated among and agreed on by participants. An analysis of whiteboard printouts from discussions at different mini-conferences provided a starting point for themes that could then be tested against the richer records found in the audio transcripts. Trustworthiness in the data analysis was increased through the use of multiple analysts and through peer review of findings. Findings were peer reviewed in preparation for conference proceedings. Research reports were also made available to participants for comment; though, few chose to do so.

Research results

The purpose of the analysis of the discussion data was to provide a coherent account of the views reported across the mini-conference program. Results were reported in the form of descriptions for each theme. Excerpts from the audio transcripts were given as illustrations of each theme. The presentation of these themes at seminars elicited recognition in ICT educators who saw the themes as reflecting their own experiences and views.

Summary of limitations

148

A main limitation of the open discussion sections of the mini-conferences is the way in which participants were recruited. Participants self-selected into the study. We can safely assume that those who participated have an interest in teaching and learning. Participants were not a random sample of ICT educators and we cannot safely make assumptions about those ICT educators who chose not to participate. We do not know why ICT educators might have chosen not to participate. Possible reasons include unavailability on the days offered, inability to invest the time required (a whole day), lack of interest in the subject of the study, or failure of departmental contacts to pass on invitations.

Observations and conclusions made here need to be qualified by an acknowledgment of the types of people who provided data. However, some comfort can be taken in the diversity found in those people who did participate, and in the consistency of issues raised despite this diversity. Participants included both junior and senior staff from a range of ICT subject areas. Though the majority of participants were teaching staff, administrators and support staff were also represented. Both men and women participated (66% and 34% respectively). Participants were from a large number of universities and each state and territory was represented. Despite this diversity, the same issues were repeatedly raised at each mini-conference.

A further limitation to the study is due to the type of data that was collected. The mini-conferences were used to collect qualitative data only. While they yielded a rich body of data on issues of concern to ICT educators, we do not have data on the relative importance of issues.

Reception of the project The mini-conference program was the first point of contact between the Project and members of the ICT education community and an important phase in terms of setting the collaborative tone of the overall project. In total, eighty-three ICT educators from twenty-nine of Australia’s universities participated in the program. The number of participants at each mini-conference ranged from four to fourteen, with fourteen being the maximum number of participants that could be accommodated on any one day. Only small numbers were attracted to the Hobart and Darwin mini-conference with only one university existed in Tasmania and the Northern Territory. The Project team was satisfied that the participants represented a range of view points from a range of different university departments. In total, fifty-one administrative units were represented, with representatives ranging across all academic levels. In terms of the data collected, saturation point was reached both for the general issues raised and the type and focus of initiatives presented.

The atmosphere of the mini-conferences was open and congenial. Informal feedback indicated that participants saw the activities as worthwhile in terms of the research and also in terms of their own time investment. Two participants provided unsolicited feedback in writing at the end of the mini-conferences in which they participated:

A participant in the second Sydney mini-conference wrote:

• I felt the time was well spent

• Well run and facilitated

149

• fun/ enjoyable/ challenging

• Met interesting colleagues

• Felt that I learnt more than I gave

• Boosted my enthusiasm and encouraged me to keep on innovating.

A participant in the Adelaide mini-conference wrote:

• well organised/ well run

• open/frank exchange of views

• free flowing format

• useful to hear others’ views/ experience.

151

APPENDIX E. Mini-conference workbook

ICT-Ed: Innovations

in ICT education

mini-conference

http://cerg.infotech.monash.edu.au/icted

Monash Conference Centre, Melbourne

February 12 and 14, 2001

M O N A S H U N I V E R S I T Y

152

AUSTRALIA

Explanatory Statement

Project Title: Educational innovations in ICT education – Mini-conferences.

Investigators: Julianne Lynch Senior Research Fellow

Francesca Collins Research Assistant

Supervising committee: Ass. Prof. John Hurst Dianne Hagan Dr. Selby Markham

Ainslie Ellis Angela Carbone

Judithe Sheard Ass. Prof. Malcolm Eley Dr. Juhani Tuovinen Allison Brown

The Computing Education Research Group, in the Faculty of Information Technology, at Monash University has been contracted to investigate teaching and learning initiatives within the Information and Communication Technology (ICT) discipline in Australian Universities. This project is funded by the Department of Education Training and Youth Affairs (DETYA) through the Australian Universities Teaching Committee (AUTC). The project is multi-facetted. Information on the various phases of the research can be found on our Web site at http://cerg.infotech.monash.edu.au/icted. The statement that follows concerns only Phase two, which makes use of mini-conferences as a means to establish links with and between ICT educators and to collect preliminary data.

Aims:

The aims of the mini-conferences are

To identify ICT educators' perceptions of the main factors driving educational innovations and best practice in the discipline

To identify current teaching and learning initiatives aimed at improving ICT education

To identify ICT educators' perceptions of the main issues currently facing the discipline

To identify issues involved in the dissemination of information on educational innovations and best practice in the discipline Participants:

153

University departments, from across Australia, that are involved in ICT education will be invited to send teaching staff to attend a one-day mini-conference in their capital city. Participation will be strictly voluntary. Each mini-conference will be attended by ICT educators from a range of universities operating in that region. In larger cities, two one-day mini-conferences will be held. Abridged versions of the mini-conferences with ICT educators based in the Northern Territory and in North Queensland will be conducted via video conferencing technology. The number of participants at each mini-conference will range from 10-20. The participants should be those teachers within each department that are involved in current teaching and learning initiatives.

Mini-conference structure:

The conferences will have three main parts. First, participants will be encouraged to brainstorm and discuss the main factors driving educational initiatives and best practice in ICT education. Second, participants will be encouraged to share details of their own teaching and learning initiatives. Finally, participants will be encouraged to identify and discuss issues involved in the dissemination of educational initiatives and the promotion of best practice in the ICT education. These sessions will be conducted over a day (from 10am to 4pm), with breaks for morning tea, lunch and afternoon tea, provided by the researchers.

Data collection:

Sessions will be video- and audio-taped. Audio tapes will be transcribed to computer files. At times during the mini-conferences, each participant will be encouraged to note ideas in a workbook containing sections for each part of the day. Workbooks will be collected by the researchers at the end of the data as a data source. Participants will be provided with photocopies of their own workbooks as a record of their participation and future resource.

Reporting:

The mini-conferences provide participants with opportunities to show case their own practice and to receive feedback from their peers. Reporting of particular educational innovations will be in the context of identifying and promoting best practice. The researchers anticipate that participants will want their contribution attributed to themselves, rather than being presented anonymously. All reporting of particular innovations will be reviewed and approved by the respective participants before publication. This member-checking will increase the credibility (internal validity) of the reporting and will enable participants to amend or restrict reports of their own practice. If any cases arise where participants wish to be anonymous, the reporting will accommodate this by the use of pseudonyms.

A final report, covering a larger study of which the mini-conferences are only a part, will be submitted to DETYA on December 1, 2001. Smaller, sub-reports and other documentation will be published on the projects Web page. Data may also be reported in the form of conference papers and journal articles.

154

Benefits to the participants:

The mini-conferences are part of a larger, nation-wide study. This phase of the study is collaborative in nature, aiming to elicit information from practitioners about their own teaching and learning initiatives, as well as their own and their peers' assessments of these initiatives and of the wider context in which they are developing. Participants are to be consulted as experts and as stakeholders in ICT education. The collaborative nature of the mini-conferences is intended to set the tone of the larger study, the ultimate aim of which is to identify and promote best practice.

Benefits to individual participants include

The opportunity to promote their own teaching and learning initiatives

The opportunity to learn about initiatives in other institutions

The opportunity to play an active role in a national study which should serve as a reference for the future directions of the discipline

Inquiries:

Please contact Julianne Lynch for more information or with any questions: Julianne Lynch School of Computer Science and Software Engineering Faculty of Information Technology Monash University Caulfield Campus Tel. 9903 1953 Fax. 9903 1077 Email. Juli.Lynch@csse.monash.edu.au

Should you have any complaint concerning the manner in which this research is conducted, please do not hesitate to contact The Standing Committee on Ethics in Research on humans at the following address:

The Standing Committee on Ethics in Research on Humans Monash University Wellington Road Clayton, VIC, 3168 Telephone (03) 9905 2052 Fax (03) 9905 1420

155

M O N A S H U N I V E R S I T Y

AUSTRALIA

Informed Consent Form

Project Title: Educational innovations in ICT education -- Mini-conferences.

I have read the description of the project and I understand what it involves. I understand that agreeing to participate means that I will

• Participate in discussions with a small group (approximately 10) of my peers about educational issues in the ICT discipline. These discussions will be video- and audio-recorded and recordings will be transcribed. Discussions will be loosely guided by a workbook, focusing participants on particular issues in ICT education. They will take place across one day (from 10am to 4pm).

• Provide details about my own teaching in the ICT discipline.

• Keep notes in a pen and paper workbook, to be provided to researchers at the end of the day. The workbook contains six open-ended questions, intended to prompt discussion, and space for notes.

• Provide my name, institution and contact information (optional) to the researchers. These details will be used to identify data and may be used if the researchers need to contact me to clarify information provided.

I understand that participation is voluntary. I also understand that, at any time during the project, I am able to withdraw my consent to participate, at which time all information provided by me will be destroyed.

I consent to participate in this project

Name (please print): ___________________________

Signature: ________________________________ Date: ___________________

I consent to be contacted in the weeks following the mini-conference should the investigators wish to clarify information collected and provide my preferred contact details below.

Institution: ________________________ Tel: ___________________________

Email: _________________________ Fax: ___________________________

156

Housekeeping

To minimise interruptions to today’s program we ask that you switch off all mobile phones and refrain from using PDA’s and laptops.

Introduction

Warm-up activity feedback

157

Brainstorm

1. What are the factors driving educational initiatives in ICT education?

2. What are the factors inhibiting educational initiatives in ICT education?

158

Notes

159

Teaching and learning initiatives

3. Please describe a teaching and learning initiative in which you or your department are currently involved.

160

3.1 Background of initiative

3.2 Goals of initiative

3.3 Life-cycle of your initiative

Planning Implementing Continuation

161

4. Deciding whether your teaching initiative is effective.

4.1 Strengths of your initiative

4.2 Limitations of your initiative

4.3 Please give details of any planned, current or completed evaluation of your teaching and learning initiative

162

Notes

163

Dissemination

5. What issues would be involved in disseminating your initiative to other ICT educators?

164

Continuation

6. What factors will influence the future of your initiative?

165

Other initiatives

Innovative colleagues

Relevant Websites

Thank you for participating in the AUTC ICT Education project. Your contributions will form the bases of further stages in the project and will, therefore, contribute to the promotion of collaboration and dissemination of innovative practice in ICT education. Your participation is much appreciated.

167

APPENDIX F. Statistical tables for industry survey data

Data from those respondents whose organisations employ recent ICT graduates Tables F1 – F29 refer to data obtained from respondents whose organisations do employ recent ICT graduates.

Demographics

Table F1. Respondents’ business size Business size Frequency Percent Cumulative

Percent less than 6 15 7.5 7.5 6 to 20 42 20.9 28.4 21 to 99 41 20.4 48.8 100 or more 103 51.2 100.0 Total 201 100.0 Missing 7

Table F2. Respondents’ industry type Industry type Frequency Percent Cumulative

Percent Communication services 15 7.5 7.5 Cultural & recreational services 5 2.5 10.0 Education 38 18.9 28.9 Electricity, gas & water supply 3 1.5 30.3 Finance & insurance 13 6.5 36.8 Government administration & defence

27 13.4 50.2

Health & community services 6 3.0 53.2 Manufacturing 3 1.5 54.7 Personal & other services 4 2.0 56.7 Property & business services 11 5.5 62.2 Retail trade 3 1.5 63.7 Transport & storage 1 .5 64.2 Other 72 35.8 100.0 Total 201 100.0 Missing 7

168

Table F3. Respondents’ state State

Frequency Percent Cumulative Percent

ACT 16 7.8 7.8 NSW 50 24.4 32.2 NT 4 2.0 34.1 Qld 30 14.6 48.8 SA 20 9.8 58.5 Tas 7 3.4 62.0 Vic 60 29.3 91.2 WA 16 7.8 99.0 Not in Aust 2 1.0 100.0 Total 205 100.0 Missing 3

Table F4. Respondents’ region Region Frequency Percent Cumulative

Percent Capital city 146 83.0 83.0 Regional centre 21 12.0 94.9 Rural 8 5.0 100.0 Total 175 100.0 Missing 33

Table F5. Respondents’ level of education Education Frequency Percent Cumulative

Percent Certificate 9 4.5 4.5 Diploma 9 4.5 9.0 Degree 90 45.2 54.3 Higher degree 88 44.2 98.5 Other 3 1.5 100.0 Total 199 100.0 Missing 9

Table F6. Organisation’s contact with universities through industrial experience

Contact Frequency Percent Cumulative

Percent Yes 102 51.0 51.0 No 81 40.5 91.5 DK 17 8.5 100.0 Total 200 100.0 Missing 13

169

Table F7. Organisation’s contact with universities through Committees Contact Frequency Valid

Percent Cumulative

Percent Yes 46 25.4 25.4 No 107 59.1 84.5 DK 28 15.5 100.0 Total 181 100.0 Missing 32

Table F8. Organisation’s contact with universities through Research & Development

Contact Frequency Valid

Percent Cumulative

Percent Yes 54 29.3 29.3 No 105 57.1 86.4 DK 25 13.6 100.0 Total 184 100.0 Missing 29

Table F9. Organisation’s contact with universities through Consulting Contact Frequency Valid

Percent Cumulative

Percent Yes 50 27.8 27.8 No 107 59.4 87.2 DK 23 12.8 100.0 Total 180 100.0 Missing 33

170

Skills, expectations and outcomes

Table F10. Importance of skills in ICT graduates Skill/knowledge

N Min Max Mean SD Skew Kurt Skills in particular computer languages

200 1 5 3.62 1.15 -0.53 -0.52

Skills in particular software applications

199 1 5 3.51 1.02 -0.23 -0.43

Foundation in theoretical principles

176 1 5 3.90 1.02 -0.74 -0.01

Ability to communicate with clients

205 2 5 4.60 0.65 -1.58 2.04

Written communication skills

205 2 5 4.37 0.72 -1.01 0.78

Ability to work in teams

205 2 5 4.55 0.62 -1.19 0.94

Project management skills

202 1 5 3.60 0.97 -0.20 -0.63

Problem solving skills

204 3 5 4.66 0.51 -1.12 0.15

Understanding of business processes

205 1 5 3.72 0.87 -0.28 -0.33

Preparedness for quick entry into job tasks

204 1 5 3.85 0.87 -0.57 0.23

Skills in languages other than English

196 1 5 1.89 0.87 0.63 -0.25

Other Skills 13 1 5 4.15 1.28 -1.74 2.42

171

Table F11. Extent to which recent ICT graduates meet organisation’s needs Skill/knowledge N Min Max Mean SD Skew Kurt Skills in particular computer languages

171 2 5 3.71 .84 -.289 -.445

Skills in particular software applications

179 1 5 3.55 .91 -.277 -.540

Foundation in theoretical principles

160 1 5 3.73 .85 -.323 -.109

Ability to communicate with clients

192 1 5 3.42 .95 .024 -.607

Written communication skills

194 1 5 3.28 .95 .057 -.572

Ability to work in teams

194 1 5 3.75 .88 -.552 .155

Project management skills

182 1 5 2.86 .97 -.113 -.505

Problem solving skills

195 1 5 3.70 .85 -.482 -.051

Understanding of business processes

182 1 5 2.90 .94 -.002 -.277

Preparedness for quick entry into job tasks

190 1 5 3.41 1.03 -.267 -.355

Skills in languages other than English

96 1 5 3.22 1.35 -.278 -.964

Other Skills 7 2 4 3.14 .69 -.174 .336

Table F12. Satisfaction with graduates Source of satisfaction N Min Max Mean SD Skew Kurt Overall performance of these graduates

196 2 5 3.87 .80 -.484 -.049

Performance compared to graduates in other disciplines

100 1 5 3.68 .96 -.354 -.478

172

Table F13. Outcome variables Variable N Min Max Mean SD Skew Kurt Would you recommend that people seeking a career in the ICT industry do a university ICT degree?

202 1 5 4.33 .97 -1.620 2.419

Would you recommend the ICT graduates your organisation has employed to other employers?

191 1 5 4.12 .89 -1.260 2.214

How responsive are university ICT departments to the needs of your industry?

146 1 5 3.05 1.09 -.501 -.367

What is the likelihood of your organisation continuing to employ university ICT graduates?

193 1 5 4.41 .92 -1.605 1.954

Table F14. Other variables Variable N Min Max Mean SD Skew Kurt As far as you know, does your organisation have difficulty finding ICT graduates to meet its needs?

201 1 5 1.72 .69 1.161 3.511

Broad education in ICT further to technical skills

188 1 5 3.98 .82 -.920 1.093

Theoretical background to support research and development

177 1 5 2.98 1.12 .045 -.632

Potential to contribute to future directions of the organisation

193 1 5 4.06 .82 -.799 .720

Time needed from entry to productivity

194 1 5 2.65 1.01 .245 -.440

173

Statistical analyses

Table F15. Important Skills - Difference from assumed mean 3.0 Skill/knowledge t df Sig.

(2-tailed) Mean

Difference Skills in particular computer languages

7.654 199 .000 .62

Skills in particular software applications

6.990 198 .000 .51

Foundation in theoretical principles

11.709 175 .000 .90

Ability to communicate with clients

34.893 204 .000 1.60

Written communication skills

27.249 204 .000 1.37

Ability to work in teams

35.756 204 .000 1.55

Project management skills

8.868 201 .000 .60

Problem solving skills

46.164 203 .000 1.66

Understanding of business processes

11.832 204 .000 .72

Preparedness for quick entry into job tasks

14.006 203 .000 .85

Skills in languages other than English

-17.747 195 .000 -1.11

Other Skills 3.248 12 .007 1.15

174

Table F16. Meet company’s needs - Difference from assumed mean 3.0 Skill/knowledge t df Sig.

(2-tailed) Mean

Difference Skills in particular computer languages

10.951 170 .000 .71

Skills in particular software applications

8.024 178 .000 .55

Foundation in theoretical principles

10.835 159 .000 .73

Ability to communicate with clients

6.145 191 .000 .42

Written communication skills

4.142 193 .000 .28

Ability to work in teams

11.863 193 .000 .75

Project management skills

-1.987 181 .048 -.14

Problem solving skills

11.418 194 .000 .70

Understanding of business processes

-1.418 181 .158 .00

Preparedness for quick entry into job tasks

5.406 189 .000 .41

Skills in languages other than English

1.591 95 .115 .22

Other Skills .548 6 .604 .14

Table F17. Satisfaction - Difference from assumed mean 3.0 Source of satisfaction t df Sig.

(2-tailed) Mean

Difference Overall performance of these graduates

15.206 195 .000 .87

Performance compared to graduates in other disciplines

7.063 99 .000 .68

175

Table F18. Outcome measures - Difference from assumed mean 3.0 Outcome variable t df Sig.

(2-tailed) Mean

Difference Would you recommend that people seeking a career in the ICT industry do a university ICT degree

19.427 201 .000 1.33

Would you recommend the ICT graduates your organisation has employed to other employers?

17.262 190 .000 1.12

How responsive are university ICT departments to the needs of your industry?

.609 145 .544 .00

What is the likelihood of your organisation continuing to employ university ICT graduates?

21.338 192 .000 1.41

Table F19. Other variables - Difference from assumed mean 3.0 Variables t df Sig.

(2-tailed) Mean

Difference Broad education in ICT further to technical skills

16.506 187 .000 .98

Theoretical background to support research and development

-.268 176 .789 .00

Potential to contribute to future directions of the organisation

17.958 192 .000 1.06

Time needed from entry to productivity

-4.775 193 .000 -.35

176

Regression

Table F20. SATISFACTION and NEEDS Coefficients Skill/knowledge B Std.

Error Beta t sig

(Constant)

.640 .500 1.279 .206

Skills in particular computer languages

.109 .110 .123 0.992 .325

Skills in particular software applications

.184 .087 .229 2.116 .039

Foundation in theoretical principles

.00 .096 .012 0.108 .914

Ability to communicate with clients

.238 .113 .315 2.112 .039

Written communication skills

.00 .111 -.017 -0.117 .907

Ability to work in teams

.00 .112 .016 0.125 .901

Project management skills

.00 .084 -.061 -0.530 .598

Problem solving skills

.258 .104 .288 2.477 .016

Understanding of business processes

.00 .107 .024 0.172 .864

Preparedness for quick entry into job tasks

.177 .082 .235 2.174 .034

Skills in languages other than English

.00 .052 -.059 -.609 .545

Dependent Variable: Overall Satisfaction with performance

177

Importance in graduates versus Meeting organisation’s needs

Note Gap data recoded to =0 no gap or negative gap

>0 is some gap

Table F21. Skills in particular computer languages Gap

Frequency Percent Cumulative Percent

.00 104 61.2 61.2 1.00 46 27.1 88.2 2.00 16 9.4 97.6 3.00 4 2.4 100.0 Total 170 100.0 Missing 38 208

Table F22. Skills in particular software applications Gap Frequency Percent Cumulative

Percent .00 113 64.6 64.6 1.00 43 24.6 89.1 2.00 14 8.0 97.1 3.00 5 2.9 100.0 Total 175 100.0 Missing 33 208

178

Table F23. Foundation in theoretical principles Gap Frequency Percent Cumulative

Percent .00 88 57.1 57.1 1.00 44 28.6 85.7 2.00 20 13.0 98.7 3.00 2 1.3 100.0 Total 154 100.0 Missing 54 208

Table F24. Ability to communicate with clients Gap

Frequency Percent Cumulative Percent

.00 53 27.6 27.6 1.00 66 34.4 62.0 2.00 49 25.5 87.5 3.00 22 11.5 99.0 4.00 2 1.0 100.0 Total 192 100.0 Missing 16 208

Table F25. Written communication skills Gap

Frequency Percent Cumulative Percent

.00 59 30.4 30.4 1.00 68 35.1 65.5 2.00 39 20.1 85.6 3.00 25 12.9 98.5 4.00 3 1.5 100.0 Total 194 100.0 Missing 14 208

179

Table F26. Ability to work in teams Gap Frequency Percent Cumulative

Percent .00 73 37.6 37.6 1.00 80 41.2 78.9 2.00 29 14.9 93.8 3.00 10 5.2 99.0 4.00 2 1.0 100.0 Total 194 100.0 Missing 14 208

Table F27. Project management skills Gap Frequency Percent Cumulative

Percent .00 74 41.1 41.1 1.00 55 30.6 71.7 2.00 35 19.4 91.1 3.00 14 7.8 98.9 4.00 2 1.1 100.0 Total 180 100.0 Missing 28 208

Table F28. Problem solving skills Gap Frequency Percent Cumulative

Percent .00 62 32.0 32.0 1.00 82 42.3 74.2 2.00 37 19.1 93.3 3.00 12 6.2 99.5 4.00 1 .5 100.0 Total 194 100.0 Missing 14 208

180

Table F29. Understanding of business processes Gap Frequency Percent Cumulative

Percent .00 75 41.2 41.2 1.00 49 26.9 68.1 2.00 41 22.5 90.7 3.00 12 6.6 97.3 4.00 5 2.7 100.0 Total 182 100.0 Missing 26 208

Table F30. Preparedness for quick entry into job tasks Gap

Frequency Percent Cumulative Percent

.00 97 51.3 51.3 1.00 58 30.7 82.0 2.00 22 11.6 93.7 3.00 10 5.3 98.9 4.00 2 1.1 100.0 Total 189 100.0 Missing 19 208

Those respondents whose organisations do not employ recent ICT graduates

Demographics

Table F31. Respondent’s business size

Size Frequency Valid

Percent Cumulative

Percent less than 6 163 54.9 54.9 6 to 20 69 23.2 78.1 21 to 99 25 8.4 86.5 100 or more 40 13.5 100.0 Total 297 100.0

181

Table F32. Respondent’s industry type Industry Frequency Valid

Percent Cumulative

Percent Accommodation, cafes & restaurants 1 .3 .3 Agriculture, forestry & fishing 2 .7 1.0 Communication services 36 12.3 13.3 Construction 4 1.4 14.7 Cultural & recreational services 11 3.8 18.4 Education 25 8.5 27.0 Electricity, gas & water supply 1 .3 27.3 Finance & insurance 8 2.7 30.0 Government administration & defence 10 3.4 33.4 Health & community services 8 2.7 36.2 Manufacturing 6 2.0 38.2 Mining 5 1.7 39.9 Personal & other services 16 5.5 45.4 Property & business services 30 10.2 55.6 Retail trade 13 4.4 60.1 Transport & storage 4 1.4 61.4 Wholesale trade 1 .3 61.8 Other 112 38.2 100.0 Total 293 100.0 Missing 4

Table F33. Respondent’s state State

Frequency Valid Percent

Cumulative Percent

ACT 14 4.7 4.7 NSW 71 24.1 28.8 NT 7 2.4 31.2 Qld 56 19.0 50.2 SA 22 7.5 57.6 Tas 4 1.4 59.0 Vic 89 30.2 89.2 WA 31 10.5 99.7 Not in Aust 1 .3 100.0 Total 295 100.0 Missing 2

Table F34. Respondent’s region Region Frequency Valid

Percent Cumulative

Percent Capital city 191 71.3 71.3 Regional centre 55 20.5 91.8 Rural 22 8.2 100.0 Total 268 100.0 Missing 29

182

Table F35. Respondent’s level of education

Education Frequency Valid

Percent Cumulative

Percent Certificate 33 11.2 11.2 Diploma 40 13.6 24.8 Degree 110 37.4 62.2 Higher degree 76 25.9 88.1 Other 10 3.4 91.5 None 25 8.5 100.0 Total 294 100.0 Missing 3

Table F36. Respondent’s contact with universities through Industrial Experience

Contact Frequency Valid

Percent Cumulative

Percent Yes 22 7.7 7.7 No 254 88.8 96.5 DK 10 3.5 100.0 Total 286 100.0 Missing 19

Table F37. Respondent’s contact with universities through Committees Contact

Frequency Valid Percent

Cumulative Percent

Yes 19 6.6 6.6 No 256 89.2 95.8 DK 12 4.2 100.0 Total 287 100.0 Missing 18

Table F38. Respondent’s contact with universities through Research & Development

Contact Frequency Valid

Percent Cumulative

Percent Yes 17 6.0 6.0 No 253 90.0 96.1 DK 11 3.9 100.0 Total 281 100.0

Missing 24

183

Table F39. Respondent’s contact with universities through Consulting Contact Frequency Valid

Percent Cumulative

Percent Yes 29 10.4 10.4 No 240 85.7 96.1 DK 11 3.9 100.0 Total 280 100.0 Missing 25

Table F40. Anticipated future employment of ICT graduates Anticipated future recruitment

Frequency Percent Cumulative Percent

Definitely will not 44 14.4 15.6 2 53 17.4 34.4 3 83 27.2 63.8 4 48 15.7 80.9 Definitely will 22 7.2 88.7 DK 32 10.5 100.0 Total 282 92.5 Missing 23 7.5

185

APPENDIX G. Methodology – Graduate interviews

In this section the aims, data collection methods, participants, analyses and reporting techniques are described for the part of the study that used semi-structured interviews to identify themes and issues in recent ICT graduates’ stories about their careers and the role played by their courses. The main limitations of the methods used are then outlined. The methods described here correspond with the second part of the phase referred to in Table 1 (Section 1) as graduate satisfaction. This phase involved a review of literature on graduate satisfaction and an investigation of graduates’ views of the role their courses played in their career paths. The methods used to investigate graduates’ views were implemented as a direct response to a suggestion of members of the Project’s Steering Committee, who suggested we conduct interviews with a small sample of graduates instead of attempting a major survey with this hard-to-access group. This represents a significant departure from the design proposed in the original Project specifications document.

Participants Graduates were identified using the Project team’s own networks. Efforts were made to maximise variation among graduates: the criteria for variation were graduate destination and graduate type. Graduate destination varied according to the size of the business for which the graduate worked, ranging from micro businesses (with six or fewer employees, including sole traders) to large businesses (with one-hundred or more people employees). Graduate status varied according to discipline of study (computer science and software engineering, information systems, or electrical engineering). Comparative data were also collected from a small number of non-graduates and a small number of graduates from non-ICT disciplines. These individuals were employed in the graduate destinations identified above. The research team’s own networks were used to identify graduates who met the criteria.

Graduate type Participants were eleven men and two women aged between 21 and 28 who have recently joined the ICT industry. Of the thirteen, eight had studied Computer Science and/or Software Engineering (CSSE), two had studied Information Systems (IS), one had studied Electrical Engineering, one had studied Biological Science and one had no tertiary qualification. Four of the male participants had commenced, but not completed their degrees in Software Engineering, Information Systems and Electrical Engineering. Three of these participants do not intend completing their degrees in the near future. Of the CSSE graduates, four had commenced degrees in other disciplines, for example, engineering and digital systems, before transferring to computing degrees.

186

Graduate Destination Seven of the participants are employed by large organisations (100+ employees) as analysts, sales professionals and programmers. One works as in a small business as a software developer. The remaining three work for micro organisations (< 5 employers), small businesses (< 20 employees) or as sole-traders contracting to larger companies. Table maps those graduates interviewed according to graduate destination and graduate type.

Table G1. Summary of interviewees by graduate destination and graduate type combinations

Micro (<6) Small (6-20) Medium (21-99) Large (100+)

CS/SE 2 1 - 5

IS/BS - - - 2

Elec Eng 1 - - -

Non-ICT grad 1 - - -

Non grad 1 - - -

CS/SE = Computer Science and/or Software Engineering; IS/BS = Information Systems/Business Systems; Elect Eng = Electrical or Electronic Engineering; Non-ICT grad = a degree from a non-ICT discipline; Non grad = no formal tertiary qualification

While the participants represent a convenience sample, their training covers a broad range of levels and specialisations. For example, levels of training represented include: no formal training, industry certificates, diplomas, pass degrees and honours degrees. Areas of specialisation represented include: software testing and quality assurance, programming, payroll systems administration, web development, software development, pre-sales engineering, database administration and financial/business analyst. Although Monash networks were used to identify graduates, interviewees had studied ICT-based degrees in five Melbourne-based universities representing a range of university types. Using Marginson and Considine’s (2000) nomenclature, the universities types represented include one Sandstone (established pre-war), one Redbrick (established post-war), one Gumtree (established 1960-75), one Unitech (former Institute of Technology) and on New university (former CAE).

Data collection The aim of the graduate interviews was to explore graduates’ beliefs about their current positions and their career paths and their attitudes towards their degrees. The approach taken was intended to identify those factors that ICT graduates view as having affected their ability to find employment and their preparedness for work.

Semi-structured conversational interviews (Patton 1990) were used to collect graduates’ accounts of their career histories. Open-ended prompts were used to elicit participants’ stories about background factors, such as family, schooling, university education and work history, and to encourage them to reflect on the role that these factors played in their careers. Prompts included instructions such as, “So you are a Software Developer in a small business, tell me the story of how you got to this point”. With participants’ permission interviews were audio-

187

recorded. Each interview ran for approximately one hour and was conducted in a location convenient to the interviewee. Follow-up telephone calls or emails were used to clarify details or to fill in any gaps where necessary.

Analysis Narrative-analytical accounts (Bassey 1999) of each participant’s career history were constructed from the audio-recordings. Accounts included sections on background (including current living situation, parents’ occupations, significant childhood memories, recreational interests, etc.), schooling, university education, employment history and aspirations. In some cases, interpretations and character generalisations were included. For example, an account might describe an interviewee as “enjoying challenges” or as “reserved”. Once compiled, accounts were returned to participants to be approved. Participants were free to make corrections and to add or delete sections.

Common themes across graduates’ stories were then identified using the constant comparative method (Silverman 2000; Strauss & Corbin 1997). Using NUDIST NVIVO (Qualitative Solutions & Research 1999), passages from each narrative account were coded into emerging themes and issues. Reports on codes were then examined and codes were merged and revised. The final coding system reflects those themes observed across graduates’ stories and those issues identified as salient to participants.

Limitations The methods used to identify common themes in graduates’ experiences and salient issues in their reflections on their careers and courses have three main limitations. These limitations concern the methods used to select graduates and the quality of engagement. Each is discussed below.

Selection methods The Project team’s own networks were used to identify graduates. This method was used for its convenience because it was anticipated that once graduates had completed their degrees, they would be difficult to locate. Even using our own networks, it was extremely difficult to find current contact details for potential participants. As a result of this approach to identifying potential participants, all interviewees were working in Melbourne and all had lived and studied in Melbourne. Seven of the participants were Monash graduates. This is a limitation because the majority of interviewees’ were reflecting on their experiences at the same institution, so themes and issues are more likely to reflect the nature of the degrees offered at Monash. For example, five of the interviewees had graduated from the same Monash course. This course involved an industry experience program that was run as a subject, in parallel with other subjects, so comments about industry experience reflect the structure of this particular program. Only one interviewee had completed industry experience that was significantly different in how it was structured. This example shows how there was little opportunity to contrast different programs or experiences.

188

Quality of engagement Each interviewee participated in only one interview that lasted approximately one hour. The length of interviews was restricted by interviewees’ availability. Interviews usually took place in interviewees’ lunch hours or after work. In qualitative research, this length of time in a single setting would not usually be seen as quality engagement, but given the restrictions on the availability of interviewees, or who were currently working in the ICT industry, and our own time and resource restrictions, longer periods were not possible. The limitations associated with the period of engagement were reduced by the use of follow-up phone calls or emails, where issues arising from, or events described in, interviews could be probed further. A further point in defence of the approach taken is that the aim of the graduate interviews was not to build a rich picture of a particular case (ie. intrinsic case study (Bassey 1999)), but to identify broad themes and issues that could be examined in subsequent research. Descriptions are not thick enough to do any more than this.

A further limitation to the quality of engagement is that the compilation of particular interviewees’ stories relied solely on the self-report of the interviewee. No collaboration was sought from other sources. Member-checking was used to confirm the validity of inferences made by the researchers; however, this does not move beyond the confirmation of a self-reported account. Interviews were conducted using a conversational style and an open approach to questioning. They were conducted in informal settings chosen by the interviewees. It is hoped that these factors increased the candidness of interviews.

189

APPENDIX H. Broad context of university teaching in Australia

Over the last fifteen years, Australia’s universities have been subject to extensive and multi-facetted change. These changes parallel those experienced by the university sectors in most developed countries around the world. They include

• Increasing mass education and increasing government focus on vocational education

• The changing policy environment for the funding and governance of higher education

• The advance of information and communication technologies

The pressures leading to these changes in Australia are evidenced in and acknowledged by a series of Government reviews and reports (eg. Dawkins 1988; Dawkins 1989; Hoang et al. 1998; Vanstone 1996; West 1998). They have been widely described and discussed in academic literature, the popular press and government reports.

Increasing mass education and increasing government focus on vocational education Participation in higher education has been increasing in all countries since the post war era. Shattock (1995) observed that, in nearly all countries, universities are suffering “something of an identity crisis” under the pressures associated with the rapid increases in student numbers. Figures for Australia can be found in Australian Bureau of Statistics (1997; 1999; 2000), Everingham (1999) and Kemp (2001, May 17). Coaldrake (2000) wrote that,

Throughout the period of the 1960s and 1970s, the Australian university system was grown rapidly to meet the needs of the 'baby boomer' generation and higher education was generally seen by government to be an important component of nation building. ... Since that time there have been changes in the volume of students moving from secondary school to higher education, and a significant increase in the numbers of older students, many of whom are returning to higher education, looking for professionally-related programs to build on their earlier studies.

Australian government policy documents have pointed variously to access and equity and to the health of the economy as rationales for this growth. Dawkins’ papers (1988; 1989) described a need for further growth, referring to an unmet demand for higher education, forecast growth in population numbers in the 17-19 year age group, increasing demands from older age groups, increased rates of school retention to Year 12, increasing demand for adult training, increasing demands of industry for higher education graduates, and increasing numbers of

190

full-fee paying overseas students. This rationale has been criticised, with many commentators noting more pragmatic explanations:

In order to absorb greater and greater numbers of those school-leavers unable to find jobs in open employment, the university system was required to take more domestic entrants. Although this was couched in terms of a greater social commitment to the 'clever country' and improved 'human capital' for the workforce, time investment in apprenticeship training by government and industry fell at the save, suggesting a more pragmatic explanation for the enrolment explosion. (Marginson & Considine 2000, p.28)

The Green Paper proposed that, “Australia’s annual output of higher education graduates [be increased] from the [1988 levels] of about 88 000 to about 125 000 by the turn of the century” (Dawkins 1989, p.13). This target was reached in 1993 (Kemp 1998). By 1998, 671 853 students were attending Australia’s publicly-funded universities (Kemp 2001).

Bessant (1996) argued that the increasing numbers of enrolments, together with the need to compete with resources led to “an over-stressed system at all levels” (Online Document). Marginson and Considine explained that, “New entrants were simply allowed to flow into existing courses and institutions with little attention to career options or ways to mark out a new pathway for less academically formed students” (Marginson & Considine 2000, p.29).

Coupled with the increasing numbers of places in higher education is an increasing government focus on vocational education. Dawkins’Green and White Papers (1988; 1989) are generally regarded as a turning point that crystallised movements towards the privileging of the economic value of education over other types of value. The focus on economic value is a challenge to the traditional values of universities and raises question about what purpose(s) higher education should serve. Marginson and Considine (2000) remind us that, as early as the Martin Report (1964), skill and professional development were described as a national resource “to be developed and exploited like any other" (p.23). The Labor Government’s focus on “human capital” and the economic value of education came to the fore with Dawkins’ papers. Bessant (1996) noted that by the late 1980s, the Labor Government had placed the education system at the centre of economic reform.

More recently, the economic advantage to be gained from education has been fore grounded not only in terms of advantages to Australian society but also in terms of individual gain (Crittenden 1999). While the seeds of this change of focus can be seen in Dawkins’ papers, particularly in the introduction of the Higher Education Contribution Scheme (HECS), it is more clearly evident in the Howard Liberal Government’s Higher Education Budget Statement 1996 (Vanstone 1996) in statements such as, “current funding arrangements do not provide a fair balance between the public and private benefits derived from higher education” (Online document) and in legislation that allowed for Australian students to pay full-fees for university courses.

Symes (1999) described this focus on the economic value of university education, both for the wider economy and for individuals, and the subsequent changes in the focus of teaching and learning and in the marketing of education, as the vocationalisation of the university, in which “the dominant imperative is learning for employment” (p.241). Employers or industry groups are seen as stakeholders in

191

education, and industry needs are seen as the key driver of students’ needs as future job seekers.

Information and Communication-based university departments and faculties in part owe their growth to the massification and the vocationalisation of university education. With the diffusion of computer-based technologies across society, graduates skilled in ICT are required for growth in the economy. Skill shortages in this area continue to be reported and forecast (see discussion in Section 2). This opportunity is also one for employment growth. With the rapid expansion of the number of students taking ICT-based courses, ICT has developed from a specialty area located in Science and Engineering faculties to a discipline in its own right.

The changing policy environment for the funding and governance of higher education The policy environment that supports the funding and governance of universities in Australia has been subject to dramatic change over the last fourteen years. The Labour Government reforms of the late 1980’s led to the formation of thirty-six large universities from the numerous higher education institutions that had previously existed. This rationalisation of the higher education sector was accompanied by cuts to government funding and a need for universities to compete for this funding, the introduction of student co-payments (HECS) and the introduction of formulae and targets that allowed for more centralised control of universities. These changes have been described as the corporatisation of universities. Marginson and Considine (2000) wrote:

Universities are no longer governed by legislation: they are more commonly ruled by formulae, incentives, targets and plans. These mechanisms are more amenable to executive-led re-engineering than are the deliberations of a council or an academic board, and less accessible to counter-strategies of resistance. They also fit with management-controlled tools such as soft money budgets, commercial companies, temporary institutes for research or teaching, fund-raising and marketing campaigns, all drawn together in a complex web of accountability tied only to the senior executive office. (2000, p.10)

The competition that was encouraged between universities, for funding and for students, was meant to improve their responsiveness to the needs of the workplace and to enhance productivity and quality (Marginson & Considine 2000). Competition for funding has been facilitated by increasing demands for accountability and quality reporting. These changes have been seen by many as challenges to traditions of collegiality, institutional autonomy and academic freedom (Bessant 1996, Online Document). Marginson and Considine (2000) wrote, “The modernisation of management and the extension of economic responsiveness seem inextricably linked to the faltering of academic identity, to reliance on money as the meta-measure of value, and a corrosive cynicism about individual motives and social purposes” (p.14).

The Vanstone budget statement (Vanstone 1996) coupled with the West Report (West 1998) applied further pressure on the administrations of universities to focus on competitiveness with each other and internationally, for students and for funding. Peters and Roberts (1999) wrote that, “Contemporary universities

192

function as performance-oriented, heavily bureaucratic, entrepreneurial organisations committed to a narrow conception of excellence generated by the imperative of international competitiveness” (p.47). Descriptions such as this are indicative of the way many academics have interpreted the changes: with a degree of cynicism. This cynicism can be best understood by exploring how the changes challenge traditional conceptions of the university and the traditional roles and motivations of academics.

The advance and convergence of information and communication technologies Another factor in the broader context of university education is the rapid advance and convergence of information and communication technologies. The convergence of these technologies is facilitating greater accessibility and greater flexibility. Increasingly, Internet-based technologies are seen as the basis to solutions to many of the problems faced by university educators, including increasing student numbers and student demands for more flexible delivery. These technologies have also assisted universities in increasing their target markets to include students who do not attend physical campuses, including students in other countries. This has been a key factor in the approach that many universities have taken to the increasing need to be more market-oriented and more entrepreneurial. Evans (1997) commented that,

It is now difficult to encounter a university that does not ‘sell’ itself in its publicity materials partly on the basis of its use of new educational technologies … and which is not seeking to reform its teaching using these technologies. (p.15)

New electronic information and communication technologies play two main roles in the broader context of university education: (1) They provide universities with a focus for advances in teaching and learning that can be used as selling points in their marketing, and (2) they promise a means of responding to the problems emerging from this new environment, such as very large numbers of students.

Both these roles are linked to the internationalisation of education. The use of new technologies is a means of attracting students from new markets and they also facilitate the delivery of educational services to off-shore fee-paying students. However, increasing internationalisation is not only an increase in the amount of international teaching; numbers of international students attending universities in Australia have also increased. Figures on trends in the recruitment of overseas students and the delivery of off-shore programs can be found in the Australian Vice-Chancellors’ Committee (AVCC) reports on Internationalisation (eg. Australian Vice-Chancellors' Committee 2000; Australian Vice-Chancellors' Committee 2001c). The AVCC reported that overseas student enrolments in Australian universities numbered 83, 111 in 1999, with the main growth areas being in enrolment in coursework higher degrees and in bachelor degrees. While overseas students come from a wide range of countries, the highest numbers come from Hong Kong, Malaysia and Singapore: together these countries supplied fifty-three percent of Australia’s overseas students in 1999. Compared with other OECD countries, Australia is well above the average in terms of overseas students as a percentage of all students: in 1999, 12.6% of students enrolled in Australian universities were from other countries compared with an OECD average of 4.8.

193

Percentages for other English speaking countries were 3.8% for Canada, 3.7% for New Zealand, 10.8% for the United Kingdom and 3.2% for the United States. Similarly, offshore programs offered by Australian universities through agreements with overseas higher education institutions have increased. The AVCC reported that by May 2001, Australian universities had 1, 009 offshore programs (Australian Vice-Chancellors' Committee 2001c). International links for the purpose of staff exchange and research are also on the rise (Australian Vice-Chancellors' Committee 2001b).

Coupled with advances in and the diffusion of information and communication technologies are other globalising forces such as the development of transport technologies and the development of legislation to better facilitate the movement of people across borders for the purpose of education. As a result of university marketing and enabling legislation, increasing numbers of fee-paying international students are studying in Australia (Australian Vice-Chancellors' Committee 2001a).

Some have been critical of universities money-raising approach to internationalisation. Bessant (1996) wrote,

University marketing has been directed towards attracting fee-paying overseas students as well as the sale of consulting and technical services. Some universities have been very successful in these ventures, but have also come to rely heavily on the finance this brings. However, there is very little evidence that programs have been tailored to meet the needs of these overseas students, apart from special English language courses. They are seen as the cream on the ice-cream bringing in desperately needed finance. (Online Document)

This criticism is borne out in the difficulties so often reported by teachers of international students.

However, others have argued that the use of information and communication technologies to facilitate the remote delivery of educational services has led to a renewed focus on teaching and learning techniques and on the needs of students. Evans (1997) wrote,

… traditional forms of education are focused around teaching opportunities through, for example, lectures, tutorials and practicals. Open and distance education, on the other hand, is focused more on facilitating opportunities for learning. (p.49)

Crittenden (1999) has also noted the attention being given to how to teach and learn using new technologies. However, he is critical of this preoccupation because it detracts from more important questions about the role of universities and the purpose of higher education.

The role of new technologies in the broad context of university education is complex. New technologies are often seen as both the cause of and the solution to problems. They offer a means of providing educational services and products to larger bodies of students, in a more diverse range of circumstances, regardless of the time/place needs of the students. They are also offered as a solution to problems associated with large diverse bodies of students. There are many reasons why the use of new technologies in education, and associated increases in the internationalisation of university education, receive luke warm responses. One reason that links the three areas of change outlined here is criticism of the

194

perceived motivations behind the changes. When viewed as part of the larger context of university education, the educational value of using new technologies appears to be secondary to other motivations.

In ICT-specific courses, the effects of the development and diffusion of computer-based technologies have yet another dimension: As new technologies are integrated into workplaces, the skills and knowledge required by graduates change and the number of graduates requiring skills in ICT increases. Developments in technology usually translate into curriculum changes, particularly in terms of content. And skill shortages that are answered by a growth in the number of students mean not only a larger student body, but also a more diverse student body.

195

APPENDIX I. Movements in university teaching and learning

Three movements are described below:

• That towards flexible delivery, which in some manifestations includes a movement towards student-centredness

• That towards quality assurance of higher education and the particular role of student evaluation of teaching questionnaires

• That towards the promotion of a scholarship of teaching and practitioner-run, targeted evaluation of educational innovations

Flexible learning, flexible delivery and student-centredness The twentieth century has seen a proliferation of movements aimed at reforming educational practice. Popular terms used in this discourse have included progressivism (often based on the ideas of Dewey (1961)), discovery learning (promoted by Bruner (1967; 1977)), generative learning (Wittrock 1974; Wittrock 1978), constructivist teaching and learning (based on the cognitive theories of Piaget and Vygotsky and discussed below), situated learning (based on Dewey, Vygotsky and more recently Lave (eg., 1988)), in the last twenty years, open learning, and more recently, flexible learning. Constructivist epistemologies have been strongly promoted as the theoretical foundation of effective teaching and continue to inform the goals of, motivations behind and evaluation of innovations in teaching and learning, particularly in teacher education (Fox 2001) and educational development departments of universities. For example, a quick glance at recent volumes of journals such as Teaching in Higher Education reveals that a large percentage of the writers (mainly educational support and development staff at universities) have adopted constructivism as the foundation of their work.

The constructivist approaches cover a wide range of epistemological orientations, from information-processing (Mayer 1992) and social constructivism (McCaslin & Hickey 2001) to radical constructivism (von_Glaserfeld 1998). Thus as each proponent of constructivism appears to advance a different variant of the construct, it is perhaps misleading to class all the adherents together, instead a more careful analysis and description of the specific characteristics of the phenomena seems to be warranted to provide the basis of beneficial discourse in teaching and learning improvement.

Many reviewers of learning theory, such as Shuell (1986), argue that, “cognitive psychology currently represents the mainstream of psychological and educational thinking” (p.441). He explained that, during the late 1960s and early 1970s, “the zeitgeist of psychology began to change from a behaviouristic to a cognitive orientation [and that] since about 1975 … cognitive psychologists have shown a growing interest in learning” (pp.411-412). The shift away from behaviourist approaches and towards constructive approaches led to a predominance of student-centred conceptions of learning. In particular, learning was seen as a

196

process that is active and cumulative rather than passive, and researchers became concerned with meaning and understanding, increasingly seeing the context of learning, prior knowledge and metacognition as important.

The focus on the active involvement of the learner has led to the promotion of a particular range of teaching strategies and learning activities. These include having learners involved in problem posing and problem solving, hypothesising, predicting and investigating, solving authentic, multidisciplinary, real world problems, working in groups, discussing their reasoning, exploring contradictions and reflecting on processes.

As a movement, constructivism has been seriously criticised on a number of grounds. For example, Anderson, Reder and Simon (unpublished) argue that many constructivist interpretations present misunderstood accounts of the nature and benefits of Piagetian assimilation and accommodation processes in knowledge acquisition, and the nature of information processing theories. They also argue that constructivists tend to misjudge the value of extensive practice and downplay the value of good instruction. Overall Anderson et al critique in detail four of the central claims of constructivists, as either misdirected or empirically incorrect. An example of different critique is presented by Fox (2001), who argues that constructivist theories are hardly different to common sense empiricist views, or else they provide misleading and incomplete views of human learning, resulting in misleading implications for teaching. He critiques seven key elements of constructivism in turn.

However, more recent educational reviews suggest that each of the three major schools of educational thought, behaviourist/empiricist, cognitive/rationalist and situative/pragmatist-sociohistoric can make worthwhile contributions to educational theory and practice leading to broader perspectives on learning and teaching (Greeno, Collins & Resnick 1996). For example, using a situated learning perspective Brown, Collins and Duguid (1989) promote the value of communication, group work and authentic activities.

Despite the heavy emphasis on constructivism in certain university subpopulations, constructivism does not appear to have penetrated very deeply into the ICT education in Australian universities. A comparative analysis of the last two years’ (2000 and 1998, no conference in 1999) conference proceedings from the Australian Computer Science (ACS) Education Conferences indicate only 11% and 6% of the papers explicitly used a constructivist basis in their studies, whereas in the last two years of the ASCILITE Conference proceedings (2000 and 1999) the proportions of papers addressing explicit constructivist philosophies were 25 % and 38 % respectively, although not always uncritically. The ASCILITE conferences are predominantly attended by university staff interested in using ICT in teaching and learning, (e.g. from multimedia education and educational development departments, or staff working closely with such departments), in contrast to the ICT educators meeting at the ACS conferences.

In fact, these results are in line with the identified plurality of educational conceptions and beliefs held by university teaching staff in general (Bain et al. 1997; Samuelowicz & Bain 2001). They classified the teachers’ beliefs about learning and teaching along a continuum from teacher-centred orientations (e.g. imparting information) to learning-centred orientations (e.g. encouraging knowledge creation), and found seven main orientations among their (small) sample of university teaching staff.

197

Many of the teaching and learning directions noted above are compatible with governments’ economic arguments for developing more adaptable, lifelong learners (Martin 1992) who are familiar with working in teams to solve problems, and with university policies on cross disciplinary graduate attributes.

In the higher education sector, the term that currently predominates in discussions of good teaching practice is flexible learning. This term has largely replaced open learning and carries with it some of the confusion that surrounded the original term. Webberley (1990) described open learning as an approach that

• is based on the needs of individual learners, not on the interests of teachers or the institution

• gives students as much control as possible over what and when and where and how they learn

• changes the role of the teacher from a source of knowledge to manager of learning and a facilitator

• commonly uses delivery methods of distance education and the facilities of educational technology (p.4)

These features can equally be said to characterise what is intended by the term flexible learning.

The slippage between terms used to describe approaches to education has been well noted (Taylor, Lopez & Quadrelli 1996). The term flexible learning is often used interchangeably with a similar and related term: flexible delivery. Flexible delivery refers to the anywhere/any time element of flexible learning and is most often associated with delivery facilitated by electronic ICT, via Web-based environments, as well as other technologically-facilitated educational provision (Bates 1995). In fact, the term flexible delivery is arguably associated more strongly with the use of particular technologies, than with an aim to offer openness or flexibility. The risk in the slippage between these two terms is that time/space independent delivery becomes associated with flexible learning regardless of the educational approach taken and the attention given or not given to creating an open learning environment. Web-based technologies can easily be used to deliver content in pre-determined sequences, for the purpose of knowledge transmission, neglecting the other learning needs of the student. This has been referred to as a learning package approach to flexible delivery (Willmot & McLean 1994), an approach that has its origins in instructional design for distance education that is based on behaviourist conceptions of learning (Taylor, Lopez & Quadrelli 1996; Lefoe 1998).

However, learning package approaches may be modified by the incorporation of collaborative ICT-based learning tools, such as email, mailing lists, Web discussion forums. Tuovinen, (2000) uses a four-interactions framework for designing and analysing flexible delivery and distance education, which overcomes some of the limitations of learning package approaches. In this framework the four key interactions involved in the educational process are identified: interactions between teacher and content, between students and content, between teacher and students, and among students. By making use of these interaction spaces, various educational activities can be sited in complementary balance with each other depending on the educational needs.

The use of hypermedia has been seen as a way of overcoming the negative implications often associated with the learning package approach noted above

198

(Ashman & Simpson 1999; Balcytiene 1999; Nicaise & Crane 1999; Jacobs 1992). Using hypermedia students can control their learning process, move among learning content in various personally selected directions, at their own pace, and according to their own motivations. However, an evaluation of the educational use of hypermedia found very few learning benefits (Dillon & Gabbard 1998).

The confusion or “imprecision” of the use of the terms flexible learning and flexible delivery, noted above, parallels that observed in the use of open learning and distance education. Rumble (1989) wrote,

The terms ‘open learning’ and ‘distance learning’ have never been used precisely … Distance education is a method of education. It differs from contiguous education. The concept of open learning describes the nature of the education offered either contiguously or at a distance. (p.28)

Similar distinctions can be made between flexible learning and flexible delivery. The distinction is important because flexible delivery can serve educational aims as well as non-educational aims, such as the development of new educational markets and the servicing of large numbers of students. While flexible delivery may serve students’ needs for time/space independent access to educational materials and environments, the term does not necessarily imply the focus on students’ learning needs implied by flexible learning. Flexible learning involves a student-centred approach to designing learning environments such that opportunities to learn are maximised; flexible delivery is only one, and not a necessary, part of this.

The move towards flexible delivery, with a focus on Web-based technologies is now one of the most remarked on movements in university education, both in Australia and internationally. Caution needs to be taken so that student-centredness is not assumed in Web-provided materials.

Quality movement and student evaluation of teaching Brennan observed that, “the growth of external assessment of higher education quality has been one of the most marked international trends in higher education in recent years” (Brennan 1997, p.23). Quality became an issue in Australia in response to the Dawkins’ reforms of the late 1980s. Massaro (1997) wrote, “vice-chancellors and other commentators began to express concern that the reforms and the reductions in funding had led to a reduction in quality” (p.53). In 1991, Peter Baldwin, the Minister for Higher Education and Employment Services, issued a policy statement, Higher Education: Quality and Diversity in the 1990 (Baldwin 1991), announcing that it was necessary to assure the community that the quality of higher education was of a high standard and allocating $70m to those institutions which could demonstrate better than adequate quality. In 1992, an independent Committee for Quality Assurance in Higher Education was established to undertake quality assessments of higher education institutions.

To promote diversity in the system, quality was to be measured against each institution’s mission statement rather than against an abstract standard (Massaro 1997). This approach mirrored that taken in Britain where quality was defined as “fitness for purpose” (Henkel 1998). Despite the intention of promoting diversity, the quality assessment process that ensued in Australia produced a tendency towards uniformity. Marginson and Considine (2000) observed a “pattern of inter-

199

institutional mimicry” (p.19). Massaro (1997) wrote that, “the results of the quality rounds may actually have reined in the more creative attempts to be different by reminding institutions that quality was still seen to reside in the old and traditional types of university” (p.55-56). He concluded that “the tendency toward homogenisation was inherent in the removal of regulation and it was over-ambitious to expect quality assurance mechanisms to arrest it” (p.55-56).

Quality assessment of teaching The quality movement that began in the 1980s brought with it a focus on internal and external assessment. University teaching, which had long been free of scrutiny, increasingly became the focus of evaluation.

It is widely recognised that quality assurance has two main purposes: accountability and improvement. In Brennan’s (1997) international review, he concluded that, although virtually all quality assessment bodies claim to have improvement as well as accountability goals, in almost all cases, statements of purpose emphasise value for money and greater accountability for the use of public funds. As a result of this emphasis and a tendency to rely on limited methods for evaluating teaching, much cynicism surrounds the quality movement. A key development that is often criticised is the proliferation of quantitative student questionnaires aimed at gauging students’ opinions about and satisfaction with particular subjects and particular teachers. While those systems commonly used in Australia for student evaluation of subjects and of teaching have been found to have validity with respect to the aim of improving student learning (Moses & Trigwell 1993), they have been criticised when they are relied on as the sole means of evaluation (Cosser 1998). A side-effect of this reliance is that evaluation of teaching methods and materials more generally are also often limited to student feedback questionnaires. Bain (Bain 1999) noted that reports of innovative teaching practice

… often limit their evidence on the influence of the innovation to students' and peers' reactions during development, and their impressions of learning achievements after implementation. Sometimes data from standard course assessments also are reported, but often without regard to the relationship between the learning encouraged by the innovation and the learning assessed by standard methods. (p.166)

Similarly, in a review of 104 teaching and learning initiatives that made use of information and communication technologies, Alexander (Alexander 1999) concluded that,

There continues to be a heavy reliance on student reaction surveys, and in some cases there is an apparent confusion between student reactions and student learning. Positive student attitudes and increased motivation may encourage better learning outcomes, but they are not in themselves evidence of improved learning. While student reaction surveys are a useful component of any evaluation, they should not be the only component. (p.181)

Stringer (1993) identified those features of teaching for which student feedback is valuable. They include course organization and structure, workload and course difficulty, marking and exams, course impact on students, breadth of coverage and

200

global or overall ratings. A distinction is made between feedback on these features of subjects and courses and the evaluation of learning outcomes. Student feedback questionnaires and their role are discussed in more detail in Section 7.

Quality assessment of teaching for improvement Some commentators do believe that quality assurance can serve the purpose of improving university teaching if it takes particular forms. Elton (1993) argued that the promotion of quality in university teaching requires the following changes:

• a rapid increase towards the professionalization of university teaching

• the establishment of TQM practices at all levels of each university

• increasing recognition and resourcing of teaching and rewards for excellence in it (p.145)

These changes are consistent with the views of others who support quality improvement of university teaching. Brennan wrote that, “quality judgements which lack legitimacy in the eyes of those on the receiving end of them are not likely to be acted upon if action can be avoided. In this respect, the nature of the involvement of the academic community as a whole is important” (Brennan 1997, p.13).

Although standard form student feedback questionnaires continue to dominate in the evaluation of university teaching in Australia, educators increasingly recognise the limitations of relying on this method and there is a move towards more formative evaluations targeted to the aims of particular teaching methods and materials. However, as Cosser (1998, 159) concluded, “unless teaching is seen to be valued in and by the institution, all attempts to introduce new systems for evaluation its effectiveness will fail.” A major flaw observed in current systems for evaluating teaching is that quality teaching is generally not highly valued or rewarded within universities. The introduction of new systems of evaluation alone will not solve this problem.

Evaluation and scholarship of teaching movement While evaluation of teaching has emerged as a key part of quality assurance in Australian higher education, the approaches taken have more often served summative purposes than they have provided meaningful information for the improvement of university teaching. This section focuses on the movement towards formative, practitioner-run evaluations for the improvement of university teaching and tailored evaluation for the identification, promotion and reward of scholarship in university teaching. Three relevant trends are outlined here:

• an increasing recognition of the value of formative, practitioner-run evaluation

• a growing interest in peer review of teaching

• a resurgence of interest in the role that scholarship might play in the promotion, recognition and reward of good teaching

Then observations about the implementation of changes to the evaluation and assessment of teaching are briefly described.

201

Formative, practitioner-run evaluation In a discussion of trends in the evaluation of university teaching, Henkel (1998, p.289-290) noted a retreat from the search for generalisable knowledge and causative modeling towards more context-specific conclusions, a greater emphasis on the need for evaluation to be more practical, a greater emphasis on organisations taking charge of evaluations themselves and incorporating them into change strategies, and a shift from summative towards formative evaluation for the purpose of improvement. Similarly, Bain (1999,) noted a growing preference for using “authentic” evidence and multiple methods rather than experimental designs that rely on a limited range of data gathering methods, to evaluate teaching. Coupled with this move away from summative, towards formative evaluations, is a move away from external evaluation towards internal evaluation of teaching for improvement. Oliver (2000, Online Document) observed a “shift of power away from the evaluator as an arbiter acting for the commissioning authority and towards evaluation as a collaborative process of building mutual understanding.” He argued that,

This approach advocates a continuous process of evaluation by practitioners (as opposed to evaluators) as a strategy for dealing with organisational change and meeting the need for responsive, adaptive organisational structures. ... Moreover, this style of evaluation represents a shift in focus away from self-contained programmes or projects to the ongoing evaluation of processes and systems. It requires the creation of a culture of reflective practice similar to that implied by action research. (Oliver 2000, Online Document).

It is often observed that formative evaluation is an integral part of university teaching and that all teachers undertake informal formative evaluations in the course of the iterative improvement of the subjects they teach. As Stringer (1993, p.94) observed, “anybody involved in curriculum development is automatically engaged in formative evaluation; thus testing work while it is still being developed and obtaining feedback will produce revisions for change.” Similarly, Brennan (1997, p21) noted that, “much educational change is invisible. It is incremental. It is slow.”

Within Australian higher educational circles, there is an increasing recognition of importance of context-specific evaluations of teaching that are more action-oriented than summative. The conduct of this type of evaluation by university teaching staff is promoted as both a means of improving teaching and a means of elevating the status of teaching.

Peer review of teaching Henkel (1998, p.285) pointed out that, "Evaluation, in the form of peer review, has traditionally been at the heart of higher education and its regulation." Similarly, Brennan (Brennan 1997, p.16) wrote, “Peer review has long been a central part of the decision-making process in higher education."

Evaluation by one's academic peers is ... part of the academic enterprise. It is built into the character of knowledge, not as a supporting plank but as its very foundation. Academic knowledge simply does not count as knowledge without it having been subjected to some kind of peer evaluation. ... It is entirely proper,

202

therefore, that peer assessment has been extended from its central place in truth discourse as such to its occupying a strategic function in activities critically associated with academic life. (Barett 1992, p.123)

Despite some criticisms, a resurgence of interest in the peer review of teaching has been observed in Europe, the United Kingdom and in the United States (Cosser 1998). Henkel (1998, p.293) explained that, "as European governments have sought to develop evaluative systems for higher education, they have placed peer review and self-evaluation at the centre of their policies." A key motivation for the growing interest in peer-review of teaching is the perception that university teaching is not highly valued. Peer review is seen as a means to elevate the status of university teaching to that of research. In the early 1990s, the American Association for Higher Education launched a major investigation into peer review. Hutchings (1994, p.4) commented,

What's needed ... are strategies for peer review that capture the scholarly substance of teaching, and which might therefore focus not only on what happens in the classroom (where the evaluation of teaching is now almost exclusively focused) but also on matters of course design, and assessment of results in terms of student learning.

Taylor (1999, np) has commented on the situation in Australia, “where academic career advancement is strongly linked to achievement in both teaching and research, much has been written about the need to recognise and reward teaching, but little progress made - research rules!" He advocates "the development of strategies to make teaching achievements as equally valued as research achievements, in terms of career progression."

In Australia, peer review is increasingly seen as an appropriate process for the identification of scholarship in university teaching. A recent government-funded report on Validating scholarship in university teaching (Taylor & Richardson 2001) places peer-review at the center of this process.

Scholarship of teaching The idea of the scholarship of teaching was introduced ten years ago by authors such as Boyer (1990) and Rice (1991). Since then a considerable literature has developed, discussing what the scholarship of teaching might look like, and how it might be encouraged and demonstrated. Kreber and Cranton (2000, p.478) argued that, “the scholarship of teaching includes both ongoing learning about teaching and the demonstration of teaching knowledge. As such, it is of critical importance in faculty development and evaluation." They noted that, to date, assessment of the scholarship of teaching has stressed outcome measures over the process of knowledge acquisition. They propose “an alternative understanding of the scholarship of teaching, one that considers it to be both learning about teaching and the demonstration of that knowledge" (Kreber & Cranton 2000, p.492).

The movement described earlier, towards formative, practitioner-run evaluation of teaching, has been picked up in particular by those interested in the scholarly use of new electronic technologies to improve learning. In Australia, the most recent work on the scholarship of teaching has been done in this area. Alexander wrote,

203

Evaluation (both formative and summative) must become as much a part of professional practice as project development. Academics have a professional responsibility to design, develop, implement and evaluate CIT [Communication and Information Technology] for learning in a scholarly way, as would be expected in their research and in all other aspects of their academic lives. (Alexander 1999, pp.181-2)

Taylor and Richardson, in their report on a national scheme for external peer review of ICT-based teaching, make the distinction between what he calls scholarship-about-teaching and scholarship-in-teaching, where scholarship-in-teaching focuses on the design process and accounting for outcomes, rather than on understanding the teaching and learning process:

Our sense is that most examples of scholarship of teaching are really discussions about teaching and learning processes, mostly in the form of conference and journal papers. We refer to this publication-focused aspect of the scholarship of teaching as 'scholarship-about-teaching'. Our interest in this work is with those aspects of the scholarship of teaching that directly inform the decisions teachers make. That is, we are interested in those aspects of the scholarship of teaching that both inform the design of pedagogical practices and are evident in teaching itself. We refer to these aspects as 'scholarship-in-teaching'. (Taylor & Richardson 2001, p.32)

In this way, the processes of design, implementation and evaluation are seen as sources of scholarship. This is consistent with Alexander’s (Alexander 1999) recommendation that evaluation be undertaken at all stages of development. The processes recommended by Taylor and Richardson involve the peer-review of resources and documentation produced by the university teacher. Documentation submitted to the proposed review system would address design, implementation and evaluation considerations. They described this review process in detail, specifying procedures for submission, review, feedback and dissemination.

The goal of the peer-review system proposed by Taylor and Richardson is not only to advance knowledge about teaching and learning, but also to recognize and reward quality teaching. However, as Cosser points out this may be a chicken and egg situation:

… the rewards for good teaching are grossly inadequate. Until there are real incentives for improving the quality of teaching, then - money; status; advancement opportunities - peer review is likely to fail. The irony however, is that only the establishment of viable methods of assessment of teaching ... will bring teaching the recognition it deserves. (Cosser 1998, p.159)

205

APPENDIX J. Teaching and learning initiatives – brief accounts

[1]

Innovators: CSU Online, Division of Information Technology, Charles Sturt University (the software was largely the work of Matthew Morton-Allen).

Presenter: William Adlong

Organisation: CELT, Charles Sturt University

Keywords: Computer-mediated Communication - Teaching Tools

Description: Adlong and colleagues at CELT CSU encourage teaching staff to focus on the constructivist communication aspects of online learning, rather than simply on content delivery. At CSU, 'Online Subject Forums' are available to all subjects. These provide for threaded asynchronous communication, file sharing, and passworded access and include moderator management facilities. They provide students with an alternative way to engage with material and they help to overcome the isolation experienced by distance education students.

To find out more: a recent paper about the online subject forums: http://ausweb.scu.edu.au/aw01/papers/refereed/burr/paper.html

You can contact Matthew Morton-Allen at: mmorton-allen@csu.edu.au

[3]

Innovators: Bill Farrow, Annette Oppenheim, Matthew Smith, Kevin Bluff, and Ahmet Sekerciouglu.

Presenter: Dr Rob Allen

Institution: School of Information Technology, Swinburne University of Technology

Key words: Assessment - Online Assignment Submission

Description: To reduce the work involved in assessing programming assignments, the Swinburne University School of IT have developed a system for the online submission and automated assessment of students' programs. Students upload their programs via a web-based system to a UNIX host; marks are automatically generated reflecting the success of the program execution. Immediate feedback is given via the web. This automatically generated feedback is complemented by online marking summaries prepared by tutors and then emailed to the students. Using this system, students may resubmit assignments (prior to the due date) as often as they wish. A prototype was in use in 2000 with a redesign/rewrite currently in progress.

To find out more: http://www.it.swin.edu.au/staff/aoppenheim/default.htm

[5]

206

Innovators: Geoffrey Roy, Jocelyn Armarego, Terry Woodings

Presenter: Jocelyn Armarego

Organisation: School of Engineering, Murdoch University

Keywords: Academic Skills - Flexible Delivery

Description: At Murdoch University, the Bachelor of Software Engineering is delivered, for the most part, online. However, lecturers in the School of Engineering have found that they cannot assume that students have the skills to manage their own online learning. In an effort to accustom students to online learning and to prepare them for life-long learning, B(SE) students are progressed through two learning environments. In the first 1-2 years of their studies, the learning environment is quite structured with material presented in a linear fashion with clear milestones and firm deadlines for the completion of tasks. An underlying goal of this environment is to provide students with the skills to manage their own learning. The second learning environment, encountered in the senior years, is more flexible; students are free to tackle course material in any order they wish as long as all topics are completed. Within these student-centred learning environments, the academic takes the role of 'guide' rather than 'teacher'.

Acknowledgements: Partial support from Murdoch University's Teaching and Learning Centre

To find out more: http://eng.murdoch.edu.au/WebTeachingDemo/MUEpage0.html

[6,21]

Innovators: Project leader: Angela Carbone 1999, Kathy Lynch 2000Other Key Staff: David Arnott, Martin Atchison, David Foot, Peter Jamieson, Don Schauder.

Presenters: Martin Atchison and Angela Carbone

Institution: School of Information Management and Systems, Monash University (Caulfield)

Key words: Experiential Learning - Studio-Based Teaching

Title: Studio-based teaching in Information Technology

Description: Based on the work of Donald Schon, studio environments are used as a core element of the School's Multimedia and Information Management and Systems degrees. The studios provide physical environments that facilitate teamwork and simulate the professional work place. The teaching approach is collaborative and team oriented, building on an integrated curriculum where work done in the studios builds on and reinforces concepts from other core subjects. The learning is based around teamwork and the production of student portfolios. The IT infrastructure includes multimedia simulations, threaded discussions, online quizzes, electronically available notes and portable notebooks.

Acknowledgements: 1998 Monash SIF Grant

To find out more: http://www.sims.monash.edu.au/studios/bims/index.html

[8]

207

Innovators: Kevin Austin

Presenter: Kevin Austin

Organisation: School of Management, University of Western Sydney

Keywords: Problem solving - Peer learning

Description: In response to an observed lack of student interest and engagement in lectures, Austin has introduced puzzles within the lecture environment of a subject on knowledge-based information systems. The goal is to increase student participation and to encourage lateral thinking and problem solving in students. At intervals during lectures, students are asked to solve puzzles that relate to the content area. Students are also encouraged to devise their own puzzles for up-coming topics. The students are required to present these puzzles, creating an interactive lecture-based learning experience.

To find out more: Further documentation on the subject refer ACE2000 - proceedings p.253 - poster presentation.

[9]

Innovators: Reuven Aviv, Orli Doron, Tami Shapira.

Presenter: Dr Reuven Aviv

Institution: School of Computer Science and Software Engineering, Monash University; Department of Computer Science, Open University, Israel

Keywords: Co-operative Learning - Computer-Mediated Communication

Description: Aviv and colleagues at the Open University in Israel conducted a content evaluation of an online learning program in an effort to identify methods for evaluating co-operative learning and to explore methods for structuring effective co-operation between online learners. A group of 10 students engaged in the online learning program over a period of 3 weeks with most communication being via the Internet. Content analysis of educational performance revealed that structuring effective co-operation resulted in a high level of reasoning in students.

Acknowledgements: This work was partially funded by the Multimedia and Networking Consortium, Israel.

To find out more: http://www.aln.org/alnweb/journal/Vol4_issue2/le/reuven/LE-reuven.htm

[10]

Innovator: Dr S Mahfuz Aziz

Presenter: Dr S Mahfuz Aziz

Organisation: School of Electrical and Information Engineering, University of South Australia

Keywords: Peer Learning - Project-based Learning

Description: In a project-based subject on computer design, students are required to work in small groups to design, implement and test a single board computer. Aziz has observed that students find this task difficult, encountering many problems in the implementation stage. As part of an action research project, Aziz

208

has introduced a class project seminar and class project meetings. The class project seminar allows student to present their designs to their peers and to receive feedback. Class project meetings provide students with a forum for sharing problems and discussing issues with their peers on a regular basis. Aziz's research project aims to investigate the impact of these initiatives on student learning outcomes and on the outcomes of the design projects.

Acknowledgements: This project is being carried out as part of the requirements for the unit The Reflective Practitioner in Higher Education in the Graduate Certificate in Higher Education programme of Queensland University of Technology (https://olt.qut.edu.au/edu/ed61/). The coordinator for the unit is Dr Sally Clarke of TALDU, QUT. Her advice and feedback have been very valuable for this action research project.

To find out more: http://people.unisa.edu.au/Mahfuz.Aziz

[14]

Innovators: John Bentley, Annemieke Craig.

Presenter: John Bentley

Institution: School of Information Systems, Victoria University

Key words: Experiential Learning - Problem-Based Learning

Description: Bentley and colleagues observed that final year students did not have good project management skills and the ability to apply/integrate prior knowledge in undertaking a real project. In response, a problem-based learning approach was developed in an introductory IS analysis and design subject. This subject is delivered primarily through students' group work as they develop solutions to given problems. Problems are designed to simulate those found in the professional world, each taking from one to five weeks to complete.

Acknowledgements: This initiative is supported by School of Information Systems Curriculum Committee, Victoria University. The problem-based learning approached used here is the subject of John Bentley's doctoral study.

To find out more: http://www.business.vu.edu.au/bentley

[16]

Innovators: Ilona Box, Marilyn Wells, Peter Jerram, Cate Jerram, Kevin Austin

Presenter: Ilona Box

Organisation: School of Management, University of Western Sydney

Keywords: Curriculum Integration - Graduate Attributes

Description: With the aid of a Teaching Development Grant, Box and colleagues are working toward integrating the teaching of first year Information Systems. The aims of the Integrating First Year Subjects project are to rationalise assessment, teach within a ‘graduate attributes' framework, better accommodate the changing nature of students and introduce first year students to university-style study as early as possible.

209

Acknowledgements: Former University of Western Sydney, Hawkesbury, Faculty of Management, Teaching Development Grant for2001

To find out more: Email Ilona Box

[17]

Innovators: Julie Brear, Jodi Tutty

Presenter: Julie Brear

Institution: School of Informatics, Northern Territory University

Keywords: Flexible Delivery - Self-Paced Learning

Description: In an effort to overcome the problems of variable student skill level and learning style, Brear and colleagues have introduced self-paced learning materials into their introductory computing subject. Students' take control of their own learning and, instead of attending regular weekly practical classes, may attend optional 'help sessions' where an instructor is present to answer questions relating to any computing topics. Replacing regular practical classes with help sessions also reduces the costs of holding classes that may be poorly attended and eliminates the problem of students who miss classes, missing important material.

To find out more: http://www.cs.ntu.edu.au/homepages/jbrear/

[18]

Innovators: The Information Environments Program, The University of Queensland, Ipswich Campus. Program Director, Michael Docherty

Presenter: Allison Brown

Organisation: Learning Resources Development Unit, University of Queensland

Keywords: Studio-based Teaching - Flexible Delivery

Description: The Ipswitch campus of UQ was established as a flexible learning-based campus where educational innovation is encouraged. The educational focus of the campus is one of learning rather than delivery. Brown and colleagues applied this ethos to the teaching of their new Bachelor of Information Environments degree. A studio-based approach has been adopted with the aim of producing IT graduates skilled in critical analysis, problem solving and communication design.

To find out more: http://www.tedi.uq.edu.au/lrdu/ and http://www.infenv.csee.uq.edu.au/

Other Initiatives:

• Development of flexible learning approaches in more traditional campuses

[20]

Innovators: Malcolm Campbell, Albert Goodman, Wanlei Zhou.

Presenter: Malcolm Campbell

Organisation: School of Computing and Mathematics, Deakin University

210

Keywords: Capstone Subjects - Experiential Learning

Description: To prepare final-year students for the reality of the IT workplace, Campbell and colleagues have developed a capstone subject in which students work in groups of 10 or more on a software development project. Project groups are self-selected to meet the job specifications for various roles, and make formalised bids for the projects offered by external and staff clients. Group members appoint a project manager who is responsible for assigning tasks and the overall running of the group. Professional communication skills are developed through weekly formal project meetings and end of project presentations and product displays.

Acknowledgements: Dr Kai Ming Ting (formally from Deakin, now at Monash University) contributed to this initiative.

To find out more: http://www.deakin.edu.au/~malcolmc/

Other Initiatives:

• E-delivery of IT degrees

• Recruiting students for IT success

[22]

Innovators: Penny Collings, Craig McDonald and Avon Richards-Smith

Presenter: Penny Collings

Organisation: School of Computing, University of Canberra

Keywords: Teaching Tools - Computer-mediated Communication

Description: To facilitate group work and refine project and quality management skills in final-year project students, Collings and colleagues have implemented a computer supported co-operative work (CSCW) approach to group-work projects. Students are required to complete a semester-long group project using Lotus Notes® for workgroups as the core communication infrastructure. CSCW has the benefit of facilitating software management and online information sharing among group members.

Acknowledgements: Funding for some of the development of CSCW was obtained from the University of Canberra and the former Committee for the Advancement of University Teaching.

To find out more: http://www.ise.canberra.edu.au/collings/

Other Initiatives:

• User interface design: poster session and online resource.

• Girls in computing in the ACT.

[23]

Innovators: Lorraine Connell

Presenter: Lorraine Connell

Organisation: School of Health, Education and Science, Northern Territory University

211

Keywords: Flexible Delivery - Technology Education

Description: Information technology is a key learning area for trainee teachers at Northern Territory University; education students are expected to utilise technology in both their own learning and their future teaching. To encourage the use of IT in non-IT students, Connell delivers much of the content for the subject, Technology in Education, online using Blackboard.com® .

To find out more: http://138.80.22.25

• search in catalogue for "Education" and then scroll down to Connell's units.

Other Initiatives: as above, only type in,

• etu114 for Arts in Education.

• etu458 for Arts in the Community

• recorder for Recorder in Education

• dance for Dance in Education

• edn531 for Teaching with Multiple Intelligences

[24]

Innovators: Annemieke Craig, Julie Fisher, Angela Scollary.

Presenter: Annemieke Craig

Institution: School of Information Systems, Victoria University

Keywords: Access and Equity - Females in Computing

Description: Craig and colleagues sought to increase the enrolments and retention of females in computing courses at Victoria University through Girls into Computing days, the development of promotional videos and a mentoring/support scheme. They also aimed to better prepare female computing students for professional life through an increased emphasis in their courses on interpersonal, communication and team-work skills.

Acknowledgements: This work is supported by a research grant from the Equity and Social Justice Branch, Victoria University.

To find out more: http://www.staff.vu.edu.au/Acraig/

Other Initiatives:

• Problem Based Learning project.

[25]

Innovators: Dr David Cropley

Presenter: Dr David Cropley

Organisation: School of Electrical and Information Engineering, University of South Australia

Keywords: Graduate Attributes - Academic Skills

Description: Creativity and innovative thinking have been identified by UNISA as desirable graduate attributes. Cropley and colleagues recognise that the

212

preferred learning style of many undergraduate engineering students does not lend itself to creativity and innovative thinking. Confronted with this pedagogical challenge, Cropley, with the assistance of an educational psychologist, has developed and implemented a second year subject that aims to develop creativity and innovative thinking in students. Before taking the subject, students complete measures of creativity and preferred learning style; creativity is again measured at the end of the subject. This has the dual purpose of identifying blocks to creativity and improvements in creative thinking. The subject has also been successfully presented to industry-based engineers in a workshop form.

To find out more: http://www.seec.unisa.edu.au/personnel/DC/DCHomepage.htm

[26]

Innovators: Jeni Paay, Jacky Hartnett, Mark Davis.

Presenter: Dr Paul Crowther

Organisation: School of Computing, University of Tasmania

Keywords: Flexible Delivery - Student Needs

Description: Crowther and colleagues have developed an introductory computing subject that can be adapted to suit the needs of students from various disciplines (eg., IT, Nursing, Human Movement). Adaptable to the educational needs and skill level of a broad range of students, the subject provides students with practical skills and a theoretical background relevant to their discipline, delivered at an appropriate pace and depth. Offered both electronically and face-to-face, the subject develops core competencies in all students and offers optional components to challenge the experienced and engage the novice.

To find out more: http://www.comp.utas.edu.au/staff/pcrowthe.html

[27]

Innovator: Gordana Culjak

Presenter: Ms Gordana Culjak

Organisation: Department of Information Systems, University of Technology, Sydney

Keywords: Student Journals - Assessment

Description: To promote class attendance and participation, Culjak and colleagues have introduced a 'research journal' component into their Internet Commerce subject. Students are required to collect and write comments upon media articles related to their course. At any time, students may be asked to present their journal to the tutorial group and are marked according to their preparedness and the depth of their understanding of the material.

To find out more: http://it.uts.edu.au/about/contact?dept=is

[identified in follow-up – international]

Innovators: Mats Daniels, Arnold Pears, Christina Björkman, Anders Berglund.

213

Organisation: Department of Computer Systems, Uppsala University

Keywords: Capstone subject – Computer-mediated communication

Description: To give students the experience of working in an international virtual workgroup, Daniels and colleagues have established a capstone subject offered jointly by Uppsala University in Sweden and Grand Valley State University in the USA. The subject is run over 10 weeks during which students work on a real world project focusing on both the technical and social skills aspects of working internationally. Project groups are assembled by instructors and managed using a fairly strict project management model. Each group comprises six students – 3 Swedes and 3 American – who are required to work collaboratively within the constraints of differences ion time zones and cultures.

Acknowledgements:

Carl Erickson and Bruce Klein - Department of Computer Science and Information Systems, Grand Valley State University, Michigan, USA

To find out more: http://www.docs.uu.se/docs/runestone/index.html

[29]

Innovators: Martin Dick, Dr Selby Markham and Judy Sheard

Presenter: Martin Dick

Institution: School of Computer Science and Software Engineering, Monash University,

Keywords: Assessment - plagiarism

Description: In response to a high rate of cheating amongst undergraduate and postgraduate ICT students, Dick and colleagues are investigating student attitudes with regard to cheating and plagiarism. The main aims of the research project are to educate students about what constitutes cheating, to improve the detection of cheating, to develop new cheating and plagiarism policies and ultimately, to prevent cheating.

To find out more: http://www.csse.monash.edu.au/cgi-bin/person_info?mdick

[31]

Innovators: Dr Rosemary King, Peter Donnan

Presenter: Peter Donnan

Organisation: CELTS, University of Canberra

Keywords: Computer-mediated Communication -- Peer Learning

Description: Donnan and colleagues recognise that although content transmission is easily possible using the Internet, student-to-student interaction is more challenging. They have designed a number of online activities in Information Technology and Education, a subject with an enrolment of approximately 150 students. To promote student interaction and critical discussions in these forums, they have linked assessment to students' participation. Students are asked to self-select their three best postings from the many they contribute to a range of online

214

interactions. They have observed an increase in the quality of students’ contributions.

To find out more: See the paper 'Strategies for promoting and moderating online discussions within a WebCT environment' by Peter Donnan at: http://www.tafe.sa.edu.au/top/webctconf/program/papers.html

Other Initiatives:

• Peter Donnan is presently involved in instructional design activities for the Flexible Delivery Development Unit that has been recently established at the University of Canberra to provide quality assurance for University priority courses. This initiative provides opportunities to implement approaches presented in the paper Donnan, P. (2000) 'E-moderating offshore electronic forums' published in the Conference Proceedings 'Offshore Education and Training in a e-Learning World' by the Centre for Research in International Education (University of Canberra).

[32]

Innovators: School of Arts and Sciences (Victoria), Australian Catholic University.

Presenter: Assoc Prof Carolyn Dowling

Institution: School of Arts and Sciences (Victoria), Australian Catholic University

Key words: Flexible Delivery - Mixed Modes

Description: As Head of School, Dowling has led a School-wide initiative that encourages staff to explore teaching and learning, particularly alternative modes of delivery, by employing mixed modes in their teaching and assessment. Modes include face-to-face lectures and tutorials, student group work and individual work, and synchronous and asynchronous electronic interaction. This initiative aims to increase staff understanding of pedagogical advantages and disadvantages of different modes of interaction with students, and to increase student awareness of their own learning styles so they can make choices about how to learn later in life.

To find out more: http://www.acu.edu.au/fas/vic/a&s/staff/page1.html

[35]

Innovator: Jenny Edwards

Presenter: Assoc Prof Jenny Edwards

Organisation: Department of Software Engineering, University of Technology, Sydney

Keywords: Technology education - Self-paced learning

Description: In response to demand for computer literacy training for IT and non-IT students, Edwards and colleagues set up a Computing Study Centre. The centre is equipped with self-teaching packages for students wanting to learn such things as word-processing, spreadsheets and databases to quite advanced levels. The centre is overseen by an academic staff member and staffed by undergraduate IT students.

215

Acknowledgements: a UTS Strategic Initiatives Grant initially funded the centre. It now forms part of a wider UTS initiative, BELL -Becoming an Effective Learner.

To find out more: http://it.uts.edu.au/about/contact?id=jenny

[36]

Innovator: Sylvia Edwards

Presenter: Sylvia Edwards

Organisation: School of Information Systems, Queensland University of Technology

Keywords: Student Journals - Graduate Attributes

Description: Within the QUT unit ITB322 Information Resources, Edwards has developed a learning environment to help students to broaden their knowledge base, and yet allow a certain flexibility to develop their own independent interests in the field of IT. In the unit, diaries and team projects are used to foster information literacy, particularly the development of communication and practical problem solving skills. Students learn to identify, retrieve and evaluate print and electronic business information resources that are relevant to a variety of problems. The aim of this subject is to encourage students to become information literate, Internet literate and resources literate, to develop good communication skills, the ability to work in teams and skills in the selection and evaluation of information. The primary method used to help students to develop these skills is a reflective journal that underpins the students' work throughout the semester.

To find out more:

• go to http://www.fit.qut.edu.au/~sylvia and follow the links to Teaching Theory Presentations

• or read:

Edwards, S. (2000). You have provided me with a new set of tools and taught me how to use them: Embedding generic skills within the IT curriculum. Paper presented at Lifelong Learning Conference: Inaugural International Lifelong Learning Conference, Yeppoon, Central Queensland, Australia, 17-19 July 2000. [Lifelong Learning Conference (1st: 2000: Rockhampton, Qld.) ISBN: 187 6674 06 7]

Other Initiatives:

• Particpation in QUT's Peer Assisted Study Scheme QUT's OLT (Online Teaching)

[37]

Innovators: Jennie Bales, Ken Eustace, Geoff Fellows, Lyn Hay and Rosalind Kentwell

Presenter: Ken Eustace

Organisation: School of Information Studies, Charles Sturt University

Keywords: Computer-mediated Communication - Teaching Tools

216

Description: Eustace and colleagues have adopted a computer supported collaborative work (CSCW) approach to teaching OO programming concepts and virtual world building. The goal of the initiative is to train students in professional workgroup practices. Using as its infrastructure, LC_MOO, an object oriented MUD learning community, students form online workgroup communities where they can engage in synchronous discussion, virtual conferences and secure meetings.

Acknowledgements: Internet Special Projects Group (ISPG), Farrer Centre Internet technology program, Centre for Studies in Teacher Librarianship, School of Information Studies and the Faculty of Science and Agriculture at Charles Sturt University.

To find out more: http://farrer.csu.edu.au/moo/jv/

Other Initiatives:

• K9MOO is a dog world theme park and virtual campus at http://ispg.csu.edu.au:9000 for developing R&D projects with a CSCW and virtual worlds approach. ISPG Zworld at http://ispg.csu.edu.au uses ZOPE ² as an object publishing environment with a similar CSCW approach. Both MOO and ZOPE products are used to teach about CSCW and Online Communities at Charles Sturt University.

• ZOPE is a Web application development system, by Digital Creations at http://zope.org

[38]

Innovators: Lynne Fowler & Maurice Allen

Presenter: Lynne Fowler

Organisation: School of Engineering, Murdoch University

Keywords: Teaching Tools - Learning Styles

Description: In an effort to increase understanding of student learning styles and to improve teaching practices, Fowler is investigating how students learn large, complex software packages. The initial research uses Rational Rose™, a professional Software Engineering CASE tool. As part of the first year Foundation Unit, Fowler administers learning style inventories to all students, who then identify their self-reported learning styles. She has developed an online tracking tool to monitor usage and movement through the CASE tool software. The goal of Fowler’s initiative is to examine the relationship between learning styles and the actual methods of usage of the software by students, as revealed by the tracking process. In this way, Fowler hopes to develop a learning methodology by which both students and lecturers become aware of the impact of learning styles on the learning process.

To find out more: http://eng.murdoch.edu.au/%7Elynne/

[39]

Innovators: Uni-u International

Presenter: Royce Gelmi

217

Organisation: School of Electronic Commerce, University of Notre Dame, Australia

Keywords: Flexible Delivery - Course Materials

Description: The University of Notre Dame, Australia, offers a first year subject in Internet fundamentals as part of their Bachelor of Electronic Commerce program. NDA aims to provide students with a sound education in electronic commerce by having students use the Internet as much as possible. He has also sought to maximise the flexibility of the subject by offering three modes of study: intensives plus online delivery, intensives plus online delivery plus face-to-face tutorials, and online delivery plus face-to-face tutorials. All online material is delivered using Blackboard®.

Acknowledgements: College of Business, School of Electronic Commerce (NDA) and Uni-u International.

To find out more: http://www.nd.edu.au/ecommerce

Other Initiatives:

• International Master of Electronic Commerce - Online Program.

[44]

Innovators: Robert Cox, Phil Scott, Mal Sutherland.

Presenter: Jennifer Goddard

Institution: Department of Information Technology, School of Management, Technology and Environment, La Trobe University(Bendigo)

Key words: Flexible Delivery - Mixed Modes

Description: To increase course accessibility and student numbers, Goddard and colleagues have been using mixed modes to deliver subjects for the Graduate Diploma in Computing. Course contact time was effectively halved by replacing thirteen three-hour lecture/tutorials with seven three-hour face-to-face tutorials offered at times convenient to distant learners (Saturday mornings and afternoons, for the participants named). Lectures were replaced by students accessing annotated course material prior to attending classes.

This initiative was pursued for four years, 1997-2000, and has been discontinued in 2001. Various factors contributed to this decision, but the major one was the majority student preference for face-to-face classes, with the Web materials as back-up rather than as the primary source. The Saturday offerings gained a few distant students, but lost those with sporting commitments, so there was little net advantage in numbers, and they were unpopular with staff.

Prior to the trial, we anticipated that IT students would be the most receptive of all to learning via ICT, and that we should be open to flexible ways to increase our courses' accessibility. Concluding that IT students do not necessarily want to learn via ICT, and that our students prefer face-to-face lectures, seems a positive contribution.

Acknowledgements: The initiative was undertaken as a trial, internally agreed within the Department. It was unfunded, and the industrial aspects of Saturday classes were not addressed.

218

To find out more: http://ironbark.bendigo.latrobe.edu.au/staff/jgoddard/Jennifer.Goddard.html

A paper describing the initiative in more detail is in preparation.

[41]

Innovators: Dr Robert Goodwin, Mr Phillip Roach and Mr Neville Williams

Presenter: Dr Robert Goodwin

Organisation: School of Informatics and Engineering, Flinders University

Keywords: Technology Education - Flexible Delivery

Description: At Flinders University, staff members are encouraged to optimise the flexibility of their teaching programs. Goodwin and colleagues are experimenting with techniques to make the delivery of computing subjects as flexible as possible without substantially increasing the workload. Initial testing is done with the materials for a first year computer literacy subject. The subject is offered in intensive and semester modes and attracts up to 700 students from disciplines as diverse as science, education, humanities, nursing and medicine. Currently, all the course material is available in print as subject booklets, on CD-ROM and as a student web site. Lectures are video-taped and made available to students. In the future, Goodwin and colleagues plan to include digital video-recordings of previous years' lectures on the CD-ROM and to add current lectures to the web site within a day of delivery of the lecture to students.

To find out more: Email Robert Goodwin: Goodwin@infoend.flinders.edu.au

[collected in follow-up]

Innovators: Dianne Hagan, Michael Kolling, Selby Markham, John Rosenberg

Presenter: Ms Dianne Hagan

Organisation: School of Computer Science and Software Engineering, Monash University

Keywords: Teaching Tools – Programming

Description: The teaching of programming to first year students is difficult because students perceive it as unrelated to anything they have done before, and because it requires problem solving, memorisation and attention to detail. In response to this difficulty, Hagan and colleagues have developed a Java development environment – BlueJ – to help first year students learn the object-oriented paradigm. Further support for students is provided by a helpdesk and two websites.

Acknowledgements: Michael Kolling, John Rosenberg, designers and developer of BlueJ, Selby Markham

To find out more: The CSE1202 website at http://www.csse.monash.edu.au/courseware/cse1202 has links that are relevant to BlueJ - as well as other things

[42]

219

Innovators: Dr Helen Hasan

Presenter: Dr Edward Gould

Organisation: Department of Information Systems, University of Wollongong

Keywords: Mixed modes - Access and equity

Description: Gould's department offers the Bachelor of Business to remote students living on the South Coast of New South Wales. Learning centres have been established at Nowra, Bateman's Bay and Bega. These centres provide local tutors and resources that complement web-based materials, video-conferencing and occasional visits by subject lecturers. This initiative responded to a government push to expand access to university education to the South Coast in the form of fully integrated degrees.

To find out more: http://www.uow.edu.au/commerce/buss/fgould-1.htm

[46]

Innovators: Jackie Hartnett

Presenter: Jacky Hartnett

Institution: School of Computing, University of Tasmania

Keywords: Course materials - Flexible Delivery

Description: Hartnett is making innovative use of the Internet in the teaching of a computer security subject at University of Tasmania. Hartnett makes a range of course materials available online including annotated lecture slides and examples of marked assignments and exam answers from past students which act as quality guides for current students. Hartnett has also plans to reduce the amount of time needed to respond to student emails by instituting a regular cross-campus interactive chat session and a subject bulletin-board on which she posts the answers to the frequently asked questions.

To find out more: http://www.comp.utas.edu.au/staff/jhartnet.html

Other Initiatives:

• 100% flexible delivery of computing degree

[50]

Innovator: Alanah Kazlauskas

Presenter: Alanah Kazlauskas

Organisation: School of Business and Informatics, Australian Catholic University

Keywords: Problem Solving - Student Needs

Description: Kazlauskas has observed that her students have difficulty with an introductory programming subject. To improve students' learning in this subject, problem solving skills are taught, along with program design and testing. Students are encouraged to pay particular attention to language aspects of solution development. Tutorial and assignment work is designed to support this approach to the subject's delivery.

To find out more: Email Alanah Kazlauskas

220

[52]

Innovators: Anjan Kundu

Presenter: Anjan Kundu

Organisation: Faculty of Technology and Industrial Engineering, Northern Territory University

Keywords: Industry Partnership - Industry Experience

Description: Kundu and colleagues at NTU are planning to forge links with industry to form a company to develop commercial products. Through the company, they plan to make the best use of the limited resources and expertise available in their region. Industry partners will benefit from access to the expertise of university staff, university staff will benefit from access to resources and opportunities to participate in research and development, and students will gain experience of working on authentic projects. This plan is modelled on an initiative that has been successfully implemented at Queens University, Belfast.

To find out more: Email Anjan Kundu

[53]

Innovators: John Lamp, Chris Goodwin, Craig Parker.

Presenter: John Lamp

Institution: School of Management Information Systems, Deakin University (Geelong)

Keywords: Communication skills - Computer-Mediated Communication

Description: Lamp takes a team-work approach to a third year project management unit. Students work in teams of approximately ten to manage authentic projects. The teams use computer-mediated communication (via First Class®) to manage their work, conducting virtual meetings. This approach is designed to emphasise the importance of communication in project management. It takes the emphasis away from the product, putting it on the project management process, communication between team members and communication with the client.

To find out more: http://mis.deakin.edu.au/pages/staff/jlamp.htm

Other initiatives:

• Deakin Electronic Trading Community®

• On- and off-campus tutorials using computer-mediated communication

[54]

Innovators: Dr Jon Osborn, School of Spatial Information Science & Mr Richard Langman, School of Engineering, University of Tasmania.

Presenter: Mr Richard Langman

Institution: Faculty of Science and Engineering, University of Tasmania

221

Keywords: Teacher Collaboration - Teaching Interest Group

Description: Langman and Osborne have established a teaching interest group that has attracted funding from the University. The purpose of the group, which at present meets bimonthly, is to share experiences and develop teaching innovations within the context of alignment of teaching and learning. Langman has used feedback from the interest group to develop strategies to improve first year students' motivation, and hence their willingness to try to understand the quite subtle concepts of electricity.

Acknowledgements: Professional advice from Dr Margaret Robertson, School of Education

To find out more: http://www.eng.utas.edu.au/cgi/staffn/langman/

[55]

Innovator: Michael Lawrence-Slater

Presenter: Michael Lawrence-Slater

Organisation: School of Information Technology and Computer Science, University of Wollongong

Keywords: Mixed Modes - Peer Learning

Description: In response to a shortage of skilled teachers and an increase in class sizes, Lawrence-Slater is developing an online ‘collaborative learning' subject, Case Studies in IT. The subject is to be simultaneously taught at three institutions - the University of Wollongong, the University of Melbourne and the Auston Open University Centre in Singapore. Subject delivery will comprise a combination of online and face-to-face teaching sessions where students from the three centres will be undertaking the production of a collaborative work using online, electronic media.

In the pilot iteration of the subject, Lawrence-Slater will travel from site to site to deliver the face-to-face practical skills component of the course.

Acknowledgements: Thanks are due for the support in this project received from Professor Joan Cooper - Head of the School of IT and Computer Science at the University of Wollongong, Professor Liz Sonenberg - Head of the Department of Information Systems at the University of Melbourne and Mr Ken Yeo - Principal of the Auston Open University Centre in Singapore.

To find out more: Email Michael Lawrence-Slater

Other Initiatives:

• Lawrence-Slater is also involved in practical research into tools and techniques for teaching student-directed learning, particularly in respect of participation within 'hybrid' classes where students may not necessarily be physically present.

[59]

Innovator: Dr Raymond Lister

Presenter: Dr Raymond Lister

222

Organisation: Faculty of Information Technology, University of Technology, Sydney

Keywords: Problem-based Learning - Assessment

Description: Lister is redesigning a large (approx 600 students) database subject. The subject is currently taught in a traditional lecture/tutorial/assignment mode and draws heavily on the Department's resources. The planned redesign involves reducing the number of lecturers by half and incorporating student-centred, problem-based learning. Lectures will be delivered in the first half of semester, with students able to sit a pass-only exam at the end of this period. Students wishing to gain higher marks will participate in problem-based learning for the remainder of the semester.

To find out more: Email Raymond Lister

Other Initiatives:

• Design for Web-Based On-Demand Multiple Choice Exams Using XML. See http://www-staff.it.uts.edu.au/~raymond/icalt/pap-046-Lis.doc

[61]

Innovators: Rod Jewell (Mackay campus), Maria Madsen (Gladstone campus) and Angela Schlotzer (Bundaberg campus)

Presenter: Jo Luck

Organisation: Faculty of Informatics and Communications, Central Queensland University

Keywords: Teaching Tools - Student Retention

Description: In an effort to standardise teaching across geographically dispersed campuses, and in turn, reduce the number of CQU students leaving for Brisbane-based universities, Luck and colleagues have adopted an Interactive System-wide Learning (ISL) approach to teaching. ISL uses video-conferencing technology to deliver interactive lectures across a number of campuses simultaneously. The lectures are interactive in that students can put questions to lecturers and hold group discussions with students at other campuses.

Acknowledgements: CQU Teaching and Learning Grant

To find out more: http://www.infocom.cqu.edu.au/Staff/Jo-Anne_Luck/

Other Initiatives:

• Staff development in the use of interactive video-conferencing for teaching and learning.

[63]

Innovator: Dr Chris McDonald

Presenter: Dr Chris McDonald

Organisation: School of Computer Science and Software Engineering, University of Western Australia

Keywords: Teaching Tools - Simulations

223

Description: McDonald has developed a network protocol simulator - cnet - for use in undergraduate computer networking courses. In development since 1992, cnet comprises simulation tools that allow students to experiment with various aspects of networking within a laboratory environment. cnet is currently used in about half of Australia's university computer science and information technology departments and in several hundred university departments around the world. At present cnet can be run on UNIX and Linux platforms.

To find out more: http://www.cs.uwa.edu.au/cnet/

Other Initiatives:

• Web-based Teaching Administration - funded by a University of Western Australia University Initiative Fund (UIF) Grant.

[64]

Innovators: Penney McFarlane

Presenter: Penney McFarlane

Organisation: School of Information Technology and Computer Science, University of Wollongong

Keywords: Flexible Delivery - Access and Equity

Description: Staff at the University of Wollongong are concerned that off-shore students be as well supported as their on-shore counterparts. To ensure uniformity of support, McFarlane and colleagues are utilising WebCT® to structure online subject materials and ensure accessibility across platforms. A goal of the initiative is to explore cultural differences in responses to the subject's web interface and modify it accordingly.

To find out more: http://www.itacs.uow.edu.au and http://www.cedir.uow.edu.au or Email Penney McFarlane

Other Initiatives:

• Penney McFarlane is also interested in investigating the use of team teaching at university level. There is a lack of research at this level of education and with flexible delivery growing its impact on teaching will only grow.

[67]

Innovators: This is a University-wide initiative, trialled in many courses. In our School Dr Timothy Ferris (course Systems Engineering Management) and myself (course Introduction to Electrical Engineering) are involved.

Presenter: Prof Andrew Nafalski

Organisation: School of Electrical and Information Engineering, University of South Australia (UniSA)

Keywords: Graduate Attributes - Teaching Tools

Description: Andrew Nafalski and colleagues are implementing a system known as Record of Achievement (RoA) in their courses. RoA enables students to develop and keep well-structured records of their achievements and experiences throughout their university studies to develop a portfolio that shows evidence of

224

their acquired knowledge and skills. In the process, students learn about graduate qualities, as described by the University, and track their development of these qualities. RoA also helps students to create résumés and job applications.

Acknowledgements: Staff of Flexible Learning Centre at UniSA

To find out more:

About the presenter:http://www.unisanet.unisa.edu.au/staff/homepage.asp?Name=Andrew.Nafalski

About RoA:http://www.unisanet.unisa.edu.au/gradquals/shappen/roa.htm

About Graduate Qualities at UniSA:http://www.unisanet.unisa.edu.au/gradquals/

Other Initiatives:

• Peer Assisted Learning, Active Learning, Learning Style Project, Intensive Modes of Delivery, Inclusive Curriculum Development.

[68]

Innovators: Connor Graham, Jon Pearce, Prof Liz Sonenberg

Presenter: Jon Pearce

Institution: Department of Information Systems, University of Melbourne

Keywords: Flexible Delivery - Mixed Modes

Description:

Project Title: Exploiting courseware delivery tools in Information Systems teaching

Amongst the staff of the University of Melbourne Department of Information Systems, there is a genuine concern about the quality of students' university experience given current trends, such as increasing large class sizes, demand for on-line resources, etc. In response to this, Pearce and colleagues are encouraging teaching staff to make 'smarter' use of teaching resources. WebCT ® is being trailed as a course management system to complement face-to-face classes by providing features such as forums, chat groups, online quizzes and other online material.

Acknowledgements: This project is funded internally by the University of Melbourne Teaching and Learning (Multimedia and Education Technology) Committee.

To find out more: http://www.dis.unimelb.edu.au/staff/Jon/

[69]

Innovators: Margot Postema, Jan Miller, Martin Dick.

Presenter: Margot Postema

Institution: School of Computer Science and Software Engineering, Monash University

225

Keywords: Course Materials - Teaching Tools

Description: To improve student performance in a second year Software Engineering course, Postema and colleagues are using a range of teaching tools that simplify manual tasks including detailed tutorial notes, exercises and technological tools. The team has also reviewed other subject materials to incorporate content from this subject with the aim of achieving a more integrated and valuable learning experience for students.

Acknowledgements: The PASE Tool was supported with two SCSSE summer scholarships.

To find out more: http://www.csse.monash.edu.au/~margot/

[70]

Innovators: Sauwan Cheah, Ruth Christie

Presenter: Michael Roggenkamp

Organisation: Faculty of Information Technology, Queensland University of Technology

Keywords: Peer Learning - Student Retention

Title: Supplementary Instruction @ QUT

Description: In an effort to reduce drop-out and failure rates in large classes (600-800 students), the Faculty of Information Technology has instituted a Peer Assisted Study Scheme (PASS) for all first year IT subjects. The scheme comprises weekly study sessions conducted by paid undergraduate students of high academic standard. PASS sessions are not additional subject tutorials; rather than focusing on what material needs to be studied for a particular subject, PASS sessions focus on how to study for success in a subject. Attendance at the tutorials is voluntary.

Acknowledgements:

1995 QUT Equity Grant Since 1996 funded as an ongoing Faculty initiative Supplementary Instruction was created by Deanna Martin, University of Missouri-Kansas City in 1973

To find out more: http://www2.fit.qut.edu.au/PASS

[72]

Innovator: Tony Sahama

Presenter: Tony Sahama

Organisation: School of Information Systems, Queensland University of Technology

Keywords: Course Materials - Plagiarism

Description: In response to changes in the student body, including larger class sizes and increased numbers of international students, Sahama is re-structuring his Information Systems Major (Programming) subjects and reviewing course content. Specific aims of this review are to motivate students by using task-driven programming exercises and to minimise plagiarism. Sahama hopes to achieve this

226

by introducing laboratory-based exams. More generally, Sahama hopes to increase student motivation to learn, thereby improving the quality of student learning processes.

To find out more: http://sky.fit.qut.edu.au/~sahama/

[73]

Innovators: main contact: Ury Szewcow (also in Dept CS, UTS)

Presenter: Dr Andy Simmonds

Organisation: Department of Computer Systems, University of Technology, Sydney

Keywords: Certification - Peer Learning

Description: In response to student demand for networking skills, Simmonds and colleagues have incorporated components of the Cisco Certified Network Associate (CCNA) course into the networking course at UTS. In addition to providing established course materials and tailored assessment, becoming a Cisco Academy has helped counter the problem of surface-learning in students by promoting peer learning and paced learning, and by ensuring there is plenty of practical work.

Acknowledgements: see Cisco information http://www.cisco.com/warp/public/10/wwtraining/certprog/lan/programs/ccna.html and the academy start page at http://cisco.netacad-intl.net/

To find out more: http://it.uts.edu.au/about/contact?id=simmonds

[75]

Innovators: Lorraine Staehr, Chris Cope.

Presenter: Lorraine Staehr

Institution: School of Management, Technology and Environment, La Trobe University (Bendigo)

Keywords: Experiential Learning - Small Group Teaching

Description: Dissatisfied with the lecture/tutorial teaching format, we have adopted a small group teaching approach to teaching a second year Information Systems Development subject at La Trobe University, Bendigo. The aim of the approach is to encourage students to take a deep approach to their learning and promote conceptual understanding through learning activities designed to teach both subject-specific content and general learning skills. Students attend workshops and tutorials in which they undertake experiential learning tasks, working in groups of 3 or 4.

Acknowledgements: La Trobe University, Bendigo Teaching Committee; School of MTE Research Advisory Group.

To find out more: http://ironbark.bendigo.latrobe.edu.au/~lorraine/pers1.html

Other Initiatives:

• Evaluation of the teaching of computer ethics

227

• Women in Computing Project.

[76]

Innovators: Assist Prof Steve Sugden

Presenter: Assist Prof Steve Sugden

Organisation: School of Information Technology, Bond University

Title of Initiative: Flexible Delivery in Elementary Tertiary Mathematics with Excel ®

Keywords: Teaching Tools - Maths Education

Description: "Discrete Maths" and "Maths & Stats for Computing" are compulsory subjects in the Bachelor of IT at Bond University. Student interest and skill level in these subjects vary considerably. To ensure that the top students are challenged and the middle to low-end students remain engaged in the subject, Sugden and colleagues take advantage of spreadsheet software. Using spreadsheets, topics such as calculus, combinatorics, induction, recurrences and many others, can be usefully modelled using tables, graphs, patterns and colours. An investigative approach is encouraged, and it is possible to illustrate a wide range of fundamental math principles with minimal algebra.

To find out more: http://www.bond.edu.au/it/staff/steve.htm

Other Initiatives:

• Currently preparing textbook for elementary discrete and continuous mathematics topics based on the Excel approach. It is aimed at low-level tertiary.

[77]

Innovators: Richard Thomas, supported by students David Erwin and Martin Ritchie

Presenter: Assoc. Prof. Richard Thomas

Organisation: Department of Computer Science and Software Engineering, The University of Western Australia

Keywords: Assessment - Student Retention

Description: Based on his earlier research findings, Thomas has hypothesised that the time it takes a first-year student to complete a lab-based software engineering task is related to their likelihood of successfully completing the whole subject. To test this hypothesis, Thomas has developed an 'early warning system' using Teachers' and Learners' Collaborascope (TLC) technology developed by the University of Glasgow. With this system, Thomas and his colleagues can track the length of time it takes individual students to complete lab-based tasks. It is anticipated that this information will aid in the identification of those students at risk of failing the subject and the allocation of teaching resources to those students most in need.

228

Acknowledgements: Funding is partially from internal Departmental resources and also via the UK EPSRC-funded GRUMPS project at the University of Glasgow.

To find out more: http://www.cs.uwa.edu.au/~richard/ and http://grumps.dcs.gla.ac.uk/

Other Initiatives:

• The teaching of generic IT skills: (i) a trial of the Computing Skills Workshop on freshers at UWA (ii) development of a course on Learning to Learn About Computers (iii) evaluation of far transfer effects in generic IT skills courses

[78]

Innovator: Michael Turk

Presenter: Michael Turk

Organisation: School of Computing and Information Technology, University of Western Sydney

Keywords: Curriculum Integration - Professional Skills

Description: Turk has incorporated integrating projects and team-related open-ended exercises into his programming and systems analysis and design subjects. His general goal is to prepare IT students for lifelong learning. The specific aims of the initiative are to equip students to effectively manage themselves and their time, to instil students with a sense of personal responsibility for their professional actions and to develop students' teamwork and self-awareness in addition to their technical IT skills.

Acknowledgements: Thanks are due for the support and encouragement of this initiative received from Professor Athula Ginige, Head of the School.

To find out more: http://fistserv.macarthur.uws.edu.au/michaelt/publicn.htmA paper describing the initiative in more detail is in preparation.

Other Initiatives:

• The Undergraduate Study Maturity Model and Personal Study Process - Studies for partial completion of Doctor of Teaching, currently in progress.

[82]

Innovators: Warren Yates

Presenter: Prof Warren Yates

Organisation: Telecommunications Engineering, University of Technology, Sydney

Keywords: Graduate Attributes - Government Project

Description: The Faculty of Engineering at UTS participated in the Australian Technology Network Project on Graduate Capabilities. As Associate Dean of Teaching an Learning, Yates has been involved in the development and delivery of a subject, Authentication and System Security, to be used as a model for subjects designed within the 'generic capabilities' framework, that is, subjects that are 'graduate attribute' based and learner centred.

229

Acknowledgements:

Associate Professor Keith Trigwell, Director, UTS Centre for Teaching and Learning Dr Keiko Yasukawa, Educational Developer, Faculty of Engineering

To find out more: http://www.uts.edu.au/div/clt/ATN.grad.cap.project.index.html and http://cea.curtin.edu.au/ATN/UTSatn1.html

231

APPENDIX K. Industry questionnaire

ICT-Ed Project FIT, Monash University Caulfield East, Vic, 3145 11 July 2001

Information & Communication Technology (ICT) Education Research

Industry Survey This survey is directed to the Information and Communication Technology (ICT) industry. It has been designed to build a picture of industry’s views of university ICT education. It has been distributed to a selection of members of the Australian Computing Society.

We are looking for responses from both those who employ recent ICT graduates and those who do not. We need to build a balanced picture. If you feel there is someone better able to complete the survey, feel free to pass it on to that person.

If you prefer to complete the survey online, please go to http://cerg.infotech.monash.edu.au/icted/industry/indexmain.html

This research has been commissioned by the Australian Universities Teaching Committee, which is under the Federal Government’s Department of Education Training and Youth Affairs. Details of the project can be found at http://cerg.csse.monash.edu.au/icted/.

Summary results of the survey will be published on our Web page (URL above), approximately four weeks after you receive this survey. This assumes, of course, that the responses are prompt.

We hope that you find the survey interesting.

Thank you for your time.

Monash University Ethics Requirements

Your response will be anonymous unless you are interested in helping us if we need further information. In that case we are asking you to give us your name and preferred contact details. We will not release any information that could identify an individual or organisation. Aggregate findings will be made available on the project’s web site.

Participation is entirely voluntary. You indicate your consent to participate by completing and returning the questionnaire. You may withdraw your consent at any time by contacting the researchers. Should you have any complaint concerning the way this research is conducted, please call The Standing Committee on Ethics in Research on humans:

The Standing Committee on Ethics in Research on Humans Monash University Clayton, VIC, 3168

232

Section 1. Demographics (All respondents to complete this section)

1.1 Your position: ________________________________________

1.2 Including contract positions, approximately how many people does your business employ?

Less than 6 1 6 – 20 2 21 – 99 3 100 or more 4

1.3 To which industry grouping does your business belong (choose one)?

Accommodation, Cafes & Restaurants 1 Agriculture, Forestry & Fishing 2 Communication Services 3 Construction 4 Cultural & Recreational Services 5 Education 6 Electricity, Gas & Water Supply 7 Finance & Insurance 8 Government Administration & Defence 9 Health & Community Services 10 Manufacturing 11 Mining 12 Personal & Other Services 13 Property & Business Services 14 Retail Trade 15 Transport & Storage 16 Wholesale Trade 17 Other -Please specify __________________________________

18

Most items will involve circling the option that applies to you.

Only use DK (Don’t Know) if you have no knowledge on the item. Similarly, use NA (Not Applicable) only when the item does not apply in your organisation.

233

1.4 What is your primary location?

i. State/territory ii. Region

ACT 1 Capital City 1 NSW 2 Regional Centre 2 Northern Territory 3 Rural 3 Queensland 4 South Australia 5 Tasmania 6 Victoria 7 Western Australia 8 Not in Australia Which country? _____________________

9

1.5 What is your highest tertiary qualification?

Certificate 1 Diploma 2 Degree 3 Higher Degree 4 Other 5 None 6

1.6 Does your organisation hire people with ICT qualifications from any of the following?

Yes No DK

TAFE Colleges 1 2 3 Private TAFE providers 1 2 3 Private Computer Training Colleges 1 2 3 Universities 1 2 3 Other Please specify_____________________________________

1.7 Does your organisation have any contact with University ICT departments for the following activities?

Yes No DK

Industrial Experience programs 1 2 3 Course or subject advisory committees 1 2 3 Research and development projects 1 2 3 Consulting 1 2 3 Other Please specify_____________________________________

234

Section 2. Complete this section only if your organisation employs recent university ICT graduates. If your organisation does not currently employ ICT graduates, go to Section 3.

2.1 Please indicate the general area(s) in which your ICT university graduate recruits have studied (select more than one area if applicable).

Computer Science/ Software Engineering 1 Information Management Systems /Business Systems 2 Distributed Software Systems 3 Multimedia (including Web design) 4 Library and Information Management 5 Other (please specify …………………………………….) 6

2.2 Please indicate how important you think each of the following skills are in ICT graduates.

Not important

Important

Skills in particular computer languages 1 2 3 4 5 DK Skills in particular software applications 1 2 3 4 5 DK Foundation in theoretical principles 1 2 3 4 5 DK Ability to communicate with clients 1 2 3 4 5 DK Written communication skills 1 2 3 4 5 DK Ability to work in teams 1 2 3 4 5 DK Project management skills 1 2 3 4 5 DK Problem solving skills 1 2 3 4 5 DK Understanding of business processes 1 2 3 4 5 DK Preparedness for quick entry into job tasks 1 2 3 4 5 DK Skills in languages other than English 1 2 3 4 5 DK Other (please specify: .………….…………..………)

1 2 3 4 5 DK

ICT graduate is used throughout this questionnaire to refer only to people who have recently graduated from a university course that focused primarily on information and computing technology.

Use DK (Don’t Know) if you have no knowledge on the item.

235

To answer the following questions think about the typical ICT graduates you have worked with recently. 2.3 Indicate the extent to which the usual ICT graduate, who has worked for you, meets your organisation’s needs for the following skills.

Doesn’t meet our needs

Meets our

needs

Skills in particular computer languages 1 2 3 4 5 DK NA Skills in particular software applications 1 2 3 4 5 DK NA Foundation in theoretical principles 1 2 3 4 5 DK

Ability to communicate with clients 1 2 3 4 5 DK

Written communication skills 1 2 3 4 5 DK

Ability to work in teams 1 2 3 4 5 DK

Project management skills 1 2 3 4 5 DK

Problem solving skills 1 2 3 4 5 DK

Understanding of business processes 1 2 3 4 5 DK

Preparedness for quick entry into job tasks 1 2 3 4 5 DK

Skills in languages other than English 1 2 3 4 5 DK

Other (please specify: .………….…………..………)

1 2 3 4 5 DK NA

2.4 Please indicate your satisfaction with these graduates:

Dissatisfied Satisfied

Overall performance of these graduates 1 2 3 4 5 DK NA Performance compared to graduates in other disciplines 1 2 3 4 5 DK

2.5 As far as you know, does your organisation have difficulty finding ICT graduates to meet its needs?

Yes 1 No 2 Don’t Know 3

2.6 Indicate the importance of the following in your organisation’s decision to recruit ICT graduates.

Unimportant Important

Broad education in ICT further to technical skills 1 2 3 4 5 DK NA Theoretical background to support R&D 1 2 3 4 5 DK

Potential to contribute to future directions of the organisation 1 2 3 4 5 DK

Other (please specify: ___________________________________ )

1 2 3 4 5 DK NA

Use NA (Not Applicable) if the skill is not important in your organisation. Use DK (Don’t Know) if you have no knowledge on the item.

236

237

2.7 Please indicate your organisation’s preference when recruiting ICT graduates for the time needed from the graduate’s entry to their productivity.

Short lead time with minimal

training needed

Longer lead time, allowing

for training

Time needed from entry to productivity 1 2 3 4 5 DK NA 2.8 Would you recommend that people seeking a career in the ICT industry do a university ICT degree?

Not Recommend

Strongly Recommend

1 2 3 4 5 DK 2.9 Would you recommend your ICT graduates to other employers?

Not Recommend

Strongly Recommend

1 2 3 4 5 DK 2.10 How responsive are university ICT departments to the needs of your industry?

Not Responsive

Very Responsive

1 2 3 4 5 DK 2.11 What do you think Australian universities can do to improve the preparation of ICT graduates for work in your industry?

2.12 What is the likelihood of your organisation continuing to employ university ICT graduates?

Not Likely

Very Likely

1 2 3 4 5 DK Thank you for helping us with this survey. We would like to be able to follow up on interesting information you might give in this survey. If you would like to be involved in a follow-up interview, please provide your preferred contact details below:

Name: ____________________________________ Email: ____________________________________ Phone: ____________________________________

Use the enclosed postage-paid envelope to return your completed questionnaire.

238

Section 3. Complete this section only if your organisation does not currently employ recent university ICT graduates.

3.1 Has your organisation employed university ICT graduates in the past?

Have never employed graduates

1

Employed once, but not currently 2

3.2 Why doesn’t your organisation currently employ ICT graduates?

3.3 Do you anticipate that your organisation will recruit university ICT graduates in the future?

Definitely Will Not

Definitely Will

1 2 3 4 5 DK Thank you for helping us with this survey. We appreciate your input. We would like to be able to follow up on interesting information you might give in this survey. If you would like to be involved in a follow-up interview, please provide your preferred contact details below.

Name:_______________________________ Email:_______________________________ Phone:_______________________________

Use the enclosed postage-paid envelope to return your completed questionnaire.

ICT graduate is used throughout this questionnaire to refer only to people who have recently graduated from a university course that focused primarily on information and computing technology.

239

APPENDIX L. Graduate cases

Case 1. Ben Gender: Male

Age: 24

Course completed: none

Business size: micro (2 employees)

Position: Manager

Background Ben is a 24-year old male who lives in the eastern suburbs of Melbourne with his wife who is a childcare worker. Ben grew up in the eastern suburbs. His father, although trained in accounting, has always worked in the computing industry, currently for IBM. His mother, once a hairdresser, has worked primarily in the home since her children were born. Ben has one sibling: a younger sister who is currently studying for a university degree in psychology/psychophysiology.

When Ben was aged 10, his father brought an IBM-XT computer into the family home. Initially Ben was not interested in the computer, but when it was replaced by a later model upon which games could be played he taught himself how to use it. His father had encouraged him to use the computer, but until the possibility of playing games presented itself, he had shown little interest. Ben taught himself how to use dos so he could get games running, experimenting with himem, memory managers, autoexec.bats, etc. This learning came relatively easy to Ben and he enjoyed his successes when he “conquered the machine” and learnt something new. Ben continued to learn more about computers through his desire to play games until early secondary school. He was motivated by the challenges and the “fun”. He had “computer friends” in the local neighbourhood with whom he played games. After his father had brought home a modem and organised a “Pegasus” connection through his work, Ben and one of his neighbourhood friends became interested in electronic bulletin boards. Ben ran a personal BBS.

Ben has a confident and out-going manner. He enjoys challenges and the satisfaction of solving problems and providing useful solutions. He says he is a natural trouble-shooter and a perfectionist. In his work, Ben is motivated by the satisfaction of doing good work and of working with “good people”.

Schooling At school, Ben excelled in and “loved” art. He pursued his art studies at secondary school, while also taking, and getting A-pluses in, information technology. His parents gave him guidance as to what subjects to take at school. When he decided to drop Science at Year 10 and then Mathematics at Year 11,

240

they were concerned that he was limiting his options, but he was confident that he would not need these subjects. Ben says that his parents have always let him have control over decisions about his education and career.

At school, Ben was in the computer club. The club was started by a teacher that Ben admired. He says the teacher had a brilliant mind and was a “nice bloke”.

In his final VCE year, Ben took Art, Media Studies, Information Technology, Graphics and English. Ben’s VCE was very successful, with a high TER score. He got top marks for Art and for Information Technology.

University education Ben says that after completing secondary school, it was “a toss up” as to what area – Art or Computing – he would pursue. In terms of a career, the two areas that interested him were graphic design and three-dimensional computer animation. He understood that neither of these required a tertiary qualification, and that “being very good” was sufficient to succeed in these fields.

Ben did not pass the selection process for a university education in graphic design, instead enrolling in a two-year TAFE certificate in 2-dimensional and 3-dimensional computer-aided art. He was aware that he was taking a risk by not going to university, but he was confident he knew what he wanted to do and he was confident in his skills. He says that his parents would have liked him to go to university, but he convinced them that he didn’t need to. Ben did not complete the TAFE course because he didn’t think it was a very good course and because

Employment history and career aspirations During the last months of his VCE, Ben worked for a computing business, specialising in multimedia, where he was exposed to the latest computing and multimedia technologies. During his TAFE course, he worked for micro computing business owned by a family friend. He helped with networking and software solutions. Through this friend he then moved into a web-design business. He worked in for this business for about 6 months, “churning out quick and nasty web-sites”. Ben was uncomfortable with this approach to business. He says he is a professional and a perfectionist. He left this job and temporarily returned to working for the family friend on a contract basis. Ben then took a job with an Internet café as a “techie” and then as “technical manager”. He observed that the business was not doing well, so decided to leave before it failed.

Reflecting on his work history, Ben says that he was not happy with the management at any of the businesses he worked for. He believed that he could make a more successful business himself and decided to “give it a sh1997, Ben has run his own business. He operated as a “one man show” for several years making small web sites. He was not making much money, but he was comfortably living in his family home. When he got engaged to be married he realised he would need to make money. He secured large contract in 2000 and brought a friend into the business and has continued in this way since.

Over the last 8 months finding work has been more difficult. Ben blames the end of the IT boom. He now has a responsibility to keep finding work because he has

241

a business partner to consider. He actively targets clients “with money”, so he does not have to compromise the product and can meet his perfectionist inclinations. When asked about his aspirations, Ben says he has moved away from graphics and can see himself in a management position. Ben has always been a “jack-of-all-trades”, so he can communicate effectively with different specialists and he understands how different areas work. He thinks his skill set should enable him to manage a small group of graphic designers and programmers, working on moderately sized web-site productions and perhaps even applications. He would be happy managing his own small business or someone else’s, but he wouldn’t like to be too removed from “the action”. Most of the work currently done by Ben’s business is web site development, but he would like to expand it to provide complete digital solutions.

Ben does not regret having not completed a university degree and is comfortable with the current direction of his career, but he sometimes thinks about the possibility of future formal education in business or management.

242

Case 2. Dave Gender: Male

Age: 28

Qualifications: BSc (Hons) in Bird Ecology; short course provided by recruitment company leading to certification in software testing.

Business size: Large (100+)

Position: Contractor - Business Analyst

Background Dave is a 28-year-old male who graduated from the Bachelor of Science Honours in 1995, specialising in bird ecology. He currently works in the credit card division of a large bank as a contractor business analyst. In his immediate family unit, Dave is the second children having an older brother and a younger sister and three younger brothers, biological and adopted. His father also has a responsibility of care for five other children, biological and adopted, from other relationships. Dave describes his father as an entrepreneur and sees himself and his three brothers as sharing his father’s entrepreneurial tendencies.

Dave’s mother, who is separated from his father, is an Occupational Therapist. His father is an Obstetrician/Gynaecologist by trade but works for himself in property development. Dave recalls that his father always did other things while practising medicine, has never been an employee and is something of an entrepreneur although “not very good at it”.

Like himself, Dave’s older brother is a contractor in creative roles. His sister is an employee in the travel industry, another brother is a contractor in the field of IT and the environment and his youngest brother is still at school. Dave describes himself as entrepreneurial, but pragmatic unlike his father who is more of a “gambler” and less successful in his entrepreneurial activities. His mother, although not entrepreneurial, nurtures this trait in her children. Dave sees his siblings and himself as milder versions of their father – “we’re in control”.

Dave sees opportunities and executes. He doesn’t um and ah, he just goes for it without worrying too much about the implications. This approach to life sometimes lands him in trouble but he figures that this is a natural consequence of being productive – some initiatives will miss but on the whole he has more hits than misses. Dave is drawn to people who are independent, self-driven and self-employed but not necessarily successfully. He describes his friends as generally care-free and happy-go-lucky.

Schooling Reflecting on his schooling, Dave says that it was alright and that he attended school, prominent Melbourne private school, regularly – “normal school, nice family”. He was never part of a clique at school and he did okay across the board.

243

He sees his current attitude as perhaps rebellion against his schooling and at odds with middle-class norms. Dave had no interest in IT in high-school and originally thought he might train as a psychologist. He had no real plan until he received his Year 12 results. He identified the courses he could apply for based on his results and decided from there.

University education Dave has a strong personal interest in ecology and conservation and enrolled in Bachelor of Science at a Melbourne metropolitan university, specialising in Zoology and Botany. His choice of subjects was not a career based decision but rather, based on interest. Part of him wanted to get into conservation and land management but he soon realised that he just enjoyed studying the natural sciences and wanted to find out more. As a university student he didn’t really care about work and money wasn’t a problem as he was living at home for the most part.

Dave completed his BSc followed by an Honours year specialising in Bird Ecology. He graduated in 1995 before accepting a PhD scholarship in Queensland. He discontinued his PhD after 10 weeks when it became apparent to him that he had been recruited as cheap research labour and would not be able to pursue his own research interests – North Queensland was the “worst place on earth for a Melbourne, tree-hugger, non-redneck”.

Employment history and career aspirations Dave returned to Victoria and took up a position managing farm operations at a trout farm in Eildon. He found this position “hermitising”, he felt out of place and returned to Melbourne to work as a builder’s labourer. For Dave, his time labouring was a period spent escaping reality and avoiding progressing.

In mid 1998, Dave was informed of a 2 week software development and quality assurance course being run by an IT recruiting firm. At the time, Dave didn’t know what IT was but figured it paid better and would be less physically gruelling than labouring and he would be able to use his intelligence. The firm was looking for non-IT graduates, particularly Arts graduates, who could look at software with a creative and analytical eye.

On completing the course, Dave worked as a contract software tester for a credit card software vendor. This position led to his being contracted to a client of the vendor as an on site consultant and involved working in Sydney as well as travel to Orlando. As his knowledge base grew, Dave became involved in more business analysis and less software testing. Dave left this company 18 months ago to work for his current employer, a large bank, as a contractor business analyst operating through an umbrella company.

Dave sees the ICT industry as somewhat mercenary. Being a contractor, his company is not committed to him and can terminate a 12 month contract with 4 weeks notice. His long-term commitment is to himself as a commercial entity and he aims to minimise his susceptibility to market ups and downs. This entails being adaptable, professional and fitting into any company – “or you won’t get the next contract.” Dave regards himself as having good business acumen but not as a capitalist.

244

He is not committed to or motivated to stay in the ICT industry. Despite this, he sees his work as interesting and challenging; not necessarily intellectually, but organisationally and in a management sense it’s very challenging. Dave stated that he can only do the things he’s interested in, no matter how much money he’s paid - “I’m not going to hang around in a field that doesn’t challenge me.” A change of industry now would not change Dave’s vision.

Dave’s vision relates to his lifestyle rather than his career, and this vision changes from time to time. For Dave, IT is just a means to an end but a means that he happens to enjoy. There is no ‘ideal job’ for Dave although he would perhaps prefer to be in an industry that involved excitement, travel and ‘do-gooding”. He knows he does not want to be a wage slave or tied to a company. In this sense, he says that he identifies with Generation Xers – he doesn’t care what he does for a living, he just wants to happy and have nice people around.

Dave has been offered permanent positions in every IT role he has taken. Some of his colleagues can’t understand why he wouldn’t take it but he prefers being in control rather than being controlled.

Commenting on the value of an ICT degree in the ICT industry, Dave distinguishes between technical and non-technical roles, such as his own. Having an ICT degree is valuable in non-technical roles, which make up a large proportion if ICT positions, but you can wing it. “I don’t know the first thing about computers and programming . . . It’s all about common sense and being able to work with people.” In regard to technical roles, the technology learnt in first year of university will likely be obsolete by graduation. In addition, the project life-cycles learnt at university may not be relevant on graduation as your employer may not adhere to them.

Dave sees those ICT professionals who enter the industry through graduate programs as having a different perspective to those who come in ‘through the back-door’. “Those who come in through the backdoor tend wonder what all the fuss is about. It’s just another industry but you get paid twice as much.” He sees a lot of people do university courses for vocational reasons because they think it’s the “Holy Grail”, that it’s going to make them happy. Dave is quite against the vocationalisation of education, believing that education is about personal development.

245

Case 3. Grant

Gender: Male

Age: 23

Qualifications: Completed one year of Bachelor of Digital Systems

Graduated from Bachelor of Computing

Business size: Micro (<5)

Position: Web developer/programmer

Background Grant is a 23-year-old male who lives with his girlfriend in a rented house in Melbourne’s Eastern suburbs. He completed a Bachelor of Computing degree in 1998 and now works on the internet programming side of a two-person web solutions business. His boss, who is a friend from high school, is the company founder and works on the design side of the business. He is reserved, somewhat cynical and not driven by financial gain so much as lifestyle. He describes himself as lazy or economical in his choices made regarding education and employment.

Mother and father, who are now divorced, are both programmers by trade. His father has worked as a technical/IT manager, then an IT academic and then as a manager of people as opposed to ‘technical’ manager. Grant’s mother currently works as an IT academic. Grant has two younger brothers: Sam, 21, who is about to commence a nursing degree after taking a few years off between high school and university, and Tom, 18, who is enrolled in an Arts degree. Tom is just as technically-minded as grant while Sam is non-technical. Grant has one other relative, an older cousin, who is a programmer however the most popular career choices in his wider family are teaching and music.

Grant was exposed to computers, programming and academia from a very early age. He first learned a programming language from his father at age 11 and spent time at university since he was about 5 years old. He has always had an interest in computer games however his academic interest in computing waned in his mid-teens when he decide he did not want to follow in his parents’ footsteps.

Grant’s sports interests include karate in which he trained for ten years, soccer and rock-climbing. Until he was about ten, Grant’s family did not own a television. He feels that this may be a factor in his siblings’ and his own interest in reading and computing in childhood.

Schooling Grant was precocious child who was reading in his pre-school years. Identified as exceptionally bright at a young age, he entered a Montessori school as did his younger brothers. On completing Grade 5, Grant skipped Grade 6 and entered

246

Year 7 at a private co-educational secondary school with a half scholarship. Grant worked well until Year 9 when he rebelled against school, authority and the study of computing. He described ‘hanging in there’ with the support of some teachers. He did well in Year 12 but not as well as could if he applied himself. Many of his friends at school did very badly and none went on to university.

Grant was often told by his parents and teachers that he had ‘potential’. Potential became a dirty word as it seemed to refer to what you could be doing rather what you are doing – “whatever you’re doing, ain’t quite up to your potential”. Grant says that he still doesn’t live up to his potential but whereas he thought this was a bad thing in his teen years, as a mature person, if had the chance again, he would have tried for better marks in high-school. Despite this, he saw potential as over-emphasised as it focused on the 1% wrong rather than the 99% right. He can now understand the motivation of parents in emphasising potential.

Grant worked hard at school until about Year 9 when he rebelled deciding that “people weren’t going to tell me what to do”. After this time, his grades dropped. Looking back at this time, he recognises that being two years younger than his class-mates caused tensions and that his defence was to “stop being smart” in order to fit in – “it’s cooler not to work”. Grant always enjoyed reading and writing at high school but has since let the writing slip. He recalls some teachers at high-school, with whom he was friends and who were supportive of him while most were not, “pushing him through”. Overall, the work he did do out-weighed the work he didn’t do. He “held on” through high-school obtaining a TER of 88 in Year 12. Although others around him were obtaining TERs in the high 90s, Grant believes that his life was more balanced while the high-scorers “didn’t have a life”.

University education Grant always knew he would go to university as this was strongly encouraged by his parents. After Year 12, attracted by the prospect of building robots, he enrolled in a digital systems degree at a Melbourne metropolitan university. On arriving at university, there was no longer anyone pushing him. He had the attitude that he was still rebelling but there was no-one around to rebel against except himself. He describes himself as lacking control and self-discipline at this time – “I slide easily”.

The digital systems course contained more maths and other subjects that did not interest him, than he expected. He did poorly overall and after the first year, transferred to a computing degree at the same university. He chose this course partly because his mother was involved in it’s organisation and she advised him that he would be well suited to it given his aptitude for computing; it seemed like the easiest way to go.

In his computing degree, Grant did well in programming subjects and poorly in others, working on the premise that achieving 51% in a subject was 1% wasted. He took 4.5 years to complete his 3-year degree as he had to repeat subjects. He began to dislike his course after the first two years as it was beginning to drag on. He began to miss classes, which he attributed to waning interest and having to rely on public transport to get to university.

He recalled completing an industrial experience subject in the third year of his degree however after 6 months of work, the project group still lacked organisation

247

and produced what he considers as a worthless solution. He feels that difficulties with this project were attributable to poor guidance on the part of the academic supervisor. Having spoken to other graduates, he feels that a sandwich (industry) year may have been more valuable than a few hours a week while continuing other studies.

Grant completed his degree in 1998. He described himself as “scraping through” and estimates that his attendance, especially toward the end of his degree, was about 60%. The time he had invested in his education and the urging of his parents pushed him towards completion. Looking back on his time at university, Grant expressed the belief that some academics are resistant to re-training and are out of touch with industry standards.

Employment history and career aspirations During his time at university, Grant commenced a job installing computer networks in schools. In this position he had contact with a wide range of clients from young children to adults. On completing his degree, he took the first job he was offered in a small custom software house which involved installing and providing remote support for a car-rental management package used by a number of nation-wide car rental companies. Grant received some informal training in this job, allowing him to hone his programming skills. This position involved phone and some on-site, client contact. Grant does not feel that his university training prepared him for client contact so much as his experience in his previous job. He was in this position 1.5 years before leaving for his current position.

The web development company that Grant currently works for was established about 4 years ago by a good friend from high-school. Grant was warned by others of the risks of working for a friend, but considered the pay-offs as worthwhile. The company is currently developing in-house software which will hopefully bring more money and clients to the company. An aspect of his job that he particularly enjoys is having total control over his programming work although this sometimes leads to him spending too much time on it, costing the company money. Grant entered the company with hope of eventually becoming a partner. He has since found out that the company structure will not allow this; he did not investigate this part of his employment very deeply.

Grant describes himself as creative and would like one day to design games however is not sure whether this will be enjoyable or will involve more maths than he prepared to do. He hopes that one day, with advances in bandwidth, he will be able to combine his internet programming skills with his interests in game design to create online, real-time games.

Although he would like one day to be running a company of 10-15 people, this is up in the air at the moment. He has kept in touch with an employment agency because he may be in need of a job if the company’s current project doesn’t come off. also, although it is stable now, delays in client payments have threatened the financial stability of the company.

Grant recognises the trade off in not applying himself one hundred percent at university and the kind of salary he can command as a graduate. He also sees this trade-off in not working long hours and financial rewards. He has seen his parents work very long hours and does not relish that lifestyle; “I would rather not have heart disease at 50”. Describing himself as not greedy by nature, he is not

248

resentful that he does not earn more, but recognises that he wasted opportunities in terms of employability. He attributes his current position to laziness; he took first job he was offered and sees that he could have exposed himself to more recruitment-directed activities during his time at university.

In describing how his degree has influenced his career, Grant identified himself as possessing more and better abstract knowledge than his non-graduate peers but states that this is not always considered valuable by employers, especially non-technical management who are more interested in speed and cost of delivery.

Grant was strongly encouraged by his parents to go to university because they felt that he would not get a well paid job without a degree. Grant disagrees with this view as his employer has had no formal training and he has seen others become successful in the IT industry without formal training. He believes that, depending on choices made at university, a graduate’s knowledge may be obsolete by graduation. He sees the importance of self-education, both for graduates and non-graduates and from this point on will primarily self-educate.

Grant was always encouraged to fulfil his potential but does not see himself as having done that to date. He sees his university career as “killing time”; his heart was not really in it. He is not driven by money and had an “economical” outlook at the time – “don’t do more than is needed”.

249

Case 4. James

Gender: Male

Age: 23

Qualifications: Completed one year of Bachelor of Engineering

Graduated from B.Bus.(Accounting)/B.Comp.

Business size: Large (100+)

Position: IT Consultant/Database Administrator

Background James is a 23-year-old male who lives with his parents and younger sister in Melbourne’s southern suburbs. He completed a double degree, Bachelor of Business (Accounting)/Bachelor of Computing, in 2000 and now works as an IT Consultant/Database Administrator for a large consulting firm. James’ father is the purchasing officer for a small car trim company and his mother is the principal’s secretary in a secondary school. His sister, 21, is completing an Arts degree at the same university that he attended.

James’ parents emigrated from India 35 years ago, arriving in Australia with is parents circumstances as immigrants have influenced James in his

attitude to education and his career. His parents did not have the opportunities that he now has and are very supportive of their children’s education. James is very proud of the fact that he is the first in his family line to obtain a tertiary education.

James recalls always being interested in computers. His family got their first computer, a PC 286, when he was in Year 8. He played ‘shoot ‘em up’ and role-playing games as a teenager and now enjoys playing games over the Internet, when he has time, to unwind. Computing came naturally to him and by Year 10 he was repairing, upgrading and configuring friends’ computers. He describes himself as something of a hardware fanatic and now that he is working full-time, he is buying all of the components and brands he always wanted to and is building his own computer.

When he has the time, James plays jazz piano, catches up with friends and fixes things around the house. His great love is his car which is his main hobby

Schooling Primary school was ‘quite tough’ for James. He recalls being ‘a very unhappy little boy’ as there was a lot of racism in his school and he was the only coloured child among his peers. Difficulties with racism continued into high-school, a private co-educational secondary school, as he and his peers entered Year 7 but phased

250

out in later years as his high-school had a more racially diverse student body than did his primary school.

In both primary and secondary school, James was rowdy and often disruptive in class which he attributes to his being very energetic. Years 7 to 10 presented no problems for James although he was ‘a last-minute worker’, a study habit he maintained until his last year of university. Year 11 came as a shock to him as he had to do homework every night. By Year 12, he had adjusted to the work load and obtained a TER of 86 with which he was quite happy. James did not start programming before starting university nor did he take any IT subjects in high-school as they were ‘secretarial’ in nature and his computer use was already quite advanced. In Year 12 he studied Accounting, English, Physics, Maths Methods and Specialist Maths.

University education James did not always think he would be suited to university as he is a very hands-on person and didn’t enjoy theoretical subjects. In his senior high-school years, James’ ambition was to become an engineer and he attended an engineering summer school. He enrolled in a Bachelor of Engineering degree but soon found it was not for him; “there’s only so much calculus I can do in a day”. He found himself working hard but receiving poor marks. He passed his first year but, after a lot of soul searching, decide to discontinue engineering and enrol in a Bachelor of Business (Accounting)/Bachelor of Computing double degree.

With the move from engineering to business and computing, James went from hating his subjects and just passing to getting HDs across the board. He found that he did not have to put in a lot of effort as he enjoyed the subject content especially in practical subjects; his computing marks were always better than his theory marks as he found them more relevant.

James completed an industrial experience subject in the final year of his computing degree. He recalls being highly unmotivated from the start of it as he had a heavy workload and he was not happy with his group; he found himself allotted the administrative and documentation tasks rather than the hands-on work which was his preferred role. However, he did prefer to study in groups rather than on his own, as he is people person and finds group study more motivating than working alone.

Toward the last 6 months of his studies, James felt very worn out because of his workload both at university and in his part-time job. He missed quite a few classes but felt this was okay if good notes were available or if the lecturers were boring. He failed one subject in his computing degree in the first semester of his final year – “I just didn’t work and thought I would make it through”. He recalls not wanting to finish in some ways because he liked the university lifestyle and socialising.

James was very social at university. He joined his university’s Accounting and Computing Society in first year and went on to become a committee member. He sees his involvement in the Society as playing a major role in his career planning as the Society had corporate sponsors who held recruitment events and offered vacation employment. He is still active in the Society, attending social and networking functions.

251

Employment history and career aspirations James has been working since he was fifteen. While at university, he worked on average 20-25 hours per week for a fast food company. In his 6.5 years with this company, he advanced from a junior role to area manager. In his penultimate year at university, James worked for three months in an on-site technical support role for an IT company.

He secured vacation employment with his present employer, a large consulting firm, in the Summer of 2000 in the area of accounting. He decided then that he preferred computing and applied for a graduate IT position through the firm’s usual recruitment process. James prefers IT because it is a very dynamic area, always changing.

James works for one of Australia’s ‘big five’ consulting firms. He was first exposed to these firms through the Society and learnt that the work and pay were the same across the companies – what mattered was whether you ‘fitted in’. He found that fitting in was, in fact, more important than university marks, describing his final results as quite poor. He also feels that his extra-curricular activities and consistent work history were influential in his being hired, reasoning that if you have not worked or been involved in social activities, it shows that you are not much of a people person which you need to be as a consultant.

He commenced with the company in March 2001 and was sent to Orlando, FL, for two months training. James enjoys his work very much as it involves variety and leading-edge technology. He has exposure to a range of industries and project roles and is involved a lot of problem solving, for which he has a natural aptitude. “That’s what it’s all about really. Always solving other people’s problems”.

At this stage, James hopes to work up the ranks to management level in his firm. He currently has a two year ‘bonding’ contract and plans to use this time to work hard, gain knowledge and get a feel for where he would like to go in the firm. While in high-school and university, he did not feel that he was working to his real potential, he believes he is doing this now.

James attributes his current employment position to a combination of personal factors and a university education. He feels strongly that if he had earned straight HDs but had not socialised, become involved in the Society and its recruitment activities and developed people skills, he would not have made it through.

Thinking about the value of theory in his computing degree, James says that, while he hated it at the time and thought he wouldn’t remember any of it, it is valued by employers. Although he was trained on the job, he realises that you cannot be trained in new skills if you don’t know the fundamentals. He now realises how much theory he has learnt and retained from his computing degree and has seen other new recruits, who did not possess the same knowledge, struggle during company training.

James believes his degree has made him highly employable but feels that course administrators should do more for students in terms of career planning and advice - “no-one gives you a kick up the bum”.

252

Case 5. Michael

Gender: Male

Age: 27

Course completed: Bachelor of Computing with Honours

Completed bachelor degree in December 1998

Completed Honours year in June 2000

Business size: micro (5 employees) unit attached to larger corporation

Position: Web Developer

Background Michael is a 27-year-old male who lives in a house owned by his parents. Michael grew up in a bayside suburb of Melbourne. He lived there with his mother, father and two younger brothers. Michael is the eldest child. The middle brother is currently completing an undergraduate degree in Science. The youngest brother is still in secondary school. His father, who once owned a general store, has worked in various trades and most recently as a wholesale salesperson, traveling around Australia. Michael’s mother is a nurse and works in a hospital close to the family home. Michael’s family has played a key role in hisfather has suffered with illness over the past several years, leading to a shift in the role that Michael plays in the family and having direct influence on decisions related to his education and career.

Michael appears to be goal oriented, competitive and highly motivated to succeed. He has a strong sense of control over his future. He does not appear to be a big risk-taker; particularly in financial matters, he prefers to research options to minimize risk and optimize security and return.

Schooling At school, Michael excelled in sports. He was known for his commitment and excellence in sport rather than in academic studies. He played competitive team and individual sports, competed in athletics and competed at the state level in martial arts. Many of the adults that influenced Michael during his schooling were sports coaches/teachers and he says they played a mentoring role. When the International Olympic Committee announced that Sydney would host the 2000 Olympic Games, Michael’s martial arts coach thought he had a chance of competing.

Michael found his school work easy and did not have to put in much effort to do well. His most enjoyable subject at school was Information Technology. Michael says he found it interesting and he was good at it. He had always had an interest in technology. As a child he would dismantle his toys and other household gadgets

253

and then reassemble them. Later, he owned a Commodore 64, which he “loved”. Initially, his interest had been in hardware and he thought might pursue this at university and become a technician. However, with the development of the Internet and the World Wide Web, he became more interested in web programming and design. This interest was further stimulated when he later enrolled in and completed a Bachelor of Computing at an institution that had a programming focus. Michael sees this interest and the particular focus of the degree that he completed as playing a role in his achieving his current position as a Web Developer.

University education While at school, Michael was not sure what he wanted to do at university. He had always assumed that he would do a university degree, receiving encouragement from his parents and from his favourite teachers. He decided to do computing because he was interested in this area.

Michael selected his university based on access by public transport from his family home. He selected one campus over another based on the proximity of a railway station. Michael continued to play and compete in sports while at university, but he also brought his need to achieve and to “be the best” to his academic life. Michael was not completely conscientious, sometimes missing classes so he could play sport, but he achieved good marks. He planned to pursue further study and eventually complete a PhD because “that is the highest you can do in education”. In his first and second year he achieved a Credit average. Michael saw Honours as the quickest way to get into a PhD program. To achieve this goal, he applied himself in third year, achieving a High Distinction average. During this year, he also worked as a sessional tutor in his department. He was then invited to do an Honours year.

During Michael’s bachelor degree, his father suffered with ill health. Midway through Michael’s Honours year, his father became extremely ill. At this point, Michael deferred his studies for several months. With his father unable to work, Michael saw a need for another “bread earner” in the family to complement his mother’s income while his younger brothers completed their education. This was a turning point in Michael’s education and career aspirations. His father’s illness not only led to a pragmatic need for income, but it prompted Michael to reassess his direction and what he valued. From that point he has focused on achieving new goals that would facilitate a lifestyle that made family life his first priority.

Michael returned to university later the same year to complete his Honours year, but he then sought full-time employment consistent with his new “plan”.

Reflecting on his university education, Michael says “they pretty much had it all covered”. He completed streams in information systems, programming and computer technology. Subjects in information systems covered documentation and presentation skills, subjects in programming covered programming and software development, and subjects in computer technology covered the technical side. He appreciates the value of an industry experience program in his third year that provided him with an opportunity to apply what he had learnt in previous years.

Thinking of the position he currently holds, Michael thinks he may have benefited from more advanced subjects of direct relevance and that some of the subjects he

254

completed were not relevant to the work he now does. However, while at university he would not have been able to anticipate what type of roles he would have in the future or what type of businesses he would work in. Given this, he believes the degree he completed, with its mix of subjects, was the right one to do. It gave him a grounding in a wide range of computing areas and, therefore, the ability to apply for a wide-range of positions. He believes that the undergraduate level is too early to specialize.

Further to the skills and knowledge Michael acquired at university, he says it also gave him a good social life and a network of lifelong friends. He says he learnt lessons of life, such as, the importance of friendship and that hard work will lead to future rewards. In terms of his working life, he believes his university education gave him are a broader perspective on the IT industry, the knowledge required to stay in tune with this rapidly changing industry and, most importantly, the knowledge that there is more than one way to solve a problem and that even incorrect solutions are a source of learning and progress.

Employment history and career aspirations Michael applied for a number of positions on completion of his Honours year. He was offered two positions in Web development: one in a university and the other in a micro business unit attached to a larger corporation. He was made aware of the latter position by one of his lecturers who then recommended him for the job. Michael chose to take this job because it seemed “more commercial”, being outside of the university environment. Michael is currently working as a Web Developer for this company. He started in this position in June 2000. His primary task is asp programming to maintain his organisation’s Web site. He continues to do some contract work through contacts with his old university.

Michael’s motivation for seeking and taking up his current position was to raise money for the service of a larger plan. Michael says that he became aware of the potential of the Web development industry while at university, through talking with his peers, reading newspapers and browsing the Internet itself, and through contact with friends who had already graduated and taken up positions in this area. Michael plans to work in this industry for the next ten years. Over this period he will save money and purchase real estate, and after this time he plans to retire and to live off his investments. He will complement this income with part-time work, possibly teaching others how to manage their investments. Michael and a friend who studied accounting have put a lot of time into researching various investment avenues and have settled on real estate and shares as the avenues that suits them. They belong to an investment club and have attended seminars on both areas. Michael is currently living quite frugally. He lives in a house owned by his family and he doesn’t run a car. This way he can save most of his income.

Michael enjoys working in his current position. The company is non-hierarchical and he is treated as a professional. The company is currently undergoing some management and structural changes. Michael hopes to continue with the company and will do if he is made a good offer (financially) after the changes have been finalized. Michael has considered starting his own business, but believes “it’s too much of a hassle”. He doesn’t want to take risks that would jeopardize his long-term plan. He has also considered working overseas where incomes are

255

substantially higher than they are locally, but at this stage he does not want to move away from his family.

The motivation behind Michael’s plan for early retirement is family. He expects that in ten years’ time he will be starting his own family, and he “wants to be there”. He says he’d rather “have more time with his While he sometimes still contemplates doing a PhD, he believes that, with the pay increases that he is experiencing in his work, he will continue to work full-time. He may reconsider the option of doing a PhD once he has attained his goal of early retirement.

256

Case 6. Steve

Gender: Male

Age: 28

Qualifications:

Did two years of Chemical Engineering

Almost completed 5-year degree in Software Engineering (currently deferred with one year to go and does not plan to complete)

Business size: small (16 employees)

Position: Software Developer – full-time

Background Steve is a 28-year old male, who lives in the eastern suburbs of Melbourne with his mother, his sister and his sister’s two children. Neither of Steve’s parents are university educated. Steve’s mother left school after grade 5 and has spent most of her working life in the furniture business. At times she has run her own business. Steve’s father completed year 12 and has worked in many different skilled and unskilled positions. Steve has two brothers and two sisters; he is the middle child. His older brother, who completed a Bachelor of Arts, works as an audiovisual technician for a high school. His older sister enrolled in but did not complete a university degree and now works part-time selling produce at a market. Steve’s younger sister, who also enrolled in a university degree without completing it, is currently between jobs. She was most recently working as an administrative assistant. His younger brother did not attend university and works as a labourer/builder

Since childhood, he has had a keen interest in science and mathematics. When he was in Year 9, his father bought the family a computer. Steve treated the new machine like a puzzle. He enjoyed the problem-solving required to get it working. Steve learnt how to use it better than anyone in the family, so he “pretty much took it over”. No one else in his family shared Steve’s interest in science and maths.

Steve is not motivated by career aspirations or money. He likes to pursue his own interests for intrinsic rewards. He is motivated to learn to satisfy his own interests, rather than for career advancement. Steve observes that he has a pattern of “dropping out”. He easily becomes bored and de-motivated with study and work. Over the years, at home, in school, at university and now at work, he has found it difficult to find peers who appreciate science and maths in the way that he does or mentors to inspire him.

Steve is reserved and quite cynical. He says he would rather work with technology than with people. He is attracted to, enjoys and is naturally good at solving logical or mathematical problems. He enjoys the satisfaction of devising elegant solutions

257

peers who could understand and appreciate good work.

Recently, Steve has started learning karate, training three nights a week. His motivation for starting this was twofold: he sees it as an opportunity to improve his fitness and to mix with people. He is not very good at mixing with people socially, so karate is a challenge for him. Because of his pattern of “dropping out”, Steve has set himself the challenge of continuing with karate for four or five years.

Schooling In Year 9, Steve’s favourite subject was Advanced Mathematics. It was in this subject that he had his only school experience of using a computer. He was required to write a simple program and he remembers enjoying it. He did not use computers in an educational context again until university.

Steve respected his Mathematics and Science teachers because they had skills and knowledge he wanted to acquire. He viewed other teachers, for example his English teachers, as impediments to his pursuit of those subjects he enjoyed. Steve says he took every opportunity to avoid attending classes in English, often going to the library to do puzzle books.

For his VCE, Steve took two Maths, Physics, Chemistry and English. He excelled in the maths and science subjects with marks ranging from the high 70s to the high 80s, but received a significantly lower mark for English. Steve felt somewhat isolated at school, seeing himself as more intelligent than his peer group. He was surprised when he started university to find that he was no longer top of the bunch.

University education Steve was attracted to the idea of doing Chemistry at university because he liked the logic of putting a and b together to get c. He enrolled in Chemical Engineering. He says that although he “loved” chemistry, there was too much rote learning required in the course, so he became bored and began to “drop out”. During this course, he rediscovered computers and began to move towards electrical engineering. He says he enjoyed the programming, but was not interested in the hardware.

Steve withdrew from the Engineering course and spent several years driving a van for his mother’s furniture business. Four years later, when he became bored with this work, he decided to return to university and pursue his interest in programming. He enrolled in a five-year Software Engineering degree. After two years of working diligently in this course he began to lose interest again. He says that the course was moving away from programming and towards project management and “paperwork”. Steve commented that the non-programming components were “too much like English and Humanities and things I don’t want to touch.” At this time, he also learned that the advanced mathematical modeling subjects that had been a key factor in Steve’s selection of this particular course had been removed from the course.

During the fourth year of the software engineering course, Steve was offered some programming work through a relative. He accepted the work and deferred his

258

studies because he was no longer enjoying his course. Steve did not defer this course because he particularly wanted to work. Nor was his enrolment in the software engineering course motivated by a desire to make a career in this area. He says he just wanted to learn and then to apply that that knowledge to pursue his own interests.

Steve has since taken up a full-time position as a software developer in a small business. He says that it seems more and more unlikely that he will complete the software engineering degree. He thinks he would pick up more useful skills by studying at home and then putting them into practice at work. Only one of the programmers at his work is formally trained. Steve believes that self taught people make better programmers because they naturally enjoy programming, but that they are limited within a team environment because they have not learnt about proper programming styles, documentation and how to work with other people, so integration is a problem.

Reflecting on his software engineering course, Steve identified three main areas of learning that were intended to prepare him for work:

• networks and programming

• project management and documentation

• group work

Steve says he learnt a lot about networks and programming and that this was both interesting and useful.

He learnt a bit about project management and documentation, but he did not apply himself to these areas while studying because they did not interest him. Nor does he apply what he learnt in his work. He knows he should, but he avoids the extra workload it would involve because he would not enjoy the work. He has not fully documented the work he has done in his current job.

Steve’s software engineering course involved a lot of group work. He did not enjoy this, recalling that it led to problems with disorganization and delegation of work: “Everyone wants to do the programming and no one wants to do the

Steve does not think his university studies prepared him very well for work, but nor does he think that this is necessarily the role of university education. He believes that the best place to develop the skills required for employment is the workplace itself; university provides a loose framework on which to build further skills or flesh out the framework. In terms of employment, Steve sees his studies as “just keeping me from working for a couple more years”.

Although Steve is highly motivated to pursue his interests in mathematics and programming, he would not consider doing a research degree because of the pressure that he would be under to conform to the expectations of others. He thinks the formal structure provided by such a degree is unnecessary for him to satisfy his own needs; he would rather pursue his interests in his own time.

Employment history and career aspirations In 2000, part-way through his software engineering course, Steve took his first computing job. This job involved making dynamic web pages and working with databases. Steve taught himself the programming language required for this work.

259

This job came to an end a year later when the business started to fail and the work was no longer available.

Early in 2001 a friend put Steve’s name forward for a job with his current employer. Steve was interviewed. His employer gave Steve a logic puzzle that he partially solved at the interview and then finished off that night. The employer was impressed and gave Steve a job as a Software Developer.

Reflecting on his work in this position, Steve says he enjoyed the first three or four months because he had to solve problems. He then had a period, involving a lot of paperwork that he did not enjoy. He is now moving onto another area that involves a language with which he is not familiar. He says this was “a bit scary” at first because he is expected to do a lot more than he is qualified to do, but it turned out he was worrying unnecessarily).

Steve does not have strong career aspirations. He is not interested in moving into management, nor does he believe he is committed or intelligent enough to achieve beyond a certain level in programming. He says he would not work as a contractor or start his own business because it would be too stressful. He likes to solve problems and would be happy in a job where he could turn up in the morning, do his work, and then go home. He is not willing to put in the work that he believes would be required to make a lot of money and he is content making enough to live on. He believes he will stay in his current position until he gets bored with it.

260

Case 7. Tim

Pseudonym: Tim

Gender: Male

Age: 22

Qualifications: Completed two years of B.Bus & Info. Systems (currently deferred and does not plan to complete)

Business size: large (100+)

Position: Sales Executive/Corporate Sales

Background Tim, a 22-year-old male, completed the first two years of a Bachelor of Business and Information Systems from which he has deferred and does not intended completing. Tim’s mother is involved in home duties and his father, who has a background in sales and marketing, established his own software development company when Tim was young. Tim’s sister is a receptionist and his brother in completing Year 10. He describes his sister as not being technically minded while his brother has an interest in technology but not as strong as Tim’s at his age.

Because of his father’s business, Tim has been around computers since about the age of 8. He recalls always having a curiosity for technology and as a child, pulling things apart to see how they worked. He first began experimenting with programming at the age of 12. His interest in programming shifted to an interest in how computers could best be used to make money, “how to make them work”. In addition to his father, Tim’s interest in computing was influenced by his high-school friends who were also interested in “geek stuff”.

Tim’s father also influenced his interest in business. He recalls discussing business decisions and processes with his father in his youth and sees his father as something of a business mentor.

Tim’s interests outside computing and business include tennis, running and weights. He and his girlfriend, a PE teacher, socialise often and Tim has an extensive network of friends and acquaintances in the ICT industry.

Schooling Tim describes his primary and secondary schooling as uneventful. In high school, he didn’t push himself too hard and “cruised through”. He took IT subjects from Year 8 onward and in Year 12 took Information Systems, which he enjoyed because he was naturally good at it. In year 12 he also took English, Maths Methods, Economics and Business management and obtained a TER of 73.

261

University education

the first two years of the course before becoming bored; he wasn’t interested in the technical subjects and feels that it was “probably the wrong thing for me as a person”. Although the course was very balanced between business and IT subjects, Tim’s professional interests had moved more toward sales and marketing. He wanted to earn money more than study and began to miss classes in order to do paid work.

Tim hopes to return to university to study economics in the next two to three years but does not think he will study full-time again as he needs to be involved in hands-on work to keep his interest up. In thinking about the value of his two years of study, he feels that he did learn things but doesn’t refer to them often.

Employment history and career aspirations Tim has been working for his current employer, a telecommunications company, since he was in Year 12. His first position was as a part-time installations officer and then he moved into sales and technical support in the company’s call centre. Eighteen months ago, Tim deferred from university to take up a full-time position with the company. Originally, he had planned to defer from his course for one year but was offered a higher paying position and decided to stay on. He is currently employed as a sales executive and is responsible for building the company’s corporate client base. He also does some pre-sales technical work preferring not to rely on pre-sales engineers where he can build solutions himself.

Although Tim would like to progress in his current company, he and two colleagues are in the process of creating a ‘side-business’ offering remote network support. While it is still in the planning stage, Tim expects the business to be up and running within months and possibly supporting himself and his two business partners within 12 months. The business is being planned in such a way that it will grow slowly allowing the partners to continue in their current jobs until its client base is large enough to support them. Tim describes himself and his business partners as ambitious and business-smart and he considers the business venture to be low risk in that the start-up costs are small. The main risk associated with the business is leaving his current job to work full-time in the new business and the new business failing. However, because of his network of contacts in the ICT industry, he believes he would have no trouble finding work again.

In addition to his father, Tim credits two previous industry figures as having influenced where he is today. The first was the owner of a hospitality company in the US for which Tim worked during a 3-month overseas holiday. This employer was very “switched on” and hard working and taught Tim about motivating people. The second was the manager who offered him his current position. A contractor from London, he was appointed to get a new division of the company up and running and his business and sales talents impressed Tim.

Thinking about the value of university education in the ICT industry, Tim noted that life experiences during your education are helpful. He sees university degrees as helpful but not regarded as highly as hard work and making a name for yourself through your performance. Having a degree can help you get a job but it is up to

262

the individual after that. Tim feels that obtaining a university degree is not the best way to go as courses provided by software and hardware vendors are more up to date and more highly valued - “switched on smart guys may not have degrees but they have industry certificates which will take them further”. However he sees university as teaching you how to acquire new knowledge and this is a bonus.

Tim is personally motivated in the ICT industry by money but a sense of satisfaction and achievement is also very important to him and he is happy with how he has done so far.

263

Case 8. Rita

Gender: Female

Age: 21

Qualifications: Bachelor of Computing

Business size: Large (100+)

Position: Analyst

Background Rita is a 21-year-old female who graduated from a Bachelor of Computing degree at a Melbourne metropolitan university. Since April 2001, she has worked as an analyst in the market unit of financial services for a ‘big five’ consulting firm.

Born in New Zealand, Rita came to Australia with her mother, her partner and her half-sister at age 10. Rita’s parents divorced when she was three years old and she is the only child from this relationship. Rita’s mother, an aged-care nurse, and her partner, a pharmaceutical engineer, have a 15-year-old daughter. Rita’s father, a car importer in New Zealand, and his partner have two sons aged 11 and 8 years.

Rita lives out of home, having left her mother’s home in the outer Southern suburbs to attend university in Melbourne.

Rita sees her dad as having influenced her career choice; he has always had an interest in IT and has the latest computing equipment in his company. His sons, Rita’s young half brothers, are also very computer literate. Rita sees her mum as having ‘no influence at all’ on her career. Like herself, Rita’s sister’s interest in computing is as a tool for completing schoolwork. In her spare time, Rita teaches highland dancing.

Schooling Rita had no exposure to computers during her primary schooling. From Year 7, she attended a Catholic high school in the outer Southern suburbs of Melbourne where she completed compulsory IT subjects involving general computing skills from Year 7 to 10. In Year 9, she got her first home PC which she used for surfing the web, for homework and as a research tool during Year 12.

In Year 12, Rita studied Information Technology along with English, International Studies, History, Economics and Literature. Rita was a little disappointed with her TER of 75.6 as she had expected to do better and the score was below the cut-off for her preferred course, Marketing. She consulted her school’s careers advisor and decided to enrol in a Bachelor of Computing degree. This decision was based on a few factors. Rita was interested in teaching at high school, but at the time, many teachers were being retrenched. She also enjoyed studying history and literature but did not see an Arts degree as leading to employment. As her

264

best performance in Year 12 was in Information Technology; computing seemed like a practical choice.

University education Rita was happy with her decision to enrol in a computing degree however “programming came as a big shock” as her high-school information technology subjects dealt mainly with commercial word-processing, database and spreadsheet products. She completed her degree in three years recording only one fail, a second year accounting-oriented project management subject. As it turned out, this subject, in which MS Project ™ was taught, is the one which has most relevance to her current position.

In first year, Rita became involved with her university’s information systems student society, starting out as a member and rising to the position of vice-president in third year. ‘Big five’ companies became sponsors of the society and conducted recruitment activities at society social functions. In the second year of her degree, she began exploring graduate employment opportunities, focusing on the ‘big five’ companies. Rita believes that her involvement in the society, and subsequent contact with ‘big five’ sponsors, influenced her employment prospects in two ways: by allowing her to familiarise herself with the companies’ recruitment practices and as a demonstration of her involvement and leadership in extra-curricular activities. Socialising with other similarly motivated society members also helped Rita crystallise her career goals.

In the final year of her degree, Rita completed a year-long industrial experience project which she regards as one of the more positive things about her course because it involved dealing with an actual client. She has heard that some universities offer six month paid industry placements and thinks this arrangement would have been more useful because it would involve working with an industry-based manager, rather than a university-based project adviser.

Employment history and career aspirations In April of this year, Rita was employed as an analyst within the financial services division of a ‘big five’ consulting firm. After attending many interviews, Rita chose her current employer over other offers because the company, and its management staff, demonstrated the most interest in their prospective employees in terms of offering time and information.

Thinking about the factors that influenced Rita’s recruitment, she stated that recruiters were interested in “well-rounded” graduates who were involved in extra-curricular activities during their degrees including paid work and student-professional societies. Rita noted that involvement in well-run student-professional societies was regarded particularly highly by industry. Rita also stated that she attended so many interviews that she became adept at the process: “I went to so many interviews I got used to the questions, so I was prepared.” She sees the interview process as favouring graduates with more open personalities and disadvantaging the quieter candidates. “Some people didn’t deal with it well . . . quiet students did not manage the recruitment process well.”

265

On commencing employment, Rita completed two weeks of local team-oriented training in HTML, Java, JavaScript, VB and asp. Shortly after, she completed 2.5 weeks of similar, although more in-depth, training in Chicago. Rita is currently involved in a two-year graduate program after which she hopes to become a consultant. She is not sure what role she would like in the longer term but sees that opportunities will open as her skills develop.

Thinking about the relevance of her degree to her current employment, Rita believes that “hands on is more useful than the theory theory is no good if it is not used in a hands-on way. … A lot of theory, you learn once and then it goes out the door”. Asked whether employers value theory, Rita stated that you need to have a basic understanding, and employers value theory, however employers also have their own practices and standards that may make theory learnt in university redundant.

In terms of practical skills, Rita found the programming languages taught in her course, Eiffel, “ a complete waste of time no-one in industry knows what it is.” Rita believes that universities should be teaching the languages and skills demanded by industry such as CRM™, Java™, asp™ and MS Project™.

When asked what universities could be doing for students, Rita stated that computing courses should include more industry knowledge rather than theory, including inviting industry professionals to speak to students. She also feels that students should be given more help in preparing for the recruitment processes, multiple interviews and presentation skills.

Rita is motivated to continue working in the ICT industry because she sees it as a “very stable, strong industry – always going to be around”. She also sees the industry as one in which you can “earn and learn a lot” although money is not that important to her: “it’s more important to me that I fehappy with the career choices she has made: “ If I stop enjoying it, I’ll do

266

Case 9. Brendan

Gender: Male

Age: 24

Course completed:

Bachelor of Computing with a double major in software engineering and information technology

Did two years of a Bachelor of Digital Systems before transferring

Business size: large

Position: Programmer

Background Brendan is a 24-year old male who lives and grew up in the South Eastern suburbs of Melbourne. His father is an electrical engineer who lectures at the University that Brendan later attended. His mother, a trained teacher, works in social work. Brendan is the youngest of three children, having two sisters: one is a lawyer; the other studied psychology at university, but now works in office administration.

Through his father’s work, Brendan was exposed to electronic technology from a very young age. When Brendan was seven, his family bought their first home computer – an Atari ST. Brendan taught himself to program on this computer, using an assembler language. He later became interested in Electronic Bulletin Boards, which he used as a resource for further learning about computers and networking. Brendan met others who were interested in computers through the Bulletin Boards (“Bulletin Board friends”), and at one stage he ran his own BBS.

Brendan says he was not very interested in school and he was distracted by family problems, and that his grades reflected this. He says that technology was his escape from his family problems. He also developed an interest in science fiction. Brendan’s parents divorced after he had finished his secondary schooling. His mother now lives in Sweden.

Brendan has a philosophical approach to life. He is outgoing and he exhibits leadership qualities. He says that when he was younger, he had aspired to be “rich and powerful”, but he has “mellowed with age” and now aspires to be happy and to have an easier and more relaxed life than he had earlier imagined he would want.

Schooling Brendan attended a public primary school that did not have computers for student use. He had an interest in science that was encouraged by a teacher that he liked. He remembers that this teachers’ husband worked in some sort of science trade and would sometimes give fun demonstrations.

267

He then attended a private school that was well funded and had computer laboratories for student use. From Year 7 he did computing subjects, mostly using LOGO and PrintShopPro. He remembers in Year 10, the school up-graded its Apple 2e laboratories, replacing the machines with IBMs. It was at this point that Brendan became interested in networking and how the computers worked.

Brendan studied Japanese at school for six years. He chose this language because he had plans to be “rich and powerful” and it was commonly believed that Japanese would be the future language of business in this region of the world. In his senior years, he studied Mathematics, Physics and Chemistry, and English which was compulsory but which he did not like. Instead of VCE, Brendan did the International Baccalaureate. As part of this diploma, Brendan also studied a subject on the theory of knowledge. Brendan liked this subject because it opened his mind to different ways of thinking. Brendan appreciates activities that will broaden his knowledge and skills, even if they do not come naturally to him. For example, he says he hates public speaking, but he takes every opportunity to practice it because “it is a good skill to have”.

University education Although he didn’t consciously plan to follow in his father’s footsteps, Brendan believes that his interest in electronics and digital systems was influenced by exposure to his father’s work and he sought entrance to a university course that would suit this interest. Brendan was in the second year of students completing the International Baccalaureate in Australia and there was no system for translating it his marks into a VCE-equivalent university entrance score. With assistance from his school principal who wrote letters to various Melbourne-based universities, Brendan organised a special entrance into a Bachelor of Digital Systems at the university of his choice.

He left this course after two years because he found the Mathematics too difficult, transferring to a Bachelor of Computing in the same university. This course moved Brendan’s interest away from electronic and towards programming.

Reflecting on his computing course, Brendan says he appreciated his exposure to the different sides of computing. Prior to his computing course, Brendan already knew most of the programming covered through his own self-learning, but he says that this course gave him a formal structure to “bind his knowledge together”, giving him an overall view of how different parts of computing relate to each other, for example, how the programming relates to the system and how the system relates to the goal and so on.

The course also introduced Brendan to new areas like project management. He says he did not enjoy the subjects in project management and documentation and he failed a project management subject. Although he was not interested this area of computing, he realized that it involved important knowledge and skills, and that it would add to his “well-roundedness”. He sees documentation and testing as necessary evils; he appreciates why they are necessary, but he finds them boring.

Brendan’s computing course involved group work in every subject. He likes working in groups if you can rely on the other people. He says it is easier doing the work by yourself, but he knows that he will always have to do group work because the whole world is about human interaction and communication. Brendan

268

believes that communication skills are the most important thing he learnt at university.

The course also included an industrial experience program that involved a project conducted over two semesters. Brendan’s group developed a distributed note-taking system. He was the project leader because he is a strong personality in a group. During this project, he learnt about the importance of group dynamics and conflict management, for example, “When to put your foot down and when to be

When asked if his course was lacking anything that may have been valuable, Brendan said he would have been interested in doing some low level programming, for example, looking at data structures for example, to suit his own interests. In retrospect, he thinks he should have done a Computer Science course, but he would not have had the marks to get in. He also realizes that if he had done Computer Science, he would not have got the general overview and the project management skills that he now appreciates.

Employment history and career aspirations During his undergraduate studies, Brendan worked as a sales person at Dick Smith Electronics for 3 year. He later became the computer technician for the "Power House" stores. He then moved on to become a Systems Adminsitrator / System Integrator / Programmer for a small financial company that wrote software to help banks and other financial institutions "better utilise their client base".

On the completion of his Bachelor of Computing, Brendan gained his current position as a Programmer in his university department. He also supervises industrial experience projects for the course that he completed.

Brendan is interested finding a career that can combine research and programming. Through a friend of his mother, he is in contact with a robotics department in a Swedish university that has offered him a position as a PhD candidate. Brendan plans to take up this opportunity once he and his partner have paid of a bank loan (possibly in 2003).

Brendan sees himself working academia in the future and likes the idea of doing research and some teaching.

269

Case 10. Shona

Gender: Female

Age: 27

Qualifications: Completing Bachelor of Computing specialising in Object-oriented programming

Business size: Large (100+)

Position: Information Analyst/C++ Programmer.

Background Shona is a 27-year-old female completing the third year of a Bachelor of Computing degree part-time at a Melbourne metropolitan university. She currently works as an Information Analyst/C++ Programmer for a consultancy company, contracted to a large telecommunications company.

Shona is the eldest of four children, having two younger sisters and a younger brother. One sister has a diploma in childcare and the other works in retail but intends returning to university to complete a teaching degree. Her younger brother has a diploma in social law and has played professional football. Shona’s parents are both in technical positions; her father works in polymer and AutoCAD engineering and her mother is employed by hospital and is involved in the sterilisation of theatre equipment.

As a teenager, Shona enjoyed swimming and running and for the last 6 years has been keen scuba diver. She also enjoys reading up on, and playing with, new computing technologies and methodologies but does not program in her own time.

Schooling Shona’s first access to a computer was in Year 7 at which time she undertook a very basic programming subject. She did not study computing again until Year 11 during which she completed an information technology subject “that turned out to just be typing.” In Year 12, her parents bought a computer that she used in her studies. In her VCE year, Shona studied Marketing, Economics, Law, English, Maths, Religious Education and Physical Education – “all the subjects that weren’t me” – obtaining a TER of 260. Information technology was not offered at Year 12 level at Shona’s school.

TAFE and University education In 1992, after VCE, Shona completed a Diploma of Marketing but feels it was “a waste of time” as she was neither suited to, nor interested in, a career in marketing – “I finished high-school; got to do something. I’ll do this.”

270

In the following year, Shona became involved in computing, first as an interest. She began to “tinker” with computers and explore operating systems basic, but more advanced than looking at the interface.” In 1995, she completed a Diploma in Computing which led immediately to her first computing position. In 1999, Shona enrolled, part-time, in a Bachelor of Computing degree as she wanted to continue her learning and improve her employment prospects – “I knew that was what I wanted to do”. She received exemptions for work completed during her diploma studies and is currently completing her final year. She plans to complete her Honours year in computing and, eventually, her PhD – “I know exactly what I want to do.”

Comparing her current experience of education to her time in high-school, Shona stated that the two experiences are vastly different in terms of her level of attention to, and pride in her work. She used to be happy with C’s and D’s but is now “a straight HD student.” When asked what her education means to her, Shona stated that “if you say it’s just a piece of paper, then I guess it’s not important to me. I just enjoy what I’m doing there’s nothing more to it.”

Employment history and career aspirations At age 16, Shona had her first paying job in a department store. Shona did not work in the year following her Diploma of Marketing, but obtained immediate employment after completing her Diploma of Computing as a database administrator in the areas of payroll and human resources with a directory company. She remained in this position for one year before using her domain expertise to secure a systems analyst/programmer position for a payroll company. She remained in this role for 2.5 years. Keen for a new challenge, Shona took on a web development position for 6 months with an insurance company before joining her current company as an information analyst/C++ programmer. Shona’s current employer is supportive of her education in terms of time release and HECS allowances.

Shona’s long-term career goal is to become an academic, teaching and researching in the areas of distributed computing, OO and re-useable software. Academia appeals to her as it offers a level of intellectual challenge “not presented too [her]

e has been offered tutoring work at her university but has declined due to work commitments. Shona expects to leave industry to join academia in the next 2-3 years. She describes herself as “not driven by money money means nothing if your not happy or being challenged”.

Reflecting on her course, Shona noted a number of areas where she believes universities can improve and increase the relevance of course content to industry. In particular, she feels that there is a great deal of duplication in her degree due to individual lecturers and departments not communicating with one another – “it’s like the departments don’t talk”. She feels subjects are poorly integrated and recalls being taught similar material at each year level. While she recognises that “re-iteration and concreting” is important, she feels this occurs too often, especially in relation to basic OO principles.

Shona also has concerns regarding the degree of redundancy in both course content and hardware referenced. In contrast, she feels that some content lacks industry relevance because it is emerging and not yet applicable in industry. Furthermore, she feels that some content, which is fundamental and will be highly

271

applicable in industry, is taught too late in the degree to be of use to the new graduate.

Thinking about the usefulness of a university education in the ICT industry, Shona noted that often students come out of degrees without having developed a systematic way of thinking and approaching tasks. She believes that this should be addressed by universities – students “need to learn to problem solve; [they are ] hand held a bit.”

272

Case 11. Mark

Gender: Male

Age: 26

Qualifications: Completed all but one subject of a Bachelor of Electrical & Computer Engineering (may complete at a later date)

Business size: Micro (<5)

Position: Consultant – Online Business Intelligence

Background Mark is a 26-year-old male who withdrew from a Bachelor of Electrical & Computer Engineering degree at a Melbourne metropolitan university after failing to complete his final assessment piece. He currently works as a consultant in online business intelligence for a large telecommunications company. He is the eldest of five children and has one younger sister and three younger brothers. He and his siblings grew up in country Victoria on their parents’ dairy farm.

Mark describes his parents as being very business-minded, having run their own books for many years using accounting software. As dairy farmers, they are “very

.

Mark’s parents always wanted their children “to go away and do something before coming back to the farm.” Mark’s sister is a sports journalist, one brother is studying geology at university and a second is a high school. Mark’s third brother “barely finished Year 12” and is working on the farm. This is of some concern to Mark as he feels his brother should gain experience off the farm - “there’s more to life than farming.”

When not working, Mark is interested in politics and current affairs and follows this in the media, particularly talk-back radio. He used to ride a bike a lot and would like to get back to it. Of late, Mark has become more interested in his health and has recently stopped smoking cigarettes and marijuana. Mark stated that he had smoked marijuana quite heavily for some years “because of lack of confidence and as a way to avoid personal issues – [it was] a really easy way to escape from it all.” He also describes himself as quite obsessive “which is why

d one reason why I smoked so much.”

Mark describes himself as being lazy yet highly motivated to become financially independent. “I’m a really lazy person . . . and I’ll always be a lazy person. That’s why I want to be successful – so I can be more lazy and do the things I enjoy.”

Schooling Good marks came easily to Mark in high school. He had a natural talent for maths, physics and chemistry and received the top VCE result for his school, 148

273

out of a possible TER of 160. Socially, high school was not a happy time for Mark. He made few friends and was the subject of bullying by his peers. He described his school as not having a lot of “smart people” and although it was a “great school”, there was no incentive for him to do more work than he had to - “I’m a bit lazy and only do what I have to do.”

University education Mark recalls not being career-focused at the end of Year 12 and enrolled in a Bachelor of Chemical Engineering/Science double degree because chemistry “fascinated” him. A fellow-student from high school who he regarded as a role model and had enrolled in Chemical Engineering at the same university the year before also influenced his decision. In his first year at university, he lived in college and then moved into a share house with friends in his second year.

After his first semester at university, Mark decided that, while fun, chemistry involved more rote learning than he preferred. Preferring formulae and rules over rote learning, he transferred into an Electrical and Computer Engineering degree at the same university.

Mark recalls having known nothing about computers prior to his first year at university when he was first introduced to the game Doom. While he had used his family’s home PC for homework during high school, with Doom he was forced to learn DOS. He did not know how to load the game and his friends, tired of loading it for him, told him he would have to learn to load the program himself. He quickly learnt DOS commands, worked through problems using help files and was soon loading the program for others. Computing came easily to Mark. He attributes this in part to his familiarity with the keyboard as he had learnt to type in secondary school - it was “one of the best things I ever learnt.”

He recalls his third year at university as being a “bad year”; he failed several subjects and started smoking a lot of marijuana. Marijuana “took up a lot of [his] time” and he sees it as contributing to his not finishing his degree wasn’t a good enough reason to stop at the time”. He recalls that there were few supports to turn to at university to help him stop smoking marijuana. He describes this time as being a “very challenging period of my life” and believes a number of other students were in a similar situation.

In his final year at university, Mark teamed up with a friend, who had failed several subjects previously, to undertake a year-long theoretical project. The pair failed to complete the project on time and, while they were given an extension, did not take advantage of the opportunity to complete the subject and their degrees. Mark feels he failed because he and his partner hated the project, had conflicting views on how to proceed and spent a lot of time smoking marijuana. Mark stopped working on the assignment once he began paid employment.

Overall, Mark remembers university as being a lot of fun but a “wasted opportunity”. When asked if he would consider returning to university to complete his degree, he stated that if asked 9 months ago he would have said “No”, but in the last month, has decided that he will take any opportunity to get the subject done including “sucking up to the lecturer and explaining what happened” despite feeling somewhat “ashamed and embarrassed at the way I approached the assignment.” Although he does not feel that completing the degree will be of benefit in the short-term, he believes it will give him more

274

security and make him more competitive in the workplace in the medium-term. Specifically, he expects that it will allow him to attain “a level of remuneration that is adequate for me to support my investment strategies . . . I’ve got big goals in the long-term.”

Employment history and career aspirations While at university, Mark worked “as little as possible”. As his parents had put money aside for their children's education from the time they were born, they were able to support Mark through university. He did not undertake paid employment for the first three years of his degree but, as his degree stretched on, found it necessary to take on a casual kitchen-hand position to supplement his allowance. “I never focused on earning more money than I needed to live on . . . which is a bit different to what my goals are now”.

In the early 2000, while still working on his final university project, Mark was offered a two week programming assignment with a service provider to a large telecommunication company. On completion of the project, Mark remained working with the service provider as a business intelligence analyst. Mark’s relationship with his employer was very strained and, very recently, he resigned to form his own company in order to provide the same consultation services directly to the telecommunications company.

Mark’s attitude toward money and career shifted in late 2000 when he attended a wealth creation seminar with his father. Mark describes his father as having been “a sceptic, sticks to the fundamentals” when it came to money, before being “blown away” by a wealth creation seminar he attended. Mark attended the seminar on his father’s recommendation and feels the experience changed his perspective on “what he wanted to do with his life and how”.

Mark reported always knowing that he would one day work for himself and be successful but until the seminar, was not sure how; attending them has “clouded what my visions were before I went to the seminar – I’m so much more certain now”. As a youth, Mark had watched how his parents ran their business and feels that his parents taught him to be both alert and knowledgeable about business and politics. He considers himself to be an entrepreneur now but is not sure that he always was –“now I know it can happen, I’m gonna make it happen.”

When asked of his career plans for the future, Mark stated that he plans to become “unashamedly filthy rich” and become a venture capitalist “throwing money at good ideas”. He sees this plan as benefiting the larger community through job creation and contributing to the economy.

He believes that graduating from his degree will provide security in the interim but he does not feel that he requires a degree to attain his long-term goals – “a degree will make it happen quicker and more securely”. Thinking of himself as a commercial entity, Mark sees tertiary education as just one asset and foresees his engineering training as being “just something that started me off” on the path toward financial independence. He describes himself presently as having more drive and commitment as opposed to actual ideas for wealth creation. He does have “ideas of [his] own that are really quite out there” but for the mean time plans to surround himself with people who are highly intelligent, drawing on novel business ideas and direct investments toward their realisation – “when I set my mind on something, I do it.”

275

Case 12. Matt

Gender: Male

Age: 21

Qualifications: Completing the final year of a Bachelor of Business (Business Information Systems)

Business size: Large (100+)

Position: Systems Engineer

Background Matt is a 21-year-old male studying part-time in the fourth and final year of a Bachelor of Business (Business Information Systems) at a Melbourne metropolitan university. He is employed full-time as a systems engineer for a large networking integration company.

Matt is the youngest of four children and lives with his parents. His father, who is an air-traffic controller by trade, has worked for his local council since retiring five years ago. Matt’s mother, who is a primary school teacher by training, is completing her PhD and currently works as an education officer. Matt’s siblings have graduated from science-based degrees, however, their careers have moved toward business (eg., PR, pharmaceutical sales and banking).

Matt’s first exposure to computing was around age 12 when his parents bought a Commodore PC. He enjoyed playing computer games with friends and taught himself basic commands and programs. It was not until he was in Year 11 that he began to tinker with hardware.

When not working or studying, Matt enjoys spending time with his friends from university and playing basketball.

Schooling Matt described himself as having been an average student in primary school: “nothing too flash back then”. In Year 7, he had some trouble making the transition from primary- to secondary school. He found the amount of class- and home-work a bit daunting but settled down to become a B+ student.

In Year 9, Matt completed an IT subject comprising mainly Microsoft™ applications and key-boarding skills. In Year 10 he completed an IT subject that involved networking content which was “a real eye-opener” for Matt as he “didn’t know you could do that sort of stuff”. He did not undertake any IT subjects in Years 11 and 12 as, at that level, the focus was on programming which was not of much interest to him. A further factor in his decision not to take IT subjects in Years 11 and 12 was that IT subjects did not carry much weight in terms of TER and he wanted to maximise his opportunities for a good score.

276

In Year 12, Matt took two maths subjects, English, physics and chemistry. Although he socialised a lot that year, he recalls Year 12 as being very stressful and was relieved when it was over.

Matt began considering university courses toward the end of Year 11. He recalls that at that time, there was a lot of ‘hype’ surrounding careers it IT; “money grows on trees [for IT professionals] – I wouldn’t mind that”. He was attracted to working with computers on an on–going basis but was also interested in business and wanted to keep his options open. At university open days, Matt discussed his needs with lecturers and found that he was best suited to a Bachelor of Business majoring in Business Information Systems that included a ‘co-op’ or ‘sandwich’ industry year at third year level. Similarly structured courses were offered by two universities, however, he chose the one that claimed to have a very high rate of industry placement for the ‘co-op’ year; the other course did make such claims and appeared to focus more on academic- rather than industry-related aspects of the course.

University education Matt performed very well in the first year of his Bachelor of Business (Business Information Systems). He won a prize for first year academic achievement sponsored by a networking integration company. The prize comprised a monetary award and an interview with the sponsoring company for a ‘co-op’ position in third year. Of the six prize-winners interviewed, Matt was one of two students who were offered a position. At the conclusion of his ‘co-op’ year, Matt was offered his current position as a full-time systems engineer with the same company. He is now completing his final year of university part-time.

Matt recalls the second year of his course as being quite stressful for two reasons. Unsure as to whether he would secure a ‘co-op’ position with the prize sponsor, who interviewed him twice, seven months apart, he attended 6-7 interviews with other potential employers. In addition, he found the second year programming subjects very frustrating and "just wanted to get working and see what was out there”.

Earlier this year, Matt travel to Italy for four weeks on an academic exchange program during which he completed the equivalent of two subjects from his degree. This has reduced the duration of his degree program by six months and he expects to complete his studies in June 2002.

Employment history and career aspirations Matt’s first paid job was at age 13, retrieving golf balls at a golf range. During the first year of his degree he worked night shifts in a supermarket. His next paid position was his ‘co-op’ year, which led to his current position.

Matt’s immediate career plan is to get as much experience as possible and perhaps move into a consulting position although he sees himself better suited to technical rather than sales roles. In terms of personal motivations, Matt would like to enjoy what he does and see how far he can go professionally. While “money growing on trees” is still a motivator, his focus is not on financial gain. He enjoys travelling and sees more travel for himself in the future and possibly working in the UK or

277

USA. Matt is inspired in his career by the success of his siblings and mother “who has ridiculous work ethics . . . always working”.

When asked what universities could be doing to better prepare students for graduate employment, Matt stated that there should be a greater emphasis placed on industry placements - “work experience is the only way of knowing what it’s all

278

Case 13. Jarrod

Gender: Male

Age: 29

Qualifications: Bachelor of Computing

Business size: Large (100+)

Position: Computer Games Producer/Project Manager

Background Jarrod is a 29-year-old male living with his partner in a Melbourne bay-side suburb. He completed a Bachelor of Computing at a Melbourne metropolitan university and currently works as a producer for a multi-national computer games company. Jarrod is the younger of two children to bookshop owning parents. His brother, Rob aged 31, graduated from a Bachelor of Applied Science in Environmental Engineering and currently works with an environmentalist organisation in Papua New Guinea.

Jarrod’s first access to computers was at age 12 when his parents purchased a Commodore VIC 20. This was upgraded to a 64 and, eventually, an Amiga. Jarrod’s initial interest was in playing computer games. As teenagers, he and his friends aspired to careers in programming and, as they became more familiar programming languages, began writing programs and ‘primitive games’ on weekends.

Apart from by school friends, Jarrod was not influenced by others in his interest in computing; his parents and brother, although technology users, are not particularly technically minded.

In addition to occasional dabbling in programming, Jarrod spends his free hours surfing, mountain-biking and reading and writing fiction. He has written a number of screenplays and sees writing as an outlet for work-related frustration; ”when I’m bored and frustrated there, I need another outlet”.

Schooling Jarrod recalls being a “terrible student” in high school, “just kind of lazy”. He socialised with the “nerd group”, a group of friends who were also interested in computing. During high school, Jarrod taught himself programming. There were no computing subjects offered at his high school until Year 12 when he completed a Group 2 HSC computing subject. In Year 12, he also took English, Biology, Geography and Graphics, obtaining an Anderson score around the 255 mark.

279

University education At the end of Year 12, Jarrod planned to pursue a career in programming. In 1990, he entered a Bachelor of Computing degree at a university campus some distance from his home. In the first year of his degree, he began working for a computer game company that was eventually bought out by his current employer.

Jarrod sees himself as an atypical student as he completed his degree over 10 years and was employed as an ICT professional throughout. In first year, he worked part-time while studying full-time. He deferred from his course for a year in 1992 and again in 1995 at which time he travelled to the UK with his partner and worked for a UK computer games company. On returning from overseas, Jarrod recommenced his studies part-time at a campus closer to home and graduated from his degree in 2000.

Jarrod recalls his academic performance at university as being ‘up and down’. In second year he earned a number of fails but by his final year was averaging Distinctions. He attributes his improved performance to “having time to study” as a part-time student: “There’s a limit to what I can concentrate on at the same time … [it was] the difference between taking four or five subjects and taking three subjects.” Earlier on in his degree, he was juggling full-time study and part-time employment and travelling over two hours a day to attend university. He also sees his earlier performance as being affected by his not having a PC at home at that time: “The course really required access [to a PC] … If you wanted to do any work, you had to go into uni.” He sees the course as being more straightforward for current students as they typically have home PCs and are able to take advantage of ftp.

In the course of his studies, Jarrod’s interests shifted from programming to project management and information systems: “turns out I’m not really a good programmer.” Moving away from programming subjects, he focused on management and information systems subjects including information security, organisational issues and project leadership for information systems and interface design. As he was employed in the ICT industry full-time in his final year, he was exempted from a core industry experience subject.

Thinking about the value of the subjects offered in his degree, Jarrod feels that the earlier programming languages taught – COBOL and PCScheme – constituted “bad programming.” These languages were obsolete, having little relevance in industry. Later in his degree, C was taught and he feels that perhaps it should have been introduced earlier to give students a solid grounding in the language.

In regard to structure, Jarrod feels the course was too rigid in that all students were required to take subjects in three core streams (computing technology, information systems and software) at each year level. He is “a fan of choice for people” and feels that course curricula should be flexible enough to allow students to “direct their degree and subjects on their own. You can’t tailor the degree to what each person needs so not everyone gets the best result out of it. You may not be good at, or interested in, some core subjects.”

Employment history and career aspirations Jarrod’s first paying job was making deliveries for a chemist at age 12. Throughout his teenage years, he worked casually, first at a supermarket and then in his parents’ bookshops.

280

Jarrod entered the computer games industry while in his first year at university. He had read about a local company and “walked in off the street and asked for a job.” He met with the owner of the company, told him that he was an avid computer game player and that this was the industry he wanted to get into. He was offered a part-time position in quality assurance as a play-tester, checking for bugs and identifying game-play issues, a role requiring an analytic approach and attention to detail. After a period in this role, he moved into game design.

In 1995, Jarrod and his partner travelled to the UK where he again worked in the quality assurance division of a UK computer games company. On his return to Australia, he was offered an associate producer position with his former employer. Three years ago he was promoted to full producer/project manager and he currently manages teams of 10-20 ICT professionals, depending on the project. As producer, he oversees game design, art and programming, in addition to his administrative duties.

Thinking about where he would like to take his career, Jarrod stated that there are few opportunities in Australia. His company is the largest computer games company in the country and there are only four or five people senior to him. He and his partner have considered moving overseas to work if the “right job” came up. However, at present, Jarrod is happy in his current role. Furthermore, his partner is completing her tertiary studies and they have their own flat, so a move overseas at this time would not be as easy as it was previously when they were both still living at home.

For Jarrod, a further disincentive for seeking a new or more senior position is that if you “go any higher … you’re not actively involved in the project.” He enjoys his current position because he is still involved and works directly with his project teams.

When asked whether he feels the Bachelor of Computing adequately prepares graduates for industry, Jarrod stated that it does in terms of programming and analytical skills and that his company frequently hires graduates from the course he completed, and other programs offered by his university, on the basis of these skills. From the perspective of a student and professional already working in the industry, Jarrod feels the course contained some “interesting stuff that supplements what you learn on the job.” Overall he found the course worthwhile although he attributes sixty percent of “what he knows” to his experience in the industry.

Commenting on the relevance of ICT programs to industry, Jarrod noted that it is difficult to make these courses practical because they are taught by academics who “don’t always know what’s going on out there.” He believes that a grounding in the fundamentals is useful “in as much as it gives you a grounding in all sorts of stuff”, but he also feels that too little time is given to specific skills: “A lot of [the time spent on fundamentals] is wasted on people; a small proportion will use all they learned – most will use a small percentage.”