Higher Education 21: 249-261, 1991. �9 1991 Kluwer Academic Publishers. Printed in the Netherlands.

Student failure: disintegrated patterns of study strategies and perceptions of the learning environment

N.J . E N T W l S T L E 1, J. H. F. M E Y E R 2 & H I L A R Y TAIT 1 I Centre for Research on Learning and Instruction, University of Edinburgh, 10, Buccleuch Place, Edinburgh, EH 8 937] Scotland 2 Teaching Methods Unit, University of Cape Town, Rondebosch 7700, South Africa

Abstract. Data describing students' study orientations, in relation to their evaluations of courses and their preferences for different kinds of learning environment, are reanalysed in the light of recent suggestions that failing students perceive their learning context in atypical ways. Factor analysis and unfolding analysis demonstrate that failing students show inter-relationships between study orienta- tions and preferences for learning environments which point to a disintegration of the coherent patterns previously reported in the full achievement range. The implications of such a disintegration of coherent patterns of perceptions are discussed in the light of case studies of individual students.

Introduction

In a previous study, Entwistle and Tait (1990) described relationships between contrasting orientations to studying, evaluations of teaching, and preferences for different learning environments. Orientations to studying bring together approach- es to learning with the forms of motivation that underpin them to describe typical ways in which students tackle their academic work. They found that students who had a meaning orientation, adopting a deep approach to learning supported by intrinsic motivation, also preferred teaching, examinations, tutorials, and courses which previous research has indicated facilitate that approach (Entwistle 1990; Entwistle and Ramsden 1983). In terms of the questionnaire items, they preferred lecturers who showed finks between course material and the real world and showed how they thought about topics. They wanted examinations which allowed them to demonstrate their own thinking and to follow their own lines. They wanted tutors who encouraged group discussions and who interacted with their ideas. Finally, they preferred courses which catered for personal interests and encouraged reading around the subject. This set of preferences coincides closely with recommendations made in the literature for improving the quality of teaching generally (Beard and Hartley 1984; Entwistle and Marton 1989; Marton and Ramsden 1988; Newble and Cannon 1989).

Yet this study also reported that students with a reproducing orientation, who adopted a surface approach to learning sustained by fear of failure or an instrumental motivation, actively preferred an academic environment which would be disparaged by innovators. Again, in terms of the items presented to the students, they wanted lecturers who told them what to put in their notes, and who entertained

250

rather than informed. They rated highly examinations which could be answered solely from their notes and which showed how much effort to put into each part. They preferred tutorial environments in which the tutors just went over the lectures and clarified them. And they wanted courses which indicated exactly which books to read, and where topics were linked directly to examination requirements.

This set of inter-relationships was considered to be coherent, and readily interpretable in terms of the model of the teaching-learning process derived from this area of research (Entwistle 1987), and yet a more recent study by Meyer, Parsons and Dunne (1990a) introduced an intriguing complication into this picture. Meyer has been carrying out a series of studies using a new statistical technique, called unfolding analysis, which allows students' approaches and perceptions to be mapped into a two- or three-dimensional space which captures the commonality of relationships between the contributory scales without the assumptions of linearity of relationship implicit in correlational techniques (Meyer and Muller 1990, a, b). Factor analysis creates a space in terms of the factors found to underlie the variables included in the analysis. Cluster analysis, separately, can take factor scores of individuals and show how these cluster to form typical sets of responses to inventories (Entwistle and Brennan 1971). Unfolding analysis goes beyond these techniques to create a space within which both the positions of scales and of individuals may be co-located, thus showing clearly to which groups of dimensions particular students are most attracted (in terms of their ratings). Within any typical sample of students, there is usually a relatively small number of individuals who show sets of preferences, in their responses to the inventory, which differ markedly from the majority of their peers. When the positions of these students are plotted within the space created by unfolding analysis, they are typically found to lie well away from the cluster of the majority of students (which is usually within the area where the scales themselves are found). The dissimilarity of these students from the main group, and also from each other, has led to them being termed 'outliers', whose individual characteristics cannot adequately be captured or represented in the analysis. In exploring the characteristics of these atypical students, Meyer found that a majority of them were academically weak. He and his colleagues therefore carried out an unfolding analysis of failing students and found that the space created by their ratings represented a total disintegration of the expected patterns of relationships between approaches to studying and perceptions of the learning context (Meyer, Parsons and Dunne 1990a). The analysis showed that, for failing students, the usual linkages between approaches to learning and perceptions of the learning enviornment were rarely found. Instead, apparently random sets of association occurred.

This intriguing finding provoked a re-examination of some of the data previously collected in Edinburgh to explore the inter-relationships between study orientations, evaluations of teaching, and preferences for the various types of teaching, examinations, tutoring, and courses, described above. The rationale for the instrument used in this study has already been described (Entwistle and Tait 1990) and is not repeated here.

251

Method

Methods of measurement

A questionnaire was designed which contained three sections. The first contained 28 items describing study orientations, with seven items within each orientation. These scales were created from items from the existing scales of the Approaches to Studying Inventory (ASI) (Entwistle and Ramsden 1983), chosen to be the best definition of each study orientation in terms of their correlations with the composite orientation scores. The items included mainly items from the defining approach and supportive motivational items. The Cronbach alpha reliability values for these short scales lay between 0.55 and 0.71. The second section contained items asking students to evaluate the course they were attending; these items were condensed on the basis of previous factor analyses (Entwistle et al. 1989) into five two-item scales describing in lectures, appropriate level and good organisation, good explanations and an enthusiastic delivery, too fast a pace linked with too heavy a workload, then more generally, availability of books and quality of handouts, and the approachability of staff together with willingness to provide advice (openness). The third part contained the preferences for contrasting academic environments which again were reduced, in this case to eight two-item scales. These scales were chosen to contrast 'deep' perceptions of lecturers, tutorials, examinations, and courses (aspects of the course normally associated with supporting a deep approach to learning), with the equivalent 'surface' perceptions.

The content areas of the individual items are indicated within Table 1 below.

Sample

The questionnaire was given to a sample of 123 first-year electrical engineering students during a lecture. End of year examination results were subsequently obtained on that course. The sample could then be subdivided into those who had passed the examination (N=80) and those who had failed (N=43).

Results

Internal consistency of the items

In considering the disintegration of relationships between scale scores, found in Meyer's previous analyses, the first possibility to be faced was that there might be a similar disintegration at item level. If that were to be found, it might well imply that the failing students, being alienated from the course, resented the request to fill in the questionnaire, and so filled it in with random, or at least unthinking, responses. Any further analysis of the scale totals could then not be justified. The first step then was to calculate the item/scale-total correlations for the successful and the failing

252

students separately. There were no systematic differences found in this analysis, although the failing students showed a slightly higher range of values (0.73 to 0.35, compared with 0.72 to 0.43). Twenty of the correlations from the successful students were above 0.50, while the equivalent figure was 18 for the failures; the Cronbach alpha values for meaning, reproducing, achieving, and non-academic orientations for achieving and failing students (in brackets) were respectively 0.68 (0.75), 0.61 (0.64), 0.50 (0.54), and 0.72 (0.51). The only substantially lower figure was for the non-academic orientation in which the failing students are more homogeneous as a group.

This analysis indicates that the disintegration reported by Meyer at scale level was not present at item level, indicating a degree of consistency of response by the failing students in responding to the individual items. It was thus appropriate to carry out factor analyses of the two samples.

Factor analyses

Maximum likelihood analyses were computed, and rotated pattern matrices were obtained for three factors with delta set at zero. The three factor solution was chosen as the one most equivalent, among the successful students, to previous analyses, although this accounted for only 43.5% of the variance (50.2% for failing students). Table 1 presents the factor analyses of the two samples. The successful students show the expected pattern of relationships, even more clearly than in the analysis of the whole sample (Entwistle and Tait 1990). The first factor links meaning orientation with those features of an academic environment expected to facilitate a deep approach to learning, while the second factor links the reproducing orientation with surface features. The third factor relates the achieving orientation to positive evaluations of lectures.

Among the failing students, however, the expected pattern does not materialize, with the exception of the factor describing a positive evaluation of teaching. The first two factors represent bizarre and uninterpretable combinations of loadings. The first factor is particularly strange as it is defined in terms of high positive loadings on all four of the orientations, in spite of the fact that two are essentially the converse of the others. The second factor makes more sense in relation to the orientations, showing reproducing associated negatively with meaning, but that is then linked to both deep and surface facets of lectures and examinations. In Meyer's earlier analyses 'disintegration' took the form of unexpected and uninterpretable linkages between approaches to learning and perceptions of the learning environment. If such an effect also existed in the present data set, it would create atypical patterns of loadings. The two factors in Table 1 contain loadings which are precisely what would be expected in correlational terms, were the phenomenon of disintegrated patterns of perceptions to have occurred here too. In particular, the combination within the same factor of both deep and surface approaches or perceptions implies the incoherence found in the previously reported unfolding analyses.

Of course, it may be objected that the samples used in these analyses were too

253

Table 1. Factor analyses of approaches, evaluations, and preferences for successful and failing students

Successful students Failing students

Scale I II III I II III

Study orientations Meaning 49 69 -44 28 Reproducing 66 75 44 Achieving 35 48 56 Non academic 59 57 -63

Evaluations Good level, well organised Pace too fast, heavy workload Good explanations, enthusiastic Books available, handouts good Staff approachable, provide advice

41

Preferences Lectures, relate outside, show thinking 52 Lectures, dictate notes, entertain 37 Exam, own thinking, general questions 45 -31 Exam, from notes, show effort required 54 Tutorials, discussions, show thinking 69 Tutorials, review lecture, chatting 41 Course, own interests, read around 29 Course, defined reading and exam topics 64

69 -27

88

36

57

33 43 53

71

56 56 27

40 46 27 30 40 39 33

65

small for factor analyses. The factor loadings obtained with such sample sizes are admittedly unstable. However, the successful students show exactly the same pattern as in two previous complete samples, while the total absence of an interpretable pattern in two of the factors from the failing students is unlikely to be simply a chance variant. One way of checking on the possibility that the lack of coherence in the factors from failing students is not simply an artefact of the small sample is to carry out an analysis based on quite different statistical principles and for which small samples create no problem - unfolding analysis.

Unfolding analyses

The complete set of successful students was included in the first unfolding analysis (see Meyer and Muller 1990 a, b, for details of the procedure). However, as is common with unfolding analysis, there was still a group of students whose preferences were not well represented in the space created essentially by the main set of students. As such students have perceptions which distort the overall pattern, a clearer picture of the space is created by omitting them. Thus a second unfolding analysis was carried out using 54 of the successful students, and this produced the solution shown in Figure 1. In interpreting the position of scales within the 'space'

Z

1 1 7

0 37

-] 91

-3 45 - ~ 0.97

AO CD

[ ]

TD

$I [ ]

r o

CS

~s

no

Y " ~ - 0 . 2 4 -5.8Q

Orientations to studying MO Meaning Orientation AO Achieving Orientation ro Reproducing Orientation no Non-academic Orientation

S 13

References for different forms of teaching LD 'Deep' ways of lecturing TD 'Deep' tutorial teaching ED 'Deep' types of examinations CD 'Deep' course demands

Is ts es cs

'Surface' ways of lecturing 'Surface' tutorial teaching 'Surface' types of examinations 'Surface' course demands

Fig. 1. 3-D joint space for achievers (n--:54). Note: Overall stress = 0.138.

scales related to a deep approach; ~ ' ~:: ~cales related to a surface approach. i

,caies considered neutral in their overall effect~ ~ ii • ~ individual student 's position in the space. Perceptions of courses OP Openness to students EE Good explanations/enthusiasm SI Level of course right pw Heavy and difficult workload cb Course books helpful and useful

Z 0 ~�84

--O 87

- 2 7 9

S

cb

~r

CO

',v E D

es ,~ ro

255

-4 714~ 4 7

u ~

- 0 3 8

Fig. 2. 3 - D joint space for failures (n=31) . Note: 1. Overal l stress = 0.126; 2. Obs. 13 plotted.

Legend: See Figure 1.

Note: The X, Y, and Z axes and the scale values are not used in interpreting the space presented in the Figures. Attention should be paid, instead, to the relative positioning of the scales within the three dimensions.

created by this analysis it is important to recognize that differences are just as important on the vertical dimension as on the horizontal.

This pattern is closely similar to those obtained with South African students (Meyer, Parsons, and Dunne 1990a), and is readily interpretable. Looking first at

256

the scales which are expected to be related to deep approach (shown as 'hearts'), meaning orientation is close on the vertical dimension to achieving orientation and a preference for features of a course which promote a deep approach. On the horizontal plane, this group of three scales is close to two 'neutral' scales describing evaluations ('squares'), which in the factor analysis formed a separate factor - 'openness' (OP) (indicating the approachability of staff) and the combination of good explanations and enthusiasm (EE). Here, the association with meaning orientation is clear, and intelligible. Another group of scales also lies in this same area - preferences for deep lectures, deep examinations, and deep tutorials, together with an evaluation of lectures as well organised and at the right level (SI). The only scale which seems out of place is surface tutorials (ts), but as this includes elements which might facilitate either approach perhaps this connection is understandable.

All the other scales expected to provoke a surface approach lie in a quite separate area of the space, with reproducing and non-academic orientations being on the extreme edge of the defined space. Related to them are the three surface preferences describing examinations, courses, and lectures. This unfolding analysis thus presents an alternative, and in some ways clearer, patterning of the relationships seen in the factor analyses for the successful students.

It proved much more difficult to find a solution for failing students, due to the disparity of their perceptions. It was necessary to exclude the most atypical outliers before any solution could be obtained. Figure 2 presents the unfolding analysis for 31 of the failing students for whom a satisfactory solution could be obtained. Although the solution was satisfactory in technical terms, the pattern of relation- ships it represents parallels the total disintegration of the pattern of relationships between study strategies and perceptions of the learning environment previously found among failing students in South Africa. Meaning orientation stays close to achieving orientation, but it has no other positive scale (heart) in its vicinity. In other words, these two orientations are associated with one another, but not with any of the other constructs, represented in this space. On the horizontal plane, a strange assembly of scales is found. Reproducing and non-academic orientations are understandably close to preferences for surface lectures, courses, and tutorials, but these failing students apparently have equivalent preferences also for deep lectures and examinations. Preferences for deep tutorials and deep courses lie as close to reproducing as to meaning orientations. This pattern of scales represents a set of functionally related concepts which is expected in theory and found in the perceptions of successful students. The lack of a similarly coherent structure among failing students, for their study strategies and perceptions of the learning environment, led to their patterns being described as 'disintegrated', compared with other students. Thus, unfolding analysis has provided evidence of disintegration similar to that implied by the factor analysis, although it is based on fundamentally different assumptions. Bringing together the two methods of analysis tends to reinforce the conclusion that the uninterpretable associations of scales now found in both analyses of failing students are more than a statistical artefact, and may prove to be an important study pathology on which intervention strategies might focus.

257

Discussion

Factor structures

The most striking feature of the two factor analyses reported in this study is the failure of the analysis of failing students to provide a conceptually interpretable pattern of associations between approaches and perceptions. This disintegration in study strategies and perceptions of the learning environment seems to be a novel finding of considerable importance. One previous study did report the lack of a factor structure in study strategy scales for students of below average ability and among those who made external attributions to account for their level of academic performance (Biggs 1985), but it is the disparity between approaches and perceptions which is of most note in our results. Other analyses have not investigated factors at different ability levels, but there have been repeated recent attempts to verify the factor structure of the full Approaches to Studying Inventory based on factor analysis of either the items or the scales (Meyer and Parsons 1989; Harper and Kember 1989; Richardson 1990; Calder 1989). Before looking further at the present findings, it is worth considering these studies in relation to the original work to establish the scales.

The original development of the scales was carried out on a large national sample and did not rely entirely on factor analysis (Entwistle and Ramsden 1983). The items were written on the basis of the defining features of each construct which had been established from interviews with students. The strong conceptual coherence created in this way is at least as important as the evidence of empirical coherence from factor analyses. Without such conceptual coherence, the scale scores become essentially uninterpretable. The next step in scale construction involved applying alpha factor analysis at item level for each of six separate subject areas. On this basis, items were selected that supported the conceptually defined scales as consistently as possible across each of the subject areas. The total sample size being over two thousand, made these analyses sufficiently stable to justify a confident interpretation of the factors. Reliance on factor analysis alone to validate scales of this type is not justifiable; it is equally important to maintain the conceptual clarity of the groups of items. Indeed it is arguable that once cleary defined scales have been created by conceptual analysis, factor analysis should be used only to confirm or refute those combinations of items, not to create alternative groupings which almost inevitably have little conceptual coherence, at least until further work to clarify their meaning has been done (Marsh 1987).

The recent studies have varied in the extent to which they have confirmed the original findings, but some of the differences may be attributable either to a small sample or a single subject area. Thus, the attempts at rescoring the ASI by both Richardson (1990) and Calder (1989) are open to criticism on one or both of these counts. However, even with a large sample (Harper and Kember 1989), or with a large and varied sample (Meyer and Parsons 1989), there are still variations in the factor analyses of scales. One or other of three main factors may merge, producing a lack of clarity, and it is often found that, beyond three or four factors, any other

258

factors are either defined mainly by a single scale (often a form of motivation) or produce uninterpretable combinations of scales.

As a result of the current study, it is possible to see both the blurring of main factors and the presence of additonal uninterpretable factors as possibly attributable to weak students in the sample. It looks as if any future attempts at verifying the ASI should be based on substantial samples, with separate analyses being conducted for each of several subject areas and for achieving and failing students, before any suggestions are made for reorganising the scales or identifying different second- order factors.

Disintegrated perceptions

This study has confirmed the existence among failing students of substantial incoherence between their study orientations and their perceptions of their learning environment, as indicated by their patterns of preference for different kinds of teaching and courses. This finding represents an empirical manifestation of a phenomenon not previously identified in analyses of either questionnaire or interview data. While the incoherence in responses stands out convincingly from both factor analysis and unfolding analysis, its meaning is still far from clear. As the lack of integration of failing students' perceptions had not been anticipated in designing the present study, it was not possible to explore its meaning further through interviewing failing students. It is possible, however, to attempt to interpret the configuration of the scales within the common joint space created by the responses of the failing students in the unfolding analysis, and also to interpret the positions of individual students within the joint space. In this way, further clarification of the nature of disintegrated perceptions may be obtained.

The first point to note in considering the disintegrated patterns found for failing students (Figure 2) is the usual separation of the four orientations, although meaning and achieving remain in close proximity to one another, indicating that the latter two are still closely associated. The reproducing and non-academic orienta- tions, on the other hand, are quite separate from the meaning and achieving orientations, as well as from each other. These positions are entirely as predicted. But the positions of the preferences for different forms of teaching are not as expected: far from it. For high achieving students the preferences for deep aspects of teaching and courses lie in the same area of space as the meaning and achieving orientations. With failing students, there are no preferences within that area of space. The preferences for deep tutorials and deep courses lie separate from all four orientations, but even more puzzling is the position of preferences for deep lectures and examinations, being both closely linked to the reproducing orientation and in the same area of space as the opposite preferences for surface tutorials, courses and examinations. Looking at the individual items, it appears that the failing students who adopt a reproducing orientation are stating, for example, that they prefer examinations which 'have questions requiring specific detailed answers' and 'can be answered directly from notes', but which also 'give an opportunity to show that I've

259

thought about the courses for myselff and 'have general questions which provide opportunities to follow a number of different lines'. It might be possible to accept failing students disagreeing with all these in a general adverse reaction to all forms of examination, but accepting all of them is uninterpretable from what is currently known.

An examination of the observed responses of the failing students (on which the unfolding analysis was based) revealed interesting individual variations in the nature of the disintegration of perceptions. Some individuals, for example, separate out the meaning and achieving orientations and isolate them from the deep perceptions of the academic context with which they are conceptually related. There are also examples of failing students who show strange, and theoretically incompatible, combinations of preferences. For example, one student produced a strange combination of ratings by combining meaning and reproducing orienta- tions, linked to preferences for both deep and surface aspects of teaching and courses. Another example, shown in Figure 2, indicates a student (+13) whose ratings lie roughly equidistant from all four orientations and also from both deep and surface preferences for all the four aspects of teaching and courses. This student seems to make no discriminations between deep and surface perceptions either within study orientations or preferences.

It is not possible to present a finer grained analysis of these strange manifestations in terms of the familiar constructs of the ASI, as its subscales were aggregated in the short questionnaire used in this study. Such an analysis has, however, been carried out in a study recently carried out in South Africa (Meyer, Parsons, and Dunne 1990b) which has described the perceptions of students showing disintegrated study orchestrations in terms of the more readily interpretable subscales of the ASI.

Analysis of inventory responses still leaves a considerable gap in seeking to interpret the world view of failing students. No hint of this effect has been found in other research, except perhaps in a rather tangential form. In a recent study in New Zealand using the ASI, Calder (1989) also decided to redistribute items on the basis of his factor analysis. Although this procedure has been criticised, one of the factors which did emerge was identified as a 'surface-confused' grouping of items which was associated with students 'appearing disorganised, highly anxious, and being unable to concentrate on their studies' (p. 269), while others within this group 'appeared to be basically deep learners who could not apply that mode of learning appropriately' (p. iii). It could be that this analysis is describing a somewhat similar phenomenon to 'disintegrated' perceptions. Another tangential finding comes from observations made during interviews with academically weak students in electrical engineering and suggests a lack of'connectedness' in their perceptions. A substantial proportion of such students continued to be unduly concerned during their first year in higher education with their home environment and their previous interests (Entwistle et al. 1989). They seemed to lack a commitment to their new academic environment, and an associated confusion with their purposes in studying. However, these interviews were conducted before the idea of disintegrated perceptions had been introduced, and so this possibility was not followed up either in questioning or in the analyses.

The earlier work of Biggs (1985) may also point towards explanations of

260

'disintegration'. He found that less able secondary school pupils, and also those who made external attributions for their academic performance, failed to produce clear factor structures in his learning processes questionnaire, He speculated that this effect might be attributable to their inability to think metacognitively about their own learning. It may be that 'disintegrated' perceptions are also, in part, a product of such a failure to be aware of the consequences of adopting surface approaches, or of a failure to interpret the implicit messages about assessment requirements which are contained in the 'hidden curriculum' to which students are exposed.

Inevitably, the small sample size in the current study leaves the findings open to the criticism made earlier of other recent factor analytic studies of the ASI. The factor analyses reported are certainly likely to be unstable with this sample size, and yet the current findings are presented with some confidence for two reasons. First, there is considerable similarity in the distinction between coherent and incoherent patterns of relationships produced by factor analysis and unfolding analysis in this sample of students. Of course, those effects could be an atypical effect found in that specific sample. However, the fact that the current analyses produce patterns of disintegration which are very similar to those reported on another entirely different sample in South Africa, using a different (although conceptually related) set of scales strongly suggests that the finding is not simply a product of a particular small sample of failing students.

It does look as if unfolding analysis may have highlighted an important facet of the problems of students who find difficulties in studying. Further research will be needed to check on the replicability of the factor structures and the unfolding analysis space of weak students, but it also seems well worthwhile to interview academically weak students to explore what form the disintegration may take in relation to their everyday studying, and how a greater coherence between approach and perception may be achieved through counselling. A recently completed exploratory study in South Africa has been confronting 'at risk' students with the implications of their 'disintegrated' perceptions. Attempts at helping these students to 'reorchestrate' their perceptions of their experiences of teaching and studying seems to be a promising intervention (Parsons and Meyer 1990), but it was still not clear from interviews with these students how these students perceived their academic context. Further interviews with students will be necessary to establish the 'world view' associated with 'disintegrated' perceptions, and how that world view leads to failure. It is hoped that others will now join the search for the phenomenon of 'disintegrated perceptions' and its educational significance.

References

Beard, R. and Hartley, J. (1984). Teaching and Learning in Higher Education. London: Harper & Row. Biggs, J. B. (1985). 'Metalearning and study processes', British Journal of Educational Psychology 55,

185-212. Calder, I. (1989). The Study and Learning Strategies of Students in a New Zealand Tertiary Institution.

Unpublished PhD Thesis, University of Waikato, New Zealand.

261

Entwistle, N. J. (1987). 'A model of teaching-learning process', in Richardson, J. T. E., Eysenck, M. W. and Warren Piper, D. (eds.), Student Learning: Research into Education and Cognitive Psychology. Milton Keynes: Open University Press.

Entwistle, N.J. (1990). 'Teaching and the quality of learning in higher Education', in Entwistle, N.J. (ed.), Handbook of educational Ideas and Practices. London: Routledge.

Entwistle, N. J. and Brennan, T. (1971). 'The academic performance of students. II. Types of successful students', British Journal of Educational Psychology 41,268-276.

Entwistle, N. J., Hounsell, D. J., Macauley, C., Situnayake, G. and Tait, H. (1989). The Performance of Electrical Engineering Students in Scottish Higher Education. Report to the Scottish Education Department. Edinburgh: Department of Education, University of Edinburgh.

Entwistle, N. 3. and Marton, F. (1989). 'The psychology of student learning', European Journal of Psychology of Education IV, 449--453.

Entwistle, N. J. and Ramsden, P. (1983). Understanding Student Learning. London: Croom Helm. Entwistle, N. J. and Tait, H. (1990). 'Approaches to learning, evaluations of teaching, and preferences for

contrasting academic environments', Higher Education 19, 169-194. Harper, G. and Kember, D. (1989). 'Interpretation of factor analyses from the Approaches to Studying

Inventory', British Journal of Educational Psychology 59, 66-74. Marsh, H. W. (1987). 'Students' evaluations of university teachers: research findings, methodological

issues, and directions for future research', International Journal of Educational Research 11 (3). Marton, F. and Ramsden, P. (1988). 'What does it take to improve learning?', in Ramsden, P. (ed.)

Improving Learning." New Perspectives. London: Kogan Page, pp. 268-278. Meyer, J. H. F. and Muller M. W. (1990a). 'An unfolding analysis of the association between perceptions

of learning context and approaches to studying', South African Journal of Higher Education 4, 46-58. Meyer, J. H. F. and Muller, M. W. (1990b). 'Evaluating the quality of student learning: I. An unfolding

analysis of the association between perceptions of learning context and approaches to studying at an individual level', Studies in Higher Education 15, 131-154.

Meyer, J. H. F. and Parsons, P. (1989). 'Approaches to studying and course perceptions using the Lancaster inventory - a comparative study', Studies in Higher Education 14, 137-154.

Meyer, J. H. F., Parsons, P. and Dunne, T.T. (1990a). 'Individual study orchestrations and their association with learning outcome', Higher Education 20, 67-89.

Meyer, J. H. F., Parsons, P. and Dunne, T.T. (1990b). 'Study orchestration and learning outcome: evidence of association over time among disadvantaged students', Higher Education 20, 245-269.

Newble, D. and Cannon, R. (1989). A Handbook for Teachers in Universities and Colleges. London: Kogan Page.

Parsons, P. and Meyer, 3. H.F. (1990). 'The academically 'at risk' student: a pilot intervention programme and its observed effects on learning outcome', Higher Education 20, 323-334.

Richardson, 3. T. E. (1990). 'The reliability and replicability of the Approaches to Studying Inventory', Studies in Higher Education 19, 155-168.