portfolio as a means of promoting autonomous learning in teacher education: a quasi-experimental...
TRANSCRIPT
This article was downloaded by: [University of Auckland Library]On: 06 November 2014, At: 13:08Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Educational ResearchPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/rere20
Portfolio as a means of promotingautonomous learning in teachereducation: a quasi-experimental studyWil Meeus a , Peter Van Petegem a & Joost Meijer ba Universiteit Antwerpen , Belgiumb Universiteit van Amsterdam , the NetherlandsPublished online: 15 Nov 2008.
To cite this article: Wil Meeus , Peter Van Petegem & Joost Meijer (2008) Portfolio as a means ofpromoting autonomous learning in teacher education: a quasi-experimental study, EducationalResearch, 50:4, 361-386, DOI: 10.1080/00131880802499837
To link to this article: http://dx.doi.org/10.1080/00131880802499837
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Portfolio as a means of promoting autonomous learning in teacher education:
a quasi-experimental study
Wil Meeusa*, Peter Van Petegema and Joost Meijerb
aUniversiteit Antwerpen, Belgium; bUniversiteit van Amsterdam, the Netherlands
(Received 17 May 2007; final version received 22 August 2008)
Background: The predominant dissertation model used in teacher education courses inFlanders is the ‘literature study with practical processing’. Despite the practicalsupplement, this traditional model does not fit sufficiently well with autonomouslearning as the objective of modern teacher education dissertations.This study reports on the development of a portfolio model to serve as an alternative tothe current model and afford greater opportunities for autonomous learning.Purpose: This quasi-experimental study aimed to examine the usefulness of portfolioas a means of promoting autonomous learning.Sample: Two teacher education courses collaborated, involving 174 teacher educationstudents and 44 supervisors in this study.Design and methods: Using a pre- and post-test quasi-experimental design, weexamined whether the use of portfolio as a dissertation model has a greater effect onthe students’ capacity for autonomous learning than the literature study with practicalprocessing, using three existing questionnaires. Differences between the experimentalgroup and the control group were revealed by means of covariance analysis.Results: Our research shows that the portfolio model results in students acquiringgreater metacognitive knowledge. It also appears that students only get the chance touse their metacognitive skills when supervisors give them sufficient autonomy. However,supervisors experience loss of control and therefore tend to restrict students’ autonomy.Conclusions: This leads us to the paradox of autonomous learning: students will onlylearn autonomously when they are given the freedom to learn autonomously.Supervisors need to transfer learner control.
Keywords: portfolio; autonomous learning; dissertation; teacher education
Introduction
Higher education in Europe is based on a two-cycle degree system: a three-yearundergraduate study leading to a Bachelor’s degree, and a contiguous graduate study, ofat least one year, culminating in a Master’s degree. The Bachelor’s degree is intended tooffer graduates better prospects on the labour market. Most Bachelor’s and Master’sdegree programmes in Europe terminate in a dissertation of some kind. Logically, thesedissertations have to be the final component in the course of study not only with regard totiming, but also with regard to content.
The dissertation in higher education is an under-researched area (McAlpine and Weiss2000; Coorough and Nelson 1997; Acker, Hill and Black 1994). A review of the existing
*Corresponding author. Email: [email protected]
Educational Research
Vol. 50, No. 4, December 2008, 361–386
ISSN 0013-1881 print/ISSN 1469-5847 online
� 2008 NFER
DOI: 10.1080/00131880802499837
http://www.informaworld.com
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
literature reveals that the fundamental objective behind the Master’s thesis is to contributeto the further development of theory within a specific discipline through academic andscientific research (Thomas and Brubaker 2000). Hence the Master’s thesis is theoretical innature. With regard to the Bachelor thesis, there is hardly any literature available,although there are a large number of practical handbooks on how this final thesis shouldbe written. In these handbooks the term ‘final thesis’ is used together with others such as‘final test’, ‘thesis’, ‘paper’, ‘dissertation’, ‘work project’ and ‘graduation project’. Despitethe differences of terminology and content in accordance with the nature of the course inquestion, all of these refer to the final thesis at Bachelor level. One general tendency isapparent, however: the Bachelor thesis is, in most cases, a derivative of the Master’s thesis.It thus follows that the theory-oriented approach also predominates at Bachelor level.
In Europe, the idea that the holder of a Master’s degree should be capable ofconducting research on a modest scale is something which is generally accepted. Indeed,the European Master’s thesis is usually a theoretical study intended to reveal the student’sresearch competencies. With regard to the Bachelor thesis, however, the theory-orientedapproach is subject to considerable criticism, even though this has not changed the factthat it continues to predominate. This predominance of the theory-oriented approach inBachelor programmes is a peculiar phenomenon. Bachelor programmes are not aimed atproviding an academic education for scientists or professional academics. They aredirectly aimed at preparing students for the labour market. The Bachelor thesis is, in fact,regarded as a bridge between higher education and the job market (Padmos 2001). Yet theBachelor thesis is primarily theoretical and research-based in nature. We have to conclude,therefore, that the ultimate aim of these programmes, and the ultimate aim behind thedissertations produced in the course of these programmes, are miles apart.
Throughout Europe the need for a more practice- and profession-oriented approach tothe Bachelor thesis is increasingly felt (Gunneng and Ahlstrand 2002). The theory-orientedapproach to the Bachelor thesis is considered fundamentally flawed. A practice-orientedapproach seems to be more appropriate. Academic or scientific research is the cornerstoneof the theory-oriented approach, but what form should the practice-oriented approachtake? The answer probably differs from programme to programme. We will focus on thethree-year teacher education programmes in Flanders. Although the Bachelor dissertationin teacher education serves several goals, it provides ample opportunity to stimulateindependent performance and to get students to engage in autonomous learning (VanLooy and Elias 2001).
In courses at teacher education colleges in Flanders the ‘literature study with practicalprocessing’ is the dominant dissertation model (Meeus, Van Looy and Van Petegem 2005).The characteristics of the literature study with practical processing are:
. a particular topic as point of departure;
. a theoretical exploration of the topic;
. a practical application of the theory to teaching practice.
In this model students choose a topic – e.g. child abuse – about which they learn morethrough a literature study. They aim to achieve a comprehensive overview of the relevantliterature and then work this up into a text of their own directed at a self-chosen targetgroup of pupils, for example, a particular grade or year group. After composing anoriginal text on the topic, the students try to make use of their study in their teachingpractice – e.g. developing and implementing a child abuse prevention programme or aninitial reading methodology in the classroom. They convert the knowledge into a practical
362 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
product for use with these pupils, such as a project, a number of lesson preparations ordidactic material. Despite the practical supplement, the traditional dissertation modelmainly is theory based and research orientated. It does not tie in sufficiently withautonomous learning as the objective of modern teacher education dissertations.
The point of departure for the present study was to bring the evaluation format – inthis case, the dissertation model – more into line with the objectives of the course (VanPetegem and Vanhoof 2002). We needed to develop a contemporary profession-orienteddissertation model. On the basis of an extensive literature study, we developed a ‘learningportfolio’ as a dissertation model for teacher education courses for pre-school and primaryschool teachers (Meeus and Van Looy 2005). The characteristics of the learning portfolioare:
. a teaching competency as point of departure;
. a cycle of action and reflection;
. the use of a variety of media and materials.
The learning portfolio dissertation can be interpreted as an extensive competency report inwhich students demonstrate their teaching and learning competencies by deliveringevidence of their teaching activities and reflecting on their performance. The use ofmultimedia is important in order to be able to present clear evidence of what the studenthas done. For example, for her final thesis a student concentrates on the competency ofdealing with children with learning disabilities. During her practice, she gathers evidenceof her competency – e.g. lesson plans, a video tape of an interesting intervention, asummary of the pupils’ learning progress, etc. Furthermore, she uses these materials toreflect explicitly on her competency.
The learning portfolio is intended to encourage students to reflect on their teaching andto get them to document their competency development. In contrast to portfolios whichare used to evaluate students’ teaching competency, the learning portfolio focuses on theirautonomous learning (Zeichner and Wray 2001). In the learning portfolio as we havedeveloped it, students start by choosing a personal learning point on the basis of astrengths/weaknesses analysis carried out in conjunction with their portfolio supervisor.They then go on to draw up a personal learning plan which they will try to implement aswell as they can. They reflect on the teaching activities they have carried out with theirfellow-students, the tutor at their teaching practice school or their portfolio supervisor orteacher educator. Students use multimedia to compile their portfolio, which is expected tocontain a large number of different items that will serve to illustrate the students’ teachingactivities. Students are not compelled to submit an electronic learning portfolio, but this iscertainly encouraged. The evaluation of the learning portfolio focuses on students’capacity for autonomous learning.
In the present paper, we report the results of a quasi-experimental study in which weplaced two dissertation models – the ‘learning portfolio’ and the ‘literature study withpractical processing’ – side by side. Experimental research makes it possible to explore therelationship between causes and effects (Mertens 1998, 61) and our study set out toexamine whether, as a dissertation model, portfolio gives students a greater learningbenefit than the literature study with practical processing; and, in particular, whether thestudents are more successful in achieving the central objective of the dissertation, which isthe promotion of autonomous learning. Autonomous learning capacity can beoperationalised in various ways. We used validated questionnaires focused onmetacognition, self-direction, cognitive processing strategies, mental learning models
Educational Research 363
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
and learning orientations. In this way, we hoped to explore the most important facets ofautonomous learning capacity.
Research method
This quasi-experimental study was carried out during the academic year 2002/03. Studentsand staff from an initial teacher education course for pre-school education (PSTE) and aninitial primary education course (PRTE) took part in the research. We used existingquestionnaires (two for the students and one for the supervisors) and opted for anexperimental model because this enabled us to gain experience with the new dissertationmodel, as well as gathering useful information about its effects. In laboratory experiments,test subjects are allocated to the experimental group or the control group entirely atrandom. In the case of our study, this was not possible which is why we opted for a quasi-experimental research design (Cook and Campbell 1979). The study is entirelyquantitative. We compared the capacity for autonomous learning among students fromthe experimental group with that of students in the control group using a before-and-aftermeasurement. The students with a portfolio undertook the experimental dissertationmodel and thus constituted the experimental group. The students with a literature studywith practical processing stuck to the traditional dissertation model and thereforeconstituted the control group. Given that this was a quasi-experimental study in which testsubjects could not be assigned at random to the respective groups, there was a risk thatcontext variables might have distorted our research results. ‘Stronger’ or ‘weaker’ studentsmight have been unequally distributed across the groups. Furthermore, we do not knowwhether these groups received supervision of equal value. For this reason, we collecteddata relating to the following context variables:
. Data relating to student performance levels, specifically their teaching practice markand their total mark for the second year.
. Data on the supervisors concerning a number of supervision-related variables, inparticular, how much work experience supervisors had; their own conceptions oflearning; and their degree of confidence in the students’ capacity for self-direction.
The students in the research group
The entire student population (n ¼ 174) of a pre-school teacher education course and aprimary teacher education course took part in the study, although in different ways. On thepre-school teacher education course the teacher educators were enthusiastic and decided tomake it compulsory for all 53 students to compile a portfolio as a dissertation. The teachereducators on the primary teacher education course, on the other hand, wished to run the twodissertation models side by side. The 121 students were allowed to sign up, voluntarily, forone or other of the dissertation models. A total of 93 (76.9%) opted for the literature studywith practical processing and 28 (23.1%) for the portfolio. In total, therefore, 93 literaturestudies with practical processing and 81 portfolios were undertaken.
When interpreting the research results, we had to take into account possible differencesbetween the courses, as there were no pre-school teacher education students in the controlgroup. The students of the two courses were therefore not equally distributed across theexperimental and the control groups. Differences between the two courses in terms ofperformance could therefore threaten the validity of the experiment if they provedprevalent in the experimental group and the control group. For this reason, we looked at
364 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
whether the students were equally divided in terms of performance level across the twogroups. As an indicator we used the marks which students from the research groupobtained for their teaching practice at the end of their second year. We can assume thatstudents on both courses were subject to the same requirements on their teaching practiceand that they were required to demonstrate the same level of teaching competency inrelation to the age category of the pupils concerned. We compared the teaching practicemarks (in percentages) obtained by students at the end of the second year of the twocourses and by students from the experimental and the control groups using a t-test(Table 1). We also looked at the differences between the experimental group and thecontrol group for the primary teacher education students only, as this would tell uswhether the competency level played a role with regard to the preference for a particulardissertation model.
There is no significant difference between the teaching practice marks of the studentsfrom the two courses. The students from both courses perform equally well in terms ofteaching competency in accordance with the level at which they have been trained to teach.There is also no significant difference in teaching practice marks between the experimentalgroup and the control group, and in terms of teaching practice marks the primary teachereducation students are equally distributed across the experimental and the control groups.The performance level of the primary teacher education students with respect to theirteaching practice therefore does not play a role in the choice of one dissertation modelrather than another.
The supervisors in the research group
On both courses, each student had a supervisor. They were allowed to choose their ownsupervisor, subject, of course, to the agreement of the person chosen. In pre-school teachereducation all 14 dissertation supervisors supervised portfolios. In primary teachereducation there were 30 dissertation supervisors in total. Ten of them supervised one ormore literature studies, as well as one or more portfolios, while the other 20 supervisedeither only portfolios or only literature studies with practical processing. For eachdissertation model considered in primary teacher education there were 27 supervisors ofliterature studies with practical processing and 13 portfolio supervisors.
In order to be able to establish possible differences between the dissertation models, wehad to be able to assume that the students of both dissertation models could count on thesame supervision. In other words, the supervisors were an intervening variable whichmight influence the possible differences between the models. In our research setting it wasneither organisationally nor psychologically permissible to pre-select the supervisors onthe basis of their supervisory or evaluatory qualities. We did, however, screen thesupervisors at the start of the study for their work experience, personal conceptions oflearning and their confidence in the students’ capacity for self-direction. In this way, weobtained a good picture of the possible interference of the supervisors in our researchresults.
The quasi-experimental study focused on differences between students, not ondifferences between supervisors. As we were working with 44 supervisors in total, thedifferences in supervision carried out by the supervisors might well have interfered withour research into the impact of the dissertation model on the students’ capacity forautonomous learning. We thus needed to examine certain aspects of the supervisionprovided, which might endanger the validity of the experiment. We were not interested inthe individual differences in supervision among the supervisors; their supervision was,
Educational Research 365
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
Table
1.
Teachingpracticemark
awarded
inthe2ndyear,bycourseanddissertationmodel
(fullandPRTE
only).
Av.PSTE
(n¼
53)
Av.PRTE
(n¼
121)
p-courses
Av.exp.
group
(n¼
81)
Av.control
group
(n¼
93)
p-m
odels
Average
exp.group
(PRTE
only;
n¼
28)
p-m
odels
(PRTE
only)
2nd-year
teaching
practice
mark
in%
61.18
(s¼
7.91)
63.75
(s¼
7.76)
0.050
(t¼
–1.973)
62.09
(s¼
8.31)
63.75
(s¼
7.44)
0.169
(t¼
–1.380)
63.75
(s¼
8.89)
1.000
(t¼
0.000)
366 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
after all, only an intervening variable. However, we were interested in the differences insupervision provided by the supervisors between the experimental group and the controlgroup.
We would not have needed to pay attention to this intervening variable if, in theideal situation, the group of supervisors in the experimental group had been the sameas the group of supervisors in the control group. For each student with a portfoliosupervised by a particular supervisor, this supervisor should ideally have alsosupervised a student with a literature study with practical processing. In that case,the experimental and the control groups would have had exactly the same supervisors,so that this intervening variable would have ceased to exist; however, the supervisorswere neither selected nor assigned to students in accordance with the needs of thisstudy. Given the importance of the free choice of supervisor from the students’ pointof view, we could not achieve this in our quasi-experiment. Only 10 supervisorssupervised both students with a portfolio and students with a literature study withpractical processing. This, then, was sufficient reason for us to look more closely at thesupervision carried out by the supervisors.
Bearing in mind the general research question, the work experience of the supervisors,their conceptions of learning and their confidence in the students’ capacity for self-direction were all taken into account. Their work experience was a possible indicatorbecause experience with the supervision of dissertations can influence the quality ofsupervision. We also regarded the supervisors’ conceptions of learning as an indicator ofsupervision style (Roelofs and Visser 2001). Moreover, differences in supervision style mayalso have influenced the learning capacity of the students. We regard confidence in thestudents’ capacity for self-direction as an important indicator of the extent to whichsupervisors give their students the chance to engage in autonomous learning (Lunenbergand Korthagen 2002).
We asked the supervisors to complete the questionnaire at the start of the study, inother words, at the start of their dissertation supervision. They were asked to indicate howmany years of work experience they had, using five categories: 1 ¼ 0–5 years; 2 ¼ 6–10years; 3 ¼ 11–15 years; 4 ¼ 16–20 years; and 5 ¼ more than 20 years. For the supervisors’conceptions of learning and their confidence in the students’ capacity for self-direction, weused a number of scales taken from the questionnaire developed by Roelofs and Visser(2001), who carried out research into learning conceptions and achievement of differentlearning environments among parents, teachers and pupils. The scales concerning‘Conceptions of learning among teachers’ and ‘Teachers’ confidence in students’ capacityfor self-direction’ proved useful in our study.
With regard to the conceptions of learning among teachers, we identified five views ofwhat learning, knowledge and studying involve and also used a separate scale ‘Confidenceof teachers in the students’ capacity for self-direction’. Respondents were asked to indicatetheir preferences for each of a total of 37 items on a four-point scale, ranging from ‘entirelydisagree’ and ‘largely disagree’ to ‘largely agree’ and ‘entirely agree’. Example items fromeach of the scales of ‘Teachers’ conceptions of learning’ and the scale ‘Confidence ofteachers in students’ capacity for self-direction’ are presented in Table 2.
The number of items per scale and the reliability coefficient as obtained by Roelofs andVisser (2001) are presented in Table 3. The reliability of the scales was sufficient for use inour own study. We also indicated the reliability coefficient as obtained for our ownresearch group of supervisors. With the exception of the scale ‘Use of knowledge’, thereliability of the scales was confirmed by our research. Possible differences on the scale‘Use of knowledge’, in our study, must therefore be seen in perspective.
Educational Research 367
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
On the basis of the Mann-Whitney test, there appeared to be no significant differencebetween the work experience of the supervisors in the experimental and control groups(p ¼ 0.321). The supervisors in both groups had, on average, between 11 and 15 years’experience of supervising dissertations. Nor did there seem to be a significant difference
Table 2. Example items from the scales.
Scale Example item
Acquisition of knowledge When doing exercises and assignments I like to be told exactly whatI have to do
Use of knowledge I must try out the study material myself in practical situationsStimulating education The teacher must encourage me to find my own solutions when I
encounter difficultiesLearning together I prefer to do assignments together with other studentsSelf-discovery learning I prefer to learn in my own way and not in the way other people
think it should be doneConfidence in self-direction Students must get the opportunity to decide for themselves about
the order and duration of their activities
Table 3. Scales for teachers’ conceptions of learning and confidence in students’ capacity for self-direction.
Learningconception Description n items
Cronbach’s a(Roelofs andVisser 1991)
Cronbach’sa (Meeus)
Acquisition ofknowledge
Learning is above all concernedwith the acquisition of factualknowledge. Learning meansremembering the study materialcovered as well as possible in orderto reproduce it later
6 0.68 0.60
Use of knowledge Learning is the acquisition ofknowledge and skills in order to putthis to practical use sooner or later
4 0.61 0.38
Stimulatingeducation
The teacher’s task is primarily thatof encouraging students toengage in a broad range of studyactivities. It is important thatteachers offer a helping handin carrying these out
4 0.78 0.88
Learning together Carrying out learning tasks withothers is of great value. Teacherswith this conception found thatmany of the students were keenand able to divide learning tasksup among themselves
5 0.75 0.77
Self-discoverylearning
Learning is looking for knowledgeyourself and processing this inyour own way as an answer toself-set learning questions
8 0.69 0.58
Confidence inself-direction
Confidence in students’ capacityfor self-direction
10 0.87 0.79
368 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
between the supervisors from the experimental group and those of the control group forany of the five conceptions of learning (Table 4). What is striking, however, is that therewas a significant difference between the supervisors from the two groups in terms of theirconfidence in students’ capacity for self-direction.
We found no significant differences regarding work experience and supervisors’conceptions of learning. We therefore assume that these variables did not interfere with theresearch results. The significant difference in the ‘Confidence of teachers in students’capacity for self-direction’ requires more research (Table 5). We compared all the sub-groups with each other:
. pre-school teacher education with primary teacher education – p(a);
. the primary teacher education experimental group with the primary teachereducation control group – p(b);
. pre-school teacher education with the primary teacher education experimentalgroup – p(c);
. pre-school teacher education with the primary teacher education controlgroup – p(d).
No significant difference was found between these different groups, although we were ableto conclude from the averages that the significant difference between the full experimentalgroup and the control group could be attributed to the greater confidence in the students’capacity for self-direction, shown by both the pre-school teacher education and theprimary teacher education supervisors.
Support for the research group
We did all we could to support the organisation of dissertations; at the end of the academicyear 2001/02, a briefing was held in order to prepare future third-year primary teachereducation students for the dissertation and their choice of a dissertation model. Thosestudents who were interested in doing a portfolio were allowed – again, on a voluntarybasis – to attend a debriefing in which examples of good practice from the previousacademic year were presented and explained by the students who had made them.Examples of good practice of portfolio were also made available in the multimedia centre.
Students were provided with two manuals: one for the literature study with practicalprocessing and one for the portfolio. All students were given a copy of the manual of thedissertation model they had chosen, and the supervisors received a copy (or copies) of the
Table 4. Conceptions of learning and confidence in students’ capacity for self-direction, bydissertation model.
Averageportfolio(n ¼ 81)
Averageliterature study
(n ¼ 93) p-models
Acquisition of knowledge 2.47 (s ¼ 0.44) 2.52 (s ¼ 0.44) 0.490 (t ¼ –0.693)Use of knowledge 3.06 (s ¼ 0.45) 3.02 (s ¼ 0.49) 0.515 (t ¼ 0.653)Stimulating education 2.30 (s ¼ 0.78) 2.23 (s ¼ 0.79) 0.545 (t ¼ 0.606)Learning together 2.51 (s ¼ 0.57) 2.61 (s ¼ 0.60) 0.301 (t ¼ –1.038)Self-discovery learning 3.32 (s ¼ 0.33) 3.27 (s ¼ 0.32) 0.412 (t ¼ 0.822)Confidence in self-direction 2.96 (s ¼ 0.37) 2.83 (s ¼ 0.34) 0.023* (t ¼ 2.289)
Educational Research 369
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
Table
5.
Confidence
incapacity
forself-direction,bycourses
anddissertationmodel
(PRTEonly).
AveragePSTE
(n¼
53)
Average
PRTE
(n¼
121)
p(a)
courses
Average
portfolio
(PRTE
only;
n¼
28)
p(b)
models
(PRTE
only)
p(c)
courses
(portfolio
only)
p(d)
PSTE
(full)
andPRTE
(literature
study)
Confidence
inself-direction
2.95
(s¼
0.38)
2.87
(s¼
0.36)
0.203
(t¼
1.279)
2.98
(s¼
0.38)
0.052
(t¼
1.970)
0.681
(t¼
–412)
0.067
(t¼
1.847)
370 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
manual(s) for the dissertation models they were due to supervise. Both in the pre-schooland primary teacher education, the dissertation is an agenda item on course councils or atspecial educational events, such as the departmental day, on a number of occasions. Theresearcher also made himself readily available to students and supervisors alike by hisfrequent physical presence and also via email, and we used the electronic learningenvironment in the electronic learning platform Blackboard.
Research instruments and techniques
We used two existing questionnaires for the students. These were discussed beforehand,and after each discussion, we formulated the research questions to be answered by theinstrument concerned. The two questionnaires were the Reporting autonomous studyingquestionnaire (RAS) and the Inventory of learning styles for higher education (ILS-HE),both of which are self-reporting instruments. The validity of self-reporting instruments hassometimes been challenged and it is indeed true that they are not as authoritative as directobservation or test instruments; after all, self-reporting instruments only provide a pictureof what respondents believe that they think and do, not necessarily of what they actuallythink and do (Winne and Jamieson-Noel 2003). That said, we found self-reportinginstruments which measured self-direction valuable for our purposes, because studentswho score high on these generally obtain better academic results than students who scorelow on them (Zimmerman 1994). These are also efficient instruments because they arecapable of measuring multiple facets of self-direction among large groups of students in ashort period of time. Given the nature of our research design and the size of our researchgroup, the use of self-reporting instruments seemed an obvious choice.
Reporting autonomous studying questionnaire (RAS)
Autonomous learning is the end-point of a continuum stretching from external tointernally directed learning. Internal direction – or self-direction – has a major influence onthe effectiveness of learning processes (Elshout-Mohr 1992; Wang, Haertel and Walberg1990). Moreover, graduates are expected to be able to learn effectively on a life-long basis,for which self-direction is a necessary condition.
One of the instruments used in our quasi-experimental study measuring students’capacity for self-direction was the Reporting autonomous studying questionnaire (RAS),according to which the capacity for self-direction can be regarded as the extent to whichstudents are able to carry out metacognitive activities. Students who engage in a largenumber of metacognitive activities obtain better results than those who undertake only asmall number of such activities. This has been demonstrated in both field (de Jong 1992)and laboratory studies (Veenman, Elshout and Meijer 1997). Metacognition is regarded asfundamental to their personal development as life-long learners.
The RAS questionnaire was developed fairly recently by the SCO-KohnstammInstitute, University of Amsterdam (Elshout-Mohr, van Daalen-Kapteijns and Meijer2001), and was used as part of a study conducted at the Faculty of Education inAmsterdam into the influence of the introduction of an electronic portfolio on themetacognition of students in teacher education. Although the research design used in thiscase was somewhat different from our own, the RAS questionnaire was clearly of use inour research, given that it examines whether the implementation of the portfolio promotesstudents’ metacognition. For the full background on composition of the questionnaire, thereader is referred to Elshout-Mohr et al. (2003).
Educational Research 371
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
The RAS questionnaire is an instrument with which students can self-report on a widevariety of aspects of their metacognitive qualities, with the aid of a list of statements.‘Autonomous studying’ is the term used to denote a form of study in which metacognitionplays a considerable role. The RAS questionnaire indeed provided an answer to thefollowing questions:
(1) How far do students, in their own opinion, possess a declarative knowledge ofstudying?
(2) How far do students, in their own opinion, have the skill to regulate their studyingsystematically?
(3) How far do students, in their own opinion, have an information-sensitive andresearch-oriented attitude with regard to information which is important forfurther development of their metacognitive knowledge and regulatory skills?
In a manner analogous to the above statement of the research problem, three maincategories could be identified within metacognition in the construction of the RASquestionnaire: metacognitive knowledge (K); metacognitive regulation (R); and metacog-nitive development (D). Each of these main categories was further subdivided into threesub-categories (Table 6). The three selected components of metacognitive knowledge werederived from the work of Flavell (1979); the three components of metacognitive regulationwere taken from the systematic approach to the problem, as described by Mettes and Pilot(1980) and Boekaerts and Simons (1995) among others; and the three components of themetacognitive development were taken from a variety of sources, including Flavell (1979),Butler and Winne (1995) and Veenman, Elshout and Meijer (1997). Some examples of thepositive and negative formulations in which the statements were put to the students foreach of the subscales are presented in Table 7.
In terms of content, the RAS questionnaire represents a selection from the broaddomain of metacognition and all statements relate to studying in higher education. Seventopics were selected which might constitute important attention pointers for students whenthinking about their studying experiences and their approach to studying. The topics weretaken from various publications concerning effective studying and study skills (see, amongothers, Elshout-Mohr and van Daalen-Kapteijns 1993; Van Hout-Wolters, Jongepier andPilot 1995), research into autonomous studying in higher education (Elshout-Mohr,
Table 6. Scales and subscales from the Reporting autonomous studying questionnaire.
Metacognitive knowledgeK1. Knowledge of studyingK2. Knowledge of study strategiesK3. Knowledge of study tasks
Metacognitive regulationR1. OrientationR2. MonitoringR3. Evaluation
Metacognitive developmentD1. Attentiveness to internal feedback in the form of metacognitive experiencesD2. Receptivity to external feedback at metacognitive levelD3. Research-oriented attitude with regard to own functioning
372 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
van Daalen-Kapteijns and Baltzer 1990; van Dyck, Leigh and Baks 1991) and meta-analyses of research into effective instruction. The seven topics are:
(1) Learning intention;(2) Involvement;(3) Collaborative learning;(4) Understanding/insight;(5) Orderliness/systematic approach;(6) External valuation;(7) Facilities.
Topics 1–3 concern the relationship between cognitive processes of autonomous studyingand three other human functions, namely: conation (striving and the exercise of the will);affect (feeling); and orientation towards others (the social drive). These topics wereincluded in the RAS questionnaire because students in higher education have to directtheir own study process, and therefore have to address their objectives; their affinity withparticular items of content and assignments; and the social aspects of their studying.
Topics 4 and 5 relate to aspects of cognitive functioning already deployed in secondaryeducation, but which require continuous attention from students in higher education.These topics concern the need to acquire understanding and insight and to prevent thevolume of information obstructing the learning process by means of orderliness and asystematic approach.
Topics 6 and 7 were included in the RAS questionnaire because their significance forstudents in higher education is different from and and greater than in the case of secondary
Table 7. Examples of formulations used in the subscales of the Reporting autonomous studyingquestionnaire.
Positive format Negative format
K1 I think that students . . .According to me, students are . . .
I don’t think that students . . .According to me, students are not . . .
K2 I know how . . .If . . ., I know how to solve it
I wouldn’t know how . . .If . . ., I don’t know how to solve it
K3 In an assignment, I can tell whether . . .I know with which assignments . . .
In an assignment, I can’t tell whether . . .I don’t know with which assignments . . .
R1 Before I . . ., I think about whether . . .If I . . ., I first check how . . .
Before I . . ., I don’t think about whether . . .If I . . ., I don’t first check how . . .
R2 During . . ., I regularly check for myselfwhether I . . .
If I . . ., I make sure that I . . .
During . . ., I don’t regularly check formyself whether I . . .
If I . . ., I don’t make sure that I . . .R3 When I’m finished with . . ., I ask myself . . .
Before I . . . finish, I check . . .When I’m finished with . . ., I don’t ask
myself . . .Before I . . . finish, I don’t check . . .
D1 During . . ., I sometimes immediately noticewhether . . .
I sometimes have the immediate feelingthat . . .
During . . ., I don’t sometimes immediatelynotice whether . . .
I don’t sometimes have the immediatefeeling that . . .
D2 If my . . . is discussed, I think about itI think it is a good idea to talk withother people about . . .
If my . . . is discussed, I don’t think about itI don’t think it is a good idea to talk
with other people about . . .D3 I’m interested in why . . .
If I . . ., I go and find out why that isI am not interested in why . . .If I . . ., I don’t go and find out why that is
Educational Research 373
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
education. For students in secondary education, external valuation is still largelydetermined by the valuation of the subject teacher, while in higher education this isincreasingly determined by a wider forum of future colleagues. Students enjoy a certainautonomy as to how they choose to study and are able to develop a personal vision ofwhat they wish to achieve through studying, but that does not mean they do not need totake account of the conceptions, standards and values of others. In secondary education,use of facilities (such as classes, tutor groups and ICT) is still directed by teachers, to alarge extent, while in higher education this is seen more as responsibility of the student.
The RAS questionnaire contains 63 statements (Table 8). Each of the seven topics isrepresented by nine statements, one statement for each of the nine metacognitive sub-components. This implies, then, that the RAS questionnaire is constructed according tothe so-called ‘facet method’ (Shavelson and Webb 1991). In the cross-tables of topics andsub-components each box is exactly filled by a single statement. For each statement thestudent circles a number on a seven-point scale, which runs from 1 (‘completely untrue forme’) through 4 (‘neutral’/’ don’t know’) to 7 (‘entirely true for me’). Half of the statementsare expressed in a positive format, the other half in a negative format.
After mirroring of the various scores for the statements in negative format, the answersto the three questions were derived from the averages of the sum scores for the 21knowledge statements, the 21 regulation statements and the 21 developments statements.The RAS questionnaire, as a whole, measures metacognition which we regard as ameasure of self-direction. The RAS scores of the students were calculated both for thequestionnaire, as a whole and for the three main categories: metacognitive knowledge,metacognitive regulation and metacognitive development. In each case, the minimumscore was 1 and the maximum score 7. The internal consistency (Cronbach’s a) of the itemsin the three main categories was satisfactory, and the a-values were confirmed in our study(Table 9). We concluded from the data that we were justified in working with separatescores for metacognitive knowledge, regulation and attitude when reporting the RASresults.
Finally, from discussion of the RAS questionnaire we formulated the research questionto be answered. This questionnaire can provide us with an answer to the following specificresearch question: Does the use of the portfolio as a dissertation model have a greatereffect on students’ metacognition (knowledge, regulation, development) than the literature
Table 8. Example items from the Reporting autonomous studying questionnaire.
Topic 5 – R2 8 While working on an assignment, I make sure that I carry out all thesub-components
Topic 6 – D3 26 If my work does not get the valuation it deserves, I am not interested inknowing why
Topic 6 – K2 27 If student performances are underestimated, I know what the solution isTopic 3 – K2 30 If collaboration between students does not seem to be useful, I don’t have
a solutionTopic 6 – R2 32 When doing an assignment, I don’t think about what others will think
of my workTopic 3 – R2 34 If I am working with others, I regularly check what I am learning from themTopic 3 – D3 44 I’m interested to know why I sometimes get so little out of my collaboration
with othersTopic 5 – D3 49 If I can’t see how to structure an assignment, I go and find out how it could
be doneTopic 5 – K2 50 If students don’t work systematically, I don’t have any solutions for this
374 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
study with practical processing? In our research with the supervisors as interveningvariable, we found a significant difference in the confidence of supervisors from theexperimental group and those from the control group in students’ capacity for self-direction. We therefore looked at the effect of this on possible differences between theexperimental group and control group, which can provide us with an answer to thefollowing additional research question: What is the influence of the confidence ofsupervisors in the students’ capacity for self-direction on the possible changes in students’metacognition?
Inventory of learning styles for higher education (ILS-HE)
The second instrument that we gave the students in our quasi-experimental study was theInventory of learning styles for higher education (ILS-HE: Vermunt 1992), in this case, aversion adapted to the Flemish situation (Van Petegem and DeMaeyer 2000). The ILS-HEis used to examine students’ learning styles which are, however, best conducted at the levelof the components which underlie those learning styles, not at the level of the learning stylesthemselves (Vermetten 1999, 136). The four components are: the students’ cognitiveprocessing strategies; regulation strategies; learning orientations and mental learningmodels. The ILS-HE has multiple scales for each of these components, a total of 16(Table 10).
The ILS-HE is a well-known instrument, extensively used in research of the presentkind, and its internal consistency (Cronbach’s a) has been amply demonstrated. Our scoresalso confirmed this. The ILS-HE is suitable for use in all forms of higher education andcontains 120 statements (Table 11). The student circles a number on a five-point scale,which runs from 1 (‘I seldom or never do this’ or ‘entirely disagree’) through 3 (‘neutral’ or‘don’t know’) to 5 (‘I almost always do this’ or ‘entirely agree’) for each statement.Negative items are not used.
Vermunt (1992) researched the relationship between cognitive processing strategies;regulation strategies; learning orientations; and mental learning models, and discoveredfour learning styles (Table 12). From the viewpoint of learning processes, the undirectedand reproduction-directed learning style can be classified as being of a qualitatively lowvalue and the meaning and application-directed learning styles can be seen as being of aqualitatively high value (Vermunt 1997). This means that students in higher educationgenerally benefit more from processing and regulation strategies, learning orientations andmental learning models, which are consistent with meaning and application-directedlearning styles. The processing and regulation strategies, learning orientations and mentallearning models which are associated with undirected and reproduction-directed learningstyles are best avoided.
Table 9. Cronbach’s a and internal correlations of the three Reporting autonomous studyingcomponents.
Metacognitiveknowledge
Metacognitiveregulation
Metacognitivedevelopment
Metacognitive knowledge a ¼ 0.80(Meeus: a ¼ 0.82)
0.559 0.591
Metacognitive regulation a ¼ 0.81(Meeus: a ¼ 0.83)
0.673
Metacognitive development a ¼ 0.78(Meeus: a ¼ 0.84)
Educational Research 375
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
The ILS-HE could provide us with answers to the following specific research questions:
(1) Does the use of portfolio as a dissertation model have a greater effect on students’cognitive processing strategies than the literature study with practical processing,in the sense that more use is made of concrete and in-depth processing strategiesand/or less use made of the step-by-step processing strategy or absence of strategy?
Table 10. Scales from the Inventory of learning styles for higher education.
Cognitive processing strategies a r
1 DP: In-depth processing strategy: a consistentuse of relating, structuring and criticalprocessing activities
0.894 0.665
2 SP: Step-by-step processing strategy: characterisedby a combined use of analytical and memoryorientated learning activities
0.813 0.558
3 CP: Concrete processing strategy: characterised by aconsistent use of specifying and applicationdirected learning activities
0.786 0.537
Regulation strategies4 SR: Self-directed regulation strategy: learning
processes are directed by the autonomous useof regulation activities
0.841 0.574
5 ER: Externally directed regulation strategy: learningprocesses are directed by the learning environment
0.693 0.403
6 UR: Undirected regulation strategy: learning processesare neither autonomous nor externally directed
0.760 0.520
Learning orientated7 CO: Certificate-driven orientation: learning motive
is focused on obtaining certificates0.725 0.501
8 PO: Profession-driven orientation: learning motiveis focused on being able to exercise a profession
0.721 0.341
9 TO: Test-driven orientation: learning motive isfocused on testing out the learner’s own capacities
0.826 0.459
10 IO: Personal interest orientation: learning motive isfocused on personal interests
0.594 0.489
11 AO: Ambivalent orientation: the learning motive isambivalent in regard to the module, course orlearning in general
0.770 0.437
Mental learning models12 AK: Acquisition of knowledge: vision in which
learning is seen as the acquisition of ready-to-use,externally available knowledge, as far as possiblein an unchanged form
0.815 0.592
13 CK: Construction of knowledge: a vision in whichlearning is seen as a construction of personalknowledge and insights
0.820 0.536
14 UK: Use of knowledge: a vision in which greatvalue is attached to learning to use knowledge
0.741 0.346
15 SE: Stimulating education: a vision in whichstimulating education is seen as important
0.895 0.442
16 LT: Learning together: a vision in which great valueis attached to collaboration and the distribution of tasks
0.880 0.574
376 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
(2) Does the use of portfolio as a dissertation model have a greater effect on students’regulation strategies than the literature study with practical processing, in the sensethat studying is approached in a more self-directed and/or less externally directedor undirected manner?
(3) Does the use of portfolio as a dissertation model have a greater effect on students’learning orientations than the literature study with practical processing, in thesense that learning is more profession orientated or study is more out of personalinterest and/or less certificate or test orientated or ambivalent?
(4) Does the use of portfolio as a dissertation model have a greater effect on the mentallearning models of the students than the literature study with practical processing,in the sense that they are more orientated towards the use and construction of
Table 11. Example items from the Inventory of learning styles for higher education.
DP I look for similarities and differences between the various theories covered on the courseSP I repeat the most important elements of study material until I know them by heartCP I pay particular importance to those parts of the course which have a practical useSR I do more than what is asked of me on the courseER I test only my learning progress by doing the questions, problems and exercises set by
the lecturer or contained in the textbookUR I find that I have difficulty processing large quantities of study materialCO My most important objective on this course is to pass the examsPO I chose this course because the work I can do with it interests me very muchTO Choosing to take a higher education degree is a challenge for meIO My course is only intended to enrich me intellectuallyAO I ask myself if the degree is worth all this effortAK The lecturer must explain clearly what I really need to know and what is less importantBM In order to learn I need to summarise what it says in the study material in my own wordsUK I must be able to use what I learn to solve practical problemsSE The lecturer must encourage me to think further on my own about how the study material
relates to the real worldLT I prefer to do assignments together with other students
Table 12. Learning style components, according to learning styles (Vermunt 1992).
Undirectedlearning style
Reproduction-directedlearning style
Meaning-directedlearning style
Application-directedlearning style
Cognitiveprocessingstrategy
Scarcelyapplicable
Step-by-step In-depth Concrete
Regulationstrategy
Undirected Externallydirected
Self-directed Both self- andexternallydirected
Learningorientation
Ambivalent Certificateand testorientated
Personalinterest
Profession orientated
Mental learningmodel
Learningtogether,stimulatinglearning
Acquisition ofknowledge
Construction ofknowledge
Use of knowledge
Educational Research 377
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
knowledge and/or less focused on the acquisition of knowledge, stimulatingeducation or learning together?
The significant difference between the experimental group and the control group in termsof the confidence of supervisors in the students’ capacity for self-direction could alsoplay a role in this case. Analogous to our additional research question concerningmetacognition, we formulate the following additional research question on learning stylestrategies: what is the influence of the supervisor’s confidence in the students’ capacity forself-direction on the possible changes in students’ learning strategies (learning stylecomponents)?
Results
We now present the results of our research into portfolio as a dissertation model for pre-school and primary school teacher education. We start by answering the researchquestions as formulated in the various questionnaires and then go on, in our conclusion, toformulate an answer to our general research question as to whether the use of portfolio asa dissertation model has a greater effect on students’ capacity for autonomous learningthan the literature study with practical processing.
The RAS questionnaire
The differences between the experimental group and the control group were revealed bycovariance analysis (ANCOVA), where the pre-test scores were regarded as covariablewith regard to the post-test scores. We carried out an ANCOVA both for metacognition asa whole and also separately for the three components of metacognition (Table 13; forparameter estimates see Table 16).
When we take the pre-test scores (as covariable) into account as part of a comparisonof the post-test scores of the experimental and control groups, we note a significantdifference for metacognitive knowledge, metacognitive regulation and metacognition as awhole, but not for metacognitive development. We explain the significant differences asfollows. The pre-test was taken at the beginning of the academic year and the post-test atthe end, which means that the interval between these amounted to the entire third year ofthe course – the year in which students are prepared for graduation and autonomousfunctioning in professional life. Moreover, in the case of most of the other subjects – andnot least during the teaching practice – much attention is paid to students’ metacognitiveknowledge, regulation and development. When we examine this effect for the dissertationalone, students who opted for portfolio score significantly better than students who optedfor the literature study with practical processing in the area of metacognition, andspecifically metacognitive knowledge and metacognitive regulation. It is noticeable that wecan observe these significant differences in metacognition, while the dissertation is onlyworth six academic credits out of a total of 60.
The ILS-HE
Here too the differences between the experimental group and the control group wererevealed via covariance analysis (ANCOVA) in which the pre-test scores were regarded ascovariable with regard to the post-test scores. We carried out an ANCOVA for the 16
378 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
Table
13.
ANCOVA
fortheRASscores.
Average
portfolio(pre)
Average
literature
study(pre)
Average
portfolio
(post)
Average
literature
study(post)
F-m
odels
p-m
odels
Metacognition
5.00(s¼
0.50)
4.92(s¼
0.51)
5.04(s¼
0.51)
4.83(s¼
0.48)
4.720
0.031*
Metacognitiveknowledge
5.03(s¼
0.59)
4.94(s¼
0.65)
5.01(s¼
0.53)
4.83(s¼
0.54)
4.351
0.038*
Metacognitiveregulation
5.18(s¼
0.57)
4.95(s¼
0.55)
5.01(s¼
0.60)
4.80(s¼
0.54)
4.162
0.043*
Metacognitivedevelopment
5.07(s¼
0.49)
4.94(s¼
0.51)
5.10(s¼
0.57)
4.85(s¼
0.57)
3.365
0.068
Note:*p5
0.05.
Educational Research 379
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
scales of the Inventory of learning styles for higher education (ILS-HE), in order to obtain apicture of the evolutions in the area of students’ cognitive processing strategies, regulationstrategies, learning orientations and mental learning models (Table 14; for parameterestimates see Table 16).
When we take the pre-test scores (as covariable) into account in comparing the post-test scores of the experimental group and the control group, we observe three significantdifferences on the 16 scales. With regard to processing and regulation strategies – the twocomponents of the ILS-HE which best tie in with our general research question intoautonomous learning – we can observe no significant differences with this instrument. Thisis somewhat surprising given that these concepts depend heavily on metacognitiveknowledge and regulation with respect to which we did indeed find significant differencesusing the Reporting autonomous studying questionnaire (RAS), and it might be that theILS-HE, as an instrument, is not sensitive enough to detect the differences between theexperimental group and the control group.
With regard to learning orientations, we observed no significant differences, exceptwith regard to ambivalent orientation. The ambivalent orientation is the component whichis most closely associated with the undirected learning style and is regarded as a lessbeneficial learning orientation. Those students who opted for portfolio score better (inother words, they are less ambivalent) than students who chose the literature study withpractical processing. On the subject of mental learning models, we found two significantdifferences, in particular, with regard to stimulating education and learning together.Students who opted for portfolio made less use of these two mental learning models. Bothare the models which are most closely associated with the undirected learning style and areregarded in the context of the ILS-HE as less beneficial learning models. In relation tothese scales it should also be noted that they represent a negative loading, namely thatstudents with these mental learning models have a considerable need for encouragementand support, either from the teacher or from their fellow-students. This can therefore notbe regarded as a positive manifestation of ‘learning together’ as promoted by the social-constructivistic theory of learning.
Interference of the confidence in students’ capacity for self-direction
In our discussion of the supervision provided by supervisors from the research group asintervening variables, we encountered a difference between the experimental group and thecontrol group with regard to their confidence in students’ capacity for self-direction. Thesupervisors who supervised students with portfolios appeared to have significantly moreconfidence in students’ capacity for self-direction than those who supervised students withliterature studies with practical processing. It is therefore possible that the significantdifferences which we found with the Reporting autonomous studying questionnaire and theInventory of learning styles for higher education between the experimental group and thecontrol group can be attributed to differences between the supervisors with regard to theirconfidence in students’ capacity for self-direction rather than differences in the dissertationmodels, per se.
In order to check this, we factored in the confidence of the supervisors in the students’capacity for self-direction as a covariable. We therefore carried out an ANCOVA with twocovariables: the pre-test scores and the confidence of the supervisors in the students’capacity for self-direction (Table 15; for parameter estimates see Table 16). We did this forall the variables from the RAS questionnaire and the ILS-HE, and this revealed asignificant difference between the experimental group and the control group.
380 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
Table
14.
ANCOVA
fortheIL
S-H
Escores.
Average
portfolio(pre)
Average
literature
study(pre)
Average
portfolio(post)
Average
literature
study(post)
F-m
odels
p-m
odels
In-depth
processingstrategy
3.25(s¼
0.60)
3.08(s¼
0.70)
3.33(s¼
0.56)
3.21(s¼
0.62)
0.065
0.799
Step-by-stepprocessingstrategy
3.31(s¼
0.57)
3.25(s¼
0.59)
3.40(s¼
0.53)
3.43(s¼
0.51)
1.182
0.279
Concreteprocessingstrategy
3.33(s¼
0.67)
2.94(s¼
0.68)
3.47(s¼
0.62)
3.23(s¼
0.64)
0.385
0.536
Self-directedregulationstrategy
2.57(s¼
0.56)
2.44(s¼
0.60)
2.71(s¼
0.52)
2.72(s¼
0.59)
1.700
0.194
Externallydirectedregulationstrategy
3.21(s¼
0.49)
3.08(s¼
0.47)
3.26(s¼
0.50)
3.30(s¼
0.44)
2.317
0.130
Undirectedregulationstrategy
2.26(s¼
0.63)
2.24(s¼
0.64)
2.39(s¼
0.62)
2.43(s¼
0.65)
0.417
0.494
Certificate-driven
orientation
2.29(s¼
0.72)
2.67(s¼
0.75)
2.88(s¼
0.70)
2.98(s¼
0.67)
3.077
0.081
Profession-driven
orientation
4.46( s¼
0.42)
4.39(s¼
0.48)
4.51(s¼
0.53)
4.35(s¼
0.55)
2.571
0.111
Test-driven
orientation
2.89(s¼
0.99)
2.74(s¼
0.99)
2.96(s¼
0.95)
2.85(s¼
0.80)
0.099
0.753
Personalinterest
orientation
2.75(s¼
0.58)
2.60(s¼
0.53)
2.85(s¼
0.62)
2.77(s¼
0.52)
0.017
0.896
Ambivalentorientation
1.65(s¼
0.63)
1.89(s¼
0.67)
1.63(s¼
0.67)
1.95(s¼
0.66)
6.467
0.012*
Acquisitionofknowledge
3.33(s¼
0.52)
3.51(s¼
0.56)
3.40(s¼
0.58)
3.52(s¼
0.59)
0.003
0.957
Constructionofknowledge
3.53(s¼
0.54)
3.48(s¼
0.45)
3.58(s¼
0.50)
3.48(s¼
0.51)
1.882
0.171
Use
ofknowledge
4.17(s¼
0.44)
4.13(s¼
0.41)
4.19(s¼
0.49)
4.05(s¼
0.50)
3.208
0.075
Stimulatingeducation
3.06(s¼
0.75)
3.21(s¼
0.76)
2.88(s¼
0.78)
3.28(s¼
0.73)
10.212
0.002**
Learningtogether
3.15(s¼
0.71)
2.95(s¼
0.75)
2.95(s¼
0.78)
3.06(s¼
0.76)
5.413
0.021*
Note:*p5
0.05;**p5
0.01.
Educational Research 381
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
Table 15. ANCOVA including ‘Confidence in self-direction’ for significant scores from the RASquestionnaire and the ILS-HE.
F-models p-models
Metacognition 3.153 0.078Metacognitive knowledge 4.119 0.044*
Metacognitive regulation 3.096 0.081Ambivalent orientation 5.877 0.017*
Stimulating education 6.075 0.015*
Learning together 6.133 0.014*
Note: *p 5 0.05.
Table 16. Parameter estimates for significant ANCOVAs.
Parameter B p Partial Z2
Dependent variable: metacognition postIntercept (literature study) 2.04 0.000*** 0.21Metacognition 0.56 0.000*** 0.33Portfolio 0.14 0.031* 0.03
Dependent variable: metacognitive knowledge postIntercept (literature study) 1.83 0.000*** 0.16Metacognitive knowledge 0.61 0.000*** 0.34Portfolio 0.14 0.038* 0.03
Dependent variable: metacognitive regulation postIntercept (literature study) 2.46 0.000*** 0.29Metacognitive regulation 0.47 0.000*** 0.27Portfolio 0.15 0.043* 0.02
Dependent variable: ambivalent orientation postIntercept (literature study) 1.17 0.000*** 0.28Ambivalent orientation 0.42 0.000*** 0.18Portfolio 70.24 0.012* 0.04
Dependent variable: stimulating education postIntercept (literature study) 1.90 0.000*** 0.28Stimulating education 0.43 0.000*** 0.19Portfolio 70.34 0.002** 0.06
Dependent variable: learning together postIntercept (literature study) 1.22 0.000*** 0.17Learning together 0.62 0.000*** 0.35Portfolio 70.23 0.021* 0.03
Dependent variable: metacognitive knowledge postIntercept (literature study) 1.81 0.000*** 0.11Confidence in self-direction (covariate) 70.05 0.612 0.00Metacognitive knowledge 0.64 0.000*** 0.37Portfolio 0.15 0.044* 0.03
Dependent variable: ambivalent orientation postIntercept (literature study) 0.38 0.353 0.01Confidence in self-direction (covariate) 0.28 0.035* 0.03Ambivalent orientation 0.39 0.000*** 0.17Portfolio 70.23 0.017* 0.04
Dependent variable: stimulating education postIntercept (literature study) 2.61 0.000*** 0.15
(continued)
382 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
We still observe significant differences between the experimental group and the controlgroup with regard to metacognitive knowledge, ambivalent learning orientation and themental learning models ‘Stimulating education’ and ‘Learning together’. Furthermore,when the confidence of the supervisors in students’ capacity for self-direction is taken intoaccount, portfolio students emerge as having acquired more metacognitive knowledge, areless ambivalently orientated and less dependent on the support of supervisors or fellow-students. We may therefore attribute these positive differences to the portfolio dissertationmodel.
However, we also note that the significant difference between the experimental groupand the control group with regard to metacognitive regulation (and thus also formetacognition as a whole) is now no longer observable. This means that the degree ofconfidence on the part of supervisors in the students’ capacity for self-direction is animportant factor in the evaluation of metacognitive regulation. Although portfolio results,by definition, in greater metacognitive knowledge, whether students are actually able toapply this knowledge depends on the confidence of the supervisors in their capacity forself-direction.
Discussion and conclusion
This paper reports the results of a quasi-experimental study in which two dissertationmodels were compared: the traditional ‘literature study with practical processing’ and thealternative ‘learning portfolio’. The aim of our study was to examine which one of thedissertation models offers students a greater benefit, in particular, more success inachieving the central objective of the dissertation, which is the promotion of autonomouslearning. We used validated questionnaires focused on metacognition, self-direction,cognitive processing strategies, mental learning models and learning orientations.
Our research indicates that the learning portfolio results in greater metacognitiveknowledge and metacognitive regulation. Moreover, the unbeneficial ambivalent learningorientation and the disadvantageous mental learning models ‘stimulating education’ and‘learning together’ appear to decline among students undertaking learning portfolios.Therefore we can conclude that the learning portfolio is more successful in increasingstudents’ capacity for autonomous learning than the literature study with practicalprocessing. In line with our original reasoning, the learning portfolio is more likely tobridge the gap between higher education and the job market.
The dissertation only constitutes one-tenth (6/60) of the academic credits for the entirethird year of the course and its impact may, in proportion, be limited. Despite the
Table 16. (Continued)
Parameter B p Partial Z2
Confidence in self-direction (covariate) 70.28 0.078 0.02Stimulating education 0.46 0.000*** 0.20Portfolio 70.29 0.015* 0.04
Dependent variable: learning together postIntercept (literature study) 1.35 0.002** 0.07Confidence in self-direction (covariate) 70.03 0.848 0.00Learning together 0.62 0.000*** 0.36Portfolio 70.25 0.014* 0.04
Note: *p 5 0.05; **p 5 0.01; ***p 5 0.001.
Educational Research 383
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
relatively small share of the dissertation in the third-year curriculum, the learning portfoliostill increases students’ capacity for autonomous learning significantly. This result is verymuch in favour of the learning portfolio. The learning portfolio is clearly preferable as adissertation model to the literature study with practical processing. If a greater weighting,in terms of more academic credits, were given to the dissertation, increasing positive effectsof the learning portfolio may be expected. To improve higher education courses, oneoption is to shift to a new curriculum. Our research study shows that small-scaleinnovations, on the level of course modules, can also have a high impact. Implementingthe learning portfolio in Bachelor teacher education courses is an opportunity of this kind.
Another important piece of data which emerges from this study is the role of thesupervisors and, specifically, their confidence in the students’ capacity for self-direction. Itappears that the learning portfolio can only really come into its own when supervisorshave sufficient confidence in students’ capacity for self-direction. They only get the chanceto apply metacognitive knowledge when supervisors give them sufficient autonomy to doso. More broadly, we can say that students get more opportunities to increase theircapacity for autonomous learning in accordance with the extent to which supervisors givethem more opportunities to learn autonomously.
This means supervisors need to extend to students enough freedom and to direct theirsupervision towards autonomous learning. The inevitable doubts which the supervisorshave in doing this can only be compensated for by having sufficient confidence in thestudents. This points to what we may call the paradox of autonomous learning: studentswill only learn autonomously when they are given the freedom to learn autonomously.When helping students towards an autonomous learning process, supervisors musttherefore take care to ensure their own doubts about whether or not students are capableof self-direction do not lead them to take control of their learning process.
Future experimental research may determine to what extent the results of the study areconsistent with test data on the one hand, and the students’ perceptions of the learningbenefits on the other hand. Functional testing of the capacity of autonomous learning is achallenging problem to be tackled. Connecting our data with students’ perceptions is partof our future research plan. Futhermore, we plan to examine the effect of increasedacademic credits for the dissertation on both of the dissertation models, and extending ourresearch also to other Bachelor courses.
References
Acker, S., T. Hill, and E. Black. 1994. Thesis supervision in social sciences: Managed or negotiated?Higher Education 28, no. 4: 483–98.
Boekaerts, M., and R.-J. Simons. 1995. Leren en instructie: Psychologie van de leerling en hetleerproces [Learning and instruction: Pupil psychology and the learning process]. Assen: VanGorcum.
Butler, D.L., and P.H. Winne. 1995. Feedback and self-regulated learning: A theoretical synthesis.Review of Educational Research 65, no. 3: 245–81.
Cook, T.D., and D.T. Campbell. 1979. Quasi-experimental design: Design and analysis issues for fieldsettings. Chicago: Rand McNally.
Coorough, C., and J. Nelson. 1997. The dissertation in education from 1950 to 1990. EducationalResearch Quarterly 20, no. 4: 3–14.
Elshout-Mohr, M. 1992. Metacognitie van lerenden in onderwijsleerprocessen Metacognition oflearning in educational learning processes]. Tijdschrift voor Onderwijs Research 23, no. 2: 121–42.
Elshout-Mohr, M., and M.M. van Daalen-Kapteijns. 2003. Goed gebruik van portfolio’s incompetentiegerichte opleidingen [Correct use of portfolios in competency-oriented courses].VELON: Tijdschrift voor Lerarenopleiders 24, no. 4: 5–13.
384 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
Elshout-Mohr, M., M.M. van Daalen-Kapteijns, and J.E. Baltzer. 1990. Zelfstandig studeren in hetHoger Beroeps Onderwijs [Autonomous studying in higher professional education]. SCO ReportNo. 232. Amsterdam: SCO-Kohnstamm Instituut.
Elshout-Mohr, M., M.M. van Daalen-Kapteijns, and J.J. Meijer. 2001. Constructie van hetinstrument: Rapportage autonoom studeren [Construction of the Reporting autonomous studyinginstrument]. Amsterdam: University of Amsterdam, SCO-Kohnstamm Instituut.
Elshout-Mohr, M., J. Meijer, M.M. van Daalen-Kapteijns, and W. Meeus. 2003. A self-reportinventory for metacognition related to academic tasks. Amsterdam: University of Amsterdam,SCO-Kohnstamm Instituut.
Flavell, J.H. 1979. Metacognition and cognitive monitoring: A new area of cognitive-developmentalinquiry. American Psychologist 34, no. 10: 906–11.
Gunneng, H., and E. Ahlstrand. 2002. Quality indicators in final theses in higher education – acomparative pilot study. Linkoping: Linkoping University.
de Jong, F.P.C.M. 1992. Zelfstandig leren: Regulatie van het leerproces en leren reguleren[Autonomous learning: Regulation of the learning process and learning to regulate]. Tilburg:Gianotten.
Lunenberg, M., and F.A.J. Korthagen. 2002. De didactische cirkel doorbroken. Het bevorderen vanzelfgestuurd leren door leraren en lerarenopleiders [The didactic circle broken. Evaluation ofself-directed learning by teachers and teacher educators. Paper presented at OnderwijsOnderzoek Dagen, May 30, in Antwerp.
McAlpine, L., and J. Weiss. 2000. Mostly true confessions: Joint meaning-making about the thesisjourney. Canadian Journal of Higher Education 30, no. 1: 1–26.
Meeus, W., and L. Van Looy. 2005. Portfolio zonder blozen. Het instrument doorgelicht vanuit hetperspectief van de lerarenopleiding [Portfolio without blushing. The instrument explained from theperspective of teacher education]. Antwerp and Apeldoorn: Garant.
Meeus, W., L. Van Looy, and P. Van Petegem. 2005. Het portfolio als professioneel gericht eindwerk.Een onderzoek in de lerarenopleidingen kleuter- en lager onderwijs [Portfolio as a profession-oriented dissertation. A study of teacher education courses for pre-school and primary education].Brussels: VUBPRESS.
Mertens, D.M. 1998. Research methods in education and psychology. Integrating diversity withquantitative and qualitative approaches. London: Sage.
Mettes, C.T.C.W., and A. Pilot. 1980. Over het leren oplossen van natuurwetenschappelijke problemen[On learning to solve physics problems]. Enschede: Technische Hogeschool Twente, CDO/AVC.
Padmos, B. 2001. De scriptiesupporter, een doeltreffende aanpak van je scriptie [The thesis supporter,an effective approach to your dissertation]. Leuven: Garant.
Roelofs, E., and J. Visser. 1991. Leeromgevingen volgens ouders en leraren: Voorkeuren en realisatie[Learning environments according to parents and teachers: Preferences and realisation].Pedagogische Studien 78, no. 3: 151–68.
Shavelson, R.J., and N.M. Webb. 1992. Generalizability theory. In Encyclopedia of educationalresearch. Vol. 2, ed. M.C. Alkin, 538–43. New York: Macmillan.
Thomas, R.M., and D.L. Brubaker. 2000. Theses and dissertations: A guide to planning, research andwriting. London: Bergin & Garvey.
Van Dyck, M., G. Leigh, and R.A.J. Baks. 1991. Studievaardigheden en de kwalificatie voor HBO enWO [Study skills and qualification for HBO and WO]. Leiden: LICOR.
Van Hout-Wolters, B.H.A.M., P. Jongepier, and A. Pilot. 1995. Studiemethoden [Study methods].Groningen: Wolters-Noordhoff.
Van Looy, L., and K. Elias. 2001. Gediversifieerde simultane verschuivingen in school- enklaspraktijk als voorwaarde voor zelfstandig leren [Diversified simultaneous shifts in school andclassroom practice as a precondition to autonomous learning]. Nieuw Tijdschrift van de VrijeUniversiteit Brussel 14, no. 1: 7–20.
Van Petegem, P., and S. De Maeyer. 2000. Het ILS-instrument voor leerstijlen: Een methodologischekanttekening [The ILS instrument for learning styles: A methodological margin note]. Tijdschriftvoor Onderwijsresearch 24, no. 3–4: 248–64.
Van Petegem, P., and J. Vanhoof. 2002. Evaluatie op de testbank: Een handboek voor het ontwikkelenvan alternatieve evaluatievormen [Evaluation on the testing bench: A manual for the development ofalternative forms of evaluation]. Malines: Wolters Plantyn.
Veenman, M.V.J., J.J. Elshout, and J. Meijer. 1997. The generality vs domain-specificity ofmetacognitive skills in novice learning across domains. Learning and Instruction 7, no. 2: 187–209.
Educational Research 385
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014
Vermetten, Y. 1999. Consistency and variability of student learning in higher education.Unpublished thesis, Catholic University Brabant.
Vermunt, J.D.H.M. 1992. Leerstijlen en sturen van leerprocessen in het hoger onderwijs – Naarprocesgerichte instructie in zelfstandig denken [Learning styles and the direction of learningprocesses in higher education – towards process-oriented instruction in autonomous thinking]. Lisse:Swets & Zeitlinger.
Vermunt, J. 1997. Leeractiviteiten van studenten [Students’ learning activities]. In OnderwijskundeHoger Onderwijs. Handboek voor docenten, ed. G. ten Dam, H. van Hout, C. Terlouw, and J.Willems, 30–45. Assen: Van Gorcum.
Wang, M.C., G.D. Haertel, and H.J. Walberg. 1990. What influences learning? A content analysis ofreview literature. Journal of Educational Research 84, no. 1: 30–43.
Winne, P.H., and D. Jamieson-Noel. 2003. Self-regulating studying by objectives for learning:Students’ reports compared to a model. Contemporary Educational Psychology 28, no. 3: 259–76.
Zeichner, K., and S. Wray. 2001. The teaching portfolio in US teacher education programs: What weknow and what we need to know. Teaching and Teacher Education 17, no. 5: 613–21.
Zimmerman, B.J. 1994. Dimensions of academic self-regulation: A conceptual framework. In Self-regulation of learning and performance, ed. D.H. Schunk, and B.J. Zimmerman, 3–21. Hillsdale,NJ: Lawrence Erlbaum.
386 W. Meeus et al.
Dow
nloa
ded
by [
Uni
vers
ity o
f A
uckl
and
Lib
rary
] at
13:
08 0
6 N
ovem
ber
2014