PBL in Teaching Computing: An overview of the Last 15 Years

Armanda Maria C. Amorim Oliveira C[n - Centro de Informatica

UFPE - Universidade Federal de Pernambuco Recife, Brasil

amcao@cin.ufpe.br

Abstract- In computing courses, the teaching and [earning

approach normally emphasizes theoretical knowledge at

the expense of practical knowledge. The major

disadvantages of this approach are learners' lack of

motivation during class and their quickly forgetting the

knowledge they have acquired. With a view to overcoming

these difficulties, Problem Based Learning (PBL), an

institutional method of teaching, has been applied to

teaching computing disciplines. Despite the growth of the

practice of PBL in various disciplines of Computing, there

is little evidence of its specific characteristics in this area,

the effectiveness of different PBL methodological

approaches, or of benefits and challenges encountered. In

this context, this paper presents a systematic mapping

study in order to identify studies which involve best

practices when using the PBL method in Computing

between 1997 and 2011, answering five research questions:

"What are the main characteristics of PBL that support

teaching in Computing?"; "What are the criteria for

applying PBL effectively in this area?"; "How is the PBL

methodology applied?", "What are the advantages and

benefits of applying PBL in Computing? "and, finally, "

What are the main challenges about learning in PBL in

Computing?" . Keywords-component; Problem Based Learning; Education;

Computing. I. INTRODUCTION

[n the evolution of the Information Technology and Communication (lCT) industry, professionals in the field of Computer face constant changes which leads to there being a need to adapt concepts, methods and techniques for specific situations that may arise in the labor market. These issues affect not only businesses, but also software engineers, who besides having as a prerequisite vast knowledge of software development tools, should also have a broad view of the problem to be solved, ie, an understanding of business, entrepreneurialism and interpersonal skills [1].

To succeed in the field of Computer Science, one needs to have knowledge of and to master a variety of skills such as: being at ease with mathematics, logic, problem solving, algorithmic thinking and programming. Unfortunately, many students struggle to develop these skills, especially when the subject is related to object-oriented programming, discrete mathematics, data structures and analysis of algorithms [2]. This happens because of the way in which teaching is approached in undergraduate courses, since what can be seen is

978-1-4673-5261-1/13/$31.00 ©2013 IEEE

Simone C. dos Santos, Vinicius Cardoso Garcia C[n - Centro de Informatica

UFPE - Universidade Federal de Pernambuco Recife, Brasil

scs@cin.ufpe.br, vcg@cin.ufpe.br

a lack of practice after being given the theory [3]. Practice along with theory enables students to make connections, and thus to linl( prior knowledge to new knowledge learned during the new learning process [4, 5].

As a consequence of this factor, what also needs to be mentioned is that teachers get greatly frustrated and students abandon their courses, as many give up working in the area due to such difficulties [2]. When students do not give up, there is poor professional training, as they do not have enough practice for the labor market. [6]

With a view to finding solutions for these problems, the instructional method of teaching, Problem Based Learning (PBL), has been applied to the teaching of computing, aiming to promote collaborative and motivating learning, based on problem solving. According to Peterson [7], PBL meets three important criteria that promote ideal learning: first, it provides an environment where students are immersed in a practical activity, secondly, they can receive guidance and support both from other students and a responsible teacher/ tutor; and, finally, learning is based on solving a real problem.

Despite the growth of the practice of PBL in various disciplines of Computing, there is little evidence of its specific characteristics in this area, of the effectiveness of different methodological PBL approaches, or of the benefits and challenges encountered. It was out of this context, the problem tackled by this research arose which sought to understand the studies that highlight best practices for using the PBL method in disciplines related to the field of computing. To this end, the method of systematic mapping was used and focused on constructing a broad overview of how the PBL method was applied in the everyday routines of these courses from 1997 to 201 [.

II. PROBLEM BASED LEARNING [N TEACHING COMPUTING SCIENCE

Problem Based Learning (PBL) is an instructional method of teaching that differs from the traditional model by virtue of its using real-life practical problems to start the learning process in order to encourage the development of problem­solving skills. These problems based on reality are worked on in groups, in order to initiate, direct, motivate and focus learning, unlike traditional methods that put the problem at the end of the presentation of content [8, 9].

This approach began in the medical field, in the 1970s, at McMaster University's Medical School [[ 0], the precursor

being the physician and educator Howard Barrows who developed methods to instruct doctors to develop their own capacities for reflection when not in medical school [11]. The goal of this method is not only the solution of the problem, but rather learning from the problem presented, ie reporting solutions based on the problem and the process used to obtain them. Barrows [12] describes six key characteristics of PBL:

• Learning is student centered;

• Learning takes place in small groups of students;

• There is the presence of a tutor who is seen as a facilitator or guide;

• The problems are presented at the start of the process.

• The problems encountered are used as instruments to attain the knowledge and skills to solve problems;

• New information is acquired through self-directed learning.

In the process of teaching, teachers must understand correctly the problem as being the basis in the learning method, which differs from the traditional model of education, which is teacher-centered. PBL directs the role of learning to the students and the teacher's role changes to a single form of teaching: that of guiding, as well as that of teaching. [6]

In Computing Science, the difficulty lies in having good professionals in the area of information technology, viz., software engineers. PBL tries to overcome this difficulty by offering students a way to acquire knowledge and develop the skills and attitudes expected of a professional.

Software engineers face many problems of adapting concepts, methods and techniques to the specific situations of problems in the market, in general influenced by the variation in scope, cost and deadlines of software projects. For the professional training of software engineers, PBL approaches can help by fostering the ability to work in teams, to solve problems and also to encourage the development of one's skills and attitudes, such as self-directed learning skills, cooperation, ethics and respect for people's other points of view. [3]

Authors like Peng [6] discuss problems with reference to textbooks on programming, and state that part of the books are written with the main focus on the logical structure of knowledge, starting with the introduction of an abstract and unclear concept, for example, the concepts of "object", "property", "event" or "method", and thus do not respect the students' cognitive rules.

Bearing in mind the benefits of the PBL method, in different areas of knowledge, it is appropriate to make a more in-depth study with the aim of supplying a broad overview of research studies that that have been undertaken in the area of Computing Science and of analyzing gaps that may yet need to be explored in research on PBL

II I. S'ySTEMA TIC MAPPING The study of Systematic Mapping (SM) is a method

designed to provide a broad overview of a particular area of research that enables the results to be identified, quantified and

analyzed, thereby establishing evidence on a particular topic [13, 14].

The Systematic Mapping conducted in this study was divided into three stages, based on the guide by Kitchenham [13], namely:

1. Planning Systematic Mapping: drawing up a protocol containing all the information of the study;

2. Driving the Systematic Mapping: carrying out searches and collecting data;

3. Presenting Systematic Mapping: analysis of results and writing a report from the information stored.

The initial research studies were conducted using digital libraries: IEEExplore Digital Library, ACM Digital Library, Elsevier Scopus, Elsevier ScienceDirect. The selection of these sources was based on the relevance and credibility of articles indexed in the area of Computer Science.

Starting out from the objective of answering the central question of the study: "How is learning based on PBL characterized and what is its contribution and challenges for teaching Computing Science?", five secondary issues were put forward for answer:

I. What are the main characteristics of PBL that were found and supporting education in Computer Science?

2. What are the criteria for the effective application of PBL in Computer Science?

3. How is the PBL methodology applied in the teaching of Computer Science?

4. What are the advantages and benefits of applying PBL in Computer Science?

5. What are the main challenges about learning in PBL in Computer Science?

For the construction of the key terms of the research, the key-words of the central question were used. In addition to the key-words, their synonyms were identified based on the area of research, which facilitated the search: Learning (learner, education, educational program, constructivism, constructivist, instruction); Problem Based Learning (PBL, problem-based learned); Contribution (advantages); Challenges (outcome, results); Computer (software, computer engineering, information technology, computer science).

The search string was constructed by combining key­words and synonyms. Two operators were, OR (or) between the synonyms and AND (e) between the key-words, as shown in Table I

TABLE I. SEARCH STRING.

Search String (learning OR learner OR education OR methodology OR approaches OR "educational program" OR constructivism OR constructivist OR instruction) AND "problem based learning" OR PBL OR "problem based learned") AND (contribution OR advantages) AND (challenges OR outcome OR results) AND (computer OR software OR "computer engineering" OR "information technology" OR "computer science")

Initially, using the search string resulted in 2,464 studies being located. From that number, the process of selecting primary studiesbbeagan and was conducted first of all on the title and abstract. As a result, 2,412 studies wwere discarded from this total.

For the classification of the studies, inclusion and exclusion criteria wer designed, which are:

• Inclusion: the relevance that the study has in relation to the proposed research question: Studies that describe research related to the topic of the PBL teaching methodology and Computer Science; studies that relate experiences of the teaching methodology being favourable for teaching on Computer Science courses.

• Exclusion: Studies published in editorials, prefaces, summary articles, interviews, news and reviews; Studies with experimental results that are inconsistent or with incomplete content; Studies that are not part of the research area.

Based on reading the introduction and conclusion of the Relevant Studies and the criteria for inclusion and exclusion, a total of 52 Primary Studies was obtained. Of the 133 ostensibly Relevant Studies, 80 of them were considered "Not Relevant" and 12 were unavailable for download, as shown in Table II.

TABLE II. RESULTS OF THE SEARCH.

Primary Studies

1.1 Phase 2DC1 Phase

Title Introduction md md

'5 Abstract Conclusion

e � Excluded Included

� "" E " c< § �

:g �

>- •

�i a .:::

] E �

;5 .fal IEEE 1,242 69 33 36

SCOPUS 681 38 30 12

SCIENCE 32 7

DIRECT

ACM 214 23 15

TOTAL 2,464 133 80 12 52

During the process of analyzing the studies, they were classified according to their year of ublication year. Thus a count could be made of studies on PBL over the last 15 years. Figure I shows a graph of the the evolution of the PBL approach over the years.

18

16

14

12

10

• • • •

• • • • • • • • • • •

Figure 1, Distribution of results published over the years shown above,

As can be seen, it is clear that PBL approaches in the area of Computer Science have been growing since 2009, with a high incidence in 2010.

IV. MAIN RESULTS The studies found were associated with categories corresponding to the secondary research questions outlined in Section III. To prove each association, evidence was collected from each "Primary Study", represented in the following subsections by the notation "EP". The following subsections present and comment the results for each category. Due to size limitations of this article, only some evidence of the complete work will be mentioned.

A. Characteristics of PBL The purpose of this question was to map the characteristics

of the PBL method in the teaching of Computing Science. The topics were defined as per the characteristics of PBL defmed by Woods [15] and Barrows [16]. Figure 2 shows the primary studies qualified for each of these topics.

Topi c Primuy Study Qty-'%

ChaDlein the rolr ofstudmt EP _OS 1- j%

Collabor adoD amoog EP _03 , EP _07 2 - 10�� partidpaou

Figure 2, Results on Characteristics of PBL

In general, the authors are clear when they affirm the need for a problem that starts the process, and that all knowledge should be acquired through solving problems, thus enabling the student to have greater contact with real-world problems, as the evidence makes clear:

EP _II: "The problem-based learning method dictates that the

students' projects must aim at solving a problem,

EP _03: "In PBL students learn by addressing ill-defined and

open-ended problems and reflecting on their experiences, thus

developing problem-solving strategies and building domain

knowledge in a self-directed manner, "

Another important feature is the need for a change in attitude and role of teachers and students, in the latter, thereby stimulating the capacity for self-initiative:

EP _08: "" , teaching based on problem solving (in which

students have an active role in their learning) allow students to

understand difficult concepts better and retain the knowledge

acquired for a long period of time, "

EP 14: "The educator should use a problem to prompt

students' experiences and interests, and then help them to be actively

involved with filrther learning. "

EP _27: "the author takes the method of problem-based learning

and suitably arranges the course design, which with a good result not

only improves the quality of teaching, but also trains students'

capacity for self-learning, active exploration and mutual

cooperation, "

Finally, the studies highlight the characteristic of collaborative learning activities with emphasis on group learning as a key point of PBL:

EP _08: "The design project is structured around a formalized

base group which exists for the duration of the subject. The base

groups are selected at random from the different degree programs

and, in general, consist of three members with no two members from

the same degree program. "

EP _11: "It is clearly an advantage of the P BL curriculum that

students have ample opportunities to practice teamwork skills. "

B. Effectiveness

The purpose of this question was to research in primary studies which of them showed their effectiveness, ie good results as to applying the PBL method. The studies were divided into eleven (11) topics that diversify the effectiveness of the PBL process, as given in Figure 3.

Topic Primary Srudr Qty·%

IDtencUollS EP _36 1-3,57%

E-Leaming s uppontool s EP _08. EP _ll. EP_48 3-7 ,14%

Critia.l\ision EP _IS 1-3,57%

Better enlironment for the EP _13 1-3,57% lessons

Homrwon. EP _2S 1-3,57%

DenJoping games EP J9. EP_46 2-1 .14%

l"seofreal problmu EP_OI,EP_43,EP_44,EP _45 4-14,29"10

Group work EP _Ol, EP _34, EP_43 3-10. Jl%

l"seof other .pproaches EP_OI.EP_03.EP_2.4, EP _26. 10-35,?% EPJ9,EP_31,EP_3S . EP _40. EP_46,EP_41,EP_48

CODJtrucOOO ofapplintions EP _13, EP _36, EP_31, EP _44 4-14, 29"10

CompetitiolU EP_42 1-3,57%

Figure 3. Results on the Effectiveness in PBL.

Most studies (35.7%) are summarized into one topic, where the authors state that PBL is more effective when combined with collaboration strategies, such as collaborative learning, and constructing motivating applications, such as developing games, competitions or applications related to market problems, as highlighted in the evidence:

EP _ 01: "This article described 0 P BL mode under the guidance

of instruction system design, constructivism learning theory,

Brunner's discovery method and PBL pedagogy. "

EP _03: "Combining PBL with cooperative learning provides a

mechanism for students to maximize their own and other group

members' learning by working in teams to accomplish a common task

or goal."

EP _29: "In a game development project, students can develop

their own requirements for a domain in which they are experts. "

EP _ 42: "... develop a National RoboCode Competition both

authors saw potential benefits to enhance their PBL initiative and to

add an interesting challenge for their students.

EP _ 43: "The cases are written in close consultation with industry

practitioners to ensure that they resemble, as realistically as

possible, real-world business problems. "

EP _44: "I examined how student self-efficacy, as it relates to

being software development professionals, changed while involved in

PBL environment."

Other criteria were also mentioned, such as the formation of teams of group work, e-Iearning support tools, development of critical vision by the students and activities outside the classroom:

EP _34: "The team includes a group of people (about 5-6 people)

that work together to learn or work on a subject or project. They have

different backgrounds, such as academic disciplines, skills and

expertise, and working experiences. "

EP n: ".. in an internet-based learning environment, it is

considerably more efficient for students to exchange information and

opinions in comparison to the traditional classroom-oriented

learning environment, which indirectly encourages communication

among students. "

EP _18: "MALESAbrain encourages learners to judge or

critically evaluate the solutions posted by others before exploring

further knowledge-content. "

EP _28: "it is also necessary to leave homework, especially

experimental projects to students so as to consolidate their

knowledge learned in the classroom. "

C. PBL Methodology

The purpose of this question was to point how the PBL methodology is applied in the teaching of Computer Science. To answer this question, five elements considered essential in a PBL methodology applied to the teaching of computing were used, which, according to the authors Santos et al. [3] and [17], are problem, environment, content, people and processes. Figure 4 shows the topics and the number of primary studies associated with these elements

Topie P rim:;u·,. Study Q ry- o ....

En,"ironme.nl EP_O S ,EP_O�EP_09,EP_)(),EP_14>EP_I� 10-21,3% EP _3 1 , EP _32, EP _3(; , EP _39

Conie.nt EP Ol.EP 02EP 03.EP 04.EP 06.EP 01. 16-34, 4% EP =08 : EP =10: EP) 3: EP )4; EP )5 : EP )6; EP _25 , EP _28 , EP _4 0, EP _45

EP 03.EP 09.EP 10.EP IlEP 14.EP 15. 13-17,6�!' EP=2S: EP=29: EP)� EP)9: EP=4il:EP= 4� EP_43

Figure 4. Results on the PBL Methodology.

Among the criteria cited, the part of content was the most discussed, 34.4% of the primary studies doing so, in which the disciplines that adopted PBL methodology were cited: software engineering, robotics, embedded systems, information systems management, operating systems, digital systems, and the development and quality of software. Some pieces of evidence comment on the proposals of these disciplines:

EP _01: "The Software engineering course was taken as the

example to illustrate the implementation of the PBL model. "

EP _02: "Topics are introduced in a way such that students are

confident with what computer studies imply despite the use of the

robot. Robotics is only the "toy" for motivating beginner students on

the subject. "

EP _16: "In order to verifY the effect of PBL in online teaching,

we have made an experiment on the course of "Object-Oriented

Programming With Java" over the last 3 years: we recorded

students' performance, their feelings and the outcome of study; by

making comparison, we analyzed and summarized the strategy for

guidance, improved the strategies and teachers' ability in guiding

students. "

The criteria associated with the processes of the PBL method, for example, the steps of teaching and assessment methods used were also highlighted in 27.66% of the primary studies:

EP _03: "Student feedback is used extensively to evaluate the

performance of both the teaching staff and the subject. These student

evaluations consist of both formalized university-wide evaluations

(JCET) and informal evaluations conducted solely for this subject by

teaching staff associated with the subject.

Another important element was the environment, mentioned in 2l.3% of the studies, in which the most diverse environments that support teaching based on PBL were discussed:

EP _08: providing a richer learning environment, offering

students alternative ways of gaining knowledge and information,

enabling more accurate assessment, and individualizing practice,

feedback and reflection. "

D. Advantages and benefits of PBL

This question aimed to investigate the advantages and benefits of using PBL methodology in Computer Science. Various benefits were listed, divided into eight (8) different topics. The most discussed topic, with 38.24%, was that of the students' improvement in developing their skills. Also from the perspective of the students, other topics were highlighted such as the ease in solving problems, a better critical view on various subjects, motivated students mmotivated by the lesson and more participatory, self-directed learning, planning study time, being better prepared for the work environment work. From the perspective of the teacher, what stands out is the improvement in the practice of teaching. Fig.S summarizes all the topics, as well as primary studies associated with them.

Topit Prima ry Stndy

Impro't'UDea t m tl.e denloplDUlt EP _28,EP_2'9, EP_l1,EP _40, ortMtMpnctX:eorte� EP _42, EP _45

Impro,'IUtl,,-ut in the dln'Uopml!.of EP _02, EP _03, EP _04, EP _06, or.dcifu EP _01, EP _09, EP _10, EP _I S,

EP _24 , EP_25, EP_39,EP_42, EP_44

F acilty ill icleDrif,-iac.Dd.lOhiac EP_O I ,EP_22, EP_H,EP_3� .... b ..... EP_41

Be-fil!.r prepl.r l bo n (or ihe jobs EP _43, EP _44 marker

.\pp&atioa iD. otlau COURS EP _O S , EP_12

Po Dfu-e re.ecdbll:.k (rom the studH.b EP _19:EP_24: EP_35,EP3 7

Beth.l'! &reatupu1ici.ti.pation i.a. EP _16 d.D

Qt,.-."

6-11,65��

13-38,24%

5-14Jl�'D

2-5,88%

2-5,BBt::'D

4-11, 15%

1-2:94�'D

1-2,94%

Figure 5. Results on the Advantages and Benefits of PBL.

Among the pieces of evidence found, are those that stand out which reinforce these advantages and benefits:

EP 40: ".. the application of this methodology have enabled

students to receive all the theoretical concepts and, also, the students

have developed transversal skills such as writing, learning and oral

expressions in the same course, thereby developing a project

consisting of a semi-complex digital system. "

EP _07: "PBL skill is very important in learning. Students are

encouraged to use their present knowledge and skills to find the

answer in PBL. Assessment is ongoing and regular so as to provide

feedback that assists, extends and improves learning. The tutor is an

essential part of providing appropriate and constructive feedback

that is meaningful to students, supports and empowers their learning,

and contributes to their development. "

EP _ 44: "Twenty-seven students indicated that because of the

course they were ready to deal with the demands of actual software

development projects, even though some seemed surprised about

their new-found confidence. "

EP _24: "Student feedback indicated that the project was

preferred as a learning experience over traditional, lecture-based

methods. The product-based method maintained student engagement

and encouraged intentional learning tendencies. "

EP _ 16: "Class participation rate has increased by 73%, and the

rate of on-time handing in assignment by 36%, and the percentage of

significant questions increased prominently too. "

E. Challenges Identified

This question aimed to list the main challenges and problems encountered during the process of teaching / learning with PBL in Computer Science. Twelve (12) different topics that address challenges and problems were identified. Figure 6 shows the 12 topics and the number of primary studies associated with them. -----------------------------

Topic Primary Study Qty- %

."-pplic:.tioao (tMlIMtltocla&om" EP_09, EP_IO,EP_28 1 -13 ,O4�"

Difficu1tiu in cllidplinu EP_28, EP_11,EP_39 3-13,04%

Kao .... �eor_tIao�· EP _09, EP _35: EP _39 3-13,O4�"

Ti me EP_25, EP_28.,EP_40 3-13.04%

F " .... Ioood probkau EP _42, EP _43 2-8,70�"

RurictioDJOn the methDd�y EP_29 1-4)5�"

,-arieQ" orbo .... W;!r or 1M .st.ft.f:s EP_14 1-4)5�"

Dissemination 0 fknoll"ledge EP_12, EP_2S 2-8,70'"

U. of olMr .edlDOIocia EP_36 1-4,35%

Intul!.Ilt orlhi! .Iltndl!Dlsin Compnter EP_46 1-4,35% Seuce

DifIXuIQ. iatMobjedn'ISn:UIMd EP_Ol, EP_07,EP_ll 3-13,04% aad:ood ruaIb

Figure 6. Results on Challenges found in PBL.

According to the studies, the main challenges of using the methodology are the lack of belief in and knowledge of the methodology, deficiencies in the formation of basic foundations of Computer Science and difficulty in applying the methodology as highlighted by some of the evidence found:

EP _28: "It is di.fficult for a teacher to organize and control the

learning process when the number of students is large. "

EP _28: "Before taking this course, students generally have

learned object-oriented programming language(s) such as C, Java,

unfortunately they usually only know the language syntax but don't

have practical project experiences. "

EP _35: " ... in a PBL environment, these students have to climb a

stiff learning curve and overcome much resistance that might pose

challenges to students in the initial stage, and limit the potential

effects of PBL."

EP 25: ".. the traditional teaching method is more efficient in

knowledge of teaching and learning; in contrast, PBL may require

teachers and students to expend more time and effort. Under the

limited class hours, the teaching timetable may not be completed in

PBL. In this regard, we believe that some appropriate additions and

adjustments be made according to the actualnumber of class hours. "

EP _43: "Unfortunately, the adoption of a real-world project is

not always a feasible option. This is mostly due to difficulty in gaining

access to suitable real-world organizations and resource constraints

in managing the projects. "

EP _29: "What challenges educators the most is often not the lack

of teaching materials, but the constraints imposed upon a course.

These include (1) type of audience, (2) the limited amount of time

available for a prearranged set of course objectives, (3) a rigid

curriculum, and (4) the choice of teaching methods."

EP _14: "The variety of student backgrounds had the potential to

cause problems when teaching the module. "

Another important point refers to the learning process, since not all students learn equally. Also, some students take on different responsibilities, thus making teachers' evaluation of this difficult. As a result, teachers very often do not know how to assess students and hence the effectiveness of the methodology, by ensuring that students have achieved the desired goals, as the evidence stresses:

EP 28: "When students are organized as groups, their

capabilities may not be developed equally. Some play a dominant role

in the group, while others are there just to make up the number. "

EP _ 11: "Besides the fundamental challenge of creating a good

problem, educators are faced with the task of deciding how to

evaluate the technique's effectiveness and how to assess whether

students have met the overall learning objectives for the course.

V. FINAL REMARKS This paper has presented and discussed the results of a

systematic mapping, and has identified the main characteristics of having used PBL in the field of Computer Science from 1997 to 20 I I. After the initial assessment of 2,464 articles, 52 primary studies were selected tthat were relevant for the research. From five research questions, it was possible to find evidence in a satisfactory way which showed that most applications of the PBL methodology are geared towards its applicability in the disciplines of Software Engineering and Programming.

From all the evidence reviewed, it was moreover possible to conclude that the PBL methodology has been applied in Computer Science, and been widely accepted by teachers and students and facilitated improving the education and training of professionals by providing a broader view of the real problems in the area. However, there is still a need to overcome several challenges and, above all, to provide good results from the application of the methodology with a view to disseminating positive points of this method and to increase belief in it by both educators as well as the students who have no knowledge or understanding about PBL.

REFERENCES

[1] 1. R. Savery; T. M. Duffy. "Problem Based Learning: An Instruction Model and its Constructivist Framework". 1995.

[2] K. Martin; D. Chinn. "Collaborative, Problem-Based Learning in Computer Science" . 2005.

[3] S. C. Santos; M. C. M. Batista; A. P. C. Cavalcanti; J. Albuquerque; S. R. L Meira. "Applying PBL in Software Engineering Education". CSEET 2009, Hyderabad, India, 2009.

[4] H. Arksey; 1. O'Malley. "Scoping Studies: Towards a Methodological Framewrok". 2005.

[5] C. R. Coles. "Is problem-based learning the only way?" . 1991. [6] W. Pengo "Practice and Experience in the Application of Problem-based

Learning in Computer Programming Course". International Conference on Educational and Information Technology. 20 I O.

[7] K. Petersen. "Systematic Mapping Studies in Software Engineering". 2007

[8] R. G. Milter; 1. E. Stinson. "Educating leaders for the newcompetitive environment". 1995.

[9] 1. R. C. Ribeiro. "A aprendizagem baseada em problemas (PBL): Uma implementa9iio na educa9iio em engenharia na voz dos atores". 2005.

[10] D. Boud; G. Feletti. "The Challenge of Problem Based Learning". 1997. [II] R. Deslisle. "How to use problem-based learning in the classroom."

Alexandria, ASCD, 1997. [12] H. S. Barrows."Problem Based Learning (PBL)". 2001. [13] B. Kitchenham. "Guidelines for performing Systematic Literature

Reviews in Software Engineering". 2007. [14] M. Peterson. "Skills to Enhance Problem Based Learning". 1997. [IS] D.R. Woods. "Problem-based learning: How to gain the most from

PBL". 1994. [16] H. S. Barrows. "Problem Based Learning in Medicine and Beyond".

1996. [17] S. C. Santos and A. Pinto, "Assessing PBL with Software Factory and

Agile Processes", CATE, Naples, Italy, 2012.