Learning Programming with Peer Support, Games,Challenges and Scratch

Roberto A. Bittencourt, David Moises B. dos Santos, Carlos A. Rodrigues,Washington P. Batista, Henderson S. Chalegre

UEFS – State University of Feira de SantanaAv. Transnordestina, s/n, Novo Horizonte

Feira de Santana, Bahia, Brazil, 44036–900Email: {roberto,davidmbs,carod}@uefs.br,{wstroks,hendersonchalegre}@gmail.com

Abstract—Helping college freshmen to learn basic computerprogramming is a longstanding research topic. Various environ-ments, tools and languages have been developed to ease the initialsteps of novice programmers. However, to be used to their fullextent, such artifacts should be better combined with an appropri-ate learning approach. This work describes pre-term workshopsoffered to Computer Engineering undergraduate freshmen thatcombines the use of the Scratch learning environment with alearning approach based on peer support, game developmentand a strategy of challenge-response. Results are derived fromquantitative data analysis: surveys and artifact analysis. Studentsevaluated the workshop well: they found it stimulant, lightweight,with good teaching, useful, well-organized, and conducive tolearning. Results suggest the effectiveness of the approach toreduce initial difficulties for freshmen learning programmingconcepts.

Keywords—peer learning, programming, environments fornovices, Scratch, game-based learning, challenges.

I. INTRODUCTION

For years, undergraduate computing programs have facedthe challenge to teach programming to freshmen with no previ-ous background in the field [1]. A large variety of concepts andskills is usually required, ranging from algorithmic thinkingto expression in a formal language, from data structures tomastering an IDE [2]. To make matters worse, faculty typicallyadopt a formal approach that is usually detached from students’worries and preferences.

Previous work is prolific with alternative approaches toteach college introductory programming, especially with en-vironments geared to the novice [2], [3]. Among such en-vironments, Scratch is one of the most popular, originallyconceived to be used by both children and teenagers, andbased on design decisions that make the environment moretinkerable, meaningful and social [4].

Nonetheless, even the most well designed environment maybe misused if not accompanied by appropriate teaching andlearning approaches. To us, students benefit more from learn-ing environments when an active learning approach is used. Fortwo years, we have been perfecting an approach that combinesthe use of Scratch, peer support, a strategy of challenge-response, and game development. We have also applied thisapproach by means of freshmen workshops, usually a weekbefore the academic term starts.

In this paper, we describe one of our latest workshops,a one-week, 20-hour pre-class workshop with freshmen of aComputer Engineering program in a state university in Brazil,offered in August 2014. The workshop intended to overcomethe initial difficulties with programming that students face.Three sophomore peer students guided the workshop presentedto 18 freshmen. The challenges proposed led to the gradualdevelopment of an animation in Scratch, and three classicgames: Pong, Bow and Arrow, and Space Invaders. One lastchallenge was the development of a simple calculator to allowa smooth transition from Scratch to the C language. To evaluateour approach, we used a quantitative approach based on artifactanalysis, i.e., the projects students produced, and two surveys:one right after the workshop was over, and another one monthafter the term started, when students finished their first projectin the CS1 course1.

Students evaluated the workshop well in various criteria:they found it stimulant, lightweight, with good teaching,useful, well-organized, and conducive to learning. Studentsconsidered stimulant the active and playful learning approach.They also evaluated Scratch as having a friendly user in-terface and simple working logic, especially the Lego-styleblock fitting. The most used blocks were interface commands(Appearance), selection and repetition structures (Control), andmovement commands (Motion), which were also the onesstudents asserted as less difficult.

Even though most students were not using Scratch afterthe workshop, 70% of them asserted that the tool helpeda lot to learn programming. Especially regarding selectionand loop structures, 80% agreed that, to some degree, theworkshop helped them to develop their first project in theCS1 course. This seems remarkably relevant since these areconcepts usually hard to grasp in regular courses. On theother hand, the same did not happen with the use of variables.Though used in the games, they were not the main studentconcern when answering the challenges.

Results point to the effectiveness of an approach based ongame development, a challenge-response strategy, peer supportand programming environments for novices. The gathering ofempirical evidence in this work also contributes to the field,helping to better evaluate learning initiatives and tools.

This paper is organized as such: Section II discusses

1CS1 is the name usually given to the computer science introductoryprogramming course.

the background for this paper and related work. Section IIIdescribes the research methods used. Then, Section IV presentsresults, followed by a discussion in Section V. Conclusionsare derived in Section VI together with suggestions for futurework.

II. BACKGROUND

In this section, we describe related work on causes fordropout and failure in CS1 courses, on programming environ-ments for novices, and on guidelines and alternative initiativesin introductory programming courses.

A. Causes for dropout and failure in programming courses

Why is programming hard to learn? This question hasbeen thoroughly pursued by the community of computingeducation for around five decades. Robins et al. surveyedthe topic and describe differences between novice and expertprogrammers and between effective and ineffective novices.They also describe the complexity of the problem by devisinga programming framework based on knowledge, strategiesand mental models needed to design, generate and evaluateprograms [5]. Program design involves knowledge of planningmethods and algorithm design, strategies for planning andproblem solving, and models of problem domain. Programgeneration requires knowledge of languages, libraries andtools, strategies for implementing algorithms and for coding,and models of a desired program. Finally, program evaluationrequires knowledge of debugging and testing tools and meth-ods, strategies for testing, tracing and repairing, and modelsof an actual program.

A survey among 67 higher education institutions all overthe world reported a high variation in fail rates, reachingvalues as high as 60% [6]. Robins discusses the likely causesdescribed by the academic literature to distinguish studentsbetween programmers and non-programmers [7]. Factors rangebetween cognitive capacity, cognitive development, cognitivestyle, attitude and motivation. However, this author suggeststhat those factors do not actually predict student success,and proposes an alternative explanation, named learning edgemomentum effect, where acquiring one concept eases thelearning of other closely related concepts.

Another work investigates students’ decisions to quit theCS1 course in high school, with reasons ranging from lackof time to lack of motivation [8]. Those reasons are affectedby factors such as perceived difficulty of the course, generaldifficulties with time management and planning studies, orthe decision to prefer something else. Low comfort level andplagiarism also play a role in dropout levels. Finally, authorsexplain that dropout reasons accumulate, making efficientintervention require a combination of various different actions.

Lewis et al. use structural equation modelling to investigatethe effect of technical and soft skills (i.e., emotional intelli-gence) on the affinity (i.e., satisfaction with the CS major)and the intention to quit the CS major [9]. Unexpectedly, theyfound that technical skills were less important than soft skillsto predict affinity. They suggest incorporating soft skills in thecurriculum of computing undergraduate programs.

Part of the difficulties with learning introductory program-ming is related to both programming languages and develop-ment environments used in the learning process. Researchershave investigated those issues [2], [3], which we shortlydescribe in the following.

B. Programming learning environments

Kelleher and Pausch have developed a taxonomy of pro-gramming environments and languages for novices [3]. Twolarge groups of environments and languages are proposed:teaching systems, which aim to help people learn to program,and empowering systems, whose goals are to empower usersto build things tailored to their own needs. Inside each group,they classify environments and languages by their approach toeither ease learning or to empower users.

Guzdial describes the evolution of learning environmentsfor novice programmers [2]. He discusses three families ofprogramming learning environments: the Logo family, the rule-based family, and the traditional programming family. Heidentifies three research trends: a clear trend towards tooldevelopment with traditional language syntax; students preferto work on computational artifacts that have meaning to them;and environments and tasks should give students immediatefeedback on their work.

The trends identified by Guzdial led to the developmentof some popular programming learning environments, wherelearning happens by developing and playing games and an-imations in microworlds, such as in Scratch [10], Greenfoot[11], Alice [12], Robocode [13] and AppInventor [14]. Uttinget al. discuss the goals, mechanisms and effects of Scratch,Alice and Greenfoot, providing an interesting account of theaffordances of each tool [15].

Scratch, the tool we use in this work, is based on threedesign concepts: being more tinkerable, more meaningful andmore social [4]. It is more tinkerable because programminghappens by fitting programming blocks together, just likeblocks in Lego toys. It is more meaningful for its investment indiversity and personalization, allowing their users to create dif-ferent project types such as games, simulations and animations.Finally, it is more social, because of its social network thatallows sharing projects, creating new projects from previousones through mixing, and cooperating with pairs to developcollaborative projects. Scratch uses a kindergarten learningcycle that fosters creative thinking [16]. The cycle uses thesteps of imagining, creating, playing, sharing and reflecting toaid in the learning process. Immediate feedback from the tooland the use of the Scratch social network allows for the cycleto be performed easily.

C. Learning introductory programming in CS1 courses

Robins and colleagues argue that most CS1 courses ina conventional curriculum are based on a lecture and labapproach, and are largely focused on knowledge, especiallyknowledge of language features [5]. They recommend in-struction focused not only on language features, but also onprogram design and the combination of those features. Theyalso suggest addressing programming mental models, such asmodels of control, data representation, program design and

problem domain. Finally, they reinforce the pedagogical roleof lab activities as case-based problem solving sessions.

A systematic review of approaches to teaching introductoryprogramming, based on 60 intervention reports, shows thatalternative teaching interventions improve passing rates in CS1courses by around one-third on average, when compared to atraditional lecture and lab approach [17].

Pears et al. collect and classify the literature of threedecades of research on teaching introductory programming,identifying influential, synthesis and emerging work [1]. Theyclassify 45 selected papers into four categories: curricula,pedagogy, language choice and tools for teaching. On thepedagogy category, they divide the studies of programmingcourses into three emphases: problem solving, learning aparticular language, and code production.

Regarding assessment of alternative approaches in CS1education, it is useful first to resort to Gross and Powers’ work,which evaluates various assessments of novice programmingenvironments [18]. They classify assessment techniques asanecdotal, analytical or empirical, and then present an eval-uation rubric for the quality of such assessments, evaluatingthe earlier assessments with the rubric.

There is a huge amount of literature of experience reportsand assessments of alternative approaches for CS1. We focushere on some papers that uses the Scratch environment forhigher education. One experience with Scratch uses it duringthe first three weeks of a CS1 course [19]. In this course,Scratch replaces the typical course sequence devoted to algo-rithms and pseudo-code. Students expressed higher motivationin contrast with the regular courses. However, no changeswere found in either dropout rates or obtained scores whencompared to the baseline course. Another experience usesScratch during a two-week period to provide scaffolding fornovices to learn basic programming concepts, and a tool foradvanced learners to remain engaged through challenging work[20]. Both experiences are similar to ours in using Scratchto introduce programming concepts during a short period.Our approach, nonetheless, is different since our workshopshappen before classes start, are led by peers, and are based onchallenges and free interaction with the environment, insteadof regular classes.

III. METHODOLOGY

The research approach used in this work is the case study.A case study is a contemporary phenomenon delimited in spaceand time [21], [22]. In this approach, the researcher is inter-ested in investigating the phenomenon in a real environment,and in all its complexity.

A. Research Design

Our case was a five-day, 20-hour workshop of introductorycomputer programming for Computer Engineering freshmenundergraduates, which happened in August, 2014. The work-shop used a particular learning approach, based on peersupport, game development, a challenge-response strategy, anda playful programming environment for novices (i.e., Scratch).The phenomenon studied was the learning process developedduring this workshop.

Our data analysis approach is both exploratory and quan-titative. In an exploratory case study, one tries to answerresearch questions related to the phenomenon with no prede-fined hypotheses. The quantitative analysis intends to measurevariables inside the case that are related to the phenomenon.To do such, we surveyed participants and we also analyzedthe artifacts (i.e., software) they produced. We also performedqualitative data analysis, but the data are still under analysisand will be reported in a later work.

Our main goal was to understand the limitations andaffordabilities of the aforementioned learning approach, andthe learning issues that develop during the workshop.

Our research questions were four:

1) How do participants assess the workshop structure?2) How do participants assess the workshop learning

approach?3) How do participants assess the Scratch learning en-

vironment?4) Which skills and knowledge do participants acquire

in the workshop?

Finally, approval for this research was obtained from theInstitutional Review Board at the State University of Feira deSantana (UEFS).

B. Participants

Eighteen students participated in the research: 15 maleand 3 female. All of them were freshmen in the ComputerEngineering Undergraduate Program at the State University ofFeira de Santana (UEFS). All participants signed an informedconsent form, allowing their participation in this research.

C. Workshop Planning

Our workshop was based on previous workshops we havebeen offering since February, 2013. The present workshophappened one week in advance of the academic term, whenstudents are free from activities of regular courses. The work-shop was conducted during five consecutive days, four hourseach day, in a computer lab with one desktop computer perstudent. Students were invited during enrollment and thoughnotices at the UEFS Computer Engineering web site.

The main learning goal was that, after the workshop, stu-dents could understand and apply basic concepts of algorithmsand programming to develop small applications in a graphicprogramming language, thus, reducing potential difficulties inthe first regular course on programming (CS1).

The learning approach used in the workshop was based ona combination of:

• peer support: three sophomore students led the work-shop, helping participants to clear their doubts;

• game development: participants developed games inorder to learn programming skills;

• challenge-response strategy: tutors challenged partici-pants with small challenges during game development,instead of providing detailed explanations;

• learning environments for novices: the Scratch en-vironment was used as the main tool to learn pro-gramming, avoiding issues with language syntax andcompilation.

The challenges proposed led to the gradual development ofan animation and three classic games: Pong, Bow and Arrow,and Space Invaders (see Table I). One last challenge was thedevelopment of a simple calculator to allow for a smoothtransition from Scratch to the C language.

TABLE I. WORKSHOP SCHEDULE

Day Challenge Main Content1 Animation Motion, Looks2 Pong Motion, Control, Sensing3 Pong / Space Invaders Motion, Looks, Control,

Sensing, Operators, Variables4 Space Invaders / Bow & Arrow Motion, Looks, Control,

Sensing, Operators, Variables5 Bow & Arrow / Calculator I/O, Variables, Operators,

Conditionals, Loops

One undergraduate tutor conducted the workshop, and wasaided by two other undergraduate assistants. Initially, the tutorpresented a short overview of the Scratch environment. Then,she proposed building an animation by interactively exploringthe environment. Later, the tutor challenged the participants byshowing the Pong game, and then asked them to produce thatgame. Then, she started to propose small challenges to developparts of the game. Students actively explored the environmentto answer to the challenges. Whenever they needed, tutor andassistants cleared their doubts. The same process happenedwith the other games.

D. Data Collection and Analysis

We used both quantitative and qualitative data collectionprocedures in order to deepen and enrich the research. Thisstrategy is known as methodological triangulation. If theresults of different procedures lead to the same conclusion,the research provides a higher degree of confidence [23], [24].

We used surveys and analysis of source code as quantitativedata, and interviews and observations as qualitative data.However, we only describe quantitative results in this paper,since qualitative data are still under analysis. Two surveys wereconducted with participants: one immediately after concludingthe workshop (response rate of 72.2%, with 23% female and77% male), and a second survey one month after term begin,right after the students had turned in their first lab assignment(response rate of 55.5%, with 20% female and 80% male).Even though the second survey was conducted in loco, theresponse rate was low because research participation wasoptional. Average age in both surveys was 18.6, with a standarddeviation of 1.25 and 1.34, respectively, for the first and secondsurveys. Regarding the source code analysis, we examinedonly those kept in the computer desktops after the workshop,totalling nine student projects. From these, only four studentsfully participated in the five days of workshop.

IV. RESULTS

All the respondents of the first survey asserted that used theInternet every day. Regarding previous experience with pro-gramming, only one had thorough knowledge of the subject,

Fig. 1. Workshop Assessment

Fig. 2. Tool Assessment

while the rest had either no or few knowledge of it. None ofthem knew or had heard of Scratch before the workshop.

A. Workshop Assessment

We used a five-point scale between two extremes to letparticipants assess the workshop. For instance, one assessedcriterion was the boring/stimulant dimension. The more tothe left (right) the answer, the more boring (stimulant) theworkshop was. The other criteria are described in Figure 1.All criteria but one had average value above 4.5. The workloadwas considered a little tiresome by some students.

B. Tool Assessment

Participants also assessed the tool, i.e., the Scratch pro-gramming environment, as shown in Figure 2. The friendlyuser interface was one of aspects best assessed, both theinterface itself and the ability to immediately see code changeresults. Another relevant aspect was that the tool motivatedparticipants’ creativity. The playful way how coding blocksare fitted was also well valued. Few students had difficultiesto use the tool (23% of partial agreement), while very few hadissues with block fitting (8% of partial agreement).

C. Learning Approach Assessment

Participants assessed the learning approach used in theworkshop as well. Figure 3 shows participants’ opinions on

Fig. 3. Learning Approach Assessment

the learning approach we used. It is worth reminding thatwe provided only brief explanations of either the tool orof programming concepts, and we fostered participants tosolve challenges we posed. The use of challenges had 76%of agreement to some degree, while 23% disagreed. Theimportance of free time to explore the tool was unanimousamong participants. The adequate quantity of tutors availableduring the workshop also had unanimous agreement (38%partially agreeing, and 62% agreeing). Everyone fully agreedwith the use of games, although there was some variationaccording to the particular game. The animation with theScratch cat character was the less well assessed item. Whenasked to grade, between 1 and 5, how much the workshopmotivated their learning, 85% of the participants graded it with5 points.

We analyzed students’ multitasking behavior, rather com-mon in this generation [25]. Parallel access to the Internetfor goals unrelated to the workshop issues was almost zero:92% asserted that had not accessed the Net (only one studentrecognized using it a little). Listening to music, although notsignificant, was more common than Internet use. One studentsaid he had listened a lot of music, while 62% asserted notlistening to any music at all.

D. Assessment of the Knowledge Construction Process

The Scratch tool groups programming blocks by theme,one tab for each theme. The analysis of student answers aboutthe degree of difficulty to learn each theme showed that:Variables, Operators and Sensing had the highest degree ofdifficulty, while Control, Motion and Looks had the lowest,as shown in Figure 4. Nonetheless, in a scale between 1 and5 (1=none; 5=substantial), no theme had average degree ofdifficulty higher than 3. The themes of Pen and Sound werenot assessed, since they were not used during the workshop.

Table II shows the number of coding blocks used by eachstudent, organized by theme, for the Space Invaders game,the most complex game in the workshop. This game wasdeveloped in two days, but students 4, 5 and 7 were presentin only one day. On average, each student used 450 blocks.Looks, Control and Motion were the most used blocks.

Table III shows the features implemented by each studentin the Space Invaders game. Each row refers to one par-

Fig. 4. Scratch Theme Assessment

TABLE II. BLOCKS PER STUDENT IN THE SPACE INVADERS GAME

Mot

ion

Loo

ks

Con

trol

Sens

ing

Ope

rato

rs

Vari

able

s

Tota

l

Student 1 73 238 441 72 39 80 943Student 2 183 106 218 58 52 0 617Student 3 94 192 248 29 2 0 565Student 4 46 47 82 31 17 28 251Student 5 61 139 131 34 6 16 387Student 6 110 98 201 37 0 0 446Student 7 29 6 28 3 0 0 66Student 8 68 42 89 21 23 30 273Student 9 111 59 180 48 28 72 498Average 86.1 103.0 179.8 37.0 18.6 25.1 449.6StandardDeviation 45.6 75.6 121.5 20.3 18.6 31.3 252.2

Total 775 927 1618 333 167 226 4,046

ticular feature, while each column refers to one participant.For fully implemented features, we assign value 1. Partiallyimplemented features receive value 0.5. And unimplementedfeatures get zero. Averaging over the features, we noticed thatseven students had a game completion average of at least 0.7.Student 7, who missed one workshop day, was below average.Students 4 and 5 had average greater or equal to 0.7, eventhough they also missed one day.

TABLE III. FEATURES PER STUDENT IN THE SPACE INVADERS GAME

Stud

ent

1

Stud

ent

2

Stud

ent

3

Stud

ent

4

Stud

ent

5

Stud

ent

6

Stud

ent

7

Stud

ent

8

Stud

ent

9

Aver

age

Life andScore 1 0 0 1 0.5 0 0 0 0.5 0.3

GameOver 1 0 1 0 0 0 0 0 1 0.3

CannonMoving 1 1 1 1 1 1 1 1 1 1.0

AlienMoving 1 1 1 1 1 1 0.5 1 1 0.9

Alien’sCostumeChanging

1 1 1 1 1 0 0 1 0 0.7

Cannon’sCostumeChanging

1 0 1 0.5 0.5 0 0 0 1 0.4

CannonShooting 1 0.5 1 0.5 0.5 0.5 0 0.5 1 0.6

AlienShooting 1 1 1 1 0.5 0.5 0 1 1 0.8

Average 1 0.7 1 0.8 0.7 0.4 0.1 0.7 0.8 -

Computing the Pearson correlation for the average blocksper student and the average features per student, we get a valueof 0.69, showing strong correlation between both variables. Inthe Bow & Arrow game, correlation grows to 0.83. Obviously,quantity does not imply quality. But quantity of blocks isimportant, in this case, to successfully implement a feature.

Table IV shows the sum of blocks used for all games but theanimation. Results show 5,550 blocks for all the nine studentswhom we retrieved the source code. Sensing, Variables andOperators were the least used blocks, while Control, Looksand Motion were the most used. We also show the numberof days students have attended the workshop. The ones whocame every day used more than 700 blocks, while the othersused less than 600.

TABLE IV. BLOCKS PER STUDENT FOR ALL THE GAMES

Day

s

Mot

ion

Loo

ks

Con

trol

Sens

ing

Ope

rato

rs

Vari

able

s

Tota

l

Student 1 5 108 288 520 80 51 100 1147Student 2 5 232 184 337 98 64 21 936Student 3 5 150 240 341 52 3 13 799Student 4 4 68 47 107 36 25 34 317Student 5 2 80 147 155 37 6 16 441Student 6 4 146 125 253 53 1 0 578Student 7 4 58 30 84 13 2 2 189Student 8 4 99 70 142 35 27 34 407Student 9 5 153 107 282 70 40 84 736Average - 121.6 137.6 246.8 52.7 24.3 33.8 616.7StandardDeviation - 55.3 86.3 144.0 26.4 32.2 55.1 321.8

Total - 1094 1238 2221 474 219 304 5,550

Finally, to get an idea of the effects of the workshops inour CS1 course, which uses the C language, we surveyedparticipants right after they finished their first coding labassignment, one month after term begin. From 10 respondents,90% asserted that either have not used or have briefly usedScratch after the workshop. Regarding the C language, 70%fully agreed that enjoyed programming in that language, while40% agreed in some degree that had difficulties to code in thatlanguage (with 75% of these with partial agreement).

When asked whether the use of Scratch before the termhelped them to learn how to solve the first assignment, 70%asserted it did so, either reasonably or a lot. The more specificwe were, the more agreement there was: 80% agreed to somedegree about the learning of control structures, while 90% saidso for logic operators.

V. DISCUSSION

Here we discuss our findings, in the light of our goals andthe literature. Discussion is ordered by our research questions(RQ), established in III-A.

A. RQ1: How do participants assess the workshop structure?

In general, the workshop was well evaluated by the stu-dents, who considered it: (1) stimulating: in general, theykept quite entertained throughout the days, working on thechallenges proposed. Being stimulating is critical in that itis their first contact with the knowledge area that they choseto pursue a career in. Moreover, the current generation hasa low threshold for boredom. In this sense, another evidence

was the lack of multitasking behavior, reported by students,and observed by researchers. This type of behavior, whichmanifests as holding two or more usually unrelated complextasks either simultaneously or alternately [26]. Recurring workin the literature point out to disruptions in learning whensuch behavior arises [27]–[29]; (2) useful: students learnedskills that would be useful in the short term, in the CS1course they began to attend a few days after the workshop;(3) organized: workshop schedule followed as planned, asdescribed in Section III, along with well-defined goals, helpedthe fluency of activities with no setbacks, as perceived byparticipants.

Furthermore, the average workshop reviews of being con-ducive to learning and with good teaching by the peers weresignificant. Only the tiresome-light binomial scored a little lowwhen compared to the others, maybe because the workshoptook all morning shifts for five consecutive days. However,one should also consider that producing almost two games perday may have triggered some exhaustion, i.e., the workshopworkload may have been excessive. Various students failedto implement all the proposed features of all the games.Therefore, either removing one of the games or rethinking theschedule in a next workshop offering should be considered.

B. RQ2: How do participants assess the workshop learningapproach?

As previously stated, the workshop learning approach wasbased on a combination of peer support, a challenge-responsestrategy, game development, and a learning environment fornovices. All participants agreed, to some degree, that theapproach was adequate. With 18 participants for three tutors,we had a ratio of six students per tutor. Peer support is awidely used strategy in introductory programming learning,as reported in the literature: both formal and informal, inclassroom or in the lab, or even outside the classroom [30].Our approach is distinct from other experiences because theworkshop is made by students and for students. Two positiveconsequences follow: (1) integration: since the workshop hap-pens a week before regular classes begin, it ends up being anevent where freshmen get to know each other, and integratewith senior students; (2) sense of belonging: being next tofellow senior students allows freshmen to feel they belong inthat community.

The use of challenges was well received by students. Thegoal was that they explored both solutions and the environmentin a self-directed way. Tutors gave just short explanations, andintervened only when needed or requested. In any case, thatstrategy led more to the use of leading questions instead ofgiving straight answers.

Challenges are one of the foundations of active learning, awidely discussed topic in pedagogy [31]. An example of activelearning is problem-based learning (PBL), which has beenwidely used in computing courses [32], [33]. In PBL, chal-lenges take the form of a problem that resembles professionalpractice, arousing students’ interest on content knowledge, andstimulating self-directed learning, among other benefits [34].A difference from our approach to PBL is that although weencourage discussion among students, it happens only in a fewmoments, while in PBL, discussion takes most of the group

meetings devoted to solving problems. By the way, similaritiesbetween challenges and PBL are welcome in our context, sinceour Computer Engineering undergraduate program is stronglybased in PBL.

One benefit of using challenges is letting students free toadvance in their own pace. Sometimes they did more thanrequested, making them more proactive. We had, for instance,some students who took their project home and improved themwith more features. Such attitude was encouraged by the tutors.

Using games as challenges has proved a right decision.It may seem obvious, but the fact that participants ratedgames like Pong or Bow and Arrow more motivating than theinitial animation suggests that they prefer more complex tasks,provided that they are motivating. Challenges that assumea playful facet, such as games, are relevant because suchactivities are part of human beings’ history, and involve variouselements such as rules, goals, tension and competition [35] —also present in the workplace. Indeed, we may say that gamedevelopment is a challenging task, since it implies a substan-tial complexity in tasks of communication between objects,logical and mathematical operators, scenery, among others. Inaddition, the use of games adds at least two important featuresto the teaching-learning process: (1) everyday life: games arepart of youth daily life. Thus, working with something thatfascinates them, that is part of their reality, is likely to increasemotivation; (2) amusement: learning by playing minimizes theview of studying as obligation, reinforcing entertainment and,hence, enhancing learning [36].

C. RQ3: How do participants assess the Scratch learningenvironment?

The choice of Scratch as the learning environment forbeginners also facilitated the decision of merging games withchallenges. First, participants strongly agreed that Scratchwas favorable for game development and learning: a playfulenvironment for creating other playful environments. Second,immediate feedback on the screen was also much appreciated.Third, avoiding issues of compilation and syntax was alsowelcome. Those features led to a good assessment by par-ticipants. They also had no bigger issues with using the tool,besides considering it user-friendly and leading to creativitydevelopment.

We must also highlight that, for the aforementioned fac-tors, it is possible to build complex programs in Scratchin a relatively short time. An undergraduate student, newto programming, would likely take a long time to developgames such as Space Invaders using traditional programminglanguages, even at the end of the first term. But they did it afteronly a few days. It is worth noting that the workshop led to anaverage of 450 blocks used per student, for developing threegames. Participants who came to every workshop day usedmore than 700 blocks. If we equate blocks to lines of code,we realize how much code our beginning programmers haddeveloped in such a short period, especially when comparedto the code students develop in their first CS1 classes.

Nevertheless, after the workshop, students made it clearthat they did not keep using Scratch. Natural reasons for thiswere: they were then focused on learning a programminglanguage, had a high workload for the courses they were

registered in, and needed to adapt to a new student routine.Another reason was that their perception of Scratch changed:Scratch was now perceived as an amateur language, whileC felt as something more professional and, therefore, morerelevant.

D. RQ4: Which skills and knowledge do participants acquirein the workshop?

The most used blocks were from the themes of Looks(user interface), Control (selection and repetition structures)and Motion (for moving objects in the game). These areprecisely the themes that showed the least degree of difficultyfor participants. Control blocks, which are usually hard forbeginning programmers, were the most used blocks, whileblocks of Operators and Variables were the least used. It isworth mentioning that the less implemented features in theSpace Invaders game were counters (e.g., for life and score),even though the final results depended on these counters. Thismay likely have happened because counters require variables,one of the difficult themes according the students.

The theme Operators, which handles logical and mathemat-ical operators, had a greater degree of difficulty than the themeControl, responsible for selection and repetition structures.Interestingly, those structures are not usually easy to learn— hence the effort to offer this workshop — and require,a priori, a more elaborate logic than such operators. Workingin a playful manner with such structures may have facilitatedlearning.

In summary, we finished the workshop with 5,550 blocksused (given that only nine participants provided their projectsfor analysis), which allows us to infer that students havereally evolved in terms of introductory programming. Sensors,Variables and Operators were the topics less practiced, whileControl, Looks and Motion, the most commonly used.

Finally, despite students have not kept on using Scratch,we may say that the experience contributed substantially totheir learning of programming during the first month of theterm. 70% asserted that the workshop helped a lot in theprocess of knowledge construction of the first CS1 lab project.80% agreed, to some degree, that it helped on selectionand repetition structures, and 90%, on logical operators. Thisresult is significant, and it was precisely one of the workshopgoals, suggesting that the learning approach together with theadopted tool provided a more fluid and meaningful learning ofprogramming.

VI. CONCLUSIONS AND FUTURE WORK

This paper described a case study of a teaching and learningapproach aimed at facilitating learning of programming con-cepts by Computer Engineering undergraduate freshmen. Ourapproach combined the use of the Scratch learning environ-ment, peer support, a strategy of challenge-response, and gamedevelopment. The case developed as a workshop for freshmenthat lasted one week, with a workload of 20 hours. Threesophomore peer students guided the workshop, presented to18 freshmen. The challenges led to the development of an-imations, games and a calculator. To evaluate our approach,we used a quantitative approach based on artifact analysis and

two surveys, one post-workshop, and another after studentsfinished their first CS1 lab project.

Students assessed the approach well in some dimensions:stimulant, lightweight, good teaching, useful, well-organized,and conducive to learning. They also evaluated the Scratchenvironment as user-friendly, and with good working mechan-ics, especially the Lego-style blocks. Commands of interface(Looks), control structures (Control), and movement (Motion)were the most used, considered less difficult by the students.Although most students have not used Scratch after the work-shop, 70% of them agreed that the environment helped them tolearn programming. Regarding control structures, 80% agreedthat the workshop aided them in developing their first CS1project to some degree. This is relevant, for these are conceptsusually hard to learn in regular courses. On the other hand,variables were not grasped so well, even though they had beenused in the games. Results suggest the effectiveness of theapproach. Moreover, the work also contributes to the field ofcomputing education with empirical evidence, helping to betterevaluate learning approaches and tools.

Further work should deal with qualitative analysis of theseworkshops, based on interviews and observation. We alsointend to add a longitudinal dimension to the work, analyzingworkshops from different terms and their evolution.

ACKNOWLEDGMENT

We would like to thank the freshmen students who vol-untarily took part in this research. We also thank FAPESB– Bahia State Foundation for Research Aid, and UEFS –State University of Feira de Santana, for their funding ofundergraduate research fellowships.

REFERENCES

[1] A. Pears, S. Seidman, L. Malmi, L. Mannila, E. Adams, J. Bennedsen,M. Devlin, and J. Paterson, “A Survey of Literature on the Teachingof Introductory Programming,” in Working Group Reports on ITiCSEon Innovation and Technology in Computer Science Education, ser.ITiCSE-WGR ’07. New York, NY, USA: ACM, 2007, pp. 204–223.[Online]. Available: http://doi.acm.org/10.1145/1345443.1345441

[2] M. Guzdial, “Programming environments for novices,” Computer sci-ence education research, vol. 2004, pp. 127–154, 2004.

[3] C. Kelleher and R. Pausch, “Lowering the Barriers to Programming: ATaxonomy of Programming Environments and Languages for NoviceProgrammers,” ACM Computing Surveys, vol. 37, no. 2, pp. 83–137,2005.

[4] M. Resnick, J. Maloney, A. M. Hernandez, N. Rusk, E. Eastmond,K. Brennan, A. Millner, E. Rosenbaum, J. Silver, B. Silverman, andY. Kafai, “Scratch: Programming for All,” Communications of the ACM,vol. 52, no. 11, pp. 60–67, 2009.

[5] A. Robins, J. Rountree, and N. Rountree, “Learning and teachingprogramming: A review and discussion,” Computer Science Education,vol. 13, no. 2, pp. 137–172, 2003.

[6] J. Bennedsen and M. E. Caspersen, “Failure Rates in IntroductoryProgramming,” SIGCSE Bull., vol. 39, no. 2, pp. 32–36, Jun. 2007.[Online]. Available: http://doi.acm.org/10.1145/1272848.1272879

[7] A. Robins, “Learning edge momentum: A new account of outcomes inCS1,” Computer Science Education, vol. 20, no. 1, pp. 37–71, 2010.

[8] A. M. Azzam and Y. Career, “Why students drop out CS1course?” in Proceedings of the Second International Workshopon Computing Education Research, vol. 64, 2006, pp. 97–108. [Online]. Available: http://search.ebscohost.com/login.aspx?direct=true\&db=ulh\&AN=24666241\&site=eds-live

[9] T. L. Lewis, W. J. Smith, F. Belanger, and K. V. Harrington,“Are Technical and Soft Skills Required?: The Use of StructuralEquation Modeling to Examine Factors Leading to Retentionin the Cs Major,” in Proceedings of the Fourth InternationalWorkshop on Computing Education Research, ser. ICER ’08. NewYork, NY, USA: ACM, 2008, pp. 91–100. [Online]. Available:http://doi.acm.org/10.1145/1404520.1404530

[10] J. Maloney, M. Resnick, N. Rusk, B. Silverman, and E. Eastmond, “TheScratch Programming Language and Environment,” ACM Transactionson Computing Education, vol. 10, no. 4, pp. 1–15, 2010.

[11] M. Kolling, “The Greenfoot Programming Environment,” Trans.Comput. Educ., vol. 10, no. 4, pp. 14:1—-14:21, Nov. 2010. [Online].Available: http://doi.acm.org/10.1145/1868358.1868361

[12] S. Cooper, “The Design of Alice,” Trans. Comput. Educ., vol. 10,no. 4, pp. 15:1—-15:16, Nov. 2010. [Online]. Available: http://doi.acm.org/10.1145/1868358.1868362

[13] K. Hartness, “Robocode: Using Games to Teach Artificial Intelligence,”J. Comput. Sci. Coll., vol. 19, no. 4, pp. 287–291, Apr. 2004. [Online].Available: http://dl.acm.org/citation.cfm?id=1050231.1050275

[14] D. Wolber, “App Inventor and Real-world Motivation,” in Proceedingsof the 42Nd ACM Technical Symposium on Computer ScienceEducation, ser. SIGCSE ’11. New York, NY, USA: ACM, 2011,pp. 601–606. [Online]. Available: http://doi.acm.org/10.1145/1953163.1953329

[15] I. Utting, S. Cooper, M. Kolling, J. Maloney, and M. Resnick, “Alice,Greenfoot, and Scratch – A Discussion,” Trans. Comput. Educ.,vol. 10, no. 4, pp. 17:1—-17:11, Nov. 2010. [Online]. Available:http://doi.acm.org/10.1145/1868358.1868364

[16] M. Resnick, “All I Really Need to Know (About Creative Thinking)I Learned (by Studying How Children Learn) in Kindergarten,” inProceedings of the 6th ACM SIGCHI Conference on Creativity &Amp;Cognition, ser. C&C ’07. New York, NY, USA: ACM, 2007, pp.1–6. [Online]. Available: http://doi.acm.org/10.1145/1254960.1254961

[17] A. Vihavainen, J. Airaksinen, and C. Watson, “A Systematic Reviewof Approaches for Teaching Introductory Programming and TheirInfluence on Success,” in Proceedings of the Tenth Annual Conferenceon International Computing Education Research, ser. ICER ’14.New York, NY, USA: ACM, 2014, pp. 19–26. [Online]. Available:http://doi.acm.org/10.1145/2632320.2632349

[18] P. Gross and K. Powers, “Evaluating Assessments of NoviceProgramming Environments,” in Proceedings of the First InternationalWorkshop on Computing Education Research, ser. ICER ’05. NewYork, NY, USA: ACM, 2005, pp. 99–110. [Online]. Available:http://doi.acm.org/10.1145/1089786.1089796

[19] I. F. de Kereki, “Scratch: Applications in Computer Science 1,” inFrontiers in Education Conference, 2008. FIE 2008. 38th Annual, Oct.2008, pp. T3B–7–T3B–11.

[20] S. Mishra, S. Balan, S. Iyer, and S. Murthy, “Effect of a 2-week Scratch Intervention in CS1 on Learners with Varying PriorKnowledge,” in Proceedings of the 2014 Conference on Innovation& Technology in Computer Science Education, ser. ITiCSE ’14.New York, NY, USA: ACM, 2014, pp. 45–50. [Online]. Available:http://doi.acm.org/10.1145/2591708.2591733

[21] B. Kitchenham, L. Pickard, and S. L. Pfleeger, “Case studies for methodand tool evaluation,” Software, IEEE, vol. 12, no. 4, pp. 52–62, Jul.1995.

[22] J. W. Creswell, Qualitative inquiry and research design: Choosingamong five approaches. Sage, 2012.

[23] V. A. Thurmond, “The point of triangulation,” Journal of nursingscholarship, vol. 33, no. 3, pp. 253–258, 2001.

[24] L. A. Guion, D. C. Diehl, and D. McDonald, “Triangulation: Establish-ing the validity of qualitative studies,” 2011.

[25] D. Tapscott, Grown Up Digital: How the Net Generation is ChangingYour World. McGraw-Hill Education, 2009.

[26] D. M. B. dos Santos, “A convergencia tecnologica lıquida no contextoda sala de aula: um recorte do ensino superior publico baiano sob aotica discente,” Thesis, Federal University of Bahia, 2012.

[27] C. B. Fried, “In-class laptop use and its effects on studentlearning,” Computers & Education, vol. 50, no. 3, pp. 906–914,

Apr. 2008. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0360131506001436

[28] N. M. Aguilar-Roca, A. E. Williams, and D. K. O’Dowd,“The impact of laptop-free zones on student performance andattitudes in large lectures,” Computers & Education, vol. 59,no. 4, pp. 1300–1308, Dec. 2012. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0360131512001236

[29] S. Fulton, D. Schweitzer, L. Scharff, and J. Boleng, “Demonstratingthe impact of multitasking in the classroom,” in 2011 Frontiersin Education Conference (FIE). IEEE, Oct. 2011, pp. F2J–1–F2J–6. [Online]. Available: http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=6142991

[30] C. E. Wills, D. Deremer, R. A. McCauley, and L. Null, “Studyingthe Use of Peer Learning in the Introductory Computer ScienceCurriculum,” Aug. 2010. [Online]. Available: http://www.tandfonline.com/doi/abs/10.1076/csed.9.2.71.3811\#

[31] M. Prince, “Does Active Learning Work ? A Review of the Research,”Journal of Engineering Education, vol. 93, no. July, pp. 223–231, 2004.

[32] C. Beaumont, A. Sackville, and C. S. Cheng, “Identifying Good Practicein the use of PBL to teach computing,” ITALICS e-journal, vol. 3, no. 1,pp. 1–19, 2004.

[33] M. J. O’Grady, “Practical Problem-Based Learning in ComputingEducation,” ACM Transactions on Computing Education, vol. 12,no. 3, pp. 1–16, Jul. 2012. [Online]. Available: http://dl.acm.org/citation.cfm?id=2275597.2275599

[34] F. M. Munshi, E. S. A. E. Zayat, and D. H. Dolmans, “Development andUtility of a Questionnaire to Evaluate the Quality of PBL Problems,”South East Asian Journal of Medical Education, vol. 2, no. 2, pp. 32–40, 2008.

[35] J. Huizinga, Homo Ludens: A Study of the Play-Element in Culture.Beacon Press, 1971.

[36] M. Prensky, “Digital game-based learning,” Computers inEntertainment, vol. 1, no. 1, p. 21, Oct. 2003. [Online]. Available:http://dl.acm.org/ft\ gateway.cfm?id=950596\&type=html