learning goals in serious games
DESCRIPTION
Presentation for PSST IP summer school 2013, Linz Author: dr. Joze Rugelj, Assoc.Prof.TRANSCRIPT
![Page 1: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/1.jpg)
Learning Goals in Serious Game Specification
Jože Rugelj University of Ljubljana Faculty of Education Chair of Didactics of Computer Science
PSST IP Linz 2013 1
![Page 2: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/2.jpg)
Introduction § Characteristics of new efficient approaches
to education: Ø student centered Ø problem based Ø motivational Ø active Ø directed to higher ordered educational goals Ø supported by ICT
§ Serious games can integrate most of the characterstics mentioned above.
§ Take the simplest and most efficient technology! PSST IP Linz 2013 2
![Page 3: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/3.jpg)
Games and learning § Children’s play is important activity where they develop
skills for life regardless of age or level of development: Ø quick adoption to new circumstances Ø handling change with ease.
§ When child plays, she discovers concepts from real world and fundamental relationships between them are made.
§ Games provide motivation for learning, thus increasing the chance that the desired learning outcomes will be achieved.
§ But motivation is just prerequisite for learning. It is not enough!
PSST IP Linz 2013 3
![Page 4: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/4.jpg)
Serious Games § Serious games must
Ø have well defined learning goals and Ø promote development of important strategies and
skills to increase cognitive and intellectual abilities of learners.
§ Important elements contributing to educational values of games are sensual stimuli, fantasy, challenge and curiosity (desire to know or learn).
PSST IP Linz 2013 4
![Page 5: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/5.jpg)
§ In well designed serious games instructional content is blurred within game characteristics.
§ Students play the game and have fun, forgetting about the “learning” part of the experience. (Eventhough they are constantly presented with new concepts which they have to adapt in order to be successful in game.)
§ Important role of pedagogs and cognitive psychologists in definition of learning goals and didactical approaches in serious learning games design.
PSST IP Linz 2013 5
![Page 6: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/6.jpg)
§ The first step in development process is to determine what you want the learning outcomes to be! (e.g. affective change, behavioural change, productivity improvement, what will be result of game intervention, ...)
§ Learning goals are exprresed in terms of learning outcomes.
§ What people learn? (i.e. different types of knowledge:) Ø facts Ø concepts Ø rules Ø procedures Ø soft skills
PSST IP Linz 2013 6
![Page 7: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/7.jpg)
§ Different types of game design techniques to effectively convey knowledge.
§ Knowledge to be taught dictates the design techniques and game mechanics.
§ ! Mistake: Using same techniques to teach different types of knowledge
PSST IP Linz 2013 7
![Page 8: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/8.jpg)
§ Classification schemes == hierarchies for defining knowledge (Bloom's taxonomy): Ø lower levels prerequisite for upper Ø identification of type of content and the domains
§ Meno’s paradox: “How can you learn something you know nothing of?”
PSST IP Linz 2013 8
![Page 9: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/9.jpg)
Game design process - SADDIE Phases in production process § Specification of “didactical problem” § Analysis
curricullum, time, resources, technology, … § Design
content, “story”, graphical, feedback, … § Development § Implementation § Testing and Evaluation
PSST IP Linz 2013 9
![Page 10: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/10.jpg)
Specification and analysis § Identification of learning goals and outcomes
§ Analysis of learning outcomes (Kapp 2012): Ø type of knowledge required Ø didactical methods (instructional strategies) needed Ø game techniques
§ Types of knowledge to be learned (?taught): Ø declarative Ø conceptual Ø rule-based Ø procedural Ø soft skills
PSST IP Linz 2013 10
![Page 11: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/11.jpg)
Web portal for serious games at FE UL
PSST IP Linz 2013 11
§ hrast.pef.uni-lj.si/igre § hrast.pef.uni-lj.si/games
![Page 12: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/12.jpg)
Declarative knowledge § = factual knowledge = verbal knowlege § Any piece of info that can be learned through
memorization § Consists of
Ø facts Ø terminology Ø jargon, acronyms
§ Needed for higher levels, facts need to be learned first § Usually based on behaviourist learning theory
PSST IP Linz 2013 12
![Page 13: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/13.jpg)
Specification phase § It is mainly didactical task § Done by teacher or other expert for teaching/learning § Identifiaction of weak points in learning process that need
special support § Some general ideas about the solution of the identified
problem. § Analysis phase should bring more info and the framework
for the final product (i.e. didactic game)
PSST IP Linz 2013 13
![Page 14: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/14.jpg)
Methods for teaching facts § Elaboration
linking new info with previous info, showing relation of new facts with previous knowledge (context)
§ Organising placing facts into logical groupings, "chunking”
§ Association linking term to its image, linking term to its definition
§ Repetition
PSST IP Linz 2013 14
![Page 15: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/15.jpg)
Game techniques for declarative knowledge Stories § Brain has natural affinity to narrative construction.
Ø Facts can be better remembered when told in a story than presented as a list.
Ø Legal arguments are more convincing in a narrative tale than in a legal precedent.
§ Elaboration techniques for embedding facts in known context.
PSST IP Linz 2013 15
![Page 16: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/16.jpg)
Sorting and matching § Learner needs to place content into appropriate location
or connect two objects/facts (on factual level or above) Ø Example: Johnny’s World of Hardware
Replayability § Playing games more times, with some changing elements,
suports memorization. § Keeping content fresh, but still conected to facts that need
to be memorized. Ø Examples: Calculus drill games
PSST IP Linz 2013 16
![Page 17: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/17.jpg)
“Johnny’s World of Hardware” § Diploma thesis § The goal of the game is to help Johnny to repair
computer. § Learning goal:
Ø getting familiar with PC components § Mini games for collecting points
(repetition, matching) § Points needed to buy damaged
components (story) § Target audience: primary school
SEGAN project - Zaragoza 2012 17
![Page 18: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/18.jpg)
PSST IP Linz 2013 18
![Page 19: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/19.jpg)
Conceptual knowledge § Grouping of similar or related ideas, events or objects § They have common attribute or set of attributes
(ex: concepts of: free market, math proofs, virus, Trojan horse ...)
§ Instructional strategies Ø Methaphoric devices Ø Examples and non-examples Ø Attribute classification
§ Game techniques Ø Matching and sorting items or contents based on the attributes
of that items, not based on memorisation Ø Experiencing the concept players are learning about Ø Experiencing examples and non-examples of the concept
PSST IP Linz 2013 19
![Page 20: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/20.jpg)
“World of Variables” game § Diploma thesis § The goal of the game is to organize the logistics for delivery
of goods to some planets in the universe. § Learning goals:
Ø variables (types, declarations,…) Ø assignments
§ Target audience: primary school § Semantic model, independent
of syntax § Understanding of concepts
20 PSST IP Linz 2013
![Page 21: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/21.jpg)
Learning goals § Pupil after playing a game
Ø understands the idea of variable, Ø knows variable has a name, Ø is aquainted with with types of variables, Ø is aware of the fact that different types are not compatible, Ø knows that different types require different amount of memory, Ø is familiar with assignement statements, Ø is aware of meaning of both “sides” of assignment, Ø knows what is the value of variable after assignment, Ø can predict the values of variables after executing part of a
program (with several assignements).
![Page 22: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/22.jpg)
Game 1: Cleaning up the mess in the lab
![Page 23: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/23.jpg)
![Page 24: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/24.jpg)
Game 2: Carriers
![Page 25: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/25.jpg)
“Sick computers” game § Student project in the Use of ICT in Education course § The learning goal of the game is to explain how viruses,
worms and Trojan horses affect computers. § Target audience:
secondary school § Understanding of concepts § Game technique:
Doctor (player) talks with the patients and tries to diagnose their “disease”. Then he can prescribe appropriate medicine.
PSST IP Linz 2013 25
![Page 26: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/26.jpg)
26 PSST IP Linz 2013
![Page 27: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/27.jpg)
Didactic game about functions/procedures
![Page 28: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/28.jpg)
![Page 29: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/29.jpg)
TIME MESH game
PSST IP Linz 2013 29
§ Serious learning game for learning history topics of common interest, important for European history
§ EU LLP Comenius 2 years project, 7 partners from 6 EU countries (PT, ES, BE, UK, EE, SI)
![Page 30: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/30.jpg)
TIME MESH
PSST IP Linz 2013 30
![Page 31: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/31.jpg)
§ Adventure game, player moves back to history, using time machine (stories).
§ The game helps to imagine life in certain history periods and to get familiar with certain concepts (e.g. Enigma machine)
§ Player is active participant in historical events.
§ Option for collaborative playing § Can be used in a classroom as introductory activity or for
revision, or as a homework § http://www.timemesh.eu
PSST IP Linz 2013 31
![Page 32: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/32.jpg)
Rule-based knowlege § Rule is a statement that express relationship between concepts. § Rules provide parameters dictating a preferred behaviour with
predicatble results (e.g. etiquete). § Instructional strategies
Ø Providing examples Ø Role playing
§ Game techniques: Ø experience consequences of not following the rule, Ø simulate work tasks, Ø playing board games that outlines and provides the rules that
must be learned, Ø sorting or matching items according to rules to be learned.
PSST IP Linz 2013 32
![Page 33: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/33.jpg)
“Nick and Social Networks” game With the help of applications will make students familiar about the pros and cons of social networks. The player helps the main character Niko to increase its popularity so that the correct decisions relating to the use of social networks to collect points and consequentlly friends. Every correct decision increases the number of friends who want to attend his birthday party.
PSST IP Linz 2013 33
![Page 34: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/34.jpg)
The objectives of the game are that the student learns get familiar with social network, its strengths and weaknesses. It can also provide information and develop the right attitude toward the protection of property rights and privacy. The game is aimed at primary school children, who learn about internet safety and social networking.
PSST IP Linz 2013 34
![Page 35: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/35.jpg)
PSST IP Linz 2013 35
![Page 36: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/36.jpg)
Procedural knowledge § A series of steps that need to bo followed in particular
order to reach a specific outcome § Step by step instructions for performing a task § Instructional strategies Ø Start with a “big picture” Ø Teach “how” and “why”
§ Game techniques Ø Practice following the procedure in difficult situation (simulator) Ø Presenting challenge to overcome following the procedure Ø Experiencing procedure in various modes
(demo, tutorial, play mode/test, free play, ...) PSST IP Linz 2013 36
![Page 37: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/37.jpg)
“Forced landing” game
PSST IP Linz 2013 37
![Page 38: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/38.jpg)
Fiona and Computer Network § Student project in the Use of ICT in Education course § The goal of the game is to help Fiona to connect to the
Internet and to set up local area network § Learning goals:
Ø to set up networking components § Target audience: secondary school § Learning procedures needed to set
up a local area network
PSST IP Linz 2013 38
![Page 39: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/39.jpg)
Learning goals and goals of the game § Learning goals are usually not identical to goals of the
game § Computer games motivate via fun, instant visual feedback,
challenge, curiosity and fantasy, active participation, intrinsic and prompt feedback, challenging (achievable) goals and mix of uncertainty and open-endedness.
§ Achivements represent goals in the game. § They affect:
Ø performance, Ø motivation, Ø attitude.
PSST IP Linz 2013 39
![Page 40: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/40.jpg)
Measurement vs. Completion Achievements § Measurement Achievements (MA)
Ø MA given to player for completing task to certain degree. Ø Performace measured against other players, own performance or
standard set by game designers. Ø MA are evalutive in nature => can be linked to feedback
=>reflection increases perception of competence =>intrinsic motivation
§ Completion Acheivements (CA) Ø offered as an award for completening task Ø no assessment of player’s performance of task Ø performance contingent vs. non-performace contingent achiv.;
(latter requires no skills or efforts to complete) PSST IP Linz 2013 40
![Page 41: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/41.jpg)
Ø rewards can decrease players sense of authonomy => less intrinsic motivation
Ø getting a reward also decrease willingness for replay Ø players less likely take risks (to get a reward earlier)
§ Boring tasks need to be “paired” with extrinsic motivations (e.g. achievements) Ø intrinsic motivation not affected by rewards
§ Interesting tasks do not need achivements, learners just need feedback.
PSST IP Linz 2013 41
![Page 42: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/42.jpg)
Achievement Difficulty § Actual difficulty needs to be on a level, that is attainable
but challenging for players. (i.e. ZPD) => best performance and enjoyment
§ Player’s efficacy must be high enough for them to feel confident in attempting it.
§ Four factors affecting self efficacy: 1. level of expertise on the subject matter 2. vicarious* experience (e.g. using leaderboard)
*experienced in the imagination through the feelings/actions of another person 3. social persuasion: givng s.o. verbal boost (e.g. “well done”) 4. how a person feels: stress level, emotional condition, percieved
physical state. PSST IP Linz 2013 42
![Page 43: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/43.jpg)
Goal Orientation § Two types of goal orientation:
Ø performance orientation Ø mastery orientation
§ Performance orientation: Ø player concerned with others’ performance Ø typical for games Ø less in-game risks Ø less exploration Ø player typically perform better only with simple, non-complex tasks
§ For complex tasks (creativity, strategies) instill mastery orientation.
PSST IP Linz 2013 43
![Page 44: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/44.jpg)
§ Permanence and visibilty of achievements § Negative achievements
Ø do not punish player for failures Ø rather provide feedback within the system
§ Achivements as currency Ø currency for completening tasks Ø use currency to enhance game Ø it shoud not be the main reason for playing
§ Competitive vs. non-competitive (cooperative) achievements Ø advanced players can help less experienced Ø keep groups small Ø metrics should assess individual performance within group
PSST IP Linz 2013 44
![Page 45: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/45.jpg)
Use of games in classroom § Limited time for use of alternative teaching/learning
resources in formal education § Hints how to overcome this problem § Each of the presented methods has some positive effects
on teaching/learning
PSST IP Linz 2013 45
![Page 46: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/46.jpg)
Suggested methodologies § Game as a motivation before the lecture § Teacher playing game during the lecture § Game as a group activity in the classroom § Game as a home activity / independent learning
PSST IP Linz 2013 46
![Page 47: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/47.jpg)
Motivation before the lecture § The importance of motivation § The game environment can be used to focus the student’s
attention on what needs to be learned § Motivational elements
Ø Illustration of the era (time and space)
Ø Identification with the main character
§ Activities: Ø Teacher showing scenes as an illustration before the lesson
Ø Teacher showing parts of the game before the lesson
§ Recommendation: the activity should be followed by gameplay PSST IP Linz 2013 47
![Page 48: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/48.jpg)
Teacher playing game during the lecture § Still traditional approach, but with the attractive/multimedia
elements § Illustration of the teacher’s explanation of the theme § Motivational elements:
Ø Illustration of the era (time and space) Ø Identification with the main character
§ Activities: Ø Teacher playing parts of a game during the lesson Ø Teacher playing minigames during the lesson
§ Recommendation: the activity should be followed by gameplay
PSST IP Linz 2013 48
![Page 49: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/49.jpg)
Game as a group activity in the classroom § In collaboration with pupils from other locations § Constructivistic approach – not classical lectures § Motivational elements:
Ø Flow experience Ø Challenge of the game that should fit the ability of the player Ø Feeling of control of the situation Ø Clear, appropriate and immediate feedback Ø Peer interaction Ø Collaboration: responsibility for the learning outcomes of the group and
others § Activities: Playing the game in groups during the lesson § Recommendation: time consuming – plan enough time (min. 2
hours) PSST IP Linz 2013 49
![Page 50: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/50.jpg)
Game as a home activity § As a continuation of the first two methods § As a independent activity, but followed by analysis and
reflection in classroom § Motivational elements:
Ø Flow experience Ø Challenge of the game that should fit the ability of the player Ø Feeling of control of the situation Ø Clear, appropriate and immediate feedback
§ Activities: Ø Home activity followed by discussion/group work in classroom based on
the game, making joint conclusion, detecting and correcting misunderstandings
§ Recomendation: some initial background is needed PSST IP Linz 2013 50
![Page 51: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/51.jpg)
Project SEGAN (LLP)
PSST IP Linz 2013 51
![Page 52: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/52.jpg)
SEGAN - Community of Practice about Serious Games
§ It produces reports on the design, development, and evaluation of Serious Games and their use in specific contexts.
§ Network supported by virtual tools and face to face events. § SEGAN results also include:
Ø development of a repository with products and projects related to Serious Games
Ø setting up of small-scale, local events on the design and development of Serious Games
Ø setting up of a series of annual European conferences and Summer Schools
http://seriousgamesnet.eu PSST IP Linz 2013 52
![Page 53: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/53.jpg)
Serious games projects § Design of serious games is suitable for teacher
education. § Students at the Faculty of Education, UL, design and
implement serious games as their study tasks § The profile of graduates from “CS in education” § Different learning goals for students:
Ø analysis of all crucial elements, Ø identification of learning goals by teachers / curriculum, Ø definition of a didactical approach, Ø specification of technical requirements, Ø implementation, Ø testing and evaluation,
53 PSST IP Linz 2013
![Page 54: Learning goals in serious games](https://reader034.vdocuments.mx/reader034/viewer/2022042714/554951d2b4c90566498b4c26/html5/thumbnails/54.jpg)
Conclusions § Game can be very “powerful” instructional technology.
§ Its use can be justified by all relevant learning theories.
§ But it can only be efficient when it is properly designed and properly integrated into learning / teaching!
§ Very challenging topic in computer science teacher education!
PSST IP Linz 2013 54