Interactive storytelling for children education

Chiara Evangelista, Veronica Neri, Massimo Bergamasco PERCRO

Scuola Superiore Sant’Anna Pisa, Italy

Marcello Carrozzino IMT

Institute of Advanced Studies Lucca, Italy

Abstract — In latest years there is a consisting trend in creating educational opportunities for schools increasingly rich and involving for children. Constructivism and technology represent an important combination to define innovative educational courses. Storytelling is the fundamental link to convey the use of technology in didactic paths purposely conceived and structured for children, exploiting their natural fondness to use elements of their everyday culture and communication. This project paper presents our experience in setting up and using a technology-based communication methodology for learning, so as to let children discover about drawing, one of the most ancient communication techniques. Starting from texts and stories, selected based on their age, children learn how to interpret and translate into images the content they wish to communicate, building the storyboard of their digital tale, and using their creativity on texts, images and other components in order to realize a multimedia application.

Keywords-component; digital storytelling, children, edutainment, interaction, drawing techniques

I. INTRODUCTION Several recent didactic approaches are valuably using

storytelling as an educational tool. A digital story is an experience, or an event, told through technological means. A set of guidelines has been setup to assist the production of these tales. Joe Lambert (director of the Center of Digital Story Telling at Berkeley) identifies seven main elements [1] in the structure of a story: Point of view (stories should be personal and “authentic”), The Dramatic Question (stories should tell something worthy), Emotional Content (stories should involve people), The Gift Of Your Voice (a way to personalize stories), Soundtrack (which helps understanding and adds aesthetical value), Economy (very often stories can be told with few words, images and music: let the “implicit” talk, use metaphors), Pacing (good stories “breath”). Using digital storytelling for educational purposes has a number of positive facets: when using technological tools together with a good pedagogy, kids are stimulated and their enthusiasm is awaken. Moreover, technological means are very attractive themselves for children, who are strongly encouraged into using them to communicate their experiences [2].

There is a great deal of projects carried out in this context; a thorough selection is hosted in [3]. In particular, in [4] it is explained how digital storytelling may literally hook children into writing, offering also useful insights on the teacher role. In [5] the involvement of family in developing multimedia tools

(in this case, e-portfolios) in early childhood education is explored.

The teaching procedure we want to set up has two main goals: the first, obvious one is the understanding of technologies and of the underlying scientific concepts. The second is to develop individual creativity. It is interesting in this context to recall the work of J. Ohler [6], introducing storytelling in order to exhort kids to tell about themselves and their personal story. Another interesting experience is presented by Judy Robertson in [7], where children are given the opportunity to tell stories through a computer game, and to develop narrative skills such as character creation, plot planning and interactive dialogue writing.

This paper presents our experience in setting up and using a technology-based communication methodology for learning, so as to let children discover about drawing, one of the most ancient communication techniques. The project is addressed to children of Primary School (6-10) and Secondary School (10-13). In this process, we have considered expression capabilities and artistic sensitiveness as the fundamental issues to build the digital tale.

II. PREVIOUS WORK In recent years many educational and edutainment software

applications have been developed, which exploit interactive technologies like video-games to recreate colorful, attractive and exciting environments fertile for the children’s learning process. Many of these applications are based on the constructivist method, an approach that nowadays is gaining more and more consent. The principle behind constructivism resides in the fact that every single individual builds his own cognitive structures drawing, directly from the external world, data which he/she reads, transforms and organizes [8]. Edutainment programs count many different types of applications, from interactive games based on novels, movies and children stories, to software with a more didactic purpose, like those involved in the learning process of school subjects.

Our teaching procedure hence intends to use the "game" metaphor as an instrument to stimulate curiosity and children motivation. Voice, sound, and artistic expression are instruments that allow children to express themselves and their creativity in a spontaneous and funny way. With this process we use technologies to digitalize voice, drawings, music, and other components, producing a new communication medium which allows to amplify or simply give a different emotional impact. As McLuhan claims, "the medium is the message"[9].

2009 Conference in Games and Virtual Worlds for Serious Applications

978-0-7695-3588-3/09 $25.00 © 2009 IEEE

DOI 10.1109/VS-GAMES.2009.22

198

2009 Conference in Games and Virtual Worlds for Serious Applications

978-0-7695-3588-3/09 $25.00 © 2009 IEEE

DOI 10.1109/VS-GAMES.2009.22

198

2009 Conference in Games and Virtual Worlds for Serious Applications

978-0-7695-3588-3/09 $25.00 © 2009 IEEE

DOI 10.1109/VS-GAMES.2009.22

198

2009 Conference in Games and Virtual Worlds for Serious Applications

978-0-7695-3588-3/09 $25.00 © 2009 IEEE

DOI 10.1109/VS-GAMES.2009.22

198

In PERCRO, a Virtual Reality laboratory of Scuola Superiore Sant’Anna, Pisa (Italy), we have been developing a consolidated experience in this kind of applications. A collaboration is ongoing with some Italian museums for the realization of a series of educational games which will take place inside the museum structure itself. One of these games, “Cubetti”, consists in the interactive reconstruction of a cube-like puzzle, where the final image is indeed a painting contained in the same museum. Using this particular approach, children are invited to observe and to play within the museum. The “Giocare con la Scienza” (Playing with Science) project, is dedicated to the world of scientific experiments. The application is based on the cartoon-like 3D reproduction of a real laboratory, where children can interact with all the objects needed to realize a series of experiments, organized in defined categories. The final application has been designed to be as versatile as possible, both as a support for a more detailed didactic material, and as an interactive game in which the constructivist method is applied.

Figure 1. “Little Prince” working sessions in the classroom

Another project, realized with two classes (Fig. 1) of an Elementary School in Lucca, was aimed to assist children in realizing a 3D interactive video-game based on Saint Exupery’s Little Prince. The educational course was organized into a sequence of steps, involving the following technical notions and skills:

• Basic computer science notions • Scanning techniques • Treatment of digital images (2D graphic software) • 3D geometry and modeling of 3D shapes • Digital sound • Digital data computing and storing

The first session has been dedicated to the basic notions of

computer science and to how all these concepts can be applied to create a cartoon showing some scenes from the Little Prince. Between the first and the second session the students prepared all the drawings. The final effect was very impressive due to the particular style that the art teacher suggested. The authors then selected some of their drawings and prepared the

storyboard. In the second session we introduced the scanning procedure and the concepts necessary to understand how a real drawing can become a digital image, and how the computer works to produce it. We also showed how the digital image can be modified to adapt it as a texture for a 3D model. The third session was devoted to 3D modeling. We showed how to build the various scenes and how to map their paintings as textures. The class then prepared the dialogs to be added as a soundtrack in the cartoon. The fourth session was dedicated to digital recording of the dialogs (with some short theoretical explanation). On this occasion, the students had fun playing the various roles of the story, trying to improve diction and acting. Then it was shown how sounds become computer files. In the fifth session some examples of animation were directed by the students. The students were enthusiastic to hear their voices and to see their paintings playing in the scene. The final result of all this work is what we can be defined a 3D interactive storytelling experience (Fig.2)

Figure 2. The “Little Prince” multimedia application

III. THE DEVELOPING FRAMEWORK XVR [10] is a technology that PERCRO and VRMedia

s.r.l. developed to manage advanced multimedia and virtual reality interactive applications. One of the most interesting aspects of XVR resides in the fact that it is designed to natively support the insertion of the developed content into a web page. However, XVR is mainly devoted to programmers, therefore we had to develop an external visual tool for the creation of interactive 3D environments, easy to use and flexible at the same time, in order to be used also by non–programmers. We chose an approach based on the Movie philosophy. The application is seen as a multimedia opera subdivided into scenes, as this fits well in the intent of recreating a story (not necessarily in a classical linear mode). The scene is the basic constitutive element of the application, the single brick that forms the entire movie, and the storyboard is the structure that binds all the scenes together, as it happens in a real movie. Within the scene it is possible to identify a stage, i.e. its setting, and the elements, i.e. the constituting objects (3D models, texts, images, videos, etc.). It is possible to associate actions and to define different interaction modes. Like a real virtual

199199199199

set, we can put lights, cameras and characters together and we can start timed animations. This way of relating to the application easily permits to manage a multimedia product that is based on storytelling, or in any case the development of a narration scheme that relates to scenes or chapters. One of the potentials of such a system, compared to the classic storytelling media, clearly resides in the interactive possibilities it offers, an approach that permits also to have a non-linear kind of narration. The authoring tool we have realized, XVR Story Builder [11], is based on the Movie approach just described, and designed to allow the creation of this kind of applications (even with a considerable level of complexity) without any need of programming skills.

IV. NEW TECHNOLOGIES TO LEARN DRAWING TECHNIQUES

Drawing is a technical and creative process which children learn in time. During the first years of their life, children approach drawing usually creating singular “scribbles”, tangles of colored lines representing at the same time gestures and words. In the course of time, guided by their experience and by their acquired knowledge, children pass to the so-called “intellectual” stage, learning how to control their gestures and managing to more freely express feelings and emotions using structured lines and signs.

Decoding the non-verbal language of children also means being able to enter their world and understand such in its multifaceted details, even when it requires meeting discomforts which the/ /children may experience: when drawing, in fact, children present their conflicts and anxieties clearer. Transposed on paper, emotional problems acquire concreteness and detachment, so as to let children look at them from a new perspective and consequently/,/ become more self-confident.

The project for the primary school we are presenting operates in this direction: it means to use classical art concepts and tools (both from an historical and a technical point of view) which, at the same time, are conveyed by means of technology. This will provide children several means of art expressions which they will consciously use to express and communicate concepts and emotions.

This course intends indeed to suggest how and which art technique to choose in order to communicate, based on the content to be transmitted. Therefore, the set goal is to divulgate the importance of art as a tool to express knowledge, and this will happen by means of and together with the first approaches to computer technology.

For kids of secondary school (10-13 years old), drawing is already a conscious communication process which, however, needs to be refined. Drawing can become an image-based means of communication useful only if well understood in its manifold expression capabilities. Technology may help a lot in this regard, as it manages to show hand-made artworks in a non-conventional way, and to speed-up the editing process, proving to more effectively convey information when tied also to multimedia

In both programs, the first step is devoted to a course of textual and iconographical analysis of a document (for instance

a poem). After an earliest in-depth reading of the selected text, children are invited to tell the plot of the tale using images. In this step traditional (ancient and contemporary) drawing techniques are also illustrated as well as printing, the first known instrument to duplicate step-by-step an artwork.

Children will then experiment by their own hands how people used to draw during centuries: how to perform a medieval miniature, how a drawing in Renaissance workshops was born, how eighteen-century techniques of pastel and watercolor worked. They are also shown all the known tools used for those purposes, and how it was possible to reproduce drawings on different scales and supports.

Every child has learned to draw in its earliest years, experimenting different techniques and tools. In this context, our aim is to make children more aware of existing tools and to show them the expressive differences that each approach offers, even mixing among them [12].

This will let them be able to effectively translate a written language into an iconographical language. After selecting the main moments of the initial tale, scenes and characters will be delineated in details by children using the drawing techniques they have just learnt. Subsequently they will realize their own drawings, using the tools able to produce the wanted visual result, in order to create a storyboard which will constitute the base of the final application.

Figure 3. Storyboard illustrating the main scenes of the 1st chant of Dante’s Inferno

The project is presented as a game, so as to stimulate technology learning and creativity in graphical representations. At the end of this first educational program, children will have acquired three basic concepts:

200200200200

• Critical analysis of a text, which leads to the decomposition of the original text into scenes.

• Text interpretation and corresponding graphical representation, where scenes are translated into images, identifying and drawing the most significant elements in terms of events, characters, environments.

• The visual language and its codes, i.e. the knowledge of the main drawing techniques, tools and actors, which leads to produce the storyboard of the multimedia application

The next step aims to provide computer basics, in particular dealing with how to create a multimedia application using the drawings realized in the first step.

Figure 4. Information Landscape presenting drawing techniques in Lucca art collections

Technological means will be therefore used to break the barrier of the physical space usually dedicated to the exposition of art works and tools. Interestingly, it is possible to exploit communication modalities [13] with a strong visual impact able to convey textual and iconographical information in an involving, entertaining and compelling way, easily understandable by a public of children and boys which are increasingly used to operate complex computer appliances and technological resources as videogames, Virtual Reality and Information Landscapes (Fig. 4).

Each protagonist will be able to re-elaborate his/her own images, creating a multimedia/multimodal application narrating a story.

V. CONCLUSIONS We presented a didactic project which makes use of digital

storytelling to structure an educational-artistic course for children of Primary and Secondary schools. Children, through the direct work experience with several art techniques and through the “multimedia processing” of the material they produce, undertake a learning path where they themselves are responsible of the process.

In our previous experiences we realized educational paths exploiting digital storytelling to let children be able to create a multimedia product, of which they are not only target users but also authors. This product was realized after a series of steps that allowed them to better understand how technologies work behind the scene. Moreover, this experience was also interesting to setup a protocol for transforming classical narration schemes into interactive environments. We expect the current project, along with these results, to provide them new elements and motivations to know better and love art, thanks to a technological frame where they play a leading role.

ACKNOWLEDGMENTS Authors want to acknowledge Andrea Baroni for his work

on the Little Prince application and in the development of the XVR Story Builder tool.

REFERENCES

[1] Lambert, J. 2006 2nd edit. Digital Storytelling: capturing lives, creating community. Berkeley,

[2] Barrett, H. (2006). Researching and Evaluating Digital Storytelling as a Deep Learning Tool. In C. Crawford et al. (Eds.), Proceedings of Society for Information Technology and Teacher Education International Conference 2006 (pp. 647-654). Chesapeake, VA: AACE

[3] http://digitalstorytelling.coe.uh.edu [4] Banaszewski, T. Digital Storytelling finds Its Place in the Classroom.

Multimedia Schools, 9 (1). [5] Barrett, HC 2004 Electronic portfolios as digital stories of deep learning:

Emerging digital tools to support reflection in learner-centered portfolios. Available from: http://electronicportfolios. org/digistory/epstory. Html

[6] Ohler, J. , Digital Stroytelling in the classroom, Corwinn Press 2008 [7] Robertson, J., An analysis of the narrative features of computer games

authored by children. In P. Brna (Ed.). Proceedings of Narrative and Interactive Learning Environments 2004 (pp. 33-41). Edinburgh, Scotland .

[8] Piaget, J. (1972). The Principles of Genetic Epistemology. New York, Basic Books.

[9] Mcluhan, M. , Understanding Media: The Extensions of Man, Cambridge, MA: MIT Press (original edition, 1964)

[10] Carrozzino, M., F. Tecchia, S. Bacinelli, M. Bergamasco (2005). “Lowering the Development Time of Multimodal Interactive Application: The Real-life Experience of the XVR project”. Proceedings of ACM SIGCHI International Conference on Advances in Computer Entertainment Technology ACE 2005.

[11] Baroni A. et al.: Building 3D interactive environments for the children's narrative: a didactic project. Advances in Computer Entertainment Technology 2005: 350-353

[12] V. Neri, “Segni ‘incisi’ sul web. La valorizzazione e la comunicazione della grafica a Lucca attraverso le nuove tecnologie”, Ph.D Thesis, IMT Institute for Advanced Studies, Lucca (Italy) 2008

[13] Ruffaldi, E., Evangelista, C., Neri, V., Carrozzino, M., and Bergamasco, M. 2008. Design of information landscapes for cultural heritage content. In Proceedings of the 3rd international Conference on Digital interactive Media in Entertainment and Arts (Athens, Greece, September 10 - 12, 2008). DIMEA '08

201201201201