JUST in Time Health Emergency Interventions: An innovative approach to

training the citizen for emergency situations using Virtual Reality Techniques and Advanced IT Tools (The Web-CD)

MANGANAS A., PhD1, TSIKNAKIS M., PhD1, LEISCH E., Dipl-Inf1, KAREFILAKI L., PhD1, MONSIEURS K., MD, PhD2, BOSSAERT L. L., MD, PhD3, GIORGINI F., PhD4 1Institute of Computer Science (ICS), Foundation for Research and Technology-Hellas (FORTH), STEP-C, Heraklion, Crete, Greece 2Emergency Department, Ghent University Hospital, De Pintelaan 185, 9000 Ghent, Belgium 3Intensive Care Department, University Hospital Antwerp, Wilrijkstraat 10, 2650 Edegem, Belgium 4GIUNTI INTERACTIVE LABS, Baia del Silenzio, Abbazia dell'Annunziata 16039 Sestri Levante (GE) Italy

This paper reports the results of the first of the two systems developed by JUST, a collaborative project supported by the European Union under the Information Society Technologies (IST) Programme. The most innovative content of the project has been the design and development of a complementary training course for non-professional health emergency operators, which supports the traditional learning phase, and which purports to improve the retention capability of the trainees. This was achieved with the use of advanced information technology techniques, which provide adequate support and can help to overcome the present weaknesses of the existing training mechanisms.

Introduction

It is a well known fact that in health emergency care the effective and timely intervention of “on-the-field” operators does not only save the lives of accident victims, but can also greatly reduce the harmful effects of injury. In a number of cases these injuries can be severe and can sometimes turn into permanent disabilities, which inevitably result in a great increase in both direct and indirect social costs. More precisely, the resulting high direct social costs stem from expenditure for medical treatment, subsequent rehabilitation and in many cases the need for extra assistance, whereas the high indirect social costs, can range from a simple absence from work, to the more permanent loss of employment due to the possible inability of the victims to perform their previous duties. Since in health emergency interventions both professional and non-professional health operators are involved, it is important to provide everybody with the proper training. This training must first of all make the trainees capable of recognizing the emergency situation at hand and secondly and most importantly, it must enable them to deal with the situation in a correct manner. However, in most European Countries, such proper training of the involved operators is still a critical issue. In general, the training of non-professionals mainly addresses three issues, namely those concerned with the content (i.e. “what to do” in the case of an emergency situation), the methodology (i.e. “what steps to take”, when faced with such an emergency) and finally with the improvement of one’s attitude (i.e. “the lowering of the psychological barriers”, in the presence of an emergency). In the traditional training methods, the trainee mainly acquires this third aspect after the end of the training course and whenever the trainee comes across a real-life emergency situation. In most cases, this psychological impact of accident scenarios on operators (such as the distress of the victim, the presence of blood and the behaviour of the patient’s relatives) is not tested in advance.

Furthermore, another very important issue with the current training methods is the one concerned with the “retention of knowledge”. According to Braslow et al. [1], “two months after the end of a traditional classroom instruction, only 36% of trainees were still rated competent in CPR”. Similar findings were also reported by Bossaert [2], who stated that “there is a sharply decreasing skills retention after 6, 12 and 24 months following a CPR course” and that “a refresher course is needed to restore initial skills”. Finally, Morgan et al. [3] stated, “6 months after the CPR course only 6.8% could perform safe and competent CPR”.

As a consequence of the above, one of the main objectives of the JUST project was to design and develop a complementary training course for non-professional health emergency operators, which supports the traditional learning phase, and which purports to improve the retention capability of the trainees. This was achieved with the use of advanced information technology techniques, which provide adequate support and can help to overcome the present weaknesses of the existing training mechanisms. The main features for this JUST training course were based on three aspects, namely those concerned with the capability of the individual for self-learning, the benefits

of interactivity between the system and the user and through the extensive use of multimedia. The result of these considerations was the employment of a hybrid technology solution, namely the Web/CD, which presents the optimal integration between the two media, even though there was an expected overhead for system and information design. Such a solution was deemed necessary for the user-friendliness, the consistency and the subsequent monitoring and updating of relevant information update and is believed to be a key issue for the market uptake of the resulting product. The training content of the Web/CD was based on current standards and recommendations, best practice guidelines, statistical data, and by taking into account the capabilities and restrictions of the target users of the system. Moreover, the content was also chosen on the basis of its potential for maximising its achievable impact from the use of the JUST Web/CD system. The training content has been designed by a User Group of recognised specialised physicians and was approved by experts in the field of emergency medicine training.

Finally, in order to verify the results of the JUST Web/CD system, four pilot sites were set up in Italy (San Martino Hospital in Genova), Greece (Medical School of the University of Crete), France (Hospital Raymond Poincare in Garches near Paris) and Spain (sites in Almeria, Jaen, Malaga and Sevilla overseen by the Emergency Medical System of the Andalusian Region). The system assessment was based on controlled clinical trials, which were performed in every pilot site. The outcome of these trials was the provision of a number of quantitative and qualitative indicators on the success of the project. Motivation and Human Factors The vision of the JUST project was to create systems that will contribute to a significant increase in the level of preparation prior to health emergency incidents through improved training. The expected improvement can be seen in terms of training frequency, training quality, assessment of trainees and cost effectiveness. As was already mentioned before, better training of key target groups can considerably improve mortality and morbidity rates in an emergency situation and can considerably reduce the social cost imposed by exaggerated rehabilitation needs and loss of productive hours. In this context, the JUST systems are designed to provide support for continuous education and training through the use of advanced information technologies, certified content and innovative Virtual Reality (VR) tools (see Manganas et. al. [4]). Furthermore, special attention was given to the European dimension through standardised (not normalized) training content, multilingual systems and cross-country evaluation at multi-national pilots.

As far as human factors are concerned, the JUST project approach was based on a requirements analysis of the type of medical emergency training that was necessary for the prospective end-users of the system. This was visualised in terms of training content, user profile and the current situation at a number of identified pilot sites. At the same time, following a thorough analysis of the needs and requirements of the potential end users of the developed system, the four target groups were defined. These were, the ambulance personnel (i.e. workers that operate within an organized system and have the appropriate training), the first responders (i.e. a group of people who have a duty to respond in emergency situations), doctors and nurses without specialization in emergency medicine (i.e. health professionals without a specific emergency medicine education, and finally, the ordinary Citizens (i.e. laypersons who have an interest in attending some first aid training course for their own personal benefit).

Having selected the previously mentioned target groups, it transpired that the learning needs of these users were somewhat different, as some of them had already been exposed to some form of medical knowledge. As a result, the decision was taken to support two distinct levels of training through the Web/CD, namely, Level 1 (Basic), which is especially targeted for the ordinary citizen, and Level 2 (Advanced), which covers the needs of ambulance personnel as well as the needs of doctors and nurses without specialization in emergency medicine. Furthermore, the decision was taken to create a system, which was compatible with the majority of the current market-standard computer platforms and devices, so that it could be installed on a large number of commercially available hardware. Finally, a number of important factors, which guarantee the best system performance and are essential for the educational value of the Web/CD system were taken into account and subsequently implemented. These factors were connected with the Interactivity, the Flexibility and Freedom, the Ease-of-use, the provision of Different Levels of Training and finally the Complimentarity of the Web/CD system. The JUST Web-CD System - Educational Approach

In order to describe the instructional approach of the JUST Web/CD it is essential to emphasize once again that during the user requirements phase it became apparent that the system was aiming to capture a non-homogeneous end-user group of people. In other words, there were two types of potential users of the Web/CD, namely those with no prior knowledge or exposure to medical information and those who already had some

exposure to medical knowledge and training but not in the field of emergency medicine. Therefore, the decision to create a system that supported one basic and one advanced level of training meant that from a pedagogical point of view a certain type of structure was needed. Especially for the multimedia content of the system, the assets were chosen and presented in a way that was appealing to both experienced and inexperienced users, i.e. to users that had or had not come across medical material beforehand. In doing this, the system’s teaching mechanism is enhanced, as the creation of familiar training environments and the accessibility of information in multiple and functionally adaptive ways helps immensely the learning processes of both types of users.

Based on the main goal of the Web/CD, essentially the development of a complimentary (to traditional courses) training course in health emergency training, the instructional purpose of the system is primarily based on the acquisition of manual and conceptual skills by its users. As a consequence, the general approach of the Web/CD is a constructivist one, i.e. an approach which aims at creating a learning process that is more centred on the practical experience than on the mere acquisition of knowledge. Moreover, the trainee can actively interact with the system and is given the freedom to choose his own learning path. However, this learning mechanism is always supported by the appropriate instruction manual, which means that a reference point is always present in the knowledge construction process.

The Web/CD is structured around two basic mechanisms: the Behavioural Algorithm, which is represented graphically by a flowchart; and the knowledge depository, which is represented by the Knowledge Book. The former is linked to a series of videos showing the correct actions and procedures in the presence of an emergency, whereas the latter is the encyclopaedic collection of training material. It must be stressed here, that these two mechanisms support in fact two alternative views of the same information: the encyclopaedic structure of the Knowledge Book forms a type of declarative knowledge representation, while the algorithmic structure forms a procedural representation of similar information.

Furthermore, the system is divided in a number of different areas, which allow the user to experience an increasingly active interaction with the system, starting from a semi-interactive presentation of the training material up to a very involved 3-D scenario in certain cases. As a result, the users start from a general introduction of the emergency topic that they wish to study, followed by a more involved “Step by Step” guide, which is an interactive presentation of the correct procedures that must be performed in an emergency situation. This sequence of events is enhanced by the use of appropriately made video clips, which show professional trainers to correctly perform certain required procedures during an emergency. From a pedagogical point of view this part of the system represents the starting point of creating the appropriate interaction between the user and the system. As a matter of fact the user is free to enjoy the video clips either sequentially or by randomly choosing the step to follow through the behavioural algorithm. Moreover, each video clip is linked to the Knowledge Book, which in turn provides extensive reference material on the topic.

Having familiarised themselves with the initial browsing of the course material of their chosen topic, the users can then proceed to interact with a more involving area of the Web/CD system, which is termed the “Learning by Doing”. This part of the Web/CD, which is based on Dewey’s doctrine (see Jonassen et. al. [5], and Woolfolk [6]), is the place where the users, besides having the possibility of another exposure to the course material, also have the opportunity of testing their understanding of the emergency procedures, i.e. their comprehension. The latter is achieved through a non-formal evaluation of the knowledge acquired thus far. As a matter of fact the system prompts the users to check their comprehension through this non-formal evaluation procedure, by offering them a series of questions, usually of the multiple-choice variety. Correct answers to the questions as well as explanations of them and hints for places where more knowledge can be found is instantly provided by the system. This kind of learning evaluation is an indispensable part of interactive training, as it is meant to help the trainee’s understanding (semantic encoding) of the material.

Having passed through the “Step by Step” and the “Learning by Doing” parts of the system, the users’ learning process continues with the “Self Evaluation” part, which is where a formal objective evaluation of their comprehension takes place. In this part, evaluation is of a more traditional type, where simple scores are used to reflect the degree of knowledge that the users have reached. No feedback is given to the users by the system until they have completed the test, thus permitting the formal assessment of this knowledge. At the end of this test the system not only provides the users with a comprehensive feedback regarding the number of correct and wrong answers, but it also offers advice to the users, as to which parts of the system are to be revisited in order to improve their understanding of the course material. It must be mentioned here that the test questions are chosen randomly by the system, so that the eventual scoring is not invalidated by the possibility that the users come across the same formal test twice. Finally, the “Self Evaluation” section includes another type of evaluation, namely a 3-D simulation that is both an evaluation and a learning tool. For this part the users are exposed to a 3-D scenario, which not only allows them to practice their acquired skills and knowledge through the course, but also to get some feedback for their actual performance.

As a concluding note to the educational approach, great care was taken when designing the Web/CD system, to incorporate all aspects of localisation issues. These issues which range from simple translation of the contents of the Web/CD to more involved ones, such as adaptation of interfaces to suit local standards and customs, were thoroughly studied and were carefully integrated into the system. Furthermore, care was taken to include pictures of medical equipment that could be easily identified by the end users and medical actions and procedures were recorded that were common across Europe. The JUST Web-CD System - Technological Approach

The development of the Web/CD system as a training and education course is a direct result of the evolution of the principles of Technology Based Training (TBT). The TBT’s main characteristic is the dynamic interaction between the teaching system and the trainee, which the TBT developers have always exploited by using the most advanced methodological and technological tools available to them. Following the rapid growth of the Internet and the DRom (CDRom/DVDRom) technology, the electronic publishing community is increasingly developing hybrid solutions that bring together the communication effectiveness of the DRom bandwidth and the capability for rapid and continuous information “update-ability” of Web based systems. Thus, in the framework of the JUST project, it was decided to follow this hybrid approach with some necessary compromises, but still utilising the best features of these two technological approaches. This decision implied that the integration level of the two technologies for on-line and off-line delivery was very critical. As a result a solution was chosen, which allowed both the maximum portability of the code and the controlled sharing of the contents. This adopted solution was based on the single versioning of control software and multimedia assets in a CDRom and Web compatible format. In other words, the contents such as text, images, videos and animations were created and delivered using powerful commercially available authoring environments, specially developed for delivering content over the Internet. Therefore the emphasis was given to the fact that the resulting application had to be efficiently delivered via the Web, which automatically meant that the chosen development platform could compress assets and optimise source code by reducing the size of files and therefore the downloading time from the network. Vector graphics were also used whenever possible, in order to the further increase the content delivery performance of the Web/CD. In this way, the CDRom and Web-based platforms that usually share the same type of information and application interfaces, guarantee the almost 100% portability of the code between the two media. This means that the same code can be stored either on a CDRom/DVDRom or published on the Internet, without major changes.

The outcome of this decision was that the creation of the very effective for teaching purposes multimedia content, such as video and audio (broadband media), which are common for off-line applications, has not compromised the feature of information updating. This is indeed a typical added value of an on-line web-based application, where data can be revised and updated at any time. The only compromise that such "update-ability" requires is the imposition of some restrictions in the use of enhanced graphics, video and audio. Continuous updates of the course stored on the CDRom/DVDRom are feasible through an FTP connection to the on-line repository. This is automatically achieved by using the “update” functionality of the Web/CD, which is capable of accessing the on-line repository and automatically download the latest versions of the contents of the Web/CD. These downloaded updates are then stored in a local storage space (the hard disk (HD) of the user’s PC) and whenever necessary are automatically retrieved. This adopted solution is rapidly becoming more relevant as new improvements in Java, General Applet Programming and Internet Delivery (such as large bandwidth) make their steady way to the marketplace. These new features and possibilities obviously provide greater scope for future improvements than those currently offered by most standard CDRom/DVDRom authoring environments. The JUST Web-CD System - Functional Approach The main functionality of the Web/CD is such that the users follow a well-defined procedure, which starts with the choice of the language for their preferred course study. The Language Selection menu offers the users the choice of one of five languages, namely English, French, Italian, Spanish or Greek. The language choice restricts the users to viewing the contents of the Web/CD in that language from then onwards. From then on, the users are presented with a Login Screen, which is essential for the intended use of the system. The login mechanism assigns a unique identification to each user and creates a storage space for each particular individual. Using a set of files that are hidden from them, the Web/CD system keeps a record of their progress as they go about learning the offered material. The information kept, is concerned with certain aspects of the user’s training and learning behaviour, such as the way they navigate, the number of completed steps, the set of tests that they take, their results, the history of their perusal of the Knowledge Book and so on. Following the general introduction to the Web/CD and the Topic Selection (where the users have the option to choose any one of the three currently available topics, namely Cardiac Arrest, Acute Asthma, and Unconsciousness), the users are then introduced to

the subject of their study. This Topic Introduction is a multimedia presentation, which provides the users with some relevant general information about their chosen course of study.

From then onwards the Web/CD functionality can be seen as a set of specialised tools whose sole aim is the enhancement of the learning experience of the trainee. These tools can be basically broken into three main sets, that is the Learning Tools, the Evaluation Tools and the Specific Tools. More precisely, the Learning Tools, are dedicated to the actual learning mechanism of the selected topic, and are characterised by three major parts, namely the “Step by Step”, the “Learning by Doing” and the Glossary. In particular, the “Step by Step” part is a multimedia presentation of each step of the decision algorithm, consisting of a number of accurate video-clips that show a realistic simulation of the incident. The “Learning by Doing” part is also based on video sequences of all the steps of the behavioural algorithm. The main difference that distinguishes the “Learning by Doing” from the “Step by Step” part is the possibility for a quick assessment of the user learning progress after each and every step of the algorithm. Moreover, the capacity of the trainee to comprehend the next action in the rescue sequence can be also evaluated. The Glossary is a list of all the medical and non-medical terms that are relevant to the specific topic, listed in alphabetical order. These terms can be reached through the menu choice “Glossary” or through the related links, which can be found inside the Knowledge Book. Integral to the learning mechanisms provided by the Learning Tools is the Knowledge Book, which is a thorough presentation in multimedia form of all the knowledge required for following each step of the algorithm, and which is available for consultation from both the “Step by Step” and the “Learning by Doing” parts. Therefore, the symptoms, the physiology, the anatomy, the required equipment are typical examples of the content of the Knowledge Book. It is structured like a multimedia encyclopaedia with text, images, sound and video sequences that describe the studied subject. The use of “Next” and “Previous” buttons allows the “linear” navigation of contents while a number of internal links permit the browsing of the Knowledge Book as a hypertext. It must be stressed here that the Knowledge Book assumes a different role during the Learning Evaluation session, where its functionality changes slightly and allows the user to consult it before answering the posed question. From an educational point of view this is quite meaningful since it helps the users to fix relevant content in their minds. The second set of tools is the Evaluation Tools of a whole topic. Their aim is to provide the necessary verification tools, which assess the acquired skills and capabilities of the trainees. These tools test the adoption of the correct decision-making procedures, the application of these procedures in a correct and coherent manner, the optimal use of the equipment, etc. Such evaluation tools are based mainly on MCQ and, whenever possible on a 3-D simulation.

Finally, the third set of tools is the Specific Tools, which are complementary to the navigation and exploitation of the Web/CD. Briefly these tools are: the Course Map (which is a table of all the contents of the Web/CD); The Bookmark facility (which is a tool for viewing, adding, and deleting bookmarks); the History facility; the Print facility; the User Notes (which is a facility that allows the trainee to add or remove textual notes); the On-line Services (which offer the possibility for accessing and consulting the Frequently Asked Questions, for asking an expert via email, or for downloading updated content or new questions for the self-evaluation part); the Search facility (which is capable for finding keywords inside the Web/CD). The final facility of the Specific Tools is Help. This facility appears under two guises. The first one is a context-dependent on-line help facility that provides relevant explanations through the help assistant Justy (which represents one of the interactive aspects of the Web/CD system) on the currently displayed content, whereas the second one is a general help facility that supplies information on every aspect of the Web/CD interface. Web/CD Prototype Trials, Validation and Evaluation Methodology

Extensive clinical trials were carried out only for one of the Web/CD topics, namely the Cardiac Arrest, for the simple reason that it was the one topic with common guidelines for the whole of Europe and as a result the outcome of the tests were amenable to cross-pilot comparison. Therefore, the four pilot sites have decided to introduce a common course in the Basic Life Support (BLS) area, which was based on guidelines, material and evaluation methodologies provided by the European Resuscitation Council (ERC). The setting up of the BLS course for the Cardiac Arrest topic had three basic objectives, namely, to enable the students: to recognize an emergency situation; to call for the appropriate help; and to safely perform efficient CPR. Thus, the main part of the course was a mixture of appropriate demonstrations given by the trainer and a number of supervised skill practice sessions undertaken by the students. The method followed for the evaluation and validation of the Web/CD was based on sound methodological guidelines and practices. All in all five types of evaluation took place during the project, namely, the examination results of students attending the Web/CD courses were compared with control groups of students attending traditional courses; the user aspects of the Web/CD were assessed through a relevant questionnaire; the Web/CD was assessed in a usability laboratory; the pilots made a follow-up study after 3 months to test the retained knowledge; the pilots made a follow-up study after 6 months to test again the retained knowledge.

As far as the Evaluation Methodology is concerned, the ERC has developed a well-structured assessment sheet, which is accompanied by a detailed instructions sheet for the assessors. In relation to JUST, this meant that when all the pilots were using this methodology, it was possible to compare skills between the test and control groups and more importantly to compare results across all four pilot sites. Acquired knowledge was assessed by mainly using tests consisting of Multiple Choice Questions (MCQ). The objective of the trials was to test the general hypothesis that interactive education can enhance traditional training in BLS-AED and more precisely to test the possible added value of the Web/CD as an interactive supplement. Especially to test that if such a supplement is continuously available to the trainees, this increased and improved their learning of skills, their retention of skills, their learning of factual knowledge and their retention of factual knowledge.

Finally, as regards the participants to the Web/CD trials, these were people from various backgrounds and varied education standards. The trainees were divided into two different groups: a Control Group (C) and a Test Group (T). The participants of the first group attended the course and only used the traditional studying material, whereas the participants of the second group attended the course and used the JUST Web/CD as the studying material. Results

Graphs 1 and 2 provide a short summary of the characteristics of the trainees that participated in the evaluation trials. All in all 501 trainees took part in the pilot trials, of which 276 were female and 225 were male. Their age range was between 14 and 60. As far as previous training was concerned this was nearly null, whereas their competence in using computers and related technologies, navigating the Internet and accessing the contents of a CDRom was medium.

The trainees, who were split into a Test Group that used the Web/CD and a Control Group that used the

traditional course material, were tested four times, using a Multiple Choice Questionnaire (MCQ), namely before the course (MCQ1), after the course (MCQ2), 3 months after the end of the course (MCQ3) and finally six months after the end of the course (MCQ4). It must be noted that of the 501 people that took part in the trials, only 120 completed all 4 MCQs. Furthermore, 233 trainees completed MCQ1, MCQ2 and MCQ3, whereas 189 trainees completed MCQ1, MCQ2 and MCQ4. The important findings from the results of the trials can be seen in the following graphs.

It is clear from Graphs 3, 4 and 5 that there was a drastic improvement for all trainees after attending the course, which was however more noticeable in the Test Group trainees. More precisely the level of the Test Group Trainees (the ones that used the Web/CD) started from an average MCQ score of MCQ1 = 5.35±1.83, moved to MCQ2 = 9.27±0.79 and for the 3 and 6 month re-tests the scores were MCQ3 = 8.04±1.43 and MCQ4 = 8.78±1.12 respectively. The corresponding results for the Control Group were MCQ1 = 5.63±1.99, MCQ2 = 8.85±1.04, MCQ3 = 7.98±1.40, MCQ4 = 8.83±1.15. The results for the

MCQs after 3 and 6 months do not show significant differences between the Test and Control Groups, possibly because there was some difference in favour of the Control Group scores when the trials began. Usability analysis of the Web/CD and User Satisfaction Results

Usability testing is one of several techniques to ensure a proper user-centred design. Two complementary methodologies were used for the Web/CD evaluation, namely a usability laboratory testing for the behavioural observations of system usability and interactivity (user testing), and a heuristic evaluation, where a team of 3-4 experts provided a categorization of the main usability issues encountered during inspection of the Web/CD. The team included both user interface experts and instructional design experts. The usability laboratory testing took part at the Usability Laboratory of ICS-FORTH. A total number of 9 participants took part in the trials (3 males and 6 females), their age range being between 25 and 38 years, with 6 out of 9 participants having been educated to a University degree level and the remaining 3 having finished secondary education. There was some prior training in first aid for 5 out of 9 participants with approximately 338 training hours on average and with practical experience for live first aid for 2 out of the 5 trained ones. The remaining 4 participants indicated that they had no first aid training and no experience in first aid practice. As regards experience in the use of computers and the Internet, their expertise was average or above average for 8 out of the 9 participants.

The user satisfaction questionnaire has proposed several statements concerning the general assessment of the Web/CD, the appreciation of its different sub-modules and the feeling of the end users about its interface. One important finding from the analysis of the user satisfaction responses, related to the question “whether the Web/CD suited their style and pace of learning”. The resulting response showed that “novice" users agreed more with the statement, when compared to the "expert" users. This higher agreement by “novice” users, strongly supports the conclusion that the tested software (Web/CD, Level 1 (Basic)) was better targeted to there educational needs (“style and pace of learning”), when compared to “expert” users. Since both types of users were tested on Level 1, which has been designed for novices in first aid, this was an appropriate finding (as expected). In other words, this is a quantitative evaluation finding which clearly supports the appropriateness of the original design goals for the Web/CD, at least regarding the Basic Level. However, for a number of other user satisfaction statements there was a higher agreement on the part of the “expert” users, when compared with the “novice” ones. This finding was true for statements such as: easy to learn how to use the different parts of the CD; never felt “lost” in the material; information was easy to remember; would like to keep using the CD after getting first aid diploma; videos illustrate well the topics; the Knowledge Book provides sufficient information; the links to Knowledge Book are at the appropriate parts of the text; Justy gives useful suggestions; and finally that there is a high overall presentation quality.

From the above findings, the following conclusion could be drawn. It is generally expected that users with prior experience in first aid would find it easier to navigate around the educational material of the Web/CD, and that information would be easier for them to remember, because they have less new information to absorb. It is also probable that they agreed more highly on statements characterising educational quality aspects of the Web/CD (e.g., videos, Knowledge Book content, Justy, overall presentation quality, long-term CD usefulness) because there prior experience gave them more confidence to make those types of judgment. Finally, the fact that the more knowledgeable users judged more positively the Web/CD’s educational quality also supports the appropriateness of the basic decisions, regarding the Web/CD’s instructional design. Conclusions

There are a number of innovative characteristics of the JUST Web/CD system, which make it an important addition to the usual emergency training mechanisms that are currently used. Some of the more important aspects

are: there is currently, no complete and certified computer-supported training compendium of European relevance; the content is certified by a team of European experts in Emergency Medicine (such as the European Resuscitation Council (ERC), a partner of the JUST consortium, that has been the body behind the Cardiopulmonary Resuscitation (CPR) Guidelines); it covers a broad range of incident types, where topics in Cardiac Arrest, Acute Asthma, and Unconsciousness are already implemented and a number of others are in the pipeline; it is a technologically sound combination of CD technology with a Web portal, thus providing a unique way of maintaining the correctness and originality of the content.

Furthermore, the Web/CD projects a European dimension, which can be seen more clearly from the following: its content is multi-lingual (initial languages being English, French, Italian, Spanish and Greek); it takes into account the different training methods that occur in the participant countries and provides statistical data at European level; it provides guidelines, recommendations and standards at European level by taking into account the individual guidelines, recommendations and standards that can be found in the various countries that participate in the JUST project; there is an in-built localisation awareness regarding protocols, legal constraints, as well as the current training situation, in the JUST project participating countries; the training methods used in the Web/CD were assessed at four different pilot sites throughout Europe, namely Italy, Spain, France and Greece, with a study also taking part in Belgium. The validation procedures used were uniform for all pilot sites and were based on scientifically sound assessment methodologies. Furthermore, the special structuring of the Web/CD provides great extensibility, which can be seen at two different levels the technical and the organisational. Especially at the technical level the Web/CD provides mechanisms for adding new training topics without too much effort and for easily updating already existing training topics, whereas at the organisational level, there is a strong commitment from the content providers to add, update, correct and generally to maintain the content of the Web/CD.

The final conclusions of this paper can be summarised as follows. The results achieved during the life cycle of the JUST project were extremely promising and in many cases they overtook the initial expectations of the project’s outcome. The Web/CD, especially the Cardiac Arrest topic, which was presented to a large audience of potential customers and end users, has been met very favourably and it consistently got rave reviews. There were always enquiries about its commercial availability and its price range. People who saw the Web/CD, although they appreciated greatly its format, its look and feel and its educational approach, they were mainly impressed by the accuracy of its content, whether written, spoken or in video form. References 1. Braslow A, Brennan RT, Newman MM, Bircher NG, Batcheller AM, Kaye W, “CPR training without an instructor: Development and

evaluation of a video self-instructional system for effective performance of cardiopulmonary resuscitation”, Resuscitation 1997; 34; pp207-220

2. Bossaert L, “Cardio-Pulmonary Resuscitation in Belgium. An evaluation of CPR-education and CPR-performance by the lay public”, Thesis geaggregeerde hoger onderwijs (PhD). UIA 6/6/91

3. Morgan CL, Donnelly PD, Lester CA, Assar DH, “Effectiveness of the BBC's 999 training roadshows on cardiopulmonary resuscitation: Video performance of cohort of unforewarned participants at home six months afterwards.” British Medical Journal 1996; 313 (12 Oct); 912-916

4. Manganas, A, Tsiknakis, M, Leisch, E, Ponder, M, Molet, T, Herbelin, B, Magnenat-Thalmann, N, Thalmann, D, Fato, M, Schenone, A, “JUST in Time Health Emergency Interventions: An Innovative Approach to Training the Citizen for Emergency Situations using Virtual Reality Techniques and Advanced IT Tools (The VR Tool)”, Proc. ICMCC - International Conference on Medical and Care Compunetics, NCC The Hague, June 2-4, 2004

5. Jonassen, D., Mayes, T., & McAleese, R., “A manifesto for a constructivist approach to uses of technology in higher education”, In T. Duffy, J. Lowyck, & D. Jonassen (Eds.), “Designing Environments for Constructivist Learning”, 1993, Berlin Heidelberg: Springer-Verlag

6. Woolfolk A., “Educational Psychology”, 2001, Allyn & Bacon. Acknowledgements JUST was a three-year European project that started in January 2000. Its total investment amounted to 3.6 MEuros and was co-funded by the Commission of the European Community within the framework of the Information Society Technologies (IST) programme (EU Contract number: IST-1999-12581). The JUST project involved 14 partners from 7 European Countries, in a multidisciplinary partnership with participants from industrial, academic, research and health sectors. Thanks are due to all the people from the following organisations that contributed to the JUST project during its 3-year period, namely the Foundation for Research and Technology – Hellas (FORTH), Greece, TSD Projects s.r.l. (TSD), Italy, the Dipartimento di Informatica, Sistemistica e Telematica dell'Universita` degli Studi di Genova (DIST), Italy, Interactive Labs s.r.l. (GIUNTI Ilabs), the Danish Centre for Health Telematics, County of Funen (FUNEN), Denmark, the Assistance Publique Hopitaux De Paris SAMU 92 (SAMU), France, the Sociedad Andaluza para el desarrollo de la informática y la electrónica SADIEL S.A. (SADIEL), Spain, University of Geneva (UNIGE), Switzerland, the Universidad Politécnica de Madrid (UPM), Spain, the Ecole Polytechnique Fédeérale de Lausanne (EPFL), Switzerland, the European Resuscitation Council (ERC), Belgium, the University of Crete (UCH), Greece, the Azienda Ospedaliera "Ospedale S. Martino" e Cliniche Universitarie Convenzionate (S.Martino), Italy, the Assistance Publique - Hopitaux de Paris (AP-HP), France, and the Empresa Publica de Emergencias Sanitarias (EPES), Spain.