verdi c-vibe overview & evaluation results

VERDI Project/Final Meeting/0204/CALT/INSEAD

VERDI C-Vibe

Overview & Evaluation Results


Future of Learning: Advanced Simulations

Research Perspective: Targeted Innovation Dimension

Technically

- Adv. User Interface

(3D & 2D real-time, multi-user Multimedia)

- Adv. User Experience

(multiuser, avatar-based navigation & interaction)

- Adv. Distribution Platform(Internet, multicasting, satellites, distributed dynamically inter-operating servers)

Conceptually

- Adv. Simulation Models

(based on Personality & Social Psychology)

- Adv. ‘Social Games’

(from quantitatively- to qualitatively-based interactions and interventions simulations)

- Adv. Scenarios/Dynamics(relevant, complex domains such as Change Management and Virtual Teams Dynamics)


Advanced Management Simulations – Research & Practice

Angehrn, A.A., Y. Doz and Jill Atherton, Business Navigator: The Next Generation of Management Development Tools, Focus, 1, 1995, pp. 24-31; or 93/37, INSEAD/CALT Working Paper 1.Angehrn, A.A. and J.F. Manzoni, A High-Tech Spin on Organizational Learning, Chief Executive, April 1996.Angehrn, A.A. and T. Nabeth, Leveraging Emerging Technologies in Management Education: Research and Experiences, European Management Journal, 15, 3, 1997.Manzoni, J.-F. and A.A. Angehrn, Understanding Organizational Dynamics of IT-Enabled Change: A Multimedia Simulation Approach, Journal of Management Information Systems, 14, 3, 1997.Angehrn, A.A. and T. Nabeth, C-VIBE: A Virtual Interactive Business Environment addressing Change Management Learning, Proceedings of ICALT 2001, Madison, USA.

humans

Agents in C-Vibe

Dynamic Models of Agent-based Scenarios

DistributedTeams

ExperientialLearning & ChangeSocio-cognitive Agents for:

- Mimiking human behavior in social contexts

- Providing socially rich Experimentation playgrounds

- Formalizing and validating models from Personality Theory, Social Psychology, Network Analysis, Innovation & Diffusion Models in different (sub)cultural contexts

EIS SimulationKernel

IP


VERDI C-Vibe Specific Focus: Change Management. Why?

75% of all transformation efforts fail50-75% of re-engineering projects fail78% of IT projects fail

Organizational resistance 82%Insufficient exec. sponsoring 72%Unrealistic expectations 65%Inadequate programme mgmt 54%Unclear business case 46%Lack of qualified resources 44%Scope of expansion/uncertainty 44%Ineffective leadership 43%

Key Factors:


VERDI C-Vibe Pedagogical Objectives

CHANGE RECIPIENTS

CHANGE STRATEGISTS

CHANGE IMPLEMENTATION

PROCESS

STRATEGY

ORGANIZATIONALDIAGNOSIS

EFFECTIVE ORG.L

INTERVENTI ONS

Adaptive

Explicit

Key driving principles(collaborativevs. competitive)

Leveraging networks& key individuals

TACTI CS

Adapted to people, timing& adoption stage

Target individuals vs groups

Open vs. covert

Collaborative vs competitive

Expectation mgmt

Process f airness dimensions

AwarenessI nterestTrialAdoption

!

I NDI VI DUALS

NETWORKS

ORG. CULTURE

Role & history

Attitude towardschange

Motivation & Resistance

I nfluence

Formal & informalnetworks

Power & diff usionnetworks

Communication culture

Positive/ negative signals

Values & vision

Mgmt style

PROCESS

UnderstandingInterpersonal and Group Dynamics(Distributed Virtual Team)

UnderstandingChange Management &

Diffusion Dynamics

PrimaryObjective

SecondaryObjective


VERDI C-Vibe Overall Value Proposition & Experience

-Knowl. Exchange-Initiatives/Practice-Play it again, Sam

-Intro experience-Agents’ advice-Other members

Motivation

Introduction to the

Challenge

3D Simulation Experience

DebriefingPhase

Follow-Up

Change Management Experience

Virtual Teamin 3D Spaces Experience

-Streaming Videos

-Discussion

-Streaming Videos

-Discussion


VERDI C-Vibe Technical Architecture (Overview)

VRSAT SERVER

C-VIBE Shared Objects

User Client 2(e.g. expert)

User Client 1(e.g.

student)

EIS Simulation

Server

Downstreamed served worlds

Upstream commands

Upstream commands

HTTP request

HTTP reply

Application Client

Upstream commands


-Peers and Professors-MBA Students

-MBA Student (INSEAD)

-Top Schools Network-Executives in Organizations (CIA, LinKS)

-Academic/Research/Education Community

-Broader Public/Local/Regional Communities

Constituencies Involved in Finetun’g, Evaluation and Dissemination

Virtual Group Dynamics &Role-Playing Lab

VERDIC-Vibe

EdComNet eLearning in

Urban Communities

Immediate Target/Impact

Next Step Target/Impact



Subject Category

Type of Subjects Size of Description of Subject Groups

Sample Size

1 Management students 10 INSEAD MBAs 10

2 Management trainers and researchers

4 faculty members from three French business schools and one English university, and 4 researchers in management

8

3 Practicing Managers 13 managers operating in France, Greece, England, Germany and France in different functions within companies including IBM, France Telecom and EdF/France, Styria/Austria, Pouliadis and Ernst &Young/Greece.

13

4 Community members/citizens from a French town (Chartrettes, Seine-et-Marne)

14 subjects representing a variety of profiles ranging from university students (3), elected municipality representatives (3), employed or self-employed managers/salespeople (5) and retirees (3)

14

Table: Subjects overview


Highlight of Lessons Learned from Evaluation Phase

Technical/Usability Dimension (1)

•3D Familiarization Challenge

•Visualization Challenge

•Real-time Coordination Challenge

Although an increasing number of people have recently been exposed to 3D applications, typically through single-user games, familiarity with 3D, multi-user, real-time environments is still not widespread. Whatever the application/game is in which users have to be involved, it is highly desirable to design brief experiences to be suggested to novice users in order to enable them to familiarize with basic navigation in 3D spaces and interaction with avatars and bots (human, and software-controlled dynamic entities inhabiting such environments). When provided enough time and guidance to get familiar with the 3D environment and with avatars interactions even the subjects with a high age (which are usually not very comfortable with computers) found basic navigation in 3D spaces not only easy but even an entertaining new experience, becoming ready for advanced applications like C-Vibe.







The design of 3D spaces with the appropriate size and which include objects with embedded textual information or which can be manipulated/re-positioned realtime, is non-trivial. Appropriate size of 3D rooms and objects, readability of textual information and visualization of dynamic characteristics of objects (e.g. ‘passive’ versus ‘active/clickable’ objects) can be achieved only through extensive user testing. In particular, in applications including a large amount of textual content like C-Vibe (explanations, narratives, etc.) it is critical to decide which information components are better presented separately in 2D windows linked to specific 3D objects.







Applications like C-Vibe (compared to the majority of 3D games and edutainment products) take advantage of the multi-user dimension and require users to coordinate real-time. For instance, the C-Vibe simulation requires distributed users to first formulate and agree upon a joint strategy. Our findings confirm that tasks like this one, which are relatively simple in face-to-face settings, can become very complex and time-consuming in distributed environments. After developing a strategy, C-Vibe (and all similar simulation/educational experiences which will be developed in a near future) requires users to take a large number of group decisions and coordinated actions. Current coordination mechanisms (like simple chat rooms and whiteboards) can provide only limited support to such tasks, and will need to be complemented with more sophisticated 3D coordination and decision support tools suited to distributed teams operating in 3D environments.



Learning Value Dimension (1)

Confirmation of underlying Hypothesis concerning Learning Value

VERDI 3D realtime, multiuser

technology

supports successfully

+

advanced expectations of how to best integrate Info Comm

Technologies (ICT) in existing learning processes

+fits

address management learning in new significant application areas (particularly change management, group/meeting dynamics,

and the development of competencies related to high-performing virtual teams)

(1) experiential and (2) collaborative eLearning approach adopted and

developed in the C-Vibe application

+

enables



Learning Value Dimension (2)Supporting Evidence

1 The capability of modelling and simulating ‘social’ and ‘inter-personal’ contexts (managerial situations involving interacting with people rather than taking quantitative decisions) is a key distinguishing characteristic of eLearning systems like C-Vibe.

2 The multi-user dimension of C-Vibe introduces a valuable ‘learning from each other’ factor enhancing the perceived usefulness of the system.

3 Requiring users to coordinate decisions and actions in distributed settings (see also last section) introduces a high level of complexity users are not familiar with and which should be particularly supported/trained in advance in order to become a productive learning experience (rather than a frustrating one).

4 The value of the 3D dimension (versus 2D applications) in systems like C-Vibe appears particularly (and probably only) when the application requires extensive interaction among groups of simultaneously online users (like when formulating a strategy and taking decisions in C-Vibe).




5 The 3D dimension introduces a ‘game’ element which can potentially negative influence/ distract from the learning focus of the application. In addition, this ‘game’ component can have a discouraging impact (‘gimmick’ association) and generate a-priori resistance on users associating learning with a ‘serious’/non-game situations. This resistance can be only eliminated after involving subjects into direct experimentation (after which, this form of a-priori resistance typically dissolves).

6 Although not intended to operate in this direction, C-Vibe helps users to learn (by doing) about group dynamics. Educational tools like C-Vibe could be effectively used to make people learn how to operate effectively in distributed teams and using or designing virtual environments supporting distributed managerial decision making and meetings in general.

7 Educational systems of the type of C-Vibe are potentially attractive for broader audience than management. Once initial barriers (scepticism towards 3D in the learning process) are overcome, they could serve the acquisition and development of ‘social skills’ also beyond organizations (other educational as well as social/political contexts).

8 Compared to other learning methods, C-Vibe provides a more intensive/immersive experience which is particularly relevant in developing management skills like the mastering of complex group dynamics.




9 The learning value of 3D simulation experiences like C-Vibe can only be realized when followed by structured debriefing sessions in which the experience is reviewed and analysed. This should be ideally supported by the possibility to ‘record and replay’ simulation sessions (like with videos). In addition, to support learning, the (behavioural) models underlying the simulation need to be made explicit to the user (‘opening the ‘black box’) after a session.

10 C-Vibe, rather than just providing a ‘virtual amphitheatre’ in which users can discuss their experience (ideally with experts supporting the analysis of their simulation sessions) should be extended with tools supporting follow-up collaborative learning among the users (from the same or from different teams) in forms of virtual learning communities operating both realtime and asynchronously. The current 3D virtual spaces (Virtual Boardroom and Virtual Amphitheatre) should be extended and adapted accordingly.

11 A specific value of learning experiences like the one provided by C-Vibe is the one of ‘forcing’ users to address challenging situations in which they can realize the limits of their competencies. Compared to more traditional learning approaches, the resulting dissatisfaction of the status quo generates motivation for further learning and competency development.



Evaluation Conclusions (1)

Evaluation provided sound basis for SPECIFIC GOIDELINES towards:

(1) improving the design of future versions of C-Vibe and similar eLearning systems

increase learning value and decrease adoption barriers

(2) identifying the dimensions along which to compare C-Vibe to other eLearning systems

factors for in-depth comparative studies focussing on learning value

(3) integrating C-Vibe in broader educational and eLearning contexts design effective blended learning approaches ‘around’ C-Vibe



Evaluation Conclusions (2)

In summary, the findings indicate a high acceptance of the approach and the recognition of a very high potential for 3D educational experiences like C-Vibe. On the technical/usability level a number of challenges have been identified thanks to the developed C-Vibe prototypes, including a 3D Familiarization Challenge, a Visualization Challenge, and a Real-time Coordination Challenge - particularly important for multi-user, real-time environments in which different players/learners need to form and coordinate their decisions as a group, like in the case of C-Vibe. At the level of the underlying concept and the learning value, the evaluation confirmed the hypotheses of the project. Provided the 3 technical challenges mentioned above are addressed adequately, 3D, multi-user, real-time environments resulted to provide a promising way of complementing and extending traditional educational approaches. This appears to be particularly the case when addressing the development of relevant but complex ‘soft skills’ or social competencies such as the capability to manage change and innovation effectively in organizational or community contexts, as well as the development of competencies related to learning to operate effectively in distributed teams and virtual environments supporting group decision making and collaboration.

verdi c-vibe overview & evaluation results

Documents

d rooms

d environment

entertaining new experience

basic navigation

appropriate size

embedded textual information

avatars interactions

advanced applications