Title: Assessment in Smart Learning Environments: Psychological Factors Affecting Perceived Learning.

Authors:

1) Thomas, Lauren Josie a Corresponding author. 2) Parsons, Michael a

3) Whitcombe, Dean a

a University of South Wales, UK.

Corresponding author information:Lauren ThomasUniversity of South WalesTreforestPontypriddCF37 1DLTel: 01443 482179Email: lauren.thomas@southwales.ac.uk

Declarations of interest: none.

Abstract: Smart learning environments (SLEs) are spaces which incorporate technology to improve student learning outcomes. This growing body of literature typically examines day-to-day learning, but very little research considers the use of these environments for formative and summative assessments.

This paper focuses on an innovative crisis simulation which took place in a unique smart learning environment at the University of South Wales. This simulation suite, one of only seven utilised in higher education globally, was used to investigate how SLEs can be used to conduct successful real time simulations for summative assessment.

Using a case study approach and PLS SEM methodology this research demonstrates that the psychological factors of social support and career relatedness show significant positive effects upon perceived learning in these environments. Surprisingly, the perceived ease of use of the technology does not significantly impact upon perceived learning. This research contributes a valuable understanding of simulation learning within SLEs, and its relationship to positive outcomes in student perceived learning. Implications for both academics and practitioners are discussed, and suggestions for future research are given.

Key words: smart learning environments, technology enabled learning, social support, ease of use, relatedness, perceived learning.

1.Introduction:The pedagogical approaches to teaching and assessment often lag behind the technology able to support them (Kinshuk et al., 2016; Price, 2015; Spector, 2014) and currently there is a need to fully explore how to use Smart Learning Environments (SLEs) for the benefit of the student (Chang et al., 2015). While the definition of SLEs has not been fully agreed (Spector, 2014), there are some noted aspects. SLEs should aim to support innovative opportunities for both students and lecturers (Spector, 2014). They should allow for students to develop skills which can be taken from context to context, and these developments in knowledge should be noticeable in the real-time environment (Kinshuk, 2016). They are environments which are effective, efficient, and engaging (Spector and Merrill, 2008), where the learning outcomes are preferably better than those of similar students in traditional environments (Spector, 2014). Within this environment, collaboration has been highlighted as a key feature where learning and problem solving are desirable outcomes (Frankl and Bitter, 2013).

The need for research on what psychological factors affect students in SLE environments in clear. The issue of a lack of empirical research within the area as highlighted by Spector (2014) remains the case today. Research on how groups can negotiate computer-mediated environments for learning has been underway for some time (Francescato et al., 2006; Gan et al., 2015; Lopez-Yanez et al., 2015), but how groups use computers and other technology as aspects of SLEs is less clear. Further to this, the use of SLEs for assessment is a rare topic in the literature (Klimova, 2015), as is work on the psychological factors that may affect these learners (Islam, 2016). The increasing use of simulation technology in industry training emphasises the relevance of these technologies for the classroom, including for assessment purposes (Andrews et al., 2014). With this in mind, this research aims to develop and test a model to explain the influence of several pertinent factors upon student’s perceived learning in SLEs.

One of the obvious benefits of an SLE is the opportunity to run deeply engaging simulations from which students can undertake both formative and summative assessment (Klimova, 2015). Research on simulations is on the rise (Beuk, 2015; Kietzmann and Pitt, 2016; Kinshuk et al., 2016), although they have been used successfully in marketing classrooms for some time (Burns and Golen, 1983; Low, 1980; Redmond, 1989). The research undertaken in this paper draws data from one such simulation, using an innovative final year undergraduate public relations module at the University of South Wales as its data collection point. The students worked in teams to handle simulated public relations crisis based on a previous scenario. This year students were asked to deal with the public relations for Fyre Festival, which famously went badly wrong leaving thousands of consumers stranded on an island with little food, shelter, or the promised entertainment (Butterly and Crookes, 2017).

It is important at this juncture to acknowledge the distinctiveness of perceived learning in comparison with the often-associated construct of actual learning. Education research such as Sitzmann et al’s (2010) meta-analysis of learning studies has established that the two concepts are markedly different. As Bacon (2016:107) observed, “The term perceived learning refers to a student’s self-report of knowledge gain, generally based on some reflection and introspection…Actual learning reflects a change in knowledge identified by a rigorous measurement of learning”. Recognizing and understanding the difference between the two concepts avoids potential confusion in the measurement of learning which can broadly be categorised in two ways: 1) direct measures of learning, such as scoring task performance based on a specific learning goal, which are employed to assess actual learning, and 2) indirect measures such as self-reports of learning which subjectively appraise perceived learning (Elbeck and Bacon, 2015). Although several scholars (e.g. Bacon, 2011;

Kennedy et al. 2002; Sitzmann et al. 2010) have questioned the validity of perceived learning as an effective measurement of pedagogical success the construct remains a key component in the study of SLEs. Recent studies of e-learning platforms (Ifinedo et al., 2018; Lin, 2018) and online social presence (Chen, 2018; Richardson et al., 2017), for example, have utilised the concept of perceived learning.

1.2 Benefits of SLEs:

Simulation activities within SLE are widely recognised as offering students unique opportunities to link theory and real-world practice (Brennan, 2014). Numerous studies have examined the use of simulations within a wide range of disciplines, including marketing (Faria & Wellington, 2004) with scholars highlighting their ability to develop the marketing skills and attitudes required to deal with some of the complexities of their future careers (Brennan & Vos, 2013; Vos & Brennan, 2010). More recently, Canhoto and Murphy’s (2016) analysis of simulations highlighted several benefits for experimental learning, notably in terms of enhancing the acquisition of tacit knowledge, offering an active approach to learning favoured by contemporary students and augmenting employability skills (Ardley & Taylor, 2010; Barker, 2014; Ganesh & Sun, 2009).

Underpinning these benefits is the common use of technology to enhance experimental learning. Whilst this trend is reflected in several studies which have examined computerised simulations (Cadotte, 2016; Flostrand et al., 2016) there nevertheless remains a noticeable lack of empirical research on the effectiveness of simulations. Indeed, several scholars have questioned the common use of instructors’ subjective impressions or select qualitative feedback from students (Fuller, 2016; Olson, 2012; Raymond & Sorensen, 2008) in the evaluation of simulation activities. As Wang (2016) observed, most published articles on simulations have focused on their operationalisation rather than investigating their impact on positive learning outcomes.

This research gap is summed up in Ranchhod et al.’s (2014, p. 75) comment that the literature on simulations has typically neglected “the way in which students make sense of this pedagogical exercise”. By investigating the psychological factors which affect students in SLE environments this paper responds to Wang (2016) and Ranchhod et al.’s (2014) comments by providing a much-needed study of the use of SLEs for assessment within the context of perceived learning. We now put forward three constructs for testing in relation to explaining perceived learning (figure 1).

1.3 Social support:

Social support has generated an extensive body of literature but despite its importance it largely fails to appear in the learning literature and has not yet been applied to SLEs. Defined as “information leading the subject to believe that he is cared for and loved, esteemed, and a member of a network of mutual obligations’ (Cobb; 1976, p.300) social support is an important factor for students (Wilcox et al., 2005; Xerri et al., 2017). Several studies, for example, have demonstrated the importance of social support in dealing with stress and reducing the effects of stressful environments (Carver et al., 1989; Cohen and Wills, 1985), such as one encounters in a summative, live, group assessment. Students can draw upon social support before, during, and after individual assessments (Folkman and Lazarus, 1985). Given the increased emphasis on teamwork as a learning objective (Riebe, et al. 2016) in response to the recognition of the role of collaborative learning on developing skills, such as communication and confliction resolution, required by employers (Britton, et al., 2017), social

support is unsurprisingly important where members of a group rely on each other to perform well to achieve a good assessment grade (Good and Adams, 2008).

The concept of social support is multi-dimensional but applied in this context it consists of two dimensions, informational support, and emotional support. Informational support includes factors such as providing guidance and suggestions, which in times of stress can assist in problem solving. Emotional support entails factors such as trust, empathy and acceptance (Langford et al., 1997; Liu and Hung, 2016; Sherbourne and Stewart, 1991; Wentzel et al., 2016; Wills, 1991). Within the context of teamwork and collaborative learning both dimensions affect group work. The absence of social support can, for example, result in interpersonal issues which can cause barriers to collaborative learning (Burns and Golen, 1983; Johnson et al., 2015). These findings were subsequently confirmed in de Janasz and Forret’s (2007) paper which highlighted the importance of providing students with opportunities to network to foster social support. Bravo et al. (2018) meanwhile demonstrated that satisfaction with the teamwork and the expected quality of the outputs can positively influence student perceptions of teamwork, while workload and task complexity have negative influences. This latter point is particularly important for simulations within a SLE context, which potentially presents added levels of technical complexities to student teamwork. Morris et al. (2004), for instance, observed that groups may struggle to share and allocate resources such as screens, and that this issue may become more difficult to manage as the number of users and the number of documents increase.

These research findings were subsequently reflected in the design and implementation of the crisis simulation. Prior to the simulation, students were accordingly given opportunities to meet each other, in line with de Janasz and Forret’s (2007) research findings. Based on the need to complete several tasks simultaneously during the simulation, screens were kept to a minimum (one laptop, one phone) for the duration of the assessment. The assessors likewise resided in a multi-display environment with multiple displays of each group producing both visual and audio information to enable real-time, non-invasive assessment of learning. Whilst reducing the number of communication devices and observing the student teams from a separate control room represented an attempt to mitigate the issues identified by Morris et al. (2004) the importance of social support cannot be underestimated. In terms of the simulation assessment, poor levels of social support could potentially cause a lack of information sharing and emotional barriers to communication, advisedly affecting learning outcomes. Supported by several studies which have highlighted the impact of social support on academic performance (Cutrona et al., 1994; Eggens et al., 2008; Levitt et al., 1994; Wentzel, 1998, 1999), we therefore hypothesise:

H.1: Social support will influence perceived learning.

1.4 Relatedness:

Perceived relatedness considers the students perceptions on how related their learning and assessment is to their desired future careers. Business schools have a duty to provide students with skills they can use in their careers (Bennis and O'Toole, 2005), and this construct is particularly important given the widely publicised skills' deficiencies in UK graduates (Frankham, 2017; Novakovich et al., 2017). Despite such criticisms the design and implementation of a crisis simulation held within a SLE offered students an opportunity to develop and demonstrate the digital competencies required by employers in the business sector. Such digital competencies include the appropriate use of social media to send messages, engage with others, and foster participation in

virtual communities (Alber et al., 2015) and appreciation of new media literacies which underline the convergence of traditional and digital journalism (Ventura et al., 2018). Whilst prior research by Tymon (2013) highlighted that first and second year Business Studies students expressed ambivalence about employability, this was not generally reflected in the student cohort who completed the crisis simulation. This can be explained by the characteristics of this student group, notably in terms of their familiarity with digital communication devices and status as final year undergraduate students. Such features can be related to studies which have demonstrated how students engage better with technology that they feel is relevant to them (Sebian and Tominc, 2015) as well as expressing a positive attitude towards tasks which are linked to team performance (Johnson et al., 2015). In this environment a good team performance will help to ensure learning. With this in mind, we hypothesise the following:

H.2: Relatedness will influence perceived learning.

1.5 Ease of use:

Defined by Davis (1989, p. 320) as "the degree to which a person believes that using a particular system would be free of effort," perceived ease of use is a central component of the Technology Acceptance Model (TAM) (Davis, 1989; Davis et al., 1989; Venkatesh et al., 2007). The model suggests that technology adoption is a function of perceived ease of use as well as perceived usefulness, a term used to describe "the degree to which a person believes that using a particular system would be free of effort” (Davis, 1989, p.320). Although the relationship between the ease of use and perceived usefulness constructs has been debated, with some authors considering them separate concepts (e.g. Sánchez and Hueros, 2010; Al-Busaidi, 2013; Yeou, 2016) whilst others have combined them (e.g. Chiu et al., 2005), TAM remains a seminal paradigm. While scholars such as Pavlou and Fygenson (2006) have criticised TAM for being incomplete an extensive body of research has nevertheless emerged aimed at improving the model’s accuracy and predictive ability (e.g. He et al., 2018; Turel et al., 2011; Venkatesh et al., 2012).

Within the context of education research perceived ease of use remains a key construct in technology enhanced and computer-mediated learning environments (Abdullah and Ward, 2016; Chang et al., 2015). Several studies of students’ acceptance of Moodle, for example, have highlighted the positive impact of perceived ease of use on technology adoption (Al-Busaidi, 2013; Ifinedo et al., 2018; Padilla-Meléndez et al., 2013; Yeou, 2016). Similar results have emerged from studies of students’ acceptance of learning systems and blogs (Chang and Yang 2013; Abdullah et al., 2016). While perceived ease of use has been identified as an important aspect of SLEs (Chang et al., 2015), little research currently examines it in relation to simulations within a SLE environment. Research has previously demonstrated that students’ perceptions of technology can create barriers to learning in simulations (Burns and Golen, 1983; Darban and Polites, 2016) and perceived ease of use is known to affect continuance intentions for business simulations games (Clark et al. 2009; Doyle and Brown 2000; Tao et al., 2009). This was demonstrated in Adobor and Daneshfar’s (2006) study of the factors that promote the effective use of simulations in management education, which highlighted the positive impact of perceived ease of use on learning outcomes.

Despite such findings there remains a lack of research into the effects of perceived ease of use on students completing simulations within SLEs. The SLE utilised for this study involved student teams completing a crisis simulation in rooms equipped with a smartphone, laptop, printer, and a TV screen with which to view incoming information and revolving social media posts. Each student

group needed to successfully interact with the technology to complete all sections of their assessment. In accordance with prior studies (e.g. Al-Busaidi, 2013; Sánchez and Hueros, 2010; Yeou, 2016) the researchers considered perceived ease of use as a separate construct and due to the potential impact of perceived ease of use on student learning outcomes hypothesised that:

H.3: Ease of use will influence perceived learning.

Figure 1: Hypothesised relationships.

2.Methodology:2.1 Scenario and Subjects:

The simulation focused on the students' ability to perform as a communications team during the Fyre Festival, which famously went badly wrong leaving thousands of consumers stranded on an island with little food, shelter, or the promised entertainment (Butterly and Crookes, 2017). The SLE was the Hydra simulation suite at the University of South Wales (displayed in figure 2). The Hydra simulation suite and methodology were designed by Professor Jonathan Crego as a unique high-fidelity training tool that enables the monitoring of real-time decision-making and leadership of emergency service personnel (police officers, social care professionals and the military) during critical incidents (Alison et al., 2012; Crego, 1995). Historically used for the training of police officers, this unique learning space now exists within Universities throughout the UK. Due to licensing restrictions, the Hydra software and methodology were not utilised during this study. However, the IT, audio-visual (microphones and HD cameras) equipment and remote access software contained within this SLE were used throughout.

Figure 2: Visual representation of a Hydra simulation suite.

Sixty-five (n = 65) students took part in the simulation. Prior to the simulation day, this cohort was split into 3 groups and each group was asked to attend at different times throughout the day. Each group was split into teams (~5 or 6 students per team) based on their grade from a previous assessment. Students were made aware of their team members approximately 2 weeks before the simulation took place. Furthermore, a mock simulation was conducted over a 3-week period (1 hour per week) prior to the simulation day to familiarise the students with the methods used to deliver exercise material.

On the day of the simulation, the first group of students reported to the plenary room at 08.50am, where they were briefed about the simulation and asked to sign a consent form, in accordance with project's University of South Wales ethical approval. Shortly after, the 4 teams entered their assigned pod room, where they worked in isolation for 2 hours. The majority of exercise material (news articles, social media posts and news broadcasts) and tasks were provided to the students in the form of a custom-made video clip (Adobe Premiere Pro CC, 2017) using the large PC monitors in the pod rooms (displayed in figure 3). Exercise material was provided in accordance with the pre-constructed storyboard and a time limit was set for each task using a countdown timer displayed on the video clip. The simulation had 3 distinct "Acts", reflecting key points in the real-life crisis designed to allow students to demonstrate and develop key crisis communication skills at appropriate points. A designated PC located in each pod room was used to complete the tasks set throughout the simulation and a whiteboard and pens were also provided. Other information was provided via networked printers and each pod room were given a mobile phone to receive journalist calls.

Figure 3: Examples of exercise material and tasks provided in the pod rooms.

Throughout the scenario, their aim was to apply skills and theories to obtain a successful outcome. This included actions that used technology in varying ways, such as emailing staff working at the festival, responding to official bodies, writing a press release for news media, and answering phone calls from journalists. Inside the pod rooms were cameras and microphones that allowed both academics and external practitioners (one practitioner assigned to each pod room) to unobtrusively observe and take notes on their performance before a debriefing session and grade being decided (displayed in figures 4 and 5).

Figure 4: Students being monitored from the control room and plenary room.

Figure 5: Students being debriefed about their performance by an external practitioner.

Following the simulation, each group returned to the plenary room where they were given a paper questionnaire. All questionnaires were returned and of a total of 65 questionnaires, 61 useable responses were obtained, all of which were anonymised before using the data. The sample was aged between 20 and 51 years old (mean age of 21 years) and 27% were male and 73% were female. Following completion of the questionnaire, students were asked to leave the plenary and asked not to confer with any other students about the simulation. The other student groups attended the Hydra suite throughout the day and completed the simulation as described above.

2.2 Measurement development:

The majority of questions included in the survey come from previous literature, with a view to increasing the reliability and validity of the survey instrument (see appendix one). A pre-pilot study resulted in some item re-wording, while the 10-student pilot survey presented no further issues. The pilot responses were not included in the research. Likert scales with 7 points were used for each question, and the responses ranged from strongly disagree (1) to strongly agree (7).

2.3 Data analysis and findings:

The analysis is conducted using partial least squares structural equation modelling (PLS SEM). Unlike more traditional variance-based SEM, PLS SEM is capable of dealing with non-normal data, such as was obtained here (Hair et al., 2017). Another determinant of this was the limited sample size available, as PLS SEM is typically capable of dealing with small sample sizes (Chin, 1998; Ringle et al., 2012). The sample size exceeded that needed for this analysis, in line with Cohen (1992). Data was analysed using SmartPLS software, version 3.2.7 (Ringle et al., 2015).

3.Measurement model:

3.1 Reliability:

When using PLS SEM the preferred method for assessing reliability is composite reliability (Hair et al., 2017). Cronbach’s alpha may also be considered also. Scores of a .70 and over indicate reliability for both composite reliability and Cronbach’s alpha. All scores were above .70 (see table 1), thus the research can be considered reliable.

3.2 Validity:

Content validity and construct validity are considered here. Content validity was ensured via a thorough literature review, and further to this, many of the questionnaire items were adapted from existing literature, enhancing the content validity further. Construct validity is comprised of convergent and discriminant validity. The AVE’s are all above .50 (see table 1), and therefore convergent validity is established (Bagozzi and Yi, 1988). In terms of discriminant validity, the Hetero-trait Mono-trait (HTMT) criterion has recently been shown to be a better measure than cross-loadings or the Fornell-Larcker criterion (Henseler et al., 2015). All loadings were below the .85 cut off and therefore discriminant validity is established (Henseler et al., 2015). With content and construct validity established, the research can be considered valid.

Social support Relatedness Ease of use Perceived learning

Social support 0.843 Relatedness 0.566 0.822 Ease of use 0.064 0.332 0.853

Perceived learning

0.583 0.574 0.197 0.847

AVE 0.711 0.676 0.728 0.718CR 0.925 0.926 0.914 0.910CA 0.899 0.905 0.893 0.870

(AVE = average variance extracted, CR = composite reliability, CA = Cronbach’s alpha. In bold is the square of correlation between constructs).

3.3 Structural model:

Two items were removed from the model (EU4 and PL3) due to loadings below the .7 level. As they were above .4, but increased the AVE when removed, they were removed in line with Hair et al. (2017). Figure 2 demonstrates the results of the model. Hypothesis testing is enabled via a bootstrapping method (table 2). Two of the three hypotheses put forward were supported, and one was rejected. The relationship between social support and perceived learning was accepted, with an effect size that proved to be large (p = .020, f2 = .178). Therefore, H1 is supported. The effect of relatedness on perceived learning was also accepted (p = .027, f2 = .115), but with a smaller effect size, meaning social support is the more influential construct within this model. This allows for H2 to be supported.

Surprisingly, H3 the relationship between ease of use and perceived learning, was rejected (p = .596, f2 = .006). This signifies that ease of use did not have any significant effects on perceived learning following the simulation. Potential factors in this outcome are considered in the discussion section. The R2 for perceived learning was .431, meaning 43% of students’ perceived learning can be explained by this model, allowing for a valuable contribution to the literature. The results of the model are summarised in figure 6.

T-statistics

P-values Path coefficient

f2 Accept/Reject

Social support > Perceived learning

2.342 0.020 0.392 0.178 Accept

Relatedness > Perceived learning 2.214 0.027 0.332 0.115 AcceptEase of use > Perceived learning 0.531 0.596 0.061 0.006 Reject

Table 2: results of the analysis.

Figure 6: Results of the model.

4.Discussion:Students undertaking assessments in an SLE are at the forefront of a wave of necessary pedagogical change. This places a significant need on developing a better understanding of the psychological antecedents to successful perceived learning in these kinds of innovative learning environments. Within this section the findings are discussed in the context of existing literature with the effects on social support, relatedness, and ease of use on perceived learning are discussed in turn.

4.1 Social support:

Social support emerged as the most influential factor within the model, clearly demonstrating that more work needs to be done on this concept in relation to group learning in SLEs. Overall student reporting of perceptions of social support were scored very high, indicating that the pre-simulation introduction processes had worked well for students (conducted in line with de Janasz and Forret, 2007), despite them not working in groups of their own choosing. This was likely affected by the fact that students were relying on overall team performance to achieve good grades (Good and Adams, 2008). This further supports the idea that social support can help to reduce the effects of stressful environments (Carver et al., 1989; Cohen and Wills, 1985), including in this specific educational context. The existence of perceived social support likely also helped facilitate a lack of barriers to learning (Burns and Golen, 1983; Johnson et al., 2015).

Both informational and emotional support were thought to play key parts during this live group assessment. Informational support is thought to be less important than if students had chosen their own teams, because students were put into teams with similar grades from a previous assessment and were therefore at similar levels of academic capability. However, the ability to provide informational support and apply problem solving skills at the right time is not necessarily spread

equally across team members, meaning it still has importance. Further, emotional support influencing factors such as trust, empathy, and acceptance, is conducive to better teamwork. It is reasonable to hypothesise that as the higher education curriculum and assessment moves towards a more collaborative approach, social support may become an even more important factor.

4.2 Relatedness:

Business schools have a duty to supply their students with skills for the job market (Bennis and O'Toole, 2005; Novakovich et al., 2017) and therefore confirming the relationship between career relatedness and perceived learning is important. Skills which have been identified as important include teamwork, communication, and conflict resolution (Riebe et al., 2016; Britton et al. 2017), and this simulation was designed to enhance them. As was demonstrated in previous research by Sebian and Tominc (2015), when students perceive a skill as relevant to themselves, they are more likely to find the platform for learning more useful also, explaining why student feedback collected on the simulation in this and previous years has been so positive in general.

While students typically gave high scores in their feelings on the ability of the simulation to provide them with skills (namely, crisis management, teamwork, digital and non-digital communication skills), they gave lower ratings to the questions on the simulation aiding future job performance and being useful in future careers. This clearly relates back to research from Tymon (2013) who found that first and second year Business Studies students had expressed ambivalence about employability. While students were clearly aware of the importance of the skills they were learning, they were less able to make the link between these skills and career relatedness. Ongoing discussions with students from simulations in previous years have revealed that they have since found the simulation was very useful in their careers in terms of the skills they gained – so questions around when and how the link is made between these skills and career relatedness continue to exist.

4.3 Ease of use:

Student's perceived ease of use of the technology in the room did not have a significant impact on perceived learning in this context. This result was unexpected due to the proven relationship between ease of use and positive learning outcomes in simulations (Adobor and Danesfhar, 2006), and ease of use and continuance intentions for business simulation games (e.g. Clark et al., 2009). This result was further surprising given the ongoing relationship with perceived ease of use and learning in similar bodies of learning literature (e.g., Abdullah and Ward, 2016; Ifinedo et al., 2018). More needs to be done to investigate and understand what factors made ease of use a redundant part of the model in this particular context. As demonstrated by Chang et al. (2015), ease of use is considered with relative frequency in the related literature, but unfortunately no literature is currently close enough to aid a direct comparison of results.

There are multiple considerations in why ease of use may not have been an influential factor for students in this instance. A significant majority of students were part of the "digital native" generation (Akcayir et al., 2016; Helsper and Eynon, 2010), for whom technology enhanced learning is typically positive, and something they feel equipped to deal with (Comer et al., 2015; Thompson, 2015). Therefore, it is possible that they didn't feel the use of technology was a significant factor in their perceived learning. Worth noting, as Jones et al. (2010) find, is that digital natives can vary widely in their perceptions and usage of technology and so this may not fully explain the outcome.

Further to this, students had participated in a mock crisis event and had been briefed about the technology they would use – this is potentially a contributing factor in that it gives students time to prepare and feel comfortable that the technology was a part of their assessment. An additional potential factor is that students were not using technology they were unfamiliar with. If students had been asked to use cloud technology, virtual reality, or other new technological developments in education, then it is anticipated that the relationship between ease of use and perceived learning would become significant in nature. This needs to be considered in future research - creating a deeper understanding of ease of use in relation to SLEs is necessary, given that poor student perceptions of technology can create barriers to learning (Burns and Golen, 1983; Darban and Polites, 2016).

5.Conclusion:

This research put forward a model that explained 43% of students' perceived learning in SLEs during a crisis simulation. Social support and the relatedness of the activity and skills to students desired careers were shown to be significantly and positively related to perceived learning. Surprisingly, ease of use of the technology used as part of the simulation was a non-significant factor, requiring future research.

5.1 Applications for this research:

This research was conducted in answer to a clear need for more research on specific aspects of SLEs (Kinshuk et al., 2016; Spector, 2014), most notably in relation to the use of SLEs for assessment (Klimova, 2015) and how psychological factors may be affecting learners in these environments (Islam, 2016). Further value is added in providing evidence for the effectiveness of simulations in SLEs on meeting learning outcomes, addressing calls from both Ranchhod et al. (2014), and Wang (2016).

Additional value can be found in this research as it adds to a growing body on literature on simulations in education, especially outside of the medical field where the majority of work has been done. Immersive and collaborative learning are key pedagogical trends that academics will need to become familiar and confident with over the coming decade, as use of these techniques within the classroom will facilitate the student learning experience. Understanding the psychological factors that affect students within this context allows for a better design, and better learning outcomes. Practitioners may wish to apply similar training styles to graduate programmes and staff training schemes, as recent history has demonstrated on numerous occasions the inability of business organisations to respond to crisis situations. At University level, the use of simulated learning can help fix this skills gap by preparing future practitioners in safe environments at a pre-career stage.

5.2 Limitations and future research:

The use of self-report studies and live scenarios prevents a fully objective exploration of actual learning, and therefore perceived learning is likely to continue being used by researchers in this field, despite questions surrounding its appropriateness (e.g. Bacon, 2011; Kennedy et al., 2002; Sitzmann

et al., 2010). As student perceptions of learning may possibly change over time the cross-sectional approach used here has clear limitations. Future research could include the use of longitudinal surveys to measure potential changes, with a view to better understanding the end result of these fluctuations. While 43% of students' perceived learning was explained by this model, making it an initial success, the most obvious question is what factors account for the other 57%? Further, are they so myriad that they prevent the development of a parsimonious model? It seems that purpose-built qualitative research is needed here to gain a clearer understanding of the psychological antecedents to success in SLEs.

On a different note, the lack of a relationship between ease of use and perceived learning was surprising. Future replicate and extension of this study to determine where, why, and how differences in this relationship might occur would be valuable. One particular area of interest is whether the incorporation of virtual / augmented reality, cloud-based applications and other new technologies would cause the relationship to become significant in nature again. Tangential to this, research on how teaching academics may in themselves form barriers to using new technology in the classroom would be beneficial. Further, a different subject area, or different cultures, may produce different results meaning the generalisability of the model cannot be fully assessed without replication.

In relation to work needed on career relatedness and the skills taught during use of SLEs, there is a need to better understand how students make the link between the skills they feel they learn in simulations, and how those skills fit into their careers. Due to the changes which take place over time in students' perceptions of these skills, longitudinal research would be the most beneficial contribution to the area here.

As was highlighted in the discussion section, the emergence of social support as the most influential factor in this model speaks to the need for further research on this construct in relation to the conduct and management of group learning in SLEs. To conclude, multiple avenues of research have opened up following the results of this paper, and we humbly invite others to consider these in future.

References:Abdullah, F., and Ward, R. (2016). ‘Developing a General Extended Technology Acceptance Model for E-Learning (GETAMEL) by analysing commonly used external factors’. Computers in Human Behavior, pp. 238-256. doi.org/10.1016/j.chb.2015.11.036

Abdullah, F., Ward, R. and Ahmed, E. (2016). ‘Investigating the Influence of the Most Commonly Used External Variables of TAM on Students’ Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) of e-Portfolios.’ Computers in Human Behavior, Vol. 63, pp. 75–90. doi.org/10.1016/j.chb.2016.05.014

Adobor, H., and Daneshfar, A. (2006). ‘Management simulations: Determining their effectiveness’. Journal of Management Development, Vol. 25, Iss. 2, pp. 151-168. doi.org/10.1108/02621710610645135

Akcayir, M., Dundar, H., and Akcayir, G. (2016). 'What makes you a digital native? Is it enough to be born after 1980?'. Computers in Human Behavior, Vol. 60, pp. 435-440. doi.org/10.1016/j.chb.2016.02.089

Akkoyunlu, B., and Yılmaz-Soylu, M. (2008). ‘Development of a scale on learners’ views on blended learning and its implementation process’. Internet and Higher Education, Vol. 11, pp. 26–32.doi.org/10.1016/j.iheduc.2007.12.006

Al-Busaidi, K.A. (2013). ‘An empirical investigation linking learners’ adoption of blended learning to their intention of full e-learning’. Behaviour & Information Technology, Vol. 32, Iss. 11, pp. 1168-1176. doi.org/10.1080/0144929X.2013.774047

Alber, J.M., Bernhardt, J.M., Stellefson, M., Weiler, R.M., Anderson-Lewis, C., Miller, M.D. and MacInnes, J. (2015). 'Designing and testing an inventory for measuring social media competency of Certified Health Education Specialists'. Journal of medical Internet research, Vol. 17, Iss. 9, pp. 221 -224. doi.org/10.2196/jmir.4943

Andrews, D. H. and Wulfeck, W. H. II. (2014). ‘Performance Assessment: Something Old, Something New’. In: J. M. Spector, M. D. Merrill, J. Elen, and M. J. Bishop. (2014). Handbook of Research on Educational Communications and Technology, pp. 303-310. doi.org/10.1007/978-1-4614-3185-5_24

Ardley, B. and Taylor, N. (2010). ‘The student practitioner: Developing skills through the marketing research consultancy project’. Marketing Intelligence & Planning, Vol. 28, pp. 847-861. doi.org/10.1108/02634501011086454

Bacon, D.R. (2011). ‘Comparing direct versus indirect measures of the pedagogical effectiveness of team testing’. Journal of Marketing Education, Vol. 33. Iss. 3, pp. 348-358. doi.org/10.1177/0273475311420243

Bacon, D. R. (2016). ‘Reporting Actual and Perceived Student Learning in Education Research’. Journal of Marketing Education, Vol. 38, Iss. 1, pp. 3-6. doi.org/10.1177/0273475316636732

Bagozzi, R. P., and Yi, Y. (1988). 'On the evaluation of structural equation models'. Journal of the Academy of Marketing Science, Vol. 16, Iss. 1, pp. 74-94. doi.org/10.1007/BF02723327

Barker, B. (2014). ‘Employability skills: Maintaining relevance in marketing education’. Marketing Review, Vol. 134, Iss. 1, pp. 29-48. doi.org/10.1362/146934714X13948909473149

Bennis, W. H., and O'Toole, J. (2005). 'How business schools lost their way'. Harvard Business School, Vol. 83, Iss. 5, pp. 96-104.

Beuk, F. (2015). ‘Sales Simulation Games: Student and Instructor Perceptions’. Journal of Marketing Education, Vol. 38, Iss. 3, pp. 170-182. doi.org/10.1177/0273475315604686

Bravo, R., Catalan, S., and Pina, J. M. (2018). ‘Analysing teamwork in higher education: an empirical study on the antecedents and consequences of team cohesiveness’. Studies in Higher Education, article in press. doi.org/10.1080/03075079.2017.1420049

Brennan, R. (2014). ‘Reflecting on experiential learning in marketing education’. Marketing Review, Vol. 14, Iss. 1, pp. 97-108. doi.org/10.1362/146934714X13948909473266

Brennan, R. and Vos, L. (2013). ‘Effects of participation in a simulation game on marketing students’ numeracy and financial skills’. Journal of Marketing Education, Vol. 35, Iss. 3, pp.259-270. doi.org/10.1177/0273475313482928

Britton,E., Simper, N., Leger,A. and Stephenson, J. (2017). ‘Assessing teamwork in undergraduate education: a measurement tool to evaluate individual teamwork skills’. In Assessment & Evaluation in Higher Education, 42, 3378–397. doi.org/10.1080/02602938.2015.1116497

Buil, I., Catalan, S., and Martinez, E. (2018). ‘Exploring students’ flow experiences in business simulation games’. Journal of Computer Assisted Learning, Vol. 34, Iss. 2, pp. 183-192.

Burns, A. C., and Golen, S. P. (1983). ‘Perceived Communication Barriers to Learning with Computer Simulation’. Journal of Marketing Education, Vol. 5, Iss. 2, pp. 13-22. doi.org/10.1177/027347538300500203

Butterly, A., and Crookes, D (2017). ‘”Luxury” Fyre Festival is cancelled with ticket-holders still stranded in Bahamas’. BBC news. Available at: http://www.bbc.co.uk/newsbeat/article/39743303/luxury-fyre-festival-is-cancelled-with-ticket-holders-still-stranded-in-bahamas (Date published: 28/04/17). (Date accessed: 11/03/18).

Cadotte, E.R. (2016). ‘Creating value in marketing and business simulations: An author’s viewpoint’. Journal of Marketing Education, Vol. 38, Iss. 2, pp.119-129. doi.org/10.1177/0273475316649741

Campion, M. A., Medsker, G. J., and Higgs, A. C. (1993). ‘Relations between work group characteristics and effectiveness: Implications for designing effective work groups’. Personnel Psychology, Vol. 46, pp. 823–850. doi.org/10.1111/j.1744-6570.1993.tb01571.x

Canhoto, A.I. and Murphy, J. (2016). ‘Learning from simulation design to develop better experiential learning initiatives: An integrative approach’. Journal of Marketing Education, Vol. 38, Iss. 2, pp.98-106. doi.org/10.1177/0273475316643746

Carver, C. S., Scheier, M. F., and Weintraub, K. J. (1989). ‘Assessing Coping Strategies: A Theoretically Based Approach’. Journal of Personality and Social Psychology, Vol. 56, Iss. 2, pp. 267-283. doi.org/10.1037/0022-3514.56.2.267

Chang, H-Y., Wang, C-Y., Lee, M-H., Wu, H-K., Liang, J-C., Lee, S. W-Y., Chiou, G-L., Lo, H-C., Lin, J-W., Hsu, C-Y., Wu, Y-T., Chen, S., Hwang, F-K., and Tsai, C-C. (2015). ‘A review of features of technology-supported learning environments based on participants’ perceptions’. Computers in Human Behavior, Vol. 53, pp. 223-237. doi.org/10.1016/j.chb.2015.06.042

Chang, Y.-S., and C. Yang. (2013). ‘Why do We Blog? From the Perspectives of Technology Acceptance and Media Choice Factors.’ Behaviour & Information Technology, Vol. 32, Iss. 4, pp. 371–386. doi.org/10.1080/0144929X.2012.656326

Chen, C., Jones, K.T. and Xu, S. (2018). ‘The Association between Students' Style of Learning Preferences, Social Presence, Collaborative Learning and Learning Outcomes’. Journal of Educators Online, Vol.15, Iss. 1, pp. 41-56.

Chin, W. W. (1998). 'Issues and opinion on structural equation modeling', editorial. MIS Quarterly. Vol. 22, Iss. 1, pp. 94-104.

Chiu, C.M., Hsu, M.H., Sun, S.Y., Lin, T.C. and Sun, P.C. (2005). ‘Usability, quality, value and e-learning continuance decisions’. Computers & Education, Vol. 45, Iss. 4, pp. 399-416. doi.org/10.1016/j.compedu.2004.06.001

Cohen, J. (1992). 'A power primer'. Psychological Bulletin, Vol. 112, pp.155-159. doi.org/10.1037/0033-2909.112.1.155

Comer, D. R., Lenaghan, J. A., and Sengupta, K. (2015). 'Factors That Affect Students' Capacity to Fulfill the Role of Online Learner'. Journal of Education for Business, Vol. 90, Iss. 3, pp. 145-155. doi.org/10.1080/08832323.2015.1007906

Chu, R. J-C., and Tsai, C-C. (2009). ‘Self-directed learning readiness, internet self-efficacy and preferences towards constructivist Internet-based learning environments among higher-aged adults’. Journal of Computer Assisted Learning, Vol. 25, pp. 489–501. doi.org/10.1111/j.1365-2729.2009.00324.x

Clark, W., Logan, K., Luckin, R., Mee, A. and Oliver, M. (2009). ‘Beyond Web 2.0: Mapping the technology landscapes of young learners’. Journal of computer assisted learning, Vol. 25, Iss.1, pp.56-69. doi.org/10.1111/j.1365-2729.2008.00305.x

Cobb, S. (1976). ‘Social support as a moderator of life stress’. Psychometric Medicine, Vol. 38, Iss. 5, pp. 300-314. doi.org/10.1097/00006842-197609000-00003

Cohen, S., and Wills, T. A. (1985). ‘Stress, Social Support, and the Buffering Hypothesis’. Psychological Bulletin, Vol. 98, Iss. 2, pp. 310-357. doi.org.ergo.southwales.ac.uk/10.1037/0033-2909.98.2.310

Cutrona, C. E., Cole, V., Colangelo, N., Assouline, S. G., and Russell, D. W. (1994). ‘Perceived parental social support and academic achievement: An attachment theory perspective’. Journal of Personality and Social Psychology, Vol. 66, Iss. 2, pp. 369-378. doi.org.ergo.southwales.ac.uk/10.1037/0022-3514.66.2.369

Darban, M. and Polites, G. L. (2016). ‘Do emotions matter in technology training? Exploring their effects on individual perceptions and willingness to learn’. Computers in Human Behaviour, Vol. 62, pp. 644-657. doi.org/10.1016/j.chb.2016.04.028

Davis, F. D. (1989). ‘Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,’ MIS Quarterly, Vol. 13, Iss. 3, pp. 318-339. doi.org/10.2307/249008

Davis, F. D., Bagozzi, R. P., and Warshaw, P. R. (1989). ‘User Acceptance of Computer Technology: A Comparison of Two Theoretical Models.’ Management Science, Vol. 35, Iss. 8), pp. 982-1003. doi.org/10.1287/mnsc.35.8.982

De Janaasz, S. C., and Forret, M. L. (2007). ‘Learning The Art of Networking: A Critical Skill for Enhancing Social Capital and Career Success’. Journal of Management Education, Vol. 32, Iss. 5, pp. 629-650. doi.org/10.1177/1052562907307637

Doyle, D. and Brown, F.W. (2000). ‘Using a business simulation to teach applied skills–the benefits and the challenges of using student teams from multiple countries’. Journal of European industrial training, Vol. 24, Iss. 6, pp.330-336. doi.org/10.1108/03090590010373316

Eggens, L., Van Der Werf, M. P. C., and Bosker, R. J. (2008). ‘The influence of personal networks and social support on study attainment of students in university education’. Higher Education, Vol. 55, Iss. 5, pp. 53-573. doi.org/10.1007/s10734-007-9074-4

Elbeck, M. and Bacon, D. R. (2015). ‘Towards universal definitions for direct and indirect assessment’. Journal of Education for Business, Vol. 90, pp. 278-283. doi.org/10.1080/08832323.2015.1034064

Faria, A.J. and Wellington, W.J. (2004). ‘A survey of simulation game users, former-users, and never-users’. Simulation & Gaming, Vol. 35, Iss. 2, pp.178-207. doi.org/10.1177/1046878104263543

He, Y., Chen, Q. and Kitkuakul, S. (2018). ‘Regulatory Focus and Technology Acceptance: Perceived Ease of Use and Usefulness as Efficacy’. Cogent Business & Management, (just-accepted), pp.1 - 52. doi.org/10.1080/23311975.2018.1459006

Flostrand, A., Ho, J.Y. and Krider, R.E. (2016). ‘Implementing “Marketing Me” A Simulation Enhanced Variant for a Student Self-Marketing Exercise’. Journal of Marketing Education, Vol. 38, Iss. 2, pp.83-89. doi.org/10.1177/0273475316643567

Folkman, S., and Lazarus, R. S. (1985). ‘If It Changes It Must Be a Process. Study of Emotion and Coping During Three Stages of a College Examination’. Journal of Personality and Social Psychology, Vol. 48, Iss. 1, pp. 150-170. doi.org.ergo.southwales.ac.uk/10.1037/0022-3514.48.1.150

Francescato, D., Porceilli, R., Mebane, M., Cudetta, M., Klobas, J., and Renzi, P. (2006). ‘Evaluation of the efficacy of collaborative learning in face-to-face and computer-supported university contexts’. Computers in Human Behavior, Vol. 22, Iss. 2, pp. 163-176. doi.org/10.1016/j.chb.2005.03.001

Frankham, J. (2017). 'Employability and higher education: the follies of the ‘Productivity Challenge’ in the Teaching Excellence Framework', Journal of Education Policy, Vol. 32, Iss. 5, pp. 628-641. doi.org/10.1080/02680939.2016.1268271

Frankl, G., and Bitter, S. (2013). ‘Collaboration is smart: Smart learning communities’. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) Volume 7947 LNCS, 2013, Pages 293-3023rd International Workshop on Human-Computer Interaction and Knowledge Discovery in Complex, Unstructured, Big Data, HCI-KDD 2013, Held at South CHI 2013; Maribor; Slovenia; 1 July 2013 through 3 July 2013; Code 97742.

Fuller, R. P. (2016). ‘The big breach: An experiential learning exercise in mindful crisis communication’. Communication Teacher, Vol. 30, pp. 27-32. doi.org/10.1080/17404622.2015.1102306

Gan, B., Menkhoff, T., and Smith, R. (2015). ‘Enhancing students’ learning process through interactive digital media: New opportunities for collaborative learning’. Computers in Human Behavior, Vol. 51, part B, pp. 652-663. doi.org/10.1016/j.chb.2014.12.048

Ganesh, G. and Sun, Q. (2009). ‘Using simulations in the undergraduate marketing capstone case course’. Marketing Education Review, Vol. 19, Iss. 1, pp.7-16. doi.org/10.1080/10528008.2009.11489054

Good, M., and Adams, G. R. (2008). ‘Linking academic social environments, ego-identity formation, ego virtues, and academic success’. Adolescence, Vol. 43, Iss. 170, pp. 221-236.

Hair, J. F. Jr., Hult, G. T. M., Ringle, C. M., and Sarstedt, M. (2017). A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM). (2nd edition). London; Sage.

Helsper, E. J., and Eynon, R. (2010). 'Digital natives: Where is the evidence?'. British Educational Research Journal, Vol. 36, Iss. 3, pp. 503-520. doi.org/10.1080/01411920902989227

Henseler, J., Ringle, C. M., and Sarstedt, M. (2015). 'A new criterion for addressing discriminant validity in variance-based structural equation modeling'. Journal of the Academy of Marketing Science, Vol. 43, pp. 115-135. doi.org/1007/s11747-014-0403-8

Ifinedo, P., Pyke, J. and Anwar, A. (2018). ‘Business undergraduates’ perceived use outcomes of Moodle in a blended learning environment: The roles of usability factors and external support’. Telematics and Informatics, Vol.35, Iss. 1, pp.93-102. doi.org/10.1016/j.tele.2017.10.001

Islam, A. K. M. N. (2016). ‘E-learning system use and its outcomes: moderating role of perceived compatibility’. Telematics and Informatics. Vol. 33, Iss. 1, pp. 48–55. doi.org/10.1016/j.tele.2015.06.010

Johnson, T. E., Top, E., and Yukselturk, E. (2015). ‘Team shared mental model as a contributing factor to team performance and students’ course satisfaction in blended courses’. Computers in Human Behavior, Vol. 27, Iss. 6, pp. 2330-2338. doi.org/10.1016/j.chb.2011.07.012

Jones, C., Ramanau, R., Cross, S., and Healing, G. (2010). 'Net generation or Digital Natives: Is there a distinct new generation entering university?'. Computers and Education, Vol. 54, Iss. 3, pp. 722-732. doi.org/10.1016/j.compedu.2009.09.022

Kennedy, E. J., Lawton, L., and Plumlee, E. L. (2002). ‘Blissful ignorance: The problem of unrecognized incompetence and academic performance’. Journal of Marketing Education, Vol. 24, pp. 243-252. doi.org/10.1177/0273475302238047

Kietzmann, J., and Pitt, L. (2016). ‘Using Simulations in the Marketing Classroom’. Journal of Marketing Education, Vol. 38, Iss. 2, pp. 71-72. doi.org/10.1177/0273475316653542

Kinshuk, Chen, N-S., Cheng, I-I., Chew, S. W. (2016). ‘Evolution Is not enough: Revolutionizing Current Learning Environments to Smart Learning Environments’. International Journal of Artificial Intelligence in Education, Vol. 26, Iss. 2, pp. 561-581. doi.org/10.1007/s40593-016-0108-x

Klimova, B. (2015). ‘Assessment in smart learning environment – a case study approach’. Smart Innovation, Systems and Technologies. Volume 41, 2015, pp. 15-24. 2nd International KES Conference on Smart Education and Smart e-Learning, SEEL 2015; Sorrento; Italy; 17th-19th June 2015; Code 157689. doi.org/10.1007/978-3-319-19875-0_2

Langford, C. P. H., Bowsher, J., Maloney, J. P., and Lillis, P. P. (1997). ‘Social support: A conceptual analysis’. Journal of Advanced Nursing, Vol. 25, Iss. 1, pp. 95-100. doi.org/10.1046/j.1365-2648.1997.1997025095.x

Levitt, M. J., Guacci-Franco, N., and Levitt, J. L. (1994). ‘Social support and achievement in childhood and early adolescence: A multicultural study’. Journal of Applied Developmental Psychology, Vol. 15, Iss. 2, pp. 207-222. doi.org/10.1016/0193-3973(94)90013-2

Lin, G.Y. (2018). ‘Anonymous versus identified peer assessment via a Facebook-based learning application: Effects on quality of peer feedback, perceived learning, perceived fairness, and attitude toward the system’. Computers & Education, Vol. 116, pp.81-92. doi.org/10.1016/j.compedu.2017.08.010

Lopez-Yanez, I., Yanez-Marquez, C., Camacho-Nieto, O., Aldape-Perez, M., and Argulles-Cruz, A-J. (2015). ‘Collaborative learning in postgraduate level courses’. Computers in Human Behaviour, Vol. 51, part B, pp. 938-944. doi.org/10.1016/j.chb.2014.11.055

Liu, Y. C., and Hung, Y. Y. (2016). ‘Self-efficacy as the moderator: Exploring driving factors of perceived social support for mainland Chinese students in Taiwan’. Computers in Human Behaviour, Vol. 64, pp. 455-462. doi.org/10.1016/j.chb.2016.07.018

Low, J. T. (1980). ‘Guidelines for the Use of Business Simulation Games’. Journal of Marketing Education, Vol. 2, Iss. 1, pp. 71-72. doi.org/10.1177/027347538000200106

Morris, M. R., Ryall, K., Shen, C., Forlines, C., and Vernier, F. (2004). ‘Beyond ”social protocols”: multi-user coordination policies for co-located groupware’. In CSCW ’04: Proceedings of the 2004 ACM conference on Computer supported cooperative work, pp. 262–265, New York, NY, USA, 2004. ACM Press. doi.org/10.1145/1031607.1031648

Nevison, C., Drewery, D., Pretti, J., and Cormier, L. (2017). ‘Using learning environments to create meaningful work for co-op students’. Higher Education Research & Development, Vol. 36, Iss. 4, pp. 807-822. doi.org/10.1080/07294360.2016.1229268

Novakovich, J., Miah, S. and Shaw, S. (2017). 'Designing curriculum to shape professional social media skills and identity in virtual communities of practice'. Computers and Education, Vol. 104, pp. 65–90. doi.org/10.1016/j.compedu.2016.11.002

Olson, K. S. (2012). ‘Making it real: Using a collaborative simulation to teach crisis communications’. Journal of Excellence in College Teaching, Vol. 23, pp. 25-47.

Paechter, M., Maier, B., and Macher, D. (2010). ‘Students’ expectations of, and experiences in e-learning: Their relation to learning achievements and course satisfaction’. Computers and Education, Vol. 54, Iss. 1, pp. 222-229. doi.org/10.1016/j.compedu.2009.08.005

Pavlou, P.A. and Fygenson, M. (2006). ‘Understanding and predicting electronic commerce adoption: An extension of the theory of planned behavior’. MIS Quarterly, Vol. 30, Iss. 1, pp. 115-143. doi.org/10.2307/25148720

Ranchhod, A., Gurău, C., Loukis, E. and Trivedi, R. (2014). ‘Evaluating the educational effectiveness of simulation games: A value generation model’. Information Sciences, Vol. 264, pp.75-90. doi.org/10.1016/j.ins.2013.09.008

Raymond, C. and Sorensen, K. (2008). ‘The use of a middle east crisis simulation in an international relations course’. PS: Political Science and Politics, Vol. 41, Iss. 1, pp. 179-182. doi.org/10.1017/S104909650808027X

Redmond, W. H. (1989). ‘On the Duration of Simulations: An Exploration of Minimum Effective Length’. Journal of Marketing Education, Vol. 11, Iss. 1, pp. 53-57. doi.org/10.1177/027347538901100110

Richardson, J.C., Maeda, Y., Lv, J. and Caskurlu, S. (2017). ‘Social presence in relation to students' satisfaction and learning in the online environment’. Computers in Human Behavior, Vol. 71(C), pp.402-417. doi.org/10.1016/j.chb.2017.02.001

Riebe, L., Girardi, A. and Whitsed, C. (2016). ‘A systematic literature review of teamwork pedagogy in higher education’. Small Group Research, Vol. 47, Iss. 6, pp. 619– 664. doi.org/10.1177/1046496416665221

Ringle, C. M., Wende, S., and Becker, J-M. (2015). ‘SmartPLS 3’. Boenningstedt: SmartPLS GmbH, http://www.smartpls.com.

Sánchez, R.A. and Hueros, A.D. (2010). ‘Motivational factors that influence the acceptance of Moodle using TAM’. Computer in Human Behaviour, Vol. 26, pp. 1632–1640. doi.org/10.1016/j.chb.2010.06.011

Sebian, U., and Tominc, P. (2015). ‘Impact of support of teacher and compatibility with need of study on usefulness of SPSS by students’. Computers in Human Behavior, Vol. 53, pp. 354-365. doi.org/10.1016/j.chb.2015.07.022

Sherbourne, C. D., and Stewart, A. L. (1991). ‘The MOS social support survey’. Social Sciences and Medicine, Vol. 32, Iss. 6, pp. 705-714. doi.org/10.1016/0277-9536(91)90150-B

Sitzmann, T., Ely, K., Brown, K. and Bauer, K. (2010). ‘Self-assessment of knowledge: A cognitive learning or affective measure?’ Academy of Management Learning & Education, Vol. 9, pp. 169-191.

Spector, J.M, and Merrill, M. D. (2008). ‘Effective, efficient and engaging (E3) learning in the digital age'. Distance Education, Vol. 29, Iss. 2, pp. 123-126.

Spector, J. M. (2014). ‘Conceptualizing the emerging field of smart learning environments’. Smart Learning Environments, Vol. 1, Iss. 2, pp. 1-10. doi.org/10.1186/s40561-014-0002-7

Thompson, P. (2015). 'How digital native learners describe themselves'. Education and Information Technologies, Vol. 20, Iss. 3, pp. 467-484. doi.org/10.1007/s10639-013-9295-3

Tsai, P-S., Tsai, C-C., and Hwang, G-J. (2012). ‘Developing a survey for assessing preferences in constructivist context-aware ubiquitous learning environments’. Journal of Computer Assisted Learning, Vol. 28, pp. 250–264. doi.org/10.1111/j.1365-2729.2011.00436.x

Turel, O., Serenko, A. and Giles, P. (2011). ‘Integrating technology addiction and use: An empirical investigation of online auction users’. Management Information Systems Quarterly, Vol. 35, Iss. 4, pp. 1043-1051.

Tymon, A. (2013). 'The Student Perspective on Employability'. Studies in Higher Education, Vol. 38, Iss. 6, pp. 841–856. doi.org/10.1080/03075079.2011.604408

Venkatesh, V. (2000). ‘Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model’. Information systems research, Vol. 11, Iss. 4, pp. 342-365. doi.org/10.1287/isre.11.4.342.11872

Venkatesh, V., Davis, F. D. and Morris, M. G. (2007). ‘Dead or alive? The development, trajectory and future of technology adoption research’. Journal of the Association for Information Systems, Vol. 8, Iss. 4, pp. 267-286.

Ventura, R., Roca-Cuberes, C. and Corral-Rodríguez, A. (2018). Interactive Digital Communication: assessment of academic competencies and professional profiles by communication professionals, faculty and students. Revista Latina de Comunicación Social, Vol. 73, pp.331-351. doi.org/10.4185/RLCS-2018-1258-17en

Vos, L. and Brennan, R. (2010). ‘Marketing simulation games: student and lecturer perspectives’. Marketing Intelligence & Planning, Vol. 28, Iss. 7, pp.882-897. doi.org/10.1108/02634501011086472

Wang, M. (2016). ‘Using Crisis Simulation to Enhance Crisis Management: The Role of Presence’, ADPR 450: Public Relations Theory and Strategy—A Peer Review of Teaching Project Inquiry Portfolio. Available at: http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1025&context=prtunl (Date accessed: 07/04/18).

Wentzel, K. R. (1998). ‘Social relationships and motivation in middle school: The role of parents, teachers, and peers’. Journal of Educational Psychology, Vol. 90, Iss. 2, pp. 202-209. doi.org/10.1037/0022-0663.90.2.202

Wentzel, K. R. (1999). ‘Social-motivational processes and interpersonal relationships: Implications for understanding motivation at school’. Journal of Educational Psychology, Vol. 91, Iss. 1, pp. 76-97. doi.org/10.1037/0022-0663.91.1.76

Wentzel, K.R., Russell, S. and Baker, S. (2016). ‘Emotional support and expectations from parents, teachers, and peers predict adolescent competence at school’. Journal of Educational Psychology, Vol. 108, Iss. 2, pp. 242. doi.org.ergo.southwales.ac.uk/10.1037/edu0000049

Werner, J. M., and Lester, S. W. (2002). ‘Applying a team effectiveness framework to the performance of student case teams’. Human Resource Development Quarterly, Vol. 12, Iss. 4, pp. 385-402. doi.org/10.1002/hrdq.1004

Wilcox, P., Winn, S., and Fyvie-Gauld, M. (2005). ‘It was nothing to do with the university, it was just the people’: The role of social support in the first-year experience of higher education’. Studies in Higher Education, Vol. 30, Iss. 6, pp. 707-722. doi.org/10.1080/03075070500340036

Wills, T. A. (1991). ‘Social support and interpersonal relationships'. In M. S. Clark (Ed.), Review of personality and social psychology’, Vol. 12. pp. 265-289.

Xerri, M.J., Radford, K. and Shacklock, K. (2017). ‘Student engagement in academic activities: a social support perspective’. Higher Education, pp.1-17. doi.org/10.1007/s10734-017-0162-9

Yeou, M. (2016). ‘An investigation of students’ acceptance of Moodle in a blended learning setting using technology acceptance model’. Journal of Educational Technology Systems, Vol. 44, Iss. 3, pp. 300-318. doi.org/10.1177/0047239515618464

Appendix 1: QuestionnaireSocial support

SS1 My team increase my chances at success.SS2 My team can help each other when necessary. SS3 My team can work well together.SS4 My team has the knowledge to succeed in this simulation. SS5 My team cares about helping each other.

RelatednessRE1 Taking part in this simulation will help me to learn crisis management skills.RE2 Taking part in this simulation will help me to learn teamwork skills.RE3 Taking part in this simulation will help me to learn digital communication skills.RE4 Taking part in this simulation will help me to learn non-digital communication skills.RE5 Taking part in this simulation will improve my future job performance.RE6 This simulation will be useful to me in my future career.

Ease of useEU1 I understand how to use the equipment in the room.EU2 Interacting with the equipment won’t require a lot of mental effort.EU3 I find smartphones, laptops, printers, and screens easy to use.EU4* It is easy to get the equipment to do what I want it to do. EU5 I feel confident about my ability to use technology during the simulation.

Perceived learningPL1 I will get a good grade in this simulation.PL2 I can effectively use the skills I learned for crisis management.

PL3* I understand the link between theory and practice for crisis communications.PL4 Live simulations are a fair way to assess my skills.PL5 I am happy to be assessed with a live simulation.

*removed from model