Computers & Education 68 (2013) 314–321

Contents lists available at SciVerse ScienceDirect

Computers & Education

journal homepage: www.elsevier .com/locate/compedu

An investigation of learners’ collaborative knowledge constructionperformances and behavior patterns in an augmented realitysimulation system

Tzung-Jin Lin a,*, Henry Been-Lirn Duh b, Nai Li c, Hung-Yuan Wang a, Chin-Chung Tsai d

aGraduate Institute of Applied Science and Technology, National Taiwan University of Science and Technology, #43, Sec.4, Keelung Rd., Taipei 106, TaiwanbDepartment of Electrical and Computer Engineering, National University of Singapore, #21 Lower Kent Ridge Road, Singapore 119077, SingaporecCommunications and New Media Programme, National University of Singapore, #21 Lower Kent Ridge Road, Singapore 119077, SingaporedGraduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, #43, Sec.4, Keelung Rd., Taipei 106, Taiwan

a r t i c l e i n f o

Article history:Received 21 March 2013Received in revised form6 May 2013Accepted 9 May 2013

Keywords:Cooperative/collaborative learningInteractive learning environmentsSimulationsVirtual realityEvaluation of CAL systems

* Corresponding author. Tel.: þ886 2 27303219; faxE-mail address: D9822302@mail.ntust.edu.tw (T.-J

0360-1315/$ – see front matter � 2013 Elsevier Ltd. Ahttp://dx.doi.org/10.1016/j.compedu.2013.05.011

a b s t r a c t

The purpose of this study was to investigate how a mobile collaborative augmented reality (AR)simulation system affects learners’ knowledge construction behaviors and learning performances. Inthis study, 40 undergraduate students were recruited and divided into dyads to discuss a given taskeither with the assistance of a mobile collaborative AR system or traditional 2D simulation system.The participants’ knowledge acquisition regarding elastic collision was evaluated through a pre-testand a post-test comparison. Learners’ knowledge construction behaviors were qualitatively identifiedaccording to an adapted three-category coding scheme including construction of problem space (PS),construction of conceptual space (CS), and construction of relations between conceptual and problemspace (CPS), and were then analyzed by adopting lag sequential analysis. The results indicated that thelearners who learned with the AR system showed significant better learning achievements than thosewho learned with the traditional 2D simulation system. Furthermore, the sequential patterns of thelearners’ behaviors were identified, including three sustained loops (PS/PS, CS/CS, CPS/CPS), a bi-directional path between the PS and CPS activities (PS4CPS), and a one way path from the PS activityto the CS activity (PS/CS). The revealed behavior patterns suggest that the AR Physics system mayserve as a supportive tool and enable dyad learners to respond quickly to the displayed results andsupport their knowledge construction processes to produce a positive outcome. Based on thebehavioral patterns found in this study, suggestions for future studies and further modifications to thesystem are proposed.

� 2013 Elsevier Ltd. All rights reserved.

1. Introduction

1.1. Augmented reality

Augmented reality (AR), as indicated by Azuma (1997), can be defined as a system or visualization technique that fulfills three mainfeatures: a combination of real and virtual worlds, real time interaction, and accurate 3D registration of virtual and real objects. In general,AR is commonly accepted as a real-time technology of a physical environment that has been augmented by adding virtual information to it(e.g., Carmigniani et al., 2011). Besides, AR can be regarded as one of the formats within the notion of the Reality–Virtuality (RV) continuum,ranging from a completely real environment to a completely virtual one (Milgram & Kishino, 1994). Due to the possibilities of the mixture ofreal and virtual worlds within a single display (i.e., a Mixed Reality (MR) environment), AR can be thought of as an MR environment, whichcontains more real world objects than virtual elements.

: þ886 2 27303222.. Lin).

ll rights reserved.

T.-J. Lin et al. / Computers & Education 68 (2013) 314–321 315

1.2. Mobile AR in education

Researchers (e.g., Chen & Tsai, 2012; Squire & Klopfer, 2007) have suggested that educational AR technology could provide learners with amore immersive and engaging learning environment without decreasing the authenticity of the real world. More recently, researchers haveshifted their attention to applying AR on handheld devices such as mobile phones (Azuma, Billinghurst, & Klinker, 2011; Martin et al., 2011).With the characteristics of AR, ubiquitous computing, and portability (Papagiannakis, Singh, & Magnenat-Thalmann, 2008), the applicationof mobile AR in education is currently rapidly increasing. In the 2011 Horizon Report, the potential pedagogical applications of mobile ARbegan to draw researchers’ attention, and more empirical studies in this regard have been called for (Johnson, Smith, Willis, Levine, &Haywood, 2011). Martin et al. (2011) also echoed this perspective based on a thorough investigation regarding the technology trends ineducation, and forecasted that the practical usage of mobile AR in education should be one of the promising research areas in the nearfuture. Yet, Azuma et al. (2011) indicated an urgent need for more related research on how to use handheld devices to deliver a compellinguser mobile AR experience, implying insufficient research on the effectiveness of and learner behaviors while using mobile AR foreducational purposes.

In addition, AR with handheld devices has revealed great possibilities for supporting face-to-face collaborative learning, and enhancingsocial interactivity between or among group learners (Behzadan, Iqbal, & Kamat, 2011; Klopfer, 2008). According to Li, Gu, Chang, and Duh(2011), handheld AR devices enable co-located learners to be involved in knowledge construction by exploring the outside environment andinteracting with each other through the guidance of AR-supported information. The referential resources provided by virtual informationcould support the development of mutual understanding among collaborators by increasing the richness of meaning negotiation (Suthers &Hundhausen, 2003). In other words, the usage of AR simulation enables learners to experience scientific phenomena (e.g., elastic collision)as well as underlying scientific constructs (e.g., kinetic energy and momentum) that are not easily observed in the real world, supportingthemwith more authentic learning practices (Klopfer & Squire, 2008). Even though physical laboratory environment may allow students tohave the opportunity to observe such phenomena, the experimental apparatuses oftentimes constrain students to make the observationsand measurements necessary to develop and confirm their scientific conceptions (Marshall & Young, 2006; Trundle & Bell, 2010). Forexample, although the phenomenon of elastic collision can be explained and experimented using a Newton’s cradle, the underlying sci-entific constructs such as kinetic energy or momentum are nearly impossible for students to recognize without the support of computersimulation. Klopfer and Squire (2008) also mentioned several advantages of using mobile or handheld devices, including portability, socialinteractivity, context sensitivity and others. In sum, it is implied that a mobile AR platform, with its mobility and ubiquitousness, mayaugment learners’ interaction experience without the restrictions of fixed locations (e.g., desktop computers) and provide the opportunitiesof “ubiquitous knowledge construction” (e.g., Peng, Su, Chou, & Tsai, 2009).

1.3. Collaborative knowledge construction

In the past decades, the studies of learners’ collaborative processes and learning outcomes in computer-supported collaborative learningenvironments have increased considerably (e.g., Fischer, Bruhn, Grasel, & Mandl, 2002; Hmelo-Silver, 2003; Jamaludin, Chee, & Ho, 2009;Weinberger & Fischer, 2006). It has been proven that, when collaborating learners are provided with content-specific visualization tools, theprocesses and outcomes of collaborative learning are improved (Gao, Baylor, & Shen, 2005). “Collaborative knowledge construction” is oneof the terms that is commonly used in research to describe learners’ cognitive processes during collaborative learning (e.g., Fischer et al.,2002; Hmelo-Silver, 2003). Rather than focusing on learners’ learning achievements and outcomes, the exploration of their discourseprocesses is also highlighted by researchers (Meier, Spada, & Rummel, 2007; Van Aalst, 2009).

It is advocated that, by analyzing the discourse between group learners, their cognitive processes of learning could be externalized (Chi,1997). For instance, Fischer et al. (2002) analyzed the discourses of collaborative knowledge construction with a sample of 32 un-dergraduates. One of the focal points of this study was to identify participants’ content of the process of collaborative construction ofknowledge (i.e., all task-related theoretical concepts, the task-relevant case information as well as the interrelations between them). Later, ahandful of researchers (Jamaludin et al., 2009; Weinberger & Fischer, 2006) further regarded this content analysis approach as a repre-sentation of externalizing participants’ epistemic activities during the process of collaborative knowledge construction. This epistemicdimension of collaborative knowledge construction offers researchers opportunities to understand how learners engage in activities to solvetheir on-task discourse or tasks (Weinberger & Fischer, 2006).

1.4. Enhancing learners’ understanding of elastic collision and momentum with technology

The concept of momentum in physics is a critical topic for science educators (e.g., Bryce &MacMillan, 2009; Graham & Berry, 1996; Ingec,2009). In an earlier study investigating students’misconceptions regardingmechanics, Camp and Clement (1994) identified a list of learners’misconceptions about collision that are closely related to applying the concepts of momentum and kinetic energy in different situations (seeCamp & Clement, 1994). Although energy and momentum are the most fundamental concepts in physics, Singh and Rosengrant (2003)highlighted that many students lack a coherent understanding of and have learning obstacles regarding these relevant concepts. Someresearchers (Bryce & MacMillan, 2009; Ingec, 2009) have advocated that a more lucrative pedagogical approach should be adopted thatmoves away from the traditional mechanistic, number-crunching, rote memorization approach. For example, George, Broadstock, andVasquez Abaz (2000) have attempted to support students’ learning of collision and concepts of energy, momentum, and conservationlaws in laboratories by using computer-aided technologies. Marshall and Young (2006) also attempted to develop prospective teachers’understanding of collisions with both physical manipulatives and an interactive 2D computer simulation called Interactive Physics. Thesestudies may suggest that the usage of technology may be regarded as beneficial in supporting students to learn the related concepts of(elastic) collision.

As previously mentioned, a technology-enhanced learning environment would be one of the approaches that shows potential. Rutten,van Joolingen, & van der Veen (2012) reviewed more than fifty publications concerning the learning effects of simulations delivered bycomputers and other equipment such as mobile devices in science education. In general, they concluded that computer simulations can not

T.-J. Lin et al. / Computers & Education 68 (2013) 314–321316

only benefit traditional instruction as a supplement, but can also even possibly serve as a replacement for traditional instructionwith regardto enhancing learners’ conceptual understanding. In addition, it is advocated that learning artifacts as shared representations can be aneffective representational tool to facilitate collaboration and knowledge construction (Gao et al., 2005). Researchers (e.g., Fischer et al., 2002)have suggested that a more content-specific structuring method, including providing structural support or visualizations of abstractcharacteristics of the task, will be able to support the learning partners in the qualitative processing of the task. It is suggested that task-relevant externalization of knowledge or abstract concepts as well as relations between concepts can be advanced with a content-specific learning artifact (Chiu & Hsiao, 2010; Slof, Erkens, Kirschner, & Jaspers, 2010).

There are, so far, still relatively few studies investigating learners’ knowledge construction processes assisted by mobile collaborative ARtechnology. As suggested by Cheng and Tsai (2013), future research is required to explore students’ learning processes by mixed methodssuch as lag sequential analysis which aims to detect whether the sequential relationship between each behavior has been achievedsignificantly and visualize the patterns. This paper also argues here that it is essential to gain an understanding of the strengths ofcollaborative AR in facilitating the knowledge construction process of collaborative learning. Accordingly, in the current study, we adoptedthis approach and developed a mobile augmented reality simulation system named “AR Physics” to promote collaborative knowledgeconstruction on the topic of elastic collision. Fischer et al. (2002) also pointed out that “individual knowledge construction is often given littleattention (p.215),” suggesting that researchers should not neglect individual learning outcomes even in the co-construction of knowledge ina given context or task. Besides, inspired by the review of Cheng and Tsai (2013), this study aimed to apply lag sequential analysis to identifyand understand learners’ behavioral patterns when using a mobile AR system. In sum, not only individual learners’ collaborative learningachievements but their collaborative knowledge construction processes are included and scrutinized in this study in order to attain a betterunderstanding regarding knowledge construction activities, and to propose suggestions for future work.

Based on the aforementioned theoretical foundations, themain purpose of this studywas to investigate the impact of amobile AR system(i.e., AR Physics) to support learners’ collaborative knowledge construction processes and enhance their learning achievements regardingthe topic of elastic collision. Through an experimental design, the differential impact of a traditional 2D mobile simulation system and a 3Dmobile simulation system (i.e., AR Physics) on the participants’ acquisition of elastic collision concepts was compared. The specific researchquestions are the following: (1) Did the learners who learnwith AR Physics have better learning achievement than those who learnwith thetraditional 2D simulation system regarding the topic of elastic collision? (2) What were the learners’ collaborative knowledge constructionbehaviors when using AR Physics?

2. Method

2.1. Participants

This preliminary study recruited 20 dyads (N ¼ 40) of undergraduate students from a university located in Singapore. The criterion ofbeing a participant was that he/she had not learned elastic collision before. The experimental group consisted of 8 males and 12 femaleswhose ages ranged from21 to 26 years old (M¼ 22.15, SD¼ 1.50). In addition, none of themhad any prior experience of using AR technology.The control group included 7 males and 13 females whose ages from 21 to 24 years old (M ¼ 21.70, SD ¼ 0.98). To ensure the two groups ofstudents had equivalent prior knowledge before the treatment, a t test was conducted in terms of their pre-test scores. The result shows thatthe learners from both groups had no statistical significant difference in their pre-test scores (t (38)¼ 0.26, p> 0.05), indicating that the twogroups had analogous prior knowledge regarding the topic of elastic collision before the start of the present treatment.

2.2. AR Physics system

The software of “AR Physics” (Fig. 1) was implemented on an HTC Desire running Android OS 2.2 with a supporting server program on alaptop. The computationallyexpensive algorithmic computations formarkerdetection andphysics simulation regardingelastic collisionweredesigned to be carried out on the server, with the results sent back and visualized on the mobile phone. The AR Physics system conforms tocurrent AR traits proposed by researchers (Cheng&Tsai, 2013;Wu, Lee, Chang, & Liang, 2013). For instance, the dyad learners in theARPhysicsgroup could manipulate the augmented 3D objects blended in the real environment. In other words, the dyad learners can inspect the 3Dcubes fromavariety of different perspectives to visualize the collisionprocess.Moreover, the systemexploits the affordances of the realworldby providing additional information that augments learners’ experience of reality as suggested by Squire and Klopfer (2007).

As shown in Fig. 1, the system enables the learners in each dyad to visualize two 3D virtual cubes on a marker shown in the left part ofFig.1. Each cube is controlled by one user who canmanipulate themass and the initial velocity of the cube through the input interface (in the

Fig. 1. Screenshots of the AR Physics system.

T.-J. Lin et al. / Computers & Education 68 (2013) 314–321 317

right part of Fig. 1) provided to simulate elastic collision. The simulation process only starts after receiving the data from both learners. Thewhole collision process is visualized with real-time numerical data of mass, velocity, momentum, and kinetic energy of the two objectsdisplayed on the left side of the screen. Furthermore, during the discussion process, the dyad learners are free to choose the appropriateperiod to use the system and are allowed to run simulations as many times as they may need to derive the answers to the questions.

2.3. Traditional 2D Physics

A traditional simulation system named “2D Physics”was developed in this study to enable the dyad learners of control group to simulatethe elastic collision process in a 2D interface (Fig. 2). The settings of “2D Physics”were similar to those in the “AR Physics” system, includingthe variables displayed on themobile phone screen and using cubes as representation. Yet, the dyad learners in the traditional 2D simulationcould only simulate the elastic collision in a 2D virtual interface.

Fig. 2. A screenshot of the traditional 2D Physics system.

2.4. Procedure

First, all participants were required to independently read the instructional material about the topic of elastic collision adopted from theSingapore published textbook prepared by the Physics department of a local Junior College since the participants have not learned elasticcollision before. The average time period of finishing reading it for individual learners was twenty minutes. Next, they were individuallyadministered the pre-test regarding the knowledge of elastic collision (multiple-choice questions) adapted from the previous literature (e.g.,Graham& Berry,1996; Singh & Rosegrant, 2003) with a highest total score of ten. The participants were then instructed to use the AR Physicssystem after the pre-test.

Two open-ended questions related to elastic collision were then presented to each dyad as a discussion task in both groups: (1) In thecontext that object B is stationary and object A moves toward B, how many kinds of subsequent motions can happen after the elasticcollision? How does the relationship between the masses of the two objects influence their subsequent motions after the collision? (2) Howdo you explain the change of motion of the two objects after elastic collision? Once the dyad learners reached consensus on the answers ofthe above-mentioned questions, they submitted a discussion summary. The duration of the treatments in both groups (i.e., discussionactivity) lasted no longer than sixty minutes for each dyad. Besides, the participants’ collaborative knowledge construction processes wererecorded by both video and audio and then transcribed into verbatim for later analysis.

Finally, to assess individual learning performance after the given task, they were required to take another version of the individual post-test (with a highest total score of ten) that was similar to the pre-test multiple-choice questions regarding the knowledge of elastic collision.It should be noted that both the pre- and post-test were verified and validated by two science education researchers to ensure theirmeasurement validities and adapted from the previous literature (e.g., Graham & Berry, 1996; Singh & Rosegrant, 2003). The two samplequestions from pre-test and post-test, respectively, are listed as follows:

(1) Pre-test sample question: “Q3. Two particles of equal mass undergo an elastic collision. Particle #1 has a speed of 10.0 m/s toward east,and particle #2 is at rest. After the collision, what is the velocities of particle #2? (A).12.0m/s, (B). 5.0 m/s, (C).10m/s, (D). zero, (E). Noneof the above”

(2) Post-test sample question: “Q4. Which object has the greatest momentum? (A). a 0.001 kg bumblebee traveling at 2 m/s, (B). a 0.1 kgbaseball traveling at 20 m/s, (C). a 5 kg bowling ball traveling at 3 m/s, (D). a 10 kg sled at rest”

The alternate-form reliability coefficient, measured by the value of the correlation coefficient between the 40 participants’ pre-test andpost-test scores, is 0.78. The reliability coefficient suggests that the tests had fairly adequate reliability for assessing the participants’knowledge of elastic collision.

2.5. Data analysis

A t-test was conducted to compare the learners’ learning achievements in terms of their post-test scores between experimental (ARPhysics) and control groups (2D Physics). In addition, the Cohen’s effect size index d (Cohen, 1988) was computed to illustrate the extent of

T.-J. Lin et al. / Computers & Education 68 (2013) 314–321318

the practical significant difference between groups. It should be noted that Cohen’s d values of 0.20, 0.50, 0.80, and 1.0 are interpreted assmall, medium, large, and very large effect sizes, respectively. Besides, in order to understand how the dyad learners engaged in specificepistemic activities during the knowledge construction process, a coding scheme for quantitative content analysis (QCA) (Rourke &Anderson, 2004) was developed and adapted based on the previous literature (e.g., Fischer et al., 2002; Weinberger & Fischer, 2006) foranalyzing the learners’ epistemic dimension of knowledge construction. In this study, the coding scheme consists of three main epistemicactivities during the knowledge construction process (also see Table 1): (1) Construction of problem space (PS): Learners select, evaluate,and relate case information for understanding a problem; (2) Construction of conceptual space (CS): Learners construct relations betweentheoretical concepts or principles to foster understanding of a theory; (3) Construction of relations between conceptual and problem space(CPS): Learners apply the relevant theoretical concepts to solve a problem or generate a solution for a given task. The three epistemicactivities were chosen because, in this study, we intended to focus on the content of collaborative knowledge construction as indicated byFischer et al. (2002) (i.e., “on content”). In other words, all off-task content as well as operational coordination were not taken intoconsideration of developing the coding scheme.

Table 1The coding scheme of learners’ collaborative knowledge construction.

Category Description Example

Construction of problem space (PS) Learners select, evaluate, and relate caseinformation for understanding a problem

“A is harder to predict because you don’t know when it stopsmoving, and if it moves, which direction it goes in, and what the speed is.”

Construction of conceptual space (CS) Learners construct relations betweentheoretical concepts or principles tofoster understanding of a theory

“The equation just has the initial speed and final speed. Itmeans it is not related with the mass. This is the perfect elastic collision.

Construction of relations betweenconceptual and problem space (CPS)

Learners apply the relevant theoreticalconcepts to solve a problem or generatea solution in a given task

“Do you want to start with both of us having the sameweight to see what the motion will be?”

To assess the inter-rater reliability of the aforementioned analyses, two researchers of this study served as coders engaged in the codingprocesses independently. The inter-rater reliability used percent agreement between the first and the second coder divided by the totalnumber of coding decisions (Lombard, Snyder-Duch, & Bracken, 2002). When the categorization discrepancies occurred, the followingprocess was used to reach final consensus on how to categorize the messages. First, each coder stated their reasons based on the codingscheme. When the two coders felt it was not clear what category a message should go, they would go over the messages again and makemodifications to resolve the ambiguities. In this study, the reliability was 0.92, indicating a highly consistent agreement between the twocoders.

To identify and visualize the learners’ behavioral patterns and provide suggestions for improving the learning system (Hou, 2012), lagsequential analysis (Bakeman & Gottman, 1997) was conducted after coding the dyad learners’ knowledge construction discourses. Asaforementioned, lag sequential analysis aims to detect whether the sequential relationship between each categorized behavior has beenachieved significantly. In other words, lag sequential analysis seeks whether the presence of one categorized behavior (often termed the“given” code) increases the probability that another categorized behavior (often termed the “target” code) will occur (e.g., Bakeman, Quera,& Gnisci, 2009). As this form of analysis requires coding in chronological order, participants’ behaviors were thus coded in the order of theiroccurrence according to the scheme. Then, we calculated the frequency of each behavioral category immediately following anotherbehavioral category. A series of matrix calculations such as frequency transition matrix, conditional probability matrix, and expected-valuematrix were then computed. Next, the computation of adjusted residuals (Z-scores) was conducted to identify sequences with statisticallysignificant differences. In the case of sequential analysis, the Z-test is computed by dividing the difference between the observed and ex-pected values by the standard deviation of this difference. The standard deviation of the difference is a function of the expected sequentialfrequency and the simple probabilities of both behaviors (for exact formula andmore details, see Bakeman & Gottman,1997). Thus, Z-scoresthat are 1.96 or greater indicate that the behavioral continuity has reached statistical significance. The significant sequences allowed us tocapture and visualize the sequential behavioral patterns.

3. Results

3.1. Group comparison of learning achievements regarding elastic collision

In order to understand the effects of AR Physics system on the individual learners’ learning achievements, a t-test was conducted tocompare the post-test cores between the two groups. As shown in Table 2, a significant difference (t (38) ¼ 2.36, p < 0.05) was found in thepost-test scores of the experimental (i.e., AR Physics) and control groups (2D Physics). The Cohen’s d value (d ¼ 0.75) also suggested nearlylarge practical effect of the difference between the two groups. This finding suggests that the learners who learned with the AR Physicssystem showed significant better learning achievements than those who learned with the traditional 2D simulation system regarding thetopic of elastic collision.

Table 2Comparison of the post-test test scores between groups.

Post-test score (Mean, SD) t-Value df Cohen’s d

Experimental group (N ¼ 20) 7.35 (1.42) �2.36 (p < 0.05) 38 0.75Control group (N ¼ 20) 6.05 (2.01)

T.-J. Lin et al. / Computers & Education 68 (2013) 314–321 319

3.2. Sequential patterns in dyad discussions using AR Physics

The above analysis indicated that the participants had significant better learning achievements regarding elastic collision after using theAR Physics system. In order to understand the interaction patterns during their knowledge construction activities, lag sequential analysis(Bakeman & Gottman, 1997) was used to detect such patterns based on the defined categories. It should be noted that the purpose ofconducting lag sequential analysis was to provide more insights about knowledge construction process of AR dyad learners. The resultsshow that six sequences reached statistically significant difference (p < 0.05). To visualize the knowledge construction activity behaviors,these sequences are illustrated in Fig. 3, where each arrow indicates the direction of transfer for each sequence and the numerical value isthe Z-score of each significant sequence. It should be noted that only those Z-scores of significant sequences were shown in Fig. 3.

As shown, the results, first, indicate that the learners sustained the three knowledge construction activities (PS, CS, and CPS) (Z ¼ 2.88,4.65, 3.89, respectively, p < 0.05), suggesting that they were mainly focusing on understanding the given questions (PS), discussingtheoretical concepts related to elastic collision (CS), and managing to generate solutions to the questions (CPS). Furthermore, the two-wayinteractions of PS (construction of problem space) and CPS (construction of relations between conceptual and problem space) (PS4CPS)(Z ¼ 3.3, 3.6, respectively, p < 0.05) suggest that, in the mobile AR collaborative learning context, learners can interpret the given questionsfirst and then generate possible solutions for the questions that might confirm their possible solutions by using the AR Physics system. Inaddition, after they see the results, they can go back to re-think the given problems. For the pattern PS/CS (Z ¼ 3.13, p < 0.05), it is impliedthat, when learners manage to understand the problems, they might seek clarification of theoretical concepts related to elastic collision.

3.13

PS CS

CPS

2.88

4.65

3.3

3.89

3.6

Fig. 3. Sequential patterns of learners’ behaviors during knowledge construction activities by using AR Physics system.

4. Discussion, conclusion, and future work

One of the purposes of the current study was to explore the effectiveness of using a developed mobile AR system for collaborativelearning called “AR Physics” on learners’ knowledge construction of elastic collision. The t-test result of their post-test scores indicates thatthe learners’ knowledge related to elastic collisionwas significantly improved by using the developed AR Physics system. Furthermore, sinceAR group learners performed significantly better in terms of their post-test scores, the AR group learners’ behavioral patterns during theirknowledge construction activities were identified through utilizing lag sequential analysis to gain more insights about knowledge con-struction process of AR dyad learners as well as further modifications to the AR system.

It is shown that there were three main sustained knowledge construction activities, including PS/PS, CS/CS, and CPS/CPS. Accordingto Weinberger and Fischer (2006), the continuous PS activity might be able to foster acquisition of knowledge between group learners inlearning scenarios. As for the continuous CS activity, it allows group learners to actively discuss, rephrase, and summarize the theoreticalconcepts and principles. Also, it can provide group learners with opportunities to clarify the relations between theoretical concepts orprinciples. The continuous CPS activity entails how learners approach a problem, as well as to what extent they are able to apply knowledgein a problem-oriented learning environment. In other words, first, the PS activities might encourage learners to interpret topic-relatedproblems. Second, the CS activities might deepen their topic-related concepts regarding elastic collision. Third, the CPS activities mightstrengthen learners’ understanding of relating theoretical concepts to given problems and transfer this knowledge to future problem cases(e.g., the post-test). Collectively, the three sustained knowledge construction activities might be one of the reasons to explain their gains onthe post-test scores.

Furthermore, the lag sequential analysis also revealed a bi-directional path between the PS and CPS activities (i.e., PS4CPS). This in-dicates that, after the dyad learners interpreted the given task, they would subsequently engage in CPS activities to generate a possiblesolution or hypothesis by linking their knowledge to the task situation (PS/CPS). In addition, for the path of CPS/PS, it might suggest that,because of the advantage of the AR Physics simulation system, learners can instantly see the displayed results on the screen and then startover their PS activity to re-interpret the given problems. Researchers (Slof et al., 2010) have pointed out that, in a simulation externalrepresentation condition, the instant results obtained from the simulation may facilitate learners in determining and negotiating theplausibility of the proposed solutions and come to a final suggestion for further discussion. Therefore, the AR Physics systemmay serve as aconfirmatory tool and enable dyad learners to respond quickly to the displayed results and support their knowledge construction processesbetween the PS and CPS activities, at least to a certain extent.

It is worth noting that, in this study, the learners would sometimes seek further clarification of the theoretical principles or conceptsrelated to elastic collision after interpreting the given problems (i.e., PS/CS). This result might indicate that, while they constructed re-lations between single theoretical concepts or distinguished concepts from each other, they would confront obstacles regarding the topic ofelastic collision and need support from their partners. It might be possible that the learners acquired inaccurate concepts from the otherdyad learner, and thus deepened their misconceptions. For future improvements to the AR Physics system, it is recommended that systemdevelopers and educators add adequate representational guidance to support learners’ cognitive behaviors, and specify the relationshipsbetween the concepts (Ertl, Kopp, & Mandl, 2008; Slof et al., 2010).

T.-J. Lin et al. / Computers & Education 68 (2013) 314–321320

In addition, no significant sequences were found between the CS and CPS activities (e.g., CS/CPS, CS4CPS or CPS/CS), suggesting that,when learners are engaged in the PS/CS sequence, it might hinder their knowledge construction process. In other words, once the learnersare engaged in this sequence, it is unlikely for them to engage in CPS activities according to the shown sequences in Fig. 2. We believe that itwill benefit learners’ knowledge construction processes if more viable sequences are shown. In the current study, dyad learners usuallymanipulated the AR Physics system during the CPS activities to confirm their solutions to the given task. It is argued that the AR Physicssystem could be a supportive learning tool to bridge the gap between CS and CPS activities and strengthen learners’ knowledge constructionsequences between the two activities. In sum, these absent sequences might show the possibilities for other future improvements to thesystem.

Since this study was an initial step toward understanding learners’ knowledge construction processes by using the AR system, futureresearchers are encouraged to adopt different angles or frameworks to unravel a more holistic outlook on learners’ knowledge constructionprocesses in the context of mobile AR collaborative learning. For example, in the framework of Weinberger and Fischer (2006), still otherdimensions such as the argument and social modes dimensions could be used to analyze the dyad learners’ knowledge constructionprocesses. Future studies should recruit more participants to further explore and verify the effects of the AR Physics system. In addition, asrecommended by Cheng and Tsai (2013), mixed methods of investigating learners’ AR learning processes such as interviews or theirattention to AR information such as eye-tracking techniques should be utilized.

Moreover, researchers are encouraged to conduct studies to further understand user experience of the AR system. Future works couldfurther investigate additional issues such as the software usability or learners’ perceived learning effectiveness of the AR system. Theseissues may be of important and of interest for advancing the current status in this line of research. Last but not least, the comparison oflearning behavior and process between the 2D simulation group and the AR system group may be beyond the scope of this study. It ispossible to assume that, by using the AR system, the participants’ learning behaviors may differ from those of using the 2D simulation. Thisspeculation could be verified in the future. Therefore, researchers are encouraged to adopt these suggested approaches to gain in-depthunderstanding of the mobile AR learning process from multiple perspectives in future studies.

References

Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355–385.Azuma, R. T., Billinghurst, M., & Klinker, G. (2011). Special section on mobile augmented reality. Computers & Graphics, 35, vii–viii.Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to lag sequential analysis (2nd ed.). New York: Cambridge University Press.Bakeman, R., Quera, V., & Gnisci, A. (2009). Observer agreement for timed-event sequential data: a comparison of time-based and event-based algorithms. Behavior Research

Methods, 41, 137–147.Behzadan, A. H., Iqbal, A., & Kamat, V. R. (2011). A collaborative augmented reality based modeling environment for construction engineering and management education.

Proceedings of the Winter Simulation Conference, 43, 3573–3581.Bryce, T. G. K., & MacMillan, K. (2009). Momentum and kinetic energy: confusable concepts in secondary school physics. Journal of Research in Science Teaching, 46(7), 739–761.Camp, C., & Clement, J. (1994). Preconceptions in mechanics: Lessons dealing with students’ conceptual difficulties. Dubuque, Iowa: Kendall/Hunt.Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., & Ivkovic, M. (2011). Augmented reality technologies, systems and applications. Multimedia Tools and Ap-

plications, 51(1), 341–377.Chen, C.-M., & Tsai, Y.-N. (2012). Interactive augmented reality system for enhancing library instruction in elementary schools. Computers & Education, 59, 638–652.Cheng, K. H., & Tsai, C. C. (2013). Affordances of augmented reality in science learning: suggestions for future research. Journal of Science Education and Technology. in press,

doi:10.1007/s10956-012-9405-9.Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: a practical guide. Journal of the Learning Sciences, 3, 271–315.Chiu, C.-H., & Hsiao, H.-F. (2010). Group differences in computer supported collaborative learning: evidence from patterns of Taiwanese students’ online communication.

Computers & Education, 54, 427–435.Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.Ertl, B., Kopp, B., & Mandl, H. (2008). Supporting learning using external representations. Computers & Education, 51, 1599–1608.Fischer, F., Bruhn, J., Grasel, C., & Mandl, H. (2002). Fostering collaborative knowledge construction with visualization tools. Learning and Instruction, 12(2), 213–232.Gao, H., Baylor, A. L., & Shen, E. (2005). Designer support for online collaboration and knowledge construction. Educational Technology & Society, 8(1), 69–79.George, E. A., Broadstock, M. J., & Vazquez Abad, J. (2000). Learning energy, momentum, and conservation concepts with computer support in an undergraduate physics

laboratory. In B. Fishman, & S. O’Connor-Divelbiss (Eds.), Fourth international conference of the learning sciences (pp. 2–3). Mahwah, NJ: Erlbum.Graham, T., & Berry, J. (1996). A hierarchical model of the development of student understanding of momentum. International Journal of Science Education, 18(1), 75–89.Hmelo-Silver, C. (2003). Analyzing collaborative knowledge construction: multiple methods for integrated understanding. Computers & Education, 41, 397–420.Hou, H. T. (2012). Exploring the behavioral patterns of learners in an educational massively multiple online role-playing game (MMORPG). Computers & Education, 58, 1225–

1233.Ingec, S. K. (2009). Analysing concept maps as an assessment tool in teaching physics and comparison with the achievement tests. International Journal of Science Education,

31(14), 1897–1915.Jamaludin, A., Chee, Y. S., & Ho, C. M. L. (2009). Fostering argumentative knowledge construction through enactive role play in Second Life. Computers & Education, 53, 317–

329.Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K. (2011). The 2011 Horizon Report. Austin, Texas: The New Media Consortium.Klopfer, E. (2008). Augmented learning: Research and design of mobile educational games. Cambridge, MA: The MIT Press.Klopfer, E., & Squire, K. (2008). Environmental detectivesdthe development of an augmented reality platform for environmental simulations. Educational Technology Research

and Development, 56(2), 203–228.Li, N., Gu, Y. X., Chang, L., & Duh, H. B. L. (2011). Sociality of mobile collaborative AR: Augmenting a dual-problem space for social interaction in collaborative social learning. Paper

presented at the 10th IEEE International Conference on Advanced Learning Technologies (ICALT 2010), Sousse, Tunisia.Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002). Content analysis in mass communication: assessment and reporting of intercoder reliability. Human Communication

Research, 28(4), 587–604.Marshall, J. A., & Young, E. S. (2006). Preservice teachers’ theory development in physical and simulated environment. Journal of Research in Science Teaching, 43, 907–937.Martin, S., Diaz, G., Sancristobal, E., Gil, R., Castro, M., & Peire, J. (2011). New technology trends in education: seven years of forecasts and convergence. Computers & Education,

57, 1893–1906.Meier, A., Spada, H., & Rummel, N. (2007). A rating scheme for assessing the quality of computer-supported collaboration processes. International Journal of Computer-

Supported Collaborative Learning, 2(1), 63–86.Milgram, P., & Kishino, A. F. (1994). Taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems, E77-D(12), 1321–1329.Papagiannakis, G., Singh, G., & Magnenat-Thalmann, N. (2008). A survey of mobile and wireless technologies for augmented reality systems. Computer Animation and Virtual

Worlds, 19(1), 3–22.Peng, H., Su, Y.-J., Chou, C., & Tsai, C.-C. (2009). Ubiquitous knowledge construction: mobile learning re-defined and a conceptual framework. Innovations in Education and

Teaching International, 46(2), 171–183.Rourke, L., & Anderson, T. (2004). Validity in quantitative content analysis. Educational Technology Research and Development, 52(1), 5–18.Rutten, N., van Joolingen, W. R., & van der Veen, J. T. (2012). The learning effects of computer simulations in science education. Computers & Education, 58, 136–153.Singh, C., & Rosegrant, D. (2003). Multiple-choice test of energy and momentum concepts. American Journal of Physics, 71(6), 607–617.

T.-J. Lin et al. / Computers & Education 68 (2013) 314–321 321

Slof, B., Erkens, G., Kirschner, P. A., & Jaspers, J. G. M. (2010). Design and effects of representational scripting on group performance. Educational Technology Research andDevelopment, 58, 589–608.

Squire, K., & Klopfer, E. (2007). Augmented reality simulations on handheld computers. Journal of the Learning Sciences, 16, 371–413.Suthers, D. D., & Hundhausen, C. D. (2003). An experimental study of the effects of representational guidance on collaborative learning processes. Journal of the Learning

Sciences, 12(2), 183–218.Trundle, K. C., & Bell, R. L. (2010). The use of a computer simulation to promote conceptual change: a quasi-experimental study. Computers & Education, 54, 1078–1088.Van Aalst, J. (2009). Distinguishing knowledge-sharing, knowledge-construction, and knowledge-creation discourses. International Journal of Computer-Supported Collabo-

rative Learning, 4, 259–287.Weinberger, A., & Fischer, F. (2006). A framework to analyze argumentative knowledge construction in computer-supported collaborative learning. Computers & Education,

46(1), 71–95.Wu, H.-K., Lee, S. W.-Y., Chang, H.-Y., & Liang, J.-C. (2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49.