Designing navigation support in hypertext systems based

on navigation patterns

SADHANA PUNTAMBEKAR1,* & AGNI STYLIANOU2

1Rm. 697, Educational Sciences, University of Wisconsin, 1025, W. Johnson Street,Madison, WI, 53706-1796, USA; 2Intercollege, Nicosia, Cyprus(*Author for Correspondence, e-mail: puntambekar@education.wisc.edu)

Abstract. In this paper, we present two studies designed to help students navigateeffectively and learn from a hypertext system, CoMPASS. Our first study (N=74)

involved an analysis of students’ navigation patterns to group them into clusters, using ak-means clustering technique. Based on this analysis, navigation patterns were groupedinto four clusters, enabling us to understand the kinds of support that students needed.

This formed the basis of our next study, in which we designed and implementedmetanavigation support to help students navigate and learn science content. Support inthe form of prompts was provided to one group (N=58) while a second group (N=58)

with no support served as the comparison group. Our results suggest that students in thesupport group performed better on a concept-mapping task. Based on the results weprovide suggestions for providing metacognitive support in hypertext systems.

Current design-based and project-based approaches to enhance sci-ence learning (Krajcik et al., 1991; Kolodner, 1997) emphasize theimportance of helping students understand cause and effect relation-ships among scientific phenomena, use of data to support explana-tions and opportunities for sustained inquiry in which studentsinvestigate questions of their own. While the hands-on activities in aninquiry approach can help students experience scientific phenomenaand the relationships therein, electronic texts in the form of hypertextand hypermedia systems (e.g., Shapiro, 2000; Azevedo & Cromley,2004) as well as digital libraries (e.g., Abbas et al., 2002; Hoffmanet al., 2003) are also increasingly being used in scientific inquiry.Digital or hypertext documents are nonlinear and flexible, and enablestudents to follow their own investigation paths.

The flexibility and non-linearity of hypertext systems, attributesthat seem to hold great promise, also present challenges for learnersand designers. On one hand, hypertext systems present material indifferent ways allowing the learner to view the same material from

Instructural Science (2005) 33: 451–481 � Springer 2005

DOI 10.1007/s11251-005-1276-5

multiple perspectives (Spiro et al., 1991). But on the other hand, thisvery attribute is believed to cause disorientation and put a greatercognitive load on learners (Wright, 1982; Marchionini, 1988). Learn-ers in a hypertext system not only have to understand the content,but they also need to understand the structure of the system, knowingwhere a particular unit falls in the big picture and what other units itis related to. As such, several mechanisms have been developed tofacilitate learning from hypertext systems. For example, hierarchies,overviews, outlines and maps (Dee-Lucas, 1996; Shapiro, 1998, 2000),multiple views, focus + context views (Bedersen & Hollan, 1995;Pirolli et al., 2001) and contextual navigation aids such as structuraland temporal context information (Park & Kim, 2000) have beenused in more static systems. Adaptive mechanisms to provide supportand personalization of content have been used in adaptive systems.Adaptive mechanisms have included adaptive presentation of content(De Bra, 1998) based on a user model that stores information aboutuser expertise, goals, interests (Brusilovsky, 1998), or adaptive naviga-tion support in the form of a customized next or continue link, anno-tated links (Brusilovsky et al., 1996, 1998), link hiding and linkdisabling. However, recent studies have suggested that the structuralaids that are available in hypertext systems might not always be intui-tive to learners (Laurillard et al., 2000). For hypertext systems to bevaluable in educational settings, learners need to negotiate what isimportant to them and what they should consider reading next, whichrequires them to regulate their navigation and learning (Hubscher &Puntambekar, 2001, 2002).

In this paper, we present two studies designed to help students navi-gate effectively and learn from a hypertext system, CoMPASS. CoM-PASS (Puntambekar & Stylianou, 2002; Puntambekar et al., 2003) is ahypertext system designed to help middle school students learn science.It presents students with descriptions of text along side a concept mapshowing the relations between science concepts. Our first study usingCoMPASS involved an analysis of students’ navigation patterns togroup them into clusters, using a k-means clustering technique. Basedon this analysis, navigation patterns were grouped into four clusters,enabling us to understand the kinds of support that students needed.This formed the basis of our next study, in which we designed andimplemented metanavigation support to help students navigate andlearn science content. Support in the form of prompts was provided toone group while a second group with no support served as the compari-son group. Our results suggest that students in the support group per-formed better on a concept-mapping task. Based on the results we

452

provide suggestions for providing metacognitive support in hypertextsystems.

Theoretical framework

Metacognition, which refers to the knowledge that we have about ourown cognitive processes, has been proven to be a significant factor fortext understanding. Reading comprehension literature suggests thatthere are individual differences among readers of different abilities inmetacognitive knowledge, experience and strategy use (Flavell, 1979;Paris & Jacobs, 1984; Brown et al., 1986; Gamer, 1987; Pressley &Afflerbach, 1995). Good readers have been found to demonstratemetacognitive differences compared to poor readers in the way theyprocess texts (Brown et al., 1985). Skilled readers often engage indeliberate activities that require planful thinking, flexible strategies,and periodic self-monitoring (Paris & Jacobs, 1984; Garner, 1987).They typically look over the text before they read, noting such thingsas the structure of the text and text sections that might be most rele-vant to their reading goals. Skilled readers also adjust their readingrates based on their goals, evaluate their own understanding as theypause, paraphrase, answer questions, or summarize information intext and monitor progress revising or modifying plans and strategiesdepending on how well they are working (Pressley & Afflerbach,1995). On the contrary, unskilled readers often seem oblivious tothese strategies and the need to use them. They are quite limited intheir metacognitive knowledge about reading and tend to focus onreading as a decoding process than as a meaning-construction process(Garner, 1987). Promoting self-awareness, monitoring and regulationof text comprehension seem to be critically important aspects ofskilled reading (Mokhtari & Reichard, 2002). Training students inmonitoring and regulating has been viewed as an important aspect ofpromoting comprehension of traditional text (Paris & Jacobs, 1984;Brown & Palincsar, 1985; Brown et al., 1986). Being aware of themetacognitive strategies that learners employ while reading can influ-ence how well they plan and monitor their understanding from text(Jacobs & Paris, 1987).

Recent studies on self regulated learning and learning fromhypertext have also indicated that learners in a hypertext environmentshould be able to regulate their own learning by planning,monitoring, controlling, and reflecting on their own learning (Hill &Hannafin, 1997; Azevedo et al., 2001). Learning from an electronic

453

text (hypertext) is different from reading a printed book. A uniqueattribute of hypertext systems is the nonlinearity of information units.Instead of looking at one unique, predetermined sequence of text,pictures or graphics, hypertext provides readers the ability to followmultiple reading paths (Landow, 1992; Jacobson et al., 1996). Readersare required to actively make decisions about which text segments toselect while learning from a hypertext system. Learners need to makefrequent and important decisions about the selection of relevant linksthat will enable them to pursue their learning goals. They need toengage in cognitive monitoring, to slow down and take a moment toconsider various paths and question why they are considering somepaths over others (Rouet, 1992; Charney, 1994). Learning fromhypertext requires planning what to read next, and closely monitoringongoing learning, both in terms of link selection and comprehension.Students therefore require metacognitive skills to select links andlearn effectively.

Some researchers have suggested that it is likely that text naviga-tion may walk hand in hand with text comprehension. Dillon andVaughan (1997) advocated that moving through the informationspace while interacting with hypertext ‘‘is frequently the same purposeas the journey, to reach an end point of comprehension – and in thiscase the journey is the destination’’ (p. 100). According to Shapiroand Niederhauser (2004), the importance of navigation is one of theimportant features that differentiates hypertext learning from thelearning of traditional text. Navigation can affect what a studentlearns from a hypertext system. In our first study therefore, our mainaim was to analyze students’ navigation patterns in order to under-stand the kinds of paths that students take while solving a complexproblem.

Study 1: Analyzing navigation patterns

Study 1 was conducted to understand how students navigate throughCoMPASS and what effect the navigation patterns have on learning.We were interested in questions such as

• How deep into the topic did students explore?• Did they visit related concepts? Did they visit the same concept

in other topics?• Was their navigation related to their understanding of the repre-

sentational structure in CoMPASS?• How did navigation affect learning?

454

In this section, we will first discuss the design of the CoMPASSsystem before we discuss details of study 1.

Design of CoMPASS

Learning science involves understanding the rich set of relationshipsamong important concepts (Ruiz-Primo & Shavelson, 1995). Accord-ing to Glynn et al. (1991), ‘‘without the construction of relations,students have no foundation and framework on which to build mean-ingful conceptual networks’’ (p. 6). Spiro et al. (1991) emphasizedthat the key feature of ill-structured domains is that they embodyknowledge that will have to be used in many different ways. Spiro etal., argue for multiple representations and multiple passes through thesame material. To enable students to understand the rich and multipleinterrelationships between science principles, concepts, and phenom-ena, we have designed a software environment, CoMPASS, whichuses external representations that mirror the structure of the domainto aid deep learning (Puntambekar & Stylianou, 2002; Puntambekaret al., 2003).

CoMPASS is a hypertext system that uses two representations –concept maps and text to facilitate multiple passes, navigation, andlearning (see Figures 1 and 2). Each screen in CoMPASS represents aconcept such as work or force, providing both a concept map (left halfof the screen) and a textual description (right half of the screen). Themaps are dynamically constructed and displayed using the fisheye tech-nique (Bedersen & Hollan, 1995; Fumas & Bedersen, 1995) – that is,the focus concept that the student has chosen is at the center of themap, with the most closely related concepts displayed in the first ringand the less closely related concepts displayed at the outer ring. Rela-tionship strength, determined in consultation with physics experts, hasbeen used to determine the spatial proximity of the concepts. Thus, thestronger the relationship between two concepts, the closer they arespatially in the concept map. The two representations, maps and text,are interlinked, such that the text contains a description of the focalconcept and the same links are used in the text and the maps. Studentscan therefore navigate through the text or the maps.

An important problem in learning from hypertext is that learnershave to place the particular text segment on the screen that they arereading relative to the other text segments. Writers of traditionaltexts provide coherence at the local level (between paragraphs) and atthe global level (between sections) (Foltz, 1996). However, inhypertext systems, it is difficult for writers to maintain macro- or

455

global-coherence because a reader can essentially jump to any section.In CoMPASS, we have strived to maintain coherence by presentingstudents with a visual representation in the form of a map. The mapsin CoMPASS show the relationships among concepts providingstudents with a visual representation of how the different nodes areconnected to each other. Students can also change views – for exam-ple, a student who is learning about work in the topic ‘pulley’ mightbe interested learning about in the phenomenon, work, in anothercontext, such as an inclined plane. The student can change views atany time (see top right of screen in Figures 1 and 2) to study thesame phenomenon (e.g., work) in other topics (contexts), allowing fora richer understanding of the content.

The CoMPASS software environment is used in conjunction withthe instructional modules that we have developed for sixth and eighthgrades, based on the pedagogical principles of Learning by DesignTM

(Kolodner et al., 2003).

Participants and design

Study 1 was conducted in a rural middle school in Connecticut with asample of 74 students in four classes taught by a single teacher. The

Figure 1. Textual and visual representation of information with ‘work’ as focus.

456

students in all four classes were from different ethnic backgrounds,academic abilities, and socioeconomic levels. Of the sample, 46%were female and 54% were male. Within each class students used lap-tops during their science classes to access CoMPASS. The data forthis study was collected during a two-week module on pulleys – thepulley challenge. In the pulley challenge, students were required towork with simple or compound pulleys so that a bottle of water thatweighed 600 grams could be lifted with the least effort. They workedwith a range of pulleys that were made available by the teachers.Students were required to put the best system together using fixed,movable pulleys or both. The task was fairly complex and open-endedand students used CoMPASS on two consecutive days to help gatherinformation to solve the problem. Students used CoMPASS for30 minutes on each of the two days to solve the problem, leading to atotal of 60 minutes on CoMPASS.

Data sources

We collected data to examine students’ navigation as well as learning.To understand how students navigated through the CoMPASSsystem, we used the log files that were collected for each of the two

Figure 2. Fisheye with work as the focus.

457

sessions that students used COMPASS. The log files kept a record ofthe topic that students chose, concept visited, the start-time and endtime for each concept, and the source of navigation, i.e., maps ortext.

In addition, a pre- and post-test and a concept mapping test (post-test only) was administered to measure students’ knowledge of thephysics that they were learning. The pre- and post-test was adminis-tered online and included fifteen multiple-choice items and threeopen-ended questions (see Appendix A for example items). The con-cept mapping test was administered at the end of the pulley challenge,in which students drew a concept map showing their understanding ofthe physics concepts and principles and the connections among them.The decision to use a post-test only concept mapping test was madebecause before the start of the unit, students did not have enoughknowledge about the domain to draw a map with the key concepts asnodes, and connect the concepts with meaningful descriptors.

Data analysis

Analysis of log filesThe log files were analyzed to understand the paths that studentstook as they read through CoMPASS. In addition, we also comparedthe time that students spent on concepts that were related to theirgoals, and on topic descriptions.

Pathfinder analysis: To analyze students’ navigation paths, thePathfinder and k-means clustering algorithms were used. Pathfinder(Schvaneveldt, 1990), a graph theoretic technique that can be used toanalyze log file data, helps create network representations consistingof nodes and links. A pathfinder network is created by converting logfile data into a ‘proximity matrix’, which shows all the ‘nodes’ that astudent visited and all of the transitions from that node. The Path-finder algorithm then computes a graphical representation where onlythose concepts are connected that are most traversed by students.Figure 3 shows the stages in using the pathfinder technique to exam-ine the navigational patterns. The figure shows a fragment of a simplelog file consisting of five concepts. This is then converted into a two-dimensional matrix showing all the transitions that students made.The matrix shows that there were seven transitions between drag andacceleration, three between drag and mass and so on. The transitionmatrix is then converted into frequencies. Higher frequencies indicatethat the path was frequently chosen by students. Finally, the Path-finder algorithm is applied and visualized as a graphic as shown in

458

Figure 3. Stages in using the Pathfinder technique.

459

the figure. The graphic shows the number of times students ‘went to’a particular concept, and the number of times they ‘went from’ it. So,drag 10:2 means that the students visited drag ten times and traversedto another concept from drag two times. This technique provided aninitial understanding of the navigation patterns of students in the fourclasses.

The clustering algorithm is then applied to the transition matrices sothat students with similar patterns can be grouped together. Clusteringof similar students is accomplished using a k-means clustering algo-rithm that assigns each student to exactly one of k groups (Kanungoet al., 2000). When students use a resource in an open-ended inquirydriven environment, they take unique investigation paths. Clusteringallows to look for similarities and differences in the paths, and to studywhether students may have missed critical information. The k-meansclustering algorithm was used to group students into clusters based ontheir navigation patterns. This mechanism uses distances to group navi-gation patterns represented by pathfinder networks into clusters, basedon their proximity to the ‘centroid’ of the cluster. Subsequently, eachpoint, which is represented by a pathfinder network, is repeatedlychosen and assigned to the ‘closest’ cluster, which is measured by thedistance between that point and the centroid of a cluster.

Time spent on related topics and goal-related concepts: We furtheranalyzed the log files to get two measures: proportion time spent ongoal-related concepts and proportion time spent on related topics.Goal-related concepts were determined in consultation with the teach-ers and experts, so that they were relevant to the task. While explora-tion is key to hypertext learning, we were interested in finding out theproportion tune that students spent on concepts that related to thepulley challenge. Higher proportion of time spent on non-goal relatedconcepts would indicate that students were floundering. The secondmeasure, proportion time on related topic also was an indication ofwhether or not students navigated to closely related topics, and alsoindicated the time that they spent on topic descriptions as opposed toconcept descriptions.

Pre- and post-test measuresA pre- and post-test, consisting of 15 multiple choice and three open-ended questions, to measure students’ knowledge of the physics ofsimple machines was administered before and after the unit. Eachcorrect response in the multiple choice test was given a score of 1.Student’s responses to the open-ended questions were scored based on

460

a threepoint scale: 0 point for an incorrect response, 1 point for apartially correct response and 2 points for a correct response. Theopen-ended questions were scored by two raters who were trained touse the scoring scheme. The inter-rater reliability was 0.95. A totalscore was calculated for each student, by adding the scores on themultiple choice and open-ended items. Another post-test, the CoM-PASS questionnaire was administered at the end of the unit. This testconsisted of 22 Likert type items that asked students about how theyused the maps in CoMPASS and how they navigated during the unit.Example items are in Appendix B.

Concept map scoresResearch has shown that that concept mapping is a powerful andpsychometrically sound method for assessing conceptual change (RuizPrimo & Shavelson, 1995). Creating concept maps engages students ina thoughtful way and encourages them to reflect on relationshipsamong concepts and complexity of ideas (Novak & Gowin, 1984).According to Jonassen and Wang (1993) students show some of theirbest thinking when they try to represent something graphically, andthinking is a necessary condition for learning.

Students’ concept maps were analyzed using a rubric that wasdeveloped in a study conducted by Puntambekar et al. (2003). Twoaspects of the maps were examined: the explanation provided for theconcepts and the explanation provided for the connections among theconcepts. Students’ responses were scored on a scale of 0–3 based onthe depth of science understanding that they demonstrated. A score of0 indicated an incorrect explanation, while a score of 3 indicated acomplete and clear explanation for the concept or the connection.Two raters scored each map. Inter-rater reliability was found to be0.96. A concept ratio was calculated for each student by dividing thescore that was given for the explanation of the concepts by the num-ber of concepts included in the concept map. This ratio was a mea-sure of student’s understanding of science principles. A connectionratio was calculated by dividing the score that was given for the expla-nation of the connections with the number of connections in the map.This ratio was a measure of the depth of understanding of therelationships among science principles.

Results

In this section, we will first discuss the analysis of students’ naviga-tion paths and then the relationship of navigation to learning.

p p y

461

Analysis of navigation patternsThe analysis revealed four different clusters (Figures 4 and 5) basedon the closeness of navigation patterns. The patterns were different interms of the richness of navigation within a topic and across topics.

Cluster 1 (N=32) shows that students visited all of the relevantconcepts within a topic. The topic overview (pulley) was the hub fromwhere most of the transitions started. Although the topic overview wasthe center, there were many transitions between related, concepts; forexample, there were transitions between power and force, work andpower, ma (mechanical advantage) and distance. This indicates thatstudents used the maps for navigation and used the related conceptsshown in the maps as a way to guide their navigation. Not only werethese concepts related to one another, but they were also goal relevantand would help students in solving the problem. This cluster illustratesmultiple ‘circular paths’ that students followed, for example, pul-ley fi ma fi distance fi pulley, pulley fi empower fi force fipulley. It also shows several double-sided arrows indicating that stu-dents were not linear in their navigation. However, students in thiscluster stayed predominantly in one topic (pulleys) and failed to navi-gate to the related topic of levers, which was a closely related topicand also failed to visit concepts in alternative views, i.e. ma in levers ascompared to ma in pulleys. The number of students in this cluster wasthe maximum indicating that a large proportion of the sample read theconcepts within a single topic. This cluster can be described as consist-ing of rich navigation within a single because of the visits to related

Figure 4. Navigation patterns of cluster 1 (left) and 2 (right). Each node has thename of the concept and the topic, thus work_p means students visited the conceptwork in the topic pulleys.

462

concepts within a particular topic. Students in this cluster visited mostof concepts in the topic of pulleys that were related to each other aswell as related to their goal, designing the best pulley system.

Cluster 2 (N=14) also had the topic overview as the center, butthe navigation pattern was very different. There was a single circularpath was the most visited, showing that students visited three con-cepts – ma, power and force – more often than other concepts. Thenetwork also shows that students went into more details for the con-cept ‘energy’ by visiting the forms of energy, while students in cluster1 stayed mostly at the first level of detail and did not go deeper intothe links that were indicative of the various manifestations of the phe-nomenon (the types of energy). Another interesting feature of cluster2 is that students also navigated to a related topic, i.e. levers. Stu-dents in this cluster seemed to have visited fewer concepts within thetopic, but did visit other related topics, such as levers. As such, thiscluster can be characterized as having breadth across topics. How-ever, visiting fewer topics meant that students did not explore all ofthe related concepts within the topic; neither did they visit many con-cepts related to their goal. Students in this cluster stayed mostly atthe level of overall topic description.

Clusters 3 and 4 (Figure 5) show navigation patterns that are com-pletely different from clusters 1 and 2. These clusters do not have aclear center; students visited numerous concepts within the topic andin other topics. Cluster 3 (N=8) shows that students visited numerousconcepts, in many of the topics, making an extensive use of the alter-native views (e.g., force_WaA, force_p, force_ip, as indicated inFigure 5) presented in CoMPASS, making this pattern rich at the‘views’ level. However, this cluster is pretty sparse at the both the

Figure 5. Navigation patterns of clusters 3 (left) and 4 (right).

463

within topic and the across topic levels because students did not havemany transitions between the related concepts. This cluster may bedescribed as exploratory. Students visited a single concept in differenttopics, so they seemed to have read about a specific phenomenonfrom different perspectives, which is important in science. However,they did not visit related concepts or concepts that would help themin their design challenge. Cluster 3 had the least number of studentsout of the four clusters. Cluster 4 (N=17) shows a more or lessrandom pattern in which students visited topics as well as conceptsthat were not related to their goal. Students in this cluster visitedseveral topics that were unrelated, such as rotational motion. Clearly,these students needed more support in order for them to navigate andlearn better.

Proportion time on goal related concepts and related topicsAs indicated before, we calculated the proportion time spent on goalrelated concepts and the transitions between related concepts. Wetook all the entries in the log files and computed the total time thatstudents spent on CoMPASS in each session. We then calculated theproportion time that students spent on the concepts that were relatedto their goal as a proportion of the total time spent on CoMPASS. Inthe same way, we calculated the proportion time that students spenton the overall topic description, i.e., pulleys, levers, etc.

Table 1 summarizes the data. We found that on the whole, stu-dents in cluster 1 spent a higher proportion of time on concepts thatwere related to their goal. A one-way analysis of variance (ANOVA)was conducted to evaluate the difference in the proportion time thatthe students in the four clusters spent on concepts that were related totheir goal. The ANOVA was significant, F(3, 67)=2.86, p=0.044.

Table 1. Navigation data for each cluster

Cluster N Proportion time on

related concepts

Proportion time on

related topics

Mean SD Mean SD

1 32 0.61 0.15 0.16 0.04

2 14 0.43 0.13 0.11 0.15

3 8 0.19* 0.17 0.11 0.03

4 17 0.17* 0.27 0.09 0.02

*Significant at the 0.01 level.

464

Follow-up tests were conducted to evaluate the pair-wise differencesbetween the four clusters. Using the Dunnett’s C test, cluster 1differed significantly from the other clusters 3 and 4. However, thedifference between clusters 1 and 2 was not significant. Similarly, aone-way ANOVA was conducted to analyze the differences betweenthe clusters on the proportion time spent on topic descriptions, ratherthan the concepts. This difference was not significant F(3, 67)=0.920;p=0.43, indicating that the students did not significantly differ in theproportion time they spent on reading the topic descriptions whichwas a ‘surface’ level description of the topic (Table 2).

Relationship between navigation and learning

In order to understand whether the differences in navigation also leadto differences in learning gains, we analyzed the pre- and post-testscores as well as scores on the concept mapping test. A multivariateanalysis of covariance was conducted with the three dependent vari-ables: post-test scores, concept ratio and connection ratio with thepre-test scores as the covariate. The analysis showed that there was asignificant difference between the clusters for the connection ratio F(3,63)=6.42; p=0.001. The effect of membership of a cluster, assessedby a partial g2 accounted for a 26% variance in the dependent vari-able (connection ratio). Follow-up t-tests showed that cluster 1 per-formed significantly better than students in cluster 2 t(43)=3.01;p<0.001, cluster 3 t(33)=1.82; p<0.001, and cluster 4 t(46)=2.67;p<0.001, i.e. they had higher connection ratios. This indicated thatstudents in cluster 1 had a better understanding of the connectionsbetween the concepts in the topic. However, the difference betweenstudents in the four clusters for the post-test scores F(3,63)=0.54;p=0.65 and the concept ratio F(3,63)=1.88; p=0.14 not significant.

Table 2. Pre- and post-test and concept mapping scores

Cluster N Pre-test Post-test Concept ratio Connection

ratio

Mean SD Mean SD Mean SD Mean SD

1 32 9.55 2.98 13.55 3.53 2.09 0.45 1.51 0.53

2 14 9.38 2.75 13.54 3.71 1.73 0.67 1.06 0.13

3 8 11.25 0.96 14.75 2.06 1.57 0.89 1.01 0.01

4 17 9.73 2.68 13.93 3.34 2.25 0.74 1.12 0.35

465

To further investigate what might have caused the differences inthe navigation, we examined students’ scores on the CoMPASS ques-tionnaire. A correlational analysis revealed that there was a significantpositive correlation between scores on the questionnaire and their per-formance on post-test scores (Pearson’s two-tailed=0.791; p=0.01)and students’ concept ratios (Pearson’s r, two-tailed=0.623; p=0.02)connection ratios (Pearson’s r, two-tailed=0.588; p=0.03).

Discussion: supporting navigation and learning

Why are navigation patterns useful? Are they merely indicative of dif-ferent strategies that students might use, or students’ experience inusing information resources? Researchers have effectively analyzed logfile data to identify patterns in problem solving (Barab et al., 1996),identify differences in students’ navigational styles by clustering theminto groups (Lawless & Kulikowich, 1996), and examine meaningfulpaths chosen by students (Rowe et al., 1996).

Cognitive flexibility theory suggests that learning in a hypertextenvironment involves the cognitive reconstruction of the domain spacethrough repeated traversals across that space (Jacobson & Spiro,1995). Therefore, the paths that users choose have a powerful influenceon learning outcomes in hypertext. A comprehensive analysis of navi-gational patterns can provide useful insights into hypertext processingand can be used to better explain the effects of self-regulated learningin hypertext learning environments (Niegemann, 2001). An importantaspect of the path record is that it creates a unique opportunity tonon-intrusively capture the dynamic processes of hypertext navigationas they unfold and gain insights into the learning process (Barab et al.,1996; Rowe et al., 1996). In study 1, we therefore investigatednavigation behavior not only to gain some insights into the paths thatstudents took as they read through the system, but also to understandwhat kind of support they needed in order to become mindful readers.

Our results suggest that clusters 1, 2 and 3 each had a very differ-ent approach to navigate through the system, while cluster 4 had amainly random navigational pattern. The navigation patternsdiscussed in the previous section suggest differences in the use of thefeatures of the system. Students in cluster 1 explored the topic of theirinvestigation in detail visiting most of the related concepts in thattopic, which were also related to their goals. Although students in thiscluster visited many concepts in the topic, they were still not using allof the features of the system. For example, they completely ignoredthe topic of levers, which was available and also related to the one

466

that they were learning about. One of the goals of the simplemachines unit was to help students understand the how the variousmachines were similar (or different) and to understand the commonscientific principles across machines. Cognitive flexibility theory (Spiroet al., 1991) recommends that for richer knowledge acquisition, it isimportant that students should visit a particular phenomenon frommultiple perspectives. In that sense, the students in cluster 1 neededsupport so that they could visit related topics.

Out of the other three clusters, cluster 2 and 3 also had strengthsin that students in these two groups seemed to have made use of theavailability of multiple perspectives. However, this was done at theexpense of exploration within the topic and students in these twoclusters did not visit the related concepts within the topic and also didnot visit concepts that were related to their goal. They thereforeneeded support to reflect on their goal and visit goal-related concepts.

A key aspect of metacognition is goal setting and planning inaccordance with the goals (e.g., Paris & Myers, 1981). This isespecially crucial in hypertext systems, as students have to make adecision before selecting each text segment that they read. In a studyby Shapiro (2000) to examine the influence of the compatibility orinconsistency of learners’ goals and the structural overviews, it wasfound that overviews often overshadow the effect of learning goals,especially in novice learners. This suggests that designers not onlyhave to provide structural aids to support navigation, but also ascer-tain that students pay attention to their learning goals when usingnavigational aids. In our study, the concept maps reflected the con-ceptual structure of the domain. We found that students in cluster 1visited a significantly higher proportion of concepts related to theirgoals. We also found that the proportion of transitions to relatedconcepts was significantly higher than the other clusters. The mapstherefore enabled students to visit related concepts but this was nottrue for students in all the clusters. Thus students needed to under-stand the affordances of the navigational aids in CoMPASS as well asto use them to achieve their learning goals.

As mentioned earlier, researchers have suggested that navigationis closely linked to comprehension in learning from hypertext sys-tems (Shapiro, 1998; Shapiro & Niederhauser, 2004). Our resultssupport this premise in that we found that students in cluster 1performed significantly better on the concept mapping test in termsof providing better concept explanations. They had higher connec-tion ratios because they included detailed explanations for theconnections between concepts that they mentioned in their concept

467

maps, indicating that they had a richer understanding of how theconcepts were related to each other. However, when it came toproviding explanations for the concepts themselves, their scoreswere higher but not significantly different from the other clusters.This might have been because they did not spend enough time onCoMPASS to get a detailed understanding of individual concepts,because they visited many concepts in the two sessions.

Our results also suggest that there was a significant correlationbetween students’ understanding of the ways in which informationwas organized in CoMPASS and the affordances of the navigationalmechanisms and their performance on the pre- and post-test and theconcept mapping test. This indicated that students who understoodthe representational structure of CoMPASS were likely to navigatemeaningfully and thereby learn better.

In a hypertext system, the reader constantly makes decisions aboutwhere to go next, about what nodes might be relevant to their currentgoals. It is therefore important for readers to be aware of andregulate both their comprehension and navigation strategies. The fourclusters in this study can be characterized based on the twodimensions how rich and focused the navigation path is, and whetherstudents visited concepts within or across topics – navigation within atopic, navigation across topics and navigation that is goal related orrandom. Based on this study, we have recognized the following typesof support that students in a hypertext environment might need:

• Our results suggest that students need support to reflect on theirgoals to make decisions about which node they should selectnext.

• Based on the finding that although students visited manyconcepts in a particular topic, they were not able provide expla-nations for single concepts, we believe that when students do visitconcepts that are related to their goals, they still need support tobe able to integrate the information found in fragments in ahypertext system, and need to monitor their understanding ofeach concept before proceeding to the next node.

• Students need to be supported to understand the representationof the information in the system, i.e., the structure of the system,especially the functions of the navigational aids provided.

• Depending on the goal that students are pursuing at a given time,they need support to visit a concept in various contexts (topics)to get a richer understanding of the domain. While this form ofsupport may not always be relevant, especially when the task is

468

not very complex, it is nevertheless an important aspect to con-sider, both in terms of design, so that students have the flexibilityto visit various topics, as well as in terms of providing supportfor them to do so.

Our results suggest that students need to be encouraged to ask thefollowing questions while reading: What are my goals? How do I getinformation from multiple texts? How do I decide where to go? Theytherefore need metacognitive strategies for navigation, i.e., metanavi-gation (Stylianou, 2003, Stylianou & Puntambekar, 2004). We definemetanavigation support as the support designed to enable students toreflect upon and monitor their link selection while navigating througha hypertext system. Metanavigation cues may be designed to enablereaders to think about the processes they employ while navigatingthrough hypertext systems and help them monitor and regulate theseprocesses in order to accomplish their learning goals. In study 2, wedesigned and implemented support for navigation in the form ofmetanavigation support.

Study 2: Implementing metanavigation support

In a follow-up study that was conducted with sixth-grade students weprovided metanavigation support to help students to monitor andregulate their navigation behavior in order to accomplish their learn-ing goal. We hypothesized that supporting middle school students touse a combination of reading comprehension and metacognitive strat-egies (i.e., monitoring and self regulation) as well as make thoughtfulnavigational decisions while using nonlinear science texts would leadto a richer text understanding.

In study 2, we provided students with metanavigation supportbased on the information collected in their log files. Study 2 wasconducted in the same school as study 1. However, the participatingstudents were not the same as in study 1, because study 2 wasconducted a year after study 1. Just as in the previous study, datawas collected during the two-day pulley challenge.

Participants and design

The participants were 121 sixth graders in four science classes taught bytwo different teachers. The students were from different ethnicbackgrounds and academic abilities. Approximately 47% (N=57) ofthe participants were girls and 53% (N=64) were boys. Each class was

469

randomly assigned to one of the two conditions (metanavigation sup-port, no support). Students collaborated in groups of three or fourwhile participating in this study. Both the conditions had 58 studentseach.

The study involved three sessions of 45 minutes during regular sci-ence classes. A pre-test of students’ prior knowledge of the physics ofsimple machines was administered before the study. This test, whichwas also used in study 1, was administered online and included 15multiple-choice items and three open-ended questions. The task was adesign challenge that required students to build a pulley device thatwould lift a bottle of water that weighed 600 grams off a table usingthe minimum amount of effort, same as in study 1. Students usedCoMPASS to read the information that was available in the simplemachines unit in CoMPASS for 25 minutes.

Procedure

The first session started with the presentation of the pulley challenge.Students then spent time on CoMPASS finding information to com-plete their challenge. In this session, their log files were collected andcustomized support was designed. The support was provided in ses-sion 2. In the third session, a post-concept mapping test was given. Inthe next few paragraphs, we have discussed details of the procedure.

At the end of the first session, log file information was collected toenable us to make decisions on the kind and amount of support thatstudents would receive in the next session. Log file information thatcaptured students’ navigation path enabled us to assess their naviga-tion behavior and decide what metanavigation prompts would be gi-ven to each group. Computer log files recorded information such asthe science concepts students visited, and the time they spent on eachconcept. Three main indices from students’ navigation paths informedour decision of what type of metanavigation support students needed:

• whether students selected concepts related to their goals (asopposed to random);

• whether students visited related concepts, i.e., they made transi-tions between related nodes; and

• whether students visited related topics.

Table 3 shows the three conditions and Table 4 provides examples foreach condition. For example, consider a navigation that patternshowed that students chose to read science topics that werenot important for solving the pulley challenge and did not read about

470

goal-related science principles such as mechanical advantage, distanceand force. The metanavigation prompts provided encouraged studentsto think about their goal and use the concept maps to make thoughtfuldecisions of what paths to follow. All the navigation patterns were ana-lyzed after the first session and customized support was designed basedon each pattern. In the second session, the support, which was tailoredto help students based on their navigation in session 1, was provided inwritten format to each student, prior to the start of the session.

In the second session students were asked to continue searching forinformation about pulleys in CoMPASS and finalize their pulleydesigns. The students in the metanavigation support conditionreceived metanavigation prompts in a written format (see an examplein Appendix C) at the beginning of the second session to guide theirexploration in CoMPASS. The third and last session included anassessment of students’ individual science knowledge through aconcept map test (Table 5).

Data sources

As indicated before, a pre-test to measure students’ prior knowledgewas administered before the unit. A concept mapping post-test wasused to measure students’ understanding of science at the end of thepulley challenge. The maps were scored in the same way as in study1. Inter-rater reliability was found to be 0.94.

Results

An analysis of covariance (ANCOVA) was used to investigatewhether the students who received metanavigation support scored

Table 3. Conditions for providing support

Log file info

(actual navigation)

Type Description

Concepts visited Random vs.

goal related

Do students visit concepts that

are relevant to their learning goal?

Transitions No coherence

vs. coherence

Do students make coherent

transitions while reading?

Dimensionality Single topic vs.

more topics

Is the navigation unidimensional

(within a topic) or multidimensional

(visit the same concept in other)?

471

significantly higher in the concept map test than students who did notget any support. Pre-test scores were used as a covariate to adjust fordifferences in prior knowledge. In analyzing students’ concept mapstwo aspects of the maps were examined: the explanation provided forthe concepts (concept ratio) and the explanation provided for the

Table 4. Rules for providing metanavigation support

Metanavigation support

rules

Metanavigation prompts

Navigation

choices

If random choice of

concepts � encourage

goal-related navigation

Think about your goal! What science

concepts will help you solve the

challenge?

If goal-related

navigation � encourage

integration of knowledge

Remember that solving the challenge is

like doing a puzzle. You need to put all

the pieces of information together and

understand how they are related. The

maps can help you do that!

Transitions If transitions are not

coherent � encourage

regulation of navigation

behavior to make

coherent transitions

between text units

while reading

USE THE MAPS! Have you ever used

the maps to help you decide where to

go? The maps in COMPASS can help

you in different ways: The maps help

you see what other science concepts are

closely related to what you are reading.

The maps show relationships among

science concepts and help you make

connections

If transitions are

coherent � encourage

integration of knowledge

Remember that solving the challenge is

like doing a puzzle, You need to put all

the pieces of information together and

understand how they are related. The

maps can help you do that!

Dimensionality

of navigation

If unidimensional

navigation � encourage

multidimensional

navigation

Think about what other topic

descriptions are related to the topic that

you are reading

If multidimensional

navigation � encourage

integration of knowledge

Think about how the topic that you

reading is similar to the one that you

read before, e.g., how are pulleys

similar to levers?

472

connections (connection ratio). The means and standard deviationsfor the two concept map indices are shown in Table 5.

The results of the ANCOVA analyses indicated that there was asignificant main effect for the treatment condition on both the con-cept ratio F(1,118)=8.77; p=0.004 as well as the connection ratio,F(1,118)=9.14; p=0.003. As a measure of the effect size we used thepartial g2 which revealed that 6.9% of variance in the concept ratiocan be explained by the provision of metanavigation support afteradjusting for pre-test science scores and 7.2% of variance in the con-nection ratio can be explained by the provision of metanavigationsupport after adjusting for prior science knowledge differences.

Overall, the ANCOVA results suggest that students’ scores in theconcept map test can be explained by the condition that they were as-signed. Students who received metanavigation provided better expla-nations of the concepts they included in their maps and richerexplanations of the connections they made among them.

Discussion

In this paper we discussed two studies in which students’ navigationpatterns were used to design support for navigation in a hypertextsystem. Providing structural cues to support navigation in hypertextsystems has been used by several researchers (e.g., Dee-Lucas, 1996;Shapiro, 1998, 2000; Shapiro & Niederhauser, 2004). Our approachwas different from earlier approaches in two ways. First, the CoM-PASS system itself has been designed to make the relations betweentext segments visible to students and using concept maps to providecoherence. Second, we designed metanavigation prompts based onstudents’ navigation and tailored the support so that students onlyreceived prompts that were relevant to the ways in which they werenavigating. We found in the first study that although some of the

Table 5. Means and standard deviations for concept map ratios

Variables Metanavigation

support

No support

N Mean SD N Mean SD

Concept ratio in concept mapa 58 1.51 0.64 63 1.14 0.58

Connection ratio in concept mapa 58 0.90 0.37 63 0.70 0.29

aThe maximum score was 3.

473

students could use the maps to navigate to related concepts (cluster1), most students needed support to select concepts relevant to theirgoals. The results from our second study suggest that metanavigationprompts enabled students to make coherent transitions among textunits and gain a better understanding of domain knowledge. Al-though our results show a positive trend in the scores on a conceptmapping test for students who received metanavigation prompts, sev-eral issues need to be taken into consideration if a designer decidesprovide support for navigation and learning: When is the right timeto provide support to encourage monitoring of navigation behavior?How much exploration should be allowed before metanavigationprompts are provided to the readers? Although there are clear defini-tions and measures of metacognitive monitoring in traditional texts(e.g. backtracking, rereading), there are not clear measures of controlwhen it comes to hypertext. For example, hypertext environmentsinherently support exploration and it is important to decide when astudent is merely exploring the domain before he or she starts to visitrelevant nodes.

Another important question is: Do all learners need support?There is empirical evidence that students found the metacognitivesupport that was provided in a hypertext environment distracting ra-ther than helpful (Lee et al., 1997). What about interaction effects onlearning outcomes? A study conducted by Lee et al. (1997) showedthat a combination of concept maps and metacognitive views was notsuperior to other treatments in a hypermedia-based genetics program.When compared to the concept maps or metacognitive cues treat-ments in isolation, there was an apparent ‘detrimental interactioneffect’. The authors assumed that interference effects such as cognitiveoverload (due to the excessive amount of information on the screen)or cognitive dissonance might have contributed to the findings of thestudy.

Designers might also consider readers’ prior domain knowledgeand their reading comprehension along with their navigation behaviorto provide adaptive support, i.e. support that is tailored based on thenavigation profiles of individual students. In order to build adaptivesupport within a hypertext system, we might need to think whetherthere is a need to apply some hierarchical priority to the rules thatinform the decisions of what type of metanavigation support shouldbe given to readers. For example, one might argue that it might bemore important to check whether readers have understood the infor-mation that they read (reading comprehension) based on their priordomain knowledge. Also prior domain knowledge might be more

474

important than navigation choices. Readers with different prior do-main knowledge might need to navigate through a hypertext environ-ment differently in order to understand the ideas presented in the text.Empirical data from future studies can provide insights into whetherthere is a need to consider a hierarchical priority scheme in the meta-navigation support rules.

Other variables might also be important for learning from hyper-text. One variable that has been suggested by some researchers to bepredictive of learning from hypertext is spatial ability (Chen & Rada,1996). Users with high spatial ability have been found to interactmore efficiently with hypertext environments than users with lowspatial ability. A study conducted by Campagnoni and Ehrlich (1989)revealed that users with good visualization ability are able to learnthe structure of a subject domain more quickly than users with poorvisualization ability. Future researchers may want to examine theeffects of differences in spatial ability on navigation behavior and onthe ability to learn with hypertext. It would also be interesting toinvestigate how metacognitive and spatial abilities interact to affectlearning from hypertext.

Acknowledgements

This research is supported by National Science Foundation’s earlycareer grant (CAREER #9985158) to the first author. We thank thestudents and teachers who participated in the studies. We also thankRoland Hubscher and Ling Wang for their help with the Pathfinderand clustering algorithms.

Appendix A

Example items from pre- and post-tests1. The amount of work done on an object is obtained by multiplying

• Input force and output force• Force and distance• Time and force• Efficiency and work

2. Neither force nor distance is multiplied in a(an)

• Inclined plane• Wheel and axle

475

• Movable pulley• Fixed pulley

3a. Look at the scales below. Which of the boxes X, Y or Z has theLEAST mass?

• X• y• Z• All three boxes have the same mass

3b. Explain why a scale (like the one in the diagram above in ques-tion 2a) is a lever.

4a. The weights of the three blocks A, B, C were compared asshown below.

Which one of the blocks weighs the most?

• A• B• C

476

4b. Explain your answer.

Appendix B

Example items from the CoMPASS questionnaire

Statement

1. The concept maps helped me understand the science conceptsbetter.

2. I preferred to use the text only for navigation and not the maps.3. Concept maps helped find information easily.4. I liked being able to control the order in which the text could be

read by selecting the concepts.5. The maps helped understand the strength of the relationships

among concepts.6. The concept maps helped me understand relationships among the

concepts that were not known before.7. I found navigating via the maps was better than navigating via the

text.

Appendix C

Example prompts in the metanavigation support conditionHello _____! Here are some useful tips that might help you while

you use CoMPASS to find information to solve the pulley challenge.1. USE THE MAPS! Have you ever used the maps to help you de-

cide where maps in CoMPASS can help you in different ways:

• The maps help you see what other science concepts are closelyrelated to what you are reading.

EXAMPLE: If you are reading about mechanical advantage (ma) in apulley the maps show that distance, force and work are science con-cepts that are closely related to mechanical advantage.

• The maps show relationships among science concepts and helpyou make connections.

477

EXAMPLE: If you look at the map there are different words that arewritten on the arrows to explain the relationship between science con-cepts. If you put your mouse over the arrows you can read how theyare related.

2. Think about your goal! What science concepts will help yousolve the challenge?

3. Remember that solving the challenge is like doing a puzzle. Youneed to put all the pieces of information together and understandhow they are related. The maps can help you do that!

References

Abbas, J., Norris, C. & Soloway, E. (2002). Middle school children’s use of the

ARTEMIS digital library. In G. Marchionini and W. Hersh, eds, Proceedings of theSecond ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 98–105. ACMPress: New York.

Azevedo, R. & Cromley, J.G. (2004). Does training on self-regulated learningfacilitate students’ learning with hypermedia? Journal of Educational Psychology96(3): 523–535.

Azevedo, R., Guthrie, I.T., Wang, H. & Mulhern, J. (2001). Do different instructional

interventions facilitate students’ ability to shift to more sophisticated mental models ofcomplex systems?. Seattle, WA: Paper presented at the Annual Conference of theAmerican Educational Research Association.

Barab, S.A., Fajen, B.R., Kulikowich, J.M. & Young, M.F. (1996). Assessinghypermedia navigation through pathfinder: prospects and limitations. Journal ofEducational Computing Research 15(3): 185–205.

Bedersen, B.B. & Hollan, J. (1995). Pad++: a zooming graphical interface forexploring alternate interface physics. Paper presented at the ACM UIST’94.

Brown, A., Armbruster, B. & Baker, L. (1986). The role of metacognition in readingand studying. In J. Orasanu, eds, Reading comprehension: from research to practice,

pp. 49–75. Lawrence Erlbaum: Hillsdale, NJ.Brown, A.L. & Palincsar, A.S. (1985). Reciprocal teaching of comprehension strategies:

a natural history of one program for enhancing learning. In J.G. Borkowski, eds,

Intelligence and Exceptionality: New Directions for Theory, Assessment, andInstructional Practice, pp. 81–132. Able: Norwood NJ.

Brusilovsky, P. (1998). Adaptive educational systems on the World-Wide-Web: a review

of available technologies. Presented at the workshop ‘‘WWW-Based Tutoring’’ at4th International Conference on Intelligent Tutoring Systems (ITS’98), SanAntonio, TX, August 16–19, 1998.

Brusilovsky, P., Eklund, J. & Schwarz, E. (1998). Web-based education for all: a tool fordeveloping adaptive courseware. Computer Networks and ISDN Systems. InProceedings of the 7th International World Wide Web Conference, pp. 291–300.Elsevier Science.

Brusilovsky, P., Schwarz, E. & Weber, G. (1996). Elm-art: an intelligent tutoring systemon the World Wide Web. Paper presented at the Third international conference onIntelligent Tutoring Systems, Montreal, Canada, June, 10–12.

478

Campagnoni, F. & Ehrlich, K. (1989). Information retrieval using hypertext-based helpsystem. ACM Transactions on Information Systems 7(3): 271–291.

Charney, D. (1994). The impact of hypertext on processes of reading and writing. In

S.J. Hilligoss and C.L. Selfe, eds, Literacy and Computers, pp. 238–263. ModernLanguage Association: New York.

Chen, C. & Rada, R. (1996). Interacting with hypertext: a meta-analysis of experimental

studies. Human–Computer Interaction I 1(2): 125–156.De Bra, P. (1998). Adaptive hypermedia on the Web: methods, technology and

applications. In Proceedings of the 3rd World Conference of the WWW, Internet

and Intranet (WebNet ’98), pp. 220–225. Charlottesville, VA: AACE.Dee-Lucas, D. (1996). Effects of overview structure on study strategies and text

representations for instructional hypertext. In J. Rouet, J.J. Levonen, A. Dillon andR.J. Spiro, eds, Hypertext and Cognition, pp. 73–107. Erlbaum: Mahwah, NJ.

Dillon, A. & Vaughan, M. (1997). ‘‘It’s the journey and the destination’’: shape and theemergent property of genre in evaluating digital documents. New Review ofMultimedia and Hypermedia 3: 91–106.

Flavell, J.H. (1979). Metacognition and cognitive monitoring: a new area of cognitivedevelopmental inquiry. American Psychologist 34(10): 906–911.

Foltz, P.W. (1996). Comprehension, coherence and strategies in hypertext and linear

text. In J. Rouet, J.J. Levonen, A. Dillon and R.J. Spiro, eds, Hypertext andCognition, pp. 109–116. Erlbaum: Mahwah, NJ.

Fumas, G.W. & Bedersen, B.B. (1995). Space-scale diagrams: understanding multiscaleinterfaces. In Proceedings of the Conference on Human Factors in Computing

Systems (CHI ’95), pp. 234–241. New York: ACM Press.Garner, R. (1987). Metacognition and Reading Comprehension. Norwood, NJ: Ablex.Glynn, S.M., Yeany, R.H. & Britton, B.K. (1991). The Psychology of Learning Science.

Hillsdale, NJ: Erlbaum.Hill, J. & Hannafin, M.J. (1997). Cognitive strategies and learning from the World-Wide

Web. Educational Technology Research and Development 45(4): 37–64.

Hoffman, J.L., Wu, H.-K., Krajcik, J.S. & Soloway, E. (2003). The nature of middleschool learners’ science content understandings with the use of on-line resources.Journal of Research in Science Teaching 40(3): 323–346.

Hubscher, R. & Puntambekar, S. (2001). Navigation support in adaptivehypermedia systems: is more indeed better?. In J.D. Moore, C.L. Redfield andW.L. Johnson, eds, Artificial Intelligence in Education, AI-ED in the Wired andWireless World, pp. 13–22. IOS Press: Netherlands.

Hubscher, R. & Puntambekar, S. (2002). Adaptive navigation for learners inhypermedia is scaffolded navigation. In P. De Bra, P. Brusilovsky and R. Conejo,eds, Adaptive Hypermedia and Adaptive Web-Based Systems, Vol. LNCS 2347,

pp. 184–192. Berlin: Springer.Jacobs, J.E. & Paris, S.G. (1987). Children’s metacognition about reading: issues in

definition, measurement, and instruction. Educational Psychologist 22: 255–278.

Jacobson, M.J. & Spiro, R.J. (1995). Hypertext learning environments, cognitiveflexibility, and the transfer of complex knowledge: an empirical investigation.Journal of Educational Computing Research 12(4): 301–333.

Jacobson, M.J., Maouri, C., Mishra, P. & Kolar, C. (1996). Learning with hypertext

learning environments: theory, design, and research. Journal of EducationalMultimedia and Hypermedia 5((3l4): 239–281.

479

Jonassen, D.H. & Wang, S. (1993). Acquiring structural knowledge from semanticallystructured hypertext. Journal of Computer Based Instruction 20(1): 1–8.

Kanungo, T., Mount, D.M., Netanyahu, N., Piatko, C., Silverman, R. &Wu, A. (2000).

The analysis of a simple k-means clustering algorithm. Symposium on Computa-tional Geometry.

Kolodner, J.L. (1997). Educational implications of analogy: a view from case-based

reasoning. American Psychologist 52(1): 57–66.Kolodner, J.L., Camp, P., Crismond, D., Fasse, B., Gray, J., Holbrook, J., Puntambekar,

S. & Ryan, M. (2003). Problem-based learning meets case-based reasoning in the

middle school science classroom: Putting Learning byDesignTM into Practice. Journalof the Learning Sciences 12(4): 485–547.

Krajcik, J.S., Blumenfeld, P.C., Marx, R.W. & Soloway, E. (1991). A collaborativemodel for helping middle grade science teachers learn project-based instruction. The

Elementary School Journal 94(5): 483–497.Landow, G.P. (1992). Hypertext: The convergence of contemporary critical theory and

technology. Baltimore, MD: Johns Hopkins University Press.

Lawless, K.A. & Kulikowich, J.M. (1996). Understanding hypertext navigation throughcluster analysis. Journal of Educational Computing Research 14(4): 385–399.

Laurillard, D., Stratfold, M., Luckin, R., Plowman, L. & Taylor, J. (2000). Affordances

for Learning in a non-linear narrative medium. Journal of Interactive Media inEducation 2, http://www-jime.open.ac.uk/00/(2)). Retrieved on January, 27, 2001.

Lee, P., Lehman, J.D., Eichinger, D. & Frederick, F. (1997). The effects of conceptmapping and metacognitive cues in a hypermedia-based genetics program, Paper

presented at the NARST conference, Chicago IL.Marchionini, G. (1988). Hypermedia and learning: freedom and chaos. Educational

Technology 28(11): 8–12.

Mokhtari, K. & Reichard, C.A. (2002). Assessing students’ metacognitive awareness ofreading strategies. Journal of Educational Psychology 94(2): 249–259.

Niegemann, H.M. (2001). Analyzing navigation patterns of learning in hypermedia

learning environments. Paper presented at the annual meeting of the AmericanEducational Research Association, Seattle WA.

Novak, J.D. & Gowin, D.B. (1984). Learning how to learn. Cambridge: Cambridge

University Press.Paris, S.G. & Myers, M. (1981). Comprehension monitoring, memory, and study

strategies of good and poor readers. Journal of Reading Behavior 13: 5–22.Paris, S.G. & Jacobs, J.E. (1984). The benefits on informed instruction for reading

awareness and comprehension skills. Child Development 55: 2083–2093.Park, J. & Kim, J. (2000). Contextual navigation aids for two World Wide Web systems.

International Journal of Human–Computer Interaction 12(2): 193–217.

Pirolli, P.L., Card, S.K. & Van Der Wege, M. (2001). Visual information foraging in afocus + context visualization. Proceedings of ACM Conference on Human Factorsand Computing Systems (CHI2001), pp. 506–513. New York: ACM Press.

Pressley, M. & Afflerbach, P. (1995). Verbal protocols of reading: The nature ofconstructively responsive reading. Hillsdale, NJ: Erlbaum.

Puntambekar, S. & Stylianou, A. (2002). CoMPASS: students’ use of externalrepresentations in science learning. In P. Bell, R. Stevens and T. Satwicz, eds,

Keeping Learning Complex: The Proceedings of the Fifth International Conference ofthe Learning Sciences (ICLS), pp. 352–358. Erlbaum: Mahwah, NJ.

Puntambekar, S., Stylianou, A. & Hubscher, R. (2003). Improving navigation and

480

learning in hypertext environments with navigable concept maps. Human–ComputerInteraction 18(4): 395–426.

Rouet, J. (1992). Cognitive processing of hyperdocuments: when does nonlinearity help?

Proceedings of the Fourth ACM Conference on Hypertext, Association for ComputingMachinery, pp. 131–140. New York: ACM Press.

Rowe, A.L., Cooke, N.J., Hall, E.P. & Halgren, T.L. (1996). Toward an on-line

knowledge assessment methodology: building on the relationship between knowingand doing. Journal of Experimental Psychology: Applied 2(1): 31–47.

Ruiz-Primo, M.A. & Shavelson, R.J. (1995). Concept maps as potential alternative

assessments in science. Paper presented at the Annual Meeting of the AmericanEducational Research Association (AERA), San Francisco, CA.

Schvaneveldt, R.W. (1990). Pathfinder Associative Networks. Norwood, NJ: Ablex.Shapiro, A.M. (1998). What do users learn from ‘‘Web’’ technologies? In Z. Berge and

M. Collins, eds, Designing the Online Classroom in Elementary and SecondarySchools, Vol. 2, pp. 185–195. Cresskill, NJ: Hampton Press.

Shapiro, A.M. (2000). The effect of interactive overviews on the development of

conceptual structure in novices learning from hypermedia. Journal of EducationalMultimedia and Hypermedia 9(1): 57–78.

Shapiro, A.M. & Niederhauser, D. (2004). Learning from hypertext: research issues and

findings. In D.H. Jonassen, eds, Handbook of Research on Educational Communi-cations and Technology, pp. 605–620. 2nd edn., Erlbaum: Mahwah, NJ.

Spiro, R.J., Feltovich, P.J., Jacobson, M.J. & Coulson, R.L. (1991). Cognitive flexibilityconstructivism, and hypertext: random access instruction for advanced knowledge

acquisition in ill-structured domains. Educational Technology 31(5): 24–33.Stylianou A. (2003). How do students navigate and learn from nonlinear science texts?

Can metanavigation support help? Unpublished doctoral dissertation, University of

Connecticut.Stylianou, A. & Puntambekar, S. (2004). Supporting middle school students use

nonlinear science texts in an inquiry classroom. In Y. Kafai, W.A. Sandoval,

N. Enyedy, S.A. Nixon and F. Herrera, eds, Embracing Diversity on the LearningSciences. Proceedings of the Sixth International Conference of the Learning Sciences,pp. 529–536. Erlbaum: Mahwah, NJ.

Wright, P. (1982). A user-oriented approach to the design of tables and flowcharts. InD.H. Jonassen, eds, The Technology of Text 1, pp. 317–340. EducationalTechnology Publications: Englewood Cliffs.

481