Research ArticleHow Influential Are Mental Models on Interaction PerformanceExploring the Gap between Usersrsquo and Designersrsquo Mental Modelsthrough a New Quantitative Method

Bingjun Xie Jia Zhou and HuilinWang

Department of Industrial Engineering Chongqing University Chongqing 400044 China

Correspondence should be addressed to Jia Zhou zhoujia07gmailcom

Received 15 May 2017 Accepted 13 September 2017 Published 18 October 2017

Academic Editor Hideyuki Nakanishi

Copyright copy 2017 Bingjun Xie et al This is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

The objective of this study is to investigate the effect of the gap between two different mental models on interaction performancethrough a quantitative way To achieve that an index called mental model similarity and a new method called path diagram toelicit mental models were introduced There are two kinds of similarity directionless similarity calculated from card sorting anddirectional similarity calculated frompath diagramAn experimentwas designed to test their influenceA total of 32 college studentsparticipated and their performance was recorded Through mathematical analysis of the results three findings were derived Fristthe more complex the information structures the lower the directional similarity Second directional similarity (rather thandirectionless similarity) had significant influence on user performance indicating that it is more effective in elicitingmental modelsusing path diagram than card sorting Third the relationship between information structures and user performance was partiallymediated by directional similarity Our findings provide practitioners with a new perspective of bridging the gap between usersrsquoand designersrsquo mental models

1 Introduction

Originating from psychology mental models are appliedto Human-Computer Interaction (HCI) to explain peoplersquosunderstanding about how computers work [1] Informationtechnology (IT) products are developed based on designersrsquomental models but what designers believed to be easy tounderstand is not necessarily true for users Users interactwith IT products in a different perspective from designersThey form their own understanding and predict feedback ofIT products [2] Users whose mental models are differentfrom those of designers encounter interaction difficultiesbut certain users with wrongincomplete mental models canalso successfully use IT products [3] Although actions (egtraining) have been taken to reduce the gap between usersrsquoand designersrsquo mental models few studies examine whetherand to what extent the gap is reduced

To address the above problem this study quantified thegap between mental models and investigated its impact onuser performance However it is challenging to elicit and

quantify the gap between mental models because they arein the usersrsquo head and are not directly observable Manyresearchers have explored mental models using IT productswith a well-defined information structure (eg web pagesand menus) that clearly reflects designersrsquo mental modelsThen usersrsquo mental models of information structures canbe elicited Thus the extent to which usersrsquo mental modelsdiffered from those of the designers can be quantified

Traditional method for eliciting mental models of infor-mation structures was card sorting but mental modelselicited by card sorting cannot represent usersrsquo understandingof specific directional relationship between elements of infor-mation structures This study proposed a new method pathdiagram to elicit mental models of information structuresThe mental models elicited by path diagram can representusersrsquo understanding of directional relationship betweenelements of information structures To further quantify thegap between the mental models this study introduced anindex called mental model similarity Two kinds of mentalmodel similarity are distinguished directionless similarity

HindawiAdvances in Human-Computer InteractionVolume 2017 Article ID 3683546 14 pageshttpsdoiorg10115520173683546

2 Advances in Human-Computer Interaction

calculated from the card sorting mainly represents the direc-tionless relationship between elements directional similaritycalculated from the path diagram mainly represents thedirectional relationship between elements Therefore tworesearch questions were considered (i) Which method ismore effective in eliciting mental models of informationstructures (ii) How influential are directional similarity anddirectionless similarity on interaction performance

In this study websites with three information structures(ie net tree and linear) were developed to elicit mentalmodels through card sorting and path diagram Then amethod to quantify the degree of match between usersrsquo anddesignersrsquo mental models was proposed Based on that theimpact of mental models on performance was analyzed

2 Literature Review

21 MentalModels The theory of mental models has obscureorigins [4 5] but the notion of mental models first appearedin a bookwritten by the psychologist Craik Craik [6] believedthat a brain could translate an external process into a modelof the world which is ldquoa small-scale model of external realityand of its own possible actions within the headrdquo Since thenit has attracted much attention from researchers particularlypsychologists

Forty years later two researchers used the term ldquomentalmodelrdquo Norman stated that ldquoin interacting with the envi-ronment with others and with the artifacts of technologypeople form internal mental models of themselves and ofthe things with which they are interacting These modelsprovide predictive and explanatory power for understandingthe interactionrdquo [3] Johnson-Laird [7] believed that peoplecould creatementalmodels thatwere structural analogs of theworld and their ability to construct manipulate and evaluatemental models had a hidden strong influence on rationalthought

Subsequent research on mental models can be approxi-mately divided into two branchesThe firstmainly focused oninternal mental processes and cognitive phenomena withinthe long-standing field of psychology Typical research inter-ests included the role of mental models in comprehension[8 9] reasoning [9 10] and deduction [11] The secondbranch stepped out of psychology and appliedmental modelsto support better interaction between people and the externalworld Typical research interests include the role of mentalmodels in learning and training [12ndash15] and using computersand appliances [16ndash19]

Among the second branch of research one noticeabletrend is a surge in the application of mental models inHuman-Computer Interaction (HCI) Norman [20] showedthat the root cause of problems in using technology productswas the gap between usersrsquo and designersrsquo mental modelsThis highlights a new perspective for HCI and workers havetried various ways to deliver a design that matches usersrsquomental models [1 21]

Originally used to explain team effectiveness TeamMental Models (TMMs) refer to the extent to which teammembers shared organized understanding and mental rep-resentation of knowledge or beliefs relevant to the key

content of the teamrsquos tasks [22 23] Mathieu et al [24]proposed that team membersrsquo mental models consist of fourparts technology job or task team interaction and otherteammatesrsquo knowledge and attitudes

Since mental models are within the head they cannot bedirectly detected Peoplersquos mental models had to be indirectlyinferred from observing and analyzing their elements Manyearly researchers followed this focus finding it challengingbecause mental models had the following five characteristics(1) incompleteness mental models are constrained by usersrsquobackground expertise and so forth (2) vague boundariesthey can be confused with similarrelated operations andsystems (3) being unstable over time they evolve as peopleforget and learn (4) they contained aspects of superstitions[20] (5) Tendency to parsimony people tend to construct alimited model of the relevant parts of a system [21]

Despite these challenges researchers have identified vari-ous techniques to elicit mental models General techniques toelicit individualsrsquo mental models and team membersrsquo sharedmental models were summarized in three comprehensivereview papers [25ndash27] Specifically in the field of HCItechniques to elicit mental models have four major cate-gories (1) verbalization through interview thinking aloudladdering and so forth verbalization is the most widely usedelicitation technique [5] However peoplersquos verbalization wasinconsistent and tended to evolve as they spoke [28 29] Onesolution to this problem is laddering a modified interviewtechnique in which people were asked to identify multipleaspects of a problem and explore the relationship betweentheir answers [30] (2) Rating people were asked to rate usingquestionnaires [30 31] but this technique is not frequentlyused because relatively few questionnaires are well estab-lished (3) Drawing sketches people were asked to visuallydraw how they thought of a concept or the pathways from thestart to a specific point in a system [18 32] (4) Card sortingthis technique is widely used in eliciting mental models ofhierarchical systems [18 33] In most cases the method iseffective in eliciting mental models This is especially truewhen dealing with information structures without consider-ing the directional relationship However card sorting is notadequate for eliciting complete mental models if we considerthe directional relationship of information structures

22 Relation between Information Structures and MentalModel Similarity Mental models are nowadays widely ap-plied to analyze user performance on web pages which arethe best carrier of information structures Many aspects ofhuman interactionwith hierarchical systems involve complexprocesses thus peoplewho interactwith hierarchical systemsmust have some type of mental models Since mental modelsrepresent usersrsquo understanding about a system including webpages we argue that simplicity of information structures hasan impact on mental model similarity Previous studies haveindicated that the card sorting is widely used in elicitingmental models of hierarchical systems [18 33] Based on thiswe proposed Hypothesis (H1a)

(H1a) The more complex the information structures thelower the directionless similarity

Advances in Human-Computer Interaction 3

The directional similarity is calculated from path dia-gram which is used to elicit mental models of informationstructures with directional relationship We add more details(eg directional relationship) to mental models Based onthis we proposed Hypothesis (H1b)

(H1b) The more complex the information structures thelower the directional similarity

23 Mental Models and User Performance Many previousstudies have shown that mental models are correlated withuser performance On the one hand mental models havea positive effect on user performance Ziefle and Bay [33]pointed out that the better the mental models of navigationsmenus the better the performance using the devices Young[34] thought that mental models could explain user perfor-mance with the systems with which they interact Dimitroff[35] found that students with more complete mental modelsmade significantly fewer errorswhen they used theUniversityof Michiganrsquos website Sasse [36] noted the significant effectsofmentalmodels on user performance using Excel Slone [37]found that usersrsquo mental models affected their performanceon websites Brandt and Uden [38] pointed out that noviceswithout strongmentalmodels for information retrieval couldnot gather information successfully

Numerous studies have also shown the relationshipbetween TMMs and team performance For example Math-ieu et al [24] considered shared mental models as twocategories task and team finding that both team-basedand task-based mental models related positively to subse-quent team process and performance Mathieu et al [39]demonstrated in a PC-based flight simulator that both taskand team models had an impact on performance the teamprocess was supported by shared mental models and task-work mental model similarity but not by teamwork mentalmodel similarity which was significantly related to both teamprocesses and team performance

On the other hand workers have found that mentalmodels had either an adverse effect or no significant effect onuser performance Halasz and Moran [40] and Borgman [41]found that whether users formed amental model of a systemor not they performed no differently on routine simple tasksNorman [3] found that users with wrongincomplete mentalmodels could use technology products successfully Payne[42] noted that evenwrongmentalmodels did not necessarilyresult in the bad usage of devices Schmettow and Sommer[43] found that the degree of match between the mentalmodel and website structure had no effect on usersrsquo browsingperformance As for TMMs Webber et al [44] found thatteam members sharing a common mental model with poorquality did not likely perform well

It is obvious that the research consequences were con-tradictory It seems that researchers cannot get the unifiedcognition of the effects of mental models on user perfor-mance One possible reason is that most researchers do theirstudies without consideringmental model similarity betweenusers and designers It is necessary for us to investigatemental modelsrsquo effects on interaction performance involvingthe mental model similarity The only study of mental model

similarity on interaction performance is seen in Schmettowand Sommer [43] They found that mental model similarityhad no effect on interaction performance However the waythey elicitedmental models was card sorting and they did notconsider the directional relationship of information struc-tures In this paper we considered more details such as direc-tional relationship to elicit a mental model of informationstructure and get the directional similarity frompath diagramand the directionless similarity from card sorting whichmay get different results from what Schmettow and Sommer[43] found Considering that there are more positive effectsthan adverse effects in the existing studies in this paper weare partial to supporting the positive effects Therefore weproposed Hypothesis (H2a) and Hypothesis (H2b)

(H2a) Directional similarity predicts the task completiontime the higher the directional similarity the less thetask completion time

(H2b) Directional similarity predicts the number of clicksthe higher the directional similarity the less the clicktimes

The effects of information structures on user performancehave been widely investigated since mental models areclosely related to usersrsquo behavior using various devices a goodmental model of information structure will likely enhanceuser performance However there are many other factorsinfluencing user performance besides mental models forexample usersrsquo age and gender Ziefle and Bay [33] pointedout that younger users were more effective in using a cellphone menu than the older users Mathieu et al [24] foundthat team processes fully mediated the relationship betweenteam mental models and team effectiveness Mathieu et al[39] found that team performance was partially mediatedby teammatesrsquo mental models Based on these findingswe have assumed that mental model similarity mediatedthe relationship between information structures and userperformance which is Hypothesis (H3a) and Hypothesis(H3b) (see Figure 1)

(H3a) Directionless similarity will mediate the relationshipbetween information structures and user perfor-mance

(H3b) Directional similarity will mediate the relationshipbetween information structures and user perfor-mance

24 Quantitative Methods of Mental Model Similarity Elic-ited mental models are widely used in two ways First thedifference between elicited mental models can be qualita-tively and quantitatively compared One common quanti-tative method is to compare the depth and width of theelicited structures Users use an interface with a well-definedinformation structure such as phone menus and web pages

Another quantitative method is to calculate the teammental model (TMM) similarity score which indicates thepercentage of team membersrsquo shared organized under-standing and mental representation of knowledge or beliefsrelevant to the key content of the teamrsquos task [22 23]

4 Advances in Human-Computer Interaction

Mental model similarity(directional similarity directionless similarity)

Information structure User performance

Figure 1 The mediation effect of mental model similarity in the relation between information structure and user performance

Specifically team members usually rated the relatedness ofpairs of statements describing team interaction processesand characteristics of team members on the Likert scaleThe response could be analyzed using multidimensionaltechniques (eg the Pathfinder technique) to calculate thesimilarity score [26 45 46]

However none of these methods can accurately elicitmental models of information structures with directionalrelationship also these methods cannot quantitatively cal-culate the degree of similarity or dissimilarity of the mentalmodels between users and designers of systems Previousstudies [43] have shown that the widely usedmethod of elicit-ing mental models of hierarchical systems is card sorting butit is inadequate to elicit mental models without consideringthe directional relationship of information structures Alsothe previous studies did not introduce a suitable method toquantify the mental model similarity Wu and Liu [47] pro-posed a new computational modeling approach which wascomposed of a simulation model of a queuing network archi-tecture and a set of mathematical equations implemented inthe simulation model to quantify mental workload to modelthe mental workload in drive and driving performanceHowever the approach proposed was not the quantitativemethod for calculating the mental model similarity It wasapplied to analyze themental workload in driver informationsystem Thus here we elicit mental models through cardsorting and path diagramThen we conducted a newmethodto quantify the degree of mental model similarity

3 Methodology

31 Equipment and Materials A notebook computer with atouch screen (ThinkPad YogaS1) was used to present webpages and a whiteboard was used to draw path diagrams Acamera (Sony HDR-PJ610E) was used to record participantsrsquoresults of card sorting and drawing the path diagrams AMorae Recorder was used for counting task completion timeand the number of clicks andwas also used to present the taskspecification for the participants Web pages with differentinformation structures were developed The web pages werefirst considered about the navigation structures which werenet tree and linear The depth of tree and net structure wasthree levels which was seen common in daily web pagesThewidth of the bottom level of tree and net structure was nineitems which was also seen in common web pages

To avoid the effects of familiar knowledge topics aboutancient inventions ancient books and ancient historical

Table 1 Experience of using technology products of participants

Mean SDLaptop 075 114Tablets 495 280Smart phones 458 342

characters of different Chinese dynasties which were notwidely known were presented in web pages as the contentThere were totally nine different Chinese dynasties and eachinformation structure was made of three different Chinesedynasties Ancient inventions ancient books and ancienthistorical characters were corresponding to their own dynas-ties Two pretests were carried out by two college studentsand the interaction forms and content layout were adjustedaccording to the resultsThememory capacity that influencesuser performance [33] was tested by a KJ-I spatial locationmemory span tester Spatial ability was tested through thepaper folding test [48] The original paper folding test wastranslated into Chinese

32 Participants A total of 32 students from ChongqingUniversity were recruited The age of the participants rangedfrom 21 to 26 years (mean = 2313 SD = 17) The gender wasbalancedThe experience of using laptops tablets and smart-phones (see Table 1) was investigated the results indicatedthat handheld devices (ie smartphones and tablets) wereused intensively

33 Task All participants completed tasks in three differentweb pages with different information structures net treeand linear To avoid learning effects and make participantsget a full understanding of the information structures of webpages each participant completed eight tasks in a web pageThe order of web pages in which participants completed thetasks is random

The tasks are searching tasks Participants found a targetand read its content The target is an item hidden in webpages which is not widely known for participants To checkwhether they read the content carefully a single-choice testwas conducted Participants wrote down answers on a pieceof paperThe test has two questions one is aboutwhat dynastythe target belonged to and the other one was about which thetarget is about Taking ldquoHuang dao you yirdquo as an examplethe task is ldquoPlease find ldquoHuang dao you yirdquo and read itsdescription and then answer the following two questions

Advances in Human-Computer Interaction 5

Q1 which dynasty ldquoHuang dao you yirdquo belongs to Q2 whatldquoHuang dao you yirdquo is aboutrdquo

34 Dependent Variables The two dependent variables weretask completion time and the number of clicks Task com-pletion time was the average of eight tasksrsquo completion timeunder each information structure and the number of clickswas the average of the click times to complete all of the eighttasksThey were both measured by the Morae Recorder onlywhen the participant obtained the correct answers for bothquestions were the missions completed

35 Independent Variable The independent variables werethe directionless similarity and directional similarity It is anew quantitative measure proposed here (see Section 37)the directionless similarity was calculated from card sortingand the directional similarity was calculated from pathdiagramThemental model similarity was computed for eachinformation structure

Demographic variables included age experience of usingtechnology product spatial ability and memory capacity Aquestionnaire was used to collect basic information aboutexperience of using technology product

The mental model similarity was a between-subject vari-able while the information structure was a within-subjectvariableThat is each participant used three prototypes In aneffort to avoid learning effects the order of using prototypeswas random

36 Procedure The experiment took each participant aboutone hour to complete It consisted of four tests a question-naire test a memory test a paper folding test and a cardsorting test

First each participant began the experiment by filling outa consent form and a general questionnaire about hisherdemographic information and using experience with tech-nology products

Secondly a memory test was conducted using the KJ-I spatial position memory span tester After completing thememory test the test scores were recorded on paper and werenot disclosed to the participants

Thirdly a paper folding test was conducted to test thespatial ability of the participants This test is divided into twoparts The time limit of each part was three minutes to avoidparticipants losing their patience The participants needed tofind the correct answer independently The final score of thetest is the correct number minus the incorrect number on thetest paperThe higher the score the better the spatial memoryability

Fourthly a brief introduction and practice about theexperiment were given to each participant Finally partici-pants completed tasks on each web page and then went onwith the card sorting The cards were the titles of each nodein the information structures Then the participants wererequired to draw a path structure of the experimental webpage navigation with a whiteboard stroke During the wholeprocess participants were left alone and questions related tothe path were not answered in a relevant way which aimed atavoiding the subjective impacts of the experimental designer

At the end the experimenter conducted a five-minute explor-atory interview with the participants to understand theirthoughts and feelings about using the three web pages Thequestions in the interview included ldquoQ1 Please score thethree web pages in this experiment considering informationsearching ease of use and user experience Q2 Pleasesequence the three web pages according to your experiencerdquo

37 Quantifying Mental Model Similarity The method pro-posed to quantify mental model similarity consists of twoparts one is the method which can be used to elicit mentalmodels of information structures with directional relation-ship and the other one is the mathematical equations whichcan be used to calculate the mental model similarity

The first part is the method of eliciting mental models Inorder to quantify mental model similarity a method whichcan be used to elicit mental models with more details such asdirectional relationship has to be used Such amethod shouldrepresent the understanding of elements and directionalrelationship of a hierarchical system which are the keyfactors in elicitingmental model of a hierarchical systemTheliterature provides several methods to elicit mental modelssuch as card sorting which is widely used in eliciting mentalmodels of hierarchical systems [18 33] However card sortingcannot elicit completemental models of hierarchical systemsparticularly the directional aspects of hierarchies

A new method is needed to reflect the directionalinformation of mental models of hierarchy structures Webnavigation is similar to the real-world navigation In real-world navigation people usually take three strategies tofind a destination They remember properties of landmarkssuch as shape and structure (ie landmark knowledge) [4950] or the sequential order of landmarks encountered anddirectional relationship between these landmarks (ie routeknowledge) [51] or an overview of the environment likea map showing spatial relationships between routes andlandmarks (ie survey knowledge) [52] to find a destinationThis knowledge is also involved in web navigation Land-mark knowledge is mainly represented through card sortingInspired by route knowledge we proposed the path diagramto elicit mental models of hierarchical systems with moredetails (eg directional relationship)

The second part is about quantifying the mental modelsimilarity The limited research in quantifying mental modelsimilarity was reported by Sinreich et al [53] They intro-duced process chart into eliciting mental models of Emer-gency Department Management and quantified similaritythrough four formulae However themethodwas not appliedto analyze the mental models of information structures

The method of quantifying the degree of mental modelsimilarity aims to reflect the extent to usersrsquo understandingof the system it can present causality and logic In additionthis method should be easy to understand so that it can bemastered by nonprofessional persons Thus in this study weextended the work of Sinreich et al [53] as our proposedquantitative method We calculated mental model similarityin twoways calculating the directionless similarity from cardsorting and calculating the directional similarity from pathdiagram

6 Advances in Human-Computer Interaction

Invention Book

View 1 View 2

Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Invention

BookHistoricalcharacter

Diao ban yinshua

Shui yun hun yi Huang dao you

yi

Page name Invention

HomepageBook Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Page name Invention

HomepageInvention

Shui yun hun yiDiao ban yin shua Huang dao you yi

Diao ban yin shua Shui yun hun yi Huang dao you yi

Ji li gu che Zhuan lun cang Lian hua lou

Chai se tao yin Jian yi Di qiu yi

Figure 2 Two views of web page interfaces in this experiment

The first component represents the elements (nodes) ofthe different content for example ldquoTang dynastyrdquo ldquoInven-tionrdquo ldquoBookrdquo and ldquoHistorical characterrdquo in Figure 2 Thedirectionless similarity measure 119886119894119895 can be obtained using

119886119894119895 = 119890119894119895119890119894119895 + 119887119894119895 + 119887119895119894 (1)

where 119890119894119895 denotes the number of identical elements in cardsortings 119894 and 119895 and 119887119894119895 denotes the number of elements thatexist in 119894 that do not exist in 119895 (see that 119887119894119895 = 119887119895119894) It isclear that 0 ≪ 119886119894119895 ≪ 1 In the case that both card sortingsare identical in terms of their elements (not necessarily theirrelationships) then we have 119886119894119895 = 1 while 119886119894119895 = 0 if nocommon elements exist and by definition 119886119894119895 = 119886119895119894

The second component represents the relationshipbetween elements (arcs) in the path diagram A relationshipis defined by the elements it connects (there may be morethan one connection between elements) and by the directionof the connecting arc for example the arcs that connectelements ldquoTang dynasty-Inventionrdquo ldquoInvention-Bookrdquoand ldquoInvention-Diao ban yin shuardquo in path diagram A inFigure 4The first step in calculating the directional similarityis to obtain the adjacency matricesThe element of adjacencymatrix was the numbers of directed segment between twonodes (eg if there were one directed segment from ldquoTangdynastyrdquo to ldquoInventionrdquo the element is 1)

Based on the adjacency matrix the sum of all the com-mon arcs 119888119894119895 and the sum of all exclusive arcs 119889119894119895 betweenany two path diagrams 119894 and 119895 can be calculated as shownin (2) and (3) respectively Finally the directional similaritymeasure 119903119894119895 can be obtained using (4)

119888119894119895 = sum119896

sum119897

min ℎ119894119896119897 ℎ119895119896119897 (2)

119889119894119895 = sum119896

sum119897

10038161003816100381610038161003816ℎ119894119896119897 minus ℎ119895119896119897

10038161003816100381610038161003816 (3)

119903119894119895 = 119888119894119895119888119894119895 + 119889119894119895 (4)

It is clear that 0 ≪ 119903119894119895 ≪ 1 In the case that both pathdiagrams are identical in terms of their relationship (arcs)then we have 119903119894119895 = 1 while 119903119894119895 = 0 if no common relationshipexists between the two path diagrams By definition 119903119894119895 = 119903119895119894

These formulae adapted from Sinreich et al [53] arevalidated and used to calculate the similarity between processcharts In this study we used them to analyze hierarchiesand extended their work by adding directions To analyzedirectional relationship an adjacency matrix was used toindicate relationship according to the graph theory Thenrelationship similarity of hierarchical systems was calculatedby using formulas (2) (3) and (4)

The process of calculating the mental model similaritycan be seen in Figure 3

In order to illustrate the calculation procedure of thesimilarity measure the following example is given (seeFigure 4)

From Figure 4 the following values are obtained 119890119894119895 = 13and 119887119894119895 = 119887119895119894 = 0

Using these values in (1) results in the directionless sim-ilarity measure of 119886119894119895 = 1 Using the graph theory the adja-cency matrix ℎ119894 can be calculated as follows

1198671 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 1 1 1 1 1 0 0 0 0 0 01 1 0 1 0 0 0 1 1 1 0 0 01 1 1 0 0 0 0 0 0 0 1 1 10 1 1 1 0 1 1 0 0 0 0 0 00 1 1 1 1 0 1 0 0 0 0 0 00 1 1 1 1 1 0 0 0 0 0 0 00 1 1 1 0 0 0 0 1 1 0 0 00 1 1 1 0 0 0 1 0 1 0 0 00 1 1 1 0 0 0 1 1 0 0 0 00 1 1 1 0 0 0 0 0 0 0 1 10 1 1 1 0 0 0 0 0 0 1 0 10 1 1 1 0 0 0 0 0 0 1 1 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

Advances in Human-Computer Interaction 7

Web page

HTML

User

Directionless similarity

Directional similarity

Website designer

Website designer

Information structure without directional relationship

Mental model

User

Information structure with directional relationship

Card sorting

Path diagram

Figure 3 The process of calculating mental model similarity

Invention

Book

Diao ban yin shua

Huang dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yingda

Tang yanqianHistoricalcharacter

Tang dynasty

Book and historical character

Invention and historical character

Invention and Book

1

Invention

Book

Diao ban yin shua

Huan dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yinda

Tang yanqianHistorical character

Tang dynasty

2

Figure 4 An example of two different path diagrams (the segment with one arrow means one-way relationship which means node A canreach node B while node B cannot reach node A the segment with two arrows means that node A and node B can reach each other)

8 Advances in Human-Computer Interaction

1198672 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 0 0 1 1 1 0 0 0 0 0 01 0 0 0 0 0 0 1 1 1 0 0 01 0 0 0 0 0 0 0 0 0 1 1 10 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

(5)

Using these values with (2) and (3) the common andexclusive path vector can be calculated as follows

11988812 = 3 + 4 + 4 + 4 + 9 = 2411988912 = 0 + 2 + 2 + 2 + 36 = 42

(6)

Based on these values and (4) the directional similaritymeasure can be calculated as follows

11990312 = 2424 + 42 = 4

11 cong 036 (7)

The directionless similarity is 1 while the directionalsimilarity is 036The calculation results show that thementalmodel similarity was totally different whenwe considered thedirectional relationship of information structures

4 Results and Discussion

The following sections first examine the effects of the infor-mation structures on mental model similarity and thenexamine the relationship between mental model similarityand user performance Finally we examine whether themediation effect was found

41 The Influence of Information Structures on Mental ModelSimilarity Simplicitycomplexity generally refers to the levelof intricacy or detail in a stimulus [54 55] Specifically thedetail could be the number of closed figures open figuresletters horizontal lines vertical lines and so forth [54]

Complexity of web pages with different informationstructures consists of two high-level notions content com-plexity which is mainly measured through the number andtypes of objects to load a web page and service complexitywhich is mainly measured through ldquothe number and con-tributions of the various servers and administrative originsrdquo[56] Specifically if the interaction semantics and UI designare not considered the structure of website is the focus ofcomplexity It could be computed through a function which

Table 2 Complexity of three information structures

Net structure Treestructure

Linearstructure

Nodes 13 13 13Line segments 33 12 8

High (net) Medium (tree)Complexity of information structure

Low (linear)

Directional or notDirectionaless similarityDirectional similarity

04

06

08

10

Men

tal M

odel

Sim

ilarit

y

Figure 5 The influence of complexity of information structures onmental model similarity

calculated the number of outgoing links and controls such asbuttons and checkboxes [57]

Regarding these studies objective metrics of complexityof information structures were the number of nodes andlinks As shown in Table 2 the numbers of nodes of all threeinformation structures (ie net tree and linear) are the samebut the net structure has more directional links between anytwo nodes than the tree structure which in turn has morelinks than the linear structure Therefore complexity of theinformation structure has three levels termed low complexitymedium complexity and high complexity

Subjective metrics of complexity were consistent withresults of objective metrics Participants rated the simplicityof three websites on a 5-point Likert scale anchored fromldquoeasyrdquo to ldquocomplexrdquoThe average rating for web net tree andlinear was 41 (SD = 058) 31 (SD = 076) and 19 (SD = 094)

To further examine the influence of complexity of infor-mation structures on two different kinds of similarity one-way repeated ANOVA analysis was conducted As shown inFigure 5 the results indicated that complexity of the infor-mation structures had significant influence on directionalsimilarity (119865(262) = 5051 119901 lt 0001) Specifically resultsof multiple comparison with the Bonferroni correctionsindicated that the complex net structure resulted in widergap between the usersrsquo and designersrsquo mental models than thethree structure (119885 = 9081 119901 lt 0001) and the linear struc-ture (119885 = 8347 119901 lt 0001) The complexity of informationstructures resulted in nodifferences in directionless similarity(119865(262) = 3056 119901 = 00542) Therefore (H1a) was rejectedand (H1b) was supported

Advances in Human-Computer Interaction 9

Table 3 Linear regression results of mental model similarity on user performance

The task completion time The number of clicks119861 119905 119901 119861 119905 119901

Directionless similarity minus399 minus083 041 299 080 043Directional similarity 723 343 000 924 635 000

42 The Influence of Mental Model Similarity on User Per-formance Linear regression was conducted The dependentvariables were the task completion time and the number ofclicks and there were seven independent variables direc-tional similarity directionless similarity information struc-tures age technology product experience spatial ability andmemory capacity The results of regression analysis wereshown in Table 3

According to Table 3 the directional similarity waslinearly related to the task completion time Participants takeless time to find the target item hidden in the informationstructureswhen the directional similaritywas lowerThismayexplain the phenomenon that users who only remember thepaths to a special item of web pages take less time to find atarget in web pages than those who get a full understandingof elements and relationship of the web pages

Table 3 also indicates that the directional similarity waslinearly related to the number of clicks Participants clickedless to complete the tasks when the directional similarity waslower and thewebpages hadmore paths to obtain targets suchas net structure Although mental model similarity betweenthe participant and designer was lower in the net structurethan in the tree structure and the linear structure the numberof clicks was smaller with the lowest similarity One possiblereason for this is that the net-structure web page has manypaths and the elements are connected to each other while thetree structure and the linear structure are notThismeans thatusers cannot return to the previous page when they need toreach other pages in the net-structure web page unlike thetree structure and the linear structure

It can also be inferred that using path diagram to elicitmental models is more effective than using card sorting Itcan also verify that the directional relationship of informationstructures is important which cannot be ignored in elicitingmental models of information structures

The mental model similarity was calculated from theinformation structure Under each information structurehow does the mental model similarity affect user perfor-mance Correlation analysis and one-way ANOVA analysiswere conducted

The results indicated that only the directional similaritywas positively correlated with the task completion time in thenet-structureweb pageHowever either directional similarityor directionless similarity had no significant impact on userperformance under each information structure

In addition the user performance was not affected bytheir age spatial ability memory capacity or technologyproduct experience Moreover no matter what kind of infor-mation structure the user performance was still not affectedby demographic variables One possible reason is that the par-ticipants chosen for this study were all college students the

differences among them were too small In other words theselection of participants has limitations It may be differentwhen expanding the scope of participants especially to theelderly children and those with lower levels of education

43 Mediation Effects of Mental Model Similarity Accordingto MacKinnon et al [58] three modes were used to estimatethe basic intervening variable model which were shown inMod 1 Mod 2 and Mod 3

Mod 1 119884 = 119888119883 + 1198901 (8)

Mod 2 119884 = 119889119883 + 119887119872 + 1198902 (9)

Mod 3 119872 = 119886119883 + 1198903 (10)

In these equations 119883 is the independent variable 119884 isthe dependent variable and119872 is the intervening variable 11989011198902 and 1198903 are the population regression intercepts in (8) (9)and (10) respectively 119888 represents the relation between theindependent and dependent variables in (8) 119889 represents therelation between the independent and dependent variablesadjusted for the effects of the intervening variable in (9) 119886represents the relation between the independent and inter-vening variables in (10) and 119887 represents the relation betweenthe intervening and the dependent variables adjusted for theeffect of the independent variable in (9) [58] According toSobelrsquos [59] test the mediation effect was investigated Theresults of statistical analysis were shown in Tables 4 and 5

For both the number of clicks and the task completiontime the directional similarity was a significant mediatorHowever the effects are only partial mediation because thedirect effect is still significant (Tables 4 and 5) The simplicityof information structures had significant effects on the taskcompletion time and the number of clicks A part of the effectwas achieved by directional similarity which means that userperformance was partially influenced by the degree of matchbetween usersrsquo mental models and information structures

Tables 6 and 7 indicate that the directionless similaritywas not a mediator there was no significant mediation effectfor directionless similarity on task completion time or thenumber of clicks One possible reason is that the directionlesssimilarity was calculated from the card sorting in whichthe details about the directional relationship of informationstructures were not considered and this would lose theaccuracy when using card sorting to elicit mental models

Both the task completion time and the number of clicksshowed partial mediation by the directional similarity Theinformation structures had a direct effect on user perfor-mance The directionless similarity did not account for therelationship between information structures and user perfor-mance while the directional similarity as the new index to

10 Advances in Human-Computer Interaction

Table 4 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and taskcompletion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

171 069 247 92

Intercept 1385 177 782 936119890minus12Pred 177 083 215 345119890minus02

Mod 2Intercept 1186 188 631 104119890minus08Pred 007 104 007 947119890minus01Med 711 274 260 110eminus02

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 5 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and thenumber of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

123 045 273 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 002 121 002 099Pred 245 067 366 000Med 513 176 291 000

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 6 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe task completion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus028 023 minus119 92

Intercept 1385 177 782 936119890minus12Pred 177 083 213 345119890minus02

Mod 2Intercept 1971 456 432 405119890minus05Pred 205 085 241 174119890minus02Med minus673 483 minus139 167eminus01

Mod 3Intercept 087 004 2175 539119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in Human-Computer Interaction 3

The directional similarity is calculated from path dia-gram which is used to elicit mental models of informationstructures with directional relationship We add more details(eg directional relationship) to mental models Based onthis we proposed Hypothesis (H1b)

(H1b) The more complex the information structures thelower the directional similarity

23 Mental Models and User Performance Many previousstudies have shown that mental models are correlated withuser performance On the one hand mental models havea positive effect on user performance Ziefle and Bay [33]pointed out that the better the mental models of navigationsmenus the better the performance using the devices Young[34] thought that mental models could explain user perfor-mance with the systems with which they interact Dimitroff[35] found that students with more complete mental modelsmade significantly fewer errorswhen they used theUniversityof Michiganrsquos website Sasse [36] noted the significant effectsofmentalmodels on user performance using Excel Slone [37]found that usersrsquo mental models affected their performanceon websites Brandt and Uden [38] pointed out that noviceswithout strongmentalmodels for information retrieval couldnot gather information successfully

Numerous studies have also shown the relationshipbetween TMMs and team performance For example Math-ieu et al [24] considered shared mental models as twocategories task and team finding that both team-basedand task-based mental models related positively to subse-quent team process and performance Mathieu et al [39]demonstrated in a PC-based flight simulator that both taskand team models had an impact on performance the teamprocess was supported by shared mental models and task-work mental model similarity but not by teamwork mentalmodel similarity which was significantly related to both teamprocesses and team performance

On the other hand workers have found that mentalmodels had either an adverse effect or no significant effect onuser performance Halasz and Moran [40] and Borgman [41]found that whether users formed amental model of a systemor not they performed no differently on routine simple tasksNorman [3] found that users with wrongincomplete mentalmodels could use technology products successfully Payne[42] noted that evenwrongmentalmodels did not necessarilyresult in the bad usage of devices Schmettow and Sommer[43] found that the degree of match between the mentalmodel and website structure had no effect on usersrsquo browsingperformance As for TMMs Webber et al [44] found thatteam members sharing a common mental model with poorquality did not likely perform well

It is obvious that the research consequences were con-tradictory It seems that researchers cannot get the unifiedcognition of the effects of mental models on user perfor-mance One possible reason is that most researchers do theirstudies without consideringmental model similarity betweenusers and designers It is necessary for us to investigatemental modelsrsquo effects on interaction performance involvingthe mental model similarity The only study of mental model

similarity on interaction performance is seen in Schmettowand Sommer [43] They found that mental model similarityhad no effect on interaction performance However the waythey elicitedmental models was card sorting and they did notconsider the directional relationship of information struc-tures In this paper we considered more details such as direc-tional relationship to elicit a mental model of informationstructure and get the directional similarity frompath diagramand the directionless similarity from card sorting whichmay get different results from what Schmettow and Sommer[43] found Considering that there are more positive effectsthan adverse effects in the existing studies in this paper weare partial to supporting the positive effects Therefore weproposed Hypothesis (H2a) and Hypothesis (H2b)

(H2a) Directional similarity predicts the task completiontime the higher the directional similarity the less thetask completion time

(H2b) Directional similarity predicts the number of clicksthe higher the directional similarity the less the clicktimes

The effects of information structures on user performancehave been widely investigated since mental models areclosely related to usersrsquo behavior using various devices a goodmental model of information structure will likely enhanceuser performance However there are many other factorsinfluencing user performance besides mental models forexample usersrsquo age and gender Ziefle and Bay [33] pointedout that younger users were more effective in using a cellphone menu than the older users Mathieu et al [24] foundthat team processes fully mediated the relationship betweenteam mental models and team effectiveness Mathieu et al[39] found that team performance was partially mediatedby teammatesrsquo mental models Based on these findingswe have assumed that mental model similarity mediatedthe relationship between information structures and userperformance which is Hypothesis (H3a) and Hypothesis(H3b) (see Figure 1)

(H3a) Directionless similarity will mediate the relationshipbetween information structures and user perfor-mance

(H3b) Directional similarity will mediate the relationshipbetween information structures and user perfor-mance

24 Quantitative Methods of Mental Model Similarity Elic-ited mental models are widely used in two ways First thedifference between elicited mental models can be qualita-tively and quantitatively compared One common quanti-tative method is to compare the depth and width of theelicited structures Users use an interface with a well-definedinformation structure such as phone menus and web pages

Another quantitative method is to calculate the teammental model (TMM) similarity score which indicates thepercentage of team membersrsquo shared organized under-standing and mental representation of knowledge or beliefsrelevant to the key content of the teamrsquos task [22 23]

4 Advances in Human-Computer Interaction

Mental model similarity(directional similarity directionless similarity)

Information structure User performance

Figure 1 The mediation effect of mental model similarity in the relation between information structure and user performance

Specifically team members usually rated the relatedness ofpairs of statements describing team interaction processesand characteristics of team members on the Likert scaleThe response could be analyzed using multidimensionaltechniques (eg the Pathfinder technique) to calculate thesimilarity score [26 45 46]

However none of these methods can accurately elicitmental models of information structures with directionalrelationship also these methods cannot quantitatively cal-culate the degree of similarity or dissimilarity of the mentalmodels between users and designers of systems Previousstudies [43] have shown that the widely usedmethod of elicit-ing mental models of hierarchical systems is card sorting butit is inadequate to elicit mental models without consideringthe directional relationship of information structures Alsothe previous studies did not introduce a suitable method toquantify the mental model similarity Wu and Liu [47] pro-posed a new computational modeling approach which wascomposed of a simulation model of a queuing network archi-tecture and a set of mathematical equations implemented inthe simulation model to quantify mental workload to modelthe mental workload in drive and driving performanceHowever the approach proposed was not the quantitativemethod for calculating the mental model similarity It wasapplied to analyze themental workload in driver informationsystem Thus here we elicit mental models through cardsorting and path diagramThen we conducted a newmethodto quantify the degree of mental model similarity

3 Methodology

31 Equipment and Materials A notebook computer with atouch screen (ThinkPad YogaS1) was used to present webpages and a whiteboard was used to draw path diagrams Acamera (Sony HDR-PJ610E) was used to record participantsrsquoresults of card sorting and drawing the path diagrams AMorae Recorder was used for counting task completion timeand the number of clicks andwas also used to present the taskspecification for the participants Web pages with differentinformation structures were developed The web pages werefirst considered about the navigation structures which werenet tree and linear The depth of tree and net structure wasthree levels which was seen common in daily web pagesThewidth of the bottom level of tree and net structure was nineitems which was also seen in common web pages

To avoid the effects of familiar knowledge topics aboutancient inventions ancient books and ancient historical

Table 1 Experience of using technology products of participants

Mean SDLaptop 075 114Tablets 495 280Smart phones 458 342

characters of different Chinese dynasties which were notwidely known were presented in web pages as the contentThere were totally nine different Chinese dynasties and eachinformation structure was made of three different Chinesedynasties Ancient inventions ancient books and ancienthistorical characters were corresponding to their own dynas-ties Two pretests were carried out by two college studentsand the interaction forms and content layout were adjustedaccording to the resultsThememory capacity that influencesuser performance [33] was tested by a KJ-I spatial locationmemory span tester Spatial ability was tested through thepaper folding test [48] The original paper folding test wastranslated into Chinese

32 Participants A total of 32 students from ChongqingUniversity were recruited The age of the participants rangedfrom 21 to 26 years (mean = 2313 SD = 17) The gender wasbalancedThe experience of using laptops tablets and smart-phones (see Table 1) was investigated the results indicatedthat handheld devices (ie smartphones and tablets) wereused intensively

33 Task All participants completed tasks in three differentweb pages with different information structures net treeand linear To avoid learning effects and make participantsget a full understanding of the information structures of webpages each participant completed eight tasks in a web pageThe order of web pages in which participants completed thetasks is random

The tasks are searching tasks Participants found a targetand read its content The target is an item hidden in webpages which is not widely known for participants To checkwhether they read the content carefully a single-choice testwas conducted Participants wrote down answers on a pieceof paperThe test has two questions one is aboutwhat dynastythe target belonged to and the other one was about which thetarget is about Taking ldquoHuang dao you yirdquo as an examplethe task is ldquoPlease find ldquoHuang dao you yirdquo and read itsdescription and then answer the following two questions

Advances in Human-Computer Interaction 5

Q1 which dynasty ldquoHuang dao you yirdquo belongs to Q2 whatldquoHuang dao you yirdquo is aboutrdquo

34 Dependent Variables The two dependent variables weretask completion time and the number of clicks Task com-pletion time was the average of eight tasksrsquo completion timeunder each information structure and the number of clickswas the average of the click times to complete all of the eighttasksThey were both measured by the Morae Recorder onlywhen the participant obtained the correct answers for bothquestions were the missions completed

35 Independent Variable The independent variables werethe directionless similarity and directional similarity It is anew quantitative measure proposed here (see Section 37)the directionless similarity was calculated from card sortingand the directional similarity was calculated from pathdiagramThemental model similarity was computed for eachinformation structure

Demographic variables included age experience of usingtechnology product spatial ability and memory capacity Aquestionnaire was used to collect basic information aboutexperience of using technology product

The mental model similarity was a between-subject vari-able while the information structure was a within-subjectvariableThat is each participant used three prototypes In aneffort to avoid learning effects the order of using prototypeswas random

36 Procedure The experiment took each participant aboutone hour to complete It consisted of four tests a question-naire test a memory test a paper folding test and a cardsorting test

First each participant began the experiment by filling outa consent form and a general questionnaire about hisherdemographic information and using experience with tech-nology products

Secondly a memory test was conducted using the KJ-I spatial position memory span tester After completing thememory test the test scores were recorded on paper and werenot disclosed to the participants

Thirdly a paper folding test was conducted to test thespatial ability of the participants This test is divided into twoparts The time limit of each part was three minutes to avoidparticipants losing their patience The participants needed tofind the correct answer independently The final score of thetest is the correct number minus the incorrect number on thetest paperThe higher the score the better the spatial memoryability

Fourthly a brief introduction and practice about theexperiment were given to each participant Finally partici-pants completed tasks on each web page and then went onwith the card sorting The cards were the titles of each nodein the information structures Then the participants wererequired to draw a path structure of the experimental webpage navigation with a whiteboard stroke During the wholeprocess participants were left alone and questions related tothe path were not answered in a relevant way which aimed atavoiding the subjective impacts of the experimental designer

At the end the experimenter conducted a five-minute explor-atory interview with the participants to understand theirthoughts and feelings about using the three web pages Thequestions in the interview included ldquoQ1 Please score thethree web pages in this experiment considering informationsearching ease of use and user experience Q2 Pleasesequence the three web pages according to your experiencerdquo

37 Quantifying Mental Model Similarity The method pro-posed to quantify mental model similarity consists of twoparts one is the method which can be used to elicit mentalmodels of information structures with directional relation-ship and the other one is the mathematical equations whichcan be used to calculate the mental model similarity

The first part is the method of eliciting mental models Inorder to quantify mental model similarity a method whichcan be used to elicit mental models with more details such asdirectional relationship has to be used Such amethod shouldrepresent the understanding of elements and directionalrelationship of a hierarchical system which are the keyfactors in elicitingmental model of a hierarchical systemTheliterature provides several methods to elicit mental modelssuch as card sorting which is widely used in eliciting mentalmodels of hierarchical systems [18 33] However card sortingcannot elicit completemental models of hierarchical systemsparticularly the directional aspects of hierarchies

A new method is needed to reflect the directionalinformation of mental models of hierarchy structures Webnavigation is similar to the real-world navigation In real-world navigation people usually take three strategies tofind a destination They remember properties of landmarkssuch as shape and structure (ie landmark knowledge) [4950] or the sequential order of landmarks encountered anddirectional relationship between these landmarks (ie routeknowledge) [51] or an overview of the environment likea map showing spatial relationships between routes andlandmarks (ie survey knowledge) [52] to find a destinationThis knowledge is also involved in web navigation Land-mark knowledge is mainly represented through card sortingInspired by route knowledge we proposed the path diagramto elicit mental models of hierarchical systems with moredetails (eg directional relationship)

The second part is about quantifying the mental modelsimilarity The limited research in quantifying mental modelsimilarity was reported by Sinreich et al [53] They intro-duced process chart into eliciting mental models of Emer-gency Department Management and quantified similaritythrough four formulae However themethodwas not appliedto analyze the mental models of information structures

The method of quantifying the degree of mental modelsimilarity aims to reflect the extent to usersrsquo understandingof the system it can present causality and logic In additionthis method should be easy to understand so that it can bemastered by nonprofessional persons Thus in this study weextended the work of Sinreich et al [53] as our proposedquantitative method We calculated mental model similarityin twoways calculating the directionless similarity from cardsorting and calculating the directional similarity from pathdiagram

6 Advances in Human-Computer Interaction

Invention Book

View 1 View 2

Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Invention

BookHistoricalcharacter

Diao ban yinshua

Shui yun hun yi Huang dao you

yi

Page name Invention

HomepageBook Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Page name Invention

HomepageInvention

Shui yun hun yiDiao ban yin shua Huang dao you yi

Diao ban yin shua Shui yun hun yi Huang dao you yi

Ji li gu che Zhuan lun cang Lian hua lou

Chai se tao yin Jian yi Di qiu yi

Figure 2 Two views of web page interfaces in this experiment

The first component represents the elements (nodes) ofthe different content for example ldquoTang dynastyrdquo ldquoInven-tionrdquo ldquoBookrdquo and ldquoHistorical characterrdquo in Figure 2 Thedirectionless similarity measure 119886119894119895 can be obtained using

119886119894119895 = 119890119894119895119890119894119895 + 119887119894119895 + 119887119895119894 (1)

where 119890119894119895 denotes the number of identical elements in cardsortings 119894 and 119895 and 119887119894119895 denotes the number of elements thatexist in 119894 that do not exist in 119895 (see that 119887119894119895 = 119887119895119894) It isclear that 0 ≪ 119886119894119895 ≪ 1 In the case that both card sortingsare identical in terms of their elements (not necessarily theirrelationships) then we have 119886119894119895 = 1 while 119886119894119895 = 0 if nocommon elements exist and by definition 119886119894119895 = 119886119895119894

The second component represents the relationshipbetween elements (arcs) in the path diagram A relationshipis defined by the elements it connects (there may be morethan one connection between elements) and by the directionof the connecting arc for example the arcs that connectelements ldquoTang dynasty-Inventionrdquo ldquoInvention-Bookrdquoand ldquoInvention-Diao ban yin shuardquo in path diagram A inFigure 4The first step in calculating the directional similarityis to obtain the adjacency matricesThe element of adjacencymatrix was the numbers of directed segment between twonodes (eg if there were one directed segment from ldquoTangdynastyrdquo to ldquoInventionrdquo the element is 1)

Based on the adjacency matrix the sum of all the com-mon arcs 119888119894119895 and the sum of all exclusive arcs 119889119894119895 betweenany two path diagrams 119894 and 119895 can be calculated as shownin (2) and (3) respectively Finally the directional similaritymeasure 119903119894119895 can be obtained using (4)

119888119894119895 = sum119896

sum119897

min ℎ119894119896119897 ℎ119895119896119897 (2)

119889119894119895 = sum119896

sum119897

10038161003816100381610038161003816ℎ119894119896119897 minus ℎ119895119896119897

10038161003816100381610038161003816 (3)

119903119894119895 = 119888119894119895119888119894119895 + 119889119894119895 (4)

It is clear that 0 ≪ 119903119894119895 ≪ 1 In the case that both pathdiagrams are identical in terms of their relationship (arcs)then we have 119903119894119895 = 1 while 119903119894119895 = 0 if no common relationshipexists between the two path diagrams By definition 119903119894119895 = 119903119895119894

These formulae adapted from Sinreich et al [53] arevalidated and used to calculate the similarity between processcharts In this study we used them to analyze hierarchiesand extended their work by adding directions To analyzedirectional relationship an adjacency matrix was used toindicate relationship according to the graph theory Thenrelationship similarity of hierarchical systems was calculatedby using formulas (2) (3) and (4)

The process of calculating the mental model similaritycan be seen in Figure 3

In order to illustrate the calculation procedure of thesimilarity measure the following example is given (seeFigure 4)

From Figure 4 the following values are obtained 119890119894119895 = 13and 119887119894119895 = 119887119895119894 = 0

Using these values in (1) results in the directionless sim-ilarity measure of 119886119894119895 = 1 Using the graph theory the adja-cency matrix ℎ119894 can be calculated as follows

1198671 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 1 1 1 1 1 0 0 0 0 0 01 1 0 1 0 0 0 1 1 1 0 0 01 1 1 0 0 0 0 0 0 0 1 1 10 1 1 1 0 1 1 0 0 0 0 0 00 1 1 1 1 0 1 0 0 0 0 0 00 1 1 1 1 1 0 0 0 0 0 0 00 1 1 1 0 0 0 0 1 1 0 0 00 1 1 1 0 0 0 1 0 1 0 0 00 1 1 1 0 0 0 1 1 0 0 0 00 1 1 1 0 0 0 0 0 0 0 1 10 1 1 1 0 0 0 0 0 0 1 0 10 1 1 1 0 0 0 0 0 0 1 1 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

Advances in Human-Computer Interaction 7

Web page

HTML

User

Directionless similarity

Directional similarity

Website designer

Website designer

Information structure without directional relationship

Mental model

User

Information structure with directional relationship

Card sorting

Path diagram

Figure 3 The process of calculating mental model similarity

Invention

Book

Diao ban yin shua

Huang dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yingda

Tang yanqianHistoricalcharacter

Tang dynasty

Book and historical character

Invention and historical character

Invention and Book

1

Invention

Book

Diao ban yin shua

Huan dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yinda

Tang yanqianHistorical character

Tang dynasty

2

Figure 4 An example of two different path diagrams (the segment with one arrow means one-way relationship which means node A canreach node B while node B cannot reach node A the segment with two arrows means that node A and node B can reach each other)

8 Advances in Human-Computer Interaction

1198672 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 0 0 1 1 1 0 0 0 0 0 01 0 0 0 0 0 0 1 1 1 0 0 01 0 0 0 0 0 0 0 0 0 1 1 10 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

(5)

Using these values with (2) and (3) the common andexclusive path vector can be calculated as follows

11988812 = 3 + 4 + 4 + 4 + 9 = 2411988912 = 0 + 2 + 2 + 2 + 36 = 42

(6)

Based on these values and (4) the directional similaritymeasure can be calculated as follows

11990312 = 2424 + 42 = 4

11 cong 036 (7)

The directionless similarity is 1 while the directionalsimilarity is 036The calculation results show that thementalmodel similarity was totally different whenwe considered thedirectional relationship of information structures

4 Results and Discussion

The following sections first examine the effects of the infor-mation structures on mental model similarity and thenexamine the relationship between mental model similarityand user performance Finally we examine whether themediation effect was found

41 The Influence of Information Structures on Mental ModelSimilarity Simplicitycomplexity generally refers to the levelof intricacy or detail in a stimulus [54 55] Specifically thedetail could be the number of closed figures open figuresletters horizontal lines vertical lines and so forth [54]

Complexity of web pages with different informationstructures consists of two high-level notions content com-plexity which is mainly measured through the number andtypes of objects to load a web page and service complexitywhich is mainly measured through ldquothe number and con-tributions of the various servers and administrative originsrdquo[56] Specifically if the interaction semantics and UI designare not considered the structure of website is the focus ofcomplexity It could be computed through a function which

Table 2 Complexity of three information structures

Net structure Treestructure

Linearstructure

Nodes 13 13 13Line segments 33 12 8

High (net) Medium (tree)Complexity of information structure

Low (linear)

Directional or notDirectionaless similarityDirectional similarity

04

06

08

10

Men

tal M

odel

Sim

ilarit

y

Figure 5 The influence of complexity of information structures onmental model similarity

calculated the number of outgoing links and controls such asbuttons and checkboxes [57]

Regarding these studies objective metrics of complexityof information structures were the number of nodes andlinks As shown in Table 2 the numbers of nodes of all threeinformation structures (ie net tree and linear) are the samebut the net structure has more directional links between anytwo nodes than the tree structure which in turn has morelinks than the linear structure Therefore complexity of theinformation structure has three levels termed low complexitymedium complexity and high complexity

Subjective metrics of complexity were consistent withresults of objective metrics Participants rated the simplicityof three websites on a 5-point Likert scale anchored fromldquoeasyrdquo to ldquocomplexrdquoThe average rating for web net tree andlinear was 41 (SD = 058) 31 (SD = 076) and 19 (SD = 094)

To further examine the influence of complexity of infor-mation structures on two different kinds of similarity one-way repeated ANOVA analysis was conducted As shown inFigure 5 the results indicated that complexity of the infor-mation structures had significant influence on directionalsimilarity (119865(262) = 5051 119901 lt 0001) Specifically resultsof multiple comparison with the Bonferroni correctionsindicated that the complex net structure resulted in widergap between the usersrsquo and designersrsquo mental models than thethree structure (119885 = 9081 119901 lt 0001) and the linear struc-ture (119885 = 8347 119901 lt 0001) The complexity of informationstructures resulted in nodifferences in directionless similarity(119865(262) = 3056 119901 = 00542) Therefore (H1a) was rejectedand (H1b) was supported

Advances in Human-Computer Interaction 9

Table 3 Linear regression results of mental model similarity on user performance

The task completion time The number of clicks119861 119905 119901 119861 119905 119901

Directionless similarity minus399 minus083 041 299 080 043Directional similarity 723 343 000 924 635 000

42 The Influence of Mental Model Similarity on User Per-formance Linear regression was conducted The dependentvariables were the task completion time and the number ofclicks and there were seven independent variables direc-tional similarity directionless similarity information struc-tures age technology product experience spatial ability andmemory capacity The results of regression analysis wereshown in Table 3

According to Table 3 the directional similarity waslinearly related to the task completion time Participants takeless time to find the target item hidden in the informationstructureswhen the directional similaritywas lowerThismayexplain the phenomenon that users who only remember thepaths to a special item of web pages take less time to find atarget in web pages than those who get a full understandingof elements and relationship of the web pages

Table 3 also indicates that the directional similarity waslinearly related to the number of clicks Participants clickedless to complete the tasks when the directional similarity waslower and thewebpages hadmore paths to obtain targets suchas net structure Although mental model similarity betweenthe participant and designer was lower in the net structurethan in the tree structure and the linear structure the numberof clicks was smaller with the lowest similarity One possiblereason for this is that the net-structure web page has manypaths and the elements are connected to each other while thetree structure and the linear structure are notThismeans thatusers cannot return to the previous page when they need toreach other pages in the net-structure web page unlike thetree structure and the linear structure

It can also be inferred that using path diagram to elicitmental models is more effective than using card sorting Itcan also verify that the directional relationship of informationstructures is important which cannot be ignored in elicitingmental models of information structures

The mental model similarity was calculated from theinformation structure Under each information structurehow does the mental model similarity affect user perfor-mance Correlation analysis and one-way ANOVA analysiswere conducted

The results indicated that only the directional similaritywas positively correlated with the task completion time in thenet-structureweb pageHowever either directional similarityor directionless similarity had no significant impact on userperformance under each information structure

In addition the user performance was not affected bytheir age spatial ability memory capacity or technologyproduct experience Moreover no matter what kind of infor-mation structure the user performance was still not affectedby demographic variables One possible reason is that the par-ticipants chosen for this study were all college students the

differences among them were too small In other words theselection of participants has limitations It may be differentwhen expanding the scope of participants especially to theelderly children and those with lower levels of education

43 Mediation Effects of Mental Model Similarity Accordingto MacKinnon et al [58] three modes were used to estimatethe basic intervening variable model which were shown inMod 1 Mod 2 and Mod 3

Mod 1 119884 = 119888119883 + 1198901 (8)

Mod 2 119884 = 119889119883 + 119887119872 + 1198902 (9)

Mod 3 119872 = 119886119883 + 1198903 (10)

In these equations 119883 is the independent variable 119884 isthe dependent variable and119872 is the intervening variable 11989011198902 and 1198903 are the population regression intercepts in (8) (9)and (10) respectively 119888 represents the relation between theindependent and dependent variables in (8) 119889 represents therelation between the independent and dependent variablesadjusted for the effects of the intervening variable in (9) 119886represents the relation between the independent and inter-vening variables in (10) and 119887 represents the relation betweenthe intervening and the dependent variables adjusted for theeffect of the independent variable in (9) [58] According toSobelrsquos [59] test the mediation effect was investigated Theresults of statistical analysis were shown in Tables 4 and 5

For both the number of clicks and the task completiontime the directional similarity was a significant mediatorHowever the effects are only partial mediation because thedirect effect is still significant (Tables 4 and 5) The simplicityof information structures had significant effects on the taskcompletion time and the number of clicks A part of the effectwas achieved by directional similarity which means that userperformance was partially influenced by the degree of matchbetween usersrsquo mental models and information structures

Tables 6 and 7 indicate that the directionless similaritywas not a mediator there was no significant mediation effectfor directionless similarity on task completion time or thenumber of clicks One possible reason is that the directionlesssimilarity was calculated from the card sorting in whichthe details about the directional relationship of informationstructures were not considered and this would lose theaccuracy when using card sorting to elicit mental models

Both the task completion time and the number of clicksshowed partial mediation by the directional similarity Theinformation structures had a direct effect on user perfor-mance The directionless similarity did not account for therelationship between information structures and user perfor-mance while the directional similarity as the new index to

10 Advances in Human-Computer Interaction

Table 4 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and taskcompletion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

171 069 247 92

Intercept 1385 177 782 936119890minus12Pred 177 083 215 345119890minus02

Mod 2Intercept 1186 188 631 104119890minus08Pred 007 104 007 947119890minus01Med 711 274 260 110eminus02

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 5 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and thenumber of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

123 045 273 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 002 121 002 099Pred 245 067 366 000Med 513 176 291 000

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 6 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe task completion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus028 023 minus119 92

Intercept 1385 177 782 936119890minus12Pred 177 083 213 345119890minus02

Mod 2Intercept 1971 456 432 405119890minus05Pred 205 085 241 174119890minus02Med minus673 483 minus139 167eminus01

Mod 3Intercept 087 004 2175 539119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

4 Advances in Human-Computer Interaction

Mental model similarity(directional similarity directionless similarity)

Information structure User performance

Figure 1 The mediation effect of mental model similarity in the relation between information structure and user performance

Specifically team members usually rated the relatedness ofpairs of statements describing team interaction processesand characteristics of team members on the Likert scaleThe response could be analyzed using multidimensionaltechniques (eg the Pathfinder technique) to calculate thesimilarity score [26 45 46]

However none of these methods can accurately elicitmental models of information structures with directionalrelationship also these methods cannot quantitatively cal-culate the degree of similarity or dissimilarity of the mentalmodels between users and designers of systems Previousstudies [43] have shown that the widely usedmethod of elicit-ing mental models of hierarchical systems is card sorting butit is inadequate to elicit mental models without consideringthe directional relationship of information structures Alsothe previous studies did not introduce a suitable method toquantify the mental model similarity Wu and Liu [47] pro-posed a new computational modeling approach which wascomposed of a simulation model of a queuing network archi-tecture and a set of mathematical equations implemented inthe simulation model to quantify mental workload to modelthe mental workload in drive and driving performanceHowever the approach proposed was not the quantitativemethod for calculating the mental model similarity It wasapplied to analyze themental workload in driver informationsystem Thus here we elicit mental models through cardsorting and path diagramThen we conducted a newmethodto quantify the degree of mental model similarity

3 Methodology

31 Equipment and Materials A notebook computer with atouch screen (ThinkPad YogaS1) was used to present webpages and a whiteboard was used to draw path diagrams Acamera (Sony HDR-PJ610E) was used to record participantsrsquoresults of card sorting and drawing the path diagrams AMorae Recorder was used for counting task completion timeand the number of clicks andwas also used to present the taskspecification for the participants Web pages with differentinformation structures were developed The web pages werefirst considered about the navigation structures which werenet tree and linear The depth of tree and net structure wasthree levels which was seen common in daily web pagesThewidth of the bottom level of tree and net structure was nineitems which was also seen in common web pages

To avoid the effects of familiar knowledge topics aboutancient inventions ancient books and ancient historical

Table 1 Experience of using technology products of participants

Mean SDLaptop 075 114Tablets 495 280Smart phones 458 342

characters of different Chinese dynasties which were notwidely known were presented in web pages as the contentThere were totally nine different Chinese dynasties and eachinformation structure was made of three different Chinesedynasties Ancient inventions ancient books and ancienthistorical characters were corresponding to their own dynas-ties Two pretests were carried out by two college studentsand the interaction forms and content layout were adjustedaccording to the resultsThememory capacity that influencesuser performance [33] was tested by a KJ-I spatial locationmemory span tester Spatial ability was tested through thepaper folding test [48] The original paper folding test wastranslated into Chinese

32 Participants A total of 32 students from ChongqingUniversity were recruited The age of the participants rangedfrom 21 to 26 years (mean = 2313 SD = 17) The gender wasbalancedThe experience of using laptops tablets and smart-phones (see Table 1) was investigated the results indicatedthat handheld devices (ie smartphones and tablets) wereused intensively

33 Task All participants completed tasks in three differentweb pages with different information structures net treeand linear To avoid learning effects and make participantsget a full understanding of the information structures of webpages each participant completed eight tasks in a web pageThe order of web pages in which participants completed thetasks is random

The tasks are searching tasks Participants found a targetand read its content The target is an item hidden in webpages which is not widely known for participants To checkwhether they read the content carefully a single-choice testwas conducted Participants wrote down answers on a pieceof paperThe test has two questions one is aboutwhat dynastythe target belonged to and the other one was about which thetarget is about Taking ldquoHuang dao you yirdquo as an examplethe task is ldquoPlease find ldquoHuang dao you yirdquo and read itsdescription and then answer the following two questions

Advances in Human-Computer Interaction 5

Q1 which dynasty ldquoHuang dao you yirdquo belongs to Q2 whatldquoHuang dao you yirdquo is aboutrdquo

34 Dependent Variables The two dependent variables weretask completion time and the number of clicks Task com-pletion time was the average of eight tasksrsquo completion timeunder each information structure and the number of clickswas the average of the click times to complete all of the eighttasksThey were both measured by the Morae Recorder onlywhen the participant obtained the correct answers for bothquestions were the missions completed

35 Independent Variable The independent variables werethe directionless similarity and directional similarity It is anew quantitative measure proposed here (see Section 37)the directionless similarity was calculated from card sortingand the directional similarity was calculated from pathdiagramThemental model similarity was computed for eachinformation structure

Demographic variables included age experience of usingtechnology product spatial ability and memory capacity Aquestionnaire was used to collect basic information aboutexperience of using technology product

The mental model similarity was a between-subject vari-able while the information structure was a within-subjectvariableThat is each participant used three prototypes In aneffort to avoid learning effects the order of using prototypeswas random

36 Procedure The experiment took each participant aboutone hour to complete It consisted of four tests a question-naire test a memory test a paper folding test and a cardsorting test

First each participant began the experiment by filling outa consent form and a general questionnaire about hisherdemographic information and using experience with tech-nology products

Secondly a memory test was conducted using the KJ-I spatial position memory span tester After completing thememory test the test scores were recorded on paper and werenot disclosed to the participants

Thirdly a paper folding test was conducted to test thespatial ability of the participants This test is divided into twoparts The time limit of each part was three minutes to avoidparticipants losing their patience The participants needed tofind the correct answer independently The final score of thetest is the correct number minus the incorrect number on thetest paperThe higher the score the better the spatial memoryability

Fourthly a brief introduction and practice about theexperiment were given to each participant Finally partici-pants completed tasks on each web page and then went onwith the card sorting The cards were the titles of each nodein the information structures Then the participants wererequired to draw a path structure of the experimental webpage navigation with a whiteboard stroke During the wholeprocess participants were left alone and questions related tothe path were not answered in a relevant way which aimed atavoiding the subjective impacts of the experimental designer

At the end the experimenter conducted a five-minute explor-atory interview with the participants to understand theirthoughts and feelings about using the three web pages Thequestions in the interview included ldquoQ1 Please score thethree web pages in this experiment considering informationsearching ease of use and user experience Q2 Pleasesequence the three web pages according to your experiencerdquo

37 Quantifying Mental Model Similarity The method pro-posed to quantify mental model similarity consists of twoparts one is the method which can be used to elicit mentalmodels of information structures with directional relation-ship and the other one is the mathematical equations whichcan be used to calculate the mental model similarity

The first part is the method of eliciting mental models Inorder to quantify mental model similarity a method whichcan be used to elicit mental models with more details such asdirectional relationship has to be used Such amethod shouldrepresent the understanding of elements and directionalrelationship of a hierarchical system which are the keyfactors in elicitingmental model of a hierarchical systemTheliterature provides several methods to elicit mental modelssuch as card sorting which is widely used in eliciting mentalmodels of hierarchical systems [18 33] However card sortingcannot elicit completemental models of hierarchical systemsparticularly the directional aspects of hierarchies

A new method is needed to reflect the directionalinformation of mental models of hierarchy structures Webnavigation is similar to the real-world navigation In real-world navigation people usually take three strategies tofind a destination They remember properties of landmarkssuch as shape and structure (ie landmark knowledge) [4950] or the sequential order of landmarks encountered anddirectional relationship between these landmarks (ie routeknowledge) [51] or an overview of the environment likea map showing spatial relationships between routes andlandmarks (ie survey knowledge) [52] to find a destinationThis knowledge is also involved in web navigation Land-mark knowledge is mainly represented through card sortingInspired by route knowledge we proposed the path diagramto elicit mental models of hierarchical systems with moredetails (eg directional relationship)

The second part is about quantifying the mental modelsimilarity The limited research in quantifying mental modelsimilarity was reported by Sinreich et al [53] They intro-duced process chart into eliciting mental models of Emer-gency Department Management and quantified similaritythrough four formulae However themethodwas not appliedto analyze the mental models of information structures

The method of quantifying the degree of mental modelsimilarity aims to reflect the extent to usersrsquo understandingof the system it can present causality and logic In additionthis method should be easy to understand so that it can bemastered by nonprofessional persons Thus in this study weextended the work of Sinreich et al [53] as our proposedquantitative method We calculated mental model similarityin twoways calculating the directionless similarity from cardsorting and calculating the directional similarity from pathdiagram

6 Advances in Human-Computer Interaction

Invention Book

View 1 View 2

Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Invention

BookHistoricalcharacter

Diao ban yinshua

Shui yun hun yi Huang dao you

yi

Page name Invention

HomepageBook Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Page name Invention

HomepageInvention

Shui yun hun yiDiao ban yin shua Huang dao you yi

Diao ban yin shua Shui yun hun yi Huang dao you yi

Ji li gu che Zhuan lun cang Lian hua lou

Chai se tao yin Jian yi Di qiu yi

Figure 2 Two views of web page interfaces in this experiment

The first component represents the elements (nodes) ofthe different content for example ldquoTang dynastyrdquo ldquoInven-tionrdquo ldquoBookrdquo and ldquoHistorical characterrdquo in Figure 2 Thedirectionless similarity measure 119886119894119895 can be obtained using

119886119894119895 = 119890119894119895119890119894119895 + 119887119894119895 + 119887119895119894 (1)

where 119890119894119895 denotes the number of identical elements in cardsortings 119894 and 119895 and 119887119894119895 denotes the number of elements thatexist in 119894 that do not exist in 119895 (see that 119887119894119895 = 119887119895119894) It isclear that 0 ≪ 119886119894119895 ≪ 1 In the case that both card sortingsare identical in terms of their elements (not necessarily theirrelationships) then we have 119886119894119895 = 1 while 119886119894119895 = 0 if nocommon elements exist and by definition 119886119894119895 = 119886119895119894

The second component represents the relationshipbetween elements (arcs) in the path diagram A relationshipis defined by the elements it connects (there may be morethan one connection between elements) and by the directionof the connecting arc for example the arcs that connectelements ldquoTang dynasty-Inventionrdquo ldquoInvention-Bookrdquoand ldquoInvention-Diao ban yin shuardquo in path diagram A inFigure 4The first step in calculating the directional similarityis to obtain the adjacency matricesThe element of adjacencymatrix was the numbers of directed segment between twonodes (eg if there were one directed segment from ldquoTangdynastyrdquo to ldquoInventionrdquo the element is 1)

Based on the adjacency matrix the sum of all the com-mon arcs 119888119894119895 and the sum of all exclusive arcs 119889119894119895 betweenany two path diagrams 119894 and 119895 can be calculated as shownin (2) and (3) respectively Finally the directional similaritymeasure 119903119894119895 can be obtained using (4)

119888119894119895 = sum119896

sum119897

min ℎ119894119896119897 ℎ119895119896119897 (2)

119889119894119895 = sum119896

sum119897

10038161003816100381610038161003816ℎ119894119896119897 minus ℎ119895119896119897

10038161003816100381610038161003816 (3)

119903119894119895 = 119888119894119895119888119894119895 + 119889119894119895 (4)

It is clear that 0 ≪ 119903119894119895 ≪ 1 In the case that both pathdiagrams are identical in terms of their relationship (arcs)then we have 119903119894119895 = 1 while 119903119894119895 = 0 if no common relationshipexists between the two path diagrams By definition 119903119894119895 = 119903119895119894

These formulae adapted from Sinreich et al [53] arevalidated and used to calculate the similarity between processcharts In this study we used them to analyze hierarchiesand extended their work by adding directions To analyzedirectional relationship an adjacency matrix was used toindicate relationship according to the graph theory Thenrelationship similarity of hierarchical systems was calculatedby using formulas (2) (3) and (4)

The process of calculating the mental model similaritycan be seen in Figure 3

In order to illustrate the calculation procedure of thesimilarity measure the following example is given (seeFigure 4)

From Figure 4 the following values are obtained 119890119894119895 = 13and 119887119894119895 = 119887119895119894 = 0

Using these values in (1) results in the directionless sim-ilarity measure of 119886119894119895 = 1 Using the graph theory the adja-cency matrix ℎ119894 can be calculated as follows

1198671 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 1 1 1 1 1 0 0 0 0 0 01 1 0 1 0 0 0 1 1 1 0 0 01 1 1 0 0 0 0 0 0 0 1 1 10 1 1 1 0 1 1 0 0 0 0 0 00 1 1 1 1 0 1 0 0 0 0 0 00 1 1 1 1 1 0 0 0 0 0 0 00 1 1 1 0 0 0 0 1 1 0 0 00 1 1 1 0 0 0 1 0 1 0 0 00 1 1 1 0 0 0 1 1 0 0 0 00 1 1 1 0 0 0 0 0 0 0 1 10 1 1 1 0 0 0 0 0 0 1 0 10 1 1 1 0 0 0 0 0 0 1 1 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

Advances in Human-Computer Interaction 7

Web page

HTML

User

Directionless similarity

Directional similarity

Website designer

Website designer

Information structure without directional relationship

Mental model

User

Information structure with directional relationship

Card sorting

Path diagram

Figure 3 The process of calculating mental model similarity

Invention

Book

Diao ban yin shua

Huang dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yingda

Tang yanqianHistoricalcharacter

Tang dynasty

Book and historical character

Invention and historical character

Invention and Book

1

Invention

Book

Diao ban yin shua

Huan dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yinda

Tang yanqianHistorical character

Tang dynasty

2

Figure 4 An example of two different path diagrams (the segment with one arrow means one-way relationship which means node A canreach node B while node B cannot reach node A the segment with two arrows means that node A and node B can reach each other)

8 Advances in Human-Computer Interaction

1198672 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 0 0 1 1 1 0 0 0 0 0 01 0 0 0 0 0 0 1 1 1 0 0 01 0 0 0 0 0 0 0 0 0 1 1 10 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

(5)

Using these values with (2) and (3) the common andexclusive path vector can be calculated as follows

11988812 = 3 + 4 + 4 + 4 + 9 = 2411988912 = 0 + 2 + 2 + 2 + 36 = 42

(6)

Based on these values and (4) the directional similaritymeasure can be calculated as follows

11990312 = 2424 + 42 = 4

11 cong 036 (7)

The directionless similarity is 1 while the directionalsimilarity is 036The calculation results show that thementalmodel similarity was totally different whenwe considered thedirectional relationship of information structures

4 Results and Discussion

The following sections first examine the effects of the infor-mation structures on mental model similarity and thenexamine the relationship between mental model similarityand user performance Finally we examine whether themediation effect was found

41 The Influence of Information Structures on Mental ModelSimilarity Simplicitycomplexity generally refers to the levelof intricacy or detail in a stimulus [54 55] Specifically thedetail could be the number of closed figures open figuresletters horizontal lines vertical lines and so forth [54]

Complexity of web pages with different informationstructures consists of two high-level notions content com-plexity which is mainly measured through the number andtypes of objects to load a web page and service complexitywhich is mainly measured through ldquothe number and con-tributions of the various servers and administrative originsrdquo[56] Specifically if the interaction semantics and UI designare not considered the structure of website is the focus ofcomplexity It could be computed through a function which

Table 2 Complexity of three information structures

Net structure Treestructure

Linearstructure

Nodes 13 13 13Line segments 33 12 8

High (net) Medium (tree)Complexity of information structure

Low (linear)

Directional or notDirectionaless similarityDirectional similarity

04

06

08

10

Men

tal M

odel

Sim

ilarit

y

Figure 5 The influence of complexity of information structures onmental model similarity

calculated the number of outgoing links and controls such asbuttons and checkboxes [57]

Regarding these studies objective metrics of complexityof information structures were the number of nodes andlinks As shown in Table 2 the numbers of nodes of all threeinformation structures (ie net tree and linear) are the samebut the net structure has more directional links between anytwo nodes than the tree structure which in turn has morelinks than the linear structure Therefore complexity of theinformation structure has three levels termed low complexitymedium complexity and high complexity

Subjective metrics of complexity were consistent withresults of objective metrics Participants rated the simplicityof three websites on a 5-point Likert scale anchored fromldquoeasyrdquo to ldquocomplexrdquoThe average rating for web net tree andlinear was 41 (SD = 058) 31 (SD = 076) and 19 (SD = 094)

To further examine the influence of complexity of infor-mation structures on two different kinds of similarity one-way repeated ANOVA analysis was conducted As shown inFigure 5 the results indicated that complexity of the infor-mation structures had significant influence on directionalsimilarity (119865(262) = 5051 119901 lt 0001) Specifically resultsof multiple comparison with the Bonferroni correctionsindicated that the complex net structure resulted in widergap between the usersrsquo and designersrsquo mental models than thethree structure (119885 = 9081 119901 lt 0001) and the linear struc-ture (119885 = 8347 119901 lt 0001) The complexity of informationstructures resulted in nodifferences in directionless similarity(119865(262) = 3056 119901 = 00542) Therefore (H1a) was rejectedand (H1b) was supported

Advances in Human-Computer Interaction 9

Table 3 Linear regression results of mental model similarity on user performance

The task completion time The number of clicks119861 119905 119901 119861 119905 119901

Directionless similarity minus399 minus083 041 299 080 043Directional similarity 723 343 000 924 635 000

42 The Influence of Mental Model Similarity on User Per-formance Linear regression was conducted The dependentvariables were the task completion time and the number ofclicks and there were seven independent variables direc-tional similarity directionless similarity information struc-tures age technology product experience spatial ability andmemory capacity The results of regression analysis wereshown in Table 3

According to Table 3 the directional similarity waslinearly related to the task completion time Participants takeless time to find the target item hidden in the informationstructureswhen the directional similaritywas lowerThismayexplain the phenomenon that users who only remember thepaths to a special item of web pages take less time to find atarget in web pages than those who get a full understandingof elements and relationship of the web pages

Table 3 also indicates that the directional similarity waslinearly related to the number of clicks Participants clickedless to complete the tasks when the directional similarity waslower and thewebpages hadmore paths to obtain targets suchas net structure Although mental model similarity betweenthe participant and designer was lower in the net structurethan in the tree structure and the linear structure the numberof clicks was smaller with the lowest similarity One possiblereason for this is that the net-structure web page has manypaths and the elements are connected to each other while thetree structure and the linear structure are notThismeans thatusers cannot return to the previous page when they need toreach other pages in the net-structure web page unlike thetree structure and the linear structure

It can also be inferred that using path diagram to elicitmental models is more effective than using card sorting Itcan also verify that the directional relationship of informationstructures is important which cannot be ignored in elicitingmental models of information structures

The mental model similarity was calculated from theinformation structure Under each information structurehow does the mental model similarity affect user perfor-mance Correlation analysis and one-way ANOVA analysiswere conducted

The results indicated that only the directional similaritywas positively correlated with the task completion time in thenet-structureweb pageHowever either directional similarityor directionless similarity had no significant impact on userperformance under each information structure

In addition the user performance was not affected bytheir age spatial ability memory capacity or technologyproduct experience Moreover no matter what kind of infor-mation structure the user performance was still not affectedby demographic variables One possible reason is that the par-ticipants chosen for this study were all college students the

differences among them were too small In other words theselection of participants has limitations It may be differentwhen expanding the scope of participants especially to theelderly children and those with lower levels of education

43 Mediation Effects of Mental Model Similarity Accordingto MacKinnon et al [58] three modes were used to estimatethe basic intervening variable model which were shown inMod 1 Mod 2 and Mod 3

Mod 1 119884 = 119888119883 + 1198901 (8)

Mod 2 119884 = 119889119883 + 119887119872 + 1198902 (9)

Mod 3 119872 = 119886119883 + 1198903 (10)

In these equations 119883 is the independent variable 119884 isthe dependent variable and119872 is the intervening variable 11989011198902 and 1198903 are the population regression intercepts in (8) (9)and (10) respectively 119888 represents the relation between theindependent and dependent variables in (8) 119889 represents therelation between the independent and dependent variablesadjusted for the effects of the intervening variable in (9) 119886represents the relation between the independent and inter-vening variables in (10) and 119887 represents the relation betweenthe intervening and the dependent variables adjusted for theeffect of the independent variable in (9) [58] According toSobelrsquos [59] test the mediation effect was investigated Theresults of statistical analysis were shown in Tables 4 and 5

For both the number of clicks and the task completiontime the directional similarity was a significant mediatorHowever the effects are only partial mediation because thedirect effect is still significant (Tables 4 and 5) The simplicityof information structures had significant effects on the taskcompletion time and the number of clicks A part of the effectwas achieved by directional similarity which means that userperformance was partially influenced by the degree of matchbetween usersrsquo mental models and information structures

Tables 6 and 7 indicate that the directionless similaritywas not a mediator there was no significant mediation effectfor directionless similarity on task completion time or thenumber of clicks One possible reason is that the directionlesssimilarity was calculated from the card sorting in whichthe details about the directional relationship of informationstructures were not considered and this would lose theaccuracy when using card sorting to elicit mental models

Both the task completion time and the number of clicksshowed partial mediation by the directional similarity Theinformation structures had a direct effect on user perfor-mance The directionless similarity did not account for therelationship between information structures and user perfor-mance while the directional similarity as the new index to

10 Advances in Human-Computer Interaction

Table 4 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and taskcompletion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

171 069 247 92

Intercept 1385 177 782 936119890minus12Pred 177 083 215 345119890minus02

Mod 2Intercept 1186 188 631 104119890minus08Pred 007 104 007 947119890minus01Med 711 274 260 110eminus02

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 5 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and thenumber of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

123 045 273 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 002 121 002 099Pred 245 067 366 000Med 513 176 291 000

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 6 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe task completion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus028 023 minus119 92

Intercept 1385 177 782 936119890minus12Pred 177 083 213 345119890minus02

Mod 2Intercept 1971 456 432 405119890minus05Pred 205 085 241 174119890minus02Med minus673 483 minus139 167eminus01

Mod 3Intercept 087 004 2175 539119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in Human-Computer Interaction 5

Q1 which dynasty ldquoHuang dao you yirdquo belongs to Q2 whatldquoHuang dao you yirdquo is aboutrdquo

34 Dependent Variables The two dependent variables weretask completion time and the number of clicks Task com-pletion time was the average of eight tasksrsquo completion timeunder each information structure and the number of clickswas the average of the click times to complete all of the eighttasksThey were both measured by the Morae Recorder onlywhen the participant obtained the correct answers for bothquestions were the missions completed

35 Independent Variable The independent variables werethe directionless similarity and directional similarity It is anew quantitative measure proposed here (see Section 37)the directionless similarity was calculated from card sortingand the directional similarity was calculated from pathdiagramThemental model similarity was computed for eachinformation structure

Demographic variables included age experience of usingtechnology product spatial ability and memory capacity Aquestionnaire was used to collect basic information aboutexperience of using technology product

The mental model similarity was a between-subject vari-able while the information structure was a within-subjectvariableThat is each participant used three prototypes In aneffort to avoid learning effects the order of using prototypeswas random

36 Procedure The experiment took each participant aboutone hour to complete It consisted of four tests a question-naire test a memory test a paper folding test and a cardsorting test

First each participant began the experiment by filling outa consent form and a general questionnaire about hisherdemographic information and using experience with tech-nology products

Secondly a memory test was conducted using the KJ-I spatial position memory span tester After completing thememory test the test scores were recorded on paper and werenot disclosed to the participants

Thirdly a paper folding test was conducted to test thespatial ability of the participants This test is divided into twoparts The time limit of each part was three minutes to avoidparticipants losing their patience The participants needed tofind the correct answer independently The final score of thetest is the correct number minus the incorrect number on thetest paperThe higher the score the better the spatial memoryability

Fourthly a brief introduction and practice about theexperiment were given to each participant Finally partici-pants completed tasks on each web page and then went onwith the card sorting The cards were the titles of each nodein the information structures Then the participants wererequired to draw a path structure of the experimental webpage navigation with a whiteboard stroke During the wholeprocess participants were left alone and questions related tothe path were not answered in a relevant way which aimed atavoiding the subjective impacts of the experimental designer

At the end the experimenter conducted a five-minute explor-atory interview with the participants to understand theirthoughts and feelings about using the three web pages Thequestions in the interview included ldquoQ1 Please score thethree web pages in this experiment considering informationsearching ease of use and user experience Q2 Pleasesequence the three web pages according to your experiencerdquo

37 Quantifying Mental Model Similarity The method pro-posed to quantify mental model similarity consists of twoparts one is the method which can be used to elicit mentalmodels of information structures with directional relation-ship and the other one is the mathematical equations whichcan be used to calculate the mental model similarity

The first part is the method of eliciting mental models Inorder to quantify mental model similarity a method whichcan be used to elicit mental models with more details such asdirectional relationship has to be used Such amethod shouldrepresent the understanding of elements and directionalrelationship of a hierarchical system which are the keyfactors in elicitingmental model of a hierarchical systemTheliterature provides several methods to elicit mental modelssuch as card sorting which is widely used in eliciting mentalmodels of hierarchical systems [18 33] However card sortingcannot elicit completemental models of hierarchical systemsparticularly the directional aspects of hierarchies

A new method is needed to reflect the directionalinformation of mental models of hierarchy structures Webnavigation is similar to the real-world navigation In real-world navigation people usually take three strategies tofind a destination They remember properties of landmarkssuch as shape and structure (ie landmark knowledge) [4950] or the sequential order of landmarks encountered anddirectional relationship between these landmarks (ie routeknowledge) [51] or an overview of the environment likea map showing spatial relationships between routes andlandmarks (ie survey knowledge) [52] to find a destinationThis knowledge is also involved in web navigation Land-mark knowledge is mainly represented through card sortingInspired by route knowledge we proposed the path diagramto elicit mental models of hierarchical systems with moredetails (eg directional relationship)

The second part is about quantifying the mental modelsimilarity The limited research in quantifying mental modelsimilarity was reported by Sinreich et al [53] They intro-duced process chart into eliciting mental models of Emer-gency Department Management and quantified similaritythrough four formulae However themethodwas not appliedto analyze the mental models of information structures

The method of quantifying the degree of mental modelsimilarity aims to reflect the extent to usersrsquo understandingof the system it can present causality and logic In additionthis method should be easy to understand so that it can bemastered by nonprofessional persons Thus in this study weextended the work of Sinreich et al [53] as our proposedquantitative method We calculated mental model similarityin twoways calculating the directionless similarity from cardsorting and calculating the directional similarity from pathdiagram

6 Advances in Human-Computer Interaction

Invention Book

View 1 View 2

Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Invention

BookHistoricalcharacter

Diao ban yinshua

Shui yun hun yi Huang dao you

yi

Page name Invention

HomepageBook Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Page name Invention

HomepageInvention

Shui yun hun yiDiao ban yin shua Huang dao you yi

Diao ban yin shua Shui yun hun yi Huang dao you yi

Ji li gu che Zhuan lun cang Lian hua lou

Chai se tao yin Jian yi Di qiu yi

Figure 2 Two views of web page interfaces in this experiment

The first component represents the elements (nodes) ofthe different content for example ldquoTang dynastyrdquo ldquoInven-tionrdquo ldquoBookrdquo and ldquoHistorical characterrdquo in Figure 2 Thedirectionless similarity measure 119886119894119895 can be obtained using

119886119894119895 = 119890119894119895119890119894119895 + 119887119894119895 + 119887119895119894 (1)

where 119890119894119895 denotes the number of identical elements in cardsortings 119894 and 119895 and 119887119894119895 denotes the number of elements thatexist in 119894 that do not exist in 119895 (see that 119887119894119895 = 119887119895119894) It isclear that 0 ≪ 119886119894119895 ≪ 1 In the case that both card sortingsare identical in terms of their elements (not necessarily theirrelationships) then we have 119886119894119895 = 1 while 119886119894119895 = 0 if nocommon elements exist and by definition 119886119894119895 = 119886119895119894

The second component represents the relationshipbetween elements (arcs) in the path diagram A relationshipis defined by the elements it connects (there may be morethan one connection between elements) and by the directionof the connecting arc for example the arcs that connectelements ldquoTang dynasty-Inventionrdquo ldquoInvention-Bookrdquoand ldquoInvention-Diao ban yin shuardquo in path diagram A inFigure 4The first step in calculating the directional similarityis to obtain the adjacency matricesThe element of adjacencymatrix was the numbers of directed segment between twonodes (eg if there were one directed segment from ldquoTangdynastyrdquo to ldquoInventionrdquo the element is 1)

Based on the adjacency matrix the sum of all the com-mon arcs 119888119894119895 and the sum of all exclusive arcs 119889119894119895 betweenany two path diagrams 119894 and 119895 can be calculated as shownin (2) and (3) respectively Finally the directional similaritymeasure 119903119894119895 can be obtained using (4)

119888119894119895 = sum119896

sum119897

min ℎ119894119896119897 ℎ119895119896119897 (2)

119889119894119895 = sum119896

sum119897

10038161003816100381610038161003816ℎ119894119896119897 minus ℎ119895119896119897

10038161003816100381610038161003816 (3)

119903119894119895 = 119888119894119895119888119894119895 + 119889119894119895 (4)

It is clear that 0 ≪ 119903119894119895 ≪ 1 In the case that both pathdiagrams are identical in terms of their relationship (arcs)then we have 119903119894119895 = 1 while 119903119894119895 = 0 if no common relationshipexists between the two path diagrams By definition 119903119894119895 = 119903119895119894

These formulae adapted from Sinreich et al [53] arevalidated and used to calculate the similarity between processcharts In this study we used them to analyze hierarchiesand extended their work by adding directions To analyzedirectional relationship an adjacency matrix was used toindicate relationship according to the graph theory Thenrelationship similarity of hierarchical systems was calculatedby using formulas (2) (3) and (4)

The process of calculating the mental model similaritycan be seen in Figure 3

In order to illustrate the calculation procedure of thesimilarity measure the following example is given (seeFigure 4)

From Figure 4 the following values are obtained 119890119894119895 = 13and 119887119894119895 = 119887119895119894 = 0

Using these values in (1) results in the directionless sim-ilarity measure of 119886119894119895 = 1 Using the graph theory the adja-cency matrix ℎ119894 can be calculated as follows

1198671 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 1 1 1 1 1 0 0 0 0 0 01 1 0 1 0 0 0 1 1 1 0 0 01 1 1 0 0 0 0 0 0 0 1 1 10 1 1 1 0 1 1 0 0 0 0 0 00 1 1 1 1 0 1 0 0 0 0 0 00 1 1 1 1 1 0 0 0 0 0 0 00 1 1 1 0 0 0 0 1 1 0 0 00 1 1 1 0 0 0 1 0 1 0 0 00 1 1 1 0 0 0 1 1 0 0 0 00 1 1 1 0 0 0 0 0 0 0 1 10 1 1 1 0 0 0 0 0 0 1 0 10 1 1 1 0 0 0 0 0 0 1 1 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

Advances in Human-Computer Interaction 7

Web page

HTML

User

Directionless similarity

Directional similarity

Website designer

Website designer

Information structure without directional relationship

Mental model

User

Information structure with directional relationship

Card sorting

Path diagram

Figure 3 The process of calculating mental model similarity

Invention

Book

Diao ban yin shua

Huang dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yingda

Tang yanqianHistoricalcharacter

Tang dynasty

Book and historical character

Invention and historical character

Invention and Book

1

Invention

Book

Diao ban yin shua

Huan dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yinda

Tang yanqianHistorical character

Tang dynasty

2

Figure 4 An example of two different path diagrams (the segment with one arrow means one-way relationship which means node A canreach node B while node B cannot reach node A the segment with two arrows means that node A and node B can reach each other)

8 Advances in Human-Computer Interaction

1198672 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 0 0 1 1 1 0 0 0 0 0 01 0 0 0 0 0 0 1 1 1 0 0 01 0 0 0 0 0 0 0 0 0 1 1 10 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

(5)

Using these values with (2) and (3) the common andexclusive path vector can be calculated as follows

11988812 = 3 + 4 + 4 + 4 + 9 = 2411988912 = 0 + 2 + 2 + 2 + 36 = 42

(6)

Based on these values and (4) the directional similaritymeasure can be calculated as follows

11990312 = 2424 + 42 = 4

11 cong 036 (7)

The directionless similarity is 1 while the directionalsimilarity is 036The calculation results show that thementalmodel similarity was totally different whenwe considered thedirectional relationship of information structures

4 Results and Discussion

The following sections first examine the effects of the infor-mation structures on mental model similarity and thenexamine the relationship between mental model similarityand user performance Finally we examine whether themediation effect was found

41 The Influence of Information Structures on Mental ModelSimilarity Simplicitycomplexity generally refers to the levelof intricacy or detail in a stimulus [54 55] Specifically thedetail could be the number of closed figures open figuresletters horizontal lines vertical lines and so forth [54]

Complexity of web pages with different informationstructures consists of two high-level notions content com-plexity which is mainly measured through the number andtypes of objects to load a web page and service complexitywhich is mainly measured through ldquothe number and con-tributions of the various servers and administrative originsrdquo[56] Specifically if the interaction semantics and UI designare not considered the structure of website is the focus ofcomplexity It could be computed through a function which

Table 2 Complexity of three information structures

Net structure Treestructure

Linearstructure

Nodes 13 13 13Line segments 33 12 8

High (net) Medium (tree)Complexity of information structure

Low (linear)

Directional or notDirectionaless similarityDirectional similarity

04

06

08

10

Men

tal M

odel

Sim

ilarit

y

Figure 5 The influence of complexity of information structures onmental model similarity

calculated the number of outgoing links and controls such asbuttons and checkboxes [57]

Regarding these studies objective metrics of complexityof information structures were the number of nodes andlinks As shown in Table 2 the numbers of nodes of all threeinformation structures (ie net tree and linear) are the samebut the net structure has more directional links between anytwo nodes than the tree structure which in turn has morelinks than the linear structure Therefore complexity of theinformation structure has three levels termed low complexitymedium complexity and high complexity

Subjective metrics of complexity were consistent withresults of objective metrics Participants rated the simplicityof three websites on a 5-point Likert scale anchored fromldquoeasyrdquo to ldquocomplexrdquoThe average rating for web net tree andlinear was 41 (SD = 058) 31 (SD = 076) and 19 (SD = 094)

To further examine the influence of complexity of infor-mation structures on two different kinds of similarity one-way repeated ANOVA analysis was conducted As shown inFigure 5 the results indicated that complexity of the infor-mation structures had significant influence on directionalsimilarity (119865(262) = 5051 119901 lt 0001) Specifically resultsof multiple comparison with the Bonferroni correctionsindicated that the complex net structure resulted in widergap between the usersrsquo and designersrsquo mental models than thethree structure (119885 = 9081 119901 lt 0001) and the linear struc-ture (119885 = 8347 119901 lt 0001) The complexity of informationstructures resulted in nodifferences in directionless similarity(119865(262) = 3056 119901 = 00542) Therefore (H1a) was rejectedand (H1b) was supported

Advances in Human-Computer Interaction 9

Table 3 Linear regression results of mental model similarity on user performance

The task completion time The number of clicks119861 119905 119901 119861 119905 119901

Directionless similarity minus399 minus083 041 299 080 043Directional similarity 723 343 000 924 635 000

42 The Influence of Mental Model Similarity on User Per-formance Linear regression was conducted The dependentvariables were the task completion time and the number ofclicks and there were seven independent variables direc-tional similarity directionless similarity information struc-tures age technology product experience spatial ability andmemory capacity The results of regression analysis wereshown in Table 3

According to Table 3 the directional similarity waslinearly related to the task completion time Participants takeless time to find the target item hidden in the informationstructureswhen the directional similaritywas lowerThismayexplain the phenomenon that users who only remember thepaths to a special item of web pages take less time to find atarget in web pages than those who get a full understandingof elements and relationship of the web pages

Table 3 also indicates that the directional similarity waslinearly related to the number of clicks Participants clickedless to complete the tasks when the directional similarity waslower and thewebpages hadmore paths to obtain targets suchas net structure Although mental model similarity betweenthe participant and designer was lower in the net structurethan in the tree structure and the linear structure the numberof clicks was smaller with the lowest similarity One possiblereason for this is that the net-structure web page has manypaths and the elements are connected to each other while thetree structure and the linear structure are notThismeans thatusers cannot return to the previous page when they need toreach other pages in the net-structure web page unlike thetree structure and the linear structure

It can also be inferred that using path diagram to elicitmental models is more effective than using card sorting Itcan also verify that the directional relationship of informationstructures is important which cannot be ignored in elicitingmental models of information structures

The mental model similarity was calculated from theinformation structure Under each information structurehow does the mental model similarity affect user perfor-mance Correlation analysis and one-way ANOVA analysiswere conducted

The results indicated that only the directional similaritywas positively correlated with the task completion time in thenet-structureweb pageHowever either directional similarityor directionless similarity had no significant impact on userperformance under each information structure

In addition the user performance was not affected bytheir age spatial ability memory capacity or technologyproduct experience Moreover no matter what kind of infor-mation structure the user performance was still not affectedby demographic variables One possible reason is that the par-ticipants chosen for this study were all college students the

differences among them were too small In other words theselection of participants has limitations It may be differentwhen expanding the scope of participants especially to theelderly children and those with lower levels of education

43 Mediation Effects of Mental Model Similarity Accordingto MacKinnon et al [58] three modes were used to estimatethe basic intervening variable model which were shown inMod 1 Mod 2 and Mod 3

Mod 1 119884 = 119888119883 + 1198901 (8)

Mod 2 119884 = 119889119883 + 119887119872 + 1198902 (9)

Mod 3 119872 = 119886119883 + 1198903 (10)

In these equations 119883 is the independent variable 119884 isthe dependent variable and119872 is the intervening variable 11989011198902 and 1198903 are the population regression intercepts in (8) (9)and (10) respectively 119888 represents the relation between theindependent and dependent variables in (8) 119889 represents therelation between the independent and dependent variablesadjusted for the effects of the intervening variable in (9) 119886represents the relation between the independent and inter-vening variables in (10) and 119887 represents the relation betweenthe intervening and the dependent variables adjusted for theeffect of the independent variable in (9) [58] According toSobelrsquos [59] test the mediation effect was investigated Theresults of statistical analysis were shown in Tables 4 and 5

For both the number of clicks and the task completiontime the directional similarity was a significant mediatorHowever the effects are only partial mediation because thedirect effect is still significant (Tables 4 and 5) The simplicityof information structures had significant effects on the taskcompletion time and the number of clicks A part of the effectwas achieved by directional similarity which means that userperformance was partially influenced by the degree of matchbetween usersrsquo mental models and information structures

Tables 6 and 7 indicate that the directionless similaritywas not a mediator there was no significant mediation effectfor directionless similarity on task completion time or thenumber of clicks One possible reason is that the directionlesssimilarity was calculated from the card sorting in whichthe details about the directional relationship of informationstructures were not considered and this would lose theaccuracy when using card sorting to elicit mental models

Both the task completion time and the number of clicksshowed partial mediation by the directional similarity Theinformation structures had a direct effect on user perfor-mance The directionless similarity did not account for therelationship between information structures and user perfor-mance while the directional similarity as the new index to

10 Advances in Human-Computer Interaction

Table 4 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and taskcompletion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

171 069 247 92

Intercept 1385 177 782 936119890minus12Pred 177 083 215 345119890minus02

Mod 2Intercept 1186 188 631 104119890minus08Pred 007 104 007 947119890minus01Med 711 274 260 110eminus02

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 5 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and thenumber of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

123 045 273 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 002 121 002 099Pred 245 067 366 000Med 513 176 291 000

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 6 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe task completion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus028 023 minus119 92

Intercept 1385 177 782 936119890minus12Pred 177 083 213 345119890minus02

Mod 2Intercept 1971 456 432 405119890minus05Pred 205 085 241 174119890minus02Med minus673 483 minus139 167eminus01

Mod 3Intercept 087 004 2175 539119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

6 Advances in Human-Computer Interaction

Invention Book

View 1 View 2

Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Invention

BookHistoricalcharacter

Diao ban yinshua

Shui yun hun yi Huang dao you

yi

Page name Invention

HomepageBook Historical character

Tang dynasty

Song dynasty

Yuan dynasty

Page name Invention

HomepageInvention

Shui yun hun yiDiao ban yin shua Huang dao you yi

Diao ban yin shua Shui yun hun yi Huang dao you yi

Ji li gu che Zhuan lun cang Lian hua lou

Chai se tao yin Jian yi Di qiu yi

Figure 2 Two views of web page interfaces in this experiment

The first component represents the elements (nodes) ofthe different content for example ldquoTang dynastyrdquo ldquoInven-tionrdquo ldquoBookrdquo and ldquoHistorical characterrdquo in Figure 2 Thedirectionless similarity measure 119886119894119895 can be obtained using

119886119894119895 = 119890119894119895119890119894119895 + 119887119894119895 + 119887119895119894 (1)

where 119890119894119895 denotes the number of identical elements in cardsortings 119894 and 119895 and 119887119894119895 denotes the number of elements thatexist in 119894 that do not exist in 119895 (see that 119887119894119895 = 119887119895119894) It isclear that 0 ≪ 119886119894119895 ≪ 1 In the case that both card sortingsare identical in terms of their elements (not necessarily theirrelationships) then we have 119886119894119895 = 1 while 119886119894119895 = 0 if nocommon elements exist and by definition 119886119894119895 = 119886119895119894

The second component represents the relationshipbetween elements (arcs) in the path diagram A relationshipis defined by the elements it connects (there may be morethan one connection between elements) and by the directionof the connecting arc for example the arcs that connectelements ldquoTang dynasty-Inventionrdquo ldquoInvention-Bookrdquoand ldquoInvention-Diao ban yin shuardquo in path diagram A inFigure 4The first step in calculating the directional similarityis to obtain the adjacency matricesThe element of adjacencymatrix was the numbers of directed segment between twonodes (eg if there were one directed segment from ldquoTangdynastyrdquo to ldquoInventionrdquo the element is 1)

Based on the adjacency matrix the sum of all the com-mon arcs 119888119894119895 and the sum of all exclusive arcs 119889119894119895 betweenany two path diagrams 119894 and 119895 can be calculated as shownin (2) and (3) respectively Finally the directional similaritymeasure 119903119894119895 can be obtained using (4)

119888119894119895 = sum119896

sum119897

min ℎ119894119896119897 ℎ119895119896119897 (2)

119889119894119895 = sum119896

sum119897

10038161003816100381610038161003816ℎ119894119896119897 minus ℎ119895119896119897

10038161003816100381610038161003816 (3)

119903119894119895 = 119888119894119895119888119894119895 + 119889119894119895 (4)

It is clear that 0 ≪ 119903119894119895 ≪ 1 In the case that both pathdiagrams are identical in terms of their relationship (arcs)then we have 119903119894119895 = 1 while 119903119894119895 = 0 if no common relationshipexists between the two path diagrams By definition 119903119894119895 = 119903119895119894

These formulae adapted from Sinreich et al [53] arevalidated and used to calculate the similarity between processcharts In this study we used them to analyze hierarchiesand extended their work by adding directions To analyzedirectional relationship an adjacency matrix was used toindicate relationship according to the graph theory Thenrelationship similarity of hierarchical systems was calculatedby using formulas (2) (3) and (4)

The process of calculating the mental model similaritycan be seen in Figure 3

In order to illustrate the calculation procedure of thesimilarity measure the following example is given (seeFigure 4)

From Figure 4 the following values are obtained 119890119894119895 = 13and 119887119894119895 = 119887119895119894 = 0

Using these values in (1) results in the directionless sim-ilarity measure of 119886119894119895 = 1 Using the graph theory the adja-cency matrix ℎ119894 can be calculated as follows

1198671 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 1 1 1 1 1 0 0 0 0 0 01 1 0 1 0 0 0 1 1 1 0 0 01 1 1 0 0 0 0 0 0 0 1 1 10 1 1 1 0 1 1 0 0 0 0 0 00 1 1 1 1 0 1 0 0 0 0 0 00 1 1 1 1 1 0 0 0 0 0 0 00 1 1 1 0 0 0 0 1 1 0 0 00 1 1 1 0 0 0 1 0 1 0 0 00 1 1 1 0 0 0 1 1 0 0 0 00 1 1 1 0 0 0 0 0 0 0 1 10 1 1 1 0 0 0 0 0 0 1 0 10 1 1 1 0 0 0 0 0 0 1 1 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

Advances in Human-Computer Interaction 7

Web page

HTML

User

Directionless similarity

Directional similarity

Website designer

Website designer

Information structure without directional relationship

Mental model

User

Information structure with directional relationship

Card sorting

Path diagram

Figure 3 The process of calculating mental model similarity

Invention

Book

Diao ban yin shua

Huang dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yingda

Tang yanqianHistoricalcharacter

Tang dynasty

Book and historical character

Invention and historical character

Invention and Book

1

Invention

Book

Diao ban yin shua

Huan dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yinda

Tang yanqianHistorical character

Tang dynasty

2

Figure 4 An example of two different path diagrams (the segment with one arrow means one-way relationship which means node A canreach node B while node B cannot reach node A the segment with two arrows means that node A and node B can reach each other)

8 Advances in Human-Computer Interaction

1198672 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 0 0 1 1 1 0 0 0 0 0 01 0 0 0 0 0 0 1 1 1 0 0 01 0 0 0 0 0 0 0 0 0 1 1 10 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

(5)

Using these values with (2) and (3) the common andexclusive path vector can be calculated as follows

11988812 = 3 + 4 + 4 + 4 + 9 = 2411988912 = 0 + 2 + 2 + 2 + 36 = 42

(6)

Based on these values and (4) the directional similaritymeasure can be calculated as follows

11990312 = 2424 + 42 = 4

11 cong 036 (7)

The directionless similarity is 1 while the directionalsimilarity is 036The calculation results show that thementalmodel similarity was totally different whenwe considered thedirectional relationship of information structures

4 Results and Discussion

The following sections first examine the effects of the infor-mation structures on mental model similarity and thenexamine the relationship between mental model similarityand user performance Finally we examine whether themediation effect was found

41 The Influence of Information Structures on Mental ModelSimilarity Simplicitycomplexity generally refers to the levelof intricacy or detail in a stimulus [54 55] Specifically thedetail could be the number of closed figures open figuresletters horizontal lines vertical lines and so forth [54]

Complexity of web pages with different informationstructures consists of two high-level notions content com-plexity which is mainly measured through the number andtypes of objects to load a web page and service complexitywhich is mainly measured through ldquothe number and con-tributions of the various servers and administrative originsrdquo[56] Specifically if the interaction semantics and UI designare not considered the structure of website is the focus ofcomplexity It could be computed through a function which

Table 2 Complexity of three information structures

Net structure Treestructure

Linearstructure

Nodes 13 13 13Line segments 33 12 8

High (net) Medium (tree)Complexity of information structure

Low (linear)

Directional or notDirectionaless similarityDirectional similarity

04

06

08

10

Men

tal M

odel

Sim

ilarit

y

Figure 5 The influence of complexity of information structures onmental model similarity

calculated the number of outgoing links and controls such asbuttons and checkboxes [57]

Regarding these studies objective metrics of complexityof information structures were the number of nodes andlinks As shown in Table 2 the numbers of nodes of all threeinformation structures (ie net tree and linear) are the samebut the net structure has more directional links between anytwo nodes than the tree structure which in turn has morelinks than the linear structure Therefore complexity of theinformation structure has three levels termed low complexitymedium complexity and high complexity

Subjective metrics of complexity were consistent withresults of objective metrics Participants rated the simplicityof three websites on a 5-point Likert scale anchored fromldquoeasyrdquo to ldquocomplexrdquoThe average rating for web net tree andlinear was 41 (SD = 058) 31 (SD = 076) and 19 (SD = 094)

To further examine the influence of complexity of infor-mation structures on two different kinds of similarity one-way repeated ANOVA analysis was conducted As shown inFigure 5 the results indicated that complexity of the infor-mation structures had significant influence on directionalsimilarity (119865(262) = 5051 119901 lt 0001) Specifically resultsof multiple comparison with the Bonferroni correctionsindicated that the complex net structure resulted in widergap between the usersrsquo and designersrsquo mental models than thethree structure (119885 = 9081 119901 lt 0001) and the linear struc-ture (119885 = 8347 119901 lt 0001) The complexity of informationstructures resulted in nodifferences in directionless similarity(119865(262) = 3056 119901 = 00542) Therefore (H1a) was rejectedand (H1b) was supported

Advances in Human-Computer Interaction 9

Table 3 Linear regression results of mental model similarity on user performance

The task completion time The number of clicks119861 119905 119901 119861 119905 119901

Directionless similarity minus399 minus083 041 299 080 043Directional similarity 723 343 000 924 635 000

42 The Influence of Mental Model Similarity on User Per-formance Linear regression was conducted The dependentvariables were the task completion time and the number ofclicks and there were seven independent variables direc-tional similarity directionless similarity information struc-tures age technology product experience spatial ability andmemory capacity The results of regression analysis wereshown in Table 3

According to Table 3 the directional similarity waslinearly related to the task completion time Participants takeless time to find the target item hidden in the informationstructureswhen the directional similaritywas lowerThismayexplain the phenomenon that users who only remember thepaths to a special item of web pages take less time to find atarget in web pages than those who get a full understandingof elements and relationship of the web pages

Table 3 also indicates that the directional similarity waslinearly related to the number of clicks Participants clickedless to complete the tasks when the directional similarity waslower and thewebpages hadmore paths to obtain targets suchas net structure Although mental model similarity betweenthe participant and designer was lower in the net structurethan in the tree structure and the linear structure the numberof clicks was smaller with the lowest similarity One possiblereason for this is that the net-structure web page has manypaths and the elements are connected to each other while thetree structure and the linear structure are notThismeans thatusers cannot return to the previous page when they need toreach other pages in the net-structure web page unlike thetree structure and the linear structure

It can also be inferred that using path diagram to elicitmental models is more effective than using card sorting Itcan also verify that the directional relationship of informationstructures is important which cannot be ignored in elicitingmental models of information structures

The mental model similarity was calculated from theinformation structure Under each information structurehow does the mental model similarity affect user perfor-mance Correlation analysis and one-way ANOVA analysiswere conducted

The results indicated that only the directional similaritywas positively correlated with the task completion time in thenet-structureweb pageHowever either directional similarityor directionless similarity had no significant impact on userperformance under each information structure

In addition the user performance was not affected bytheir age spatial ability memory capacity or technologyproduct experience Moreover no matter what kind of infor-mation structure the user performance was still not affectedby demographic variables One possible reason is that the par-ticipants chosen for this study were all college students the

differences among them were too small In other words theselection of participants has limitations It may be differentwhen expanding the scope of participants especially to theelderly children and those with lower levels of education

43 Mediation Effects of Mental Model Similarity Accordingto MacKinnon et al [58] three modes were used to estimatethe basic intervening variable model which were shown inMod 1 Mod 2 and Mod 3

Mod 1 119884 = 119888119883 + 1198901 (8)

Mod 2 119884 = 119889119883 + 119887119872 + 1198902 (9)

Mod 3 119872 = 119886119883 + 1198903 (10)

In these equations 119883 is the independent variable 119884 isthe dependent variable and119872 is the intervening variable 11989011198902 and 1198903 are the population regression intercepts in (8) (9)and (10) respectively 119888 represents the relation between theindependent and dependent variables in (8) 119889 represents therelation between the independent and dependent variablesadjusted for the effects of the intervening variable in (9) 119886represents the relation between the independent and inter-vening variables in (10) and 119887 represents the relation betweenthe intervening and the dependent variables adjusted for theeffect of the independent variable in (9) [58] According toSobelrsquos [59] test the mediation effect was investigated Theresults of statistical analysis were shown in Tables 4 and 5

For both the number of clicks and the task completiontime the directional similarity was a significant mediatorHowever the effects are only partial mediation because thedirect effect is still significant (Tables 4 and 5) The simplicityof information structures had significant effects on the taskcompletion time and the number of clicks A part of the effectwas achieved by directional similarity which means that userperformance was partially influenced by the degree of matchbetween usersrsquo mental models and information structures

Tables 6 and 7 indicate that the directionless similaritywas not a mediator there was no significant mediation effectfor directionless similarity on task completion time or thenumber of clicks One possible reason is that the directionlesssimilarity was calculated from the card sorting in whichthe details about the directional relationship of informationstructures were not considered and this would lose theaccuracy when using card sorting to elicit mental models

Both the task completion time and the number of clicksshowed partial mediation by the directional similarity Theinformation structures had a direct effect on user perfor-mance The directionless similarity did not account for therelationship between information structures and user perfor-mance while the directional similarity as the new index to

10 Advances in Human-Computer Interaction

Table 4 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and taskcompletion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

171 069 247 92

Intercept 1385 177 782 936119890minus12Pred 177 083 215 345119890minus02

Mod 2Intercept 1186 188 631 104119890minus08Pred 007 104 007 947119890minus01Med 711 274 260 110eminus02

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 5 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and thenumber of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

123 045 273 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 002 121 002 099Pred 245 067 366 000Med 513 176 291 000

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 6 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe task completion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus028 023 minus119 92

Intercept 1385 177 782 936119890minus12Pred 177 083 213 345119890minus02

Mod 2Intercept 1971 456 432 405119890minus05Pred 205 085 241 174119890minus02Med minus673 483 minus139 167eminus01

Mod 3Intercept 087 004 2175 539119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in Human-Computer Interaction 7

Web page

HTML

User

Directionless similarity

Directional similarity

Website designer

Website designer

Information structure without directional relationship

Mental model

User

Information structure with directional relationship

Card sorting

Path diagram

Figure 3 The process of calculating mental model similarity

Invention

Book

Diao ban yin shua

Huang dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yingda

Tang yanqianHistoricalcharacter

Tang dynasty

Book and historical character

Invention and historical character

Invention and Book

1

Invention

Book

Diao ban yin shua

Huan dao you yi

Yi wen lei ju

Shi tong

Shui yun hun yi

Pi rixiu

Kuo di zhi

Kong yinda

Tang yanqianHistorical character

Tang dynasty

2

Figure 4 An example of two different path diagrams (the segment with one arrow means one-way relationship which means node A canreach node B while node B cannot reach node A the segment with two arrows means that node A and node B can reach each other)

8 Advances in Human-Computer Interaction

1198672 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 0 0 1 1 1 0 0 0 0 0 01 0 0 0 0 0 0 1 1 1 0 0 01 0 0 0 0 0 0 0 0 0 1 1 10 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

(5)

Using these values with (2) and (3) the common andexclusive path vector can be calculated as follows

11988812 = 3 + 4 + 4 + 4 + 9 = 2411988912 = 0 + 2 + 2 + 2 + 36 = 42

(6)

Based on these values and (4) the directional similaritymeasure can be calculated as follows

11990312 = 2424 + 42 = 4

11 cong 036 (7)

The directionless similarity is 1 while the directionalsimilarity is 036The calculation results show that thementalmodel similarity was totally different whenwe considered thedirectional relationship of information structures

4 Results and Discussion

The following sections first examine the effects of the infor-mation structures on mental model similarity and thenexamine the relationship between mental model similarityand user performance Finally we examine whether themediation effect was found

41 The Influence of Information Structures on Mental ModelSimilarity Simplicitycomplexity generally refers to the levelof intricacy or detail in a stimulus [54 55] Specifically thedetail could be the number of closed figures open figuresletters horizontal lines vertical lines and so forth [54]

Complexity of web pages with different informationstructures consists of two high-level notions content com-plexity which is mainly measured through the number andtypes of objects to load a web page and service complexitywhich is mainly measured through ldquothe number and con-tributions of the various servers and administrative originsrdquo[56] Specifically if the interaction semantics and UI designare not considered the structure of website is the focus ofcomplexity It could be computed through a function which

Table 2 Complexity of three information structures

Net structure Treestructure

Linearstructure

Nodes 13 13 13Line segments 33 12 8

High (net) Medium (tree)Complexity of information structure

Low (linear)

Directional or notDirectionaless similarityDirectional similarity

04

06

08

10

Men

tal M

odel

Sim

ilarit

y

Figure 5 The influence of complexity of information structures onmental model similarity

calculated the number of outgoing links and controls such asbuttons and checkboxes [57]

Regarding these studies objective metrics of complexityof information structures were the number of nodes andlinks As shown in Table 2 the numbers of nodes of all threeinformation structures (ie net tree and linear) are the samebut the net structure has more directional links between anytwo nodes than the tree structure which in turn has morelinks than the linear structure Therefore complexity of theinformation structure has three levels termed low complexitymedium complexity and high complexity

Subjective metrics of complexity were consistent withresults of objective metrics Participants rated the simplicityof three websites on a 5-point Likert scale anchored fromldquoeasyrdquo to ldquocomplexrdquoThe average rating for web net tree andlinear was 41 (SD = 058) 31 (SD = 076) and 19 (SD = 094)

To further examine the influence of complexity of infor-mation structures on two different kinds of similarity one-way repeated ANOVA analysis was conducted As shown inFigure 5 the results indicated that complexity of the infor-mation structures had significant influence on directionalsimilarity (119865(262) = 5051 119901 lt 0001) Specifically resultsof multiple comparison with the Bonferroni correctionsindicated that the complex net structure resulted in widergap between the usersrsquo and designersrsquo mental models than thethree structure (119885 = 9081 119901 lt 0001) and the linear struc-ture (119885 = 8347 119901 lt 0001) The complexity of informationstructures resulted in nodifferences in directionless similarity(119865(262) = 3056 119901 = 00542) Therefore (H1a) was rejectedand (H1b) was supported

Advances in Human-Computer Interaction 9

Table 3 Linear regression results of mental model similarity on user performance

The task completion time The number of clicks119861 119905 119901 119861 119905 119901

Directionless similarity minus399 minus083 041 299 080 043Directional similarity 723 343 000 924 635 000

42 The Influence of Mental Model Similarity on User Per-formance Linear regression was conducted The dependentvariables were the task completion time and the number ofclicks and there were seven independent variables direc-tional similarity directionless similarity information struc-tures age technology product experience spatial ability andmemory capacity The results of regression analysis wereshown in Table 3

According to Table 3 the directional similarity waslinearly related to the task completion time Participants takeless time to find the target item hidden in the informationstructureswhen the directional similaritywas lowerThismayexplain the phenomenon that users who only remember thepaths to a special item of web pages take less time to find atarget in web pages than those who get a full understandingof elements and relationship of the web pages

Table 3 also indicates that the directional similarity waslinearly related to the number of clicks Participants clickedless to complete the tasks when the directional similarity waslower and thewebpages hadmore paths to obtain targets suchas net structure Although mental model similarity betweenthe participant and designer was lower in the net structurethan in the tree structure and the linear structure the numberof clicks was smaller with the lowest similarity One possiblereason for this is that the net-structure web page has manypaths and the elements are connected to each other while thetree structure and the linear structure are notThismeans thatusers cannot return to the previous page when they need toreach other pages in the net-structure web page unlike thetree structure and the linear structure

It can also be inferred that using path diagram to elicitmental models is more effective than using card sorting Itcan also verify that the directional relationship of informationstructures is important which cannot be ignored in elicitingmental models of information structures

The mental model similarity was calculated from theinformation structure Under each information structurehow does the mental model similarity affect user perfor-mance Correlation analysis and one-way ANOVA analysiswere conducted

The results indicated that only the directional similaritywas positively correlated with the task completion time in thenet-structureweb pageHowever either directional similarityor directionless similarity had no significant impact on userperformance under each information structure

In addition the user performance was not affected bytheir age spatial ability memory capacity or technologyproduct experience Moreover no matter what kind of infor-mation structure the user performance was still not affectedby demographic variables One possible reason is that the par-ticipants chosen for this study were all college students the

differences among them were too small In other words theselection of participants has limitations It may be differentwhen expanding the scope of participants especially to theelderly children and those with lower levels of education

43 Mediation Effects of Mental Model Similarity Accordingto MacKinnon et al [58] three modes were used to estimatethe basic intervening variable model which were shown inMod 1 Mod 2 and Mod 3

Mod 1 119884 = 119888119883 + 1198901 (8)

Mod 2 119884 = 119889119883 + 119887119872 + 1198902 (9)

Mod 3 119872 = 119886119883 + 1198903 (10)

In these equations 119883 is the independent variable 119884 isthe dependent variable and119872 is the intervening variable 11989011198902 and 1198903 are the population regression intercepts in (8) (9)and (10) respectively 119888 represents the relation between theindependent and dependent variables in (8) 119889 represents therelation between the independent and dependent variablesadjusted for the effects of the intervening variable in (9) 119886represents the relation between the independent and inter-vening variables in (10) and 119887 represents the relation betweenthe intervening and the dependent variables adjusted for theeffect of the independent variable in (9) [58] According toSobelrsquos [59] test the mediation effect was investigated Theresults of statistical analysis were shown in Tables 4 and 5

For both the number of clicks and the task completiontime the directional similarity was a significant mediatorHowever the effects are only partial mediation because thedirect effect is still significant (Tables 4 and 5) The simplicityof information structures had significant effects on the taskcompletion time and the number of clicks A part of the effectwas achieved by directional similarity which means that userperformance was partially influenced by the degree of matchbetween usersrsquo mental models and information structures

Tables 6 and 7 indicate that the directionless similaritywas not a mediator there was no significant mediation effectfor directionless similarity on task completion time or thenumber of clicks One possible reason is that the directionlesssimilarity was calculated from the card sorting in whichthe details about the directional relationship of informationstructures were not considered and this would lose theaccuracy when using card sorting to elicit mental models

Both the task completion time and the number of clicksshowed partial mediation by the directional similarity Theinformation structures had a direct effect on user perfor-mance The directionless similarity did not account for therelationship between information structures and user perfor-mance while the directional similarity as the new index to

10 Advances in Human-Computer Interaction

Table 4 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and taskcompletion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

171 069 247 92

Intercept 1385 177 782 936119890minus12Pred 177 083 215 345119890minus02

Mod 2Intercept 1186 188 631 104119890minus08Pred 007 104 007 947119890minus01Med 711 274 260 110eminus02

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 5 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and thenumber of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

123 045 273 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 002 121 002 099Pred 245 067 366 000Med 513 176 291 000

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 6 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe task completion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus028 023 minus119 92

Intercept 1385 177 782 936119890minus12Pred 177 083 213 345119890minus02

Mod 2Intercept 1971 456 432 405119890minus05Pred 205 085 241 174119890minus02Med minus673 483 minus139 167eminus01

Mod 3Intercept 087 004 2175 539119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

8 Advances in Human-Computer Interaction

1198672 =

[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[

0 1 1 1 0 0 0 0 0 0 0 0 01 0 0 0 1 1 1 0 0 0 0 0 01 0 0 0 0 0 0 1 1 1 0 0 01 0 0 0 0 0 0 0 0 0 1 1 10 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 1 0 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 00 0 0 1 0 0 0 0 0 0 0 0 0

]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

(5)

Using these values with (2) and (3) the common andexclusive path vector can be calculated as follows

11988812 = 3 + 4 + 4 + 4 + 9 = 2411988912 = 0 + 2 + 2 + 2 + 36 = 42

(6)

Based on these values and (4) the directional similaritymeasure can be calculated as follows

11990312 = 2424 + 42 = 4

11 cong 036 (7)

The directionless similarity is 1 while the directionalsimilarity is 036The calculation results show that thementalmodel similarity was totally different whenwe considered thedirectional relationship of information structures

4 Results and Discussion

The following sections first examine the effects of the infor-mation structures on mental model similarity and thenexamine the relationship between mental model similarityand user performance Finally we examine whether themediation effect was found

41 The Influence of Information Structures on Mental ModelSimilarity Simplicitycomplexity generally refers to the levelof intricacy or detail in a stimulus [54 55] Specifically thedetail could be the number of closed figures open figuresletters horizontal lines vertical lines and so forth [54]

Complexity of web pages with different informationstructures consists of two high-level notions content com-plexity which is mainly measured through the number andtypes of objects to load a web page and service complexitywhich is mainly measured through ldquothe number and con-tributions of the various servers and administrative originsrdquo[56] Specifically if the interaction semantics and UI designare not considered the structure of website is the focus ofcomplexity It could be computed through a function which

Table 2 Complexity of three information structures

Net structure Treestructure

Linearstructure

Nodes 13 13 13Line segments 33 12 8

High (net) Medium (tree)Complexity of information structure

Low (linear)

Directional or notDirectionaless similarityDirectional similarity

04

06

08

10

Men

tal M

odel

Sim

ilarit

y

Figure 5 The influence of complexity of information structures onmental model similarity

calculated the number of outgoing links and controls such asbuttons and checkboxes [57]

Regarding these studies objective metrics of complexityof information structures were the number of nodes andlinks As shown in Table 2 the numbers of nodes of all threeinformation structures (ie net tree and linear) are the samebut the net structure has more directional links between anytwo nodes than the tree structure which in turn has morelinks than the linear structure Therefore complexity of theinformation structure has three levels termed low complexitymedium complexity and high complexity

Subjective metrics of complexity were consistent withresults of objective metrics Participants rated the simplicityof three websites on a 5-point Likert scale anchored fromldquoeasyrdquo to ldquocomplexrdquoThe average rating for web net tree andlinear was 41 (SD = 058) 31 (SD = 076) and 19 (SD = 094)

To further examine the influence of complexity of infor-mation structures on two different kinds of similarity one-way repeated ANOVA analysis was conducted As shown inFigure 5 the results indicated that complexity of the infor-mation structures had significant influence on directionalsimilarity (119865(262) = 5051 119901 lt 0001) Specifically resultsof multiple comparison with the Bonferroni correctionsindicated that the complex net structure resulted in widergap between the usersrsquo and designersrsquo mental models than thethree structure (119885 = 9081 119901 lt 0001) and the linear struc-ture (119885 = 8347 119901 lt 0001) The complexity of informationstructures resulted in nodifferences in directionless similarity(119865(262) = 3056 119901 = 00542) Therefore (H1a) was rejectedand (H1b) was supported

Advances in Human-Computer Interaction 9

Table 3 Linear regression results of mental model similarity on user performance

The task completion time The number of clicks119861 119905 119901 119861 119905 119901

Directionless similarity minus399 minus083 041 299 080 043Directional similarity 723 343 000 924 635 000

42 The Influence of Mental Model Similarity on User Per-formance Linear regression was conducted The dependentvariables were the task completion time and the number ofclicks and there were seven independent variables direc-tional similarity directionless similarity information struc-tures age technology product experience spatial ability andmemory capacity The results of regression analysis wereshown in Table 3

According to Table 3 the directional similarity waslinearly related to the task completion time Participants takeless time to find the target item hidden in the informationstructureswhen the directional similaritywas lowerThismayexplain the phenomenon that users who only remember thepaths to a special item of web pages take less time to find atarget in web pages than those who get a full understandingof elements and relationship of the web pages

Table 3 also indicates that the directional similarity waslinearly related to the number of clicks Participants clickedless to complete the tasks when the directional similarity waslower and thewebpages hadmore paths to obtain targets suchas net structure Although mental model similarity betweenthe participant and designer was lower in the net structurethan in the tree structure and the linear structure the numberof clicks was smaller with the lowest similarity One possiblereason for this is that the net-structure web page has manypaths and the elements are connected to each other while thetree structure and the linear structure are notThismeans thatusers cannot return to the previous page when they need toreach other pages in the net-structure web page unlike thetree structure and the linear structure

It can also be inferred that using path diagram to elicitmental models is more effective than using card sorting Itcan also verify that the directional relationship of informationstructures is important which cannot be ignored in elicitingmental models of information structures

The mental model similarity was calculated from theinformation structure Under each information structurehow does the mental model similarity affect user perfor-mance Correlation analysis and one-way ANOVA analysiswere conducted

The results indicated that only the directional similaritywas positively correlated with the task completion time in thenet-structureweb pageHowever either directional similarityor directionless similarity had no significant impact on userperformance under each information structure

In addition the user performance was not affected bytheir age spatial ability memory capacity or technologyproduct experience Moreover no matter what kind of infor-mation structure the user performance was still not affectedby demographic variables One possible reason is that the par-ticipants chosen for this study were all college students the

differences among them were too small In other words theselection of participants has limitations It may be differentwhen expanding the scope of participants especially to theelderly children and those with lower levels of education

43 Mediation Effects of Mental Model Similarity Accordingto MacKinnon et al [58] three modes were used to estimatethe basic intervening variable model which were shown inMod 1 Mod 2 and Mod 3

Mod 1 119884 = 119888119883 + 1198901 (8)

Mod 2 119884 = 119889119883 + 119887119872 + 1198902 (9)

Mod 3 119872 = 119886119883 + 1198903 (10)

In these equations 119883 is the independent variable 119884 isthe dependent variable and119872 is the intervening variable 11989011198902 and 1198903 are the population regression intercepts in (8) (9)and (10) respectively 119888 represents the relation between theindependent and dependent variables in (8) 119889 represents therelation between the independent and dependent variablesadjusted for the effects of the intervening variable in (9) 119886represents the relation between the independent and inter-vening variables in (10) and 119887 represents the relation betweenthe intervening and the dependent variables adjusted for theeffect of the independent variable in (9) [58] According toSobelrsquos [59] test the mediation effect was investigated Theresults of statistical analysis were shown in Tables 4 and 5

For both the number of clicks and the task completiontime the directional similarity was a significant mediatorHowever the effects are only partial mediation because thedirect effect is still significant (Tables 4 and 5) The simplicityof information structures had significant effects on the taskcompletion time and the number of clicks A part of the effectwas achieved by directional similarity which means that userperformance was partially influenced by the degree of matchbetween usersrsquo mental models and information structures

Tables 6 and 7 indicate that the directionless similaritywas not a mediator there was no significant mediation effectfor directionless similarity on task completion time or thenumber of clicks One possible reason is that the directionlesssimilarity was calculated from the card sorting in whichthe details about the directional relationship of informationstructures were not considered and this would lose theaccuracy when using card sorting to elicit mental models

Both the task completion time and the number of clicksshowed partial mediation by the directional similarity Theinformation structures had a direct effect on user perfor-mance The directionless similarity did not account for therelationship between information structures and user perfor-mance while the directional similarity as the new index to

10 Advances in Human-Computer Interaction

Table 4 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and taskcompletion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

171 069 247 92

Intercept 1385 177 782 936119890minus12Pred 177 083 215 345119890minus02

Mod 2Intercept 1186 188 631 104119890minus08Pred 007 104 007 947119890minus01Med 711 274 260 110eminus02

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 5 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and thenumber of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

123 045 273 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 002 121 002 099Pred 245 067 366 000Med 513 176 291 000

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 6 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe task completion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus028 023 minus119 92

Intercept 1385 177 782 936119890minus12Pred 177 083 213 345119890minus02

Mod 2Intercept 1971 456 432 405119890minus05Pred 205 085 241 174119890minus02Med minus673 483 minus139 167eminus01

Mod 3Intercept 087 004 2175 539119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in Human-Computer Interaction 9

Table 3 Linear regression results of mental model similarity on user performance

The task completion time The number of clicks119861 119905 119901 119861 119905 119901

Directionless similarity minus399 minus083 041 299 080 043Directional similarity 723 343 000 924 635 000

42 The Influence of Mental Model Similarity on User Per-formance Linear regression was conducted The dependentvariables were the task completion time and the number ofclicks and there were seven independent variables direc-tional similarity directionless similarity information struc-tures age technology product experience spatial ability andmemory capacity The results of regression analysis wereshown in Table 3

According to Table 3 the directional similarity waslinearly related to the task completion time Participants takeless time to find the target item hidden in the informationstructureswhen the directional similaritywas lowerThismayexplain the phenomenon that users who only remember thepaths to a special item of web pages take less time to find atarget in web pages than those who get a full understandingof elements and relationship of the web pages

Table 3 also indicates that the directional similarity waslinearly related to the number of clicks Participants clickedless to complete the tasks when the directional similarity waslower and thewebpages hadmore paths to obtain targets suchas net structure Although mental model similarity betweenthe participant and designer was lower in the net structurethan in the tree structure and the linear structure the numberof clicks was smaller with the lowest similarity One possiblereason for this is that the net-structure web page has manypaths and the elements are connected to each other while thetree structure and the linear structure are notThismeans thatusers cannot return to the previous page when they need toreach other pages in the net-structure web page unlike thetree structure and the linear structure

It can also be inferred that using path diagram to elicitmental models is more effective than using card sorting Itcan also verify that the directional relationship of informationstructures is important which cannot be ignored in elicitingmental models of information structures

The mental model similarity was calculated from theinformation structure Under each information structurehow does the mental model similarity affect user perfor-mance Correlation analysis and one-way ANOVA analysiswere conducted

The results indicated that only the directional similaritywas positively correlated with the task completion time in thenet-structureweb pageHowever either directional similarityor directionless similarity had no significant impact on userperformance under each information structure

In addition the user performance was not affected bytheir age spatial ability memory capacity or technologyproduct experience Moreover no matter what kind of infor-mation structure the user performance was still not affectedby demographic variables One possible reason is that the par-ticipants chosen for this study were all college students the

differences among them were too small In other words theselection of participants has limitations It may be differentwhen expanding the scope of participants especially to theelderly children and those with lower levels of education

43 Mediation Effects of Mental Model Similarity Accordingto MacKinnon et al [58] three modes were used to estimatethe basic intervening variable model which were shown inMod 1 Mod 2 and Mod 3

Mod 1 119884 = 119888119883 + 1198901 (8)

Mod 2 119884 = 119889119883 + 119887119872 + 1198902 (9)

Mod 3 119872 = 119886119883 + 1198903 (10)

In these equations 119883 is the independent variable 119884 isthe dependent variable and119872 is the intervening variable 11989011198902 and 1198903 are the population regression intercepts in (8) (9)and (10) respectively 119888 represents the relation between theindependent and dependent variables in (8) 119889 represents therelation between the independent and dependent variablesadjusted for the effects of the intervening variable in (9) 119886represents the relation between the independent and inter-vening variables in (10) and 119887 represents the relation betweenthe intervening and the dependent variables adjusted for theeffect of the independent variable in (9) [58] According toSobelrsquos [59] test the mediation effect was investigated Theresults of statistical analysis were shown in Tables 4 and 5

For both the number of clicks and the task completiontime the directional similarity was a significant mediatorHowever the effects are only partial mediation because thedirect effect is still significant (Tables 4 and 5) The simplicityof information structures had significant effects on the taskcompletion time and the number of clicks A part of the effectwas achieved by directional similarity which means that userperformance was partially influenced by the degree of matchbetween usersrsquo mental models and information structures

Tables 6 and 7 indicate that the directionless similaritywas not a mediator there was no significant mediation effectfor directionless similarity on task completion time or thenumber of clicks One possible reason is that the directionlesssimilarity was calculated from the card sorting in whichthe details about the directional relationship of informationstructures were not considered and this would lose theaccuracy when using card sorting to elicit mental models

Both the task completion time and the number of clicksshowed partial mediation by the directional similarity Theinformation structures had a direct effect on user perfor-mance The directionless similarity did not account for therelationship between information structures and user perfor-mance while the directional similarity as the new index to

10 Advances in Human-Computer Interaction

Table 4 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and taskcompletion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

171 069 247 92

Intercept 1385 177 782 936119890minus12Pred 177 083 215 345119890minus02

Mod 2Intercept 1186 188 631 104119890minus08Pred 007 104 007 947119890minus01Med 711 274 260 110eminus02

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 5 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and thenumber of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

123 045 273 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 002 121 002 099Pred 245 067 366 000Med 513 176 291 000

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 6 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe task completion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus028 023 minus119 92

Intercept 1385 177 782 936119890minus12Pred 177 083 213 345119890minus02

Mod 2Intercept 1971 456 432 405119890minus05Pred 205 085 241 174119890minus02Med minus673 483 minus139 167eminus01

Mod 3Intercept 087 004 2175 539119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

10 Advances in Human-Computer Interaction

Table 4 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and taskcompletion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

171 069 247 92

Intercept 1385 177 782 936119890minus12Pred 177 083 215 345119890minus02

Mod 2Intercept 1186 188 631 104119890minus08Pred 007 104 007 947119890minus01Med 711 274 260 110eminus02

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 5 Mediation analysis of the directional similarity as mediator of the relationship between simplicity of information structure and thenumber of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

123 045 273 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 002 121 002 099Pred 245 067 366 000Med 513 176 291 000

Mod 3Intercept 028 007 400 564119890minus05Pred 024 003 800 118119890minus11

Note Med refers to mediator Pred refers to independent variable

Table 6 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe task completion time

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus028 023 minus119 92

Intercept 1385 177 782 936119890minus12Pred 177 083 213 345119890minus02

Mod 2Intercept 1971 456 432 405119890minus05Pred 205 085 241 174119890minus02Med minus673 483 minus139 167eminus01

Mod 3Intercept 087 004 2175 539119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in Human-Computer Interaction 11

Table 7 Mediation analysis of the directionless similarity as mediator of the relationship between simplicity of information structure andthe number of clicks

Point estimate SE 119905 119901 Indirect effect SE 119911 119873Mod 1

minus008 013 minus062 92

Intercept 145 115 126 210119890minus01Pred 368 054 681 861119890minus10

Mod 2Intercept 322 299 108 284119890minus01Pred 376 055 684 119119890minus09Med minus203 317 minus064 523eminus01

Mod 3Intercept 087 004 2175 5390119890minus39Pred 004 002 200 249119890minus02

Note Med refers to mediator Pred refers to independent variable

measure the degree of match between mental models was asignificantmediatorThus (H3b) was partially supported and(H3a) was rejected

44 Discussion Usersrsquo activities provide objective informa-tion of web navigation behaviors and thus could complementusersrsquo subjective understanding of web pages The subjectiveunderstanding of web pages is usually elicited through cardsorting However card sorting is not adequate for elicit-ing complete mental models if we consider the directionalrelationship of information structures To get more objec-tive information from usersrsquo activities we proposed a newmethod called path diagram to elicit mental models withdirectional information of hierarchical systems To furtherquantify the difference between mental models mentalmodel similarity was calculated through the mathematicalequations It might be a quick and dirty way to predict userperformance In addition designers can get more preciseinformation about how users think about the system byapplying path diagram into the two phases of interactionprocess the designer-to-user communication phase and theuser-system interaction phase [60]

The mental model similarity has two major theoreticaland practical implications (1) the mental model similarityprovides an index to check if the designersrsquo improvement ontheir websites is effective which is quite different from whendesigner can only check if their improvement is workingby means of their feelings (2) the mental model similarityalso provides an index of measuring the usability of websitesPeople can get amore precise understanding of which kind ofwebsite was preferred by comparing mental model similarityof various websites

Hypothesis (H1a) was rejected and Hypothesis (H1b)was supported Many studies have shown that informationstructures are correlated with mental models For exampleGregor and Dickinson [61] thought that good mental mod-els could design good information structures Roth et al[62] pointed out that different information structures haddifferent mental models However hardly any studies haveinvestigated the relationship between information structuresand mental model similarity between users and designers

Here we have explored this relationship the results showedthat information structures had a significant effect on thedirectional similarity The more complex the informationstructures the lower the directional similarity

The results also indicated that information structureshad no significant effect on directionless similarity Previousstudies also indicated that card sorting lost its validity whendealing with the complex websites such asmunicipal websiteswhich are complex information structures [43]

Hypothesis (H2a) and Hypothesis (H2b) were rejectedThe results indicated that the directional similarity waspositively correlated with the task completion time and thenumber of clicks When the directional similarity was lowerthe participants took less time and smaller number of clicksto find the target This is different from the findings ofSchmettow and Sommer [43] they discovered that mentalmodel similarity between users and designers had no effecton usersrsquo browsing performance of municipal websites Thepossible reasons for this may be as follows (1) path diagraminvolves specific directional relationship between variouselements of an information structure while card sortingmainly represents a userrsquos understanding of the directionlessrelationship (2) culture difference might influence the wayusers are navigating in websites Chinese users will benefitfrom a thematically organized information structure of aGUIsystem whereas American users will benefit from a function-ally organized structure [63] Specifically in the card sortingtasks Chinese subjects were more likely to stress the categoryby identifying the relationship between different entitieswhile the Danish subjects preferred to stress the categoryname by its physical attributes [64] The participants in thestudy of Schmettow and Sommer were from Netherland andthe web pages used were functionally organized structurewhile the participants of this study were Chinese and thewebsites used were thematically organized structure Possibleinfluence of cultural difference might be considered in futureworkHowever the navigation strategy is systematic focusedand directed when individuals have specific targets or goals[65] Card sorting cannot describe the usersrsquo mental modelscompletely when it relates to information structures withdirectional relationship In other words a better way to elicit

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

12 Advances in Human-Computer Interaction

mental models was using path diagram rather than cardsorting In addition the method we proposed to elicit themental model is predictive

Demographic variables such as age spatial ability mem-ory capacity and technology product experience had nocorrelation with user performance This is quite different tothe findings of Arning and Ziefle [66] who indicated thatuser age and spatial ability were major factors affecting userperformanceThe possible reasons for this may be as follows(1) participants in this study were all younger students whileparticipants in the study of Arning and Ziefle [66] wereyounger and older adults (2) This study focused on webnavigation on computers while the study of Arning andZiefle [66] focused on menu navigation on Personal DigitalAssistants whose small screen makes navigation more chal-lenging and thus set higher requirements for spatial ability

Hypothesis (H3b) was partially supported and Hypothe-sis (H3a) was rejected The mediation effect of directionlesssimilarity on user performance was not found but a partiallymediated effect was found between directional similarityand user performance The results are different from thoseof Ziefle and Bay who thought that the more similar themental models between the user and designer the betterthe performance using the device [33] One possible reasonis that the tasks in the study of Ziefle and Bay [33] wereabout browsing tasks while they were searching tasks in thisstudy It is necessary to distinguish browsing without specificgoals and searching specific goals in future studies Anywayresults implied that designing hierarchical systems accordingto usersrsquo mental models was not the only way to solveproblems caused by the gap of mental models between usersand designers Alternatives such as providing navigation aidsin complex websites and training may be considered [67]

This study only considered individuals and the resultsmay not apply to groups where interaction with other peopleinfluences constructions of mental models Mathieu et al[24] found team processes fully mediating the relationshipbetween team mental models and team effectiveness How-ever their subsequent study [39] indicated that team perfor-mance was partially mediated by teammatesrsquo mental models

In addition the results showed that information struc-tures had a direct effect on user performance They seem totestify that ldquoa meaningful information structure will promoteefficient navigation to ensure that information is organized ina way that is meaningful to its target users is essential whendesigning websitesrdquo [68]

5 Conclusion and Future Research

We have discussed the impact of the mental model similaritybetween users and designers A new method path diagramwas applied to elicit mental models and calculate the similar-ity and comparably tested to the traditional method

Path diagram is more effective than card sorting in elicit-ing mental models of hierarchical systems particularly con-sidering the directional relationship of hierarchical systemsFor general information structures directionless similaritycannot predict user performance while directional similaritycan predict both task completion time and the number

of clicks Users will take less time and fewer clicks whenthe directional similarity is lower However for a specificinformation structure neither the directionless similarity northe directional similarity has a significant impact on userperformance

In addition user performance is not affected by theirage spatial ability memory capacity or technology productexperience The results have also shown that it is moregenerally effective in eliciting the mental model using pathdiagram compared to card sorting

Limitations of this study should be noted (i) participantswere sampling from young students who were not represen-tative Future studies may consider older adults and thosewith lower education level (ii) web pages with mixed infor-mation structures and various content were not considered(iii) multiple tasks (eg browsing tasks) were not involved(iv) possible impact of cultural difference was not examined(v) changes of mental models were not tracked over time (vi)similarity calculation required additional human efforts sofuture work may explore ways to automatically calculate it

Conflicts of Interest

The authors declare that they have no conflicts of interest

Acknowledgments

This work was supported by the National Natural Sci-ence Foundation of China under Grants nos 71401018and 71661167006 and Chongqing Municipal Natural ScienceFoundation under Grant no cstc2016jcyjA0406

References

[1] M Helander T Landauer and P Prabhu ldquoMental models andusermodelsrdquo inHandbook of Human-Computer Interaction pp49ndash63 Elsevier 1997

[2] Y Zhang ldquoUndergraduate studentsrsquo mental models of the Webas an information retrieval systemrdquo Journal of the Associationfor Information Science and Technology vol 59 no 13 pp 2087ndash2098 2008

[3] D A Norman ldquoDesign Rules Based on Analyses of HumanErrorrdquo Communications of the ACM vol 26 no 4 pp 254ndash2581983

[4] P N Johnson-Laird ldquoMental models and thoughtrdquo The Cam-bridge Handbook of Thinking and Reasoning pp 185ndash208 2005

[5] M Volkamer and K Renaud ldquoMental modelsgeneral introduc-tion and review of their application to human-centred securityrdquoin Number Theory and Cryptography pp 255ndash280 SpringerBerlin Heidelberg 2013

[6] K J W Craik ldquoThe nature of explanationrdquo CUP Archive vol445 1967

[7] PN Johnson-LairdMentalmodels Towards a Cognitive Scienceof Language Inference and Consciousness Harvard UniversityPress 6 edition 1983

[8] A Garnham and J Oakhill ldquoThe mental models theory of lan-guage comprehensionrdquo Models of Understanding Text pp 313ndash339 1996

[9] P N Johnson-Laird ldquoMental models and cognitive changerdquoJournal of Cognitive Psychology vol 25 no 2 pp 131ndash138 2013

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in Human-Computer Interaction 13

[10] J H Holland Induction Processes of Inference Learning andDiscovery Mit Press 1989

[11] P N Johnson-Laird ldquoMental models and deductionrdquo Trends inCognitive Sciences vol 5 no 10 pp 434ndash442 2001

[12] R A Schmidt D E Young S Swinnen and D C ShapiroldquoSummary Knowledge of Results for Skill Acquisition Supportfor the Guidance Hypothesisrdquo Journal of Experimental Psychol-ogy Learning Memory and Cognition vol 15 no 2 pp 352ndash359 1989

[13] R A Schmidt and R A Bjork ldquoNew Conceptualizations ofPractice Common Principles inThree Paradigms Suggest NewConcepts for Trainingrdquo Psychological Science vol 3 no 4 pp207ndash218 1992

[14] J I Tollman and A D Benson ldquoMental models and web-basedlearning examining the change in personal learning models ofgraduate students enrolled in an online library media courserdquoJournal of education for library and information science pp 207ndash233 2000

[15] I M Greca and M A Moreira ldquoMental models conceptualmodels and modellingrdquo International Journal of Science Edu-cation vol 22 no 1 pp 1ndash11 2000

[16] Y-F Shih and S M Alessi ldquoMental models and transfer oflearning in computer programmingrdquo Journal of Research onComputing in Education vol 26 no 2 pp 154ndash175 1993

[17] P Fuchs-Frothnhofen E A Hartmann D Brandt and DWeydandt ldquoDesigning human-machine interfaces to match theuserrsquos mental modelsrdquo Engineering Practice vol 4 no 1 pp 13ndash18 1996

[18] Y-CHsu ldquoThe effects ofmetaphors on novice and expert learn-ersrsquo performance and mental-model developmentrdquo Interactingwith Computers vol 18 no 4 pp 770ndash792 2006

[19] K M A Revell and N A Stanton ldquoCase studies of mentalmodels in home heat control Searching for feedback valvetimer and switch theoriesrdquo Applied Ergonomics vol 45 no 3pp 363ndash378 2014

[20] D A Norman The psychology of everyday things (The designof everyday things) 1988

[21] M D C P Melguizo and G C van der Veer ldquoMental modelsrdquoHuman-Computer Interaction Handbook pp 52ndash80 2002

[22] R Klimoski and S Mohammed ldquoTeam mental model Con-struct or metaphorrdquo Journal of Management vol 20 no 2 pp403ndash437 1994

[23] S Mohammed L Ferzandi and K Hamilton ldquoMetaphor nomore A 15-year review of the team mental model constructrdquoJournal of Management vol 36 no 4 pp 876ndash910 2010

[24] J E Mathieu G F Goodwin T S Heffner E Salas and J ACannon-Bowers ldquoThe influence of shared mental models onteam process and performancerdquo Journal of Applied Psychologyvol 85 no 2 pp 273ndash283 2000

[25] J R Olson and K J Biolsi 10 Techniques for representing expertknowledge Toward a general theory of expertise Prospects andlimits 1991

[26] S Mohammed R Klimoski and J R Rentsch ldquoThe measure-ment of team mental models We have no shared schemardquoOrganizational Research Methods vol 3 no 2 pp 123ndash1652000

[27] J Langan-Fox S Code and K Langfield-Smith ldquoTeam men-tal models Techniques methods and analytic approachesrdquoHuman Factors The Journal of the Human Factors and Ergo-nomics Society vol 42 no 2 pp 242ndash271 2000

[28] S J Payne ldquoA descriptive study of mental modelsrdquo Behaviour ampInformation Technology vol 10 no 1 pp 3ndash21 1991

[29] Y Zhang ldquoThe impact of task complexity on peoplersquos mentalmodels ofMedlinePlusrdquo Information ProcessingampManagementvol 48 no 1 pp 107ndash119 2012

[30] A L Rowe andN J Cooke ldquoMeasuringmental models Choos-ing the right tools for the jobrdquo Human Resource DevelopmentQuarterly vol 6 no 3 pp 243ndash255 1995

[31] Y Yamada K Ishihara and T Yamaoka ldquoA study on an usabilitymeasurement based on the mental model Access in Human-Computer Interactionrdquo Design for All and Einclusion pp 168ndash173 2011

[32] S Y Rieh J Y Yang E Yakel and K Markey ldquoConceptualizinginstitutional repositories Using co-discovery to uncovermentalmodelsrdquo in In Proceedings of the third symposiumon Informationinteraction in context pp 165ndash174 ACM 2010

[33] M Ziefle and S Bay ldquoMental models of a cellular phone menuComparing older and younger novice usersrdquo in Proceedingsof the International Conference on Mobile Human-ComputerInteraction pp 25ndash37 International Berlin Germany 2004

[34] R Young Surrogates and mappings two kinds of conceptualmodels for interactive 1983

[35] A DimitroffMental models and error behavior in an interactivebibliographic retrieval system dissertation [Doctoral thesis]1990

[36] M A Sasse Eliciting and describing usersrsquo models of computersystems dissertation [Doctoral thesis] University of Birming-ham 1997

[37] D J Slone ldquoThe influence ofmental models and goals on searchpatterns during web interactionrdquo Journal of the Association forInformation Science andTechnology vol 53 no 13 pp 1152ndash11692002

[38] D S Brandt and L Uden ldquoInsight intomental models of noviceinternet searchersrdquo Communications of the ACM vol 46 no 7pp 133ndash136 2003

[39] J E Mathieu T S Heffner G F Goodwin J A Cannon-Bowers and E Salas ldquoScaling the quality of teammatesrsquo mentalmodels Equifinality and normative comparisonsrdquo Journal ofOrganizational Behavior vol 26 no 1 pp 37ndash56 2005

[40] F G Halasz and T P Moran ldquoMental models and problemsolving in using a calculatorrdquo in Proceedings of the SIGCHI con-ference on Human Factors in Computing Systems pp 212ndash216ACM 1983

[41] C L Borgman ldquoThe userrsquos mental model of an informationretrieval systemrdquo in Proceedings of the 8th annual internationalACMSIGIR conference onResearch and development in informa-tion retrieval pp 268ndash273 ACM Montreal Canada June 1985

[42] S Payne ldquoMental models in human-computer interactionrdquo inTheHuman-Computer InteractionHandbook vol 20071544 pp63ndash76 CRC Press 2007

[43] M Schmettow and J Sommer ldquoLinking card sorting to brows-ing performancemdashare congruent municipal websites moreefficient to userdquo Behaviour amp Information Technology vol 35no 6 pp 452ndash470 2016

[44] S S Webber G Chen S C Payne S M Marsh and S J Zac-caro ldquoEnhancing team mental model measurement with per-formance appraisal practicesrdquo Organizational Research Meth-ods vol 3 no 4 pp 307ndash322 2000

[45] B-C Lim and K J Klein ldquoTeam mental models and teamperformance A field study of the effects of team mental modelsimilarity and accuracyrdquo Journal of Organizational Behaviorvol 27 no 4 pp 403ndash418 2006

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

14 Advances in Human-Computer Interaction

[46] T Biemann T Ellwart and O Rack ldquoQuantifying similarity ofteammental models An introduction of the rRG indexrdquo GroupProcesses and Intergroup Relations vol 17 no 1 pp 125ndash1402014

[47] C Wu and Y Liu ldquoQueuing network modeling of driver work-load and performancerdquo IEEE Transactions on Intelligent Trans-portation Systems vol 8 no 3 pp 528ndash537 2007

[48] R N Shepard and C Feng ldquoA chronometric study of mentalpaper foldingrdquo Cognitive Psychology vol 3 no 2 pp 228ndash2431972

[49] S Werner B Krieg-Bruckner H A Mallot K Schweizer andC Freksa ldquoSpatial cognition The role of landmark route andsurvey knowledge in human and robot navigationrdquo in Infor-matikrsquo97 Informatik als Innovationsmotor pp 41ndash50 SpringerBerlin Germany 1997

[50] R G Golledge T R Smith JW Pellegrino S Doherty and S PMarshall ldquoA conceptual model and empirical analysis of child-renrsquos acquisition of spatial knowledgerdquo Journal of EnvironmentalPsychology vol 5 no 2 pp 125ndash152 1985

[51] E K Farran Y Courbois J Van Herwegen and M BladesldquoHow useful are landmarks when learning a route in a virtualenvironment Evidence from typical development andWilliamssyndromerdquo Journal of Experimental Child Psychology vol 111no 4 pp 571ndash586 2012

[52] MNys V Gyselinck E Orriols andMHickmann ldquoLandmarkand route knowledge in childrenrsquos spatial representation of avirtual environmentrdquo Frontiers in Psychology vol 6 article no522 2015

[53] D Sinreich D Gopher S Ben-Barak Y Marmor and R LahatldquoMental models as a practical tool in the engineerrsquos toolboxrdquoInternational Journal of Production Research vol 43 no 14 pp2977ndash2996 2005

[54] MGarc AN Badre and J T Stasko ldquoDevelopment and valida-tion of icons varying in their abstractnessrdquo Interacting withComputers vol 6 no 2 pp 191ndash211 1994

[55] T J Lloyd-Jones and L Luckhurst ldquoEffects of plane rotationtask and complexity on recognition of familiar and chimericobjectsrdquoMemory amp Cognition vol 30 no 4 pp 499ndash510 2002

[56] M ButkiewiczHVMadhyastha andV Sekar ldquoUnderstandingwebsite complexity Measurements metrics and implicationsrdquoin Proceedings of the 2011 ACM SIGCOMM Internet Measure-ment Conference IMCrsquo11 pp 313ndash328 deu November 2011

[57] P Chandra andGManjunath ldquoNavigational complexity in webinteractionsrdquo in Proceedings of the 19th international conferenceon World wide web pp 1075-1076 ACM 2010

[58] D P MacKinnon C M Lockwood J M Hoffman S G Westand V Sheets ldquoA comparison of methods to test mediation andother intervening variable effectsrdquo Psychological Methods vol 7no 1 pp 83ndash104 2002

[59] M E Sobel ldquoAsymptotic confidence intervals for indirect effectsin structural equationmodelsrdquo SociologicalMethodology vol 13pp 290ndash312 1982

[60] C S De Souza and C F Leitao ldquoSemiotic engineering methodsfor scientific research in HCIrdquo Lectures on Human-CenteredInformatics vol 2 no 1 p 122 2009

[61] P Gregor and A Dickinson ldquoCognitive difficulties and accessto information systems An interaction design perspectiverdquoUniversal Access in the Information Society vol 5 no 4 pp 393ndash400 2007

[62] S P Roth P Schmutz S L Pauwels J A Bargas-Avila and KOpwis ldquoMental models for web objects Where do users expect

to find the most frequent objects in online shops news portalsand company web pagesrdquo Interacting with Computers vol 22no 2 pp 140ndash152 2010

[63] P L P Rau T Plocher and Y Y Choong Cross-Cultural Designfor IT Products and Services CRC Press 2012

[64] A Nawaz T Plocher T Clemmensen W Qu and X SunldquoCultural differences in the structure of categories in Denmarkand ChinardquoDepartment of Informatics CBS vol article 3 2007

[65] G Marchionini Information Seeking in Electronic Environ-ments Cambridge University Press Cambridge UK 1995

[66] K Arning andM Ziefle ldquoEffects of age cognitive and personalfactors on PDA menu navigation performancerdquo Behaviour ampInformation Technology vol 28 no 3 pp 251ndash268 2009

[67] J Park and J Kim ldquoEffects of contextual navigation aids onbrowsing diverse Web systemsrdquo in Proceedings of the SIGCHIconference on Human Factors in Computing Systems pp 257ndash264 ACM 2000

[68] M L Bernard and B S Chaparro ldquoSearching within websitesA comparison of three types of sitemap menu structuresrdquo inIn Proceedings of the Human Factors and Ergonomics SocietyAnnual Meeting vol 44 pp 441ndash444 Los Angeles Calif USA2000

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Submit your manuscripts athttpswwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Distributed Sensor Networks

International Journal of

Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014

International Journal of

ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Electrical and Computer Engineering

Journal of

Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia

International Journal of

Biomedical Imaging

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Advances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 201

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Computational Intelligence and Neuroscience

Industrial EngineeringJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014