sequential analysis for learning modes of browsingfaculty.bscb.cornell.edu/~hooker/slam.pdf ·...
TRANSCRIPT
Sequential Analysis for Learning Modes of Browsing
Giles HookerDepartment of Statistics
Stanford UniversityStanford, CA, 94305
Matthew FinkelmanDepartment of Statistics
Stanford UniversityStanford, CA, 94305
ABSTRACTIt is well-known that different users navigate websites dif-
ferently, being more or less inclined to browse or search andso forth. It is also very likely that the same user will ex-hibit different behaviors at different times - looking for aparticular item one time, and browsing without a great dealof direction another. Knowing the type of behavior a userexhibits in a session would allow a website to tailor the in-formation it displays to that behavior, and even to affect thebehavior being displayed.
We present a mathematical framework in which we di-rectly try to learn a user’s mode of browsing during a givensession. This framework is inspired by sequential analysis inthe setting of educational testing. We demonstrate its fea-sibility and utility in the context of click-stream data andexplore the range of models and variations that this frame-work makes available.
Categories and Subject DescriptorsG.3 [Probability and Statistics]: Nonparametric statis-
tics; I.5.2 [Pattern Recognition]: Models; I.6.5 [Simulationand Modeling]: Model Development—Modeling Method-ologies
General TermsAlgorithms, Measurement, Design, Experimentation
KeywordsWeb Mining, Browsing, Types of Use, Classification, Se-
quential Analysis, Utility Maximization
1. INTRODUCTIONThe development of the World Wide Web in commerce
has allowed for increasingly large amounts of automateddata collection, providing an enormous repository of con-sumer behavioral data. Many companies in e-commerce nowmaintain huge data sets of customer behavior, allowing for a
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.KDD’04, August 22–25, 2004, Seattle, Washington, USA.Copyright 2004 ACM 1-58113-888-1/04/0008 ...$5.00.
very deep empirical analysis of this behavior and providingsignificant evidence even for very small effects.
User click-streams on websites, combined with originat-ing IP addresses and demographic data, allow companiesto develop sophisticated profiles of users, enabling them topresent specific products or information tailored to the his-torical interests of a known user. However, little attentionhas been paid so far to different modes of use by a singleuser. There is a general belief, based in part upon anecdotalevidence, that users may at different times use a company’swebsite in qualitatively different manners. They may belooking for a single product, either to buy directly or fora price comparison, they might be generally interested insome collection of products and are looking for more infor-mation, or they may merely be killing time. Knowing theparticular mode of browsing that a customer is employing ina given session allows a company to provide a more refinedset of information or advertisements to that user. Someoneinterested in a particular product is unlikely to be distractedby very different products, but might respond well to a spe-cial offer. On the other hand, those browsing a website atrandom, while unlikely to purchase, offer a company theopportunity to raise their awareness of product lines thatthey have not viewed or services that they have not used. Acompany may also be able to increase click-through rates foradvertising and product placement for users showing partic-ular modes of browsing. Obviously, quickly establishing themode of use enables a website to present appropriate mate-rial sooner, providing greater user exposure to informationtargeted to their behavior.
In this paper, we develop a methodology for learning themodality of use in a particular session through the machin-ery of sequential analysis. Educational testing, a main areaof application for this branch of statistics, bears some simi-larity to the problems companies face in providing individ-ualized webpages. Both allow an interaction between theoptions presented and the user selection of those options.In the former case, these are questions and correct or incor-rect answers, and we are interested in determining a user’sability. In the latter, the space of options comprises possi-ble actions that the user can take, and we are interested indetermining a classification of the mode of browsing.
In this paper we will provide an introduction to sequen-tial analysis in the setting of educational testing in §2. Wewill then develop a statistical framework to translate thisinto an analysis of user modalities on webpages in §3.§4 and§5 will discuss how to formulate sequential models for spe-cific websites. We will present an initial simulation study in§6. §7 will then cover strategies of website design to min-
imize time-to-classification, reassessing classification accu-racy, and more sophisticated techniques for using this modelin choosing information or advertising to present.
2. SEQUENTIAL ANALYSIS AND THE THE-ORY OF TESTING
There is a long and well-developed statistical literaturein sequential analysis (c.f. Wald, 1947; Siegmund, 1985).This section is designed to provide an illustration of themethodology with respect to the field of educational testing,which we believe provides a useful analogy to user behaviorin web browsing.
Consider a simplified situation in which subjects are beingtested for adequate mastery of some field. A test is admin-istered that consists of N questions. Each question Qi willhave probability p1
i of being answered correctly by mastersand p0
i by non-masters. For a given subject the results ofthe test are recorded in an N -vector U with entries ui = 0or ui = 1, respectively, for incorrect or correct answers toquestion Qi. Suppose that we have administered k of theN questions to the subject, and therefore have access to thefirst k entries in U . The likelihood of mastery is then givenby
p1(u1, . . . , uk) =
k∏i=1
(p1i )
ui(1− p1i )
1−ui ,
with the product on the right hand side being taken un-der the assumption of independence between test answers.Similarly, the likelihood of non-mastery can be written as
p0(u1, . . . , uk) =
k∏i=1
(p0i )
ui(1− p0i )
1−ui .
The result of this is a time series of two likelihoods astesting proceeds. After N questions, we classify the studentas a master if and only if p1(U)/p0(U) ≥ C, where C is aconstant chosen to have the appropriate false positive andfalse negative rates.
Given a common test, this time series is of no particu-lar value since we base a decision on the ultimate result ofN responses. The testing scenario is somewhat simplified,however. Typically, the probability of success, pi, is indexedby an innate measure of ability, θ, and we are trying todetermine whether the student possesses adequate exper-tise (θ > θ0) or to reduce the variance of our estimate ofθ. In this case, the time series can be used either to selectquestions with the most discriminative power about θ (c.f.Chang & Ying, 1996; Owen, 1975), or to curtail the test oncewe have established sufficient confidence (via the likelihoodratio) in expertise or our estimate of θ (Finkelman, 2003).Within the framework of user behavior for a webpage, webelieve that a sequential approach may be profitable, evenwithout attempting to maximize the discriminative powerof the information displayed.
3. A STATISTICAL FRAMEWORK FORLEARNING USER MODALITIES
In educational testing, the goal is to determine the valueof θ, an index of ability. The probabilistic setup for usermodalities will implicitly use a discrete θ to index a set ofmodalities {ki}K
i=1. We will suppress θ in favor of i. At the
tth page visited , there is also a set of possible actions on thecurrent webpage {at
j}Nj=1, defined by possible clicks. We can
regard this set of actions as being “answers” with the currentwebpage representing a “question” about the user’s mode ofbrowsing. Since only one click is possible for any page, eachmodality ki will define a multinomial distribution over theset of possible actions, giving probability pt
ij to action atj
when in modality ki.At this point the general machinery of sequential anal-
ysis can be employed. We believe it reasonable to makea Markov-type assumption that a user’s actions at a givenwebpage and given a particular mode of browsing, will beindependent of the previous pages visited. Under this as-sumption, the log-likelihood of the click-stream Xt undermodality ki is given by
Lki (Xt) =
t∑s=1
log(psij), (1)
with Xt = {asj}t
s=1, the click-stream until current time t.In addition to the user’s current click-stream, websites typ-ically collect a signature for customers, which can incorpo-rate information about the frequency with which each useremploys a particular browsing modality. In the absence ofsuch information for a particular user, aggregate informa-tion about browsing types over the population as a wholecan be collected, and the current user may be regarded asbeing randomly sampled from that population. Thus eachmodality ki has a prior probability ξ0
i that can be incor-porated to give us a posterior probability1 for ki at timet:
ξti = P (ki|Xt) =
ξ0i
∏ts=1 ps
ij∑kl=1 ξ0
l
∏ts=1 ps
lj
.
This information could then (somewhat naively) be used,with a cut off at P (ki|Xt) > α, to regard the user as ex-hibiting modality ki once this confidence has been reached.From that point on, the administrator could present the userwith material tailored specifically to that modality. The restof this paper will explore the power of this statistical frame-work and more sophisticated uses of this information.
4. SEQUENTIAL ANALYSIS IN A WWWFRAMEWORK
Although outside the mathematical framework of this pa-per, the development of the models for ps
ij needs to be dis-cussed and we will provide an initial methodology to do so.This will also present a motivation for the simulation studypresented in §6.
We will begin by developing a representation of a genericwebsite. Standard e-commerce sites consist of hundreds ofpages. In fact, there is potentially an infinite number ofpages, given the ability to generate new pages tailored to thedemands of a customer. Each page will likely contain manypotential actions. A survey of the Amazon.com homepage,for example, found 184 hyperlinks. While the large volumeof Internet traffic allows the basic rates of clicks on each ofthese page-action pairs to be estimated with high confidence,the extreme dimensionality of such a representation becomes
1The likelihood framework above implicitly uses posteriorξ0
i = 1/K.
problematic when we wish to consider user sessions thatoften consist of a very small number of clicks.
The pages on most e-commerce sites can be readily groupedinto identifiable categories {Wm}n
m=1: a main page, prod-uct pages, search results and so forth. Each page within acategory may display different links, and even different num-bers of links, and we will therefore also cluster these linksinto predetermined categories {Am
j }nmj=1: searches, brows-
ing links, links to products, advertisements, and, of course,leaving the site. We can now represent a user-session bythe frequency of clicks on each of the
∑nm=1 nm page-type-
link-type pairs. User-sessions are comparable in this lower-dimensional representation and we can perform a clustering.This should be done in a semi-directed manner since appro-priate webpage material must be identified for each cluster.
This clustering can most effectively be done via an EMalgorithm (Demptser et. al. (1977)), although a simple hi-erarchical clustering might be used to determine the numberof clusters and initialize cluster assignments. The algorithmis then very simple:
Algorithm 4.1. Initialize: Data set
{Xl = {a1l , . . . , a
Tll }}M
l=1
with initial cluster probabilities for each session {ξ01l, . . . , ξ
0Kl}M
l=1.Iterate until Convergence:
1. ξ0i =
∑Ml=1 ξ0
il/M , i = 1, . . . , k
2. pmij =
∑Ml=1
∑Tls=1 ξ0
il1(asl =Am
j )∑nmj′=1
∑Ml=1
∑Tls=1 ξ0
il1(as
l=Am
j′ ),
i = 1, . . . , K, j = 1, . . . , nm, m = 1, . . . , n.
3. ξ0il =
ξ0i
∏Tls=1 ps
ij∑kl=1 ξ0
l
∏Tls=1 ps
lj
,
i = 1, . . . , K, l = 1, . . . , M
Output: {ξ0il, i = 1, . . . , K, l = 1, . . . , M}, {ξ0
i }Ki=1,
{pmij , j = 1, . . . , nm, m = 1, . . . , n, i = 1, . . . , K}.
We successively estimate the prior probabilities of each modeof browsing, ξ0
i , and the multinomial distributions pslj by
aggregating the posterior probabilities of each session. Thisthen allows us to re-calculate the posterior probabilities basedon the new parameters and the procedure is iterated. Itcan be shown that each iteration increases the likelihood ofthe observed data given the parameter estimates. Both thisand Algorithm 7.1 have been written out for clarity of un-derstanding rather than numerical stability, which can beimproved by simplifying some calculations and working ona log scale.
The total frequency counts for each page-type-link-typepair in each cluster will then provide estimates for ps
ij in (1).Correspondingly, the relative sizes of the clusters themselvescan provide an initial estimate for the priors ξ0
i .In order to direct this clustering, we can also make use of
known examples {X0l }M0
l=1 whose cluster assignments {ξ0il}M0
l=1
are not updated in the final step of Algorithm 4.1. Thesecan be chosen as examples where we are confident of theirclassification and serve as archetypes to guide the clusters.They also help to provide an interpretable set of clusters sothat appropriate webpage material can be designed for eachcluster.
Naturally, a classification of sessions for a single user canmake a refinement on the prior ξ0
il which can be updated in
a signature for that user as more session data are collected.Throughout the simulations below and the rest of this paper,we will assume that these clusters and probabilities havealready been estimated satisfactorily.
The representation developed above makes use of the Markovassumption of independence of past click-stream history. Itwould be possible to build in a time-dependent representa-tion of a user-session and to modify the likelihood (1).
5. MAKING USE OF MODALITIESThe above procedures have focused on an eventual cor-
rect determination of a user’s modality. This is of no useto the webpage by itself. However, there may be utility inproviding specific types of information to the user. A web-site’s choice of information will be constrained to a largeextent - there must be links to the home page, a searchpage and browsing pages, the information requested by theuser and a host of other constraints including general layoutand design. Those parts of a webpage that are mutable willtherefore largely consist of the set of advertisements shownto the user. More complex situations are imaginable here- the order of search results, page layout and so forth maybe important - but this will be highly dependent on specificwebsite content. In general, there is a notion of differen-tial utility for displaying certain information to a user, andthis utility may depend upon the user’s modality: it maybe advantageous to display a broad range of products to atime-killer in order to raise awareness, whereas somebodyresearching a particular product may be better served byadvertising accessories, similar items or discounts on thatproduct. Time-killers may also be targets for external ad-vertising; being unlikely to make a website money by directpurchases, they can earn a website click-through revenueand knowing the type of behavior a user is displaying canhelp determine whether a website should advertise its ownservices or a competitor’s.
Let Uei be the utility derived from showing information2
e to a user with true modality i. We would like to maximizethe total utility
∑Ts=1 Us
ei of the information displayed overthe total length T of the user’s session. An initial approachis a straightforward greedy algorithm: at click t, select theinformation e∗t that maximizes the expected utility for thenext page only:
e∗t = arg maxe
K∑i=1
ξtiUei. (2)
Of course, this may be made more sophisticated with look-ahead algorithms. It may also be advantageous to tailor theinformation presented to differentiate modality types. Thiswill be considered in a later section.
6. A SIMULATION STUDY
6.1 Estimating a ModelFor the purposes of exploring the effectiveness of this
mathematical framework, we will produce a simulation basedon a simplified version of the model described in §4. Our
2The use of e here follows Chernoff (1972) to designate an“experiment” designed to maximize discriminative powerbetween alternatives. This is a notion that we will return toin §7.
Front Page
prob
abili
ty
0.0
0.4
0.8
Type1 Type2 Type3 Type4
Browse Page
prob
abili
ty
0.0
0.4
0.8
Type1 Type2 Type3 Type4
Product Page
prob
abili
ty
0.0
0.4
0.8
Type1 Type2 Type3 Type4
Search Results
prob
abili
ty
0.0
0.4
0.8
Type1 Type2 Type3 Type4
Figure 1: Multinomial distributions for the set ofpossible clicks on each page type induced by modal-ity. Distributions have been grouped into graphsby page type. Within each graph the bar plots aregrouped by modality. Bar heights give the proba-bility distributions for each mode in the order Exit,Search, Browse, Product, Advertisement
website will have four page types: a front page, browsingpages, product pages, and search result pages. Each page-type will have the same set of possible link-types: search(which results in a search-result page), browse (resulting ina browsing page), product-link (resulting in a product page),advertisement (resulting in a browsing page) and leave, whichends the session. For simplicity, we have dispensed withlinks directly related to the process of shopping: addingproducts to a shopping cart, checkout pages and so forth.
The set of users that we have will be divided into fourcategories: time killers, targeted browsers, product groupresearchers and searchers for a specific product. These cor-respond roughly to increasing specificity in the set of infor-mation of interest. The probabilities induced by each modal-ity on each webpage are graphed in Figure 1. Roughly, asthe information of interest becomes more specific, the usersare more likely to perform searches and less likely to viewadvertisements. A uniform prior for each modality has beenused throughout.
We simulated 10,000 user-sessions equally divided betweenmodality types. After 50 clicks, all sessions had ended dueto exit. A dendogram resulting from a complete-linkagehierarchical clustering of 10% of these sessions using therepresentation above is given in Figure 2. Here, four sep-arate groups are apparent. Assigning each cluster to theappropriate modality type produced a correct classificationrate of 83.4%. The squared distance between the set of80 true parameters and those estimated with this cluster-ing was 0.00074. For comparison, a random assignment toclusters produces an expected estimated parameter vectorwith squared distance 0.009 from the truth. Using the es-timated clusters on this 10% sample as fixed, archetypal
0.0
0.2
0.4
0.6
0.8
Initial Session Clustering
hclust (*, "complete")Sessions
Hei
ght
Figure 2: A dendogram for a complete-linkage hi-erarchical clustering of 1000 user sessions. Fourmodality types are clearly visible.
examples and employing Algorithm 4.1 on the full set ofsessions, with random cluster assignments for the remaining90%, reduced the squared error in the parameter vector to0.00023. In doing this, we maintained the incorrect clusterassignments for the archetypal clusters in recognition thatthis is likely to happen in a realistic setting. The results ofthe EM algorithm indicate that these estimates are robustto a reasonable error rate in archetypes.
6.2 Posterior ProbabilitiesWe then used the parameter estimates obtained from the
procedure above to evaluate posterior probabilities and util-ity scores on a further 10,000 sessions generated from theoriginal process. The results show that the posterior prob-abilities very quickly tend to place high probability masson the correct modality type, suggesting that this approachmay provide some advantage even for relatively short ses-sions. Figure 3 provides plots of the posterior distributionsbroken down by true modality type at clicks 5 and 15. Wehave only included sessions that have not been terminatedby those times.
For a more detailed view of the probabilities, Figures 8and 9 provide time-series box plots for each time step for theposterior probability of each mode of browsing for each truemode of browsing. The model has difficulty distinguishingType 3 browsers from the relatively similar Type 2. This isexacerbated by high exit probabilities for Type 3, makingthem easier to distinguish once the session is terminated –something that may not be useful within a session but canstill help to update prior probabilities for type of browsingfor an individual user. Overall, we are very quickly able toprovide an accurate classification of mode of browsing.
6.3 Utility MaximizationWe have noted in §5 that there is little point in calculating
the modality of a user-session without making use of that
After 5 clicks
●●●●●●●●●●●●●●
●●●
●●●●●
●●●●●●●●●●●●●●●
●●●
●
●●
●
●●●
●●●
●●
●
●
●●
●
●
●
●●
●
●●
●
●●●
●●●●
●●●
●
●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●
●
●
●
●●
●
●●●●
●
●●●
●●●●
●
●
●
●
●●
●●
●
●
●
●
●●●●●
●●
●
●
●●●
●
●●
●●●●●
●
●●
●●●
X1 X2 X3 X4
0.0
0.4
0.8
Type1
Pos
terio
r P
roba
bilit
y
●
●●●●●
●●●●
●●●
●
●
●
●●●●●●
●●●●●●●●●●●
●
●●
●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●
●●
●
●
●
●●●●
●
●●
●
●●●●●●
●
●●
●
●●●●
●●
●
●●●
●●●●●
●●●●
●●
●
●●
●●●●●●●●●●
●
●
●
●●●●
●●●
●
●
●
●●●●●●●●
●
●●
●
●
●
●
●
●
●●●●●
X1 X2 X3 X4
0.0
0.4
0.8
Type2
Pos
terio
r P
roba
bilit
y
●●
●
●●
●
●
●
●●●●●●
●
●●●
●●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●●●
●
●
●●
●
●●●●●●●●●●●●●●●●●●●
X1 X2 X3 X4
0.0
0.4
0.8
Type3
Pos
terio
r P
roba
bilit
y
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●
●●
●
●
●●●
●●
●
●
●●●●●
●
●●
●
●
●●●
●
●●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●●
●●
●●
●
●●
●
●●●●
●●
●
●
●
●●
●
●●
●●
●
●
●●
●●
●
●●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●●●
●●●●●●●●
●
●
●
●
●
●
●
●●
●
●●●
●
●●
●
●●●●●
●●●●●●●●●●
●●
●●●●
●
●●●●●●●
X1 X2 X3 X4
0.0
0.4
0.8
Type4
Pos
terio
r P
roba
bilit
y
After 15 clicks
●●
●
●
●●
●
●
●●
●
●
●
●
●●●●●●●
●
●●
●●
●
●●●
X1 X2 X3 X4
0.0
0.4
0.8
Type1
Pos
terio
r P
roba
bilit
y
●
●●
●
●
●●
●
●
●
●
●●●●
●
●●●●
●
●
●●
●●
●●●●●●
●
●
●●●●
●
●
●
●
●●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●●●
●
●
X1 X2 X3 X4
0.0
0.4
0.8
Type2
Pos
terio
r P
roba
bilit
y
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
X1 X2 X3 X4
0.0
0.4
0.8
Type3
Pos
terio
r P
roba
bilit
y
●●
●
●●●●
●
●
●
●
●
X1 X2 X3 X4
0.0
0.4
0.8
Type4
Pos
terio
r P
roba
bilit
y
Figure 3: Posterior probability estimates for brows-ing modality after 5 clicks (top) and 15 clicks (bot-tom). Each plot is labeled by the true modality andthe posterior probabilities for each possible modal-ity are given in bar plots.
2 4 6 8
24
68
Utility Score By Item Choice and Modality
Advertisement IndexU
tility
Figure 4: Utility scores for each type of user-sessionagainst advertisement index. We have given averageutility across a uniform prior by the dashed line.
information, and we introduced the concept of utility fordisplaying particular information. Here we will examine theeffect of seeking utility-maximizing information in a simpli-fied setting. Let us suppose that there are 9 advertisements(or types of advertisements) that can be shown. Each ofthese advertisements is given a utility score between 1 and9 for each type of user. Figure 4 presents the utility scoresthat we have used. These have been chosen so that a naiverandom selection of advertisement has an expected utility of5 for each modality.
These advertisements and utility scores will be the samefor each page type. Further, we will assume that the ad-vertisements presented do not affect the multinomial distri-butions already described. We calculated the advertisementwith the maximum utility score as in (2) for each page-viewin the simulation above. The plots in Figures 5 through 7present the true utilities achieved by this strategy. We pro-vide a histogram of the final utility averaged over the ses-sion in Figure 5, a time series of cumulative mean utilitiesamong those sessions still active in Figure 6 and a graph offinal cumulative utility against stopping time for the sessionin Figure 7. A naive approach is to pick an advertisementat random each time (giving an expected utility of 5) and areference line for this is provided in each graph. It is clearthat we very quickly beat this target.
7. MORE SOPHISTICATED MODELSThe mathematical framework presented above considered
the multinomial distributions for each webpage and eachbrowsing modality to be fixed and independent of variationin the mutable part of the website. This is not only anover-simplification, it discards useful information that maybe incorporated into various strategies. In particular, wecan select information to display, not merely on the basisof a predefined utility score, but also on its ability to dis-
Mean Utility at End of Session
Mean Utility
Fre
quen
cy
2 4 6 8
050
010
0015
0020
00
Figure 5: A histogram of utility averaged over thelife of a session. Vertical lines give mean averageutility (solid) and expected utility under random se-lection (dashed).
●●●●●●
●●●●●●
●
●
●
●●●●
●
●
●●●
●●●●●
●
●
●●●●●
●●
●
●●
●●●●●●●
●●●●●●
●
●●
●
●
●●●●●●●●●●●●●
●
●
●
●●●●●●●●
●●
●●
●●●●
●●
●
●
●●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●●●●
●●●
●●
●
●●●●●
●
●
●
●
●●●●●●●●
1 4 7 11 15 19 23 27 31 35 39 43 47
24
68
Mean Utility Time Series
Mea
n U
tility
Figure 6: A time series of box plots of mean utilityup to 50 clicks with a horizontal line at the expectedscore for random selection.
●●
●
●●
●●
●●●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●●
●
●●
●
●
●●
●
●●
●
●●●
●
●
●
●
●●
●●● ●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●●
●●
●
●●
●●●
●●
●● ●
●●
●
●●●
●
●
●●
●●●
●
●
●●●●
●
●●
●
●
●
●● ●
●
●●
●●●
●
●
●●●
●●●
●
●
●
●
●●
●
●
●●●
●●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●●
●●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●●
●
●
●
●●●
●
●●
●
●
●●
●
●●
●
●●●
●●
●
●●●●
●
●● ●
●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●●●●
●
●
●
●
●●●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●
●●
● ●●
●
●
●●
●●
●●
●
●
●●
●
●●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●● ●
●●
●
●
●
●
●
●●
●●
●●
●
●●●
●
●
●●
●
●
●●
● ●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●●●
●
●
●
●●●
●
●
●
●●●
●
●● ●●●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●●●●
●●●
●
●●●●
●●
●
●
●
●●
●●
●
●●●●●●●
●
●
●●●●●
●●
●
●
●●
●●
●
●●●
●
●
●●
●
●
●
●●
●
●
●●●
●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●●●●
●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●●●
● ●
●●
●●
●
●●
●●●
●
●●
●
●
●
●●
●
●●●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●●●
●●
●●●
●
●●●
●●
●●
●●
●
●
●●
●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●●
●●●
●
●
●
●
●
●
●●
●
●
●●●
●
●●
●
●●
●●
●●●
●
●
●●●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●●
●
●●●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●
●●
●
●●●●
●●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●
●
●●●
●
●●●
●
●
●
●
●
●●●
●●
●
●●
●
● ●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
● ●●●
●●
●
●
●●●
●
●●
●
●
●●
●
●●
●
●
● ●
●
●●●
●●●
●
●
●
●
●
●●
●
●
●
●●●●
●
●
●
●●
●●●●
●
●
●
●●
●
●
●
●
●●
●
●●●●●
●
●
●
●●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●
●
●●●
●
●●
●
●●
●
●
●●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●●
●●
●
●
●
●
●●
●●
●
●
●●
●●
●
●
●●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●
●
●
●●●
●
●●
●●● ●●●
●
●
●
● ●
●
●
●●
●
●●●
●
●
●
●
●●
●●
●●●
●●
●●●
●
●●
●
●
●●
●
●●●●
●
●
●●
●
●●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●●●●
●●
●
●
●
●●●●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
● ●●
●
●
●●
●●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●●●
●
●●
●
●
●
●
●●
●
●
●
●●●
●●
●
●
●●
●●
●
●●●
●●
●●
●
●●
●
●●●
●
●
●
●
●●●
●
●●
●●
●
●
●
●
●
●
●●
●● ●
●
●
●●●
●
●●
●
●●● ●
●●
●●●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
● ●●●
●
●
●
●●
●●
●●●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●●
●
●●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●●
●●
●
●
●
●●
●●
●●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●
● ●●
●
●
●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●●
●●
● ●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●●
●● ●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●
●●● ●
●
●
●
●
●
●
●●
●●●●●
●
●
●●
●
●●
●●●
●●
●●
●
●
●
●
●
●●●●
●
●
●
●●●
●
●
●
●
●
●●●
● ●
●
●
●
●
●
●●
●●●●
●
●
●
●●
●
●
●●●
●
●
●●●●
●
●
●
●
●●●
●
●
●●
●●●
●
●
●●
●
●
●●
●●●
●●
●
●●
●●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●●●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●●
●
●●●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●●
●
●
● ●
●●
●●
●
●
●●●
●●●
●
●
●●
●
●
●●
●●●●
●
●●●●●
●
●
●
●
●●
●
●●
●●
●
●
●●●
●
●
●●
●●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●●●
●
●
●
●●●●●●
●
●
●●
●
●
●●●●●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●
●●●●
●
●
●
●●
●●
●
●
●●
●
●
●●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●●●●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●●
●
●
●
●
●●●●
●
●●●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●●
●●●
●
●●
●
●●●
●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●●
●●
●●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●
●●
●●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
● ●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●● ●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●●
●
●●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
● ●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
● ●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
● ●●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●●●
●
●●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●●●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
●●
●
●●
●
●
●●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
● ●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●
●
●●
●●●●●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●●
●●
●
●
●
●
●●●●●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
● ●●
●
●●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
●●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●●●●
●●
●
●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●● ●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●●●
●●●
●
●
●
●
●●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●
●●●
● ●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
● ●
●●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●
●
●
●●●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●
● ●
●
●
●
●● ●
●●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
● ●●
●●
●
●●●
●●
●
●
●
●●
●
●
●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
● ●
●●
●
● ●●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
● ●
●●●
●●●
●
●●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●●
●
●
●●
●
●
●●
● ●
●●
●
● ●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●●
●
●●
●●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
● ●
● ●
●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●●
●●●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●
●
●
●
●
●
●●●
●
●●
●
●●
●
●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●●● ●
●
●
●
●●
●
●
●●●
●●
●
●
●●
●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●●●
●
●●
●
●●
●
●
●
●
● ●
●
●
●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●●
●
● ●
● ●
●
●●
●●
●
●●
●
●
●
●●●
●
●
●●
●
●
●●●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●●
●
●
●●● ●●
●●
●
●
● ●●
●
●●●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
● ●
●●●
●
●●
●●
●
●
●
●
●
●
●●● ●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●● ●
●●
●
●
●
●
●
●●●
●●
●
●●
●
●●
●
●
●●●
●
●●
●
●
●
●●
●●●
●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●●
●●●●●
●●
●
●
●
●
●●●
●●
●●
●
●
●
●●
●
●●●
●●
●
●
●
●
●●●
●●
●
●
●
●●●●●●
●●
●
●
●
●●
●
●●
●●
●
●●●
●●
●
●
●●●
●
●
●
●
●
●
●●●
●●●
●
●
●●●
●●●
●
●
●
●
●
●
●●
●●●●
●
●● ●●●
●●
●
●
●
●●
●●
●●
●●
●●●●●
●
●●●
●
●
●
●● ●●●
●
●
●
●
●
●●●
●
●
●●●●●
●
●
●
●
●
●●●●●
●
●
●
●●●●●
●
●●●
●●
●●
●
●
●
●●
●●
●
●●●
●
●●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●●
●●●
●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●●
●
●
●
●●
●●
●
●●●●
●●
●● ●
●
●
●●
●●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
● ●
●●●
●
●
●
●
●
●●●
●
●
● ●●
●
●
●
●●
●
●
●●
●●●
●●
●●
●●
●
●
●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●
●●●
●●
●●
●
●●●●
●●
●
●●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
● ●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●●●●
●
●
●●
●●
●●
●●
●
●
●
●
●●●
●●
●
●
●●
●●
●
●
●
●●
●
●●
●
●
●●●
●●
●●
●●●
●
●
●●
●
●●
●
●
●
●
●●●
●●● ●●
●●
●
●
●●
●
●
●
●
●●●
●
●●
●●
●●
●●●●●
●
●●
●
●●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●●●●
●●●
●
●●
●
●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●●
●●●
●
●
●
●●
●
●●●
● ●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●●
●
●
●
●●●
●
●
●
●●●●
● ●●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●●●●
●
●
●
●●●
●●●
●
●
●●
●
●●
●
●●
●●
● ●
●●●
●●
●
●
●
●
●●●
●
●●●●
●
●
●
●
●●●
●
●●
●●●
●●●
●
●●
●
●
●
●●●●●
●
●●●●
●
●
●
●●
●
●
●●
●●●
●
●
●
●
●
●
●●●
●●
● ●●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
● ●●
●
●●
●
●
●
●●●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●●
●
●
●
●
●
●● ●
●
●●
●●
●
●
●
●
●●●●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●●
●● ●
●●
●
●
● ●
●
●
●
●
●
●●●●
●●
●
●●●
●
●
●
●
●
●
●●●●
●●
●
●
●
●●
●
●
●●●
●
●●●
● ●●●
●
●●
●
●
●
●● ● ●
●
●●
●
●●
●
●●
●
●
●●●
●●
●●
●
●●●●
●
●●
●
●●
●●
●●
●
●●
●
●●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●●●●
●
● ●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●●
●
●
●●
●●
●
●
●●●
●●
●
●
●●●
●
●●
●
●
●
●●●
●●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●●
●
●
●●●
●
●
●
●
●
●●●●
●
●●●
●
●
●
●
●
●
●
●●
●
●●●
●●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●●
●●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●●●●●
●
●●
●
●
●●● ●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●
●
●●●
●●●●●
●●
●●
●
●●
●
●
●●
●
●●●
●
●
●
●
●●●●
●●
●
●
●
●
●
● ●
●●●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●●●●
●●
●●●●
●●●
●
●
●
●●
●
●
●●●
●●
●●●
●
●
●●●
●●
● ●●●
●
●●●
●●
●
●●●
●
●●
● ●●●
●
●
●
●●
●
●
●●
●
●●
●
●
●●
●●●
●●
●
●●
● ●●●
●
●●
●● ●●●
●●●
●
●
●
●● ●
●●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●●
●
●●
●
●●
●
●●
●●
●
●
●●
●
● ●●
●●
●
●
●●●●
●●●
●
●
●
●
●●
●
●
●
●●
●
●
●●●●●
●
●
●
●
●●
●
●●
●
●●
●●
●●
●
●
●●
●●●
●●
●●●
●●
●
●●
●
●
●●●●●●
●●●●●●●●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
● ●
●●●
●●
●
●●●
●●
●
●●
●
●
●●●
●
●
●●
●●●●
●
●
●●●●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●●●
●
●
●●
●
●
●
●●
●
●●●
● ●
●
●●
●
●●●●
●
●●●
●
●
●●●
●
●
●●
●
●
●
●●●●
●
●
●●
●●
●
●
●●
●
●
●
●●●
●
●●●●
●●●
●● ●
●●
●● ●●
●
●
●●●●
●●●●
●●
●
●
●
●
●
●●●●●●
●
●●●●
●
●
●● ●● ●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●●●●●●
●
●●
●● ●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●●
●
●●
●
●●●
●
●
●●
●
●●
●●●
●●●
●
●
●
●
●●
●
●
●●●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●●●●●●
●
●
●
●●●
●
●
●●●
●
●●●●
●●● ●
●
●●●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●●
●●●●
●
●
●●●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●●●
●●●
●●
●
●
●●●
●
●
●●
●●
●●
●●
●
●●
●
●●●
●
● ●●
●●●
●
●
●●●
●
●
●
●●●●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●●●● ●
●●
●
●
●
●
●●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●●
●●
●●●
●
●
●●
●
●●●
●
●
●
●●
●●●
●●
●
●●● ●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●
●●
●
●●●
●●
●●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●
●
●●
●●●●
●
●●
●●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●●
●
●●
●
●
●●●
●
●
●
●●
●
●
● ●
●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●●
●
●●●●●
●
●●
●
●
●
●
●
●
● ●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●
●●
●●●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●●
●
●●●
●
●●●●●●
●
●
●
●
●
●
●●●●●
●●
●●●
●
●●●●●
●
●●
●●
●●●
●●
●
●●
●
●
●●
●
●●
●
● ●
●●
●
●
●●
●●●●●
●●
●●
●● ●●
●
●
●
●
●
●●●
●
●
●
●
●
●●●
●
●●
●
●
●●●
●●●
●●
●●
●
●●
●●
●●
●
●
●●
●
●
●
●●●
●
●●●
●●●
●●
●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
● ●
●●
●
●
●
●
●●
●
●●
●●
●●
●
●
●●
●●
●
●●●
●
●●
●
●●
●
●●●
●
●
●●
●●
●
● ●
●
●●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●●
●●
●●
●
●
●
●
●
●●●●
●
●
●●
●
●
●
●
●
●●
●●●●
●●
●●
●
●
●●
●
●
●●
●●
●●
●●
●
●
●
●
●
●
●●
●●●●●
●
●
●●
●
●
●
●●●●●
●●
●
●●
● ●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●● ●
●
●●
●
●
●
●●●
●
●●
●●●●
●
●●●●●
●●
●
● ●●
●
●
●●
●
●●● ●
●●●
●
●●●
●●
●
●
●
●
●●
●
●●●●
●●
●
●
●●●
●
●
●●
●●●
●●●
●●
●
●
●●
●●
●●
●
●
●●●
●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●●
●●
●●
●
●●
●
●●●●●
●●
●●
●●
●
●
●
●●●
●
●●●
●
●
●
●
●●●
●●
●
●●
●
●●
●●
●
●●
●●●
●●
●●●
●●
●
●
●●
●
●
●
●
●●
●
●●●
●
●
●
●
●●
●
● ●●
●
●
●●
●●
●
●●●
●
●
●●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●●
●●
●●
●●
●●●●●
●●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●●
● ●
●
●●
●●
●●
●
●●
●
● ●●
●●
●
●●●●
●●
●
●●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●●
●●●
●●
●●
●
●
●●● ●
●●●●●●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●●
●
●●●
●
●●
●●●●
●
●
●●
●●●
●
●
●●
●
●
●
●●●
●●
●
●
●●●
●
●
●
●●
●●●●
●
●
●● ●●
●●
●
●
●
●
●
●●
●
●●
●
●
●●●
●●
●
●●
●
● ●
●
●●●
●●
●●
●●●
●●
● ●
●
●●
●
●
●●
●●●
●●● ●
●
●●
●●
●
● ●●●
●
●●
●
●
●
●●
●●
●
●●
●● ●●●●
●
●●
●
●●
●
●
●
●●
●●●
●
●
●
●●
●●
●
●
●
●●●
●●
●
●
●
●
●
●●
●●
●
●●
●●
●
●
●
●●●
●
●●
●●
●
●●
●
●
●
●●●●●
●
●
●●●●
●
●●
●●● ●●● ●
●●
●
●
●●●●
●
●●
●
●● ●
●
●● ●
●
● ●
●
●●●
●
●●
●●
●
●
● ●
●
●●
●
●
●
●●
●
●●
●
●
●●
●
●●●●●
●
●
●
●
●
●
●
●●
●
●●
●●
●●●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
● ●●
●
●●●
●
●●●
●
●● ●●
●●
●
●
●●●
●●
●
●
●
●
●
● ●●
●
●
●●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●●●
●●●
●
●
●●●
●●●
●
●
●
●
●●●
●●
●●●
●
●●
●
●●
●
●
●●
●
●
●●●
●●●●●
●●
●●
●
●
●●
●●
●
●
●●
●
●
●
●
●●
●●
●●●●
●●
●
●
●●
●
●●
●
●
●
●●
●●●●
●
●●●●●
●●●●●● ●●●
●
●●●● ●
●
●●
●
●●
●
●
●●
●●
●
●●
●
●●
●
●●
●● ●
●
●
●
●
●
●●
●●
●●●●
●●
●
●
●
●
●●●
●●●●
●●
●
● ●
●
●
●
●●
●
●●●
●
●
●
●
●
●●●●●
●
●●●
●●
●
●
●●
●●
●●
●
●
●●
●●●●
●
●
●●●
● ●
●●
●●
●●
●
●
●●●●●
●
●
●
●●
●
●
●●
●●●
●
●●●
●
● ●●●●
●
●●
●●●
●
●
●●
●
●●
●
●
● ●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●●
●●
●●●
●
●
●
●
●
●
●●● ●●●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●●●
● ●●
●●
●
●●
●
●●
●●
●●
●
● ●●
●
●
●●
●
●
●●
●●
●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●●
●●
●
●●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●●
●
●
●●
●●●
●●
●
●●
●
●
●●
●
●
●
●●●
●●
●● ●●
●●
●●●
●
●
●●
●
●
●●●
●
● ●●
●
●
● ●
●●
●
●
●
●●●●
●●
●
●●
●●
●●
●
●●●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●
●●●●
●●●
●
●
●
●●●
●
●
●
●●●
●
●●●
●●
●
●●●●
●
● ●
●
●
●
●
●
●●●
●
●
●●
●●
●●●
●
●
●
● ●●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●●
●
●●
●●●●
●●
●
●●
●●
●●
●●
●
●
●
●●
●
●●●●
●
●●
●
●
●
●●
●
●
●●
●
●●●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●●●
●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●●
●●●●
● ●
●
●
●●
●●●
●
●●
●●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●●●
●
●●●
●
●●
●
●●
●
●
●
●●●
●●
●●
●●
●●
● ●●●
●●
●●
●
●
●
●●●●
●●
●●
●●
●
●
●
●● ●● ●
●
●
●●
●●
●
●
●
●
●
●●
●
●
●●●● ●
●
●
●
●
●
●
●
●●●●●● ●
●●
●●
●
●●●
●●●
●
●
●●●●● ●●
●
●
●
●● ●
●●●
●
●
●●●
●
●
●●●●
●
●●
●●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
● ●●
●
●●● ●
●●●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●●
●●
●
●
●●●
●
●
●
●● ●●
●●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
● ●
●
●
●
●●
●
●
●
●
●●
●●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●●● ●●
●●
●
●●●
●
●●
● ●●
● ●●●
●●
●●●●
●●
●
●●
●
●
●●●
●
●●●
●●
●
●●
●●●●●●●
●
●●●
●●
●
●●
●
●
●●●●
●
●
●
●● ●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●
●
●
●●●●
●
●
●
●●
●
●
●
●●
●●
●
●
●●
●●
●
●
●
●●
●●●
●
●
●
●
●● ●●
● ●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●●
●
●●
●
●
●●
●
●●
●●
●
●
●●
●●
●
●
●
●
●
●●
●●●●
●
●●
●●
●●
●●
●●●●●●●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●●●
●
●
●●●●●●●●
●
●●●●
●
●
●●●●
●
●●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●●●
●
●●●●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●
● ●
●
●
●●
●
●
●
●
●
●
●
● ●
●
●●●●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●●●
●
●
●●
●●
●
●
●
●
●
●
●
●●●●
●●
●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●●●●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●●●
●
●●●
●
●
●
●●●●
●
●
●
●
●
● ●
●
●
●●
●
●
●●
●
●
●
●●
10 20 30 40 50
010
020
030
040
0
Total Utility at End of Session
Stopping TimeC
umul
ativ
e U
tility
Figure 7: Cumulative utility plotted against stop-ping time, the line representing expected cumulativeutility using random selection.
criminate between users or to extend the life of the session.This section will consider a number of variations that can bemade to the basic framework above and outline the poten-tial richness and power of the sequential framework in thiscontext.
7.1 Model Specification and EstimationThe representation developed in §4 assumed that the multi-
nomial distributions that each modality ki induced on thelink-types of each page-type were independent of the spe-cific contents of those pages. This rather strong assumptionis necessary in order to produce a low-dimensional space inwhich user-sessions are comparable and can be clustered.Such an initial clustering is vital to producing a comprehen-sible set of clusters for which appropriate information canbe tailored.
That simplified representation is not necessary beyond theinitial clustering. Given a set of clusters, the number andsize of the multinomial distributions that can be estimateddepends only on the amount of data available. In the con-text of web usage, this can be enormous. Click-throughprobabilities are still estimated by counts as in Algorithm4.1 and we only require enough data to estimate these tohigh accuracy; 40,000 examples of a given type of browsingon a page are needed to estimate the parameters of its dis-tribution to within 0.01 with 95% confidence. We observethat in this context, Algorithm 4.1 can also be run withan expanded model to refine the initial cluster definitions.Where a sufficient volume of observations is unavailable, asfor a new advertisement, for example, the continual streamof user data makes direct experimentation possible to refineprobability estimates for a particular page.
Expanding our representation of a website has a num-ber of advantages. To begin with, we can gain considerablygreater information about a user’s behavior; somebody re-searching camcorders is likely to move from a product page
for one camcorder to another, but not from a camcorderproduct page to a clothing product page. Such a distinctionmight be lost on the initial, simplified, representation. Moreimportantly, at each click, a website has a choice of pagesto display. In an expanded representation, each of thosepages will have a different multinomial distribution associ-ated with it for each modality type. These distributions canalso be important factors in selecting which page to display.Different distributions will provide more or less informationabout the type of browsing displayed in a single session, canshorten or extend the expected length of a session or canincrease the probability of gaining revenue from advertis-ing click-throughs. We explore these possibilities and theirtrade-offs in the rest of this section.
7.2 Item Selection and DiscriminationIn a simple framework, the goal of this analysis is to esti-
mate which of K possible modes of browsing represents thetruth about a particular user-session. If we allow that theinformation presented to the user can change the probabilitydistribution of the next click, we can select pages that havehigh power to discriminate between user modalities. Thereis a large literature in the selection of informative data; forclassical examples in sequential analysis, see Bessler (1960);Box & Hill (1967); and Chernoff (1972). In this setup thereare multiple candidate “experiments” that are available tothe administrator at a given time. It is desired to selectthe experiment that will be maximally informative to thewebsite, based on the history of clicks so far observed in thesession.
Denoting a candidate website configuration as e, we willselect a “winning” candidate e∗t to present to the user afterthe tth click (e∗0 is the first page shown to the user, beforeany clicks have been made). Since the task is to figure out anindividual user’s modality, the mutable part of the websiteshould be designed to give as much information as possibleabout the user, conditional on their t previous clicks. Thereare many candidate scores for the information available inan experiment. We will employ the multinomial entropy(c.f. Box & Hill, 1967; Cover & Thomas, 1991) as being acommon and natural measure.
The entropy of a distribution with K possible states andassociated probabilities πi, i = 1, ..., K, is
H = −K∑
i=1
πi log πi. (3)
The minimum expected entropy metric chooses the avail-able experiment e∗t that gives the smallest expected entropyafter the (t + 1)th click, conditional on the t clicks alreadyobserved. Let Ht+1 denote the entropy of the posterior prob-
abilities ξ(t+1)i after the (t + 1)th click. Then
e∗t = arg mine
E(Ht+1|Xt, e)
= arg mine
E(−K∑
i=1
ξ(t+1)i log ξ
(t+1)i |Xt, e), (4)
where the expectation is taken with respect to ξti , the pos-
terior probabilities after t clicks, and supposing that e is thenext experiment chosen.
7.3 Maximizing Life of Session andClick-Through Revenue
For many commercial websites, there is an advantage tobe gained in extending the length of a user-session, bothin gaining more product exposure and in providing moreexposure for external advertisers3. A website may thereforewish to display information that decreases the probabilityof a user’s leaving. This can be done with varying degreesof look-ahead, from trying to minimize the probability ofexit at the next click, to encouraging a user to click ontosome other page that has a low probability of exit. Anappropriate strategy for life-of-session maximization wouldbe to use expected session length or some surrogate as partof the utility in the framework above.
One possible, easily calculable, surrogate is the probabil-ity of exit in the next n clicks, for modality ki this givenrecursively by
P (exit within n|ki, Xt) = ptid +∑
j 6=d
ptijP (exit within n− 1|ki, Xt ∪ {pt
id}) (5)
where d is the index of the exit click and
P (exit within 1|ki, Xt) = ptid
and the final probability is given by
K∑i=1
ξtiP (exit within n|ki, Xt).
This estimate also needs to incorporate the probability dis-tribution for the next page shown if the user does not exiton the next click. This will be affected by the page that wechoose to show. In this case the algorithm used to make thatchoice can easily be incorporated into the recursion above.
A variation on this is that many websites gain advertisingrevenue from having users leave their pages by clicking onadvertising. Another strategy is to try to maximize expectedrevenue in the next n clicks. This can be carried out as aboveby multiplying pt
id by Did – the payoff for having a userexhibiting modality i click on advertisement d – throughout.
These two strategies are mutually contradictory; wishingto reduce the probability of a user terminating the sessionand trying to maximize revenue when they do. Both scorescan appropriately be incorporated into an overall utility, thespecifics of which will depend on each particular website.
7.4 Discrimination and Utility Trade-offsIn some cases, the administrator may consider certain
types of error to be more grievous than other types. Forinstance, to conclude incorrectly that a user is merely brows-ing, when in fact the user is looking to buy a particular item,might be considered a worse mistake than the converse. Inthe former scenario, the administrator may not only missout on a possible sale, but also run the risk of frustratingthe user by making it difficult to find the desired item easily.In this case, a loss function may be introduced to quantifythe penalty for a certain conclusion that is different from
3The opposite is true of other websites - such as search en-gines - in which quick visits denote effective search resultsand a likelihood of return.
the true state of nature. Let L(i, j) be the loss in classifyinga user into modality j when the true modality is i. Thisintroduces a modification to the entropy score used to give
e∗t = arg mine
E
(−
K∑i=1
(
K∑j=1
L(i, j)ξ(t+1)i ) log ξ
(t+1)i |Xt, e
).
(6)It may well be the case that the set of information maxi-
mizing our information concerning a user’s modality is dif-ferent from that maximizing expected utility; in the simu-lations in §6, for example, exiting from a product page ishighly discriminatory for being in “search” mode, but maynot necessarily be desirable. There therefore is a need totrade-off these two quantities. One approach to this trade-off is to incorporate utility into the loss function used in (6).An appropriate, purely utility-based loss might be
L(i, j) = Ue∗j i − Ue∗i i
where e∗j is the optimal action to take for modality j. Thisequates simply to the utility lost by mistakenly classifying tomodality j if the truth is i. This is a greedy strategy, regard-ing the next click as ending the session and the utility forthe current page as the entire pay-off. Here, of course, otherfactors than a particular utility measure may also be incor-porated into the loss, and the utility itself may be composite,incorporating innate values such as consciousness-raising aswell as expected advertising revenue, expected life of ses-sion and expected utility over that life. These quantities arelikely to be highly dependent on the specific website.
An alternative compromise between information and util-ity would be to use one of the information-maximizing ap-proaches if the modality is quite unclear, and to use theutility-maximizing approach if the modality has become ap-parent. For example, choose e∗t to maximize information ifHt ≥ Ct, and choose e∗t to maximize utility if Ht < Ct. Forpurposes of generality, Ct may depend on t. It may also de-pend on the quality of experiments available at stage t, theexpected number of clicks for a given user, or the relativeimportances of accurate classification and utility accordingto the administrator.
7.5 Changes of ModalityThe above framework contains the assumption that a user
operates under a given modality for the entire length of asession. This may also be unrealistic - a user may begin bykilling time but become interested in some specific product;decide to switch products of interest; or become distractedby some advertising and forget their original intentions. Itmay therefore be useful for a website to try to estimatechanges in modality. This also provides potential for im-mediate sales maximization if the website can increase thechance that a browser turns into a buyer.
Supposing that the probabilities of changing modality af-ter click t are given by a K × K transition matrix Rt: Rt
ij
being the probability of changing from modality ki to modal-ity kj given the page displayed after click t. The posteriorprobability of being in mode i at time t is then given recur-sively by
ptil
∑Kj=1 Rt
jiP (j|Xt−1)∑Ki=1 pt
il
∑Kj=1 Rt
jiP (j|Xt−1),
with ptil being the probability of clicking on link l at time t
under modality i. This merely adds a single matrix multipli-cation at each click to the estimates defined in §3. We havewritten Rt to emphasize that the transition probabilities canchange depending on the page shown. For the remainder ofthe paper we regard Rm as being different for each webpage,although in practice this may need to be simplified.
Allowing such a process complicates the models signifi-cantly, and the procedures above need to be altered accord-ingly. We believe changes in user behavior to be rare eventsand therefore feel that the initial clustering in a page-type-link-type representation can remain unaltered. It is intendedto provide a guide to the number of clusters and to findingsession-type archetypes. If only a small proportion of ses-sions include a change of modality, they are unlikely to affecta broad-scale clustering.
A more demanding task is the re-estimation of model pa-rameters. In particular the entries of each Rm must be es-timated jointly with the multinomial distributions for eachwebpage and Algorithm 4.1 can no longer be employed. Theproblem can be made tractable by viewing it as a HiddenMarkov Model. We regard the sequence of modality typesfor each session l, {kt
l}Tlt=1, as being a simple Markov Chain
with transition probabilities at each time t given by {Rtl}
Tlt=1.
The observed click is then the “emission” of this chain, cho-sen with multinomial distribution {pt
ktlj}
ntj=1 corresponding
to the modality type ktl . All the parameters in the model
can then estimated with the following Baum-Welch algo-rithm (Durbin et. al. (1998)), modified to accommodatematrices varying with the webpages that are shown.
In this case the major addition to the calculations inAlgorithm 4.1 are estimates for the posterior probabilities{ξt
il, t = 0, . . . , Tl, l = 1, . . . , M, i = 1 . . . , K} of each sessionl being in state ki at time t. The added complexity of thecalculations requires an expansion of our notation. We willassociate ws
l ∈ {wm}nm=1 with the webpage, shown to ses-
sion l at time s: a parallel time series to Xt. We will alsouse pi(a
sl ) to denote the probability of the click as
l undermodality ki, based upon the current parameter estimates.We begin by assuming no transitions between states.
Algorithm 7.1. Initialize: Data set
{Xl = {w1l , a1
l , . . . , wTll , a
Tll }}M
l=1
with initial cluster probabilities for each session {ξ0il, . . . , ξ
0Kl}M
l=1
and initial multinomial distributions
{pmij , j = 1, . . . , nm, m = 1, . . . , n, i = 1, . . . , K}.
Take
Rmik = 1(i = k), i = 1, . . . , K, k = 1, . . . , K, m = 1, . . . , n.
Iterate Until Convergence:
1. ξ0i =
∑Ml=1 ξ0
il, i = 1, . . . , K.
2. Loop l = 1, . . . , M :
(a) f1il = pi(a
1l )ξ
0i , fs
il = pi(asl )∑K
k=1 Rslikf i−1
kl ,s = 2, . . . , Tl, i = 1, . . . , K.
(b) bTlil = ξ0
i , bsil =
∑Kk=1 Rs+1
lik pk(as+1l )bs+1
kl ,s = 1, . . . , Tl − 1, i = 1, . . . , K.
(c) ξsil =
fsilb
sil∑
ξ0i f
Tlil
,
s = 1, . . . , Tl, i = 1, . . . , K.
3. pmij =
∑Ml=1
∑Tls=1 ξs
il1(asl =Am
j )∑nmj′=1
∑Ml=1
∑Tls=1 ξs
il1(as
l=Am
j′ ),
j = 1, . . . , nm, m = 1, . . . , n, i = 1, . . . , k.
4. Rmik =
∑Ml=1
∑Tl−1s=0 ξs
ilξs+1kl
1(wsl =wm)∑K
k′=1
∑Ml=1
∑Tl−1s=0 ξs
ilξs+1
k′l 1(wsl=wm)
,
m = 1, . . . , n, i = 1, . . . , K, k = 1, . . . , K
Output: {Rmik, m = 1, . . . , n, i = 1, . . . , K, j = k, . . . , K},
{pmij , j = 1, . . . , nm, m = 1, . . . , n, i = 1, . . . , K},
{ξ0i , i = 1, . . . , K} .
Here the recursive calculations undertaken in Step 2 arenecessary to calculate the posterior probability of each modal-ity for each click in each session, adding significantly to thecomputational complexity. Once these have been calculated,estimates of the multinomial probabilities pm
ij and transitionprobabilities Rm
ik are the standard, naive quantities based onthe posterior probabilities ξt
il.In this case, the set of parameters has been significantly
expanded, especially when we allow the transition proba-bilities of the Markov Chain to vary. This increase in di-mensionality allows for significantly greater variance in pa-rameter estimates. However, within a web mining context,the abundance of data should allow the whole model to beestimated well. Of somewhat more concern would be theincreased computational cost of the modifications to thelook-ahead quantities (5), which may have to be curtailed inscope. The benefit of the scheme is that if the page displayedmodifies the transition probabilities, R, a website can seekto display material more likely to turn a user into a browser,or more likely to generate click-through revenue.
8. CONCLUSIONSIn this paper we have introduced a sequential analysis
framework in which to estimate types of user behavior withina user-session, and we have tried to indicate the wealth, va-riety and power of models available within this framework.For a small simulation study in the simplest case, we haveshown that estimated types of behavior quickly convergetoward providing a correct classification and that these pos-terior probabilities can successfully be employed to provideutility scores that are superior to those obtained by display-ing information at random.
This paper has not discussed the selection of those util-ity scores, believing this to be the proper concern of par-ticular websites. Our purpose has been to demonstrate theframework for this approach and we have therefore only beenmarginally concerned with issues such as estimating the pa-rameters in the model. These may be quite numerous, es-pecially if we allow different types of information to changethem. We have shown the EM algorithm, together with thevery large volume of Internet traffic and a website’s abilityto run real-time experiments, provides a direct estimationfor these.
We have provided a basic programmatic framework forcreating a type-of-browsing model for web usage that canbe summarized as
1. Produce a small-scale model of the website based onpage-type-link-type pairs.
2. Use click rates on this model to initially cluster indi-vidual sessions, decide on an appropriate number of
clusters, interpret the type of behavior that they ex-hibit and select a set of archetypal examples to guidecluster refinements.
3. Refine the clusters chosen in the larger page-link modelof the website using the EM algorithm and use this toalso provide a set of multinomial distributions for eachpage-modality pair.
4. Establish utility scores for each candidate page to showwhen a choice of pages is available.
5. Perform the sequential analysis with item selection asoutlined above.
Beyond this basic model, we can represent possible changesin modes of browsing through a Hidden Markov Model,which, once estimated, can be easily incorporated into thissequential analysis, allowing for far more elaborate utilityscores, but also allowing a website greater ability to expandsales. These ideas have not yet been implemented in prac-tice and their real-world applicability and usefulness is notyet known.
Beyond straight click-streams, it is also possible to in-clude a variety of other measurables about web usage - timeto click, scrolling information and so forth may all have im-portant discriminatory power between modes of browsing.This mathematical formulation of the problem encouragesa huge array of variations that which deserve further explo-ration and experimentation.
AcknowledgmentsThe authors would like to thank Debashis Paul for helpful
discussion and comments.
9. REFERENCES
Bessler, S. (1960). Theory and applications of the sequen-tial design of experiments, k-actions and infinitely manyexperiments: Part I–Theory. Tech. Rep. 55, Depart-ment of Statistics, Stanford University, 1960.
Box, G.E.P., & Hill, W.J. (1967). Discrimination amongmechanistic models. Technometrics, 9, 57-71.
Chernoff, H. (1972). Sequential Analysis and Optimal De-sign. Philadelphia, PA: Society for Industrial and Ap-plied Mathematics.
Chang, H.-H., & Ying, Z. (1996). A global informationapproach to computerized adaptive testing. AppliedPsychological Measurement, 20, 213-229.
Cover, T., & Thomas, J. (1991). Elements of InformationTheory. New York: John Wiley & Sons, Inc.
Dempster, A., Laird, N., & Rubin, D., (1977). “Maxi-mum Likelihood from Incomplete Data via the EMAlgorithm” Journal of the Royal Statistical Society,39 (series B), 1-38.
Durbin, R., Eddy, S., Krogh, A., & Mitchison, G. (1998).Biological Sequence Analysis. Cambridge: CambridgeUniversity Press.
True Type 1
●
●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type1
Pos
terio
r P
roba
bilit
y
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●
●●●●
●
●●●●
●●●
●
●
●●●●●●●
●●●●●
●●
●
●
●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type2
Pos
terio
r P
roba
bilit
y
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●
●●●●●●●●●●●●●●●
●●●
●
●●●
●
●●●
●
●●●
●
●●●●
●
●
●●●●
●●
●
●●●●
●
●●●●●
●
●●●●●●●●
●●
●●●
●
●●●●
●●
●●●
●
●●
●
●●●
●
●
●
●
●
●
●●●●●●●
●●
●
●●●
●●●●●●●
●
●
●
●●●●●
●●●●
●
●
●
●●●●
●
●●●
●
●
●
●●●●
●
●
●
●
●●
●●
●●●●●
●
●
●
●●
●
●●
●
●●●●●
●
●
●
●●●●
●●
●
●●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●●●●
●●
●●
●●
●
●
●●●
●
●
●●
●
●
●
●●
●●
●
●
●●
●
●●
●●●
●
●
●●
●●●
●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●●
●●●●
●
●
●●
●
●
●●
●
●
●
●
●●●
●
●
●●●●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●
●
●
●●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●●●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type3
Pos
terio
r P
roba
bilit
y
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●
●●
●
●●●
●
●●
●
●●
●
●●●
●●●
●●
●
●●
●
●●●●●
●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●●●●●
●
●
●
●●●
●
●●
●
●
●●●●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●●
●●
●●●●
●
●●●
●
●●●●●●●●
●
●●●
●●
●
●●
●●
●●
●
●●●●
●
●●●●●●●●
●●
●●●●●●
●
●●
●
●●
●●
●
●
●●●●●
●●
●●●●
●
●●
●
●
●●●
●●●
●●
●
●
●●
●
●
●
●●
●
●●
●
●●●
●●●●
●●●
●
●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●
●●
●
●●●
●
●●
●
●
●
●
●
●●
●
●●●●
●
●●●
●●●●
●
●
●
●
●●
●●
●
●
●
●
●●●●●
●●
●
●
●●●
●
●●
●●●●●
●
●●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●●●●●●●●●
●
●●
●●
●●
●●●
●
●
●
●
●●●
●
●●
●
●
●
●●
●
●●●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●●●●
●
●
●
●
●
●●●●●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●●
●
●
●●●●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●●●●
●
●●
●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●
●
●●●●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●●
●●●
●
●●●●●
●
●●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●●●
●
●
●
●●●
●
●
●
●
●●
●●
●
●●●●●
●
●●
●
●
●
●
●●●●
●
●
●
●●
●●●
●
●
●
●●●
●●●●
●
●●●
●●●
●●
●●
●
●
●
●
●●
●
●
●
●
●●
●●●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●●
●
●
●●●●
●
●
●●●●
●●●●●●●
●
●●
●
●
●
●
●
●
●●
●●
●●●●●●
●
●●
●
●
●●
●●
●
●
●
●
●●
●●
●
●●●●
●
●
●
●●●●●●●●
●
●
●
●
●
●●
●
●
●
●●●●●●●●●●●●●●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●●●●●●
●
●●●
●●●●
●●●
●●●●●●●
●
●
●●●●●
●
●●●●●
●
●●●●●●●
●●●
●●●●●
●
●●●●●●●●●●●●
●
●●
●●
●
●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type4
Pos
terio
r P
roba
bilit
y
True Type 2
●●●
●●
●●
●
●●●
●
●
●
●
●●●●●●●●
●
●
●
●●●●●●●●●
●●
●●
●●
●●
●●
●●●●●●●●●●●●●
●●●●●
●●●●
●●●
●
●
●
●●●●●●
●●●●●●●●●●●
●
●●
●
●●●●
●●●●
●●
●●●●●
●
●
●●●
●
●
●●
●●●
●
●●
●
●
●
●
●
●
●●●
●●
●●
●
●
●
●●●●●
●●●
●●●
●
●●
●
●
●
●
●
●
●●
●
●
●●●●●
●
●●●
●
●
●●●
●
●
●
●
●
●●
●●●●●
●
●●●
●
●
●
●
●
●●●
●●●●●
●●
●●
●●●●●●
●●●●
●●●
●
●
●
●●●●●
●
●●
●●●
●●
●
●
●
●●
●●●
●●
●
●●
●
●●●
●●●
●
●
●
●●
●
●●
●
●●●
●●●●●●
●
●
●
●●
●●
●
●
●●●●
●
●●
●●●
●
●●●
●●
●
●
●●●
●
●●
●●●
●
●
●●
●●●●●
●
●
●
●
●●●
●●●
●●●●
●●
●●
●
●●
●●●●●
●
●
●
●●
●●●
●
●●●●●●
●
●
●●●●●●●●
●
●●
●
●●
●
●●●●
●●●●●●●
●●
●●
●●●●●
●
●●
●
●
●
●●
●●●●
●
●
●●
●
●
●●
●
●
●
●
●●●●
●
●●●●
●
●●
●
●
●●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●
●
●●●
●●●●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type1
Pos
terio
r P
roba
bilit
y
●●
●●
●●●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type2
Pos
terio
r P
roba
bilit
y
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●
●●●
●●●●
●●●
●●
●
●●
●●
●
●●
●●
●●●●●●
●
●●●●●●●●
●
●●
●●●●●●●●●●
●●●●●●●●●●
●
●●●●●
●
●●●●●
●●●●
●●
●●●●●
●
●
●●●
●●●
●●
●
●●
●●●
●
●
●●
●●
●
●●●
●●●●
●
●
●
●●●
●●●●
●
●●●
●
●●
●
●
●
●
●
●●●●●
●
●●
●●
●●●
●
●
●
●●●●●
●●●●●●●●
●
●●●●
●●●●●●●
●
●●●
●●
●
●●
●
●
●●●
●
●●
●
●
●●●
●●●●
●
●
●●
●●
●●●●●●
●
●
●●●●
●
●
●
●
●●●
●
●●
●●●
●
●●
●
●
●●
●
●
●
●
●●●●
●
●●
●●●●●●●●
●●
●●●
●
●●
●
●
●
●●●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●●
●
●●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●●●●●●●●●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type3
Pos
terio
r P
roba
bilit
y
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●
●
●●
●●
●●●●●
●●
●
●
●●
●●
●
●●●●●●
●
●●●●●
●●
●
●●
●
●
●
●
●
●●
●●
●
●●●
●
●
●●
●●
●●
●●
●●
●
●
●●●
●
●
●
●
●
●
●●●
●●
●●●
●
●
●●
●●
●
●
●
●●●●
●
●●
●
●●●●●●
●
●●
●
●●●●
●●
●
●●●
●●●●●
●●●●
●●
●
●●
●●●●●●●●●●
●
●
●
●●●●
●●●
●
●
●
●●●●●●●●
●
●●
●
●
●
●
●
●
●●●●●
●
●
●●●
●
●
●●
●●
●
●●
●
●
●●●●●
●●
●●●
●●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●●●●
●
●●
●
●
●●
●
●●●
●
●
●●●
●
●●●●●
●
●
●●●
●
●
●
●●●
●
●●
●●
●●●●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●●●●
●
●
●●●
●
●●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●●●●●
●
●●
●●
●●●●
●
●
●
●●●
●●
●●
●
●
●
●
●●
●●●●
●
●
●●●
●
●●●
●
●
●
●
●●
●●
●
●●
●
●●
●
●●●●
●●
●
●●●
●●●
●
●●
●●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●●
●
●
●
●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●●
●●
●
●●●
●●
●●
●
●
●
●●
●
●
●
●
●●●
●●
●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●●●
●
●●
●
●●
●
●●
●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●●
●●
●●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●●
●●
●●
●
●●
●
●●●
●
●
●
●
●
●
●
●●
●●●
●●
●
●●
●
●●
●
●●●
●
●
●
●
●
●●●
●
●●●●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●
●●
●
●
●●
●●●
●
●
●●
●●
●●
●●
●●
●●●●●●
●
●
●●●●
●
●●●
●
●●
●●
●
●
●●●●
●
●●●●
●●●●●
●
●
●●●
●
●●
●
●
●
●●
●
●●●
●
●●●●
●
●
●
●●●●●
●●●●
●
●
●
●
●●●●●●●●
●●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●●
●●
●●●●
●
●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●●
●●●
●●●●●●
●
●●●●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●●●●
●
●●
●
●●●
●●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●●
●
●●●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●●●●
●
●●●●●●
●●●●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●●●●
●●●●●●
●
●●
●
●●
●
●●
●
●
●●●
●
●
●
●●●●
●
●●●
●●●●●
●
●●●●
●●●●●●●●●●●●●●
●
●●●●●●
●●●●●●●●●●●
●
●●●●●
●
●●●●●●
●
●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type4
Pos
terio
r P
roba
bilit
y
Figure 8: Time series of boxplots of posterior prob-ability estimates for Type 1 (top) and Type 2 (bot-tom) browsers. Only those sessions still alive at agiven time are included.
Finkelman, M. (2003). An Adaptation of Stochastic Cur-tailment to Truncate Wald’s SPRT in ComputerizedAdaptive Testing. National Center for Research onEvaluation, Standards, and Student Testing, October2003.
Hastie, T., Tibshirani, R., & Friedman, J.H. (2001). TheElements of Statistical Learning. New York: Springer-Verlag, 2001.
Owen, R.J. (1975). A Bayesian sequential procedure forquantal response in the context of adaptive mental
True Type 3
●●
●●
●
●●●
●
●
●
●●
●●
●
●●
●●●●
●
●
●●
●
●●
●●●
●
●
●●●●●
●●
●
●
●●
●
●●●
●
●
●
●●
●●
●●
●
●
●
●
●●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●●●
●
●●
●●●
●●
●●
●●
●
●●
●
●
●●●
●●
●
●●●
●
●●●
●●
●
●●●
●
●
●
●●●
●●●●
●
●
●●●●●
●●
●
●
●●
●
●●
●●●
●
●
●●●●●
●
●●
●●●
●●●●●
●●
●●
●●
●
●
●●
●
●
●●●
●●●●
●●
●●●
●
●●
●
●
●●
●
●●
●●●
●
●●
●●
●
●●
●●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●
●●●●●
●●●
●
●
●●●
●●●
●●●
●●
●●●●
●
●
●●
●●●
●●●
●●●
●●●●
●
●
●●●
●●●●●●●●●●●●●●●
●
●●●●●
●
●
●
●●●●●
●
●
●
●●●●●●
●
●●●●●
●
●●●●●●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●●●●
●●
●●●
●
●●●●●
●
●●
●
●
●●●
●
●
●
●●●●●●●●
●
●●●●●
●
●●●
●
●
●
●●●●●●●●
●
●●
●●
●●●
●
●
●
●
●
●●
●●
●
●●●●
●
●
●●●
●
●●
●
●●
●●
●
●
●
●
●
●●●●●●
●●
●
●
●●
●●●●●●●
●
●
●
●●
●
●
●●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●●
●●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●●●●●●●●●●
●
●
●●
●●●●
●
●●●●●
●
●●
●
●
●●●
●●●●
●●
●●●●●●
●
●
●●
●
●●
●
●●
●
●
●
●●●●●●
●
●●●
●
●●
●
●
●
●
●●
●●
●
●●
●
●
●
●
●●
●
●
●●
●●●
●●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●●●●
●
●
●●
●
●●
●
●
●●●
●
●
●●
●●
●
●
●
●●●
●●●
●●
●●
●
●
●●
●●●
●●
●
●
●●
●●
●●
●●
●●
●
●
●
●
●●
●
●●
●
●●
●
●
●●
●●●
●
●
●●●●
●
●
●●●●●●●
●●
●●
●
●●●●●●●●
●●
●
●●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●●●●
●
●
●●●●
●●
●
●●
●
●
●
●
●
●●
●
●
●●●●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●●●●●
●
●●●
●
●
●
●●
●
●
●●●●●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●●
●
●●
●●
●●
●●●●●
●
●
●
●
●●
●●
●
●●
●●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●
●●
●●
●
●
●
●●●
●
●
●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●●●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type1
Pos
terio
r P
roba
bilit
y
●●●●●●●●●●●●●●●●●●●●
●
●
●●●●
●●●●●●
●●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type2
Pos
terio
r P
roba
bilit
y
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●● ●
●
●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type3
Pos
terio
r P
roba
bilit
y
●●●●●●●●●●●
●
●
●
●●●
●●
●●●●●●
●
●●
●●●
●●●●●●●●
●
●
●●
●●
●
●
●●
●
●●●●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●●●●●
●
●●●
●
●
●
●
●
●●●
●
●●
●
●●
●
●●
●
●
●
●
●●
●
●
●●●●
●
●
●
●
●●
●●●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●●●
●●
●
●
●●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type4
Pos
terio
r P
roba
bilit
y
True Type 4
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●
●●●
●
●●●●
●
●
●●
●
●●●●●●
●
●
●
●●●●●●
●●
●●●●●●
●●●
●●●●●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●●●●
●
●●
●
●●
●
●●●●●●●●●●●●●●●●●
●
●●●●●
●
●●●●●
●
●
●
●
●●●●●
●●
●●●
●●●●●●●●●●●●●●●●
●
●
●●●●●●●
●
●
●
●●●●●●
●
●
●●
●●
●
●
●●●
●
●
●●
●
●
●●●●●
●
●
●
●
●●●●
●
●●
●
●
●
●●●●●●●●●●●●●
●
●●
●
●
●●●
●●●●●●
●
●●
●
●
●
●
●●●
●
●
●
●●●●
●
●
●
●
●
●
●●●
●●
●●●●●
●
●●●●
●
●
●
●●
●●●
●●●●●●●
●●●●
●●●
●
●●●
●
●
●●●●●
●
●
●
●●●
●
●
●
●●
●●
●
●●●●
●●
●
●
●
●
●●●
●
●
●●
●
●
●
●●●●●●●●●
●
●●●
●
●●●●
●
●
●●●
●●●●●●●
●●●●●
●
●
●
●●●●
●
●●
●
●
●
●●
●●
●●●
●●●
●
●●●
●
●●
●
●●
●●
●
●●●
●●●●
●●
●●
●
●
●●
●
●●●●
●●
●●●
●
●●●
●
●
●●
●
●
●●●●
●●●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●
●●
●
●
●●●
●●
●
●
●●●●●
●
●
●
●
●
●●●
●
●●●
●●
●
●●
●
●
●
●
●
●
●●●
●
●●
●●
●●
●
●●
●
●●●●
●●
●
●
●
●●
●
●●
●●
●
●
●●
●●
●
●●
●
●●
●
●
●
●●●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●
●●●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●●●
●●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●●●●
●
●
●●●
●
●
●●●●●●
●●●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●●
●
●●●
●
●
●
●●●●
●●
●
●
●
●●
●
●●●●
●
●
●
●
●
●●●●●
●
●
●
●●●
●
●
●●
●
●
●●●●●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●●●
●
●●●
●●●
●●
●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●●
●●
●
●
●
●
●●●
●●●●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●●
●
●●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●●●●●
●
●
●
●
●
●
●●
●
●●●●●●●
●●
●
●
●●
●
●●●
●
●●●●●●
●
●●●●●
●
●●●●●●●●
●●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type1
Pos
terio
r P
roba
bilit
y
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●
●
●
●
●
●
●
●●
●
●●●●●●●●●●●●
●●●●●●
●●●●
●●
●●●
●
●
●
1 7 14 22 30 38 460.
00.
40.
8
Type2
Pos
terio
r P
roba
bilit
y
●
●●
●●●●
●●
●●
●●
●
●●
●
●
●
●
●
●●●
●
●●
●●●
●
●●
●
●●
●●
●●●
●
●
●
●●●
●●●●
●●
●
●●
●
●●
●
●●
●●●●
●
●
●
●●●
●●
●
●
●
●●
●●●●●●
●●●
●●
●
●●
●●
●●●●●
●●●●●●
●●●●●
●●
●
●●
●●
●●●
●
●
●
●●
●
●
●
●
●●●●●
●
●
●●●●●
●
●
●
●
●●
●
●●●
●
●
●●●●
●
●●
●●●
●●
●●
●
●
●●
●
●
●
●●●
●
●
●●
●
●●●●
●●
●●●●●●
●
●
●
●●●
●●●●●
●
●
●
●●●
●
●●
●●
●
●●
●
●
●●
●●
●
●
●
●
●●●
●●
●
●●
●●●●●●●●●
●
●
●●●
●●●
●●●
●
●●
●●
●●●●
●●●
●●
●●●●
●●●
●●●
●
●●●●
●
●
●●
●●
●●
●●●●
●
●
●●
●
●●●●
●●●
●
●
●●
●
●●
●●●●●
●●
●
●●
●●●
●●●●
●
●
●
●●●●
●●●
●●
●
●
●
●
●
●
●
●
●●●
●●●
●
●●●●●
●●●●●●
●●
●●
●
●
●●
●
●
●●●●●●
●●
●●●
●●
●●
●●●
●●
●
●
●●●
●
●●
●●
●●●●●●
●
●
●●
●●●
●●
●
●●
●●
●
●
●
●
●●
●
●●
●
●
●
●●●●
●●●
●
●
●
●
●●●●●
●
●●●●●●
●●●
●
●
●●●
●●●
●
●●
●
●●●
●
●
●
●
●
●
●●●●●●
●
●●
●
●
●
●●
●
●●
●●
●
●●
●
●
●●●●●
●
●
●●
●
●●●●●●
●
●
●●●●
●●●●
●●
●
●●
●●
●
●●●●
●
●●●
●
●●●
●
●●
●
●●●●
●●●
●●●●
●
●●
●
●●
●●●
●●
●
●
●●●●●
●
●●●
●●
●
●
●●●
●●
●●●
●
●●
●
●●
●●
●●
●
●●●●●●
●●
●●
●
●
●
●●
●●
●●
●
●●●
●●
●
●●●●
●●
●
●●●
●●●●●●
●
●●●
●
●
●●●●
●●●●
●●●
●
●
●
●
●
●●
●●
●
●●
●●
●●●
●
●●
●
●●●
●●●●
●●
●
●
●
●
●
●●
●●
●
●●●●
●●●
●●●●
●
●
●
●
●
●●●
●●
●●
●●●●
●●
●
●●●●●●
●●
●●
●●
●
●
●
●●
●●
●●●●●●
●
●●
●
●●●
●
●
●●
●●●
●
●
●●●●
●
●●
●●
●
●●●●●
●
●●
●●
●●
●
●
●●●●
●●●●
●
●
●●●
●
●
●
●●●●●
●
●
●●
●
●
●
●
●
●●●
●
●
●●●●●●
●●●
●●
●●●●
●●●
●●
●●
●
●●
●●
●●
●●●●
●●
●
●
●
●●●
●●●
●
●●
●●●
●
●
●
●●●●
●
●●
●
●
●●●
●●
●
●●●●●●●●
●●
●
●
●
●
●●●
●●●●
●
●●
●●●●●
●
●●●●●
●●
●
●
●●
●●
●●
●●
●●●
●●●
●
●
●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●
●
●●●●●
●●●●●●●●●●
●●
●●●●
●
●●●●●●●●●●●
●
●
●●
●
●●●●●●●●●●●
●
●●●●●●●
●●●
●
●
●
●
●
●
●●●●●●●●
●●●
●
●●
●
●●●
●
●●
●
●●●●●
●
●
●●●●●
●
●
●
●
●
●●●●●●●●
●
●
●
●●
●●
●●●●●
●
●●
●
●●●
●
●
●
●●
●
●●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●●●●●
1 7 14 22 30 38 46
0.0
0.4
0.8
Type3
Pos
terio
r P
roba
bilit
y
1 7 14 22 30 38 46
0.0
0.4
0.8
Type4
Pos
terio
r P
roba
bilit
y
Figure 9: Time series of boxplots of posterior prob-ability estimates for Type 3 (top) and Type 4 (bot-tom) browsers. Only those sessions still alive at agiven time are included.
testing. Journal of the American Statistical Associ-ation, 70, 351-356.
Siegmund, D. (1985). Sequential Analysis: Tests and Con-fidence Intervals. New York: Springer-Verlag.
Wald, A. (1947). Sequential analysis. New York: Wiley.