Measuring Spontaneous Devaluations in User Preferences

Komal Kapoor, Nisheeth Srivastava, Jaideep SrivastavaDepartment of Computer Science

University of Minnesota200 Union St SE

Minneapolis, MN 55455kapoo031@umn.edu, nisheeths@gmail.com, srivasta@cs.umn.edu

Paul SchraterDepartment of Psychology and Computer Science

University of Minnesota75 East River Road

Minneapolis, MN 55455schrater@umn.edu

ABSTRACTSpontaneous devaluation in preferences is ubiquitous, whereyesterday’s hit is today’s affliction. Despite technologicaladvances facilitating access to a wide range of media com-modities, finding engaging content is a major enterprise withfew principled solutions. Systems tracking spontaneous de-valuation in user preferences can allow prediction of the on-set of boredom in users potentially catering to their changedneeds. In this work, we study the music listening historiesof Last.fm users focusing on the changes in their preferencesbased on their choices for different artists at different pointsin time. A hazard function, commonly used in statisticsfor survival analysis, is used to capture the rate at whicha user returns to an artist as a function of exposure to theartist. The analysis provides the first evidence of sponta-neous devaluation in preferences of music listeners. Betterunderstanding of the temporal dynamics of this phenomenoncan inform solutions to the similarity-diversity dilemma ofrecommender systems.

Categories and Subject DescriptorsI.6 [Simulation and Modeling]: Applications

General TermsHuman Factors

KeywordsDynamic Preferences, Recommender Systems, Temporal Mod-els, User Behavior Modeling

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from permissions@acm.org.

1. INTRODUCTIONRecommendation systems have become a popular means

of suggesting relevant content to the user. Methods in rec-ommendations have focused on constructing estimates ofuser preferences based on their history of choices. Thesepreference estimates are then used to suggest new contentto the user using content-based or collaborative methods.Content-based methods use a user’s preference estimates tofind similar content, while collaborative methods use a user’spreference estimates to identify similar users (neighborhood)and recommend content popular in the identified neighbor-hood. But, it’s not sufficient for a recommender agent toonly estimate a user’s past preferences; it’s also importantto predict their future preferences given past experiences.This makes the task of a recommender even more challeng-ing by requiring it to predict when and how a user’s pref-erences will change in the future. The recommendationscommunity, however, lacks models which can predict chang-ing preferences of users and doing so is generally acceptedas a hard problem. On the other hand, user’s recent choiceshave been found to be a good predictor of their future be-havior. Efforts in modeling temporal recommendations haveexploited this aspect of user choices by designing recommen-dation systems which systematically emphasize recency withgood results. The critical shortcoming of this formulationis that such a system merely reacts to preference changesrather than trying to predict them.

While little work has been done on predicting changes inuser preferences in the recommendation literature, psychol-ogists and behaviorists have long studied the dynamics ofindividual preferences. Several theories have been proposedto explain why individuals seek out new content (noveltyseeking, exploratory and information seeking behavior) [2].Other studies talk about individuals making choices to ac-tively seek an optimal level of stimulation in their environ-ment [21]. The theory of flow [20] suggests that an envi-ronment which provides an optimal level of challenge for agiven level of skill leads to a desirable state of flow. Despitesuch theoretical developments, it has been difficult to op-erationalize these aspects of individual choices to solve realworld problems. However, modeling properties of individ-

KDD’13, August 11–14, 2013, Chicago, Illinois, USA.Copyright 2013 ACM 978-1-4503-2174-7/13/08 ...$15.00.

1061

ual behavior is critical for advancing designs of automatedagents which interact with individuals on a daily basis.

In this work, we study one aspect of dynamic individualpreferences. Individuals are often found to develop disinter-est and even dislike for their dearly preferred content bothtemporarily and lastingly. It’s common to find that one’sclothes, food, entertainment, jobs etc. have grown boring de-spite being enjoyable in the past. We call this phenomenon aspontaneous devaluation of one’s preferences or boredom fora stimulus. Spontaneous devaluation is seen to arise whenrepeated exposure to a stimulus creates a feeling of satia-tion towards it leading to a loss in interest [7]. Alternatively,spontaneous devaluation has been linked to lost opportunityfor novel experiences when similar experiences are repeatedtoo often [18]. Both theories concur in suggesting that, incontrast to recency-based expectations, repeated exposure tofamiliar choices spontaneously devalues one’s preference forthem.

Human behavior driven by these dynamics could be mod-eled as systematically alternating between one’s set of choices,assuming that the time spent in experiencing other stimuliis sufficient to mitigate the effects of boredom for a par-ticular stimulus. Several studies on user purchase behaviorhave found buyers to alternate among their preferred alter-natives [11, 18, 9] etc. However, in practice users have anon-uniform liking for different alternatives in their choicespace. Furthermore, users have a pronounced tendency tostick to their recent choices [11] which has been responsi-ble for the success of the previously proposed recommendermodels. We call this behavior the ‘sticky’ behavior in users.This phenomenon has also been called reinforcement or in-ertial behavior. Such behavior can be explained to arise dueto an actual increase in liking on exposure [9] or a tendencyto avoid switching costs.

The presence of both stickiness and devaluation effectsin user preferences make predicting the temporal choices ofa user non-trivial. In this paper, we analyze user music lis-tening behavior to extract signals of stickiness and boredom.Our analysis is limited to the music domain due to availabil-ity of public datasets, nevertheless, we expect our results togeneralize to other items like movies, videos, books, vacationpackages, shopping etc. which are fairly susceptible to bore-dom effects. We demonstrate the use of hazard functionsfor measuring these phenomena. Our work provides the firstproof of spontaneous devaluation in music listening prefer-ences of users and its impact on user choices. This workcan inform design of future methods that incorporate thesedynamics, producing agents that can cater to new needs ofusers suffering from boredom.

The rest of the paper is organized as follows: Section 2provides a summary of the related work. Section 3 givesan overview of the dataset and pre-processing details. Sec-tion 4 lays out terminology relevant to our analysis. Sec-tion 5 provides details of our methodology. Our results aresummarized in Section 6. We end with a discussion of thecontributions of this work and possible future extensions inSection 7.

2. RELATED WORK

2.1 Dynamic PreferencesStimulus satiation was initially used by researchers to ex-

plain spontaneous alternation in rats [7]. Rats were placed

in a T-shaped maze and provided an unlimited supply offood at the left and the right corners of the maze at equaldistances. The experiment was set up such that that the rathad to return to the starting point before each trial. It wasseen that rats chose to alternate between the left and theright ends on repeated trials. Glanzer [10] suggested thatsuch a behavior arose due to stimulus satiation such thateach time the organism was exposed to the stimulus, sati-ation for the stimulus increased causing the rat to switchdirections. Further, satiation for the stimulus diminishedwhen the organism could no longer perceive the stimulusand the rat returned back to the same direction.Researchers have found individuals to engage in more com-plex forms of variety seeking behavior while making choices.McAlister proposed a taxonomy of factors responsible forvaried behavior in individuals [18]. These were classifiedinto two categories based on whether they arose due to ex-ternal factors (such as unavailability of a product, launch ofnew products etc.) or due to internal motivations. Whenarising out of internal motivations, variety seeing behaviorwas suggested to manifest in two forms; a desire for unfamil-iar alternatives or a desire to alternate among familiar al-ternatives. The former was linked to individuals seeking anoptimal level of stimulation [2, 21], while, the latter was seenas a weak form of exploratory behavior. It was also linked todevaluation in preferences due to satiation. A single peakedpreference function was proposed to characterize the attrac-tiveness of a stimulus on repeated exposure [6]. McAlisteralso proposed a dynamic attribute satiation model [17] whichassumed an ideal level of inventory for different attributesof the items. The inventory was designed to dwindle overtime to incorporate the effects of forgetting.Researchers have subsequently focused on modeling the choiceprobabilities of consumers directly given their past choices.Consumers were found to exhibit either a short term loy-alty for their last purchased brand (inertia) or devaluationfor the last purchased brand (variety seeking) [11, 9]. Kahn[12] compared seven models for user choice behavior withsimilar results. Bawa et al [1] used a single peaked func-tion, to model the conditional probability of repeat purchasegiven the number of times the brand was re-purchased sinceuser’s last switch (run length). Chintagunta [5] used hazardrates to model the level of inertia and variety seeking as afunction of time between purchases. Recent efforts have ex-panded these models to incorporate heterogeneities betweenconsumers and external environment variables affecting userchoices [13].Most of the research in this area, however, has been limitedto panel datasets and analysis of user surveys and question-naires. In this work we have adopted a data driven approachto elicit changes in user preferences towards a stimulus as afunction of their past exposure to it. Our efforts do not lookat variety seeking or inertial behavior in users in general,but at changes in choice probabilities with respect to partic-ular stimulus, grounding ourselves in psychological theoriesof boredom and novelty seeking, which provides a causalexplanation for the existence of these patterns.

2.2 Recommender SystemsState-of-the-art methods in recommender systems have

assumed a static view of human preferences. Ding et al. [8]showed that the static view of user preferences used whilegenerating recommendations was flawed as it did not take

1062

changing user interests into account. They used a decayfunction to gradually devalue the impact of a user’s past his-tory while making prediction of his future likings. Recently,a temporal model of recommendation was developed [15,14] which was an important part of the solution to the KDDCup on Yahoo Music dataset and the Netflix challenge. Themodel incorporated several time-sensitive user and item bi-ases in the standard factor model. Gradual changes in userpreferences over time were captured using a linear function.Their model showed that modeling temporal dynamics inuser choices was essential for improving the performanceof the recommender. Sahoo [22] has proposed a dynamicmodel of blog reading behavior in employees. He used a Hid-den Markov Model to predict future interests of employeesbased on their previous choices. However, user transitionsare assumed to be driven by a static transition matrix. Atpresent, the recommendation community lacks models thatpredict changes in user preferences.

Also related to our work are methods to introduce di-versity and novelty in recommendations. Lathia et al. [16]showed that popular recommendations methods such as kNNand SVD produced recommendations which were very sim-ilar (low in temporal diversity) on iterated train-test exper-iments on temporally ordered data. Many methods thatsystematically introduce diversity in the recommendationshave been proposed [19, 23, 24, 3]. However, these meth-ods focus on jointly optimizing both similarity and diversityindices described on the space of items being recommendedrather than predicting changes in user preferences.

3. DATAOur analysis is based on complete temporal music listen-

ing histories of users provided by Last.fm. Last.fm is a pop-ular music website with millions of active users. It allowsusers to purchase tracks, listen to online radios and playlistsetc. and has additional social networking features as well.Recently, Last.fm made available a dataset of complete mu-sic listening histories of around 1000 users as recorded tillMay 2009 [4]. This is the only publicly available dataset,to our knowledge, to provide complete temporal records ofuser choices. Because Last.fm hosts several online radios,it is quite probable that parts of the user histories captureradios, and playlists rather than active user choices. Wefiltered these effects by using the time gap between two con-secutive tracks played by the user. Last.fm has a generouslist of API’s available to developers. The API, track.getInfo,was used to retrieve the duration of most of the songs in ourdataset. We compared the time gap between song 1 and song2 in that temporal order in the user history with the lengthof song 1. If the time gap was found to be more than thelength of song 1 by less than 5 seconds, song 2 was identifiedto belong to an automated play list. All tracks ‘not on auto-play ’ were assumed to be active user choices. We could notremove auto-play effects for the songs whose lengths wereunavailable through the API. This corresponded to 0.05% ofthe songs. We only considered the first 1 year of each userhistory in our analysis. All the users which had less than 30records of activity were eliminated from the dataset. Also,we only kept those artists in the user history which the userhad listened to 15 or more times in that period of 1 year.We summarize some important statistics about the datasetin Table 1.

Property Value# unique tracks 1,084,872# unique artists 174,091# Users 957Mean history length -# songs heard

6716

Mean history length -# active days

177

Mean # unique artistsheard

37

Table 1: Statistics from the Last.fm dataset

4. TERMINOLOGYBased on both the novelty-seeking and stimulus satiation

theories of devaluation of preferences, repeated exposure toa stimulus causes devaluation in one’s preferences towardsit. Additionally, devalued preferences can get reinstated af-ter a period of reduced or no exposure. A music piece canstimulate the listeners because of the combined effect of itsmultiple features (artist, genre, tempo, strong female vocals,etc.). For simplicity and ease of access, we use the artist ofthe songs as our basic stimulus. More sophisticated stimu-lus definitions that model the interaction between multiplefeatures of a song can enhance our method.

Preferences have been linked to choice probabilities in thepast. It is only a logical extension to relate changes in pref-erences to changes in choice probabilities, and in our caseconditional choice probabilities. We suspect that the phe-nomenon of devaluation produces two different patterns inthe choice probabilities of users for an artist.

Hypothesis 1: The probability that a user will listento an artist again will decrease after he has listened to theartist some number of times. When this happens, we saythat the user’s preferences for the artist have devalued.

Hypothesis 2: Devalued preferences can get reinstatedafter a sufficient period of non/reduced exposure to the artist.

Through our experiments, we look for signals suggestive ofspontaneous devaluation in choices probabilities of Last.fmusers. By doing so, we establish a methodology for detectingthis phenomenon and analyzing its properties.

We consider the state of the user at some time t to bedefined by the artist of the song the user was listening to atthat time. The temporal history of the user comprises thesequence of states visited by him as a function of time; i.e.Hu(t) = sa if user u was listening to artist a at time t.User u is said to enter a state a at time t if Hu(t) = sa andHu(t− 1) 6= sa. A user u is said to exit a state a at time tif Hu(t) 6= sa and Hu(t− 1) = sa. We can now define thefollowing conditional choice probabilities:

1. Conditional probability of exit: This is the con-ditional probability of a user u exiting state a at timet given that he last entered state a at time t − r andhas not exited state a yet. Formally, the probability isequal to P (Hu(t) = sa|Hu(t − 1) = sa, . . . , H

u(t −r) = sa, H

u(t− r− 1) 6= sa). Here, r is the time spentlistening to the artist and corresponds to the idea of arun length in Bawa’s model [1]. We make the simplify-ing assumption that this probability depends only onr. Hence, we can also represent the conditional prob-

1063

ability of exiting state a by user u when time spent instate is r as Pua(exit|time spent in state a = r).

2. Conditional probability of entry: This is the con-ditional probability of user u entering a state a at timet given that the user last exited state a at time t−(o+1). Formally, this corresponds to P (Hu(t) = sa|Hu(t−1) 6= sa, . . . , H

u(t−o) 6= sa, Hu(t−o−1) = sa). Here,

o is the time spent not listening to the artist a. Again,for simplicity, we assume that this probability dependsonly on o. We later relax this assumption with interest-ing effects, described in Section 6.3. Thus, this proba-bility can also be represented as the conditional proba-bility of entering state a after having exited it o units oftime ago or Pua(entry|time spent out of state a = o).

The definition of time has been kept ambiguous in the defi-nitions above. We now define it more formally. Time can bedefined in terms of the order in which songs are heard by theuser such that Hu(t) refers to the t-th song heard by useru. Such a definition, however, does not take the actual timegap between consecutive listenings into account. It is impor-tant to consider the actual time gap between user choices.This is because a user satiated with an artist can get unsa-tiated both by listening to other artists or due to forgettingif he returns to the system after a long time. To analyze theimpact of actual clock time on the satiation level, we definetime in terms of days since the first historical record of theuser. Accordingly, Hu(t) refers to the state of the user ont-th day since day 1. For simplicity, the state of the user ona day is defined by the artist listened to most frequently byhim on that day.

5. METHODOLOGYSurvival Analysis is a statistical method commonly used

for modeling time-to-event data. The purpose of this kindof analysis is to model the probability of survival (wherethe occurrence of the event corresponds to death) beyonda certain point in time. For simplicity, we use a discretemeasures of time t ∈ N. The survivor function at time t isdefined as:

S(t) = P (T > t) (1)

Where, T is a random variable denoting the time of death.The instantaneous rate of occurrence of the event at timet, conditioned on having survived up to time t, is capturedusing the hazard function. The hazard function is also calledthe conditional failure rate and is defined as:

λ(t) = lim∆t→0|

P (t ≤ T < t+ ∆t|T ≥ t)∆t

= −S′(t)/S(t) (2)

We use the hazard rate function to compute the exit and en-try conditional probabilities defined in the previous section.We set ∆t = 1. This allows us to use the terms hazardrate and conditional probability of death interchangeably.We can construct the two different hazard curves based onhow we define our events.

1. Exit Hazard Rate: Here, we measure time from thepoint when a user u entered a state a. The event cor-responds to his ‘exit’ from the state. The random vari-able Tuaexit denotes the time of exit or death. This haz-ard rate captures the conditional probability of exitingthe state at time t+ 1 having survived in the state fortime t or greater; λuaexit(t) = Pua(Tuaexit = t|Tuaexit ≥ t).

2. Entry Hazard Rate: Here, we measure time fromthe point when a user u exited a state a. The eventcorresponds to his ‘entry’ back into the state. The ran-dom variable Tuaentry denotes the time of entry or death.This hazard rate captures the conditional probabilityof entering a state at time t having survived outsidethe state for time t or greater;λuaentry(t) = Pua(Tuaentry = t|Tuaentry ≥ t).

An exit and entry hazard rate can be defined for each artista user listens to. For our analysis, we pool across the differ-ent users and the artist choices to compute an average exitand entry hazard rate for the entire dataset. We normalizethe time of entry and exit variables to mitigate the effectsof differences in a user’s preferences for different artists anddifferences across users. The time of event variable is logtransformed as well as it becomes harder to exactly predictthe time of an event as time for which the event has nothappened increases. In other words, this means that if auser has not returned to an artist in a month, its more dif-ficult to predict the exact day of his return, than, when hehas has not returned to the artist for a day. The log trans-form accommodates this non-linearity in the predictabilityof return time.

TNi =

log2(Tuai )

log2( 1Pu(a))

(3)

for a user u and artist a and i ∈ {‘entry′, ‘exit′}. Pu(a) isthe prior probability of user u being in state a.

Pu(a) =Nu(a)

Lu(4)

where, Nu(a) is the number of times user u was in state aand Lu is the length of user u’s history. The average hazardrates for the normalized time of event variable can then becomputed across users and artists:

λi(t) = P (TNi = t/TN

i ≥ t) (5)

We discretize t into intervals (0, 0.1], (0.1, 0.2] and so on.The hypothesis presented by us in section 4 can now berepresented using the hazard rates.

1. Hypothesis 1 The exit hazard rate for an artist shouldbe an increasing function of time. This indicates that auser’s preferences for an artist decrease with increasedexposure to the artist.

2. Hypothesis 2 The entry hazard rate for an artistshould be an increasing function of time. This indi-cates that user preferences for the artist are reinstatedafter sufficient time gap.

The sticky or inertial view of user choices, on the other hand,suggest that a user’s probability of visiting a state would in-crease on having visited it. Contrary to the devaluation hy-pothesis, the conditional probability of visiting a state againwould increase as time spent in the state increases. This im-plies that the exit hazard rate for an artist is a decreasingfunction of time for sticky users. The entry hazard rate,would also be a decreasing function of time as a user wouldbe less likely to visit a state which they has not visited forlong periods of time.

A common analysis methodology is to compare the hazardrate of interest in an analysis with that generated from a

1064

(a) Expected hazard rate for a sticky and boredom-prone user (b) Expected hazard rates for the baseline models

(c) Expected ∆ Hazard Rates for sticky users (d) Expected ∆ Hazard Rates for boredom-prone users

Figure 1: Figure (a) and (b) depicts the expected hazard rates for sticky and boredom-prone users and thebaseline models. Both the entry and exit hazard rates should decrease with time for sticky users and increasewith time for uses susceptible to boredom. Figure (c) and (d) shows the expected ∆ hazard rates computedagainst each baseline model for sticky and boredom-prone users.

control experiment. This is done to remove the effects ofcovariates not being considered in the analysis. We definefour baseline models to serve as controls. We constructedlistening sequences by simulating user histories using eachof the baseline models for every user. The user historieswere simulated by sampling randomly from the temporalpreference vector (Pref) generated by each of the model. Inorder to make the baseline models as close to the real dataas possible, the parameters of the models were fitted to theactual user histories.

1. Random (R) The user is assumed to sample statesrandomly from his average preference vector (Pu).Prefu(t) = Pu

2. 1st order Markov (M1) A user’s switching proba-bility from one state to the other is assumed to be con-

trolled by a 1st order Markov model. The dynamics ofthe Markov model are controlled by a static transitionmatrix (Tu) which is learnt for each user u’s historyusing maximum likelihood estimation. Prefu(t) =Prefu(t− 1) ∗ Tu

3. Time weighted (TW) We use a recency based modelfor generating user histories. Prefu(t) = αu∗Prefu(t−1) + cu(t− 1), where, cu(t− 1) is 1 ∗ |A| choice vector,which is set to 1 at index i if Hu(t− 1) = si, and is 0otherwise. The parameter αu is a |A|*1 vector whichwas fit to the user u’s history using stochastic gradientdescent. We introduced a small exploratory compo-nent to this model to prevent extremely long lengths ofcontinuous listening of the same artist. Therefore, ourmodified preference vector is computed as Pref ′u(t) =0.95 ∗ Prefu(t) + 0.05 ∗ Pu

1065

4. Linearly increasing or decreasing (L) We used thetemporal model of user preference used by Koren [14].

Prefu(t) = Pu + sign(t − Lu/2) ∗ (t − L/2)βu

. Theparameter βu is a |A|*1 vector and was fitted to theuser u’s history using stochastic gradient descent.

The Log-Rank test can be used to test whether the survivaldistributions generated by the simulated models are suffi-ciently different from that of the real data. The hypothesistest is defined as:

Ho: The real data and the simulated data have differentsurvivor function

Ha: The real data and the simulated data have the samesurvivor function

The Log-Rank test on the real and the simulated survivalfunctions rejects the null hypothesis with a p-value < 10−6.The discrepancy between the real data and the baselinemodel predictions can be quantified using a ∆ hazard rateobtained by subtracting the simulated hazard rates from thehazard rates computed on real data.

λ∆(t) = − S′real(t)

Sreal(t)− − S′(simulated)(t)

S(simulated)(t)(6)

We generate four ∆ hazard rates for both the entry andexit time events for our analysis, namely real vs. random(λA−Ri ) , real vs. Markov (λA−M1

i ), real vs. time weighted(λA−TWi ) and real vs. linear (λA−Li ), where i ∈ {‘entry′, ‘exit′}.

We display the entry and exit hazard rates expected forthe event times obtained from the ‘sticky’ and ‘boredom-prone’ models and those expected from the baseline modelsin Figure 1. The entry and the exit hazard rates for a ran-dom, markovian and linear model should be independent oftime spent in the state. A TW model on the other hand, isessentially a sticky model. Hence, the exit and entry hazardrates for TW model would decrease with time. The objec-tive of this study is to understand the form of the exit andentry hazard rates for the real data. Figure 1 displays theexpected ∆ hazard rates if the real data follows the stickyand the boredom-prone model, respectively.

6. RESULTSIn this section we examine the obtained ∆ exit and ∆

entry hazard rates in close detail.

6.1 ∆ Exit Hazard RatesFigure 2 displays the survivor functions for the exit time

for the real data and data generated by each simulatedmodel. It also depicts the obtained ∆ exit hazard rates.The changes in λA−Rexit , λA−M1

exit and λA−Lexit , directly represent

changes in the λexit for the real data. Changes in λA−TWexit

would depict changes in the exit hazard rate for real dataagainst a decreasing baseline.

1. Real Vs.Random, Markov and Linear models: TheλA−Rexit and λA−M1

exit are negative throughout suggestingthat the exit rate for the real data is lower than thatexpected for the baseline models. This supports thesticky view of user preferences suggesting that a userhas a lower rate of exiting a state after having vis-ited it. However, contrary to what is expected for thesticky model, the ∆ exit hazard rate increases with

time after a point. We expect the ∆ hazard rate toeventually flatten out, becoming uninformative. Thesurvival function for R, M1 and L models drops sharplyindicating a lower probability for large sequences thanthose observed in the real data. The L model has thesharpest drop in survival probability, such that we didnot enough samples of exit times greater than 0.1.

2. Real vs. Time-Weighted model: λA−TWexit is negativefor low values of t, suggesting larger stickiness in usersthan generated by the TW model. However, the ∆ exitrate increases thereafter, becoming positive after sometime. Since, the exit hazard rate for the TW model isexpected to decrease with time, this suggests that theexit hazard rate for real data increases more than thedecrease observed in the TW model.

From these observations we can conclude that users havehigh stickiness towards the state on entering the state. How-ever, the stickiness for a state reduces with time and the dy-namics driven by boredom start dominating as time spentin the state increases. A user is thus likely to stick to hisprevious state at a higher rate initially and a decreased rateas time in the state increases.

6.2 ∆ Entry Hazard RatesFigure 3 displays the survivor functions computed for the

entry time variable for real and simulated data and the ob-tained ∆ entry hazard rates. Similar to the ∆ exit hazardrates, the changes in λA−Rentry , λA−M1

entry and λA−Lentry functionswould depict changes in the entry hazard rate for the actualdata. The TW model is expected to have a declining entryhazard rate, being a sticky model. The changes in λA−TWentry

should reflect changes in the entry hazard rate for the realdata against a decreasing baseline.

1. Real Vs.Random, Markov and Linear models: TheλA−Rentry , λA−M1

entry and λA−Lentry functions are positive ini-tially suggesting that the users have a higher rate ofentry than that expected from the baseline models.This again can be attributed to the sticky nature ofuser choices, such that users have a high rate of return-ing to the artists they had listened to recently. The ∆hazard rates decrease for intermediate values of t sug-gesting a prominent devaluation in preferences. The∆ hazard rates eventually increase for larger values oft. However, they do not cross the 0-line again suggest-ing that a user always has a lower rate of return thanthat generated by the baseline models. This can beattributed to phasing out of an artist who is not beingactively sampled.

2. Real vs. Time-Weighted model: The λA−TWentry functionis slightly negative at the beginning suggesting that theactual entry hazard rate is lower than that of a TWmodel. Our TW model is seen to pull back users whichhave just left an artist at a higher rate than observedin real data. The hazard rate increases thereafter in-dicating the actual data seems to have a larger rate ofreturn than that of the TW model.

The analysis on the ∆ entry hazard rates reveals aspectsof sticky behavior in users which produces quick switchesin and out of the artist. Also, we find indicators of deval-ued preference for intermediate values of time spent out of

1066

(a) Kaplan-Meier survival functions and 95% confidence interval (b) Nelson-Aalen ∆ exit hazard functions

Figure 2: The figure illustrates the survival and the hazard functions computed for the exit time variable. The

negative ∆ exit rates for low values of t are indicative of sticky behavior, while the increase in ∆ exit hazard rate

indicate a devaluation in preference.

(a) Kaplan-Meier survival functions and 95% confidence interval b) Nelson-Aalen ∆ exit hazard functions

Figure 3: This figure illustrates the survival and the hazard functions computed for the entry time variable. The ∆

hazard rates are positive for all the model for low values of t which is indicative of sticky behavior. A decline in the ∆

entry hazard rates corresponding to the R, M1 and L models for intermediate values of t indicate that the preferences

were temporally devalued. The increase in the ∆ entry hazard rates corresponding to all the models for larger values

of t suggest that preferences were reinstated

the state. Preferences are reinstated after longer periods oftime spent away from the artist, however, the rate of returneventually flattens out becoming uninformative.

6.3 Previous Return TimeIn our previous analyses, we found evidence suggesting

that users quickly switch in and out of an artist in a shortspan of time. Such a characteristic of user temporal choicessuggest that a user’s level of exposure to an artist is not com-pletely defined by the ‘in time’. A user who has just switchedout of the artist and has switched back in almost immedi-ately after, somewhat continues to be in state a. Therefore,we suspect that the previous return time (PRT) TN,Pentry alsoindicates how much a user has been exposed to the artistrecently. A low PRT indicates higher exposure to the artistthan a larger PRT. A corollary to hypothesis 1 in terms ofthe TN,Pentry for the artist follows:

Corollary 1’ The probability that a user listens to anartist again will depend on his PRT to the artist. We suspectthat of if the user has returned to the artist quite quickly

previously, he will have a lower rate of returning quickly tothe artist in the future.

In order to test this hypothesis we generate two condi-tional entry hazard rates.

1. λLPentry Entry Hazard Rate given a low PRT, TN,Pentry < 1

2. λHPentry Entry Hazard Rate given a high PRT, 1 < TN,Pentry <1.5

We compute the ∆ hazard rate for the two conditional entryhazard rates.

λLP-HPentry = λLPentry − λHPentry (7)

λLP-HPentry function is computed for the real data and data sim-

ulated using a Markov model. The simulated data servesas a comparison. Figure 4 displays the obtained λLP-HP

entry

functions and the survival functions for λLPentry and λLP

entry forthe real data and simulated data. The log rank test is re-jected with a p-value of less than 10−4 on the conditional

1067

(a) Kaplan-Meier survival functions (real data) (b) Kaplan-Meier survival functions (M1-Model)

(c) Nelson-Aalen ∆ exit hazard functions (real data) (d) Nelson-Aalen ∆ exit hazard functions (M1-Model)

Figure 4: This figure illustrates the survival and the hazard functions computed for the entry time variable conditioned

on the PRT. The conditioned survival function for the simulated data are coincident but vary significantly for the real

data. The positive values of the ∆ hazard function λLP−HPentry for low values of t indicate an increased stickiness when

conditioned on lower values of PRT. The negative values of the ∆ hazard function for larger values of t are indicative

of increased boredom effects when conditioned on lower values of PRT.

survival functions of the simulated and the real data. How-ever λLP-HP

entry varies by very small amounts. On the contrary,

λLP-HPentry on the real data varies in an interesting way. We

see that λLP-HPentry is highly positive initially, which indicates

increased stickiness when PRT is low. However, λLP-HPentry de-

creases and becomes negative eventually which indicates alower rate of return for larger values of t when PRT is lowthan when PRT is high. Hence, once a user is out of thestate he has a lower rate of returning back to the state whenprevious return time is low than rate of return for a user-artist pair for whom previous return time was high.

7. DISCUSSIONIn this work we have outlined a methodology for analyz-

ing music listening histories of Last.fm users for studying thephenomenon of spontaneous devaluation in user preferencesor boredom. We constructed hypothesis about boredom-prone behavior in Last.fm users and tested them throughexperiments on real and simulated data. Exploratory anal-ysis of dynamic hazard rates computed on both the real andsimulated data suggest that real data has strong evidenceof spontaneous devaluation of preferences, as hypothesized.We also found strong evidence suggesting stickiness or rein-forcement nature of past choices in users. Crucially, sticki-ness and boredom effects on user choices were found to be

spaced out in time suggesting that methods can be designedto systematically appease the two driving forces effectinguser temporal needs. The results obtained from this analy-sis motivate the design of sophisticated dynamic models ofuser choices impacting recommendation methods, productdesign and advertising.

Our findings suggest that methods which only focus onmaximizing similarity, or focus on maximizing both simi-larity and diversity at all times, accommodate only someaspects of user behavior, leaving useful temporal informa-tion on the table. Sophisticated temporal models of individ-ual preferences, well grounded in cognitive and psychologicalanalysis of the dynamics of their choices, are required for thedesign of automated methods that can predict user temporalneeds well.

Being able to say when a user is likely to be bored shouldyield considerably more responsive and accurate productrecommendations. However, the gap between this exploratoryanalysis and usable applications, while bridgeable, is non-trivial. We suspect heterogeneities to exist among users andtheir behavior towards different items, which this analysishas not considered. This is principally because extricatinggood estimates of dynamic hazard rates for different user-item pairs requires large amounts of historical data, while wewere limited in our analysis to the Last.fm publicly releasedataset. Unavailability of datasets providing complete tem-

1068

poral histories of users makes procurement of data a chal-lenge. While gaining access to more data would be the bestsolution, clustering methods can reduce the data scarcityproblem in the interim. Additionally, for simplicity, we haveassumed the user behavior for an item is independent of theother items experienced by him. However, one can expectsimilar/dissimilar items to increase/decrease one’s level ofsatiation with an item. Extending our approach into a full-fledged recommendation system would require us to addressuser and item level heterogeneities and similarities betweenitems in a single framework. Potential solutions can benefitfrom hierarchical approaches to cluster items using multiplefeatures allowing estimation of the impact of history on thehazard rates for similar items.

Our work constitutes the first study on dynamics of prefer-ences of online music listeners, and demonstrates that thereis significant value in trying to study the temporal browsinghistory of users along the lines we have suggested. We hopeour work will motivate further studies on this topic in thefuture. Also, larger datasets would be made accessible forstudying aspects of user choices, allowing advancement inthe design of predictive agents of temporal user choices.

8. ACKNOWLEDGEMENTThe research reported herein was supported by the Social

Media and Business Analytics Collaborative (SOBACO) atthe University of Minnesota. We would like to thank thefunding agency for their support.

9. REFERENCES[1] K. Bawa. Modeling inertia and variety seeking

tendencies in brand choice behavior. MarketingScience, 9(3):263–278, 1990.

[2] D. E. Berlyne. Conflict, arousal, and curiosity. 1960.

[3] O. Celma. Music Recommendation and Discovery inthe Long Tail. Springer, 2010.

[4] O. Celma. Music recommendation datasets forresearch. 2010.

[5] P. K. Chintagunta. Inertia and variety seeking in amodel of brand-purchase timing. Marketing Science,17(3):253–270, 1998.

[6] C. H. Coombs and G. S. Avrunin. Single-peakedfunctions and the theory of preference. PsychologicalReview, 84(2):216, 1977.

[7] W. N. Dember and H. Fowler. Spontaneous alternationbehavior. Psychological Bulletin, 55(6):412, 1958.

[8] Y. Ding and X. Li. Time weight collaborative filtering.In Proceedings of the 14th ACM internationalconference on Information and knowledgemanagement, pages 485–492. ACM, 2005.

[9] M. Givon. Variety seeking through brand switching.Marketing Science, 3(1):1–22, 1984.

[10] M. Glanzer. The role of stimulus satiation inspontaneous alternation. Journal of experimentalpsychology, 45(6):387, 1953.

[11] A. P. Jeuland. Brand choice inertia as one aspect ofthe notion of brand loyalty. Management Science,25(7):671–682, 1979.

[12] B. Kahn, M. Kalwani, and D. Morrison. Measuringvariety-seeking and reinforcement behaviors usingpanel data. Journal of Marketing Research, pages89–100, 1986.

[13] T. Kamai and Y. Kanazawa. The latent class model ofbrand choice behaviors incorporating variety-seekingand state dependence. 2012.

[14] N. Koenigstein, G. Dror, and Y. Koren. Yahoo! musicrecommendations: modeling music ratings withtemporal dynamics and item taxonomy. In Proceedingsof the fifth ACM conference on Recommender systems,pages 165–172. ACM, 2011.

[15] Y. Koren. Collaborative filtering with temporaldynamics. Communications of the ACM, 53(4):89–97,2010.

[16] N. Lathia, S. Hailes, L. Capra, and X. Amatriain.Temporal diversity in recommender systems. In Proc.of the 33rd Annual International ACM SIGIRConference on Research and Development inInformation Retrieval (SIGIRaAZ10), pages 210–217,2010.

[17] L. McAlister. A dynamic attribute satiation model ofvariety-seeking behavior. Journal of ConsumerResearch, pages 141–150, 1982.

[18] L. McAlister and E. Pessemier. Variety seekingbehavior: An interdisciplinary review. Journal ofConsumer research, pages 311–322, 1982.

[19] S. McNee, J. Riedl, and J. Konstan. Being accurate isnot enough: how accuracy metrics have hurtrecommender systems. In CHI’06 extended abstractson Human factors in computing systems, pages1097–1101. ACM, 2006.

[20] J. Nakamura and M. Csikszentmihalyi. The concept offlow. Handbook of positive psychology, pages 89–105,2002.

[21] P. S. Raju. Optimum stimulation level: Itsrelationship to personality, demographics, andexploratory behavior. Journal of Consumer Research,pages 272–282, 1980.

[22] N. Sahoo, P. V. Singh, and T. Mukhopadhyay. Adynamic model of employee blog reading behavior.

[23] T. Zhou, Z. Kuscsik, J. Liu, M. Medo, J. Wakeling,and Y. Zhang. Solving the apparent diversity-accuracydilemma of recommender systems. Proceedings of theNational Academy of Sciences, 107(10):4511–4515,2010.

[24] C. Ziegler, S. McNee, J. Konstan, and G. Lausen.Improving recommendation lists through topicdiversification. In Proceedings of the 14th internationalconference on World Wide Web, pages 22–32. ACM,2005.

1069