SADNA – Ad Auctionlecture #3

Time Series

Yishay MansourMariano Schain

135791113151719212325272931333537394143454749

-1.5

-1

-0.5

0

0.5

1

1.5

Series1Series2Series3

1 5 9 13 17 21 25 29 33 37 41 45 49 53 570

5

10

15

20

25

30

Series1

1 5 9 13 17 21 25 29 33 37 41 45 49 53 570

10

20

30

40

50

60

70

80

90

Series1

1 5 9 13 17 21 25 29 33 37 41 45 49 53 570

5

10

15

20

25

30

35

Series1

1 5 9 13 17 21 25 29 33 37 41 45 49 53 570

5

10

15

20

25

30

35

40

Series1

Pg-dvd Pg-null

null-dvdnull-null

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 590

5

10

15

20

25

30

35

40

Series1Series2Series3

Bounded Window

• Parameter W• Take the average of the last W samples– Smooth the samples– Larger W – smoother outcome– Smaller W – better following recent trends

• Computationally – Requires keeping last W samples– Simple update:• St+1 St - yt-W /W + yt/W

Exponential Weight Moving average

• Parameter 0 < α < 1• Formula:

St+1 αyt + (1- α)St

• Effect of α:– Smaller α: larger weight to history– Large α: short reaction to trend– Effective window size: 1/ α

• Why “exponential”: St+1= α ∑ (1- α)j yt-j

135791113151719212325272931333537394143454749

-1.5

-1

-0.5

0

0.5

1

1.5

Series1Series2Series3

Discussion

• Setting up the parameters:– α = depends on the stability of the data

• Can be found by minimizing objective function• Rt = yt – St

• Minimize MSE = minα ∑ Rt2

– why MSE ?

• Note St+1 = St + α Rt

– S1 = undefined (need to initialize somehow)

• Problems:– Trend

Double Exponential Smoothing

• Handles trend• Parameters: α, γ• Formula:

St+1 αyt + (1- α)(St + bt)

equivalent: St+1 St + α(yt - (St + bt))

bt+1 γ (St+1 - St) + (1- γ) bt

• Motivation– St tracks the smoothed point

– bt tracks the smoothed slope

• Forecasting: Ft+1 = St + bt

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 490

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Series1Series2Series3Series4

Double Exponential Smoothing

• Simple Example:– yt = t

– Then: bt = 1 and St =t-1

• Residual Analysis– Rt = yt – St

– Again St+1 = St + α Rt

• Why not: bt+1 γ (yt – yt-1) + (1- γ) bt

Triple Exponential Smoothing

• Handles seasonality of cycle L– (Holt-Winters)

• Parameters: α, β, γ• Formula:

St+1 αyt /It+ (1- α)(St + bt)

bt+1 γ (St+1 - St) + (1- γ) bt

It+1 β yt / St + (1- β)It-L

Ft+1 (St + bt) It-L

Triple Exponential Smoothing

• Average and trend as before• It should be cyclic – cycle size L (how to find it?)– measures that ratio of current value to average.

• Illustrative:I=2

I=1

I=0.5

I=1

I=2

Linear Regression: Basics

• Simple model to fit the data• Basic example

a1 X1 + … + ap Xp

• Goal: minimize square error (MSE)∑t (a1 X1

t + … + ap Xpt – Yt)2

– The constants are the X’s and Y’s– We are solving for the coefficients: a’s

Linear Regression

• What counts as linear ?!– a0 + a1 x

– a0 + a1 x + a2x2 + a3 x3

– a1x1 + a2 x2 + a3 x1x2 + a4 x12 + a5 x2

2

– a1 log x1 + a2 exp(x2)

– a0 + a1 a0 x1

– a + x/a

Linear Regression: Computation

• Simple case: ax + b• Minimize ∑t (a xt + b – yt)2

• Need to solve for a and b∑t 2 (a xt + b – yt) xt = 0

∑t 2 (a xt + b – yt) =0

• The equations:a*avg(x2) +b*avg(x) = avg (xy)

a * avg(x) + b = avg(y)

• The Solution

xayb

xx

yyxxa

i

ii

2)(

))((

Non-Linear Regression

• We need to optimize the MSE• Now the derivatives are not linear– in the a’s

• Need a more complicate solver• There are software out there that do it– non-linear fitting procedures

Autoregressive Model (AR)

• A model of the current value – given previous observed values

• Model– Xt = c0 + a1 Xt-1 + a2 Xt-2 + … + ap Xt-p + εt

• usually ∑ ai < 1

• c0 = (1- ∑ ai ) E[X]

– Need to solve for the coefficients– Simple linear regression

Moving Average Model (MA)

• A model of the current value – given previous unobserved residuals

• Model:– Xt = μ + εt - b1 ε t-1 - b2 ε t-2 - … - bq ε t-q

• This is a linear regression in the residuals ε’s• PROBLEM: we do not observe the residuals

directly– non-linear fitting procedures

ARIMA(p,q)

• Combines – an autoregressive model (AR)• p values back

– a moving average (MA)• q values back

• Model:Xt = c0 + a1 Xt-1 + a2 Xt-2 + … + ap Xt-p

+ εt - b1 ε t-1 - b2 ε t-2 - … - bq ε t-q

Detecting change

• When is there a shift• Consider the Residuals– Rt = yt – Ft

• Stable residuals No change• Much higher residuals maybe change• In the simulation: detect a burst

How can you use this

• Understand the concepts• Understand the alternatives• There is enough software you can use– you will need to select the model– specify the input– understand what the output means

• If you use software, remember:– document which software you use– document where you use it