time series analysis negar koochakzadeh. outline introduction: time series data stationary /...
Post on 28-Mar-2015
217 Views
Preview:
TRANSCRIPT
Time Series Analysis
Negar Koochakzadeh
Outline Introduction:
Time Series Data Stationary / Non-stationary TS Data Existing TSA Models
AR (Auto-Regression) MA (Moving Average) ARMA (Auto-Regression Moving Average) ARIMA (Auto-Regression Integrated Moving Average) SARIMA (Seasonal ARIMA)
Examples Example 1: International Airline Passenger Example 2&3: Energy Load Prediction
Time Series Data Mining Time Series Classification (SVM)
Example Example 4: Stock Market Analysis
Time Series Data In many fields of study, data is collected from
a system over time. This sequence of observations generated a
time series: Examples:
Closing prices of the stock market A country’s unemployment rate Temperature readings of an industrial furnace Sea level changes in coastal regions Number of flu cases in a region Inventory levels at a production site
Temporal Behaviour Most physical processes do not change
quickly, often makes consecutive observation correlated. Correlation between consecutive observation is
called autocorrelation.
Most of the standard modeling methods based on the assumption of independent observations can be misleading.
We need to consider alternative methods that take into account the serial dependence in the data.
Stationary Time Series Data Stationary time series are characterized by
having a distribution that is independent of time shifts.
Mean and variance of these time series are constants
If arbitrary snapshots of the time series we study exhibit similar behaviour in central tendency and spread, we can assume that the time series is indeed stationary.
Stationary or Non-Stationary? In practice, there is no clear demarcation line
between a stationary and a non-stationary process.
Some methods to identify: Visual inspection Using intuition and knowledge about the process Autocorrelation Function (ACF) Variogram
Visual Inspection A properly constructed graph of a time series
can dramatically improve the statistical analysis and accelerate the discovery of the hidden information in the data.
“You can observe a lot by watching.” This is particularly true with time series data analysis! [Yogi Berra, 1963]
Intuition and knowledge Inspection Does it make sense...
for a tightly controlled chemical process to exhibit similar behaviour in mean and variance in time?
to expect the stock market out it “to remain in equilibrium about a constant mean level”
The selection of a stationary or non-stationary model must often be made on the basis of not only the data but also a physical understanding of the process.
Autocorrelation Function (ACF) Autocorrelation is the cross-
correlation of a time series data with itself based on lag k
ACF summarizes as a function of k, how correlated the observations that are k lags apart are.
If the ACF does not dampen out then the process is likely not stationary(If a time series is non-stationary, the ACF will not die out quickly)
𝐴𝐶𝐹(𝐾) = 𝐶𝑜𝑟𝑟(𝑍𝑡,𝑍𝑡−𝐾)
Variogram The Variogram Gk measures the variance of
differences k time units apart relative to the variance of the differences one time unit apart
For stationary process, Gk when plotted as a function of k will reach an asymptote line. However, if the process is non-stationary, Gk will increase monotonically.
𝐺𝑘 = 𝑉 { 𝑍𝑡+𝑘 − 𝑍𝑡 }𝑉 { 𝑍𝑡+1 − 𝑍𝑡 }
Modeling and Prediction “If we wish to make predictions, then clearly we must
assume that something does not vary with time.” [Brockwell and Davis, 2002]
Let’s try to predict and build a model for our time series process based on: Serial Dependency Leading Indicators Disturbance
True disturbances caused by unknown and/or uncontrollable factors that have direct impact on the process.
It is impossible to come up with a comprehensive deterministic model to account for all these possible disturbances, since by definition they are unknown.
In these cases, a probabilistic or stochastic model will be more appropriate to describe the behaviour of the process.
Notations Backshift Operator
∇𝑍𝑡 = 𝑍𝑡 − 𝑍𝑡−1 ∇2𝑍𝑡 = ∇ሺ∇𝑍𝑡ሻ= 𝑍𝑡 − 2𝑍𝑡−1 + 𝑍𝑡−2 ∇𝑠𝑍𝑡 = 𝑍𝑡 − 𝑍𝑡−𝑠
𝐵𝑍𝑡 = 𝑍𝑡−1
∇𝑍𝑡 = (1− 𝐵)𝑍𝑡
∇ሺ∇𝑍𝑡ሻ = (1− 𝐵2)𝑍𝑡
∇𝑠𝑍𝑡 = (1− 𝐵)𝑠𝑍𝑡
𝑍෨𝑡 = 𝑍𝑡 − 𝜇 𝑍መ𝑡−𝑠(𝑘) = 𝑓𝑜𝑟𝑐𝑎𝑠𝑡 𝑜𝑓 𝑍𝑡−𝑠+𝑘 𝑎𝑡 𝑡𝑖𝑚𝑒 𝑡− 𝑠
Auto-Regressive Models AR(P)
Where at is an error term (called white error) assumed to be uncorrelated with zero mean and constant variance.
The random error at cannot be observed. Instead we estimate it by using the one-step-ahead forecast error
The regression coefficients , i = 1, ... , p, are parameters to be estimated from the data
𝑎𝑡 = 𝑍𝑡 − 𝑍መ𝑡−1(1)
𝜑𝑖
𝑍𝑡 = 𝜑1𝑍𝑡−1+ ...+ 𝜑1𝑍𝑡−𝑝 + 𝑎𝑡
Moving Average Current and previous disturbances affect the
value. We have a sequence of random shocks
bombarding the system and not just a single shock.
MA(q)
Uncorrelated random shocks with zero mean and constant variance
The coefficients , i = 1, ... , q are parameters to be determined from the data
𝑍𝑡 = 𝑎𝑡 − 𝜃1𝑎𝑡−1− ...− 𝜃𝑞𝑎𝑡−𝑞
𝜃𝑖
Auto-Regressive Moving Average ARMA(p,q)
Typical stationary time series models come in three general classes, auto-regressive (AR) models, moving average (MA) models, or a combination of the two (ARMA).
𝑍𝑡 = 𝜑1𝑍𝑡−1+ ...+ 𝜑1𝑍𝑡−𝑝 + 𝑎𝑡 − 𝜃1𝑎𝑡−1− ...− 𝜃𝑞𝑎𝑡−𝑞
Identifying appropriate Model
The ACF plays an extremely crucial role in the identification of time series models
The identification of the particular model within ARMA class of models is determined by looking at the ACF and PACF.
Partial Autocorrelation Function (PACF) Partial Autocorrelation is the partial cross-correlation of
a time series data with itself based on lag k Partial correlation is a conditional correlation:
It is the correlation between two variables under the assumption that we know and take into account the values of some other set of variables
How Zt and Zt-k are correlated taking into account how both Zt and Zt-k are related to Zt-1 , Zt-2 , ... , Zt-k+1
The kth order PACF measure correlation between Zt and Zt+k after adjustments have been made for the intermediate observations Zt-1 , Zt-2 , ... , Zt-k+1
where denotes the projection of x onto the space spanned by Zt-1 , Zt-2 , ... , Zt-k+1
𝑃𝑡,𝑘(𝑥)
𝑃𝐴𝐶𝐹(𝐾) = 𝐶𝑜𝑟𝑟 (𝑍𝑡 − 𝑃𝑡,𝑘(𝑍𝑡) ,𝑍𝑡−𝐾− 𝑃𝑡,𝑘(𝑍𝑡−𝐾))
ARMA Model identification from ACF and PACF
ACF
PACF
AR(p) MA(q) ARMA(p, q)
Infinite damped exponentials and/or damped sine waves; Tails off
Infinite damped exponentials and/or damped sine waves; Tails off
Infinite damped exponentials and/or damped sine waves; Tails off
Infinite damped exponentials and/or damped sine waves; Tails off
Finite; cuts off after q lags
Finite; cuts off after p lags
Source: Adapted from BJR
Examples
Models for Non-Stationary Data Standard autoregressive moving average (ARMA)
time series models apply only to stationary time series.
The assumption that a time series is stationary is quite unrealistic. (Stationary is not natural!)
For a system to exhibit a stationary behaviour, it has to be tightly controlled and maintained in time.
Otherwise, systems will tend to drift away from stationary
Converting Non-Stationary Data to Stationary
More realistic is to claim that the changes to a process, or the first difference, form a stationary process.
And if that is not realistic, we mat try to see if the changes of the changes, the second difference, form a stationary process.
If that is the case, we can then model the changes, make forecasts about the future values of these changes, and from the model of the changes build models and create forecasts of the original non-stationary time series.
In practice, we seldom need to go beyond second order differencing.
Auto Regressive Integrated Moving Average(ARIMA) In the case of non-stationary data,
differencing before we use the (stationary) ARMA model to fit the (differenced) data is appropriate.
Because the inverse operation of differencing is summing or integrating, an ARMA model applied to d differenced data is called an autoregressive integrated moving average process, ARIMA (p, d, q).
In practice, the orders p, d, and q are seldom higher than 2.
wt = ∇𝑍𝑡 = 𝑍𝑡 − 𝑍𝑡−1
𝑤𝑡 = ∇𝑑𝑍𝑡
Stages of the time series model building process using ARIMA
Consider a generalARIMA Model
Identify the appropriatedegree of differencing if
neededUsing ACF and
PACF, find a tentative model
Estimate the parameters of the
model using appropriate software
Perform the residual analysis.
Is the model adequate?
Start forecasting
Model Evaluation Once a model has been fitted to the data, we
process to conduct a number of diagnostic checks.
If the model fits well, the residuals should essentially behave like white noise.
In other words, the residuals should be uncorrelated with constant variance.
Standard checks are to compute the ACF and PACF of the residuals.
If they appear in the confidence interval there is no alarm indications that the model does not fit well.
Exponentially Weighted Moving Average Special case of ARIMA model: EWMA
Unlike a regular average that assigns equal weight to all observation, an EWMA has a relatively short memory that assigns decreasing weights to past observations.
EWMA made practical sense that a forecast should be a weighted average that assigns most weight to the most immediate past observation, somewhat less weight to the second to the last observation, and so on.
It just made good practical sense.
𝑍𝑡 = (1− 𝜃)(𝑍𝑡−1 + 𝜃𝑍𝑡−2 + 𝜃2𝑍𝑡−3+...) + 𝑎𝑡
|𝜃| < 1
Seasonal Models For ARIMA models, the serial dependence of
the current observation to the previous observations was often strongest for the immediate past and followed a decaying pattern as we move further back in time.
For some systems, this dependence shows a repeating, cyclic behaviour.
This cyclic pattern or as more commonly called seasonal pattern can be effectively used to further improve the forecasting performance.
The ARIMA models are flexible enough to allow for modeling both seasonal and non-seasonal dependence.
Example 1: International Airline Passengers
Trend and Seasonal Relationship Two relationship going on simultaneously:
Between observations for successive months within the same year
Between observation for the same month in successive years.
Therefore, we essentially need to build two time series models, and then combine the two.
If the season is s period long, in this example s = 12 months, then observation that are s time intervals apart are alike.
Log Transformation
Pre-Processing
Apply Differencing on Seasonal Data For seasonal data, we may need to use not
only regular difference but also a seasonal difference .
Sometimes, we may even need both (e.g., ) to obtain an ACF that dies out sufficiently quickly.
∇𝑍𝑡 ∇𝑠𝑍𝑡
∇∇𝑠𝑍𝑡
Investigate ACFs Only the last one (combination of regular
difference and seasonal difference) is stationary:
Model Identification Identifying stationary seasonal models is a
modification of the one used for regular ARMA time series models where the patterns of the sample ACF and PACF provide guidance.
First, look for similarities that are 12 lags apart. ACF seems to cut off after the first one (in k=12). This is a sign of a Moving Average Model applied to
the 12-month seasonal pattern. Second, look for patterns between successive months
ACF seems to cut off after the first one First order MA term in the regular model
AC
FPA
CF
Model Evaluation ACF of the residuals after fitting a first order
SMA model to :
We see that the ACF shows a significant negative spike at lag 1, indicating that we need an additional regular moving average term
∇∇12𝑍𝑡
ARIMA (p,d,q)*(P,D,Q)12
𝑊𝑡 = 𝑏𝑡 − Θ1bt−12
𝑏𝑡 = 𝑎1 − 𝜃1𝑎𝑡−1
∇∇12𝑍𝑡 = ሺ𝑎1 − 𝜃1𝑎𝑡−1ሻ− Θ1 (𝑎𝑡−12 − 𝜃1𝑎𝑡−13) 𝑍𝑡 − 𝑍𝑡−1 − 𝑍𝑡−12 + 𝑍𝑡−13 = 𝑎1 − 𝜃1𝑎𝑡−1 − Θ1𝑎𝑡−12 − Θ1𝜃1𝑎𝑡−13
Example 2: Energy Peak Load Prediction The hourly peak load follows a daily periodic pattern
S=24 hours Covert peak load values into and then apply ARMA
∇24𝑍𝑡
AC
F
PAC
F
Example 3: Energy Load Prediction Daily, weekly, and monthly periodic patterns
Exogenous Variables (Temperature)
They proposed to apply Periodic Auto-Regression (PAR)
* An auto-regression is periodic when the parameters are allowed to vary across seasons.
Example 3 (cont’d) Proposed model template:
Seasonality varying intercept term
Dummy variable for weekly seasonal
Dummy variable for monthly seasonal
Exogenous variable for temperaturesensitivity
Time Series Data Mining Using Serial Dependency of forecasting variable to
build the training set.
Leading indicators might exhibit similar behaviour to forecasting variable
The important task is to find out whether there exists a lagged relationship between indicators and predicted variable
If such a relationship exists, then from the current and past behaviour of the leading indicators, it may be possible to determine how the sales will behave in the near future.
Time Series SVM Optimization problem in SVM:
Error in SVM:
Error in Modified SVM:
Example 4: Stock Market Analysis Portfolio optimization is the decision process
of asset selection and weighting, such that the collection of assets satisfies an investor’s objectives
Serial dependency or Lagged Relationship between stock performance and financial indicators from the companies.
Bloomberg Mnemonic
Description
PROF_MARGINIndicates how much out of every dollar of sales, the company actually keeps in earning : Net Income / Revenue
RETURN_ON_ASSET
Quantifies the companies success of effort to earn a profit with respect to its total asset:Net Income / Total Assets
RETURN_ON_CAPQuantifies the companies success of effort to earn a profit with respect to its capital:Net Income / (Total Assets - Current Liabilities)
ROA_TO_ROE
Quantifies the ratios of Return On Asset to Return On Equity (ROE: Net Income as a percentage of shareholders' equity):Shareholder's Equity / Total Assets
ROA_BASED_ON_BOTTOM_EPS
Indicates Return On Asset calculated based on the last line of the company's income statement. This reflects the fact that all expenses have already been taken out of revenues, and there is nothing left to subtract.
REVENUE_PER_SH
Indicates Revenue with respect to each share price. Revenue is the income that a company receives from its normal business activities, usually from the sale of goods and services to customers.
RETENTION_RATIO
Quantifies the percent of earnings credited to retained earnings:(Net Income - Dividends)/Net Income
TOT_DEBT_TO_TOT_ASSET
Quantifies company's financial risk by determining how much of the company's assets have been financed by debt:Debt / Total Assets
TOT_DEBT_TO_TOT_CAP
Quantifies company's financial leverage:Debt / (Shareholder's Equity + Debt)
TOT_DEBT_TO_TOT_EQY
Indicates what proportion of equity and debt the company is using to finance its assets:Total Liabilities / Shareholder's Equity
TOT_INVEST_TO_TOT_LIAB
Indicates total Investement of the company to the total liabilities:Total Investment / Total Liabilities
Stock Ranking Learn relationship between stocks’ current
features and their future rank score. (Lagged Relationship)
By Applying modified version of SVM Rank Algorithm for time series based on exponential weighted error.t0 t1 t2
…
tk tcΔtS Δtr
Training Set Testing Set
Stoc
k N
ame
f1 f2
futu
re R
OI
Targ
etRa
nk S
core
S1 0.5 0.9 0.5 1
S2 0.3 0.1 -0.1 3
S3 0.4 0.5 0.2 2
Stoc
k N
ame
f1 f2
futu
re R
OI
Targ
etRa
nk S
core
S1 0.3 0.7 0.6 1
S2 0.2 0.3 0.6 1
S3 0.7 0.1 0.4 2
Stoc
k N
ame
f1 f2
futu
re R
OI
Targ
etRa
nk S
core
S1 0.4 0.2 -0.3 3
S2 0.5 0.6 0.1 2
S3 0.1 0.7 0.7 1
Stoc
k N
ame
f1 f2
futu
re R
OI
Targ
etRa
nk S
core
S1 0.5 0.9 0.4 2
S2 0.8 0.2 0.5 1
S3 0.7 0.7 0.1 3
Stoc
k N
ame
f1 f2
futu
re R
OI
Targ
etRa
nk S
core
Pred
icte
dRa
nk S
core
S1 0.4 0.3 0.4 2 1
S2 0.5 0.9 -0.7 3 3
S3 0.7 0.2 0.6 1 2
References[1] Søren Bisgaard and M. Kulahci, TIME SERIES ANALYSIS AND
FORECASTING BY EXAMPLE: A JOHN WILEY & SONS, INC., 2011.
[2] Rayman Preet Singh, Peter Xiang Gao, and Daniel J. Lizotte, "On Hourly Home Peak Load Prediction," in IEEE SmartGridComm, 2012.
[3] Marcelo Espinoza, Caroline Joye, Ronnie Belmans, and Bart De Moor, "Short-Term Load Forecasting, Profile Identification, and Customer Segmentation: A Methodology Based on Periodic Time Series," Power Systems, vol. 20, pp. 1622-1630, 2005.
[4] F. E. H. Tay and L. Cao, "Modified support vector machines in financial time series forecasting," Neurocomputing, vol. 48, pp. 847-861, 2002
Questions?
top related