impact of used methodology in risk prediction ...revistadestatistica.ro/supliment/wp-content/... ·...

Revista Română de Statistică - Supliment nr. 9 / 2019 135

IMPACT OF USED METHODOLOGY IN RISK PREDICTION: VARIANCE-COVARIANCE VS.

HISTORICAL SIMULATION

Associate lecturer Dumitru-Cristian OANEA PhD ([email protected])

„Artifex” University of Bucharest

Abstract

The purpose of this article is to identify the diff erence between

applying the variance-covariance method (used by the RiskMetrics model)

and the historical simulation, in quantifying the risk. As a result of the diff erent

assumptions behind these models, we conclude that the risk is more correctly

or less correctly identifi ed.

The main analyzed hypotheses that infl uence the ability of the models

to quantify the risk are: the way of estimating the decay factor, the supposed

distribution of returns, as well as the methodological framework used.

Keywords: RiskMetrics, decay factor, variance-covariance, historical

simulation

JEL Classifi cation: D81, G32

Introduction

The economic literature has delimited three approaches to quantifying

Value at Risk, namely: the variance-covariance approach (advantage =

easy application; disadvantage = predetermined hypothesis regarding the

distribution of the set of returns), the historical simulation and the Monte

Carlo simulation.

For the practical analysis, we will use the daily values of the stock

exchange indices for the capital markets of Central and Eastern Europe: BET

(Romania), BUX (Hungary), PX (Czech Republic), SAX (Slovakia), SOFIX

(Bulgaria) and WIG (Poland). The sample consists of 6 capital markets from

Central and Eastern Europe, including only those countries that are members

of the European Union and from which Romania imported more than 1% of the total value of imports from 2011-2013, according to the data, obtained from the offi cial website of the National Institute of Statistics.

Research methodology, data, results and discussions

Variance-covariance method

We will start from using the methodology proposed by JP. Morgan in

risk estimation, namely RiskMetrics model, based on which we estimated 3

types of models, namely:

Romanian Statistical Review - Supplement nr. 9 / 2019136

►RM1 – 0=l 0.94 and 2 types of distribution for returns: Normal

distribution (RiskMetrics-1994) and t-Student distribution (RiskMetrics-2006);

► RM2 – λ is estimated based on error checking function, and 2

types of distribution for returns: Normal distribution (RiskMetrics-1994) and

t-Student distribution (RiskMetrics-2006);

► RM3 – λ is estimated based on square errors, and 2 types of

distribution for returns: Normal distribution (RiskMetrics-1994) and t-Student

distribution (RiskMetrics-2006);

A fi rst fi nding concerns the manner of infl uence of a mathematical

model in the estimation can refer to the existence of diff erences in the

estimation of the decay factor. The diff erences are presented in table 1. Easelly

we can see that the hypothesize regarding the distribution of returns, and also

the selected level of confi dence infl uence in the end the estimated value for

decay factor. In this respect, a direct proportional relationship is observed

between the decay factor value and selected level of confi dence.

Decay factor estimation

Table 1+��'�)��

�

�

5��E�

4�� 4��!��

�

/�F �

G�� G��1��

;�#;�F/� � ;H#;�

F/� �;#;�

F/� � ;�#;�

F/� � ;H#;�

F/� � �;#;�

F/� �

*)+ ;'B:� ;'B8� ;'B<� ;'IB� � ;'BH� ;'B<� ;'IB�

*-. ;'BH� ;'BH� ;'B8� ;'B8� � ;'BC� ;'B8� ;'B8�

(.� ;'B;� 8�9: ;'B;� ;'IB� � ;'BH� ;'B:� ;'B;�

12.� ;'BJ� ;'B8� 8�98 8�.9 � ;'BH� 8�92 8�98

1345.� ;'B:� ;'BH� 8�.8 8�;: � ;'BC� 8�./ 8�;9

657� ;'B<� ;'B8� ;'B8� ;'BH� � 8�9; 8�99 ;'BH�0� ��#-�/�,��"��#��#�� #��1��2��,��

� � � � � � �

� � � � �

� � � �

� �

� � � �

��

� � � � � �

��

� � � � � � �

� � � � �

Legend: The table presents the estimates of the decay factor for the entire sample. Bold values

show discrepancies greater than 4% between the two functions based on which we made the

estimation

The estimation of decay factor value based on the function of the

square errors is presented in fi gure 1.


Estimation of the decay factor based on the function of the squared

errors

Fig. 1�

;'II

;'B;

;'B:

;'BC

;'B8

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

*)+

�

;'IC

;'II

;'B:

;'B8

�';;

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

*-.

��

;'IH

;'II

;'B;

;'B<

;'BH

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

(.

�

;'B;

;'B<

;'BH

;'BI

�';;

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

12.

;'IC

;'II

;'B:

;'B8:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

1345.

��

;'B;

;'B:

;'BC

;'B8

;'BI

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

657

��5 ��6��7�� #��,�"�� #��

� � � � � � �

� � � � �

� � � �

� �

� � � �

��

� � � � � �

��

� � � � � � �

� � � � �

Performance of estimated Value at Risk based on the model RM1 Table 2

IndexNormal distribution t-Student distribution10% 5% 1% 10% 5% 1%

The unconditional coverage test (Kupiec)Crisis period: 2008 – 2009BET 0.57 2.40 6.76 6.87 5.62 1.72 BUX 0.13 0.27 0.05 8.89 1.90 0.03 PX 0.25 8.12 2.72 8.19 12.08 0.36 SAX 2.74 1.47 25.98 0.08 1.90 10.21 SOFIX 0.03 5.62 23.42 4.04 11.02 6.76 WIG 0.25 1.90 14.17 11.96 3.54 3.90

Post-crisis period: 2010 – 2016BET 7.58 0.59 17.01 0.06 0.29 5.73 BUX 1.07 0.29 7.80 5.65 2.50 0.48 PX 0.06 6.24 23.50 8.41 15.02 3.94 SAX 3.71 0.92 32.69 0.01 5.27 7.80 SOFIX 2.82 1.36 12.71 1.74 3.20 4.80 WIG 1.07 5.27 25.24 3.42 15.02 3.16

The conditional coverage test (Christoff ersen)Crisis period: 2008 – 2009BET 3.57 4.54 7.43 13.34 7.90 2.07 BUX 1.81 0.98 0.23 13.96 2.14 0.16 PX 2.69 9.66 4.59 9.57 13.87 3.41 SAX 4.78 1.66 27.76 1.05 2.15 11.08 SOFIX 4.14 6.66 27.77 10.17 12.00 11.37 WIG 2.69 6.25 15.26 17.89 6.75 4.40


Note: Critical values for 2

)1(c are 2.706 (90%), 3.841 (95%) and 6.635 (99%). Critical values


for 2

)2(c are 4.605 (90%), 5.991 (95%) and 9.210 (99%). Bold values indicate that the model is

accepted (the probability of failure is equal to the selected confi dence level – α).

Performance of estimated Value at Risk based on the model RM2Table 3

IndexNormal distribution t-Student distribution

10% 5% 1% 10% 5% 1%The unconditional coverage test (Kupiec)

Crisis period: 2008 – 2009BET 0.78 4.18 6.76 8.19 5.62 0.36 BUX 0.06 0.03 0.05 6.25 2.94 0.05 PX 0.39 10.01 10.21 7.52 14.30 1.72 SAX 0.93 4.18 23.42 0.00 4.18 14.17 SOFIX 4.04 14.30 23.42 18.30 20.55 6.76 WIG 0.18 0.12 14.17 4.55 1.47 0.93






for 2



Finally, we checked the performance of the models used in the

estimation respectively RM1 (decay factor is 0.94), RM2 - decay factor is

estimated based on the error checking function and RM3 - decay factor is

estimated based on the squared error function for each index.


Performance of estimated Value at Risk based on the model RM3Table 4

IndexNormal distribution t-Student distribution


Crisis period: 2008 – 2009BET 0.57 3.54 6.76 6.87 6.41 1.72 BUX 0.25 0.75 0.36 9.62 2.40 0.03 PX 1.29 11.02 5.25 8.19 14.30 0.05 SAX 2.29 1.47 23.42 0.48 1.90 14.17 SOFIX 0.06 7.24 28.63 7.52 9.05 6.76 WIG 0.01 1.08 12.13 10.37 4.18 2.72






for 2



Analyzing the results presented in tables 2, 3 and 4, we fi nd that the best models that manage to capture the risk are represented by RM1 ( .0=l0.94) and RM2 (λ is estimated based on the error checking function).

The two models are accepted for over 50% of the indexes analyzed, both during the fi nancial crisis (2008 - 2009) and in the post-crisis period (2010 - 2016).

Historical simulation method

Using the second method of estimating individual risk, namely historical simulation, we estimated VaR for the same market indices analyzed.


VaR for period 2008 – 2016

Fig. 2�

��;9

�H9

;9

H9

�;9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

)��

*)+

��

��;9

�H9

;9

H9

�;9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

)��

*-.

�I9

�C9

;9

C9

I9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

)��

(.

�

��;9

�H9

;9

H9

�;9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

��/�L��L

*)+

��;9

�H9

;9

H9

�;9:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

��/�L��L

*-.

�I9

�C9

;9

C9

I9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

��/�L��L

(.

��

�I9

�C9

;9

C9

I9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

)��

12.

��

�I9

�C9

;9

C9

I9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

)��

1345.

�I9

�C9

;9

C9

I9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

)��

657

�

�I9

�C9

;9

C9

I9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

��/�L��L

12.

��

�I9

�C9

;9

C9

I9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

��/�L��L

1345.

�I9

�C9

;9

C9

I9

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

��/�L��L

657

��5 ��;��2�$��;88@�A�;86=��2�$�A�>>4'��2�$�A�>94'��2�$�A�>84��:

;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

:;;J

:;;I

:;;B

:;�;

:;��

:;�:

:;�<

:;�C

:;�H

:;�8

��5 ��;��2�$��;88@�A�;86=��2�$�A�>>4'��2�$�A�>94'��2�$�A�>84��

In order to highlight the impact of the size of the estimation sample,

we estimate the Value at Risk in two ways: taking into account all available

historical values, and also based on the use of a fi xed sample, using the rolling window method. The results obtained for the Value at risk can be seen in fi gure 2


Performance of estimated Value at Risk based on historical simulation

method

Table 5

IndexFull sample Rolling window


Crisis period: 2008 – 2009BET 68.14 82.35 31.36 68.14 77.73 25.98BUX 48.54 48.33 8.42 48.54 46.44 12.13PX 42.64 35.76 16.32 56.37 40.96 18.58SAX 6.30 0.33 0.36 3.75 0.00 0.36SOFIX 57.99 39.19 14.17 69.89 42.76 16.32WIG 44.09 24.75 23.42 45.55 27.73 23.42

Post-crisis period: 2010 – 2016BET 64.73 53.93 10.90 87.23 45.64 13.29BUX 16.92 13.63 3.19 40.23 33.32 4.28PX 6.69 12.73 5.57 12.34 17.58 7.08SAX 4.01 1.89 1.55 7.57 1.62 3.19SOFIX 52.96 38.28 16.08 54.35 41.85 13.29WIG 37.93 17.58 0.97 24.88 18.67 3.19

The conditional coverage test (Christoff ersen)Crisis period: 2008 – 2009BET 73.52 87.81 37.49 74.57 82.78 33.30BUX 51.67 54.11 9.32 53.28 52.70 12.71PX 45.33 41.64 18.81 57.98 47.05 20.76SAX 12.33 0.76 0.59 7.99 3.04 0.59SOFIX 66.91 47.61 20.68 79.74 51.91 22.23WIG 51.44 30.52 27.77 52.36 34.32 27.77

Post-crisis period: 2010 – 2016BET 66.57 54.30 10.94 91.52 45.81 13.32BUX 26.25 22.90 3.34 55.85 39.03 4.40PX 19.79 35.99 5.67 28.49 35.96 12.08SAX 9.88 5.80 4.62 14.59 3.53 6.90SOFIX 74.03 57.73 16.10 70.21 62.78 13.32WIG 60.58 44.65 16.40 50.70 41.96 3.34



for 2



The effi ciency of the estimates is tested using the two tests well known

in the economic literature, namely: the unconditional coverage test and the

conditional coverage test.

The results obtained are summarized in table 5, based on which we

can see that the Value at Risk manages to better capture the risk on the capital

market, only if we consider the confi dence level of 1%, and especially for the post-crisis period.


Comparison of variance-covariance methodology and historical

simulation

The purpose of this article was to estimate the individual risk existing

on the capital market, using for this purpose the indices of the capital markets

of Bulgaria, the Czech Republic, Poland, Romania, Slovakia and Hungary,

these being the most important capital markets in Central and Estern Europe.

When the variance-covariance method is applied in the risk estimation,

it can be observed that the models are mainly accepted in case of a t-Student

distribution of the returns, as well as of using 1% signifi cance level, which can be observed in the results presented in table 6.

The degree of acceptance of the variance-covariance models

Table 6

ModelNormal distribution t-Student distribution


Crisis period: 2008 – 2009RM1 83.3% 66.7% 33.3% 16.7% 50.0% 66.7%RM2 83.3% 33.3% 16.7% 16.7% 33.3% 66.7%RM3 100.0% 66.7% 33.3% 16.7% 33.3% 66.7%

Post-crisis period: 2010 – 2016RM1 50.0% 66.7% 0.0% 50.0% 50.0% 83.3%RM2 83.3% 50.0% 16.7% 50.0% 33.3% 83.3%RM3 66.7% 66.7% 16.7% 33.3% 50.0% 50.0%

The conditional coverage test (Christoff ersen)Crisis period: 2008 – 2009RM1 83.3% 50.0% 50.0% 16.7% 33.3% 66.7%RM2 50.0% 66.7% 33.3% 16.7% 50.0% 66.7%RM3 66.7% 50.0% 50.0% 16.7% 33.3% 66.7%

Post-crisis period: 2010 – 2016RM1 16.7% 50.0% 16.7% 33.3% 16.7% 100.0%RM2 16.7% 16.7% 16.7% 16.7% 16.7% 83.3%RM3 33.3% 50.0% 16.7% 16.7% 33.3% 66.7%

Table 7 presents the test results in the case of historical simulation. Based on these results, the importance of choosing the level of confi dence in risk estimation on capital markets is emphasized once again, as the only acceptable performance areas of historical estimation are the choice of a statistical signifi cance threshold of 1%, regardless of the estimation method: complete sample or rolling window. Comparing tables 6 and 7 we conclude that the variance-covariance methodology is more useful in estimating the risk on the capital market, generating better results compared to the historical simulation. In the same time, investors must also be aware of the impact that the assumption regarding the distribution of returns, as well as the selection of the degree of signifi cance have on the correctness of estimating the Value at Risk.


The degree of acceptance of the historical simulation models

Table 7

Model

Full sample Rolling window

10% 5% 1% 10% 5% 1%

The unconditional coverage test (Kupiec)

Crisis period: 2008 – 2009

Degree of acceptance 0.0% 16.7% 16.7% 0.0% 16.7% 16.7%

Post-crisis period: 2010 – 2016


The conditional coverage test (Christoff ersen)

Crisis period: 2008 – 2009


Post-crisis period: 2010 – 2016


Conclusions

This research aims to identify the impact of main mathematical assumptions of the models used in the estimation of the Value at risk, using two main estimation methods: the variance-covariance approach and the historical simulation. We have shown the impact on the results of the estimation of the following main hypotheses: the chosen confi dence level, the assumed distribution of the returns, the way of estimating the decay factor in the RiskMetrics model, as well as the size of the selected sample. In this regards, it can be observed that the models are accepted especially in the case of the assumption of a t-Student distribution of the used returns, as well as of the threshold of 1% statistical signifi cance.

The choice of the statistical confi dence level in the estimation of the

risk is of particular importance in the estimation, since the only acceptable

performance areas of the historical estimation are represented by the choice of

a statistical signifi cance threshold of 1%.

Finally, it can be observed that the variance-covariance methodology

is more useful in estimating the risk on the capital market, generating better

results compared to the historical simulation.


References

1. Anghelache, V. G.; Jakova, S.; Oanea, D. C., (2018). Politica fi scal-bugetară și piața de capital, Editura Economică, ISBN 978-973-709-840-5, București.

2. Anghelache, V. G.; Oanea, D. C.; Zugravu, B., (2013). General Aspects Regarding the Methodology for Prediction Risk, Romanian Statistical Review, Supplement

no.2, 66-71. 3. Billio, M., Getmansky, M., Lo, A. W., & Pelizzon, L. (2012). Econometric measures

of connectedness and systemic risk in the fi nance and insurance sectors. Journal of

Financial Economics, 104(3), 535-559. 4. Christoff ersen, P., (1998). Evaluating Interval Forecasts, International Economic

Review, 39(4), 841-862.

5. Gonzales-Rivera, G., Lee, T.H., Yoldas, E., (2007). Optimality of the RiskMetrics

VaR model. Finance Research Letters, 4, 137-145.

6. Granger, C. W. (1969). Investigating causal relations by econometric models and

cross-spectral methods. Econometrica: Journal of the Econometric Society, 424-

438.

7. Kupiec, P.H., (1995). Techniques for Verifying the Accuracy of Risk Measurement

Models. The Journal of Derivatives, 3, 73-84.

8. Markowitz, H. (1952). Portfolio selection. The journal of fi nance, 7(1), 77-91.

9. Morgan, J.P. (1996). Risk Metrics technology document (4th ed.).

10. Oanea, D. C.; Anghelache, V. G.; Zugravu, B. (2013), Econometric Model for

Risk Forecasting, Romanian Statistical Review, Supplement no.2, 123-127.

11. Roengpitya, R. & Rungcharoenkitkul, P. (2011). Measuring Systemic Risk

and Financial Linkages in the Thai Banking System, Systemic Risk, Basel III.

Financial Stability and Regulation 2011.

12. Soros G. (2008). The New Paradigm of Financial Markets. The credit crisis of

2008 and what it means, PublicAff airs, New York.

13. Tobin J., (1969), „A General Equilibrium Approach to Monetary Theory”, Journal

of Money, Credit and Banking, vol. 1, issue 1, 15-29, link: http://www.jstor.org/

stable/1991374.

14. de Vries, A. (2000). The Value at Risk, Working Paper, FH Südwestfalen

University of Applied Sciences, Germany.

15. Zugravu, B.; Oanea, D. C.; Anghelache, V. G. (2013), Analysis Based on the Risk

Metrics Model, Romanian Statistical Review, Supplement no.2, 145-154.

16. Zumbach, G., (2006). A gentle introduction to the RM 2006 methodology.

RiskMetrics Group, New York

impact of used methodology in risk prediction ...revistadestatistica.ro/supliment/wp-content/... ·...

Documents