11

European Real Estate Society European Real Estate Society ConferenceConference

Stockholm, Sweden, 24-27 June 2009Stockholm, Sweden, 24-27 June 2009

Greg CostelloGreg CostelloCurtin University of TechnologyCurtin University of Technology

Perth, Western AustraliaPerth, Western Australia

Yen Min GohYen Min GohDepartment of FinanceDepartment of Finance

The University of MelbourneThe University of Melbourne

Greg SchwannGreg SchwannDepartment of FinanceDepartment of Finance

The University of MelbourneThe University of Melbourne

The Accuracy and Robustness of The Accuracy and Robustness of Real Estate Price Index MethodsReal Estate Price Index Methods

2

Research QuestionResearch Question

► How accurate and robust are How accurate and robust are different house price index methods different house price index methods when subjected to finer levels of when subjected to finer levels of aggregation:aggregation:

1.1. Temporal – monthly time intervalsTemporal – monthly time intervals

2.2. Geographic – suburb specificationsGeographic – suburb specifications

Chart 1:transaction volume by Chart 1:transaction volume by monthmonth

0

500

1000

1500

2000

2500

3000

3500

4000

4500

Jan-

88

Jul-8

8

Jan-

89

Jul-8

9

Jan-

90

Jul-9

0

Jan-

91

Jul-9

1

Jan-

92

Jul-9

2

Jan-

93

Jul-9

3

Jan-

94

Jul-9

4

Jan-

95

Jul-9

5

Jan-

96

Jul-9

6

Jan-

97

Jul-9

7

Jan-

98

Jul-9

8

Jan-

99

Jul-9

9

Jan-

00

Jul-0

0

Jan-

01

Jul-0

1

Jan-

02

Jul-0

2

Jan-

03

Jul-0

3

Jan-

04

Jul-0

4

Jan-

05

Month

Tra

nsa

ctio

n V

olu

me

Transaction Volume Average Transaction Volume Per Month

Chart 2:Chart 2: Transaction volume by Transaction volume by suburbsuburb

0

2000

4000

6000

8000

10000

Suburb

Tra

ns

ac

tio

n V

olu

me

Transaction Volume Average Transaction Volume Per Suburb

5

Two important issuesTwo important issues

1.1. Why do we need to pool data?Why do we need to pool data? Sample sizeSample size Does the pooled sample represent the Does the pooled sample represent the

equivalent subsamples?equivalent subsamples?

2.2. What is the “true” price trend?What is the “true” price trend? It is unobservableIt is unobservable How do we proxy a true price trend to How do we proxy a true price trend to

compare relative performance of compare relative performance of indexes?indexes?

6

Two important papersTwo important papers

Englund Quigley Redfearn (1999) “The Englund Quigley Redfearn (1999) “The choice of methodology for computing choice of methodology for computing housing price indexes: Comparisons of housing price indexes: Comparisons of temporal aggregation and sample temporal aggregation and sample definition” (JREFE)definition” (JREFE)

Diewert Heravi Silver (2007) “Hedonic Diewert Heravi Silver (2007) “Hedonic imputation versus time dummy hedonic imputation versus time dummy hedonic indexes” indexes” International Monetary Fund International Monetary Fund Working Paper No. 07/234Working Paper No. 07/234..

7

Our approachOur approach

1.1. Rigorously test different indexes with Rigorously test different indexes with different aggregation formatsdifferent aggregation formats

2.2. Use out of sample technique 75% of Use out of sample technique 75% of data to estimate, then 25% to data to estimate, then 25% to forecast in order to overcome forecast in order to overcome unobservable “true” price trendunobservable “true” price trend

Table 1: DataTable 1: Data

Mean (Standard Deviation)

Split-samples (% of Full Sample) Hedonic Characteristics Full Sample

75% 25% Number of Single-Sale Transactions 252,470 189,351 63,118 Number of Repeat-Sale Transactions 254,748 191,061 63,687

Total Number of Transactions 507,218 380,412 126,805 Number of Sub-regions 3

Number of Districts 42 Number of Suburbs 299

173,801 173,790 173,835

General

Sale Price ($) (125,252) (125,235) (125,304)

653.0 653.1 652.9 Area (square meters)

(487.9) (487.8) (488.2) 7.542 7.542 7.542

Total Number of Rooms (2.217) (2.217) (2.218)

0.4542 0.4545 0.4535

Size

Ratio of Bathroom(s) to Bedroom(s) (0.1617) (0.1623) (0.1600)

Testing the influence of Geographic and Testing the influence of Geographic and Temporal AggregationTemporal Aggregation

1 1 1 1

T k T k

it t t i i it itt i t i

v T L x

Table 2: Framework for Testing Geographic and Table 2: Framework for Testing Geographic and

Temporal AggregationTemporal Aggregation

Suburb-Level District-Level Sub-regional Regional

Case 1 (Base Case) Case 2 Case 3 Case 4 Monthly

(0) (257) (296) (298)

Case 5 Case 6 Case 7 Case 8 Quarterly

(140) (397) (436) (438)

Case 9 Case 10 Case 11 Case 12 Semi-Annually

(175) (432) (471) (473)

Case 13 Case 14 Case 15 Case 16 Annually

(192) (449) (488) (490)

Base case is the highest level of aggregation, monthly-suburb, “unrestricted”

Table 2: Framework for Testing Geographic and Table 2: Framework for Testing Geographic and

Temporal AggregationTemporal Aggregation

Suburb-Level District-Level Sub-regional Regional

Case 1 (Base Case) Case 2 Case 3 Case 4 Monthly

(0) (257) (296) (298)

Case 5 Case 6 Case 7 Case 8 Quarterly

(140) (397) (436) (438)

Case 9 Case 10 Case 11 Case 12 Semi-Annually

(175) (432) (471) (473)

Case 13 Case 14 Case 15 Case 16 Annually

(192) (449) (488) (490)

Case 16 is the lowest level of aggregation, annual-region, most restricted

Table 3: F Statistics for Different Model Table 3: F Statistics for Different Model RestrictionsRestrictions

Panel (a): F-ratios comparing different models of decreasing geographic and temporal aggregation to the most disaggregated (monthly-suburb) model

Panel (a): F-ratios comparing different models of decreasing geographic and temporal aggregation

to the most disaggregated (monthly-suburb) model

F-statistics (Critical F Value = 1.17)

Suburb District Sub-region Region

Monthly 0.00 374.40 1416.58 1607.48

Quarterly 2.99 243.58 1051.97 1095.13

Semi Annually 11.24 227.32 977.24 1017.43

Annually 31.13 227.01 949.97 988.11

Table 3: F Statistics for Different Model Table 3: F Statistics for Different Model RestrictionsRestrictions

Panel (b): F-ratios comparing different models of decreasing geographic aggregation

F-statistics (Critical F Value = 1.17)

Districts Sub-regions Regions

Suburbs 374.40 1416.58 1607.48

Districts 6946.47 7829.05

Sub-region 16221.97

Panel (c): F-ratios comparing different models of decreasing temporal aggregation

F-statistics (Critical F Value = 1.17)

Quarters Half Years Years

Months 2.99 11.25 31.15

Quarters 44.25 106.89

Half Years 235.15

Note pronounced influence of geographic

disaggregation

14

Five house price index Five house price index methodsmethods

i.i. The hedonic imputation modelThe hedonic imputation model

ii.ii. The longitudinal hedonic approachThe longitudinal hedonic approach

iii.iii. An augmented weighted repeat-sales An augmented weighted repeat-sales (WRS) model(WRS) model

iv.iv. The Quigley (1995) hybrid modelThe Quigley (1995) hybrid model

v.v. The mix-adjusted median methodThe mix-adjusted median method

Table 4: Index Accuracy - Mean Squared ErrorsTable 4: Index Accuracy - Mean Squared Errors Region Sub-region District Suburb

Mean Squared Error

Hedonic Imputation

Annually 0.163 0.150 0.088 0.069

Semi-Annually 0.163 0.149 0.087 0.069

Quarterly 0.162 0.149 0.087 0.068

Monthly 0.162 0.149 0.087 0.068

Longitudinal Hedonic

Annually 0.214 0.227 0.242 0.311

Semi-Annually 0.221 0.236 0.252 0.308

Quarterly 0.255 0.267 0.279 0.310

Monthly 0.321 0.328 0.328 0.318

Hybrid

Annually 0.188 0.199 0.212 0.273

Semi-Annually 0.194 0.206 0.221 0.269

Quarterly 0.223 0.233 0.244 0.272

Monthly 0.281 0.287 0.287 0.278

Repeat-sales

Annually 0.609 0.608 0.604 0.609

Semi-Annually 0.617 0.618 0.615 0.640

Quarterly 0.824 0.830 0.873 2.785

Monthly 0.989 1.977 3.659 127.463

Mix-Adjusted Median

Annually 0.424 0.450 0.479 0.616

Semi-Annually 0.438 0.466 0.499 0.608

Quarterly 0.505 0.528 0.551 0.614

Monthly 0.636 0.648 0.650 0.628

Chart 3A: MSE Surface Plot-The Hedonic Imputation Chart 3A: MSE Surface Plot-The Hedonic Imputation ModelModel

Table 4: Index Accuracy - Mean Squared ErrorsTable 4: Index Accuracy - Mean Squared Errors Region Sub-region District Suburb

Mean Squared Error

Hedonic Imputation

Annually 0.163 0.150 0.088 0.069

Semi-Annually 0.163 0.149 0.087 0.069

Quarterly 0.162 0.149 0.087 0.068

Monthly 0.162 0.149 0.087 0.068

Longitudinal Hedonic

Annually 0.214 0.227 0.242 0.311

Semi-Annually 0.221 0.236 0.252 0.308

Quarterly 0.255 0.267 0.279 0.310

Monthly 0.321 0.328 0.328 0.318

Hybrid

Annually 0.188 0.199 0.212 0.273

Semi-Annually 0.194 0.206 0.221 0.269

Quarterly 0.223 0.233 0.244 0.272

Monthly 0.281 0.287 0.287 0.278

Repeat-sales

Annually 0.609 0.608 0.604 0.609

Semi-Annually 0.617 0.618 0.615 0.640

Quarterly 0.824 0.830 0.873 2.785

Monthly 0.989 1.977 3.659 127.463

Mix-Adjusted Median

Annually 0.424 0.450 0.479 0.616

Semi-Annually 0.438 0.466 0.499 0.608

Quarterly 0.505 0.528 0.551 0.614

Monthly 0.636 0.648 0.650 0.628

Chart 3D: MSE Surface Plot - The Augmented Repeat-sales ModelChart 3D: MSE Surface Plot - The Augmented Repeat-sales Model

19

ConclusionsConclusions

1.1. The aggregation of data, whether along The aggregation of data, whether along temporal or geographic definitions, is temporal or geographic definitions, is generally unwarrantedgenerally unwarranted

2.2. Price indexes should be estimated using the Price indexes should be estimated using the most disaggregated dataset availablemost disaggregated dataset available

3.3. Convincing evidence that the hedonic Convincing evidence that the hedonic imputation method performs significantly imputation method performs significantly better than four other methods considered better than four other methods considered on all measures of accuracy and robustnesson all measures of accuracy and robustness

**assuming that high-quality data is **assuming that high-quality data is available**available**

20

Further research?Further research?

1.1. The economic significance of The economic significance of differences in various index methods, differences in various index methods, does it really matter?does it really matter?

2.2. IImportant within the context of mportant within the context of developing derivative products developing derivative products applied to property marketsapplied to property markets

3.3. The values of derivative contracts in The values of derivative contracts in housing markets would be sensitive housing markets would be sensitive to any underlying house price index?to any underlying house price index?