Upload File

module 2. statistical learning example 2 - … filemodule 2. statistical learning example 2. ... •...

  • Home
  • Documents
  • Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First
21
Eurostat THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION Module 2. Statistical learning Example 2

Upload: dinhdung

Post on 28-Aug-2018

221 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

THE CONTRACTOR IS ACTING UNDER A FRAMEWORK CONTRACT CONCLUDED WITH THE COMMISSION

Module 2. Statistical learning

Example 2

Page 2: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

2

First steps with R

Example: data of example 1. Recall

• Regression Tree for classification.

• Basics

• Pruning

• Cross Validation

• Regression Tree for prediction.

• Bagging

• Random forest

• Boosting

Page 3: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

3

First steps with R

Example: data of example 1. Recall

• Objective: Mass appraisal, or automatic valuation of alarge number of real estate assets.

• Data base: information of 2,141 apartments in the cityof Alicante (Spain):• Apartment characteristics

Price Area Floor Rooms

Bedrooms Bathrooms Terrace

• Block characteristics

Number of apart Lifts Number of floors

Age Community spaces

• Neighborhood characteristics

Location Commerce Neighborhood

Page 4: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

4

First steps with R

Example 2: Descriptive

mean sd median min max

Price 196234.21 151422.03 165000 27550 1745000

Area 103.25 33.09 100 25 300

Number_of_apartments 19.79 29.84 15 1 496

Number_of_floors 6.84 3.23 6 3 35

Floor 4.36 2.42 4 1 24

Rooms 6.54 1.28 7 2 14

Bedrooms 2.95 0.86 3 1 6

Bathrooms 1.60 0.55 2 1 5

Age 18.65 11.74 25 0 40

Neighborhood* 0.91 0.29 1 0 1

Commerce* 0.92 0.27 1 0 1

Community_spaces** 0.21 0.41 0 0 1

Location* 0.26 0.44 0 0 1

Lifts*** 0.70 0.46 1 0 1

Terrace*** 0.13 0.34 0 0 1

* Binary variable: 1 Good; 0 Fair/Bad

** Binary variable: 1 With; 0 Without

*** Binary variable: 1 Yes; 0 No

Page 5: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

5

First steps with R

Example 2: Regression Tree Approach

• Data set: 2,141 apartments in Alicante, Spain.• 1,070 observation in the training set• 1,071 observation in the test set

• Ten-fold cross-validation

Page 6: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

6

First steps with R

Example 2: Regression Tree

• 10 terminal nodes• 5 variables

• Lifts• Area• Neighborhood,• Number of floors• Number of apartments

• Error in training data 9,2% • Error in test data 9,8%

Predicion is theaverage log price

in the node

Page 7: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

7

First steps with R

Example 2: Regression Tree – Tree pruning (Cross-

Validation)

Optimalnumber of

nodes

Page 8: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

8

First steps with R

Example 2: Regression Tree - Cross-validation

Region 1(73.800 €)

Region 2(115.800 €)

Region 3(166.000 €)

Region 5(334.400 €)Region 4

(224.100 €)

Region 6(680.100 €)

• 6 terminal nodes• 3 variables

• Lifts,• Neighborhood• Area

• Error on training data 10,7% • Error test data 10,7%

Page 9: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

9

First steps with R

Example 2: Two-dimensional classification

Page 10: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

10

First steps with R

Example 2: Regression Tree - Bagging

• Error on training data 6,8% • Error test data 6,3%

Page 11: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

11

First steps with R

Example 2: Regression Tree - Random Forest

• p = 5• Error on training data 6,3% • Error test data 6,3%

Page 12: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

12

First steps with R

Example 2: Regression Tree - Random Forest

Page 13: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

13

First steps with R

Example 2: Regression Tree - Boosting

• depth = 3• Error on training data 6,6% • Error test data 6,8%

Page 14: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

14

First steps with R

Example 2: Regression Tree - Boosting

Page 15: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

15

First steps with R

Example 2: R code#-------------------------------------------------------------------------------

#

# Mass Appraisal of Residential Real Estate Using Regression Trees

#

#-------------------------------------------------------------------------------

# library

library(tree)

library(randomForest)

library(gbm)

#-------------------------------------------------------------------------------

#

#

#

#-------------------------------------------------------------------------------

# Read data

#-------------------------------------------------------------------------------

# Lectura de datos

apartments<-read.table('Alicante.csv',header=TRUE,sep=';',dec='.',

colClasses=c(rep('double',15)))

Page 16: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

16

First steps with R

Example 2: R code#-------------------------------------------------------------------------------

# ANALYSIS (DEP. VAR LOG(PRICE))

#-------------------------------------------------------------------------------

#tree.apartments <- tree(log(Price)~.,apartments)

#summary(tree.apartments)

#plot(tree.apartments)

#text(tree.apartments,pretty=0)

#tree.apartments

#-------------------------------------------------------------------------------

# VALIDATION TEST (package tree requiered)

#-------------------------------------------------------------------------------

set.seed(1)

train <- sample(1:nrow(apartments), nrow(apartments)/2)

apartments.test <- apartments[-train,]

tree.apartments.train <- tree(log(Price)~.,apartments,subset=train)

summary(tree.apartments.train)

layout(1)

plot(tree.apartments.train)

text(tree.apartments.train,pretty=0)

tree.apartments.train

yhat.all <- predict(tree.apartments.train,newdata=apartments.test)

mean((yhat.all - log(apartments.test$Price))^2)

Page 17: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

17

First steps with R

Example 2: R code#-------------------------------------------------------------------------------

# CROSS VALIDATION

#-------------------------------------------------------------------------------

set.seed(2)

cv.apartments <- cv.tree(tree.apartments.train)

cv.apartments

par(mfrow=c(1,2))

plot(cv.apartments$size,cv.apartments$dev,type="b",

xlab='Terminal Nodes',ylab='Deviation')

points(cv.apartments$size[4],cv.apartments$dev[4],pch=19, col='red',)

plot(cv.apartments$k,cv.apartments$dev,type="b",

xlab='Parameter alpha',ylab='Deviation')

layout(1)

prune.apartments <- prune.tree(tree.apartments.train,best=6)

plot(prune.apartments)

text(prune.apartments,pretty=0)

prune.apartments

yhat.prune <- predict(prune.apartments,newdata=apartments[train,])

mean((yhat.prune - log(apartments$Price[train]))^2)

yhat.prune <- predict(prune.apartments,newdata=apartments.test)

mean((yhat.prune - log(apartments.test$Price))^2)

Page 18: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

18

First steps with R

Example 2: R code#-------------------------------------------------------------------------------

# GRAPH

#-------------------------------------------------------------------------------

colour <- rep('Black',nrow(apartments))

colour[apartments$Lifts==0 & apartments$Neighborhood==0] <- 'Red'

colour[apartments$Lifts==0 & apartments$Neighborhood==1] <- 'Blue'

plot(apartments$Area,log(apartments$Price),pch=20,

xlab='Area (square meters)',ylab='Price (log euros)',col=colour)

abline(v=189,lty=2)

abline(v=149.5,,lty=2)

abline(v=108.9,,lty=2)

text(x=250,y=14.3,'Region 6',cex=.9)

text(x=170,y=14.3,'Region 5',cex=.9)

text(x=130,y=14.3,'Region 4',cex=.9)

text(x=70,y=14.3,'Region 3',cex=.9)

legend(x=240,y=11.5,

legend=c('Region 1','Region 2'),col=c('red','blue'),pch=20,cex=.9)

Page 19: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

19

First steps with R

Example 2: R code#-------------------------------------------------------------------------------

# BAGGING (package ramdomForest requiered)

#-------------------------------------------------------------------------------

set.seed(1)

bag.apartments <- randomForest(log(Price)~.,data=apartments,subset=train,

mtry=14,importance=TRUE,ntree=500)

bag.apartments

yhat.bag <- predict(bag.apartments,newdata=apartments.test)

plot(exp(yhat.bag), apartments.test$Price,

ylab='Estimated Prices (euros)',

xlab='Observed Prices (euros)')

abline(0,1,col='red',lwd=2)

mean((yhat.bag-log(apartments.test$Price))^2)

Page 20: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

20

First steps with R

Example 2: R code#-------------------------------------------------------------------------------

# BOOSTING (package gbm requiered)

#-------------------------------------------------------------------------------

set.seed(1)

boost.apartments <- gbm(log(Price)~.,data=apartments[train,],distribution="gaussian",

n.trees=5000,interaction.depth=3,shrinkage=0.001)

layout(1)

summary(boost.apartments)

par(mar=c(5,10,4,2))

barplot(summary(boost.apartments)$rel.inf, horiz=TRUE,

names.arg=summary(boost.apartments)$var,

xlab='Importance',las=1)

par(mar=c(5,4,4,2))

par(mfrow=c(1,2))

plot(boost.apartments,i="Area")

plot(boost.apartments,i="Lifts")

yhat.boost <- predict(boost.apartments,newdata=apartments[train,],n.trees=5000)

mean((yhat.boost-log(apartments$Price[train]))^2)

yhat.boost <- predict(boost.apartments,newdata=apartments.test,n.trees=5000)

mean((yhat.boost-log(apartments.test$Price))^2)

Page 21: Module 2. Statistical learning Example 2 - … fileModule 2. Statistical learning Example 2. ... • 1,071 observation in the test set • Ten-fold cross-validation. Eurostat 6 First

Eurostat

21

First steps with R

Example 2: R code

layout(1)

plot(exp(yhat.boost), apartments.test$Price,

ylab='Estimated Prices (euros)',

xlab='Observed Prices (euros)')

abline(0,1,col='red',lwd=2)


The POPPI 1 Example: Statistical Comments

EVALUATION OF MANUFACTURING TOLERANCE USING A STATISTICAL ... · PDF fileEVALUATION OF MANUFACTURING TOLERANCE USING A STATISTICAL METHOD AND EXPERIMENTATION . ... [13] for example

STATISTICAL DATA ANALYSIS IN EXCEL - SABLab.netedu.sablab.net/sdae2013/handouts/SDAE2013_L4-ANOVA.pdf · Statistical data analysis in Excel. 4. ANOVA 15 INTRODUCTION TO ANOVA Example

Statistical Methods for Data Analysis a RooStats example Luca Lista INFN Napoli

Statistical Learning Theory · “statistical learning theory” What is the measure of simplicity? For example, number of features = the number of non-zero elements in

STATISTICAL DATA ANALYSIS IN EXCELedu.sablab.net/sdae2010/handout/Nazarov_StatExcel_part1.pdf · Statistical data analysis in Excel 5 TABULAR AND GRAPHICAL PRESENTATION Example: Pancreatitis

Division Information Type Title Double-Blind (Sponsor ... · 8. SECONDARY STATISTICAL ANALYSES ... 10.11.2. Mock Example Shell Referencing ... Statistical Analysis Software (SAS)

30 · 1,071 975 29 659 271 206

Statistics meets machine learning: the example of …Statistics meets machine learning: the example of statistical boosting Data science, statistics, machine learning:statistics vs

An Example Using PET for Statistical Parametric Mappingric.uthscsa.edu/personalpages/lancaster/SPM_Class/PET_Stuttering... · An Example Using PET for Statistical Parametric Mapping

Large-scale deployment of statistical machine translation · Large-scale deployment of statistical machine translation Example Microsoft [email protected] ... User Interface

Security Control Methods for Statistical Database - …lxiong/cs573_s12/share/slides… ·  · 2009-02-06Security Control Methods for Statistical Database ... Example: U.S. Census

An Interactive Java Statistical Image Segmentation System ... · 4 An Interactive Java Statistical Image Segmentation System: GemIdent Figure 3: An example of training in a microscopic

Detection: chapter 3 Statistical detection theory I Example of detection

Confidence Intervals and Hypothesis Tests (Statistical ... · Confidence Intervals and Hypothesis Tests (Statistical Inference) Ian Jolliffe Introduction Illustrative Example Types

A Statistical Analysis Example of A Full Functional Utilization of An Engineering Calculator

Metadata management in National Statistical Institutes and researcher access: an example Zoltán Vereczkei Hungarian Central Statistical Office Methodology

Statistical Sampling - Weebly · 2018. 9. 5. · Statistical Sampling 5 A. Divide population into smaller populations Example 4: Stratified Sample Population of 11,200 equipment transactions

Comparing Example-Based & Statistical Machine Translation

Statistical Methods in Medical Researchtrials as an example of complex interventions, we review statistical methods which crucially evaluate both direct and indirect effects in the

[1] Confidence Intervals. [2] Statistical Estimation sample statistic = parameter estimate = s=s= Example:

Introduction to Statistical InferenceWhat is statistical inference? Making evidence based claims about a population by using data. Example: An administrator claims that undergraduate

A statistical approach to PVA validation: an example using

An example of statistical data ... - Crop and Soil Science

Statistical Analysis of DNA Microarray. An Example of HDLSS in Genetics

Introduction to Statistical Inference - Radford Universitytpierce/610 files/Data Analysis for Professional Psychologists...An example of a statistical inference using raw scores The

The Argument for Using Statistics Weighing the Evidence Statistical Inference: An Overview Applying Statistical Inference: An Example Going Beyond Testing

Nonequilibrium Thermodynamics and Statistical … · NONEQUILIBRIUM THERMODYNAMICS AND STATISTICAL PHYSICS 49 conditions containing as constitutive coefficients, for example, the

Part II: Statistical Inference - GitHub Pages...Statistical Inference 1 Francisella tularensis Example 2 Hypothesis testing 3 Multiple testing 4 Moderated statistics 5 Experimental

Example: Statistical Estimation

1 1 Establishing a register-based statistical system Example: Population and housing censuses in Norway Statistical Training Course Use of Administrative

Statistical Inference After Model Selectionberkr/selection copy.pdfshow though a criminal justice example using real data how proper statistical inference in principle may be obtained

Alternatives to rule-based MT: statistical and example-based MT

Module 5: Statistical Analysesunix.eng.ua.edu/~rpitt/Class... · 2007-10-11 · Module 5: Statistical Analyses ... EXAMPLE OF ANOVA USING EXCEL ... This module reviews a number of

Comparable statistics in the EU: ESS, an example of an effective regional statistical system Claudia Junker, Eurostat, head of unit "Statistical cooperation"