correction of daily values for inhomogeneities

Post on 03-Jan-2016

38 Views

Category:

Documents

3 Downloads

Preview:

Click to see full reader

DESCRIPTION

Correction of daily values for inhomogeneities. P. Štěpánek. Czech Hydrometeorological Institute, Regional Office Brno, Czech Republic. E-mail: petr.stepanek@chmi.cz. COST-ESO601 meeting, Tarragona, 9-11 March 2009. Using daily data for inhomogeniety detection , is it meaningful ?. - PowerPoint PPT Presentation

TRANSCRIPT

Correction of daily valuesCorrection of daily values for for inhomogeneitiesinhomogeneities

P. Štěpánek

Czech Hydrometeorological Institute, Regional Office Brno, Czech Republic

E-mail: petr.stepanek@chmi.cz

COST-ESO601 meeting, Tarragona, 9-11 March 2009

Using daily data for inhomogeniety Using daily data for inhomogeniety detectiondetection, , is it meaningfulis it meaningful??

Homogenization of Homogenization of daily values daily values –– precipitationprecipitation series series

working with individual monthly values (to get rid of annual cycle)

It is still needed to adapt data to approximate to normal distribution

One of the possibilities: consider values above 0.1 mm only

Additional transformation of series of ratios

(e.g. with square root)

Original values - far from normal distribution

(ratios tested/reference series)(ratios tested/reference series) FrequenciesFrequencies

Homogenization of precipitation Homogenization of precipitation – daily values– daily values

Homogenization of precipitation Homogenization of precipitation – daily values– daily values

Limit value 0.1 mm

(ratios tested/reference series)(ratios tested/reference series) FrequenciesFrequencies

Limit value 0.1 mm, square root transformation (of ratios)

(ratios tested/reference series)(ratios tested/reference series) FrequenciesFrequencies

Homogenization of precipitation Homogenization of precipitation – daily values– daily values

Problem of independeProblem of independennce, ce, PrecipitationPrecipitation above 1 mm above 1 mm

August, Autocorrelations

Problem of independece,Problem of independece,

TTemperatureemperature

August, Autocorrelations

Problem of independece,Problem of independece,

TTemperature differences emperature differences (reference – candidate)(reference – candidate)

August, Autocorrelations

HomogenizationHomogenization Detection (preferably on monthly, seasonal and annual values) Correction – for daily values

WP1 WP1 SURVEYSURVEY (Enric Aguilar) (Enric Aguilar) Daily dataDaily data - - CorrectionCorrection (WP4) (WP4)

Very few approaches actually calculate special corrections for daily data.

Most approaches either

– Do nothing (discard data)

– Apply monthly factors

– Interpolate monthly factors

The survey points out several other alternatives that WG5 needs to investigate

0

2

4

6

8

10

12

Ap

ply

mo

nth

lyfa

cto

rs

Ch

an

ge

sN

LR

C N

Dis

card

da

ta

Em

pir

ica

lva

lue

s

Inte

rpo

late

mo

nth

ly

Tra

nsf

er

fun

ctio

ns

CD

FO

verl

ap

pin

gre

cord

s &

LM

Re

fere

nce

s+

mo

de

llin

go

f ho

m.

Lin

ea

ra

dju

stm

en

ts

Trust metadata only

Use a technique to detect breaks

Detect on lower resolution

Daily data correction mDaily data correction methodsethods

„Delta“ methods Variable correction methods – one element Variable correction methods – several

elements

Daily data correction mDaily data correction methodsethods

Interpolation of monthly factors– MASH– Vincent et al (2002) - cublic spline interpolation

Nearest neighbour resampling models, by Brandsma and Können (2006)

Higher Order Moments (HOM), by Della Marta and Wanner (2006) Two phase non-linear regression (O. Mestre) Modified percentiles approach, by Stepanek Using weather types classifications (HOWCLASS), by I. Garcia-

Borés, E. Aguilar, ...

AdjustAdjustinging daily values daily values for inhomogeneitiesfor inhomogeneities, , from from monthlymonthly versus versus dailydaily adjustmentsadjustments(„delta“ method)(„delta“ method)

AdjustingAdjusting from from monthlymonthly data data

monthly adjustments smoothed with Gaussian low pass filter (weights approximately 1:2:1)

smoothed monthly adjustments are then evenly distributed among individual days

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

1.1.

1.2.

1.3.

1.4.

1.5.

1.6.

1.7.

1.8.

1.9.

1.10

.

1.11

.

1.12

.

°C

UnSmoothed

B2BPIS01_T_21:00

-1.0

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

0.8

1.0

1.1.

1.2.

1.3.

1.4.

1.5.

1.6.

1.7.

1.8.

1.9.

1.10

.

1.11

.

1.12

.

°C

ADJ_ORIG ADJ_C_INC

B2BPIS01_T_21:00

AdjustingAdjusting straight from straight from dailydaily data data

Adjustment estimated for each individual day (series of 1st Jan, 2nd Jan etc.)

Daily adjustments smoothed with Gaussian low pass filter for 90 days (annual cycle 3 times to solve margin values)

-5.0

-4.5

-4.0

-3.5

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

1.1

.

1.2

.

1.3

.

1.4

.

1.5

.

1.6

.

1.7

.

1.8

.

1.9

.

1.1

0.

1.1

1.

1.1

2.

hP

a

ADJ_ORIG ADJ_SMOOTHB2BPIS01_P_14:00

AdjustmentsAdjustments ((Delta methodDelta method))

-1.5

-1.0

-0.5

0.0

0.5

1.0

01-J

an

01-F

eb

01-M

ar

01-A

pr

01-M

ay

01-J

un

01-J

ul

01-A

ug

01-S

ep

01-O

ct

01-N

ov

01-D

ec

°C

UnSmoothedADJ_C_INCADJ_ORIG

1 3 2

a)

-1.5

-1.0

-0.5

0.0

0.5

1.0

01-J

an

01-F

eb

01-M

ar

01-A

pr

01-M

ay

01-J

un

01-J

ul

01-A

ug

01-S

ep

01-O

ct

01-N

ov

01-D

ec

°C

ADJ_ORIGADJ_SMOOTH30ADJ_SMOOTH60ADJ_SMOOTH90

4 5 6 7

b)

The same final adjustments may be obtained from either monthly averages or through direct use of daily data

(for the daily-values-based approach, it seems reasonable to smooth with a low-pass filter for 60 days. The same results may be derived using a low-pass filter for two months (weights approximately 1:2:1) and

subsequently distributing the smoothed monthly adjustments into daily values)

(1 – raw adjustments, 2 – smoothed adjustments, 3 – smoothed adjustments distributed into individual days), b) daily-based approach (4 – individual calendar day adjustments, 5 – daily adjustments smoothed by low-pass filter for 30 days, 6 – for 60 days, 7 – for 90 days)

Spline through monthly temperature Spline through monthly temperature adjustmentsadjustments („delta“ method)(„delta“ method)

Easy to implement No assumptions about changes in variance Integrated daily adjustments = monthly adjustments But, is it natural?

Variable correction Variable correction

f(C(d)|R), function build with the reference dataset R, d – daily data

cdf, and thus the pdf of the adjusted candidate series C*(d) is exactly the same as the cdf or pdf of the original candidate series C(d)

Trewin & Trevitt (1996) method: Use simultaneous observations of old and new conditions

Variable correctionVariable correction

Variable correctionVariable correction

1996

The HOM method concept: Fitting a modelThe HOM method concept: Fitting a model Locally weighted regression (LOESS)

(Cleveland & Devlin,1998)

HSP2 HSP1

The HOM method concept: Calculating the The HOM method concept: Calculating the binned difference seriesbinned difference series

Decile 1, k=1

Decile 10, k=10

The HOM method concept: The The HOM method concept: The binned differencesbinned differences

DELLA-MARTA AND WANNER,

JOURNAL OF CLIMATE 19 (2006)

4179-4197

SPLIDHOM (SPLIDHOM (SPLIne Daily HOMogenization), Olivier Mestre), Olivier Mestre direct non-linear

spline regression approach (x rather

than a correction based on quantiles),

cubic smoothing splines for estimating regression functions

Variable correctionVariable correction, , q-q functionq-q function

Michel Déqué, Global and Planetary Change 57 (2007) 16–26

Our modified percentiles based Our modified percentiles based approachapproach

0.000

5.000

10.000

15.000

20.000

25.000

30.000

0.00

0

0.04

0

0.10

0

0.16

0

0.22

0

0.28

0

0.34

0

0.40

0

0.46

0

0.52

0

0.58

0

0.64

0

0.70

0

0.76

0

0.82

0

0.88

0

0.94

0

0.99

5

Q_CAND_BE

Q_REF_BE

0.000

5.000

10.000

15.000

20.000

25.000

30.000

0.00

0

0.04

0

0.10

0

0.16

0

0.22

0

0.28

0

0.34

0

0.40

0

0.46

0

0.52

0

0.58

0

0.64

0

0.70

0

0.76

0

0.82

0

0.88

0

0.94

0

0.99

5

Q_CAND_AF

Q_REF_AF

-1.000

-0.500

0.000

0.500

1.000

1.500

0.00

0

0.04

0

0.10

0

0.16

0

0.22

0

0.28

0

0.34

0

0.40

0

0.46

0

0.52

0

0.58

0

0.64

0

0.70

0

0.76

0

0.82

0

0.88

0

0.94

0

0.99

5

Q_DIFF_BE

Q_DIFF_AF

-1.400

-1.200

-1.000

-0.800

-0.600

-0.400

-0.200

0.000

0.200

0.00

0

0.04

0

0.10

0

0.16

0

0.22

0

0.28

0

0.34

0

0.40

0

0.46

0

0.52

0

0.58

0

0.64

0

0.70

0

0.76

0

0.82

0

0.88

0

0.94

0

0.99

5

Q_DIFF

Q_DIFF_SM

-1.400

-1.200

-1.000

-0.800

-0.600

-0.400

-0.200

0.000

0.200

1 8 15 22 29 36 43 50 57 64 71 78 85 92 99

Our percentiles based approachOur percentiles based approach

-1.400

-1.200

-1.000

-0.800

-0.600

-0.400

-0.200

0.000

0.200

0.00

0

0.04

0

0.10

0

0.16

0

0.22

0

0.28

0

0.34

0

0.40

0

0.46

0

0.52

0

0.58

0

0.64

0

0.70

0

0.76

0

0.82

0

0.88

0

0.94

0

0.99

5

Q_DIFF

Q_DIFF_SM

PERC sm25

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

-1.400

-1.200

-1.000

-0.800

-0.600

-0.400

-0.200

0.000

0.200

1 8 15 22 29 36 43 50 57 64 71 78 85 92 99

Q_DIFF_SM

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

Variable correction methods – Variable correction methods – complex approach complex approach (several elements)(several elements)

not yet available …

Comparison of the methods, Comparison of the methods, ProClimDB softwareProClimDB software

Correction methods comparisonCorrection methods comparison

PERC

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

EMPIR

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

HOM

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

SPLIDHOM

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

PERC sm50

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

PECR sm75

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

EMPI sm75

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

Correction methods comparison, Correction methods comparison, different parameters settingsdifferent parameters settings

PERC

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

EMPIR

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

HOM

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

SPLIDHOM

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

PERC sm50

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

PECR sm75

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

EMPI sm75

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

PERC

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

EMPIR

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

SPLIDHOM

Correction methods comparison, Correction methods comparison, different parameters settingsdifferent parameters settings

PERC

-0.900

-0.800

-0.700

-0.600

-0.500

-0.400

-0.300

-0.200

-0.100

0.000

0.000 5.000 10.000 15.000 20.000 25.000 30.000

EMPIR

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

HOM

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

SPLIDHOM

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0 5 10 15 20 25 30

SPLIDHOM

-1.0

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

0.0 5.0 10.0 15.0 20.0 25.0 30.0

HOM

-1.0

-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

0.0 5.0 10.0 15.0 20.0 25.0 30.0

Correction of daily valuesCorrection of daily values

We have some methods … - but we have to validate them -> benchmark

dataset on daily data Do we know how inhomogeneites in daily data

behave?we should analyse real datawho and when?, what method for data

comparison?

top related