rosa – rossa validation results r. notarpietro, g. perona, m. cucca riccardo.notarpietro@polito.it

ROSA – ROSSA Validation resultsROSA – ROSSA Validation results

R. Notarpietro, G. Perona, M. CuccaR. Notarpietro, G. Perona, M. Cucca

riccardo.notarpietro@polito.itriccardo.notarpietro@polito.it

COSMIC Data

- COSMIC 1: 75 events (10 rising) well distributed in time (2 days for each 06-07 season) and space

- COSMIC 2: data related 31/12/2007 (1120 events)

CHAMP Data

- 120 setting events related to 22/01/2004 observations:

INPUT DATAINPUT DATA

INPUT DATAINPUT DATA

- COSMIC 2: data related 31/12/2007 (> 1200 events)

90° 65° North Polar regions65° 35° North Temperate regions

35° 35° Tropics

35° 65° South Temp. regions

65° 90° South Polar regions

24 56 8056 188 244

68 336 404

59 149 20817 47 64

R S TOT*

* These figures are related to data couples compared at different levels after outliers removal

CHAMP DataCHAMP Data

RO

SA

-RO

SS

A

CD

AA

C (

CO

SM

IC)

ISD

C (

CH

AM

P)

COSMIC DataCOSMIC Data

We can quantify differences due to different algorithm implementations (CHAMP Data, COSMIC Data or future ROSA-ROSSA products ARE NOT First Level Benchmark)

This analysis has been carried out even if it is not an absolute validation of ROSA-ROSSA software

1

Validation considering Third Level Benchmarks (output of other RO SWs)

COSMIC 1/2 Data COSMIC 1/2 Data 2006/20072006/2007

NNee(h)(h)

COSMIC 2 Data COSMIC 2 Data 31/12/2007 31/12/2007

T(h)T(h)Specific humidity (h)Specific humidity (h)ppwetwet(h) (h)

stratosph. optimized (a)(a) iono-free

COSMIC 2 Data COSMIC 2 Data 31/12/200731/12/2007

n(h)n(h)

CHAMP Data CHAMP Data 22/01/0422/01/04

OrbitsOrbits

Excess Excess phasesphases

1

Validation considering Third Level Benchmarks (output of other RO SWs)

2

Validation considering Second Level Benchmarks (ECMWF Re-analysis)

RO

SA

-RO

SS

A

CD

AA

C (

CO

SM

IC)

ISD

C (

CH

AM

P)

NNRSARSA(h)(h) NNCOSCOS(h)(h)

TTRSARSA(h)(h) TTCOSCOS(h)(h)

WVWVRSARSA(h)(h) WVWVCOSCOS(h)(h)

Dati COSMIC 2 Dati COSMIC 2 31/12/200731/12/2007

2

Validation considering Second Level Benchmarks (ECMWF Re-analysis)

RO

SA

-RO

SS

A

CD

AA

C (

CO

SM

IC)

ISD

C (

CH

AM

P)

NNRSARSA(h)(h) NNCOSCOS(h)(h)

ECMWF Re-analysis spatially

and temporally

co-located

NNECMECM(h)(h)

TTRSARSA(h)(h) TTCOSCOS(h)(h)TTECMECM(h)(h)

WVWVRSARSA(h)(h) WVWVCOSCOS(h)(h)WVWVECMECM(h)(h)

following Tangent Points

Dati COSMIC 2 Dati COSMIC 2 31/12/200731/12/2007

2

Validation considering Second Level Benchmarks (ECMWF Re-analysis)

RO

SA

-RO

SS

A

CD

AA

C (

CO

SM

IC)

ISD

C (

CH

AM

P)

NNRSARSA(h)(h) NNCOSCOS(h)(h)

ECMWF Re-analysis spatially

and temporally

co-located

NNECMECM(h)(h)

TTRSARSA(h)(h) TTCOSCOS(h)(h)TTECMECM(h)(h)

WVWVRSARSA(h)(h) WVWVCOSCOS(h)(h)WVWVECMECM(h)(h)

following Tangent Points

Dati COSMIC 2 Dati COSMIC 2 31/12/200731/12/2007

2

Validation considering Second Level Benchmarks (ECMWF Re-analysis)

RO

SA

-RO

SS

A

CD

AA

C (

CO

SM

IC)

ISD

C (

CH

AM

P)

NNRSARSA(h)(h) NNCOSCOS(h)(h)

ECMWF Re-analysis spatially

and temporally

co-located

NNECMECM(h)(h)

TTRSARSA(h)(h) TTCOSCOS(h)(h)TTECMECM(h)(h)

WVWVRSARSA(h)(h) WVWVCOSCOS(h)(h)WVWVECMECM(h)(h)

following Tangent Points

Outliers identification and removal and Mean Fractional Error evaluation

Dati COSMIC 2 Dati COSMIC 2 31/12/200731/12/2007

Statistic definition

100hN

hNhNh

COS

COSRSAN%,

Statistic definition

100hN

hNhNh

COS

COSRSAN%,

Statistic definition

Outlier Identification through T-Student test ( 3 threshold) and rejection

Mea

n +

Std

Mea

n -

Std

mea

n

Statistic definition

OPEN PROBLEMS WITH THE FIRST ROSA-ROSSA RELEASE

Cosmic Raw Excess-phase profiles filtering

OPEN PROBLEMS WITH THE FIRST ROSA-ROSSA RELEASE

Cosmic Raw Excess-phase profiles filtering

OPEN PROBLEMS WITH THE FIRST ROSA-ROSSA RELEASE

Cosmic Raw Excess-phases filtering

Statospheric Bending optimization (we are actually using CIRA-Q climatology; ROSA-ROSSA VE will adopt data coming from Numerical Weather Prediction Models)

Wave Optics techniques for Bending angle extraction in low troposphere (foreseen for ROSA-ROSSA VE). Actually DG_ATMO is validated giving COSMIC data in input.

Electron Density profile extraction through ROSA observations is critical given ROSA configuration and observation scheduling

RSARSA(a)(a)

CD

AA

C

(CO

SM

IC)

COSCOS(a)(a)

1

Validation considering Third Level Benchmarks (output of other RO SWs)

BENDING LEVEL

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Rising Setting North Pole

North Temperate

Tropics

South Temperate

South Pole

Number of events

(a)(a)

a%, hmean

North Pole

North Temperate

Tropics

South Temperate

South Pole

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

(a)(a)

a%, hmean

North Pole

North Temperate

Tropics

South Temperate

South Pole

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

(a)(a)

a%, hmean

North Pole

North Temperate

Tropics

South Temperate

South Pole

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

(a)(a) North Pole

North Temperate

Tropics

South Temperate

South Pole

stdmeana%, h

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

NNRSARSA(h)(h)

CD

AA

C

(CO

SM

IC)

NNCOSCOS(h)(h)

1REFRACTIVITY LEVEL

Validation considering Third Level Benchmarks (output of other RO SWs)

NN (h)(h)

hmean N%,

North Pole

North Temperate

Tropics

South Temperate

South Pole

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

NN (h)(h) North Pole

North Temperate

Tropics

South Temperate

South Pole

stdmeanN%, h

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Validation considering Third Level Benchmarks (output of other RO SWs)

TTRSARSA(h)(h)

CD

AA

C

(CO

SM

IC)

TTCOSCOS(h)(h)

1TEMPERATURE and humidity LEVEL

DG_ATMO

qqRSARSA(h)(h) qqCOSCOS(h)(h)

eeRSARSA(h)(h) eeCOSCOS(h)(h)

• DG_ATMO is a 1-D VAR scheme developed for retrieving temperature, humidity and pressure (only as dependent variable) for the ROSA observations.

• The Background profile is extracted by NCEP long-term mean (365 days) reanalysis.

• Simplified version of error covariance matrices

DG_ATMO

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

T(h)T(h)

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

e(h)e(h)

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

q(h)q(h)

• Our scheme differs from the COSMIC one.– Formulation of hypotheses to develop the

1D-VAR scheme– Formulation of error covariance matrices– Background fields

• Results are not any different!

DG_ATMO

Validation considering Third Level Benchmarks (output of other RO SWs)

NeNeRSARSA(h)(h)

CD

AA

C

(CO

SM

IC)

NeNeCOSCOS(h)(h)

1ELECTRON DENSITY LEVEL

DG_DELN

• The DG_DELN Data Generator evaluates the electron density profile in the ionosphere, using the Onion Peeling algorithm.

• Since the ray bending in the ionosphere is small enough, the straight-line propagation from GPS to LEO satellites has been assumed for the GPS signals.

• As required by the inversion technique adopted, the spherical symmetry for the electron density of the ionosphere has been assumed.

• Excess phase measurements at L1 and L2 GPS frequencies during one occultation event are used to compute the TEC in the shell determined by the LEO orbit (calibrated TEC).

DG_DELN

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Ne(h)Ne(h)

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Ne(h)Ne(h)

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Ne(h)Ne(h)

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Ne(h)Ne(h)

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Ne(h)Ne(h)

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Ne(h)Ne(h)

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Ne(h)Ne(h)

DG_DELNCOSMIC

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Ne(h)Ne(h)

average +/- st.dev.average

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

Ne(h)Ne(h)

RO

SA

-RO

SS

A

CD

AA

C (

CO

SM

IC)

NNRSARSA(h)(h) NNCOSCOS(h)(h)

ECMWF Re-analysis

spatially and

temporally co-located

NNECMECM(h)(h)

2

Validation considering Second Level Benchmarks (ECMWF Re-analysis)

REFRACTIVITY LEVEL

North Pole

North Temperate

Tropics

South Temperate

South Pole

RISING

Number of events

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

NN (h)(h)North Pole

North Temperate

Tropics

South Temperate

South Pole

RISING50

03

71012

16

40

30

He

igh

t [k

m]

25

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

NN (h)(h)North Pole

North Temperate

Tropics

South Temperate

South Pole

RISING50

03

71012

16

40

30

He

igh

t [k

m]

25

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

North Pole

North Temperate

Tropics

South Temperate

South Pole

SETTING

Number of events

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

NN (h)(h)North Pole

North Temperate

Tropics

South Temperate

South Pole

50

03

71012

16

40

30

He

igh

t [k

m]

25

SETTING

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

NN (h)(h)North Pole

North Temperate

Tropics

South Temperate

South Pole

50

03

71012

16

40

30

He

igh

t [k

m]

25

SETTING

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

2TEMPERATURE and Spec. Humidity LEVEL

RO

SA

-RO

SS

A

TTRSARSA(h)(h)

ECMWF Re-analysis

spatially and

temporally co-located

TTECMECM(h)(h)DG_ATMO

qqRSARSA(h)(h) qqECMECM(h)(h)

Validation considering Second Level Benchmarks (ECMWF Re-analysis)

• Use the closest horizontal grid point (in the space-time) domain per each vertical level, this means reconstructing an ECMWF “slanted” vertical profile

• Interpolation of ECMWF and ROSA on a common vertical grid (100 m vertical resolution like the COSMIC one).

• 25 Pressure levels ECMWF data, 0.25 degree horizontal resolution

DG_ATMO

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

T(h)T(h)

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

q(h)q(h)

2ELECTRON DENSITY LEVEL

RO

SA

-RO

SS

A

NeNeRSARSA(h)(h)

IONOGRAMS

NeNetruetrue(h)(h)DG_DELN

Validation considering Second Level Benchmarks (ECMWF Re-analysis)

• DG_DELN derived profiles are compared with ionosonde derived (bottomside) profiles.

• The distance between the Ionosonde location and the DG_DELN derived peak location has been chosen to be less than [2.5° x 2.5°] in [lat. x lon.] and 15 min in time.

• The ionograms have been selected and scaled by K. Alazo of the Instituto Geofisica y Astronomia, La Habana, Cuba in the framework of the STEP program of ICTP.

DG_DELN

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

DG_DELNIONOSONDE

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

Ne(h)Ne(h)

Statistic definition

from Kursinsky et al., JGR1997

Mea

n +

Std

Mea

n -

Std

mea

n

Statistic definition

Outlier Identification through T-Student test ( 3 threshold) and rejection

NN (h)(h)

hmean N%,

North Pole

North Temperate

Tropics

South Temperate

South Pole

SW ROSA-ROSSA 3.0

VALIDATION VS COSMIC DATAVALIDATION VS COSMIC DATA

NN (h)(h)North Pole

North Temperate

Tropics

South Temperate

South Pole

RISING50

03

71012

16

40

30

He

igh

t [k

m]

25

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF

NN (h)(h)North Pole

North Temperate

Tropics

South Temperate

South Pole

50

03

71012

16

40

30

He

igh

t [k

m]

25

SETTING

SW ROSA-ROSSA 3.0

VALIDATION VS ECMWFVALIDATION VS ECMWF