the open-source air pollution project - openair€¦ · the open-source air pollution project...

Introduction Examples of openair functions Outlook and concluding remarks

The open-source air pollution projectopenair

David Carslaw

The UK Air Quality Forecasting Seminar16th July 2009

David Carslaw — The open-source air pollution project 1/25


Outline

1 Introduction

2 Examples of openair functions

3 Outlook and concluding remarks



Outline

1 Introduction





Opportunities and barriersAnalysis of measurement and model output data

Opportunities

The analysis of air quality data can provide importantinsights into air pollution

There is a huge amount of data available

Insightful analysis provides the evidence to support airquality management decisions

Barriers

No consistent set of tools available to carry out analysis

Tools can be spread across many different softwareapplications

Many useful approaches are simply unavailable

Lack of time, money or ideas about what can be done



The openair projectSummary of project

Key points

3-year NERC project to October 2011

Develop and make available open-source data analysistools to AQ community

Use R statistical software as the platform

Highly capable software for “programming with data”Develop a “package” of tools and progressively includeadvanced methods not widely availableTransparency — all code open to scrutiny



openair websiteCentral resource for the project

Available atwww.openair-project.org

openair package –development version

All documentation, data setsetc.

Mailing list and newsletter

Sister NERC project AirTrackat the University of Lancasterwith complimentary aims

Joint openair/AirTrackworkshop, London, 1stOctober 2009


http://www.openair-project.org


Data analysisHow best to analyse data?

Data analysis is most useful when built around specificquestions, however. . .

Exploratory data analysis can be very insightful and isunder-used — but time consuming

John Tukey sums it up:

“The combination of some data and an achingdesire for an answer does not ensure that areasonable answer can be extracted from a givenbody of data.”



Outline

1 Introduction





Example analysis at a background siteThurrock — east of the M25

Import a few years of data fora range of pollutants

Examples of how some of theopenair functions can beapplied

Exampletk1 = import(“d:/mydata/thurrock.csv”)



Quickly summarise dataThe summarise function

Always a good idea to look atdata first before doinganything more serious

The summarise functionprovides a way to do thisrapidly

Examplesummarise(tk1)

date

1998 2000 2002 2004 2006 2008

missing = 9045 (9.4%)

min = 0

max = 173

mean = 2.3

median = 1.2

95th percentile = 7.8

36.3 % 97.8 % 96.9 % 96.1 % 97.3 % 91.1 % 97.7 % 93.9 % 97.3 % 97.5 % 94.9 %

SO

2

missing = 7837 (8.1%)

min = 0

max = 830

mean = 34.5

median = 21.8

95th percentile = 105

87.1 % 97.5 % 92.9 % 95.9 % 94.2 % 93.5 % 89.8 % 84.6 % 91.8 % 87.1 % 96.2 %

NO

X

missing = 2520 (2.6%)

min = 0

max = 360

mean = 199

median = 220

95th percentile = 340

98.6 % 98.1 % 97.2 % 96.7 % 97.7 % 98.7 % 99 % 97.3 % 94.1 % 97.5 % 96.4 %

win

d di

r.

missing = 1963 (2%)

min = −5.3

max = 37.3

mean = 11.7

median = 11.5

95th percentile = 21.6

98.6 % 98.1 % 97.2 % 96.7 % 97.7 % 98.7 % 99 % 98.5 % 96.7 % 99 % 97.5 %

tem

pera

ture

value

Per

cent

of T

otal

0

5

10

15

20

25

0 5 10 15

0

5

10

15

0 50 100 150 200

0

2

4

6

0 100 200 300

0

2

4

6

0 10 20



What do the met conditions look like?The wind.rose function

Plot a traditional wind roseusing the wind.rose function

lots of options to control howthe data are plotted

Examplewind.rose(tk1)

5

10

15

20

25

30

calm = 0.9%

01 January 1998 to 31 December 2008

0−2 2−4 4−6 6+ (m s−−1)



Wind roses by yearPlot by year, month, hour of the day. . .

510

1520

2530

3540

45

calm = 1.0%

1998

510

1520

2530

3540

45

calm = 1.9%

1999

510

1520

2530

3540

45

calm = 2.0%

2000

510

1520

2530

3540

45

calm = 1.9%

2001

510

1520

2530

3540

45

calm = 1.0%

2002

510

1520

2530

3540

45

calm = 0.2%

2003

510

1520

2530

3540

4550

calm = 0.2%

2004

510

1520

2530

3540

4550

calm = 0.2%

2005

510

1520

2530

3540

4550

calm = 0.8%

2006

510

1520

2530

3540

4550

calm = 0.3%

2007

510

1520

2530

3540

4550

calm = 0.3%

2008

0−2 2−4 4−6 6+ (m s−−1)

Examplewind.rose(tk1, type = “year”)



How do concentrations vary in time?The time.variation function

The temporal variation in concentrations can provideimportant clues as to the source

Sources can vary differently by hour of the day, day of theweek and season

Enhanced with further information

Traffic dataMeteorological data e.g. boundary layer height,atmospheric stability

Excellent for model evaluation

Exampletime.variation(tk1, pollutant = “nox”)



How do concentrations vary in time?The time.variation function

hour

NO

X

20

30

40

50

60

70

0 6 12 18 23

Monday

0 6 12 18 23

Tuesday

0 6 12 18 23

Wednesday

0 6 12 18 23

Thursday

0 6 12 18 23

Friday

0 6 12 18 23

Saturday

0 6 12 18 23

Sunday

NOX

hour

NO

X

25

30

35

40

45

50

0 6 12 18 23

month

NO

X

30

40

50

J F M A M J J A S O N D

●●

●

●

●

● ●

●

●

●

●

●

weekdayN

OX

25

30

35

40

Mon Tue Wed Thu Fri Sat Sun

●

●

● ●

●

●

●



How do concentrations vary in time?Two or more pollutants at once (SO2 and NOX)

hour

norm

alis

ed le

vel

0.5

1.0

1.5

2.0

0 6 12 18 23

Monday

0 6 12 18 23

Tuesday

0 6 12 18 23

Wednesday

0 6 12 18 23

Thursday

0 6 12 18 23

Friday

0 6 12 18 23

Saturday

0 6 12 18 23

Sunday

NOX SO2

hour

norm

alis

ed le

vel

0.8

1.0

1.2

1.4

0 6 12 18 23

month

norm

alis

ed le

vel

0.6

0.8

1.0

1.2

1.4


●●

●

●

●

● ●

●

●

●

●

●

●

●

● ●

●

● ●

●

●

●

●●

weekday

norm

alis

ed le

vel

0.7

0.8

0.9

1.0

1.1


●

●

● ●

●

●

●

●

● ● ●●

●

●

Normalising the concentrations helps greatly when comparingdifferent pollutants



How do concentrations vary in time?Two or more pollutants at once (SO2, NOX and O3)

hour

norm

alis

ed le

vel

0.5

1.0

1.5

2.0

0 6 12 18 23

Monday

0 6 12 18 23

Tuesday

0 6 12 18 23

Wednesday

0 6 12 18 23

Thursday

0 6 12 18 23

Friday

0 6 12 18 23

Saturday

0 6 12 18 23

Sunday

NOX SO2 O3

hour

norm

alis

ed le

vel

0.8

1.0

1.2

1.4

0 6 12 18 23

month

norm

alis

ed le

vel

0.6

0.8

1.0

1.2

1.4


●●

●

●

●● ●

●

●

●

●●

●●

● ●

●

● ●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

● ●

weekday

norm

alis

ed le

vel

0.7

0.8

0.9

1.0

1.1

1.2


●

●● ●

●

●

●

●

● ● ●●

●

●

●

● ●● ●

●

●

Very different behaviours and underlying reasons for differences



Polar plots and source identificationConcentrations by wind speed and direction

Variation with wind speed and direction can help identifysources and source characteristics1

Tall stack emissions vs.ground-level sources

Wind-blown sources e.g. particlere-suspension

Hot buoyant plumes e.g. aircraftjets

Local street canyon mixing

Examplepolar.plot(tk1, pollutant = “nox”)

W

S

N

E

10

20

30

40

50

60

70

80

1Carslaw et al. (2006). Detecting and quantifying aircraft and other on-airport contributions to ambientnitrogen oxides in the vicinity of a large international airport. Atmos. Env., 40 (28), 5424-5434.



Polar plots and source identificationConcentrations by wind speed and direction

The plot for SO2 is markedlydifferent to NOX

Evidence of at least three sources

Can be shown to be a refinery,power station and industrialsources

Examplepolar.plot(tk1, pollutant = “so2”)

W

S

N

E

0

1

2

3

4

5

6



Temporal polar plotsConcentrations by wind direction and time

Plot as an annulus

Consider how concentrations varyby hour of the day, day of the week,season or trend by wind direction

For NOX highest concentrations atnight from north-west

Examplepolar.annulus(tk1, pollutant = “nox”)



Temporal polar plotsConcentrations by wind direction and time

The SO2 plot is markedly differentto NOX

Concentrations highest duringdaytime and from south-east

Examplepolar.annulus(tk1, pollutant = “so2”)

Can choose type = “weekday”,“season” and “trend”



TrendsTrend analysis in openair

Trends are an importantcomponent of air quality analysis

Mann-Kendall analysis often usedfor environmental time series

Consider monthly trends withoption to de-seasonalise the data

Use bootstrap simulationtechniques to estimate uncertaintiesand block bootstrap to deal withautocorrelation

ExampleMannKendall(tk1, pollutant = “so2”)

year

SO

2

1

2

3

4

5

6

1998 2000 2002 2004 2006 2008

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●●

●●●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●●●

●

●

●

●●●●●

●●

●

●

●

●

●

●

●●●●

●

●

●●

●

●

●●●

●●

●

●

●

●

●

●

●●●

●

●

●●

−0.24 [−0.27, −0.2] units/year ***



TrendsCan consider trends in many different ways

year

SO

2

0

5

10

15

20

●●●

●

●

●

●●●●●●●●●

●

●●●

●●●●●●●●

●

●

●

●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●

●

●●●●●●●●●●●●●●●●●

●

●●●●●

●

●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●●●●

●●●●●●●●●

−0.23 [−0.28, −0.18] units/year ***

NW

1998 2000 2002 2004 2006 2008

●●●

●

●

●●

●●●●

●●●●

●

●

●

●●●●●●●●●

●●●

●

●●●●●

●

●●●●●

●

●

●●●●●●●●●

●

●●●

●●●●●●●●●●●●

●●●●●●●

●●●●●●●●●●●●

●●●●●●●●

●

●●●●

●●●●●●●●●●●●

●

●●●●●●●●●●●●

−0.23 [−0.3, −0.16] units/year ***

N

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●●

●●●

●

●●●

●●●

●●●●●

●

●●●●●

●

●●●●●●●●●●●●●●●

●

●

●●

●

●●●●●●●●●●●●

●●●

●●●

●

●●●●●●●●●●●

●●●●●●●

●

●●●●●●●

●●●●●●●●

●●●●●●●●●●

−0.21 [−0.29, −0.14] units/year ***

NE

●

●

●

●●●●●●●

●

●●●●●●●

●

●●●●●●

●

●●●●

●●●●●●●●

●●●●●

●●●●●●●●●

●●●●

●●

●●●●●●●

●●●●●●●●●●●●●●●●●

●

●●●●●

●●●●●●●●●●●●●●●●●

●

●

●

●●●●●●●●●●●●●●●●●●

−0.26 [−0.3, −0.22] units/year ***

W

0

5

10

15

20

●

●

●

●

●●

●

●

●●●●

●

●●●

●●●●

●

●●●

●

●●●●●

●●

●

●

●●●●●●●●●●●●

●

●

●

●

●●

●

●●●●●

●

●●●

●

●●●●

●

●●●●

●●

●

●●

●

●●●●●●●

●

●

●

●

●●

●●●●●●

●

●●

●

●●●●

●●●●●●●●

●

●●●●●●●

●

●●●●

−0.26 [−0.34, −0.18] units/year ***

E

0

5

10

15

20

1998 2000 2002 2004 2006 2008

●●●●●

●●●●●●●●●●●

●

●●●●●

●

●●●●●●●

●●●●●●●●●●

●●

●●●●●●●●●

●●●●●●●●●●●

●●●●●●●●

●●●●●●●

●●●●●●●●●●●●●●

●

●●●●●●●●●●

●●●●●●●●●

●●●●●●●●●●●●●●

●

−0.15 [−0.18, −0.12] units/year ***

SW

●●●●

●

●

●●●●●●●

●

●●●●●●

●●

●●●●●●●●●●

●●●●●●

●

●●●●●●●●●●

●

●●●●

●

●●●●

●●●●●●●●●●●●

●●●●●●●●●●●●

●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

−0.27 [−0.33, −0.23] units/year ***

S

1998 2000 2002 2004 2006 2008

●●

●

●●●●

●

●●●

●

●●

●

●

●●

●

●

●

●●●●●●●●●●●●●●●●●

●

●●●●●●●●●

●

●●●

●

●●●●

●●●

●●

●●

●●●●●●

●

●

●

●

●

●

●●●●●●

●●

●

●●

●

●

●

●

●

●

●●●●●●●

●●●●●●●

●●●●

●

●●●●

●●

●

●●●●●

−0.4 [−0.49, −0.31] units/year ***

SE

Trends can be ‘conditioned’ by many different variables—hereby wind direction



Outline

1 Introduction





Developments

1 Reviewing scientific literature and will adopt promisingapproaches

Examples include, improved temporal characterisatione.g. Fourier analysis, change-point detectionBetter quantitative analysis

2 Better support for model evaluation

Automate the process of evaluating modelsDevelop a range of metrics

3 The openair package

Graphical-user interface (GUI)?Remote repository with full version control and easierinstallationDevelop documentationReproducible analyses using Sweave, R and LATEX



Thank you for you attention!

Questions?

David [email protected]


the open-source air pollution project - openair€¦ · the open-source air pollution project...

Documents