Science of the Total Environment 488–489 (2014) 197–207

Contents lists available at ScienceDirect

Science of the Total Environment

j ourna l homepage: www.e lsev ie r .com/ locate /sc i totenv

Can artificial neural networks be used to predict the origin ofozone episodes?

T. Fontes a,b,⁎, L.M. Silva c,d, M.P. Silva a, N. Barros a, A.C. Carvalho e

a University Fernando Pessoa, Global Change, Energy, Environment and Bioengineering Center (CIAGEB), Praça 9 de Abril, 349, 4249-004 Porto, Portugalb University of Aveiro, Department of Mechanical Engineering/Centre for Mechanical Technology and Automation, Campus Universitário de Santiago, 3810-193 Aveiro, Portugalc University of Aveiro, Department of Mathematics, Campus Universitário de Santiago, 3810-193 Aveiro, Portugald INEB— Instituto de Engenharia Biomédica, Rua do Campo Alegre, 823, 4150-180 Porto, Portugale New University of Lisbon, Faculty of Sciences and Technology/Center for Environmental and Sustainability Research (CENSE), Quinta da Torre, 2829-516 Caparica, Portugal

H I G H L I G H T S

• ANN can classify the origin of an O3 episode with a mean error around 2-7%.• The best classification is obtained when a simpler input combination is used.• ANN can help authorities to foster O3 action plans to control exceedances.

⁎ Corresponding author at: University of Aveiro, DepartmCentre for Mechanical Technology and Automation, Cam3810-193 Aveiro, Portugal.

E-mail address: trfontes@ua.pt (T. Fontes).

http://dx.doi.org/10.1016/j.scitotenv.2014.04.0770048-9697/© 2014 Elsevier B.V. All rights reserved.

a b s t r a c t

a r t i c l e i n f o

Article history:Received 5 February 2014Received in revised form 7 April 2014Accepted 20 April 2014Available online xxxx

Editor: P. Kassomenos

Keywords:Human healthOzoneStratosphereTroposphereClassificationArtificial neural network

Tropospheric ozone is a secondary pollutant having a negative impact on health and environment. To control andminimize such impact the European Community established regulations to promote a clean air all over Europe.However, when an episode is related with natural mechanisms as Stratosphere–Troposphere Exchanges (STE),the benefits of an action plan to minimize precursor emissions are inefficient. Therefore, this work aims todevelop a tool to identify the sources of ozone episodes in order to minimize misclassification and thus avoidthe implementation of inappropriate air quality plans. For this purpose, an artificial neural network model –the Multilayer Perceptron – is used as a binary classifier of the source of an ozone episode. Long data series, be-tween 2001 and 2010, considering the ozone precursors, 7Be activity and meteorological conditions were used.With this model, 2–7% of a mean error was achieved, which is considered as a good generalization. Accuracymeasures for imbalanced data are also discussed. The MCC values show a good performance of the model(0.65–0.92). Precision and F1-measure indicate that the model specifies a little better the rare class. Thus, theresults demonstrate that such a tool can be used to help authorities in the management of ozone, namelywhen its thresholds are exceeded due natural causes, as the above mentioned STE. Therefore, the resourcesused to implement an action plan to minimize ozone precursors could be better managed avoiding theimplementation of inappropriate measures.

© 2014 Elsevier B.V. All rights reserved.

1. Introduction

Due to its oxidative characteristics, the adverse effects of tropospher-ic ozone (O3) on human health and on the environment are fully recog-nized (Agrawal et al., 2003; Dimitriou et al., 2011; Heal et al., 2013). Themajor health issues found to be related with high ozone concentrationsare associated with the respiratory system, provoking or increasinglung irritations and asthma (Halonen et al., 2010; Hamade et al., 2008;

ent ofMechanical Engineering/pus Universitário de Santiago,

Ebi and McGregor, 2008;WHO, 2006; Srebot et al., 2009). This pollutantis classified as a greenhouse gas and is responsible for inducing a reduc-tion of the photosynthetic process thus affecting vegetation growth andreproduction, and hence crop productivity (EEA, 2011). Thereby, theprediction of ozone concentrations and, as possible the identification ofits sources are fundamental to improve the effectiveness of publicawareness and policies for human health protection, as well as vegeta-tion, and increase the knowledge on the interactions between theozone concentrations, weather and climate.

To promote a cleaner air in Europe, guidelines, programmes andstandards from the World Health Organization (WHO) were includedin the Directive 2008/50/EC, of 21 May 2008. This Directive definesthe main rules concerning the ambient air quality as well as strategies

198 T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

to reduce, minimize and inform citizens about the adverse effects of airpollution. As a result, information thresholds (180 μg·m−3) and alertthresholds (240 μg·m−3) were defined for ozone. Moreover, for placeswhere one of these limits is exceeded, Member States shall ensure theimplementation of a programme, and/or an action plan, to reduce theozone concentration and minimize its effects. Nevertheless, the recom-mendedmeasuresmaygive resultswhen anthropogenic sources are themain ozone precursors and can be useless if the origin of the episode isnatural.

The main origin of tropospheric ozone is a series of complex photo-chemical reactions regulated by natural and anthropogenic precursoremissions, such as volatile organic compounds (VOCs) and nitrogen ox-ides (NOx), in the presence of solar radiation (wavelength b 424 nm)(Crutzen et al., 1999; Fishman and Crutzen, 1978). However,the ozone concentrations can also increase due to Stratosphere–Troposphere Exchanges (STE). Fishman and Crutzen (1978) reasonedthat the enhancement of tropospheric ozone concentration over theNorthern Hemisphere is based on photochemical reactions but its fluxis not sufficient to explain the measured concentrations. Stevensonet al. (2006), and Hess and Zbinden (2013) show the importance ofthe STE in the tropospheric ozone burden. Studies conducted by Hessand Lamarque (2007), and Solomon et al. (2007), conclude that STEare responsible for nearly 20% of the global tropospheric ozone burdenand Hegglin and Shepherd (2009) suggest that the stratospheric fluxof ozone has been increasing at a nearly constant rate in the NorthHemisphere of approximately 2% per decade since 1970. Therefore,this last process cannot be neglected as a dynamical mechanism thatalso contributes to the tropospheric ozone budget.

The exchange of airmasses between the stratosphere and the tropo-sphere in the extratropics results from irreversible diabatic erosion ofcut-off lows and diffusive mixing connected with tropopause folds(Elbern et al., 1997), associated with upper level frontogenesis andrapid surface cyclogenesis (Davies and Schuepbach, 1994). Othermech-anisms include mesoscale convective systems, thunderstorms andbreaking gravity waves (Stohl et al., 2000). The shape of the STE in theextratropics is highly variable in space and time, which induces a highvariability in the surface measured ozone concentrations influencedby these dynamical processes.

Stratospheric intrusions are responsible for high or very high levelsof ozone in the troposphere with an unknown impact at a regional tolocal scale (Arsićet et al., 2011; Barros et al., 2004; Carvalho et al.,2005; Fontes et al., 2013; San José et al., 2005). Generally, the signatureof this phenomenon follows fine three-dimensional atmosphericmove-ments that only by chance are measured. Very few studies report thesehigh ozone levels. In Madrid (Spain) between 2 h00 and 6 h00 in the29th of April, 2000 several monitoring stations reported ozone concen-tration levels up to 1190 μg·m−3 (San José et al., 2005). In Bor (Servia)ozone concentration reached values of 3000 μg·m−3 in two episodes,which lasted 3–5 days in November 2010 (Arsić et al., 2011). Otherstudies report similar results (e.g. Fadnavis et al., 2010; Ganguly, 2012).

In spite of scarcity of extremely high surface ozone measurements,the actual contribution to ozone episodes near the legal values present-ed above is up to now, difficult to justify and forecast. Unknowing thereal reason for an ozone episode may be costly when action plans areenforced by law (EEA, 1999). Thus, to improve the effectiveness ofthese plans more research on this field is needed in order to identifythe origin of these episodes.

In the last decades, the research has been focused in quantifying theozone levels from different sources and understanding the main pro-cesses relatedwith their formation. In fact, few studies have been devel-oped in order to present tools which can help authorities to optimizethe costs of an action plan and thus minimize the negative effects ofozone. Some exceptions are the studies presented by Grewe (2006)and Emmons et al. (2012). Grewe (2006) applied a chemistry–climatemodel E39/C to determine the origin of ozone (using a classificationmodel), and Emmons et al. (2012) present a technique of tagging

ozone from various source regions by using artificial tracers of NO andits oxidation products. These are interesting approaches to identify theorigin of ozone, but complex and time consuming to implement onthe air quality management networks and services.

Instead of using deterministic models, as the ones used in theabovementioned works, other groups of researchers have been focusedon the prediction of ozone concentrations using statistical models.These linear and non-linear methods can be used for constructingnon-deterministic models that can be used not only to forecast theozone concentration but also to predict the concentrations of other pol-lutants as PM10 and NOx (using regression model). Although linearmodels are acceptable, non-linear models capture the non-linearity ofozone response and thus have been one of the preferred statisticaltools for ozone prediction in the last years (Chattopadhyay andBandyopadhyay, 2007; Hájek and Olej, 2012; Sellitto et al., 2011;Taormina et al., 2011; Tsai et al., 2009; Wang et al., 2003).

In these studies, chemical and/ormeteorological input variableswereused. Most of the studies use chemical variables related with ozone andits precursors' concentrations (NO, NO2) (e.g. Chattopadhyay andChattopadhyay, 2012; Moustris et al., 2012). In addition, in some of thestudies other pollutants, such as carbon oxide — CO, particles — PM(PM10 and/or PM2.5), and sulfur dioxide — SO2, are also included (e.g.Hájek and Olej, 2012; Özbay et al. 2011; Zhang et al., 2012). Concerningmeteorological variables, themost commonly used variables are temper-ature, wind speed, wind direction, solar radiation and relative humidity.Additionally, some authors include other meteorological variables suchas the Solar Zenith Angle (SZA), the reflectance data and the TotalOzone Column (TOC) (Chattopadhyay et al., 2012; Sellitto et al., 2011),rain (Chattopadhyay and Chattopadhyay, 2012; Özbay et al., 2011) andthe cloud cover (Chattopadhyay and Chattopadhyay, 2012; Tsai et al.,2009).

In general, the literature review shows a lack of knowledge about theprediction of the origin of ozone episodes, with the underlying assump-tion that the near surface ozone concentrations are mainly resultingfrom photochemistry reaction products. In addition, the literature re-view on ozone regressionmodels shows that the large majority of stud-ies were developed to predict the surface ozone concentrations usingchemical and meteorological data. Although these regression models(e.g. Hájek and Olej, 2012; Sellitto et al., 2011) are simpler than thoseclassification models (e.g. Emmons et al., 2012; Grewe, 2006) theygive information only on the ozone concentration level but their originis not addressed.

Therefore, themain purpose of thework presented here is to fill thisgap and include the ozone origin into an artificial neural networkmodelfor ozone forecast purpose. To achieve this goal the MLP architecturewas used to build a binary classifier of the origin of ozone episodes, an-thropogenic vs natural. The objective of the development here present-ed is to understand if an intelligent computational tool can be used tohelp authorities to improve the efficiency of ozone concentration man-agement mainly when ozone concentrations exceed threshold valuesdue to natural causes, like STE. Although STE are classified as rareevents, in this case the implementation of control programmes is clearlyineffective. For this reason, knowing the origin of ozone may be of par-amount importance in order to improve the actuation of the air qualitymanagement services improving the use of the available resources.

2. Materials and methods

To predict the origin of ozone episodes between anthropogenic (e.g.photochemical due to the presence of anthropogenic precursors) andnatural (in the present study mainly due to STE), four main steps weredefined: (i) firstly, air quality and meteorological data were collectedfrom different sources (see Subsection 2.1); (ii) secondly, the classifica-tion of ozone episodes was then performed by experts in the field andalso based on knowledge available in the literature (see Subsection2.2); (iii) in the third step, different input scenarios were evaluated, in

199T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

order to identify which variables are more important for this discrimi-nation problemaswell as to foresee localswhere a small number of var-iables are available (see Subsection 2.3); and (iv) finally, the wholemodelling process is described, including a discussion on alternativeaccuracy measures for imbalanced data (see Subsection 2.4). Fig. 1presents an overview of the overall methodology.

2.1. Data collection

High beryllium activities, in its isotope 7 (7Be), may be an indicativefor stratospheric intrusions. Several studies consider 7Be as a parameterto identify stratospheric airmasses (eventuallywith high ozone concen-trations) (Allen et al., 2003; Cristofanelli et al., 2006; Carvalho et al.,

Definsc

Classification of episodes by

experts

Data collection: •7Be activity•Air pollutants concentrations •Meteorological data

30 rep

Staan

Model

Training set ( )

Drando

Model search

Cross-

No

Fig. 1.Methodology overview for

2013). Because the 7Be isotope is a chemical tracer of atmospheric airmasses of high troposphere and low stratosphere, it is the one of the pa-rameters to consider in the study, as also indicated by theWMO (2001).

Therefore, the 7Be measurements collected in the region of Lisbonand Tagus Valley were included in the database. This location is theonly area over Portugal'smainlandwhere 7Be activity (mBq·m−3)mea-surements are available on a regular basis since 2001. The 7Be activitywas measured in aerosol particles collected from the PortugueseNuclear Technological Institute (ITN) weekly between 2001 and 2010(as described in Carvalho et al., 2013).

Additionally, air quality pollutant concentrations andmeteorologicaldata were acquired, inside the study domain, by the air quality andme-teorological national network stations, respectively. Concentrations

ition of enarios

5-fold

etitions?

tisticalalysis

evaluation

Test set ( )

For each scenario:

atasetmization

validation

Yes

the ANN model construction.

200 T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

(μg·m−3) of O3, NO2 and NO, recorded in 25 air quality stations locatedat the Lisbon and Tagus Valley agglomerationwere used. In this analysisthe type of environment of each air quality station was analysed(1: urban; 2: suburban; and 3: rural). Pressure (hPa), precipitation(mm·h−1), temperature (°C), relative humidity (%) and specific humid-ity (g·kg−1) were measured at the Gago Coutinho meteorological sta-tion (serving the Lisbon International Airport).

Fig. 2 shows the location of the different air quality andmeteorolog-ical stations used.

Table 1 presents the various variables considered in this study. Datafrom the air quality stations were validated according to the require-ments of Directive 2008/50/EC. To ensure accuracy of the measure-ments and compliance with the data quality objectives, this Directiverequires an integrated system of quality control and maintenance ofthe measurement equipment fulfilling of ISO/IEC 17025:2005 norm(IPQ, 2005).

2.2. Classification of episodes

From all the available data only cases with hourly concentrations ofozone above 180 μg·m−3 were kept. This value is the threshold overwhich a brief exposure is a risk tohumanhealth, particularly for the sen-sitive sections of the population for which immediate and appropriateinformation is necessary (Directive 2008/50/EC). In these situations au-thorities need to implement an action plan in order to control emissionprecursors and minimize the effects of ozone concentrations. At thispoint, the data consists of a matrix with 153 rows (number of cases)and 22 columns (number of variables measured). The values were

Fig. 2. Spatial distribution of the 7Be, the meteorological and air quality station

summarized on a weekly mean basis, in order to be in accordancewith the 7Be sample frequency.

Two origins for ozone episodes are defined: (i) only local/mesoscalephotochemical origin (class 1); or (ii) other origins, stratospheric (STE)and/or advection (class 2). The episode labellingwas then performed bytwo of the authors of the paper, experts on this research area. This workwas specifically developed for this paper.

Previous knowledge based on a literature review was also consid-ered. The analysis of the literature shows important details that can in-fluence the ozone concentrations, nevertheless, the definition of specificthresholds to classify the origin of these concentrations is difficult todefine.

Meteorological variables are important elements that can directlyinfluence ozone production, accumulation and depletion (Zhang et al.,2012). The study performed by June and Hui-Jun (2010) suggests thatregional climate (e.g. temperature andwind), in a linearway, is respon-sible for 80% of the variance of total ozone. Some studies proved a highcorrelation between air temperature and ozone concentration, resultingin an increase of surface ozone concentrationswith the rise of tempera-ture during photochemical ozone episodes (Dawson et al., 2007; Sousaet al., 2007; David and Nair, 2011; Fernández-Fernández et al., 2011).Moustris et al. (2012) explain that during an advection ozone episode,wind and temperature influence surface ozone concentrations due toits effect in the distribution of air pollutants by dispersion and horizon-tal transport. Moreover, some studies show a high correlation betweenozone and relative humidity which correspond to a variable with directinfluence on the ozone processes (Mintz et al., 2005; Sousa et al., 2007).Camalier et al. (2007) confirm that ozone generally increases with in-creasing temperature and decreases with increasing relative humidity.

s in Lisbon and Tagus Valley used in the construction of the ANN models.

Table 1Measured variables used in the non-linear model construction.

Variable Acronym Description/Unit Time period

General variables Month of occurrence M 1: Jan; 2: Feb; 3: Mar, … –

Beryllium activity concentration 7Be mBq·m−3 WeeklyAir quality variables Type of environment TE 1: urban; 2: suburban; 3: rural Hourly

Ozone concentration O3 μg·m−3 HourlyNitrogen monoxide concentration NO μg·m−3 HourlyNitrogen dioxide concentration NO2 μg·m−3 HourlyNumber of hours of ozone episode NH – Hourly

Meteorological variables Temperature T °C HourlyPressure P hPa HourlyPartial pressure PP mm·h−1 HourlyRelative humidity RH % HourlySpecific humidity SH g·kg−1 Hourly

Table 2Spearman's correlation coefficient (rs) between different O3 measures (minimum,average and maximum) and several air quality and meteorological measures.

VariableO

3

Minimum Average Maximum

Generalvariables

Month of occurrence -0.013 0.184* 0.128

Air qualitymeasurements

Type of environment 0.199* 0.112 -0.077No. of hours with O

3 ≥ 100 μg·m-3 0.317** 0.624** 0.147

No. of hours with O3 ≥ 180 μg·m-3 -0.014 0.117 0.664**

NO2

Minimum -0.020 0.243** -0.045

Average -0.415** 0.019 0.108

Maximum -0.500** -0.294** 0.228**

NO

Minimum 0.198* 0.105 -0.082

Average -0.545** -0.313** 0.091

Maximum -0.648** -0.438** 0.109

Meteorologicalmeasurements

Pressure

Minimum -0.033 -0.122 -0.260**

Average -0.261** -0.545** -0.122

Maximum -0.212** -0.612** -0.103

Temperature

Minimum 0.056 0.324** -0.089

Average -0.001 0.383** -0.008

Maximum -0.221** -0.427** 0.185*

Relative humidity

Minimum 0.113 0.384** 0.101

Average -0.163* -0.548** 0.128

Maximum -0.286** -0.433** 0.106

Specific humidity

Minimum -0.050 -0.109 0.192*

Average -0.109 -0.081 0.154

Maximum -0.328** -0.204* 0.137

Notes: Bold values point out significant values of correlation at: *5% level; and **1% level.Correlation: Strong (|rs| ≥ 75%), Moderate (75% N |rs| ≥ 50%), Low (|rs| b 50%).

201T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

This last relation could be linked to the existence of greater cloud coverand atmospheric instability leading to reduction of photochemical pro-cess. Specific humidity and potential vorticity are two of the tracers wellknown to determinate the presence of ozone in the upper troposphere(Felker et al., 2011).

When photochemical processes are more intense, high correlationsare found between ozone concentrations and several chemical parame-ters asNO, NO2 andNOx (Imet al., 2013,Wang et al., 2003). Another im-portant factor that may explain surface ozone concentration is the7Beisotope activity measured at surface. This isotope is a chemical tracerof atmospheric air masses with origin in the high troposphere–lowstratosphere (WMO, 2001). Reiter et al. (1983) explain that the pres-ence of high 7Be activity is related with stratospheric intrusions. Severalrecent studies consider this isotope a parameter to identify stratospher-ic air masses (Allen et al., 2003; Cristofanelli et al., 2006; Carvalho et al.,2013). In the Zugspitze station (2962 m a.s.l.) Slàdkovic and Munzert(1990) and Stohl et al. (2000) identified the value of 8 mBq·m−3 asthe threshold value to characterize stratospheric intrusions. This valuewas also proposed by Reiter et al. (1983) as a threshold above whichan air mass probably has stratospheric characteristics. Although thisvalue cannot be applied to other study domains Cristofanelli et al.(2006) used it to selected stratospheric intrusion episodes in theCimone mountain (2165 m a.s.l.).

2.3. Input scenarios

It is usually difficult in real world situations to collect a great diversi-ty of variables, such as the case of this study. Therefore, it is important tounderstand the impact on themodel's accuracywhen different/reducedsubsets of variables are used. Thereby, to select the main variables withpredictive capacity of ozone origin, nine scenarios were defined consid-ering different sets of inputs.

For that purpose, a statistical evaluation of the input variables wasperformed in two steps. Firstly, the correlation between ozone, its pre-cursors and some meteorological parameters (NO, NO2, T, P, PP, RHand SH)was analysed. Precursors andmeteorological variables withoutsignificant correlation (p N 0.05) with ozone are candidates to bediscarded. Spearman's correlation is used as the Kolmogorov–Smirnovtest shows no evidence that the variables follow a normal distribution.Secondly, the same data are used to analyse the discriminative powerof each variable by applying a t-test or a Mann–Whitney test (groupsare defined by the classes). The Mann–Whitney test is preferred when-ever the Kolmogorov–Smirnov test shows no evidence that the groupsfollow a normal distribution. This analysis was conducted using SPSSVersion 21 (IBM, 2012).

Table 2 presents the Spearman correlation coefficient (rs) betweenozone concentrations and its precursors as well as with meteorologicalparameters. Statistically significant (p b 0.05 or p b 0.001) values of cor-relation are appropriately marked (* 5% level; ** 1% level). With somefew exceptions low correlations are found. In the air quality and in the

meteorological group of variables moderate correlations are foundwith high statistical significance (p b 0.001).

Table 3 presents the significance value for the t-test and theMann–Whitney test for each variable. Significant values (p b 0.05or p b 0.001) mean that for one of the classes the values of the specif-ic variable are significantly higher (regardless of the specific vari-able). This hints some discriminative power of that variable. On theother hand, for variables with p ≥ 0.05, significant differences be-tween the classes are not expected. Note, however, that despite thefact that the variable shows lack of discriminative power per se, itmight give a good contribution for the discriminative power of aset of variables. It may be observed from Table 2 that some variablessuch as the type of environment, the number of hours with concen-trations higher than 100 μg·m−3, the NO and pressure have a poten-tial lack of discriminative power and are candidates to be discardedin some scenario. All the other variables have, in some way, signifi-cant differences between the classes and can be considered as poten-tially useful for discrimination.

Based on the results provided in Tables 2 and 3, nine scenarios weredefined (S1–S9) based on different combinations of the input variables.

S1 uses all the available variables. The remaining combinationswerecreated by analysing the statistical significance values of the Spearman

Table 3Significance value (p-value) for the t-test orMann–Whitney test for the groups defined bythe two classes of ozone episodes.

Variable p-Value

General variables Month of occurrence 0.001Air qualitymeasurements

Type of environment 0.670No. of hours with O3 ≥ 100 μg·m−3 0.472No. of hours with O3 ≥ 180 μg·m−3 0.000O3 Minimum 0.057

Average 0.042Maximum 0.680

NO2 Minimum 0.000Average 0.000Maximum 0.004

NO Minimum 0.021Average 0.096Maximum 0.376

Meteorologicalmeasurements

Pressure Minimum 0.182Average 0.349Maximum 0.100

Temperature Minimum 0.022Average 0.090Maximum 0.041

Relative humidity Minimum 0.715Average 0.001Maximum 0.565

Specific humidity Minimum 0.000Average 0.000Maximum 0.000

Bold values: p-value b 0.05.

202 T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

correlation (S2–S5), and the statistical significance values of the t-test orMann–Whitney test (S6–S9). For each of these groups, the procedurefollows with the analysis of the significance values of the minimum(S2 and S6), average (S3 and S7), and maximum values (S4 and S8) ofeach variable. In these groups the month of occurrence, the type of en-vironment and/or the number of hours of ozone concentration higherthan 100or 180 μg·m−3were also included ,whenever statistical signif-icance is achieved. S5 includes all variables with significant values of

Table 4Set of input variables for each scenario.

Variables Scenar

S1

General variables Month of occurrence xAir quality measurements Type of environment x

No. of hours with O3 ≥ 100 μg·m−3 xNo. of hours with O3 ≥ 180 μg·m−3 xO3 Minimum x

Average xMaximum x

NO2 Minimum xAverage xMaximum x

NO Minimum xAverage xMaximum x

Meteorological measurements Pressure Minimum xAverage xMaximum x

Temperature Minimum xAverage xMaximum x

Relative humidity Minimum xAverage xMaximum x

Specific humidity Minimum xAverage xMaximum x

Total 25

Spearman correlation and S9 includes all variables with significantvalues of the t-test or Mann–Whitney test. Table 4 shows a summaryof the variables considered by these scenarios.

2.4. Modelling

ANNs are a broad set of statisticalmodels developedwith an inspira-tion from the biological neural networks of the brain. They have beenused with success in several areas of knowledge, from engineering ap-plications to social sciences.

In the present section we describe the specific model used, how it isoptimized and how it is applied to obtain binary predictions for the or-igin of ozone episodes. For a comprehensive approach on ANNs pleaserefer to Bishop (1995) or Haykin (2009). All the implementations andcomputations were performed using MATLAB (MathWorks, 2012).

2.4.1. The modelWe consider the Multilayer Perceptron (MLP) a feedforward artifi-

cial neural network model that maps a set of inputs to a set of outputsusing nonlinear compositions of functions. An MLP can be seen as a di-rected graph with a stacked arrangement of layers composed of pro-cessing units (neurons). Layers between the inputs and the outputsare known as hidden layers.

We restrict here to the case of a single hidden layer as this type ofMLP is known to be a universal approximator provided that the numberof neurons in that layer (hidden neurons) is sufficient (Cybenko, 1989).Also, one output neuron is sufficient as we are performing binary classi-fication. In a formalway, for d inputs and nhidhidden neurons, themodelcan be expressed as:

y ¼ φXnhidj¼1

w 2ð Þj h j þ b 2ð Þ

0@

1A

¼ φ

Xnhid

j¼1

w 2ð Þj φ

Xdk¼1

w 1ð Þkj xk þ b 1ð Þ

j

!þ b 2ð Þ

0@

1A;

hj

ð1Þ

io

S2 S3 S4 S5 S6 S7 S8 S9

x x x x x xx x xx x x x

x x x x x x

x x

x x x xx x x xx x x x x xx x x xx x xx x x

x xx x xx x x

x x x xx x

x x x x x xx x

x x x x xx x x

x x x xx x

x x x x x13 15 5 21 6 6 5 15

203T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

where:

xk k-th input variable;wkj

(1) Parameter/weight connecting input k to hidden neuron j;bj(1) Bias term connected to hidden neuron j;

hj Output of hidden neuron j;wj

(2) Parameter/weight connecting hidden neuron j to the outputneuron;

b(2) Bias term connected to the output neuron;y Output/response of the MLP.φ Activation function: φ að Þ ¼ 1

1þe−a :

TheMLP architecture can thus be represented in the form d : nhid : 1,which means it has d inputs (depending on the complexity of eachscenario used, see Subsection 2.3), nhid hidden neurons and 1 outputneuron.

2.4.2. Model searchTo search for the optimal set of weights (also known as training

phase), the batch backpropagation iterative algorithm was used alongwith the gradient descent optimization technique.

Two different objective (cost) functions were used. Let n be thenumber of instances, ti ∈ {0, 1} is the target value (class code) for in-stance i and yi ∈ [0, 1] is the MLP output for instance i. The Cross–Entropy (CE) cost function (Bishop, 1995) is expressed as:

RCE ¼ −Xni¼1

yi log yið Þ þ 1−yið Þ log 1−yið Þ; ð2Þ

while the Exponential (EXP) cost function is given by

REXP ¼Xni¼1

τ expt i−yið Þ2

τ

!: ð3Þ

The latter is a parameterized family of cost functions with the abilityto emulate the behaviour of other cost functions (Silva et al., 2008). EXPhas the advantage of adapting the cost function to the particular prob-lem at hand by controlling τ. The learning rate η controls the rate ofweight updating and is an essential parameter in the training phase.We used an adaptive learning rate as described in Marques de Sá et al.(2012).

To set the number of training epochs (iterations of the optimizationalgorithm), the number of hidden neurons nhid and the τ parameter ofEXP a series of preliminary experiments were carried out as now de-scribed. For each value of nhid in the case of CE and each pair (nhid, τ)in the case of EXP, 10 repetitions of a stratified 5-fold cross-validationwere performed. For each repetition, thewhole datasetwas randomizedand each training fold was normalized, to have inputs with zero meanand unit standard deviation, and the corresponding test fold was nor-malized using the parameters of the training fold; theMLPwas then ini-tialized with small random weights and trained during 3000 epochs.Along this process, we kept track of the test error (misclassifications)which was then used to choose an appropriate number of epochs, nhidand τ. For this purpose, nhid was varied from 2 to 20 and τ was variedin the interval [−3, 3].

The parameterswere then chosen using a combination of the follow-ing three rules: lowest mean test error and standard deviation; lowestcomplexity (lowest number of hidden neurons); and homogeneity ofthe model across the scenarios (i.e. as much as possible the same archi-tecture across scenarios). This strategy prevents picking overfittedmodels which happen when a too complex model fits the trainingdata and not the distribution of the data (despite the training error de-creases, the test error increases).

The complete set of parameters for each pair scenario/cost functionis shown in Table 6. As you can see, depending on the inputs and the

cost function of the model, different optimal configurations were re-corded. However, no correlation was found between the architectureand the complexity of the model (number of inputs and hidden neu-rons), which show the non-linearity of the problem. Moreover, foreach scenario, no significant differences were found between themean errors for the different architectures (~error b 5%), which suggestthat simple models can provide satisfactory results.

3. Results and discussion

Althoughwewant a simplemodelwith the lowestmean error, accu-racy measures must be used to assess the degree of closeness of themodel predictions with the “real” values. This evaluation can be veryimportant, when we have imbalanced data, as in this case. Thus, inorder to analyse these questions, Subsection 3.1 presents the obtainedresults and discusses the imbalanced problem and accuracy estimation.In addition, policy implications related with the application of themodel are discussed in Subsection 3.2.

3.1. The imbalance problem and accuracy estimation

It is known that imbalanced data can compromise learning in twoways. First, by compromising the performance of learning algorithmsthat are not usually prepared for such imbalanced class distributions.Second, traditional accuracy measures are distribution dependent anddo not provide a clear picture of the classifier's functionality. Severalstrategies have been suggested, and tested, to solve (or at least, allevi-ate) the effects of class imbalance, either by adapting, in some sense,the traditional classifiers (usually by using sampling strategies) or byconsidering different measures of performance. We followed the latterapproach. For that purpose we considered a set of seven performancemeasures based on the confusion matrix of the test set: Accuracy, Errorrate, Precision, Recall, F1-measure, balanced error rate (BER) and Mat-thews Correlation Coefficient (MCC) (see Appendix A for details).

The estimation of such quantities is carried out with a similar exper-imental procedure described before (Subsection 2.4.2) with the excep-tion that now 30 repetitions of a 5-fold cross-validation are performed(the data is randomly shuffled in each repetition). Moreover, insteadof the singlemean test error, we nowkeep track of all the confusionma-trices generated.

Figs. 3 and 4 present the results of the above accuracy measuresusing the CE and EXP cost functions respectively, obtained for each ofthe nine scenarios previously defined. It can be observed that the resultsare, in general, promising with expected differences across scenarios. Amean error between 2% and 7% was observed. This means that if wehave 100 ozone episodes with the application of this model at least 93can be classified correctly. However, to understand if our rare class(and our main goal) is correctly classified, an analysis of the other accu-racy measures must be done.

A ranking obtainedwithMCC reveals both for CE andEXP that the bestscenarios are from S1 to S5, pointing S3 and S4 as the best ones. For CE, S3is the best scenario as the Precision and F1-measure values are slightlyhigher, indicating that the model specifies a little better in the rare class(natural origin of ozone), while EXP elects S4 as the best one. The latterhas the advantage of being a simpler model (requiring less inputs).Hence, depending on the available information (variables collected) atthe specific site one may choose to use S3 or S4. Counter intuitively, wenote that S1 is not the best scenario, indicating, eventually, that some ofthe variables are uninformative or redundant (but all of them bringsome noise) and can be discarded from the model. This fact supportsthe importance of defining and studying different (input) scenarios.

It is also interesting to observe the impact of the class imbalance onthe results, namelywhen comparing Errorwith BER. The latter essential-ly doubles the value of the former and, therefore, the accuracy in themost prevalent class is masking the behaviour of the model in theother class. This is more evident for scenarios S6 through S9 where

Fig. 3. Classification results using the CE cost function: a) Precision, Recall and F1-measures; and b) Error, BER and MCC. ExceptMCC, all the measures are presented in %.

204 T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

BER is above 10%. This can be related with the type of variables used toperform the prediction. In fact, these scenarios (S6–S9) did not includeany variable related with pressure. The overall error should not, in thiscase, be the preferred measure to analyse. Note also that the standarddeviations are high in some cases. This is expected as the dataset issmall; a single misclassified instance has a great impact on theestimates.

It is not clear whether one of the cost functions performs betterthan the other. Despite the trial to choose τ as similar as possibleacross scenarios, it was found that EXP was quite sensitive to thechoice of this parameter essentially due to the small size of thedataset. Nevertheless, EXP consistently prefers (except for S9)lower complexity models.

It was also verified which variables are most important for this dis-crimination problem. For example, when the number of hours withozone concentrations higher than 180 μg·m−3, the minimum pressureand the minimum specific humidity variables are introduced on theS3 scenario, corresponding to S5, a lower performance is obtained. Thesame happens between S4 and S5. Moreover, the scenarios whichinclude the maximum NO2 and maximum temperature can producebetter results.

3.2. Policy implications

In this work, several scenarios were analysed in order to understandwhich variables allow a better prediction (binary classification) of theorigin of ozone episodes. The results across scenarios have shown tobe very similar. This suppleness is very important because it demon-strates that different combinations of inputs provide similar results. Inaddition, the accuracy of the model gives very promising results. Therare events, as the ones introduced into the surface through strato-spheric–tropospheric exchanges (STE), are very well classified showingmean errors of 2% to 7%. Accuracy measures confirm the good perfor-mance of the model showing MCC values ranging between 0.65 and0.92. Precision and F1-measure indicate that the model specifies a littlebetter the rare class. Hence, although the model was developed to theLisbon area, the results obtained suggest that for other areas with limit-ed information it can be possible to adapt the inputs, maintaining thesame error levels.

This capacity of adaptation of themodel is very important since highcosts of ozone pollution have been recorded in the last decade. EU esti-mated, for the year 2000, that ozone pollution had an impact on healthcosts and crop losses of about 11.9 and 2.8 billion euros, respectively

Fig. 4. Classification results using the EXP cost function: a) Precision, Recall and F-measures; and b) Error, BER andMCC. ExceptMCC, all the measures are presented in %.

205T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

(EEA, 2008). Although the proposed tool did not cut the problem, someof those features demonstrate that such a tool has a high potential inhelping authorities to develop ozone action plans with more accuracy.Moreover, coupling this classification model with meteorological andair quality prediction models, is possible to predict the origin of ozoneexceedances and thus improve the use of the resources available tomin-imize the impacts on environment and human health. This is speciallyimportant during the occurrence of STE where the control of ozoneemission precursors as VOC and NOx has a reduced impact in the tropo-spheric ozone levels. Thus instead of using such resources to control

Table 5Number of epochs, hidden neurons and τ selected for each of the nine scenarios.

Cost function Parameters Scenarios

S1 S2 S3

CE Epochs 1000 1000 1000nhid 10 10 10

EXP Epochs 1500 2000 2500nhid 8 6 6τ 3 1 1

ozone emission precursors, those resources can be used to reinforcethe plans of human health protection.

4. Conclusions

Amultilayer perceptron with one hidden layer was applied to auto-mate the classification of the origin of ozone episodes given several airquality and meteorological variables. It was found that with a smallcomplexity model (with 4 to 10 hidden neurons) a mean error around2% to 7% may be achieved (depending on the scenario at hand) which

S4 S5 S6 S7 S8 S9

2000 1000 2000 2000 2000 200010 10 10 8 8 4

2000 2500 3000 100 100 1004 6 6 6 6 62 2 2 2 1 1

Table 6Confusion matrix for a binary classification problem.

Predicted values

Positive(P)

Negative(N)

Actual values Positive (P) True positives (TP) False negatives (FN)Negative (N) False positives (FP) True negatives (TN)

206 T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

can be considered as a good generalization. The analysis of the accura-cy measures shows that the learning process is affected by the classimbalance present in the data. Thus, measures such as the meanerror (or accuracy) are not the most appropriate. However, theMCC values show a good performance of the model (0.65–0.92). Inaddition, Precision and F1-measure indicate that the model specifiesa little better the rare class. Although, the best results are obtainedwhen a simpler input combination is included, as is the case ofscenarios S3 and S4 (please see details in Table 5) no correlationwas found between the model complexity and accuracy measures.Due to the small dataset size available for this study, results shouldbenefit from the collection of more data or the application ofsampling strategies as suggested by some authors.

The present work demonstrates that such a tool has a potential ap-plication value on helping authorities to foster ozone action plans, inparticular to control ozone exceedances due to natural causes (as theones introduced into the surface through stratospheric–troposphericexchanges — STE). This allows governments to better justify to theEuropean Commission and also better assess the potential benefits ofthe introduction of an action plan to minimize emissions of ozone pre-cursors, therefore minimizing the implementation of inappropriate airquality plans.

Conflict of interest

There are no known conflicts of interest associatedwith this publica-tion and there has been no significant financial support for this workthat could have influenced its outcome.

Acknowledgements

This work was partially funded by FEDER Funds through the Opera-tional Programme “Factores de Competitividade — COMPETE” and byNational Funds through FCT — Portuguese Science and TechnologyFoundation within the projects PTDC/CTE — ATM/105507/2008, PTDC/EIA — EIA/119004/2010 and the Strategic Project PEst-C/EME/UI0481/2014. The authors also acknowledges to the Campus Tecnológico e Nu-clear, of the Instituto Superior Técnico-Universidade de Lisboa and tothe Portuguese Environmental Agency (APA) that made the berylliumactivity and the air quality available, respectively.

Appendix A

Wediscuss accuracymeasures for the case of a two-class imbalancedproblem, that is, where one of the classes has fewer cases (say, less than10%) than the other one. The basis for this approach is the confusionma-trix, a two-way table that summarizes the performance of a classifier ineach class. Considering one of the classes as the positive (P) class (usu-ally the rare one) and the other as the negative (N) class, four quantitiesmay be defined: the true positives (TP), the true negatives (TN), thefalse positives (FP) and the false negatives (FN). For example, TP arethe number of instances of the positive class that were correctly classi-fied. The confusion matrix is represented in Table 6.

Accuracy or error rate can be readily computed from the confusionmatrix by

Accuracy ¼ TP þ TNTP þ TN þ FP þ FN

; ð4Þ

Error rate ¼ 1−Accuracy: ð5Þ

However, thesemeasures are not the most appropriate to analyse inthe presence of imbalanced data and so, several measures have beensuggested for this purpose (a good overview can be found in He andGarcia (2009)). In this work we have considered:

Precision ¼ TPTP þ FP

; ð6Þ

Recall ¼ TPTP þ FN

; ð7Þ

F1−measure ¼ 2� Precision� RecallPrecisionþ Recall

; ð8Þ

BER ¼ 12

FPTN þ FP

þ FNTP þ FN

� �; ð9Þ

MCC ¼ TP � TN−FP � FNffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiTP þ FPð Þ TP þ FNð Þ TN þ FPð Þ TN þ FNð Þp : ð10Þ

Precision gives an insight on how the classifier specifies on the posi-tive class, giving a measure of how many positive predicted instancesare in fact positive. This does not mean however that all the positive in-stances have been correctly classified. On the other hand, Recall gives aninsight of the classifier's performance on the positive class, measuringhow well the whole positive class is recognized. In general, Precisionand Recall share an inverse relationship, that is, increasing Precision im-plies a reduction in Recall (but not necessarily, as in a perfect classifica-tion scenario, with no errors, one would have Precision= Recall=1). Itis obvious that these measures should not be analysed separately andthe F1-measure is an attempt to combine both (it is the harmonicmean of Precision and Recall). It varies between 0 and 1 and highervalues imply higher (trade-off) values for Precision and Recall. BERstands for Balanced Error Rate and essentially computes an equallyweighted average of the errors in each class (in contrast to Accuracywhich weights class errors by their proportions in the data). This givesa fairer estimate of the performance and functionality of the classifier.Finally, MCC stands for Matthews Correlation Coefficient and is consid-ered as one of the best measures to summarize a confusion matrix (in asingle value) (He andGarcia (2009)). Note that thismeasure uses all theTP, TN, FP and FN values. The classifier is better as MCC tends to one (itsmaximum value) and no better than a toss of a coin for MCC= 0 (MCCalso takes value in [−1,0] but in this case one would just change theclass labels to get MCC back to [0,1]). Except for MCC, all the measurescan be multiplied by 100 to get the results in %. The joint analysis ofthis pack of measures can give a more clear view of the classifier's per-formance and value.

References

Agrawal M, Singh B, Rajput M, Marshall F, Bell J. Effect of air pollution on periurban agri-culture: a case study. Envron Pollut 2003;126(3):323–39.

Allen DJ, Dibb JE, Ridley B, Pickering KE, Talbot RW. An estimate of the stratospheric con-tribution to springtime tropospheric ozone maxima using TOPSE measurements andberyllium-7 simulations. J Geophys Res 2003;108(D4):8355.

207T. Fontes et al. / Science of the Total Environment 488–489 (2014) 197–207

Arsićet M, NikolićD, Djordjević P,Mihajlović I, Živković Z. Episodes of extremely high con-centrations of tropospheric ozone in the urban environment in Bor — Serbia. AtmosEnviron 2011;45(32):5716–24.

Barros N, Borrego C, Fontes T, Carvalho AC, Moreira N, Leitão P, et al. Stratospheric ozoneinto the troposphere over Portugal In Air Pollution 2004, Rhodes, Greece, 30/06 a 02/07; 2004. p. 315–25.

Bishop CM. Neural networks for pattern recognition. Oxford, U.K.: Oxford UniversityPress; 1995

Camalier L, Cox W, Dolwick P. The effects of meteorology on ozone in urban areas andtheir use in assessing ozone trends. Atmos Environ 2007;41(33):7127–37.

Carvalho A, Moreira N, Leitão P, Fontes T, Barros N, Borrego C. A winter ozone episodeover Portugal. Geophys Res. Abs. European Geosciences Union 24–29/04, Vol. 7,(SRef-ID: 1607-7962/gra//EGU05-A-06926). Vienna, Austria; 2005.

Carvalho AC, Reis M, Silva L, Madruga MJ. A decade of 7Be and 210Pb activity in surfaceaerosols measured over the Western Iberian Peninsula. Atmos Environ 2013;67:193–202.

Chattopadhyay S, Bandyopadhyay G. Artificial neural network with backpropagationlearning to predict mean monthly total ozone in Arosa, Switzerland. Int J RemoteSens 2007;28(20):4471–82.

Chattopadhyay G, Chattopadhyay S. Modeling and prediction of monthly total ozone con-centrations by use of an artificial neural network based on principal component anal-ysis. Pure Appl Geophys 2012;169:1891–908.

Chattopadhyay G, Chattopadhyay S, Chakraborthy P. Principal component analysis andneurocomputing-basedmodels for total ozone concentration over different urban re-gions of India. Theor Appl Climatol 2012;109:221–31.

Cristofanelli P, Bonasoni P, Tositti L, Bonafe U, Calzolari F, Evangelisti F, et al. A 6-year anal-ysis of stratospheric intrusions and their influence on ozone at Mt. Cimone (2165 mabove sea level). J Geophys Res 2006;111:D03306.

Crutzen P, Lawrence M, Poschl U. On the background photochemistry of troposphericozone. Tellus 1999;A 51:123–46.

Cybenko G. Approximation by superpositions of a sigmoidal function. Math Control Sig-nals Syst 1989;2:303–14.

David LM, Nair PR. Diurnal and seasonal variability of surface ozone and NOx at a tropicalcoastal site: association with mesoscale and synoptic meteorological conditions. JGeophys Res 2011;116:D10303.

Davies TD, Schuepbach E. Episodes of high ozone concentrations at the earth's surfaceresulting from transport down from the upper troposphere/lower stratosphere: a re-view and case studies. Atmos Environ 1994;28(1):53–68.

Dawson JP, Adam PJ, Pandis SN. Sensitivity of ozone to summertime climate in the easternUSA: a modeling case study. Atmos Environ 2007;41:1494–511.

Dimitriou K, Kassomenos PA, Paschalidou AK. Assessing the relative risk of daily mortalityassociated with short term exposure to air pollutants in the European Union throughair quality indices. Ecol Indic 2011;27:108–15.

Directive 2008/50/EC of the European Parliament and of the Council of 21 May 2008 onambient air quality and cleaner air for Europe. Off J Eur Union, L 152/1–44.

Ebi K,McGregor G. Climate change, tropospheric ozone and particulate matter, and healthimpacts. Environ Health Perspect 2008;116(11):1449–55.

EEA. Ozone position paper (final version). Prepared by the Ad-Hoc Working Group onOzone Directive and Reduction Strategy Development. Luxembourg: Office for Offi-cial Publications of the European Communities; 1999.

EEA. Costs of ozone pollution. [Available on:] http://www.eea.europa.eu/themes/air/air-quality/more-about-air-pollutants/ozone-1/impacts/costs-of-ozone-pollution, 2008.

EEA. Air quality in Europe — 2011 report. Copenhagen: Publications Office of theEuropean Union; 2011.

Elbern H, Kowol J, Sladkovic R, Ebel A. Deep stratospheric intrusions. A statistical assess-ment with model guided analyses. Atmos Environ 1997;31(19):3207–26.

Emmons LK, Hess PG, Lamarque J-F, Pfister GG. Tagged ozone mechanism for MOZART-4,CAM-chem and other chemical transport models. Geosci Model Dev 2012;5:1531–42.

Fadnavis S, Chakraborty T, Beig G. Seasonal stratospheric intrusion of ozone in the uppertroposphere over India. Ann Geophys 2010;28:2149–59.

Felker SR, Moody JL, Wimmers AJ, Osterman G, Bowman K. A multi-sensor upper tropo-spheric ozone product (MUTOP) based on TES Ozone and GOES water vapour: deri-vation. Atmos Chem Phys 2011;11:6515–27.

Fernández-Fernández MI, Gallego MC, García JA, Acero FJ. A study of surface ozone vari-ability over the Iberian Peninsula during the last fifty years. Atmos Environ 2011;45:1946–59.

Fishman J, Crutzen P. The origin of ozone in troposphere. Nature 1978;274:855.Fontes T, Silva LM, SilvaMP, Barros N, Carvalho AC. Predicting the origin of ozone episodes

using artificial neural networks. 16th International Union of Air Pollution Preventionand Environmental Protection Associations (IUAPPA) Congress, Cape Town, SouthAfrica; 2013. p. 94.

Ganguly N. Influence of stratospheric intrusion on the surface ozone levels in India. ISRNMeteorology; 2012 [Article ID625318].

Grewe V. The origin of ozone. Atmos Chem Phys 2006;6:1495–511.Hájek P, Olej V. Ozone prediction on the basis of neural networks, support vector regres-

sion and methods with uncertainty. Ecol Informat 2012;12:31–42.Halonen J, Lanki T, Tiittanen P, Niemi J, Loh M, Pekkanen J. Ozone and cause-specific car-

diorespiratory morbidity and mortality. J Epidemiol Community Health 2010;64:814–20.

Hamade A, Rabold R, Tankersley C. Adverse cardiovascular effects with acute particulatematter and ozone exposures: interstrain variation in mice. Environ Health Perspect2008;116(8):1033–9.

Haykin S. Neural networks and learning machines. 3rd Edition. Prentice Hall; 2009.He H, Garcia E. Learning form imbalanced data. IEEE Trans Knowl Data Eng 2009;21(9):

1263–84.Heal MR, Heaviside C, Doherty RM, Vieno M, Stevenson DS, Vardoulakis S. Health

burdens of surface ozone in the UK for a range of future scenarios. Environ Int2013;61:36–44.

Hegglin MI, Shepherd TG. Large climate induced changes in ultraviolet index andstratosphere-to-troposphere ozone flux. Nat Geosci 2009;2:687–91.

Hess PG, Lamarque J. Ozone source attribution and its modulation by the arctic oscillationduring the spring months. J Geophys Res 2007;112:D11303.

Hess PG, Zbinden R. Stratospheric impact on tropospheric ozone variability and trends:1990–2009. Atmos Chem Phys 2013;13:649–74.

IBM. IBM SPSS Statistics, Version 21.0, 2012, Chicago, IL, USA; 2012.Im U, Incecik S, Guler M, Tek A, Topcu A, Unal YS, et al. Analysis of surface ozone and ni-

trogen oxides at urban, semi-rural and rural sites in Istanbul, Turkey. Sci Total Envi-ron 2013;443:920–31.

IPQ. Norma Portuguesa - Requisitos gerais de competência para laboratórios de ensaio ecalibração (ISO/IEC 17025:2005). Instituto Português da Qualidade; 2005.

June W, Hui-Jun W. The relationship between total ozone and local climate at Kunmingusing Dobson and TOMS data. Atmos Ocean Sci Lett 2010;3:207–12.

Marques de Sá J, Silva LM, Santos JM, Alexandre LA. Minimum error entropy classification.Studies in computational intelligence. Springer; 2012. p. 420.

MathWorks MATLAB and Statistics Toolbox Release 2012. Natick, Massachusetts, UnitedStates: The MathWorks, Inc.; 2012.

Mintz R, Young BR, Svrcek WY. Fuzzy logic modeling of surface ozone concentrations.Comput Chem Eng 2005;29:2049–59.

Moustris KP, Nastos PT, Larissi IK, Paliatsos AG. Application of Multiple Linear RegressionModels and Artificial Neural Networks on the surface ozone forecast in the GreaterAthens Area, Greece. Adv Meteorol 2012;2012. Article ID 894714, 8.

Özbay B, Keskin GA, Doğruparmak ŞÇ, Ayberk S. Predicting tropospheric ozone concentra-tions in different temporal scales by using multilayer perceptron models. EcolInformat 2011;6(3–4):242–7.

Reiter R, Munzert K, Kanter HJ, Poetzl K. Cosmogenic radionuclides and ozone at a moun-tain station at 3.0 km a.s.l. Arch Meterol Geophys Bioklimatol 1983:131–60.

San José R, Stohl A, Karatzas K, Bohler T, James P, Pérez JL. A modelling study of an ex-traordinary night time ozone episode over Madrid domain. Environ Model Software2005;20(5):587–93.

Sellitto P, Bojkov BR, Liu X, Chance K, Del Frate F. Tropospheric ozone column retrieval atnorthern mid-latitudes from the Ozone Monitoring Instrument by means of a neuralnetwork algorithm. Atmos Meas Tech 2011;4:2375–88.

Silva L, Marques de Sá J, Alexandre LA. Data classification with multilayer perceptronsusing a generalized error function. Neural Netw 2008;21(9):1302–10.

Slàdkovic R, Munzert K. Lufthygienisc-klimatologische Uberwachung im bayrischenAlpenarum: Ozonspitzen auf der Zugspitze durch Zustrom aus der Stratosphäre.Final report 908080— VI.4. Fraunhofer-Institut für AtmosphärischeUmweltforschung;1990.

Solomon S, Qin D, ManningM, Chen Z, Marquis M, Averyt KB, et al. Contribution ofWork-ing Group I to the fourth assessment report of the Intergovernmental Panel onClimate Change. Cambridge, United Kingdom and New York, USA: Cambridge Uni-versity Press; 2007 [http://www.ipcc.ch/publications_and_data/ar4/wg1/en/ch7s7-4-4.html#table-7-9].

Sousa SIV, Martins FG, Alvim-Ferraz MCM, Pereira MC. Multiple linear regression and ar-tificial neural networks based on principal components to predict ozone concentra-tions. Environ Model Software 2007;22(1):97–103.

Srebot V, Gianicolo EAL, Rainaldi G Trivella M, Sicari R. Ozone and cardiovascular injury.Cardiovasc Ultrasound 2009;7:30.

Stevenson DS, Dentener FJ, Schultz MG, van Noije TPC Ellingsen K, Wild O, Zeng G, et al.Multimodel ensemble simulations of present-day and near-future troposphericozone. J Geophys Res 2006;111(D8):27.

Stohl A, Spichtinger-Rakowsky N, Bonasoni P, Feldmann H,MemmesheimerM, Scheel HE,et al. The influence of stratospheric intrusions on alpine ozone concentrations. AtmosEnviron 2000;34(9):1323–54.

Taormina R, Mesin L, Orione F, Pasero E. Forecasting tropospheric ozone concentrationswith adaptive neural networks. Proceedings of International Joint Conference onNeural Networks, San Jose, California, USA, July 31–August 5, 2011; 2011.

Tsai CH, Chang LC, Chiang HC. Forecasting of ozone episode days by cost-sensitive neuralnetwork methods. Sci Total Environ 2009;407(6):2124–35.

Wang W, Lu W, Wang X, Leung A. Prediction of maximum daily ozone level using com-bined neural network and statistical characteristics. Environ Int 2003;29:555–62.

WHO. Air quality guidelines — global update 2005. Copenhagen, Denmark: World HealthOrganization Regional Office for Europe; 2006.

WMO. WMO TD No. 1073 — global atmosphere watch measurements guide. [Last accesson 30 November of 2012] http://www.wmo.int/, 2001.

Zhang H, ZhangW, Palazoglu A, SunW. Prediction of ozone levels using a HiddenMarkovModel (HMM) with Gamma distribution. Atmos Environ 2012;62:64–73.