Solar Phys (2017) 292:173DOI 10.1007/s11207-017-1196-y

Solar Energetic Particle Forecasting Algorithmsand Associated False Alarms

B. Swalwell1 · S. Dalla1 · R.W. Walsh1

Received: 12 April 2017 / Accepted: 25 October 2017 / Published online: 10 November 2017© The Author(s) 2017. This article is published with open access at Springerlink.com

Abstract Solar energetic particle (SEP) events are known to occur following solar flaresand coronal mass ejections (CMEs). However, some high-energy solar events do not resultin SEPs being detected at Earth, and it is these types of event which may be termed “falsealarms”.

We define two simple SEP forecasting algorithms based upon the occurrence of a mag-netically well-connected CME with a speed in excess of 1500 km s−1 (a “fast” CME) or awell-connected X-class flare and analyse them with respect to historical datasets. We com-pare the parameters of those solar events which produced an enhancement of >40 MeVprotons at Earth (an “SEP event”) and the parameters of false alarms.

We find that an SEP forecasting algorithm based solely upon the occurrence of a well-connected fast CME produces fewer false alarms (28.8%) than an algorithm which is basedsolely upon a well-connected X-class flare (50.6%). Both algorithms fail to forecast a rela-tively high percentage of SEP events (53.2% and 50.6%, respectively).

Our analysis of the historical datasets shows that false-alarm X-class flares were eithernot associated with any CME, or were associated with a CME slower than 500 km s−1;false-alarm fast CMEs tended to be associated with flare classes lower than M3.

A better approach to forecasting would be an algorithm which takes as its base the oc-currence of both CMEs and flares. We define a new forecasting algorithm which uses acombination of CME and flare parameters, and we show that the false-alarm ratio is similarto that for the algorithm based upon fast CMEs (29.6%), but the percentage of SEP eventsnot forecast is reduced to 32.4%.

Lists of the solar events which gave rise to >40 MeV protons and the false alarms havebeen derived and are made available to aid further study.

Keywords False alarms · Solar energetic particles · Coronal mass ejections · Solar flares

Electronic supplementary material The online version of this article(doi:10.1007/s11207-017-1196-y) contains supplementary material, which is available to authorizedusers.

B B. Swalwellbswalwell@uclan.ac.uk

1 Jeremiah Horrocks Institute, University of Central Lancashire, Preston, PR1 2HE, UK

173 Page 2 of 27 B. Swalwell et al.

1. Introduction

Solar energetic particles (SEPs) pose a significant radiation hazard to humans in space (Hoff,Townsend, and Zapp, 2004) and in high-flying aircraft, particularly at high latitudes (Becket al., 2005). They may also cause serious damage to satellites (Feynman and Gabriel, 2000)and make high-frequency radio communications either difficult or impossible (Hargreaves,2005). Accurate forecasting of the arrival of SEPs at locations near Earth is consequentlyvital.

SEPs are known to be energised by flares and coronal mass ejections (CMEs), processeswhich can take place within the same active region on the Sun in close temporal association.Flares exhibiting high levels of energy emission in soft X-rays (SXR) and CMEs with highspeeds have long been associated with a high likelihood of SEPs being detected at Earth (seee.g. Dierckxsens et al., 2015). The bases for making such associations are studies of largenumbers of events which are directed towards demonstrating the connection between flareand CME properties, and SEP events. These studies proceed to look for correlations betweenevent parameters and the proportion of associated solar event SEPs (e.g. Belov et al., 2005;Cliver et al., 2012).

Whether SEPs are actually detected at Earth, however, may depend upon many differ-ent factors: the mechanism behind their acceleration, the energy and efficiency of that ac-celeration, the location of the acceleration site, whether the particles can escape into theinterplanetary medium, and how they travel through it.

It is not the case that SEPs are detected at Earth following all large flares and fast CMEs(e.g. Klein et al., 2011). Solar events of this type, which might reasonably be expectedto produce SEPs at Earth but which do not, may be termed “false alarms”. Furthermore,some SEP events may follow smaller solar events, so that they are “missed events” for SEPforecasting algorithms based on intense flares and/or fast CMEs.

Many SEP forecasting tools base their prediction upon the observation of intense solarflares and/or radio bursts. For example, the proton prediction system proposed by Smart andShea (1989) makes a forecast based upon flare intensity and position. It produces almostequal numbers of correct forecasts, false alarms, and missed events (Kahler, Cliver, andLing, 2007).

The National Oceanic and Atmospheric Administration (NOAA) Space Weather Predic-tion Center (SPWC) uses a system named “Protons” which is described by Balch (1999).The tool aims to forecast the arrival of SEPs near Earth following the detection of solar flaresand radio bursts. Balch (2008) validated the system over a period between 1986 and 2004,and found that its false-alarm rate was 55%. The tool, however, is only used as a decisionaid, and the actual forecasts issued by SWPC have improved over time.1 Kahler and Ling(2015) combined SEP event statistics with real-time SEP observations to produce a forecastwhich changes dynamically.

Laurenza et al. (2009) developed the empirical model for solar proton events real-timealert (ESPERTA) method of SEP forecasting based upon flare size, flare location, and evi-dence of particle acceleration and escape. Their emphasis was to maximise the time betweenthe issue of an SEP event warning and the arrival of the particles, and their aim was to pro-duce an automated forecasting tool with a view to issuing warnings of SEP events withouthuman intervention. Whilst it is a significant improvement over the Protons tool, the false-alarm rate was, nevertheless, between 30% and 42% (Alberti et al., 2017). The forcastingsolar particle events and flares (FORSPEF) model, proposed by Papaioannou et al. (2015),

1http://www.swpc.noaa.gov/sites/default/files/images/u30/S1 Proton Events.pdf.

SEP Forecasting Algorithms and Associated False Alarms Page 3 of 27 173

aims to make forecasts of both flares and SEPs. Its SEP forecasting algorithm is based upona purely statistical approach and has not yet been validated.

Other forecasting tools use different methods. It has also been shown that type II radiobursts at decametric–hectometric (DH) wavelengths may be used to aid the forecasting ofSEP events. Winter and Ledbetter (2015) have described a statistical relationship betweenDH type II radio bursts, the properties of the associated type III burst, and peak proton flux.During the period they analysed (2010 to 2013), they were able to make predictions of anSEP event with a false-alarm rate of 22%.

The relativistic electron alert system for exploration (REleASE) SEP forecasting tool(Posner, 2007) relies upon the fact that electrons will travel faster than protons and willtherefore arrive at 1 AU first. A forecast of expected proton flux is made based upon thereal-time electron flux measurements.

Although the majority of currently operational data-based forecasting schemes make useof flare information, it is widely thought that the use of CME information would substan-tially improve algorithm performance. While from an operational point of view it is currentlynot trivial to obtain CME parameters in real time, it is important to compare the performanceof flare-based versus CME-based algorithms and determine whether a combination of flareand CME parameters within a forecasting tool may be beneficial.

Along with empirical forecasting algorithms which are based upon solar observations,several physics-based space weather forecasting tools have recently been developed, e.g. theSOlar Particle ENgineering COde (SOLPENCO) (Aran, Sanahuja, and Lario, 2006), a solarwind simulation including a cone model of CMEs (Luhmann et al., 2010), and the SolarParticle Radiation SWx (SPARX) model (Marsh et al., 2015).

A catalogue of 314 SEP events and their parent solar events between 1984 and 2013 hasbeen produced by Papaioannou et al. (2016). It is expected that this database will providea solid basis for the analysis of SEP events and the characteristics of their parent solarevent. The catalogue does not, however, include information on solar events which werefalse alarms. In order to improve SEP forecasting tools for space weather applications, ananalysis of the characteristics of false-alarm events should be carried out with a view togaining an understanding of why SEPs were not observed.

Some statistical studies of SEP events and false alarms have been undertaken. Most takethe same approach as Papaioannou et al. and Laurenza et al., starting by considering theSEP events and then looking for the possible parent solar events. Gopalswamy et al. (2014)examined solar events during the early part of Solar Cycle 24, and considered why somewhich had very fast CMEs and large flares did not produce ground-level enhancementsof energetic particles, as might have been expected. They suggested that poor latitudinalmagnetic connectivity between the solar event and Earth may have been an important factor.

Marqué, Posner, and Klein (2006) examined a small number of CMEs with a speedgreater than 900 km s−1 which had no radio signature of flare-related acceleration, and foundthat none produced conspicuous SEP events at Earth. These authors argued therefore that aCME shock without an associated flare is not sufficient to produce SEPs.

Wang and Zhang (2007) suggested that X-class flares not associated with any CME mayoccur closer to the magnetic centre of their source active region and may therefore be con-fined by overlying arcade magnetic fields. Klein, Trottet, and Klassen (2010) investigated asmall number of these “CME-less” flares further and argued that no SEP event might be ex-pected following a flare which shows high peak emission in soft X-rays but does not exhibitradio emission at decimetre and longer wavelengths.

Most of the large sample studies described above started by considering SEP events andthen looked for possible parent solar events. In this article, we take a different approach. We

173 Page 4 of 27 B. Swalwell et al.

start our analysis by considering solar events and determining whether an SEP event wasmeasured at Earth a short time thereafter. We focus on intense flares and fast CMEs anddefine two possible forecasting algorithms, the first based solely on the occurrence of an in-tense flare, and the second on that of a fast CME. The performance of the algorithms is quan-tified by evaluating them over historical datasets and the characteristics of the false alarmswhich were studied. In addition, missed events, i.e. SEP events which were not forecast, arealso identified and studied. Finally, we discuss how a new algorithm which combines flareand CME properties may be introduced to achieve a better performance.

We provide lists of false alarms based upon the forecasting algorithms in order that theymay form the basis of future studies and comparisons, together with a list of the solar eventswhich produced >40 MeV protons. We analyse the properties of the false-alarm events todetermine whether reasons can be identified that would explain why they did not produceSEPs at Earth.

2. False Alarms and Forecasting Algorithms

A false alarm may simply be defined as a solar event which is predicted by a forecastingalgorithm to produce SEPs at Earth, but which fails to do so. Specification of a forecast-ing algorithm and determination of its associated false alarms requires identification of thefollowing three points:

1. The criteria and observational data sets by which a solar event is assigned a high likeli-hood of producing SEPs at Earth. Typically, this will include identification of the typeof solar event (e.g. flare or CME) expected to produce SEPs, of a requirement on theintensity of the event (e.g. a flare with a peak SXR flux, fsxr, which exceeds a specifiedthreshold intensity, fthr, or a CME with a speed vCME which is faster than a thresholdspeed vthr), of a positional requirement (e.g. an event with a source region West of agiven longitude), and possibly of other parameters.

2. The criteria by which it is determined that an SEP event has occurred or not. Thesewill typically include specification of the instrument being used to measure particleflux intensity, of the species of particle examined and its energy range, and of the SEPintensity threshold, Ithr, used to establish whether an SEP event was detected followinga particular solar event.

3. The method by which the solar event is associated with the SEP event.

We discuss each of these requirements in Sections 2.1, 2.4, and 2.5, respectively.

2.1. Solar Event Parameters

As our source for CME data we have used the CME catalogue of the Co-ordinated DataAnalysis Workshop (CDAW)2 (Gopalswamy et al., 2009). This catalogue is produced man-ually, CMEs being identified visually from images obtained by the C2 and C3 coronagraphsof the Large Angle and Spectrometric Coronagraph Experiment (LASCO) (Brueckner et al.,1995) on board the Solar and Heliospheric Observatory (SOHO) spacecraft.

Information is published in the catalogue on various CME parameters including, interalia, the time it is first seen in the LASCO images, its width, and its position angle. CDAWpublishes three values for the speed of CMEs in its catalogue, each calculated by differentmeans: we use the first, the “linear” speed, which is obtained simply by fitting a straight line

2http://cdaw.gsfc.nasa.gov/CME_list/index.html.

SEP Forecasting Algorithms and Associated False Alarms Page 5 of 27 173

to the height–time measurements. Importantly, there is no information directly availablefrom the catalogue as to whether the CME is Earth directed, or from where on the solardisc it originated. This imposes serious limitations in analysing whether a particular CMEis likely to produce SEPs at Earth.

Solar flares are classified by their peak SXR emission as measured in the 1 – 8 Å channelof the Geostationary Observational Environmental Satellites (GOES) (Grubb, 1975) X-raySensor (XRS) instruments. Flares with a peak flux in this energy channel above 10−4 W m−2

are designated to be of class X; those with a peak flux between 10−5 and 10−4 W m−2 areof class M; and classes C, B, and A are defined in a similar fashion. No single instrumenthas been in continuous operation since 1975, although the design has changed little over theyears (Garcia, 1994).

As our source for solar flare data, we have used the GOES SXR Flare List, which hasbeen continuously maintained since 1975, and which may be downloaded from the website3

of the Heliophysics Integrated Observatory (Bentley et al., 2011).In addition to reporting the maximum SXR intensity and the time of the start, peak,

and end of the flare, the GOES SXR Flare List also usually reports its heliographic co-ordinates. However, there is a significant number of flares for which the list does not providethis information. In these cases, we have used values for co-ordinates from the followingsources:

1. Co-ordinates reported in the SolarSoft Latest Events Flares List (gevloc) (which mayalso be obtained through Helio).

2. The reported co-ordinates of the active region (AR) from which the flare originatedaccording to the GOES SXR flare list.

3. Making our own estimate of co-ordinates by watching movies of 195 Å images takenby the Extreme ultraviolet Imaging Telescope (EIT) on board the SOHO spacecraft orof 195 Å images taken by the Atmospheric Imaging Assembly (AIA) on board the SolarDynamics Observatory (SDO).

CMEs and solar flares, particularly high-energy events, often occur within a short timeof each other from the same solar active region. Making associations between these solarevents is required so as to gain an understanding of the type of event which did or did notproduce SEPs at Earth: it also allows an estimate to be made of the site of origin of the CMEfrom the reported heliographic coordinates of its associated flare.

We developed a method of making associations between CMEs and flares automatically,which we set out in Appendix A. Whilst we are confident that the method produces correctassociations in over 90% of cases, to be sure, we also viewed 195 Å (obtained by the EIT onboard SOHO) and 193 Å (obtained by the AIA on board SDO) movies of each solar event.We confirmed the associations made by the automatic method in 156 cases, changed themin six cases, and were unable to confirm the associations in a further 17 cases because EITor AIA images were not available.

2.2. Location Criterion for Solar Events

It is well known that solar events with origin in the West of the Sun as observed by an ob-server on Earth are more likely to produce SEPs than those originating in the East. Thereforeit is common to introduce a positional criterion within SEP forecasting algorithms. Figure 1shows the heliographic longitude of the 171 SEP-producing events between 1 April 1980and 31 March 2013 for which we were able to determine coordinates. Of these, 86.5%

3http://www.helio-vo.eu/.

173 Page 6 of 27 B. Swalwell et al.

Figure 1 Heliographic longitude and latitude of solar events which produced an SEP event according to thecriteria defined in Section 2.4 between 1 April 1980 and 31 March 2013.

(148/171) had their origin in a solar event which occurred at a site West of E20, hence ourchoice of positional requirement in the forecasting algorithms. We call solar events whichhave their origin West of E20 “western events”.

2.3. The Forecasting Algorithms

The two forecasting algorithms we investigate in this work are based upon the fact that themore energetic the solar event, the greater the likelihood of that event producing SEPs atEarth, particularly if it was magnetically well connected (e.g. Dierckxsens et al., 2015). Thealgorithms are:

A.1 A frontside CME with a reported speed of 1500 km s−1 or greater (a “fast” CME)occurring West of E20 on the solar disc will result in an SEP event being detected atEarth.

A.2 An X-class flare occurring West of E20 on the solar disc will result in an SEP eventbeing detected at Earth.

We evaluate both the forecasting algorithms over the time range from 11 January 1996until 31 March 2013 (“time range 1”); for algorithm A.2 we are also able to examine alonger period, between 1 April 1980 and 31 March 2013 (“time range 2”). In time range 1,there were 143 fast CMEs (according to our definition set out in A.1) reported by CDAWand 140 X-class flares. In time range 2, there were 403 X-class flares.

Table 1 sets out the numbers of solar events which we have examined in this study.A number of solar events have had to be excluded from our analysis because of data gaps,the saturation of detectors, or another cause, or because it was not possible to determine theheliographic co-ordinates.

2.4. SEP Event Parameters

The definition of an SEP event typically includes a specification of the instrument beingused to measure the particle flux, of the species of particle which is examined and its energy

SEP Forecasting Algorithms and Associated False Alarms Page 7 of 27 173

Table 1 Numbers of solar events we studied. Column 1 shows the time range over which data have beenanalysed, column 2 the type of solar event considered, column 3 the total number of solar events within theperiod investigated, column 4 the number of events for which we were able to determine coordinates (afterremoval of events which were discarded because of data gaps, saturation of detectors, or other reasons), andcolumn 5 the number of events which occurred West of E20.

Time range Event type Total numberof events

Events for which coordinateswere determined

Analysed eventsWest of E20

Time range 1(Jan. 1996 to Mar. 2013)

Fast CMEs 143 93 52X-class flares 140 139 79

Time range 2(Apr. 1980 to Mar. 2013)

X-class flares 403 377 197

range, and of the SEP intensity threshold, Ithr, which is used to establish whether an SEPevent was detected following a particular solar event.

Particles accelerated by solar events include electrons, protons, and heavier ions, butwe have chosen to analyse high-energy (>40 MeV) protons. The threshold considered isslightly higher than the >10 MeV threshold used by NOAA, making our event list lessbiased towards interplanetary shock-accelerated events. This choice also avoids proton en-hancements caused by magnetospheric effects.

Because our threshold energy for protons is higher than that used by NOAA, we com-pared peak >40 MeV fluxes for our event sample with the peak >10 MeV fluxes for thesame events. For each of our events a value for >10 MeV flux was obtained from theNOAA SEP list.4. Eleven of the SEP events at >40 MeV did not reach the NOAA thresholdof 10 pfu at >10 MeV, and for these we estimated the peak flux by visual analysis of theplots of each event.5 Figure 2 is a plot of the peak flux of >10 MeV protons plotted againstthe peak proton flux in the ∼40 – 80 MeV energy channel of the GOES EPS instruments forthe SEP events in time range 1. The dotted horizontal line is at the NOAA threshold of tenparticles cm−2 s−1 sr−1 (pfu). The highest value for the maximum peak flux at >40 MeV intime range 1 was approximately 100 pfu – the same event at >10 MeV produced 31700 pfuaccording to NOAA.

All instruments which detect proton intensities are subject to slight fluctuations, and notall of these can properly be said to be SEP events. The definition of the intensity threshold,Ithr, must be high enough so as to exclude the normal fluctuations in measurements, but lowenough to ensure that rises which are genuinely due to solar events are included. We set Ithr

to be a 2.5-fold increase in proton intensity over the quiet-time background level.For this study we have used GOES SEP data because they allow us to study SEP events

over a time period of more than 30 years. No one instrument has been in continuous oper-ation during that time, and so we have had to use data from a number of different GOESsatellites. Table 2 sets out which spacecraft we have used and the energy channel we consid-ered to establish the occurrence of an SEP event. There are slight differences in the energychannels, particularly in the case of GOES 2, but we take the view that the differences are sosmall as to have a negligible effect upon our results. We downloaded data from the EuropeanSpace Agency’s Solar Energetic Particle Environment Monitor (SEPEM) website (Crosbyet al., 2010).6 Data from 1 April 1987 onwards had been cleaned and intercalibrated by theSEPEM team; prior to that date, we used their raw data.

4ftp://ftp.swpc.noaa.gov/pub/indices/SPE.txt.5Downloaded from https://solarmonitor.org/.6http://dev.sepem.oma.be/.

173 Page 8 of 27 B. Swalwell et al.

Figure 2 Peak flux of>10 MeV protons as reported byNOAA plotted against the peakproton flux in the GOES∼40 – 80 MeV energy channel intime range 1. The dottedhorizontal line is at the NOAAthreshold of 10 pfu.

Table 2 Instruments used to obtain data on proton intensity, the dates between which data from that instru-ment were used, and the energy channels which were analysed. Column 1 gives the name of the spacecraftfrom which the data were taken, column 2 the date from which we began to use these data, and column 3the date when we ceased using these data. Column 4 shows the range of proton energies measured by theinstrument we used, and column 5 whether the data were raw or had been cleaned by the SEPEM team.

Spacecraft Start date End date Energy channel (MeV) Raw data/Cleaned

GOES 2 1 April 1980 31 December 1983 36.0 – 500.0 Raw dataGOES 6 1 January 1984 31 March 1987 39.0 – 82.0 Raw dataGOES 7 1 April 1987 28 February 1995 39.0 – 82.0 CleanedGOES 8 1 March 1995 7 January 2003 40.0 – 80.0 CleanedGOES 12 8 January 2003 31 December 2009 40.0 – 80.0 CleanedGOES 11 1 January 2010 31 December 2010 40.0 – 80.0 CleanedGOES 13 1 January 2011 31 March 2013 38.0 – 82.0 Cleaned

It is not always easy to determine whether an SEP event had occurred if the instrumentwere still recording high-energy protons from a previous event. When the intensity levelhad not returned to within 2.5 times the quiet-time background level by the time of the startof the solar event we investigated, that solar event was disregarded – we were unable todetermine whether that event produced SEPs at Earth. The only exceptions were those caseswhere there was a clear increase in proton intensity which could only be attributed to thesolar event in question, in which case it was treated as an SEP event.

We determined that during time range 2, there had been 221 flux enhancements in theGOES >40 MeV proton channel which satisfied our definition of an SEP event.

2.5. Association of Solar Events and SEP Events

A criterion for associating solar events and SEP enhancements is necessary. First we tookthe start time of the solar event. For CMEs not associated with a flare, we used the time atwhich the CME was first reported in the CDAW catalogue; for CMEs which were associatedwith a flare and for all flares, we used the reported start time of the flare.

SEP Forecasting Algorithms and Associated False Alarms Page 9 of 27 173

We then searched the GOES proton data for a subsequent SEP event. In most cases,the SEP enhancement began before another solar event was reported, in which case theassociation between the solar event and the SEP enhancement was made. In some instances,however, another solar event was reported before the SEP enhancement commenced. Forthese cases, it was assumed that this new solar event accelerated the particles, unless thatevent was so close in time to the arrival of the SEPs (∼20 minutes) that it was unlikelythat the new event could have been the cause. None of our confirmed solar event – SEPassociation time differences was as short as 20 minutes.

A number of solar events had to be discarded because they coincided with gaps in SEPdata, meaning that we were unable to determine whether they had produced an SEP enhance-ment. However, if there had been short outages (∼3 hours), and there was no evidence ofan SEP event either side of the outage, the solar event was counted as a false alarm.

We also associated solar events with all of the 221 proton events we identified. In somecases, the associated flare was of a class smaller than X and/or the associated CME was nota fast one according to our definition. Of these 221 events, we were not able to determineco-ordinates of the parent solar event for 50. The event was a western one in 148 of theremaining 171 cases.

3. Identification of False Alarms and Evaluation of the ForecastingAlgorithms

We applied the forecasting algorithms described in Section 2.3 to the historical datasets wecollected. We evaluated both algorithms over time range 1 (1996 to 2013), and in addition,we evaluated algorithm A.2 over the longer time range 2 (1980 to 2013).

3.1. Algorithms A.1 and A.2 over Time Range 1

Figure 3 shows the results of applying the two SEP forecasting algorithms to the data setfor time range 1. The number of correctly forecast SEP events is shown by the blue bar andnamed α; the number of false alarms is represented by the red bar and named β; and the

Figure 3 Numbers of correctlyforecast SEP events, false alarms,and SEP events which were notforecast for the two forecastingalgorithms during time range 1.

173 Page 10 of 27 B. Swalwell et al.

number of SEP events which occurred but were not forecast by the algorithm (the “missedevents”) is shown as the green bar and named γ . There was a total of 107 SEP events in timerange 1. Of the 86 SEP events for which we were able to determine the coordinates of theparent solar event, 91.9% (79/86) were western events.

Algorithm A.1 considers western fast CMEs. There were 52 such events during the periodin question, and 71.2% (37/52) produced SEPs at Earth. Thus the false-alarm rate was 28.8%(15/52), but the algorithm failed to forecast 53.2% (42/79) of the SEP events for which theparent solar event was a western one. Of all the SEP events for which coordinates could bedetermined, it missed 57.0% (49/86).

Algorithm A.2 uses western X-class flares as the basis for the forecast. There were 79such flares in time range 1, and 49.4% (39/79) produced SEPs at Earth. The false-alarm ratewas therefore 50.6% (40/79), and the algorithm failed to forecast 50.6% (40/79) of the SEPevents for which the parent solar event was a western one. Of all the SEP events for whichcoordinates could be determined, it missed 54.7% (47/86).

Table 5 in Appendix B provides the list of false alarms for algorithm A.1 and Table 6 inAppendix C contains the false alarms for A.2; the same lists are available electronically assupplementary material.

As well as reaching for an understanding of the underlying physical differences betweenthose solar events which produced SEPs at Earth and the false alarms, we also aim to mea-sure the efficacy of the forecasting algorithms. A high percentage of correctly forecast SEPevents (α) coupled with a low number of false alarms (β) is desirable, but not at the expenseof failing to forecast a large number of the SEP events which did occur (γ ). We used tworatios in our evaluation:

1. The “false-alarm ratio” (FAR) gives the fraction of forecast events which actually didoccur. It is defined as

FAR = β

α + β. (1)

The FAR is sensitive to the number of false alarms, but takes no account of missedevents. Possible scores range from 0 to 1, with the “perfect” score being 0.

2. The “critical success index” (CSI) is a measure of how well the forecast events corre-spond to the observed events. It is defined as

CSI = α

α + β + γ. (2)

Possible scores range from 0 to 1, with the “perfect” score being 1.

3.2. Forecasting Algorithm A.1: Fast CMEs

All the CMEs in our sample were from the front-side of the Sun and had an associated flarewhich was used to determine the coordinates. The FAR for algorithm A.1 is 0.29 and theCSI, not taking account of the missed eastern events, is 0.39. If the eastern events wereto be included within the calculation for the CSI, its value would be reduced to 0.37. Theevaluation scores for this algorithm over time range 1, and for algorithm A.2 over both timeranges, are summarised in Table 3. It is not clear whether the high number of missed eventsis due to the fact that the measured velocity of the CME, vCME, is the plane-of-the-sky speed,if in general the speeds measured by examination of coronagraph images are not sufficientlyaccurate, or if more physics need to be included in the analysis.

In Figure 4 we plot the peak SXR intensity of the CME associated flare against its speedfor those solar events in time range 1 which produced SEPs at Earth (top left, blue circles);

SEP Forecasting Algorithms and Associated False Alarms Page 11 of 27 173

Table 3 Summary of the evaluation scores for the two forecasting algorithms: the “false-alarm ratio” (FAR)and the “critical success index” (CSI) over time range 1. Algorithm A.2 is also evaluated over time range 2.Column 1 shows the forecasting algorithm being considered, column 2 the time range over which the analysiswas made, column 3 lists the FAR for that algorithm, column 4 the critical success index (CSI) not taking intoaccount the missed eastern events, and column 5 the CSI were these additional missed events to be included.

Forecasting algorithm Time range FAR CSI not including missedeastern events

CSI including missedeastern events

A.1 (Fast CMEs) 1 0.29 0.39 0.37A.2 (X-class flares) 1 0.51 0.33 0.31A.2 (X-class flares) 2 0.61 0.29 0.26

Figure 4 Flare class versus associated CME speed for the solar events which produced SEPs >40 MeV atEarth in time range 1 (top left, blue circles); for fast CMEs which were false alarms according to forecastingalgorithm A.1 (top right, red squares); for SEP events missed by algorithm A.1 (bottom left, green diamonds);and for all events together (bottom right).

for those events in the same period which were false alarms according to algorithm A.1 (topright, red squares); for SEP events missed by algorithm A.1 (bottom left, green diamonds);and for all events together (bottom right). This shows that many of the fast CME false alarms

173 Page 12 of 27 B. Swalwell et al.

Figure 5 Plots of �δ against time for algorithm A.1, together with histograms of �δ. The top plots presentthe results for the solar events which were correctly forecast to produce SEPs at Earth (shown in blue); thebottom plots show the false alarms (shown in red).

occur close to the threshold speed, vthr, which means that increasing the threshold would re-duce the number of false alarms, although it would also increase the number of missedevents. A significant fraction of SEP events were associated with CMEs of a reported speedmuch slower than 1500 km s−1. It is also clear that many of the false alarms have a flareintensity <M3.

Gopalswamy et al. (2014) studied major solar eruptions during the first 62 months of So-lar Cycle 24 and suggested that among other parameters, the separation in latitude betweenthe flare and the footpoint to Earth may be an important factor in determining whetherhigh-energy particle events are detected. Therefore we define a parameter, �δ, the differ-ence between the latitude of the flare, δflare, and the latitude of the Earth’s footpoint, δEarth,i.e. the parameter �δ takes into account the inclination of Earth’s orbit. In Figure 5 weplot �δ against time for algorithm A.1, together with histograms for �δ. The events cor-rectly forecast to produce SEPs are presented in the top plots (shown in blue), and the falsealarms in the bottom plots (shown in red). Of the fast CMEs which had their origin within±10 degrees of the Earth’s footpoint, 64.7% (11/17) produced SEPs; of those which hadtheir origin outside this range, 74.3% (26/35) produced SEPs. Overall, there does not appearto be a significant difference between the distribution in �δ for SEP events and false alarms.

Figure 6 shows histograms of the heliographic longitude of solar events in time range 1correctly forecast by algorithm A.1 to produce an SEP event (top left), of algorithm A.1false alarms (top right), of SEP events missed by algorithm A.1 (bottom left), and of all SEPevents (bottom right). There is a peak of SEP-producing fast CMEs between W50 and W90.The false alarms for algorithm A.1 are relatively evenly distributed, as are the SEP eventsnot forecast by A.1.

In Figure 7 we plot �δ against the longitude of the 37 western fast CMEs which producedan SEP event in time range 1. The size of the marker reflects the peak SXR intensity of the

SEP Forecasting Algorithms and Associated False Alarms Page 13 of 27 173

Figure 6 Histograms of the heliographic longitude of solar events in time range 1 of algorithm A.1 SEPevents (top left); of algorithm A.1 false alarms (top right); of SEP events missed by algorithm A.1 (bottomleft); and of all SEP events (bottom right).

associated flare, and its colour is representative of the width of the CME. The bottom plotgives the same information, but for the false alarms according to algorithm A.1. On average,the size of the markers in the middle plot is smaller than the size of those in for the SEP-producing events. Thus, the peak SXR intensity of a flare associated with a fast CME isrelevant to the question as to whether SEPs will arrive at Earth.

Also apparent from Figure 7 is that the CME width is an important parameter. Of the 37SEP-producing CMEs, 86.5% (32/37) were reported to be haloes by the CDAW catalogue.By contrast, for the false alarms of algorithm A.1, only 46.7% (7/15) were haloes. Thereforewe find that halo CMEs are more likely to produce SEPs than non-haloes. This result isconsistent with the findings of Park, Moon, and Gopalswamy (2012), who found that solarevents which had the highest probability of producing 10 MeV protons were full-halo CMEswith a speed exceeding 1500 km s−1.

Kwon, Zhang, and Vourlidas (2015) examined 62 halo CMEs (as reported by the CDAWcatalogue) which occurred between 2010 and 2012 and were observed by three spacecraftseparated in longitude by nearly 180◦. They found that 42 were observed to be haloes byall three spacecraft. They concluded that a CME may appear to be a halo as a result offast magnetosonic waves or shocks, and that apparent width does not represent an accuratemeasure of CME ejecta size.

3.3. Forecasting Algorithm A.2: X-Class Flares

Algorithm A.2 has an FAR of 0.51. Whilst it makes almost exactly the same number of cor-rect forecasts as algorithm A.1, the percentage of correct forecasts is lower. The proportionof missed SEP events is also relatively high, leading to a CSI of 0.33 without accounting forthe missed eastern events, or of 0.31 if the missed eastern events were to be included.

173 Page 14 of 27 B. Swalwell et al.

Figure 7 �δ versus heliographic longitude for the western fast CMEs which produced SEPs at Earth in timerange 1 (top plot); and for those which were false alarms according to algorithm A.1 (bottom plot). The sizeof the marker represents the peak SXR intensity of the flare: for example, the point at S20W95 in the top plotwas an M1.8 flare, whereas the point at S21E08 in the same plot was an X17.2 flare. The colour of the markerrepresents the CME width.

In Figure 8 we plot the SXR intensity for the solar flares above the threshold of A.2against associated CME speed, and for SEP events missed by algorithm A.2 in the sameformat as in Figure 4. There is some symmetry with Figure 4 in that many of the falsealarms fall close to the chosen threshold. Not all events above the A.2 threshold have anassociated CME. Of the 122 X-class flares which occurred in time range 1 (and which didnot coincide with a LASCO data gap), 14.8% (18/122) had no associated CME. However,the percentage of A.2 false alarms which did not coincide with a LASCO data gap and whichdid not have an associated CME is 26.5% (9/34).

In Figure 9 we show histograms of the heliographic longitude of solar events in timerange 1 for algorithm A.2 in the same format as Figure 6. There appears to be no significantdifference in the longitudinal distribution of western X-class flares which produced an SEPevent and those which were false alarms, but in this case, the SEP events which were notforecast by algorithm A.2 do have a clear peak between W20 and W80.

In the top plot of Figure 10 we show �δ against the longitude of the 39 western X-classflares which produced an SEP event in time range 1. As in Figure 7, the colour of the markeris representative of the width of the flare’s associated CME as reported by CDAW, but in thecase of Figure 10, the size of the marker reflects the duration of the flare itself. The bottomplot gives the same information, but for the false alarms according to algorithm A.2.

SEP Forecasting Algorithms and Associated False Alarms Page 15 of 27 173

Figure 8 Flare class versus associated CME speed for the solar events which produced SEPs >40 MeVat Earth in time range 1 (top left, blue circles); for X-class flares which were false alarms according toforecasting algorithm A.2 (top right, red squares); for SEP events missed by algorithm A.2 (bottom left,green diamonds); and for all events together (bottom right).

X-class flares which were false alarms tended to be shorter than those which producedSEPs. The average flare duration for the SEP-producing X-class flares was 46.3 minutes,and 25.6% were longer than 60 minutes (“long-duration flares”). For the false alarms, theaverage flare duration was 24.9 minutes, and only 5.0% (2/40) were long-duration flares.It has previously been shown that there is an association between long-duration flares andCMEs (Yashiro et al., 2006), therefore the trend with duration may be connected with thefact that large flares without CMEs are more likely to be false alarms.

In this case, the width of the associated CME is also an important parameter. Of the 39western X-class flares which produced SEPs at Earth, we were able definitively to associate37 with a CME (the other two occurring during times when LASCO did not produce anydata). Of those 37, 86.5% (32/37) were halo CMEs. In contrast, for the false alarms, wewere able to confirm associations with CMEs in 25 cases. Of these 25, only 44.0% (11/25)were haloes.

3.4. Algorithm A.2 over Time Range 2

Over the longer period of time range 2, we analysed 197 western X-class flares, and 39.1%(77/197) produced SEPs at Earth. The false-alarm rate was thus 60.9% (120/197), and the

173 Page 16 of 27 B. Swalwell et al.

Figure 9 Histograms of the heliographic longitude of solar events in time range 1 of algorithm A.2 SEPevents (top left); of algorithm A.2 false alarms (top right); of SEP events missed by algorithm A.2 (bottomleft); and of all SEP events (bottom right).

algorithm failed to forecast 47.8% (71/148) of SEP events. Of all the SEP events for whichcoordinates could be determined, it missed 55.0% (94/171). Therefore the FAR was 0.61,and the CSI was 0.29 without the missed eastern events and 0.26 with them. The FAR ishigher for this longer time period than that for time range 1. Table 7 in Appendix D providesthe list of false alarms for algorithm A.2 over time range 2.

In Figure 11 we plot �δ against date for this longer time period together with histogramsfor �δ. In the left-hand plots the duration of the flare is denoted by the size of the marker.Figure 11 shows a significant difference in the �δ distribution for events which producedSEPs and false alarms. For the former, the distribution is rather flat, whereas for the latter,very many events are characterised by large �δ.

A significantly higher number of false alarms originated in the southern solar hemisphereduring Solar Cycle 22 (taken to be 1 January 1987 until 31 December 1995; 80% or 40/50)than from the north (20% or 10/50). Furthermore, in Solar Cycle 24 (taken to be from 1January 2010 onwards), only two western X-class flares were false alarms.

Moreover, X-class flares between 1980 and 1995 were on average longer than those post1995. Table 2 shows that we have taken data from GOES 7 and its predecessors for datesbefore 1 March 1995, and from GOES 8 and its successors after that date. We are not awareof any reason why a change of instrument should produce such a result, nor are we aware ofany change in the way the flare duration has been measured.

4. Improvement of the Forecasting Algorithms

We examined ways in which the performance of the forecasting algorithms might be im-proved. We note in particular

SEP Forecasting Algorithms and Associated False Alarms Page 17 of 27 173

Figure 10 �δ versus heliographic longitude for the western X-class flares which produced SEPs at Earth intime range 1 (top plot); and for those which were false alarms according to algorithm A.2 (bottom plot). Thesize of the marker represents the relative duration of the flare: for example, the flare marked at S18W33 inthe top plot had a duration of 10 minutes, whereas the flare at S03W38 in the same plot lasted 120 minutes.The colour of the marker represents CME width.

1. that algorithm A.1 produced the lowest number of false alarms, and that many of thesehad an associated flare intensity <M3; and

2. that X-class flares without an associated CME, or associated with a CME of speed lessthan 500 km s−1, did not produce SEPs.

We therefore define a third forecasting algorithm as follows:

A.3 A front-side CME with a reported speed of 1500 km s−1 or greater occurring West ofE20 on the solar disc which is associated with a flare of class M3 or greater, or a solarflare of class X or greater which occurs West of E 20 on the solar disc and is associatedwith a CME of speed greater than 500 km s−1 will result in an SEP event being detectedat Earth.

There were 71 such events in time range 1 and 70.4% (50/71) produced SEPs at Earth.For this algorithm, we have had to discard five of the SEP events which occurred duringa time when there were no data from the LASCO coronagraph. Thus the false-alarm ratewas 29.6% (21/71) and the algorithm missed 32.4% (24/74) of the SEP events for which theparent solar event was a western one, or 38.3% (31/81) of all SEP events. The false-alarmratio is thus comparable to that produced by algorithm A.1, but A.3 misses far fewer SEPevents, and consequently, the CSI is significantly higher at 0.53, not including the missed

173 Page 18 of 27 B. Swalwell et al.

Figure 11 Plots of �δ against time for algorithm A.2 over time range 2, together with histograms of �δ.The top plots present the results for the solar events which were correctly forecast to produce SEPs at Earth(shown in blue); the bottom plots the false alarms (shown in red). The size of the marker in the left-hand plotsis representative of the duration of the flare: for example, the flare in October 1989 shown at S35 in the topplot lasted 8 hours 48 minutes, whereas the flare in November 1998 shown at S29 in the same plot lasted19 minutes.

Table 4 Summary of the evaluation scores for algorithm A.3 in the same format as Table 3.

Forecasting algorithm Time range FAR CSI not including missedeastern events

CSI including missedeastern events

A.3 1 0.30 0.53 0.49

eastern events, or 0.49 were they to be included. The result is summarised in Table 4. Wealso show the result graphically in Figure 12, which is in the same format as Figure 3. Itmay be possible to formulate better forecasting algorithms, but we suggest that increasedforecasting accuracy will only come if the properties of both flares and CMEs are taken intoaccount.

5. Summary and Conclusions

We have used historical datasets in order to assess the efficacy of two simple SEP forecastingalgorithms which were based upon the occurrence of magnetically well-connected energeticsolar events: western fast CMEs and X-class flares. We used in our definition of SEP eventa threshold value for a proton energy of >40 MeV.

An algorithm purely based on the detection of a fast CME (A.1) performs reasonably wellin terms of false alarms (with a false-alarm ratio of 28.8%), but misses a significant fractionof actual SEP events (53.1%). It is unclear whether this is due to experimental limitations inthe determination of the CME speed, or if there are other physical properties which would

SEP Forecasting Algorithms and Associated False Alarms Page 19 of 27 173

Figure 12 Number of correctly forecast SEP events, false alarms, and SEP events which were not forecastfor the three forecasting algorithms during time range 1.

need to be measured and included in the algorithm to assess the SEP producing potential ofa CME more accurately. False alarms for this type of algorithm tend to be associated withflares of magnitude smaller than M3. There does not seem to be any positional trend in thesource location of the false alarms.

An algorithm purely based on the detection of an intense flare (A.2) correctly forecasts al-most the same number of SEP events as A.1, but has a much higher false-alarm rate (50.6%).Like A.1, it misses a significant fraction of SEP events (also 50.6%). We found that falsealarms for this algorithm tend to be flare events of shorter duration, compared to those thatdid produce SEPs. Of these false alarms, 37% were not associated with a CME. An earlierstudy has analysed confined flares (CME-less flares) and emphasised that this type of eventtends not to produce SEPs (Klein, Trottet, and Klassen, 2010). In terms of their longitudinallocation, A.2 false-alarm events were quite uniformly distributed. We also determined thatSEP events not forecast by algorithm A.2 were preferentially located in the well-connectedregion (between W20 and W80), suggesting that for this region, a lower flare magnitudethreshold may need to be used.

When evaluated over a longer time range which includes Solar Cycle 21 (time range 2),algorithm A.2 performs less well than over time range 1. This may be due to instrumentaleffects associated with different GOES detectors being employed at different times, or itmay be a real physical effect. We found that there is a systematic trend for flare durations tobe longer in Cycle 22 than in Cycle 23, and this may be an instrumental effect.

It has previously been suggested that the latitudinal separation, �δ, between the flarelocation and the footpoint of the observing spacecraft plays a role in whether high-energyparticles are detected (Gopalswamy et al., 2014). In our analysis, carried out over a widertime range, we found that false alarms for algorithm A.2 tended to be associated with a largelatitudinal separation �δ, whilst this was not the case for algorithm A.1.

We defined a new forecasting algorithm, A.3, based upon the parameters of both flaresand CMEs. This algorithm performed better than the algorithms based solely upon one typeof solar event: it correctly forecast 70.4% of SEP events during time range 1 and thus had afalse-alarm rate comparable to that of algorithm A.1 (29.6%). It also missed far fewer SEP

173 Page 20 of 27 B. Swalwell et al.

events (32.4%, or 38.3% if eastern events were to be included) than both algorithms A.1and A.2.

In test particle simulations it has been shown that SEPs may exhibit significant cross-fielddrift velocities depending on the configuration of the interplanetary magnetic field (Dallaet al., 2013; Marsh et al., 2013). Future work will assess whether the specific polarity of themagnetic field may influence the detection or non-detection of SEPs at a given location.

We have made available, in electronic form as supplementary material, lists of the>40 MeV proton false alarms according to each of the algorithms we analysed, togetherwith a list of the solar events which produced the >40 MeV SEP events. We hope that theselists can be used as the basis for further studies and comparisons.

Acknowledgements The CDAW catalogue is generated and maintained at the CDAW Data Center byNASA and The Catholic University of America in cooperation with the Naval Research Laboratory. Thecatalogue may be accessed at http://cdaw.gsfc.nasa.gov/CME_list/index.html. The GOES SXR Flare List canbe found at the Heliophysics Integrated Observatory website, http://hfe.helio-vo.eu, and we also used data ofheliographic coordinates from the SolarSoft Latest Events Flares List (gevloc) which can be obtained from thesame site. Our proton intensity data were downloaded from the SEPEM website, http://dev.sepem.oma.be.We are very grateful to all those who maintain these data repositories, as this work could not have beencompleted without them.

We have also used videos taken by the EIT instrument on board the SOHO spacecraft and the AIAinstrument on board the SDO spacecraft. SOHO is a project of international cooperation between ESA andNASA. We are grateful to them, and to the AIA, EVE, and HMI science teams.

Thanks also go to the anonymous referee for their helpful comments. S. Dalla acknowledges support fromthe UK Science and Technology Facilities Council (STFC, grant ST/M00760X/1).

Disclosure of Potential Conflicts of Interest The authors declare that they have no conflicts of interest.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 Inter-national License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution,and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source,provide a link to the Creative Commons license, and indicate if changes were made.

Appendix A: Association of Solar Flares and CMEs

It has long been accepted that solar flares and CMEs, particularly energetic events, oftenoccur within a short time of each other from the same solar active region, but making as-sociations between them is no trivial exercise. There is no standard approach: for example,Reinard and Andrews (2006) associated a flare with a CME if the CME occurred within atwo-hour window centred on the time of the peak of the flare; others made associations us-ing both temporal and spatial criteria (Vršnak, Sudar, and Ruždjak, 2005; Dumbovic et al.,2015). Below we describe a method of making associations between CMEs and flares auto-matically, and evaluate its accuracy.

In the light of the connection between high-energy eruptive events and SEPs, we decidedto look for associations involving CMEs reported by CDAW to have a speed of 1000 km s−1

or faster (“rapid CMEs”), and flares reported in the GOES SXR list to be of class M5 orgreater (“intense flares”). We examined all such events between 1 July 2011 and 31 August2012, this period being chosen solely because it provided a dataset which was small enoughto allow individual observation of each event, yet large enough to allow wider conclusionsto be drawn.

There were 55 rapid CMEs and 32 intense flares reported in the 13-month period underinvestigation. Of these, we did not study further 3 of the rapid CMEs and 1 of the intenseflares because they coincided with data gaps. Hence there were 83 events which formed thebasis of our study of flare–CME associations.

SEP Forecasting Algorithms and Associated False Alarms Page 21 of 27 173

In order to set a benchmark against which any automated method of associating CMEsand flares could be judged, we needed to know unambiguously whether any of the 83 ener-getic events were associated with another solar event. Consequently, we watched movies at193 Å of each one of these events, each movie having been created from data obtained bythe Atmospheric Imaging Assembly (AIA) on board the Solar Dynamics Observatory (SDO)spacecraft.

For each of the intense flares, identification was made visually from the AIA/SDOmovies. We looked for increases in intensity on the solar surface at the time of the flarespecified by the GOES SXR list, but when the site of the flare was not obvious, we acceptedthe reported coordinates. Whilst watching the movies of the intense flares, we also searchedfor evidence of an associated CME. If we were able to see any ejected material, any loopdistortion, or any coronal dimming consistent with the flare site within one hour either sideof the reported time of the flare, we associated that flare with a CME (regardless of whetherthis was a rapid CME).

Rapid CMEs were identified by searching visually for evidence of any ejected material,any loop distortion, or any coronal dimming at the time reported by CDAW. If such evidencewas present (and was consistent with a front-side event), the CME was regarded as havingoccurred on the face of the disc; if there was no such evidence, the CME was regarded asa back-side event. Associations were made between a rapid CME and a flare (regardless ofwhether this was an intense flare) if the reported time of the CME (i.e. the time the CMEwas first seen in the LASCO C2 images) fell between one hour before the reported start ofthe flare and one hour after its reported end, and the evidence of the CME was consistentwith the flare site.

As a result of making the associations manually, we found that 35 of the 52 fast CMEswere on the face of the disc. This proportion is slightly higher than might have been expected(given that we can only see one side of the Sun at any one time, we might expect that onlyhalf of the CMEs we see would be from the face of the disc), but can be explained by twofactors: first, there were large numbers of CMEs from the same active regions (two activeregions produced five each, and one other eight), and this may slightly distort the figures;secondly, 17 of the 52 events were reported to occur very close to the limb, meaning that wemay have seen a CME which originated from just behind the limb.

Of the 35 rapid CMEs which occurred on the face of the disc, all were associated witha flare of some kind; 46% (16/35) were associated with an intense flare. Of the 31 intenseflares, 84% (26/31) were associated with a CME.

In every instance where we had associated a solar flare with a CME, the flare was reportedin the GOES SXR list as having commenced before the CME was first reported in theCDAW catalogue. It should be noted that this is not an indication of actual chronology –as an example of where there is evidence of a CME lifting off before its associated flare,see Harrison and Bewsher (2007) – but it is of significance when devising a method ofautomatically making associations between flares and CMEs.

CDAW reports the time of a CME as being when it is first seen in images produced bythe LASCO C2 coronagraph. This instrument, however, has a field of view between abouttwo and six solar radii (as measured from the Sun’s centre), and the images used by CDAWhave a cadence of, at best, 12 minutes and sometimes much longer. The combination ofthese factors means that the reported time of the CME may be many minutes after is actual“lift-off” time, to.

Any attempt to make an estimate of to faces a number of difficulties: there is no informa-tion as to the height of the CME when it was first ejected; no information as to whether ithas accelerated or decelerated before its first appearance in the C2 images; and no informa-tion as to the direction of the CME. Nevertheless, finding a first approximation of to is more

173 Page 22 of 27 B. Swalwell et al.

likely to result in accurate associations between CMEs and flares than using the time of theCME as reported by CDAW.

We make the simple assumptions that by the time the CME reaches the field of view ofthe C2 coronagraph, it has travelled (at least) one solar radius and has undergone neithersignificant acceleration nor deceleration. An estimate for to is then obtained by using thereported speed of the CME.

In order to take into account of the difficulties caused by the cadence of the images, wedefine �t as a number of minutes both before and after a flare. For example, if we take�t = 12, we compare to with a time window opening 12 minutes before the flare began andclosing 12 minutes after it ended. Plainly, the greater �t , the more likely it is that to will fallwithin the window, and hence the greater the likelihood of false associations being made.

A good correlation could be found between those flare–CME associations which hadbeen made manually and those using a value of �t of just 30 minutes. We did investigatewhether it may be possible to improve the accuracy of the method by imposing a spatialcriterion, for example by requiring the position angle of the CME to agree with the latitudeand longitude of the flare to within a particular number of degrees. We found that the overallaccuracy was not improved by the imposition of such a criterion.

There will, of course, always be a small number of (usually) false associations whenusing this automatic method given that occasionally apparently unconnected solar eventssometimes occur almost simultaneously. Nevertheless, in our sample the method correctlyidentified 98% (60/61) associations and correctly identified 86% (19/22) non-associations,an overall success rate in 95% (79/83) of cases.

Appendix B: Algorithm 1: False Alarms in Time Range 1

Table 5 List of fast CMEs between 1 January 1996 and 31 March 2013 which were false alarms. Column 1gives the time the CME was first reported, column 2 its speed, and column 3 its acceleration all as reportedby CDAW. Column 4 is the position angle at which the height–time measurements had been made (called byCDAW the “measurement position angle”). Column 5 is the CME width. Columns 6 to 10 give the parametersof the associated flare: its start time, heliographic latitude, heliographic longitude, class, and duration.

First reported CME parameters Time start Flare parametersV(km s−1)

Accel.(km s−2)

Mpa(degrees)

Width(degrees)

Lat Lon Class Duration(hrs:mins)

1998-11-24T02:30 1798 −12.5 225 360 1998-11-24T02:07 −20 94 X1.0 00:302000-01-06T07:31 1813 10.7 342 67 2000-01-06T06:45 24 35 C5.8 00:212000-05-05T15:50 1594 −103.4 265 360 2000-05-05T15:19 −15 97 M1.5 02:092000-06-25T07:54 1617 −17.5 274 165 2000-06-25T07:17 16 55 M1.9 01:042001-07-19T10:30 1668 −11.6 252 166 2001-07-19T09:52 −8 62 M1.8 00:252002-03-22T11:06 1750 −22.5 259 360 2002-03-22T10:12 −10 95 M1.6 01:402002-05-30T05:06 1625 67.0 275 144 2002-05-30T04:24 5 95 M1.3 01:492002-08-16T12:30 1585 −67.1 121 360 2002-08-16T11:32 −14 −20 M5.2 01:352003-03-18T12:30 1601 −13.3 266 209 2003-03-18T11:51 −15 46 X1.5 00:292003-06-02T00:30 1656 42.5 248 172 2003-06-02T00:07 −8 89 M6.5 00:362003-11-18T08:50 1660 −3.3 206 360 2003-11-18T08:12 −2 −18 M3.9 00:472004-04-11T04:30 1645 −77.6 237 314 2004-04-11T03:54 −16 46 C9.6 00:412005-07-09T22:30 1540 −168.5 328 360 2005-07-09T21:47 12 28 M2.8 00:322005-09-13T20:00 1866 11.5 149 360 2005-09-13T19:19 −11 −3 X1.5 01:382013-02-06T00:24 1867 −8.2 31 271 2013-02-06T00:04 22 −19 C8.7 00:37

SEP Forecasting Algorithms and Associated False Alarms Page 23 of 27 173

Appendix C: Algorithm 2: False Alarms in Time Range 1

Table 6 List of X-class flares between 1 January 1996 and 31 March 2013 which were false alarms. Column1 gives the time the CME was first reported, column 2 its speed, and column 3 its acceleration all as reportedby CDAW. Column 4 is the position angle at which the height–time measurements had been made (called byCDAW the “measurement position angle”). Column 5 is the CME width. Columns 6 to 10 give the parametersof the associated flare: its start time, heliographic latitude, heliographic longitude, class, and duration.

First reported CME parameters Time start Flare parametersV(km s−1)

Accel.(km s−2)

Mpa(degrees)

Width(degrees)

Lat Lon Class Duration(hrs:mins)

LASCO data gap 1996-07-09T09:01 −10 30 X2.6 00:48

LASCO data gap 1998-11-22T16:10 −30 89 X2.5 00:22

LASCO data gap 1998-11-23T06:28 −28 89 X2.2 00:30

1998-11-24T02:30 1798 −12.5 225 360 1998-11-24T02:07 −20 94 X1.0 00:30

1999-08-02T22:26 292 0.9 264 157 1999-08-02T21:18 −18 46 X1.4 00:20

1999-08-28T18:26 462 1.1 221 245 1999-08-28T17:52 −26 14 X1.1 00:26

1999-11-27T12:54 235 64.2 256 68 1999-11-27T12:05 −15 68 X1.4 00:11

2000-03-02T08:54 776 0.8 233 62 2000-03-02T08:20 −11 70 X1.1 00:11

2000-03-22T19:31 478 −92.0 312 154 2000-03-22T18:34 14 57 X1.1 00:22

2000-03-24T07:41 16 82 X1.8 00:18

2000-06-06T13:30 22 −10 X1.1 00:16

2000-06-06T15:54 1119 1.5 47 360 2000-06-06T14:58 21 −9 X2.3 00:42

2000-06-07T16:30 842 59.8 309 360 2000-06-07T15:34 23 −3 X1.2 00:32

2000-06-18T02:10 629 −1.2 318 132 2000-06-18T01:52 23 85 X1.0 00:11

2000-09-30T23:13 7 91 X1.2 00:15

2001-04-02T10:04 17 60 X1.4 00:16

2001-04-02T11:26 992 3.0 278 80 2001-04-02T10:58 15 65 X1.1 01:07

2001-10-25T15:26 1092 −1.4 175 360 2001-10-25T14:42 −16 21 X1.3 00:46

2001-12-13T14:54 864 −11.4 37 360 2001-12-13T14:20 16 −9 X6.2 00:15

2002-07-03T02:54 265 −9.9 274 73 2002-07-03T02:08 −20 51 X1.5 00:08

2002-07-15T20:30 1151 −25.6 35 360 2002-07-15T19:59 19 1 X3.0 00:15

2002-07-18T08:06 1099 −30.2 354 360 2002-07-18T07:24 19 30 X1.8 00:25

2002-08-03T19:31 1150 −18.8 272 138 2002-08-03T18:59 −16 76 X1.0 00:12

2002-08-21T06:06 268 −19.6 266 66 2002-08-21T05:28 −12 51 X1.0 00:08

2002-10-31T17:06 1061 −33.4 96 43 2002-10-31T16:47 28 87 X1.2 00:08

2003-03-17T19:54 1020 −5.5 264 96 2003-03-17T18:50 −14 39 X1.5 00:26

2003-03-18T12:30 1601 −13.3 266 209 2003-03-18T11:51 −15 46 X1.5 00:29

2003-05-27T23:50 964 −9.6 67 360 2003-05-27T22:56 −7 17 X1.3 00:17

2003-06-09T21:31 12 34 X1.7 00:12

LASCO data gap 2003-06-10T23:19 10 40 X1.3 00:53

LASCO data gap 2003-06-11T20:01 14 57 X1.6 00:26

2004-02-26T01:50 14 14 X1.1 00:20

2004-08-13T18:07 −13 23 X1.0 00:08

2004-08-18T17:54 602 0.5 250 120 2004-08-18T17:29 −14 90 X1.8 00:25

2004-10-30T12:30 427 12.4 269 360 2004-10-30T11:38 13 25 X1.2 00:12

2005-01-15T00:22 14 −8 X1.2 00:40

2005-09-13T20:00 1866 11.5 149 360 2005-09-13T19:19 −11 −3 X1.5 01:38

2005-09-15T08:30 −11 24 X1.1 00:16

2011-02-15T02:24 669 −18.3 189 360 2011-02-15T01:44 −20 10 X2.2 00:22

2011-03-09T23:13 8 9 X1.5 00:16

173 Page 24 of 27 B. Swalwell et al.

Appendix D: Algorithm 2: False Alarms for Time Range 2

Table 7 List of X-class flaresbetween 1 April 1980 and 31December 1995 which were falsealarms. Column 1 gives the starttime of the flare, column 2 itsheliographic latitude, and column3 its heliographic longitude.Column 4 is the class of the flare,and column 5 its duration.

Time start Flare parameters

Lat Lon Class Duration (hrs:mins)

1980-05-21T20:51 −14 15 X1.4 00:35

1980-05-28T19:24 −18 33 X1.1 01:29

1980-06-04T22:57 −14 69 X2.2 00:17

1980-06-21T01:17 20 90 X2.6 00:43

1980-07-01T16:22 −12 38 X2.5 00:49

1980-10-14T05:42 −9 7 X3.3 01:52

1980-10-25T09:42 19 59 X3.9 00:28

1980-11-07T01:56 7 11 X2.7 01:19

1980-11-08T13:33 8 28 X3.3 02:05

1980-11-12T04:46 10 72 X2.5 00:06

1980-11-15T15:40 −12 83 X1.9 01:51

1981-02-17T18:12 20 20 X1 05:30

1981-02-20T06:40 19 49 X2.4 01:07

1981-03-25T20:39 9 89 X2.2 00:44

1981-04-02T11:03 −43 68 X2.2 00:25

1981-07-19T05:32 −37 56 X2.7 01:05

1981-07-26T07:57 −14 18 X1 00:35

1981-07-27T17:24 −13 −11 X1.5 01:24

1981-08-12T06:24 −10 28 X2.6 00:56

1981-09-15T21:13 10 78 X2.3 00:15

1982-02-07T12:50 −14 72 X1 01:21

1982-02-08T12:50 −13 88 X1.4 00:29

1982-02-09T03:57 −13 90 X1.2 00:26

1982-03-30T05:22 13 11 X2.8 03:04

1982-06-26T00:42 16 5 X1.9 01:26

1982-06-26T19:09 15 73 X2.1 01:04

1982-07-17T10:28 14 32 X3.2 00:53

1982-12-22T08:26 −9 82 X2.4 00:31

1982-12-29T06:43 −13 12 X1.9 00:34

1983-06-06T13:31 −11 15 X1.4 02:01

1988-06-23T08:56 −19 34 X1.6 01:07

1988-06-24T04:18 −18 45 X1.3 02:43

1988-06-24T16:03 −17 52 X2.4 00:51

1988-10-03T14:53 −27 16 X3.2 00:49

1988-10-03T23:22 −27 20 X1.1 00:57

1988-12-30T17:25 −19 30 X1.4 02:23

1989-01-13T08:29 −31 5 X2.3 02:16

1989-01-14T02:54 −32 10 X2.1 02:25

1989-01-14T21:45 −29 26 X1.1 01:24

1989-01-18T07:02 −30 65 X1.4 00:11

1989-01-27T19:08 −19 −17 X1.1 01:38

SEP Forecasting Algorithms and Associated False Alarms Page 25 of 27 173

Table 7 (Continued)Time start Flare parameters

Lat Lon Class Duration (hrs:mins)

1989-03-14T16:46 33 21 X1.1 05:02

1989-03-16T15:24 36 47 X3.6 01:21

1989-03-16T20:35 29 60 X1.4 00:56

1989-05-05T07:23 30 −1 X2.4 03:12

1989-06-15T18:13 −21 −8 X4.1 02:28

1989-06-16T04:19 −17 −3 X3 00:26

1989-09-03T14:28 −18 −16 X1.2 00:32

1989-09-04T08:57 −18 −19 X1.1 00:49

1989-09-09T19:28 −15 67 X1.3 00:28

1989-11-12T06:21 13 39 X1.5 00:46

1989-11-19T06:19 −24 25 X1.1 00:23

1989-11-20T21:25 −27 43 X1 00:36

1989-11-21T13:32 −26 53 X4 00:59

1989-11-25T22:55 30 −5 X1 03:40

1989-12-30T04:09 −19 −9 X1 01:05

1989-12-31T09:32 −25 51 X2.8 00:45

1991-01-30T08:49 −8 34 X1 01:36

1991-01-31T01:58 −17 35 X1.3 03:21

1991-03-16T00:47 −9 −9 X1.8 00:22

1991-03-17T20:54 −10 13 X1 02:11

1991-03-29T06:42 −28 60 X2.4 00:52

1991-03-31T19:11 −22 88 X1 00:08

1991-04-20T08:27 8 50 X1 02:57

1991-05-18T05:06 32 85 X2.8 02:42

1991-07-31T00:46 −17 −11 X2.3 01:29

1991-08-02T03:07 25 −15 X1.5 00:52

1991-09-07T19:11 −11 50 X3.3 01:10

1991-09-08T09:06 −13 58 X1 00:43

1991-10-26T18:53 −9 −20 X1.7 04:32

1991-10-27T02:06 −11 −20 X1.9 00:51

1991-10-27T05:38 −13 −15 X6.1 01:20

1991-11-09T15:32 −16 57 X1.1 01:37

1991-11-15T22:34 −13 19 X1.5 00:43

1991-12-24T10:13 −17 −14 X1.4 01:20

1992-01-26T15:23 −16 66 X1 01:02

1992-02-16T12:32 −13 17 X1.4 01:09

1992-02-27T09:22 6 2 X3.3 03:41

1992-09-06T18:42 −11 41 X1.7 02:09

1992-09-06T20:50 −11 46 X1.3 00:26

References

Alberti, T., Laurenza, M., Cliver, E.W., Storini, M., Consolini, G., Lepreti, F.: 2017, Solar activity from 2006to 2014 and short-term forecasts of solar proton events using the ESPERTA model. Astrophys. J. 838(1),59. DOI. ADS.

173 Page 26 of 27 B. Swalwell et al.

Aran, A., Sanahuja, B., Lario, D.: 2006, SOLPENCO: A solar particle engineering code. Adv. Space Res. 37,1240. DOI. ADS.

Balch, C.C.: 2008, Updated verification of the Space Weather Prediction Center’s solar energetic particleprediction model. Space Weather 6, S01001. DOI. ADS.

Balch, C.C.: 1999, SEC proton prediction model: Verification and analysis. Radiat. Meas. 30, 231. DOI.Beck, P., Latocha, M., Rollet, S., Stehno, G.: 2005, TEPC reference measurements at aircraft altitudes during

a solar storm. Adv. Space Res. 36, 1627. DOI. ADS.Belov, A., Garcia, H., Kurt, V., Mavromichalaki, H., Gerontidou, M.: 2005, Proton enhancements and their

relation to the X-ray flares during the three last solar cycles. Solar Phys. 229, 135. DOI. ADS.Bentley, R.D., Csillaghy, A., Aboudarham, J., Jacquey, C., Hapgood, M.A., Bocchialini, K., Messerotti,

M., Brooke, J., Gallagher, P., Fox, P., Hurlburt, N., Roberts, D.A., Duarte, L.S.: 2011, HELIO: TheHeliophysics Integrated Observatory. Adv. Space Res. 47, 2235. DOI. ADS.

Brueckner, G.E., Howard, R.A., Koomen, M.J., Korendyke, C.M., Michels, D.J., Moses, J.D., Socker, D.G.,Dere, K.P., Lamy, P.L., Llebaria, A., Bout, M.V., Schwenn, R., Simnett, G.M., Bedford, D.K., Eyles,C.J.: 1995, The Large Angle Spectroscopic Coronagraph (LASCO). Solar Phys. 162, 357. DOI. ADS.

Cliver, E.W., Ling, A.G., Belov, A., Yashiro, S.: 2012, Size distributions of solar flares and solar energeticparticle events. Astrophys. J. Lett. 756, L29. DOI. ADS.

Crosby, N.B., Glover, A., Aran, A., Bonnevie, C., Dyer, C., Gabriel, S., Hands, A., Heynderickx, D., Jacobs,C., Jiggens, P., King, D., Lawrence, G., Poedts, S., Sanahuja, B., Truscott, P.: 2010, SEPEM – SolarEnergetic Particle Environment Modelling. In: COSPAR Meeting 38, 4. ADS.

Dalla, S., Marsh, M.S., Kelly, J., Laitinen, T.: 2013, Solar energetic particle drifts in the Parker spiral. J.Geophys. Res. 118(10), 5979. DOI.

Dierckxsens, M., Tziotziou, K., Dalla, S., Patsou, I., Marsh, M.S., Crosby, N.B., Malandraki, O., Tsiropoula,G.: 2015, Relationship between solar energetic particles and properties of flares and CMEs: Statisticalanalysis of Solar Cycle 23 events. Solar Phys. 290, 841. DOI. ADS.

Dumbovic, M., Devos, A., Vršnak, B., Sudar, D., Rodriguez, L., Ruždjak, D., Leer, K., Vennerstrøm, S.,Veronig, A.: 2015, Geoeffectiveness of coronal mass ejections in the SOHO era. Solar Phys. 290, 579.DOI. ADS.

Feynman, J., Gabriel, S.B.: 2000, On space weather consequences and predictions. J. Geophys. Res. 105,10543. DOI. ADS.

Garcia, H.A.: 1994, Temperature and emission measure from GOES soft X-ray measurements. Solar Phys.154, 275. DOI. ADS.

Gopalswamy, N., Yashiro, S., Michalek, G., Stenborg, G., Vourlidas, A., Freeland, S., Howard, R.: 2009, TheSOHO/LASCO CME catalog. Earth Moon Planets 104, 295. DOI. ADS.

Gopalswamy, N., Xie, H., Akiyama, S., Mäkelä, P.A., Yashiro, S.: 2014, Major solar eruptions and high-energy particle events during Solar Cycle 24. Earth Planets Space 66, 104. DOI. ADS.

Grubb, R.N.: 1975, The SMS/GOES space environment monitor subsystem. NASA STI/Recon. Tech. Rep. N.76. ADS.

Hargreaves, J.K.: 2005, A new method of studying the relation between ionization rates and radio-waveabsorption in polar-cap absorption events. Ann. Geophys. 23, 359. DOI. ADS.

Harrison, R.A., Bewsher, D.: 2007, A benchmark event sequence for mass ejection onset studies. A flareassociated CME with coronal dimming, ascending pre-flare loops and a transient cool loop. Astron.Astrophys. 461, 1155. DOI. ADS.

Hoff, J.L., Townsend, L.W., Zapp, E.N.: 2004, Interplanetary crew doses and dose equivalents: variationsamong different bone marrow and skin sites. Adv. Space Res. 34, 1347. DOI. ADS.

Kahler, S.W., Ling, A.: 2015, Dynamic sep event probability forecasts. Space Weather 13(10), 665. DOI.Kahler, S.W., Cliver, E.W., Ling, A.G.: 2007, Validating the proton prediction system (PPS). J. Atmos. Solar-

Terr. Phys. 69, 43. DOI. ADS.Klein, K.-L., Trottet, G., Klassen, A.: 2010, Energetic particle acceleration and propagation in strong CME-

less flares. Solar Phys. 263, 185. DOI. ADS.Klein, K.-L., Trottet, G., Samwel, S., Malandraki, O.: 2011, Particle acceleration and propagation in strong

flares without major solar energetic particle events. Solar Phys. 269, 309. DOI. ADS.Kwon, R.-Y., Zhang, J., Vourlidas, A.: 2015, Are halo-like solar coronal mass ejections merely a matter of

geometric projection effects? Astrophys. J. Lett. 799, L29. DOI. ADS.Laurenza, M., Cliver, E.W., Hewitt, J., Storini, M., Ling, A.G., Balch, C.C., Kaiser, M.L.: 2009, A technique

for short-term warning of solar energetic particle events based on flare location, flare size, and evidenceof particle escape. Space Weather 7, 4008. DOI. ADS.

Luhmann, J.G., Ledvina, S.A., Odstrcil, D., Owens, M.J., Zhao, X.-P., Liu, Y., Riley, P.: 2010, Cone model-based SEP event calculations for applications to multipoint observations. Adv. Space Res. 46, 1. DOI.ADS.

SEP Forecasting Algorithms and Associated False Alarms Page 27 of 27 173

Marqué, C., Posner, A., Klein, K.-L.: 2006, Solar energetic particles and radio-silent fast coronal mass ejec-tions. Astrophys. J. 642, 1222. DOI. ADS.

Marsh, M.S., Dalla, S., Kelly, J., Laitinen, T.: 2013, Drift-induced perpendicular transport of solar energeticparticles. Astrophys. J. 774, 4. DOI. ADS.

Marsh, M.S., Dalla, S., Dierckxsens, M., Laitinen, T., Crosby, N.B.: 2015, SPARX: A modeling system forsolar energetic particle radiation space weather forecasting. Space Weather 13(6), 386. DOI.

Papaioannou, A., Anastasiadis, A., Sandberg, I., Georgoulis, M.K., Tsiropoula, G., Tziotziou, K., Jiggens, P.,Hilgers, A.: 2015, A novel forecasting system for solar particle events and flares (FORSPEF). J. Phys.Conf. Ser. 632(1), 012075. DOI. ADS.

Papaioannou, A., Sandberg, I., Anastasiadis, A., Kouloumvakos, A., Georgoulis, M.K., Tziotziou, K.,Tsiropoula, G., Jiggens, P., Hilgers, A.: 2016, Solar flares, coronal mass ejections and solar energeticparticle event characteristics. J. Space Weather Space 6(27), A42. DOI. ADS.

Park, J., Moon, Y.-J., Gopalswamy, N.: 2012, Dependence of solar proton events on their associated activities:Coronal mass ejection parameters. J. Geophys. Res. 117, A08108. DOI. ADS.

Posner, A.: 2007, Up to 1-hour forecasting of radiation hazards from solar energetic ion events with relativisticelectrons. Space Weather 5(5), S05001. DOI.

Reinard, A.A., Andrews, M.A.: 2006, Comparison of CME characteristics for SEP and non-SEP relatedevents. Adv. Space Res. 38, 480. DOI. ADS.

Smart, D.F., Shea, M.A.: 1989, PPS-87 – A new event oriented solar proton prediction model. Adv. SpaceRes. 9, 281. DOI. ADS.

Vršnak, B., Sudar, D., Ruždjak, D.: 2005, The CME-flare relationship: Are there really two types of CMEs?Astron. Astrophys. 435, 1149. DOI. ADS.

Wang, Y., Zhang, J.: 2007, A comparative study between eruptive X-class flares associated with coronal massejections and confined X-class flares. Astrophys. J. 665, 1428. DOI. ADS.

Winter, L.M., Ledbetter, K.: 2015, Type II and type III radio bursts and their correlation with solar energeticproton events. Astrophys. J. 809(1), 105. DOI. ADS.

Yashiro, S., Akiyama, S., Gopalswamy, N., Howard, R.A.: 2006, Different power-law indices in the frequencydistributions of flares with and without coronal mass ejections. Astrophys. J. Lett. 650, L143. DOI. ADS.