protecting the leaders—syndromic surveillance for the g8 summit in scotland

69june2006

Protecting the leaders—syndromic surveillance

for the G8 summit in Scotland

The G8 summit was held at the Gleneagles Hotel, Scotland, from July 6th to 8th, 2005. Leaders of the eight largest economies, and their entourages, assembled to discuss economic and political issues and the state of the world, including terrorism. Previous G8 summits had sparked off waves of protests and violence. Consequently, a huge security exercise took place in central Scotland for the 2005 summit. With Bush, Putin, Blair and a clutch of other premiers on hand, their safety had to be protected—as did their health, and that of those around them. Chris Robertson was charged with that task.

Health Protection Scotland (HPS) is part of the NHS in Scotland and is charged with working, in partnership with others, to protect the Scottish public from being exposed to hazards that could damage their health and to limit any impact on health when such exposures cannot be avoided. Our concern in the period surrounding the G8 summit was with the health and security issues that might arise from a deliberate release of a chemical or biological agent. Also of concern was the possible introduction of unusual diseases to Scotland; diseases which might be carried by the thousands of people who were expected to visit the country around the time of the G8 summit, principally demonstrators, but also the politicians’ entourages them-selves. Consequently, HPS was given the task of monitor-ing diseases in and around the summit site in the period immediately before, during and after the summit.

In the early stages of a disease epidemic there is usually a slow rise in the number of cases before the World leaders gather…

3(2)_07 Robertson_G8Summit.indd 693(2)_07 Robertson_G8Summit.indd 69 09/05/2006 12:29:5809/05/2006 12:29:58

70 june2006

huge increase, and the purpose of syndromic surveillance is to monitor a large number of syndromes associated with the diseases of concern. By comparing observed counts with expected counts, it is hoped to obtain an early detection of a disease outbreak. In this way medical interventions, such as prophylactic antibiotics, can be advanced with the expec-tation that the peak of the epidemic will be reduced1.

Syndromic surveillance systems have been used in the recent past at times of large gath-erings of individuals and have received much more attention following the September 11th terrorist attacks in the USA, where such sys-tems are now increasingly popular. Th ey have also been used at political party conventions and large sporting events, such as the 2004 Athens Olympics and the 2006 Turin Winter Olympics2. England and Wales have used a sys-tem based on the NHS Direct telephone sys-tem3. Perhaps the most popular system is the Early Aberration Reporting System (EARS) of the Center for Disease Control, Atlanta4.

In early April 2005 we heard that HPS was to run a syndromic surveillance system for the G8 summit over the period July 4th to 15th. Prior to the this, HPS did not have a such a system. Th is news did not bother us

too much as HPS does have a weekly excep-tion reporting system for counts of the organ-isms that are routinely reported. Th is is based on the exceedance method5, and we also rou-tinely use cusum methods for looking at trends in MRSA bacteraemia rates, for example. We started work on the system in early May and the statistical modelling and reporting aspects were ready by early June, but most of the test-ing was based on simulated data as very few of the data streams were routinely supplying data.

Th e syndromic surveillance system was to collect daily information from a number of data providers in central Scotland. Th e syn-dromes would cover symptoms associated with a number of diseases. Th e critical bioterrorism agents selected for surveillance included an-thrax, plague, smallpox, botulism, viral haem-orrhagic fevers and tularaemia. Th e system was

also designed to pick up symptoms related to common infectious diseases (for example, no-rovirus and legionella) and unusual ones (such as avian infl uenza and severe acute respiratory syndrome (SARS)). In total the system moni-tored daily data from 42 syndromes, ranging from colds and fl u, vomiting, coughs, eye prob-lems and rashes, to botulism-like symptoms, abdominal pain, gastrointestinal bleeding, shortness of breath, collapsed adult and unex-pected death.

Data were obtained from a number of local providers (see map), who routinely col-lected data electronically for purposes other than surveillance. Th ese included: the general practices in the two villages or towns closest to Gleneagles, Auchterarder and Dunblane; the bacteriology and virology laboratories of Stir-ling, Perth and Dundee hospitals, which cover the area surrounding Gleneagles; accident and emergency departments at Ninewells hospital, Dundee, and Perth hospital; and telephone calls to the three call centres of the NHS 24 hotline. Over the 10-day period of the surveil-lance—from 2 days before the summit to a week after—there were 23 193 telephone calls to NHS 24; 2566 laboratory test orders; 225 A&E visits; and 188 visits to GPs. Historical data for the NHS 24 hotline, Auchterarder general practice and the hospital laboratories was available from January 2005; from Febru-ary 2005 for the Dunblane general practice; and from June 2005 for the hospital A&E de-partments.

Many exception reporting systems are based on the premise of using past data to

predict a future value and then comparing the observed to the expected to determine wheth-er there has been an exceptionally high count in a particular day. We decided to use two methods for reporting an exception, one based on the exceedance method5 and the other on the cusum6,7. Th ese have diff erent properties and the cusum is better able to detect small incremental increases, whereas the exceedance method can detect a single instantaneous ex-ception.

In both cases the prediction model was based on an over-dispersed Poisson regres-sion model. For some of the data sources there was only one month’s data on which to base the prediction model, but for most there was six months’. For most of the syndromes we would expect seasonal or weekly fl uctuations in the counts and, after much investigation, we found that the most important eff ect was the diff erences between weekdays and weekends. For most of the relatively rare syndromes there was very little evidence of any trend over the 6 months of previous data, and the predicted val-ues for the cusum were obtained from an over-dispersed Poisson model where the parameters were estimated using all previous available data (where there was no evidence of any trend) or data from the previous 4 weeks (where there was evidence of a trend). Th e parameters of the cusum were set such that the expected time between false alarms for any one cusum—if the process was not changing—was 400 days. However, the high number of syndromes and data providers meant that a total of 1224 cu-sums were run (102 for each of the 12 days of

Gleneagle

Dundee (A&E, Pop 300,000

Perth (A&E, ab) Pop 50,000

Kirkcaldy Pop 47,155

Dunfermline Pop 55,083

Edinburgh

Glasgow

Oban

Mallaig

Inverness

Aberdeen

Stirling (A&E,Pop 134,610

Dunblane(GP) Pop 9,904

Crieff(GP)Pop 18,248

(GP & on-site hotel clinic & occ. health)

Pop 8,122

‘NHS 24’covers all Scotland

Gleneagles

Dundee (A&E) Pop 300 000

Perth (A&E, ab) Pop 50 000

Kirkcaldy Pop 47 155

Dunfermline Pop 55 083

Edinburgh

Glasgow

Oban

Mallaig

Inverness

Aberdeen

Stirling (A&E)Pop 134 610

Dunblane(GP) Pop 9904

Crieff(GP)Pop 18 248

(GP & on-site hotel clinic & occ. health)

Pop 8122

‘NHS 24’covers all Scotland

GleGlenneagleeagles

“Critical bioterrorism agents included anthrax, plague,

smallpox and botulism”


71june2006

the surveillance), leading almost inevitably to false alarms.

Shorter-term predictions were used for the exceedance method in case some of the syndromes had local trends. Th e expected syndrome counts for each day were calculated using an over-dispersed Poisson regression model incorporating a term to diff erentiate weekend days from weekdays. Th ree diff erent baseline periods were used to fi t the models: the previous 7 days; days 3 to 9 in the past; and days 3 to 30 in the past (that is, the pre-vious 4 weeks). Th e fi rst two correspond to C1 and C2 of the EARS4. Th ree reference periods were chosen to respond to potentially

diff erent recent trends during the surveillance period.

Th e sensitivity and specifi city of the sys-tem was investigated using simulated data, with outbreaks of random commencement and duration over a 1-month surveillance pe-riod. With the exceedance limit set at 5% the sensitivity was about 85% and the specifi city of detecting an outbreak in the fi rst 3 days was over 95%. Th e specifi city of detecting an out-break on the fi rst day was only 70%. In all hon-esty, this simulation could have been done in a great more detail and in a much more system-atic fashion. Ideally, it should have been based on the type of data that we were expecting to

deal with, but most of the data did not begin to arrive until late June, and by that time we were more concerned with making sure that the day-to-day working of the system was val-id rather than in investigating the appropriate prediction models to use over the wide variety of syndromes.

Th e biggest headache we had in the run-up to the beginning of the surveillance period was the data quality and data checking. Th e laboratory data came as a line listing by syn-drome and was very easy to use. It was extract-ed for us by laboratory technicians and came in the same format each day and was available from the beginning of June. Th e GP data were held on local computer systems through the General Practice Administration System for Scotland (GPASS) and programmers were contracted to go to the surgeries to install extraction programs. Th e GP data came as a line listing of Read codes, a coded thesaurus of clinical terms which enables clinicians ac-curately to code each consultation. Th e Read code is named after James Read, who used to be a GP. Writing a program to recode these in to the syndromes was no easy task.

Th e NHS 24 data sometimes came as an Excel worksheet, with total calls by syndrome, and at other times (weekends mainly) as a line listing of every call. Th e text in the line list-ing had to be scanned for the presence of key words and phrases denoting the syndromes. In fact, once written it was easier to process the line listing daily rather than the Excel fi le, as the latter was created manually in NHS 24 each day and was not always in an identical format, whereas the line listing was a database dump and easy to deal with.

…for round-table discussions.


72 june2006

Th e baseline A&E data only arrived at HPS 2 or 3 days before the syndromic sur-veillance system was due to be in place. In fact, a specialist registrar had to go to the hospitals and run the required data extraction programs. Th ese programs had to be specially written, as the A&E database was constructed for a dif-ferent purpose—to manage emergency admis-sions to hospital—and, naturally enough, data extraction for the G8 summit surveillance was not the prime concern of the A&E staff . Th e extraction programs produced Excel fi les, which looked pretty, but had with them all the necessary details about the admissions laid out all over the worksheet. Th e location of the data items in the worksheet varied from day to day and from hospital to hospital in a seem-ingly random fashion. Initially, we thought that this important data would be useless, but

we were able to write some code in S-PLUS to interrogate each Excel worksheet and look for the keywords as, fortunately, the required data was adjacent to or below a text fi eld. So, with a lot of late work on writing code for automatic data manipulation, we were in a position to use the system on Monday, 4th July.

Th at day was very quiet. Th e previous day’s A&E data arrived in the morning, as did the laboratory data. NHS 24 data came in the afternoon, and GP data between 5.00 and 6.00 in the evening. Th e whole process oper-ated in real time. As soon as the data from a centre were received, they were processed, us-ing either R or Stata, and a list of the excep-tions was reported to the epidemiologists and the public health consultants. Two separate systems were used, as my colleague Gwen Al-lardice and I were each familiar with diff er-ent programs and we had to operate a failsafe system so that, in the event of one of us not being able to work, then parts of the system could still function. Data were uploaded to a Web-based system for interrogation by inves-tigators.

On Tuesday, July 5th there were a whole host of exceptions for the NHS 24 data from the Edinburgh centre for Monday. It turned out that Monday had been a public holiday in Edinburgh and we had forgotten to include a term for public holidays in the prediction model. Th is was a silly mistake.

Th ereafter, the whole process went rela-tively easily, although events were overshad-owed by the Tube and bus bombing tragedies in London. However, this also served to con-centrate our minds on the task in hand. In all, from the combination of 121 syndromes and data centres investigated daily over the 12 days of the surveillance period, the exceedance method and cusum systems together produced 95 signals; and 13 of these were investigated in detail. One cluster of gastroenteritis infec-tion was detected at Perth Royal Infi rmary A&E department on July 7th as a result of this system. Th is arose from three police offi cers admitting themselves with severe abdominal pain at separate times during the day. Further investigation revealed another two cases, and the public health system engaged in activities to prevent further infections among the police by increased cleaning of the on-site toilet facili-ties for police and security offi cers and distri-bution of infection control information.

In retrospect, we would have been a lot happier had we carried out a lot more checking on the sensitivity and specifi city of the system before using it live. We would also have been better equipped had the data sources provided regular data sooner, and we should have been more proactive in going out to see them so that we had much more control over the format in which the data arrived. We had to solve a lot of (unnecessary) problems stemming from the data being stored in diff erent types of data-bases and the fact that day-to-day users of the databases were not familiar with the best ways of extracting the data.

In all, this was a very interesting process to have been involved in. Statisticians are not often faced with designing a real-time system that has to work the fi rst time it is used and be delivered within a short time. Th is meant that a number of operational decisions—such as the type of estimation and prediction meth-od to be used and the method for fl agging up observations as exceptions—were based on our previous experiences rather than on a sys-tematic study. Also, we were using experiences from exception reporting systems in other re-lated areas.

Of course, it could have gone badly wrong. It is not often the case that a statisti-cian has to make decisions about exceedances or exceptions right away. Our general strategy is to report outliers and then ask the investi-gator to check the experiment to see if any-thing strange had taken place that might have given rise to the outliers. Th is process can take weeks, and we generally have plenty of time to consider the appropriate response. In this in-

stance we had data arriving, which were then processed relatively automatically—based on our best choices for prediction models and exceedance methods—and within 30 minutes we had to report on any important exceptions. Potentially one of these exceptions could have led to a major security alert, which in turn could have led to closure of the G8 summit and the leaders dispersing early—a premature end to the summit that would have reverber-ated worldwide, and which could easily have been based upon a false positive. Th is would have left us looking, and feeling, very foolish. On the other hand, it would have been much worse to miss the beginnings of a positive outbreak, particularly if it had started as a re-sult of a deliberate release of an infectious or chemical agent. We did not feel that our vigi-lance was unnecessary.

References1. Lawson, A. B. and Kleinman, K. (eds)

(2005) Spatial and Syndromic Surveillance for Public Health. Chichester: Wiley.

2. Eurosurveillance (2006) Surveillance sys-tem in place for the 2006 Winter Olympic Games, Torino, Italy. Surveillance Report, 11-2. (Available from http://www.eurosurveillance.org/ew/2006/060209.asp.)

3. Cooper, D. L., Smith, G. E., Joseph, C., Hollyoak, V. and Dickens, J. (2002) Th e develop-ment of a new national surveillance system in the UK for “infl uenza-like illness”. Clinical Microbiology and Infection, 8(Suppl 1).

4. Hutwagner, L., Th ompson, W., Seeman, G. M. and Treadwell, T. (2003) Th e bioterrorism preparedness and response Early Aberration Re-porting System (EARS). Journal of Urban Health, 80-2, (Suppl 1):I, 89–96.

5. Farrington, C. P., Andrews, N. J., Beale, A. D. and Catchpole, M. A. (1996) A statistical algo-rithm for early detection of outbreaks of infectious disease. Journal of the Royal Statistical Society, Series A, 159, 547–563.

6. Hawkins, D. and Olwell, D. (1998) Cu-mulative Sum Charts and Charting for Quality Im-provement. New York: Springer.

7. Rogerson, P. A. and Yamada, I. (2004) Approaches to syndromic surveillance when data consist of small regional counts. Morbidity and Mortality Weekly Report, 53, S, 79–85.

Chris Robertson is Head of Statistics at HPS and Pro-fessor of Public Health Epidemiology at Strathclyde University. His main research interests are in statisti-cal applications and modelling in epidemiology. The G8 summit surveillance was a collaborative effort involving colleagues at HPS—Dr Jim McMeniman, Consultant Epidemiologist for Respiratory Infection, and Dr Nadia Meyer, Epidemiologist—and Dr Gwen Allardice, Senior Lecturer in Statistics at Strathclyde University.

“Tube and bus bombings in London concentrated our

minds on the task in hand”


protecting the leaders—syndromic surveillance for the g8 summit in scotland

Documents