social media usage reveals how regions recover after natural … · 2019-08-21 · social media...

37
Social media usage reveals how regions recover after natural disaster Robert Eyre a , Flavia de Luca b , and Filippo Simini a,* a University of Bristol, Department of Engineering Mathematics, Bristol, BS8 1UB, UK b University of Bristol, Department of Civil Engineering, Bristol, BS8 1UB, UK * To whom correspondence should be addressed; E-mail: [email protected] Abstract The challenge of nowcasting and forecasting the effect of natural disasters (e.g. earthquakes, floods, hurri- canes) on assets, people and society is of primary importance for assessing the ability of such systems to recover from extreme events. Traditional disaster recovery estimates, such as surveys and interviews, are usually costly, time consuming and do not scale. Here we present a methodology to indirectly estimate the post-emergency recovery status (‘downtime’) of small businesses in urban areas looking at their online posting activity on so- cial media. Analysing the time series of posts before and after an event, we quantify the downtime of small businesses for three natural disasters occurred in Nepal, Puerto Rico and Mexico. A convenient and reliable method for nowcasting the post-emergency recovery status of economic activities could help local governments and decision makers to better target their interventions and distribute the available resources more effectively. Natural disaster management is of interest to organisations that deliver aid, provide insurance, and operate in areas affected by disasters. The steps in natural disaster management are categorised into “phases”, with previous literature identifying anywhere from four to eight phases during the occurrence of a natural disaster [1]. For the purpose of this article, we take the phases used by the Federal Emergency Management Agency (FEMA) - mitigation, preparedness, response and recovery [2]. Of the four stages of the disaster life cycle, recovery is the least understood and investigated [3]. Factors such as deaths and building damage have been used to immediately quantify the magnitude of a disaster, but are not necessarily indicative of the longer-term recovery process. Multiple indicators for the recov- ery process of an area have been proposed [4], with many of them focusing on economic indicators of business activity thought to capture the long-term efforts of a region to return to a normal state. In particular, we refer herein to the recovery time as “downtime” relying on the definition within the Performance-Based Engineering framework; i.e., “the time necessary to plan, finance, and complete repairs on facilities damaged in earthquakes or other disasters” [5] and applying the downtime concept to small businesses. The definition of actual business downtime, i.e. the time in which a business is either closed or not running as expected, has been debated strongly in the literature. Chang’s framework for urban disaster recovery [6] highlights three such definitions of recovery; (a) returning to the same level of activity before the disaster, (b) reaching the level of activity that the business would have been attained without the disaster happening, and (c) attaining some stable level of activity that is different from these definitions. Studies on long-term business recovery have been made using large-scale economic indicators, such as the number of reported businesses in Kobe City over a ten year period following the Kobe City earthquake in 1995 [6]. These large-scale economic indicators are not as readily available or relevant for disasters of a more 1 arXiv:1908.07384v1 [physics.soc-ph] 20 Aug 2019

Upload: others

Post on 28-May-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

  • Social media usage reveals how regions recover after naturaldisaster

    Robert Eyrea, Flavia de Lucab, and Filippo Siminia,*

    aUniversity of Bristol, Department of Engineering Mathematics, Bristol, BS8 1UB, UKbUniversity of Bristol, Department of Civil Engineering, Bristol, BS8 1UB, UK

    *To whom correspondence should be addressed; E-mail: [email protected]

    Abstract

    The challenge of nowcasting and forecasting the effect of natural disasters (e.g. earthquakes, floods, hurri-canes) on assets, people and society is of primary importance for assessing the ability of such systems to recoverfrom extreme events. Traditional disaster recovery estimates, such as surveys and interviews, are usually costly,time consuming and do not scale. Here we present a methodology to indirectly estimate the post-emergencyrecovery status (‘downtime’) of small businesses in urban areas looking at their online posting activity on so-cial media. Analysing the time series of posts before and after an event, we quantify the downtime of smallbusinesses for three natural disasters occurred in Nepal, Puerto Rico and Mexico. A convenient and reliablemethod for nowcasting the post-emergency recovery status of economic activities could help local governmentsand decision makers to better target their interventions and distribute the available resources more effectively.

    Natural disaster management is of interest to organisations that deliver aid, provide insurance, and operatein areas affected by disasters. The steps in natural disaster management are categorised into “phases”, withprevious literature identifying anywhere from four to eight phases during the occurrence of a natural disaster [1].For the purpose of this article, we take the phases used by the Federal Emergency Management Agency (FEMA)- mitigation, preparedness, response and recovery [2]. Of the four stages of the disaster life cycle, recovery isthe least understood and investigated [3].

    Factors such as deaths and building damage have been used to immediately quantify the magnitude of adisaster, but are not necessarily indicative of the longer-term recovery process. Multiple indicators for the recov-ery process of an area have been proposed [4], with many of them focusing on economic indicators of businessactivity thought to capture the long-term efforts of a region to return to a normal state. In particular, we referherein to the recovery time as “downtime” relying on the definition within the Performance-Based Engineeringframework; i.e., “the time necessary to plan, finance, and complete repairs on facilities damaged in earthquakesor other disasters” [5] and applying the downtime concept to small businesses.

    The definition of actual business downtime, i.e. the time in which a business is either closed or not runningas expected, has been debated strongly in the literature. Chang’s framework for urban disaster recovery [6]highlights three such definitions of recovery; (a) returning to the same level of activity before the disaster, (b)reaching the level of activity that the business would have been attained without the disaster happening, and (c)attaining some stable level of activity that is different from these definitions.

    Studies on long-term business recovery have been made using large-scale economic indicators, such asthe number of reported businesses in Kobe City over a ten year period following the Kobe City earthquake in1995 [6]. These large-scale economic indicators are not as readily available or relevant for disasters of a more

    1

    arX

    iv:1

    908.

    0738

    4v1

    [ph

    ysic

    s.so

    c-ph

    ] 2

    0 A

    ug 2

    019

  • moderate scale, inspiring the use of alternative statistics as predictors for business recovery, such as changes inpedestrian foot traffic [7], manually reporting on when businesses closed and opened [8] and changes in parkingtransactions [9], allowing for a much smaller scale to be studied. However, surveys and traditional direct datasources of businesses downtime have either been too costly or too inconvenient for widespread use.

    Remote sensing has been shown to be vital in rapid damage detection and relief efforts after natural disas-ters. For example, the European Union’s Copernicus Emergency Management Service and the Jet PropulsionLaboratory’s Advanced Rapid Imaging and Analysis project use satellite imagery to provide assessment of re-gions devastated by different types of disasters. The main application of post-disaster satellite imagery is onemergency response and large-scale damage estimation [10]. Recent efforts to predict downtime using satelliteimagery include counting the number of vehicles in imagery collected by Google Earth, correlating the presenceof vehicles in business zones to the activity of businesses [11]. This method requires to have multiple satelliteimages of the region over time and may not be reliable in regions with developed public transport links, wherethe presence (or lack) of vehicles may not always correlate to small businesses’ level of economic activity. Adesirable secondary source of data for downtime estimation should be more readily accessible (easy and cheapto collect), have a high spatio-temporal resolution and, in the context of natural disaster, be rapidly obtainable.It is for these reasons that studies have turned to social media as a source of indicators for damage predictionafter a natural disaster.

    Social media data has been shown to be effective at rapidly detecting the presence of natural disasterssuch as earthquakes and floods, however the underlying process of social media use during natural disasteris not completely understood [12]. The micro-blogging service Twitter is often a source of related data, due tothe nature of the platform - only allowing short 280 character maximum (often geo-tagged) posts promotes theexchange of informative data. Other social media and content sharing websites have also been shown to exhibitcorrelating behaviour to natural disaster events, such as the photo sharing website Flickr [13, 14]. Many currentsocial-media methods rely on sentiment analysis to filter messages by relevancy and quantify the severity ofresponse to an event [15, 12, 16, 17, 18]. These methods offer rapid assessment of an event’s infancy andare useful tools for understanding human behaviour during emergencies and to improve the delivery of hazardinformation in a region. Links between damage indicators such as assistance data from the Federal EmergencyManagement Agency (FEMA), and tweets have also been found [19]. Mobile phone data has been used toobtain insights on human behaviour during emergencies. Bagrow et al. [20] show that mobile phone data can beused to identify how human behaviour changes during periods of unfamiliar conditions, in a variety of situationsranging from bombings to festivals.

    Dynamic mapping of population through the use of mobile phone records [21] have been used to measurepopulation displacement after the 2015 Gorhka earthquake in Nepal, where negative inflows were recorded inKathmandu for roughly 60 days after the event [22]. Mobile phone data was also used in the case of the 2010Haiti Earthquake to show that movement actually became more predictable up to three months after the eventin comparison to normal activity [23]. One of the main limitations of mobile phone data is that it is usually notpublicly available because of privacy concerns, hence obtaining mobile phone records for a specific region is notalways possible. Most of the studies in the literature often focus on emergency management and rapid responsewithin the short term, identifying sharp spikes in activity to determine the immediate impact of a disaster in agiven region.

    In this paper we show that downtime can be estimated in real time using the public posts of local businesseson the social media site ‘Facebook’ collected before, during and after a natural disaster, without the need fornatural language processing or semantic analysis. In particular, we consider three natural disasters of differenttypes (two earthquakes and one hurricane) which occurred in countries with different indicators of socioeconomicdevelopment (Nepal, Mexico and Puerto Rico). The locations of the businesses collected in the three regionsconsidered are shown in Figure 1a-c and the respective time series of the total number of posts retrieved areshown in Figure 1d-f (See Materials and Methods for a description for each natural disaster)

    2

  • 85.28 85.30 85.32 85.34 85.36longitude

    27.68

    27.69

    27.70

    27.71

    27.72

    27.73

    27.74

    latit

    ude

    (a) Kathmandu, Nepal

    66.10 66.05 66.00longitude

    18.300

    18.325

    18.350

    18.375

    18.400

    18.425

    18.450

    latit

    ude

    (b) San Juan, Puerto Rico

    95.03 95.02 95.01longitude

    16.420

    16.425

    16.430

    16.435

    16.440

    16.445

    latit

    ude

    (c) Juchitán de Zaragoza, Mexico

    2013 2014 2015 2016 2017 2018date

    0

    1000

    2000

    3000

    post

    s

    (d) Gorkha Earthquake, 2015raw datarolling two week meanevent

    2013 2014 2015 2016 2017 2018date

    0

    1000

    2000

    3000

    post

    s

    (e) Hurricane Maria, 2017

    2013 2014 2015 2016 2017 2018date

    0

    100

    200

    300

    post

    s

    (f) Chiapas Earthquake, 2017

    Figure 1: (a-c) Regions in which Facebook posts have been collected, each business highlighted in red (roadnetworks collected from OpenStreetMap [24] using OSMnx [25]). (d-f) Time series for the number of posts madein each region. The vertical dashed lines denote the date of the natural disasters.

    1 ResultsTo gauge the impact of a natural disaster (or event) on the businesses within a specific region, we consider thebusinesses’ posting activity on Facebook. Specifically, we compare the time series of number of posts on abusiness’ page in the time period after the event with the typical posting activity before the event.

    The framework is composed of three phases , schematically illustrated in Figure 2: (i) Data collection(Sect. 3.2); (ii) Data processing (Sect. 3.3); (iii) Downtime detection (Sect. 3.7). In phase (i) publicly avail-able data on the businesses’ posting activity on Facebook is collected. In phase (ii) the time series of the postingactivity is processed in order to remove trends, periodicities, seasonality effects, heterogeneous posting ratesand correlations, that are present in the raw data. In phase (iii) we define an automatic method to determinewhether the posting activity presents anomalies, such as a posting rate significantly lower than the average rate,and use this to estimate the length of the downtime.

    Using this framework, we find downtime in Kathmandu (50 days), Juchitán de Zaragoza (52 days) and SanJuan (118 days) on the dates of their respective natural disaster events. The events detected in the threeregions are shown in Figure 3; the start date, end date and the length of downtime of all the events detectedare reported in the Supplementary Materials. Given that we are using a weekly rolling mean to detect the start,

    3

  • x1(t) x2(t) ... xB(t)Data

    Colle

    ctio

    nPhase

    1:

    q1(t) q2(t) ... qB(t)

    rPIT (t) =∑Bi=1 qi(t)

    rN (t) =rPIT (t)−(n(t)/2)√

    n(t)/12

    r̃N (t) =rN (t)−µ(t)

    σ(t)

    rU (t) = PN (r̃(t))

    Data

    pro

    cessin

    gPhase

    2:

    T ∗, Threshold detection

    Event detection

    Downtim

    edetectio

    nPhase

    3:

    Figure 2: Schematic for the three phases of the proposed framework to detect the impact of natural disasters onsmall businesses.

    4

  • 0.0

    0.2

    0.4

    0.6

    0.8

    1.0r U

    (t)Downtime events (Kathmandu, Nepal)

    0.0

    0.2

    0.4

    0.6

    0.8

    1.0

    r U(t)

    Downtime events (San Juan, Puerto Rico)

    2013 2014 2015 2016 2017 2018Date

    0.0

    0.2

    0.4

    0.6

    0.8

    1.0

    r U(t)

    Downtime events (Juchitán de Zaragoza, Mexico)

    Figure 3: Downtime events longer than seven days detected in each region. One week rolling mean shownin black, raw data shown in grey, red dashed line denotes the threshold estimated with the ‘elbow method’ (seeSection 3.7). The thresholds T ∗ are marked with dashed horizontal red lines and the corresponding events arehighlighted in red (natural disasters) and blue (other events).

    end and duration of the downtime, we expect our estimates to have an uncertainty of around one week. Thefollowing sections discuss the validation of our downtime estimates in each of the three regions considered.

    1.1 Kathmandu, NepalA downtime of 50 days is predicted over the entire city after the Gorkha Earthquake in 2015. This sits comfort-ably between the lower bound of 40 days found with the surveys [11] (see Supplementary Materials), and theupper bound of approximately 56 days found by mobility data in [22]. To further test the downtime prediction,we also applied the process to Region A (around the UNESCO Heritage site of Kathmandu Durbar Square, seeSupplementary Materials) which contains the majority of the surveyed businesses, and report a downtime of 33days - close to the 28 days of average downtime reported by the surveys in this region. We also detect shorterdowntimes in other times of the year and this further supports the validity of our method (see SupplementaryMaterials). Indeed, the majority of the Nepalese people (84.7%) practice Hinduism and celebrate multiple festi-vals throughout the year. The main festival, Dashain, is celebrated for two weeks in September or October andis of such importance to the religious and cultural identity of the country, that businesses and other organisationsare completely closed for 10 to 15 days to celebrate. We detect downtime during Dashain for each year in which

    5

  • we have data. Additionally, we detect other festivals that follow Dashain, such as the second largest festival inNepal called Tihar.

    1.2 San Juan, Puerto RicoTo determine the actual downtime in San Juan after Hurricane Maria, we took tourism data from the PuertoRico Tourism company [26], listing the monthly number of cruise passengers in the port of Old San Juan (seeSupplementary Materials). We apply the elbow method (see Section 3.7) to this data and estimate a downtimeof 102 days. From the analysis of Facebook posts, the 118 days of downtime that we predict can be split intodowntime from the Hurricane (103 days), and downtime from the Christmas to New Years period (14 days). Infact, in San Juan, we found periodic downtime events during the periods between Christmas and New Year (seeSupplementary Materials). Compared to Nepal, Puerto Rico has a majority Christian population, which explainsthe downtime during the Christmas period.

    1.3 Juchitán de Zaragoza, MexicoThe events that were found in Juchitán de Zaragoza proved to be the most difficult to verify. Whilst the finaloutput is noisy, we still see downtime during the Chiapas earthquake event, supported by a limited survey sentto the businesses that were collected. Of the businesses that responded, the average closure time reported(ignoring businesses that never reopened) was 63 days. This sits in between our predicted downtime of 52 daysand 66 days (if we also count the 12 consecutive additional days of downtime presumed to be due to the AllSaints festivity, see Supplementary Materials).

    1.4 Sensitivity analysisThe reported downtime for each region was calculated using all of the collected businesses that had posted atleast once. We tested the sensitivity of our method to the overall number of businesses considered. We ob-serve that the methodology gives consistent estimates when the number of businesses is large (i.e. thousands),whereas the length of downtime might be underestimated if the number of businesses is small. We verified thisin Nepal and Puerto Rico, by randomly sampling a subset of businesses and computing the average length ofdowntime over 1000 realisations as a function of the sample size (see Supplementary Materials). Establishinga precise relationship between the number of businesses and the average downtime is difficult because thatestimate depends not only on the number of the businesses but also on other factors, such as their geographicdensity and mean posting rate. Indeed, to assess to what extent the estimate of downtime depends on theposting activity of the businesses, we filtered businesses by their daily posting rate and by the total number ofposts they have made. The results reported in the Supplementary Materials show that the overall downtime ofthe region is not affected by the filtering, except in cases of very high thresholds. In particular, we note that wecan obtain an accurate estimate of the downtime with a small number of businesses (few hundreds) that postfrequently (e.g. more than once per week).

    1.5 Downtime detection in real-timeWhen sufficient data from social media is available, the proposed system can be applied in real-time givingestimates of the recovery status during the weeks immediately after an event. We simulate the collection ofdata in real time by cropping our data in the weeks following the event and we calculate dRT(t), the real-timeestimate of the downtime t weeks after the event, using just the posts published until week t. Results forKathmandu are shown in Figure 4, for the other regions see Supplementary Materials. To evaluate the accuracy

    6

  • of the real-time estimate, we measure the root mean squared distance (RMSD) between dRT(t) and the groundtruth downtime, dGT(t) = min(t, d∗), where d∗ is the downtime estimated using all data (d∗ = 50 days for

    Kathmandu). Computing the RMSD =√∑d∗

    t=0(dRT(t)− dGT(t))2/d∗ from the date of the event t = 0 untilt = d∗, i.e. for a time period of the length of the actual downtime from the date of the event, we obtain 2.75,0.88, 3.5 days for Kathmandu, San Juan and Juchitán de Zaragoza respectively. In all cases the errors are withinthe method’s accuracy of plus/minus one week (see Methods), demonstrating the possibility to obtain accuratereal-time estimates of downtime. The error is larger in Juchitán de Zaragoza because of the larger fluctuationsof the estimates due to a lower sample of businesses.

    Jan2013

    Apr Jul Oct Jan2014

    Apr Jul Oct Jan2015

    Apr0.0

    0.5

    1.0Posts up to 2015-05-23

    Jan2013

    Apr Jul Oct Jan2014

    Apr Jul Oct Jan2015

    Apr0.0

    0.5

    1.0Posts up to 2015-06-20

    Jan2013

    Jan2014

    Jan2015

    Jul Jul Jul

    Date

    0.0

    0.5

    1.0Posts up to 2015-07-18

    Apr2015

    May Jun Jul Aug Sep Oct Nov

    Date

    0

    10

    20

    30

    40

    50

    Down

    time

    (day

    s)

    Real-time estimate, dRT(t) Ground truth, dGT(t)Event

    Figure 4: Event detection shown in real-time. Data was cropped at different end dates (up to seven months afterthe event), and the process was applied to this data. Shown are the time series for one, two and three monthsafter the event in Kathmandu respectively. The black dashed line indicates the perfect real-time detection for thedowntime predicted using all of the data.

    2 DiscussionA framework has been proposed to determine the recovery status of small businesses in the aftermath of naturaldisasters through the analysis of the businesses’ posting activity on their public pages on the social media siteFacebook.

    This framework relies on the assumption that businesses tend to publish more posts when they are open andfewer when they are closed, hence analysing the aggregated posting activity of a group of businesses over timeit is possible to infer when they are open or closed. This assumption is verified in three case studies, where wesuccessfully estimate the downtime of businesses in three urban areas affected by different natural disasters:the 2015 Gorkha earthquake in Nepal, the 2017 Chiapas earthquake in Mexico and the 2017 hurricane Mariain Puerto Rico. We validate our predictions using various data sources: field interviews, official tourism data,Facebook surveys and other studies available in literature.

    7

  • The methodology proposed has the potential to be highly informative to predict the return to activity of busi-nesses in a specific area with an accuracy of plus/minus one week, provided that there is a sufficient use ofFacebook by the businesses in the area. Given the widespread diffusion of social media websites, the methodcan be easily applied to multiple regions at the same time, enabling to nowcast the recovery status of areasaffected by natural and artificial disasters on a global scale. The proposed methodology for the detection ofanomalous events in non-stationary time series is of general applicability to all cases where the main signalis composed by the aggregation of a large number of individual components, for example phone users’ callingactivities or internet users’ visits to web pages.

    3 Materials and Methods

    3.1 Natural disastersWe report a short description for each natural disaster:

    2015 Gorkha Earthquake (25-04-2015; Kathmandu, Nepal).

    According to worldwide news results, the 2015 Gorkha Earthquake was one of the worst natural disasters inNepalese history, killing around 9,000 people, and injuring nearly 22,000 people. It occurred on April 25th, witha magnitude of Mw 7.8. Its epicentre was located 77km North West of Kathmandu (east of the Gorkha district).Hundreds of aftershocks were reported following the earthquake, five of which registered Mw above 6. The mostnotable aftershock occurred on the 12th of May, killing a further 200 people and injuring a further 2,500. 8.1million Nepalese citizens were thought to be affected by the earthquake [27]. The International Monetary Fund[28] label Nepal as an emerging and developing country. The event affected significantly the all country withdetected damage in the city of Kathmandu especially in the North West area of the city [29].

    3.1.1 Chiapas Earthquake (07-09-2017; Chiapas, Mexico).

    The 2017 Mw 8.2 Chiapas Earthquake was the second strongest earthquake recorded in Mexico’s history (andmost intense globally in 2017), triggered in the Gulf of Tehuantepec, just off the south coast of Mexico. Multiplebuildings in the city closest to the epicentre, Juchitán de Zaragoza, were reduced to rubble with people sleepingoutdoors due to fears of further damage [30]. The main hospital in Juchitán also collapsed causing severedisruption to medical services in the area, with 90 reported dead in Mexico. Food shops were also affected,with prices rises due to closures, and fears from looting causing more closures. According to local authorities,roughly 30% of houses were damaged in the earthquake. This is probably aggregated by the lack of earthquakeresilient houses, predominantly made of adobe block with concrete bond beam and mud wall constructions [31].$150 million dollars was transferred to the National Disaster Fund (FONDEN) to aid health, education and roadsinfrastructure/reconstruction efforts [32].

    3.1.2 Hurricane Maria (20-09-2017; San Juan, Puerto Rico).

    Puerto Rico, along with Florida and the US Virgin Islands, was hit by a Category 5 hurricane on September 20th,2017 causing significant damage to infrastructure [33], affecting 100% of the population [34], with an estimated45% of islanders being without power for three months after the event [35]. Hurricane Maria is estimated to havecaused $94 billion in damage, with as many as 60,000 homes still lacking roofs as of December 2016 [36]. Asof August 2018, FEMA have approved 50,650 loans to small businesses proving $1.7 billion to aid in recovery,in total obliging $21.8 billion in relief efforts [34].

    8

  • 3.2 Data collectionOn the social media website Facebook, businesses can set up their own pages (called “walls”) to advertiseand interact with users, with approximately 65 million local businesses pages created as of 2017 [37]. Thesebusiness pages include the latitude and longitude of the business, allowing for the possibility of spatial analysison the posts created in a given region.

    Publicly available business data has been retrieved from Facebook via the Facebook Graph API, in areasaffected by natural disasters; we found 11,818 businesses that published 1.1m posts in Kathmandu, NP, 10,894businesses that published 2.2m posts in San Juan, PR and 1,728 businesses that published 62k posts in Juchitánde Zaragoza, MX. The locations of the businesses in the three regions considered are shown in Figure 1a-c.It should be noted that a visible post on the wall of a business can be made by the owner of the business, ora member of the public. Facebook differentiates these messages by offering API access to the wall’s posts(messages made by the owner) or the wall’s feed (messages made by everyone, including the author). For thepurpose of this study, we collect only the messages made by the owner of the page.

    Once businesses’ pages had been collected from Facebook, an additional set of queries were ran using theAPI to collect the time-stamps in which each business made a post. The contents of the posts were not stored.

    3.3 Data processingWe define the aggregated time series describing the behaviour of the entire region, r(t), as the time series ofthe total number of posts made by all the businesses, B:

    r(t) =∑

    i∈Bxi(t) (1)

    where xi(t) is the number of posts made by business i on day t. The raw time series r(t) display a reducedactivity during a period of time following the disasters, as shown in Figure 1d-f. However, it is difficult to estimatethe significance and the length of the downtime from the raw time series because it is non-stationary. In particu-lar, there is an increasing trend in the average number of posts, due to widespread adoption to Facebook by localbusinesses over time. Additionally, the time series present various seasonality effects, such as reduced postingrates during holidays like Christmas and New Year in Puerto Rico, which can be more or less pronounced ondifferent years. Some of these recurring patterns are quasi-periodic, for example Dashain Festival in Nepal doesnot occur on the same day every year, and this makes it harder to remove them. Moreover, businesses havevery different posting rates, with some being up to two orders of magnitude more active than the average. Suchvery active businesses can introduce significant biases and lead to over or under estimate the activity level of theentire region, when the overall number of businesses is small. This is the case of the activity spikes in Juchitánde Zaragoza in 2015, which are caused by a very small number of businesses.

    To account for these issues, we develop a method to transform the raw time series into a detrended andrescaled series that allows us to clearly identify the periods of anomalous activity and measure their length.The methodology to process the time series is composed of four steps: (1) Single business Probability IntegralTransformation (PIT) and aggregation, (2) Shift and rescale, (3) Mean and variance correction, (4) AggregatedProbability Integral Transformation.

    Step 1: Single business PIT and aggregation.In step (1) the raw time series of the posts of each business is transformed into the time series of the correspond-ing “mid-quantiles”. Formally, this is obtained using a procedure similar to the Probability Integral Transform (PIT)[38]. Let xi(t) be the number of posts of business i on a given day t and let PXi(x) be the empirical CumulativeDistribution Function (CDF) denoting the fraction of days of the current year when business i made less than xposts. We define the corresponding mid-quantile for xi(t) as qi(t) = [PXi(xi(t)) + PXi(xi(t) + 1)]/2. We use

    9

  • the CDF of the current year, instead of considering the business’ entire lifetime, to take into account long-termchanges in a business’ posting behaviour. Using the mid-quantile variable q instead of the raw number of postsx helps to reduce the bias due to outliers, for example days with an unusually high number of posts, and thebias caused by businesses with mean posting rates much higher than the average. Aggregating the transformeddata, we have the time series

    rPIT (t) =∑

    i∈Bqi(t) . (2)

    2013 2014 2015 2016 2017 20180

    50010001500

    r(t)

    (a) Raw aggregate data (Kathmandu, Nepal)

    2013 2014 2015 2016 2017 20180

    2500

    r PIT

    (t)

    (b) Step 1 : Single businesses PIT and aggregation

    2013 2014 2015 2016 2017 201810

    010

    r N(t)

    (c) Step 2 : Shift and Rescale

    2013 2014 2015 2016 2017 20180

    1

    r U(t)

    (d) Step 4 : Aggregated PIT

    0.0 0.2 0.4 0.6 0.8threshold T

    0

    20

    40

    60

    80

    100

    120

    num

    ber o

    f eve

    nts

    f(T)

    (e)y = f(T)y = TT* = argmax[ T f(T)]

    Figure 5: Steps of the process to estimate the downtime of small businesses applied to Kathmandu, Nepal. (a)Raw data (b) PIT transformed data (c) Shifted and rescaled data (d) Probability Integral Transform applied on theaggregated and transformed time series (e) Threshold and event detection.

    3.4 Step 2: Shift and rescale.After the PIT, the mid-quantiles qi(t) are expected to be uniformly distributed between 0 and 1. Hence, theaggregated transformed data rPIT (t) of Equation 2 is expected to follow the Irwin-Hall distribution, which isthe distribution of the sum of n(t) uncorrelated Uniform random variables between 0 and 1. Here n(t) is thetotal number of businesses with a Facebook account on day t. When n(t) is large, the Central Limit Theoremensures that the distribution of rPIT (t) is well approximated by a Normal distribution with mean n(t)/2 andvariance n(t)/12. Hence, by appropriately shifting and rescaling each day of rPIT (t) we define the time series

    rN (t) = (rPIT (t)− n(t)/2)/√n(t)/12 , (3)

    10

  • whose distribution is a Standard Normal with mean 0 and variance 1 for all days t. To perform the transformationof Equation 3 we have to estimate n(t). The number of businesses with an active Facebook account on agiven day is estimated by recording the first and last post date for each business i, denoted as t(f)i and t

    (l)i

    respectively. We make the assumption that a business remains active between these dates, plus a period of twomore days to be sure that it has closed. This period was chosen to give an unbiased estimate of the number ofactive businesses at the tail end of the data collected. Formally, we estimate the number of businesses with anactive Facebook account on day t as n(t) = |{i : t(f)i ≤ t ≤ t

    (l)i + 2}|. The time series rN (t) for Kathmandu is

    shown in Figure 5c and those for the other regions considered are shown in the Supplementary Materials.

    3.5 Step 3: Mean and variance correction.The assumption that the businesses’ posting activities are independent and the estimate of the number of busi-nesses on Facebook n(t) are reasonable, but may not be perfectly exact. In fact, a small correlation betweenbusinesses’ posting activity may be present even during normal (non-emergency and non-holiday) conditionsand this would affect the variance of rPIT . On the other hand, deviations in the estimate of n(t) could alsochange the mean of rPIT . To correct for these possible sources of bias, we fit and remove a linear trend fromrN (t), so that its mean becomes zero, and we divide it by its standard deviation, so that its variance becomesone. The resulting corrected time series, r̃N (t), for the three regions are shown in the Supplementary Materials.

    3.6 Step 4: Aggregated PIT.Finally, we apply the Probability Integral Transform to the aggregated time series of normally distributed variables,r̃N (t), to obtain a time series of variables uniformly distributed between 0 and 1. This is done using the CDF ofthe Standard Normal distribution, PN :

    rU (t) = PN (r̃N (t)) . (4)

    The final time series rU (t) for Kathmandu is shown in Figure 5d and those for all the regions are shown inFigure 3. The four steps of the data processing for all the regions considered are shown in the SupplementaryMaterials.

    3.7 Downtime detectionThe length of downtime is estimated as follows.

    We consider the rolling weekly mean of the transformed time series rU (t). This is primarily to account fordiffering posting behaviour over the course of a week (e.g. weekdays vs. weekends). We define an event as aperiod of time when the weekly mean of rU (t) is below a given threshold value, T , for more than 7 consecutivedays. The value of the threshold is set using a method aimed at detecting all periods when posting activity issignificantly below the characteristic range of normal activity fluctuations. This value is found using the following“elbow method”. First, the number of potential downtime events are recorded at multiple threshold values; thenumber of events for a given threshold value is denoted as f(T ) and it is shown in Figure 5e for Kathmandu.

    Second, we identify the threshold value, T ∗, at which the number of events begins to rapidly rise. This pointmarks the transition between the size of activity fluctuations due to anomalous events and the characteristic sizeof normal activity fluctuations. We define T ∗ as the abscissa of the point on the “elbow” of function f(T ), i.e. thevalue of T that maximises the distance between f(T ) and the line connecting the origin to a point on f such thatits slope, θ, is maximum (see Figure 5e): T ∗ = argmaxT (θT − f(T )). The dashed vertical line in Figure 5edenotes the value of T ∗ for Kathmandu. The events detected in the three regions using this method are shownin Figure 3.

    11

  • 3.8 ValidationResearch field data. Surveys from four areas in Kathmandu were used to verify the results. These surveys werepart of a research field mission in 2016 to examine the extent of the earthquake damage, with regions chosenthat vary in population density, construction methods and traffic concentration [11]. These surveys record thelength of downtime for each business, along with the reason for closure. Regions A and B both report damageas the main reason for business closure. Regions C and D both report scare as reason for closure in the region,it was reported that shop owners feared another aftershock and remained closed until the risk of danger wasreduced.

    Tourism data. Historical tourism data has been retrieved from the Puerto Rico Tourism Company [26],listing the cruise passenger movement in port in Old San Juan. We compute the Year over Year (YoY) change,preserving seasonal trends, calculating the percentage change of each month to the same month in the previousyear.

    Facebook surveys. To collect validation for the region collected in Mexico, we sent surveys to the recordedbusinesses asking the question “Were you affected by the disaster, and if so, for how long were you closed?”.We sent this survey to 52 of the businesses found via Facebook in Juchitán de Zaragoza, with a response rateof 30% (16 responses).

    4 Data availabilityThe raw data that supports the findings of this study is not available due to the Facebook Platform Policy. De-rived (anonymised) data that supports the findings of this study are available from the authors upon reasonablerequest.

    5 Author contributionsAll authors contributed to the conception and design of the study. FDL conducted the field interviews, REcollected the social media data and analysed the results. All authors wrote and reviewed the manuscript.

    6 Competing interestsThe authors declare no competing interests.

    7 AcknowledgementsRE is grateful for the support received by EPSRC and the 2017/18 Cabot Institute Innovation Fund. FDL ac-knowledges the support of the Leverhulme Trust (RPG2017-006, GENESIS project). FS is supported by EPSRCFirst Grant EP/P012906/1.

    References[1] D. M. Neal, “Reconsidering the phases of disasters,” International Journal of Mass Emergencies and Dis-

    asters, vol. 15, no. 2, pp. 239–264, 1997.

    12

  • [2] D. E. Alexander, Principles of emergency planning and management. Oxford University Press on Demand,2002.

    [3] R. B. Olshansky, L. D. Hopkins, and L. A. Johnson, “Disaster and recovery: Processes compressed in time,”Natural Hazards Review, vol. 13, pp. 173–178, aug 2012.

    [4] J. Horney, C. Dwyer, M. Aminto, P. Berke, and G. Smith, “Developing indicators to measure post-disastercommunity recovery in the united states,” Disasters, vol. 41, p. 124–149, Mar 2016.

    [5] M. C. Comerio, “Estimating downtime in loss modeling,” Earthquake Spectra, vol. 22, pp. 349–365, may2006.

    [6] S. E. Chang, “Urban disaster recovery: a measurement framework and its application to the 1995 kobeearthquake,” Disasters, vol. 34, p. 303–327, Mar 2010.

    [7] A. Harding and F. Powell, “Variations in pedestrian traffic count in christchurch due to the september 2010darfield (canterbury) earthquake,” in Proceedings of the Ninth Pacific Conference on Earthquake Engineer-ing: Building an Earthquake-Resilient Society, pp. 14–16, 2011.

    [8] R. Campanella, “Street survey of business reopenings in post-katrina new orleans,” 2007.

    [9] M. Hino, S. T. Belanger, C. B. Field, A. R. Davies, and K. J. Mach, “High-tide flooding disrupts local economicactivity,” Science advances, vol. 5, no. 2, p. eaau2736, 2019.

    [10] E. Booth, K. Saito, R. Spence, G. Madabhushi, and R. T. Eguchi, “Validating assessments of seismicdamage made from remote sensing,” Earthquake Spectra, vol. 27, p. S157–S177, Oct 2011.

    [11] F. De Luca, D. Aldamen, J. Kurton, M. Wray, R. Mohan Pokhrel, and M. Werner, “Traffic data as proxyof business downtime after natural disasters: The case of kathmandu,” in 11th National Conference onEarthquake Engineering, 6 2018.

    [12] D. Murthy and A. J. Gross, “Social media processes in disasters: Implications of emergent technology use,”Social Science Research, vol. 63, p. 356–370, Mar 2017.

    [13] T. Preis, H. S. Moat, S. R. Bishop, P. Treleaven, and H. E. Stanley, “Quantifying the digital traces of hurricanesandy on flickr,” Scientific Reports, vol. 3, Nov 2013.

    [14] N. Tkachenko, S. Jarvis, and R. Procter, “Predicting floods with flickr tags,” PLOS ONE, vol. 12,p. e0172870, Feb 2017.

    [15] L. Zou, N. S. N. Lam, H. Cai, and Y. Qiang, “Mining twitter data for improved understanding of disasterresilience,” Annals of the American Association of Geographers, vol. 108, p. 1422–1441, Mar 2018.

    [16] P. S. Earle, D. C. Bowden, and M. Guy, “Twitter earthquake detection: earthquake monitoring in a socialworld,” Annals of Geophysics, vol. 54, no. 6, 2012.

    [17] S. Vieweg, A. L. Hughes, K. Starbird, and L. Palen, “Microblogging during two natural hazards events,”Proceedings of the 28th international conference on Human factors in computing systems - CHI ’10, 2010.

    [18] J. Sutton, E. S. Spiro, B. Johnson, S. Fitzhugh, B. Gibson, and C. T. Butts, “Warning tweets: Serial trans-mission of messages during the warning phase of a disaster event,” Information, Communication & Society,vol. 17, no. 6, pp. 765–787, 2014.

    [19] Y. Kryvasheyeu, H. Chen, N. Obradovich, E. Moro, P. Van Hentenryck, J. Fowler, and M. Cebrian, “Rapidassessment of disaster damage using social media activity,” Science Advances, vol. 2, p. e1500779, Mar2016.

    [20] J. P. Bagrow, D. Wang, and A.-L. Barabási, “Collective response of human populations to large-scale emer-gencies,” PLoS ONE, vol. 6, p. e17680, Mar 2011.

    [21] P. Deville, C. Linard, S. Martin, M. Gilbert, F. R. Stevens, A. E. Gaughan, V. D. Blondel, and A. J. Tatem,“Dynamic population mapping using mobile phone data,” Proceedings of the National Academy of Sciences,vol. 111, p. 15888–15893, Oct 2014.

    13

  • [22] R. Wilson, E. zu Erbach-Schoenberg, M. Albert, D. Power, S. Tudge, M. Gonzalez, S. Guthrie, H. Chamber-lain, C. Brooks, C. Hughes, and et al., “Rapid and near real-time assessments of population displacementusing mobile phone data following disasters: The 2015 nepal earthquake,” PLoS Currents, 2016.

    [23] X. Lu, L. Bengtsson, and P. Holme, “Predictability of population displacement after the 2010 haiti earth-quake,” Proceedings of the National Academy of Sciences, vol. 109, p. 11576–11581, Jun 2012.

    [24] OpenStreetMap contributors, “Planet dump retrieved from https://planet.osm.org .” https://www.openstreetmap.org, 2018.

    [25] G. Boeing, “Osmnx: New methods for acquiring, constructing, analyzing, and visualizing complex streetnetworks,” Computers, Environment and Urban Systems, vol. 65, p. 126–139, Sep 2017.

    [26] “Estadísticas.PR: Instituto de estadísticas de pr.” https://www.estadisticas.pr. Accessed: 2019-02-11.

    [27] Office of the Resident Coordinator, “Nepal : Earthquake 2015,” vol. 05, no. 05, pp. 1–9, 2015.

    [28] International Monetary Fund, “World economic outlook database,” World Econ Financ Surv, 2018.

    [29] K. Goda, T. Kiyota, R. M. Pokhrel, G. Chiaro, T. Katagiri, K. Sharma, and S. Wilkinson, “The 2015 gorkhanepal earthquake: Insights from earthquake damage survey,” Frontiers in Built Environment, vol. 1, jun2015.

    [30] ACAPS, “MEXICO AND GUATEMALA Earthquake Epicentre: Pacific Ocean, 8km southwest of Pijijia-pan, Chiapas state in Mexico Magnitude in Mexico: 8.1 Magnitude in Guatemala: 7.6 Crisis overview,”no. September, pp. 1–6, 2017.

    [31] UNITAR-UNOSAT, “Pijijiapan earthquake M w 8 . 1 , Mexico Population Exposure in Mexico - PopulationExposure Analysis,” vol. 4020, no. September, 2017.

    [32] UNICEF, “Mexico Humanitarian Situation Report No.6,” no. 6, pp. 1–4, 2017.

    [33] United States Department of Energy, Infrastructure Security and Energy Restoration, “Hurricanes Maria,Irma, and Harvey September 20 Event Summary (Report #39),” tech. rep., 2017.

    [34] FEMA, “2017 Hurricane Season FEMA After-Action Report,” 2018.

    [35] J. Shermeyer, “Assessment of electrical and infrastructure recovery in puerto rico following hurricane mariausing a multisource time series of satellite imagery,” in Earth Resources and Environmental Remote Sens-ing/GIS Applications IX, vol. 10790, p. 1079010, International Society for Optics and Photonics, 2018.

    [36] “Quick facts: Hurricane maria’s effect on puerto rico january 19, 2018,” 2018.

    [37] Facebook, “Helping small businesses succeed in a mobile world.” Accessed: 2018-07-24.

    [38] J. E. Angus, “The probability integral transform and related results,” SIAM Review, vol. 36, p. 652–654, Dec1994.

    14

  • Social media usage reveals how regionsrecover after natural disaster: Supplementary

    Materials

    Robert Eyre, Flavia de Luca, Filippo Simini

    1 Data collectionAs Facebook’s search does not return every business from a given location andsearch radius, we divided each region into 250m sections and pass over thesesections when searching (instead of searching the whole region). Searches aremade at radii 250m, 375m and 500m to ensure as many businesses are returned aspossible. Duplicate results were then subsequently removed.

    2 Survey locationsWe highlight the locations in which surveys were taken in Kathmandu, Nepal. 92surveys were taken in total. To validate our results, we also ran the describedprocess on each area, finding all businesses in the smallest circle to contain allsurveys from the centre of the survey region (Figure 1). We find 532 businesses inregion A, 163 businesses in region B, 24 businesses in region C, and 73 businessesbusiness in region D. We report the average downtime in each region in Table 1.

    3 Process for each regionWe show the framework applied to the two natural disasters not shown in the maintext (Figure 2, Figure 3).

    1

    arX

    iv:1

    908.

    0738

    4v1

    [ph

    ysic

    s.so

    c-ph

    ] 2

    0 A

    ug 2

    019

  • 4 Variance correctionWe make the assumption of independence between businesses’ posting habits,and also the assumption that businesses are always active between their first andlast post date. Whilst this is reasonable, it does not always hold - for examplecorrelation between businesses’ posting activity may be present during normal(non-emergency and non-holiday) conditions, affecting the variance of rPIT . Alsodeviations in the estimate of n(t), where we have not perfectly predicted the num-ber of active businesses could also change the mean of rPIT . To correct for thesepossible sources of bias, we fit and remove a linear trend from rN(t), so that itsmean becomes zero, and we divide it by its standard deviation, so that its variancebecomes one.

    5 Choice of business filterThe reported downtime for each region was calculated using all of the collectedbusinesses that had posted at least once. We can filter businesses by the rate atwhich they post and the total number of posts they have made to determine howdependent the process is on business “quality”. In the case of Kathmandu, ourpredictions stay relatively consistent (Table 5), with less downtime being reportedas we introduce higher filters - the majority of businesses that meet the criteriain these cases are located in the most densely commercially populated area in thecentre of the city (Figure 7).

    6 Business samplingWe highlight the impact that under-sampling a region can have on the final predic-tion for the entire region. For each sample size, we draw 1000 realisations (withreplacement) and computer the average downtime predicted using our framework.In Kathmandu we are able to see the downtime for the whole region (within onestandard deviation from the mean) within 3000 businesses. In San Juan we areable to see the downtime for the whole region (within one standard deviation fromthe mean) within 2000 businesses (Figure 9).

    2

  • 7 Validation, San JuanTo estimate the true downtime in San Juan, we look at the Year over Year reportedincoming tourists via the port in Old San Juan [1].

    8 Downtime detectionThe start date, end date and the length of downtime of all the events detected inthe three regions analysed are reported in Table 2 (Kathmandu, Nepal), Table 3(San Juan, Puerto Rico), Table 4 (Juchitán de Zaragoza, Mexico).

    9 Real time detectionWe show the downtime predicted for different cut-off dates in the regions of SanJuan (Figure 10) and Juchit/’an de Zaragoza (Figure 11).

    References[1] “Estadı́sticas.PR: Instituto de estadı́sticas de pr.” https://www.

    estadisticas.pr. Accessed: 2019-02-11.

    [2] F. De Luca, D. Aldamen, J. Kurton, M. Wray, R. Mohan Pokhrel, andM. Werner, “Traffic data as proxy of business downtime after natural dis-asters: The case of kathmandu,” in 11th National Conference on EarthquakeEngineering, 6 2018.

    3

  • List of Figures1 Survey locations in the study by De Luca et al. [2] . . . . . . . . . 52 Steps of the process applied to San Juan. (a) Raw data (b) PIT

    transformed data (c) Shifted and rescaled data (d-e) Threshold andevent detection. . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

    3 Steps of the process applied to Juchitán de Zaragoza. (a) Rawdata (b) PIT transformed data (c) Shifted and rescaled data (d-e)Threshold and event detection. . . . . . . . . . . . . . . . . . . . 7

    4 Variance correction applied to rN(t) in Kathmandu, Nepal. . . . . 85 Variance correction applied to rN(t) in San Juan, Puerto Rico . . . 96 Variance correction applied to rN(t) in Juchitán de Zaragoza, Mex-

    ico. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Imposing a high threshold on the quality of the business (at least

    1000 posts, at least 1 post per day), the majority of the businessesfound are in the centre of Kathmandu, meaning that the final esti-mate is not representative of the whole region. . . . . . . . . . . . 11

    8 Normalised Year-Over-Year tourism data in San Juan (solid) vsnormalised signal data from our framework (dotted). . . . . . . . 12

    9 Business samples of different sizes are taken in Kathmandu, Nepal(left) and San Juan, Puerto Rico (right). Downtimes are calcu-lated, and then this process is repeated. Average downtimes shownwith two standard deviations from the mean. . . . . . . . . . . . 13

    10 Event detection shown in real-time. Data was cropped at differentend dates (up to seven months after the event), and the processwas applied to this data. Shown are the time series for one, twoand three months after the event in San Juan respectively. . . . . 14

    11 Event detection shown in real-time. Data was cropped at differentend dates (up to seven months after the event), and the process wasapplied to this data. Shown are the time series for one, two andthree months after the event in Juchitán de Zaragoza respectively. 15

    4

  • Figure 1: Survey locations in the study by De Luca et al. [2]

    5

  • 2013 2014 2015 2016 2017 2018

    100020003000

    r(t)

    (a) Raw aggregate data (San Juan, Puerto Rico)

    2013 2014 2015 2016 2017 20180

    10002000

    r PIT

    (t)

    (b) Step 1 : Single businesses PIT and aggregation

    2013 2014 2015 2016 2017 201820

    0

    r N(t)

    (c) Step 2 : Shift and Rescale

    2013 2014 2015 2016 2017 20180

    1

    r U(t)

    (d) Step 4 : Aggregated PIT

    0.0 0.2 0.4 0.6 0.8threshold T

    0

    20

    40

    60

    80

    num

    ber o

    f eve

    nts

    f(T)

    (e)y = f(T)y = TT* = argmax[ T f(T)]

    Figure 2: Steps of the process applied to San Juan. (a) Raw data (b) PIT trans-formed data (c) Shifted and rescaled data (d-e) Threshold and event detection.

    6

  • 2013

    2014

    2015

    2016

    2017

    2018

    0

    250

    r(t)

    (a) Raw aggregate data (Juchitán de Zaragoza, Mexico)

    2013 2014 2015 2016 2017 20180

    100

    r PIT

    (t)

    (b) Step 1 : Single businesses PIT and aggregation

    2013 2014 2015 2016 2017 20182.50.02.5

    r N(t)

    (c) Step 2 : Shift and Rescale

    2013 2014 2015 2016 2017 20180

    1

    r U(t)

    (d) Step 4 : Aggregated PIT0.0 0.2 0.4 0.6 0.8

    threshold T

    0

    50

    100

    150

    200

    num

    ber o

    f eve

    nts

    f(T)

    (e) Threshold detectiony = f(T)y = TT* = argmax[ T f(T)]

    Figure 3: Steps of the process applied to Juchitán de Zaragoza. (a) Raw data(b) PIT transformed data (c) Shifted and rescaled data (d-e) Threshold andevent detection.

    7

  • 250

    2501: Shift mean by n/2

    4 0 4 4 0 4

    Distribution of daily values

    4 0 4

    10

    102: Rescale variance by n/12

    4 0 4 4 0 4 4 0 4

    10

    10

    Tran

    sfor

    med

    val

    ue

    3: Remove linear trend (zero mean)

    4 0 4norm

    ed d

    ensi

    ty

    4 0 4 4 0 4

    3

    34: Unit variance

    4 0 4 4 0 4 4 0 4

    2013 2014 2015 2016 2017 2018Date

    0

    15: Transform to uniform

    0 1 0 1histogram bins

    0 1

    Variance correction (Kathmandu, Nepal)

    Figure 4: Variance correction applied to rN(t) in Kathmandu, Nepal.

    8

  • 250

    2501: Shift mean by n/2

    4 0 4 4 0 4

    Distribution of daily values

    4 0 4

    20

    202: Rescale variance by n/12

    4 0 4 4 0 4 4 0 4

    20

    20

    Tran

    sfor

    med

    val

    ue

    3: Remove linear trend (zero mean)

    4 0 4norm

    ed d

    ensi

    ty

    4 0 4 4 0 4

    3

    34: Unit variance

    4 0 4 4 0 4 4 0 4

    2013 2014 2015 2016 2017 2018Date

    0

    15: Transform to uniform

    0 1 0 1histogram bins

    0 1

    Variance correction (San Juan, Puerto Rico)

    Figure 5: Variance correction applied to rN(t) in San Juan, Puerto Rico

    9

  • 10

    101: Shift mean by n/2

    4 0 4 4 0 4

    Distribution of daily values

    4 0 4

    3

    32: Rescale variance by n/12

    4 0 4 4 0 4 4 0 4

    3

    3

    Tran

    sfor

    med

    val

    ue

    3: Remove linear trend (zero mean)

    4 0 4norm

    ed d

    ensi

    ty

    4 0 4 4 0 4

    3

    34: Unit variance

    4 0 4 4 0 4 4 0 4

    2013 2014 2015 2016 2017 2018Date

    0

    15: Transform to uniform

    0 1 0 1histogram bins

    0 1

    Variance correction (Juchitán de Zaragoza, Mexico)

    Figure 6: Variance correction applied to rN(t) in Juchitán de Zaragoza, Mex-ico.

    10

  • 85.27 85.39longitude

    27.64

    27.75

    latit

    ude

    (a) p > 0, r > 0

    85.27 85.39longitude

    27.64

    27.75

    latit

    ude

    (b) p > 1000, r > 1

    Kathmandu, Nepal

    Figure 7: Imposing a high threshold on the quality of the business (at least1000 posts, at least 1 post per day), the majority of the businesses found are inthe centre of Kathmandu, meaning that the final estimate is not representativeof the whole region.

    11

  • 2014 2015 2016 2017 2018Date

    0.0

    0.2

    0.4

    0.6

    0.8

    1.0

    Nor

    mal

    ised

    dat

    aYoY tourism dataFacebook dataHurricane MariaDowntime end (Facebook)Downtime end (tourism)

    Figure 8: Normalised Year-Over-Year tourism data in San Juan (solid) vs nor-malised signal data from our framework (dotted).

    12

  • 0 2000 4000 6000 8000 10000Samples

    10

    0

    10

    20

    30

    40

    50

    Dow

    ntim

    e (d

    ays)

    Kathmandu, Nepal

    Downtime±± 2

    0 2000 4000 6000 8000Samples

    0

    20

    40

    60

    80

    100

    120D

    ownt

    ime

    (day

    s)

    San Juan, Puerto Rico

    Figure 9: Business samples of different sizes are taken in Kathmandu, Nepal(left) and San Juan, Puerto Rico (right). Downtimes are calculated, and thenthis process is repeated. Average downtimes shown with two standard devia-tions from the mean.

    13

  • 2013 2014 2015 2016 20170.0

    0.5

    1.0Posts up to 2017-10-18

    2013 2014 2015 2016 20170.0

    0.5

    1.0Posts up to 2017-11-15

    2013 2014 2015 2016 2017Date

    0.0

    0.5

    1.0Posts up to 2017-12-13

    Sep Oct Nov Dec Jan2018

    Feb Mar Apr

    Date

    0

    20

    40

    60

    80

    100

    120

    Down

    time

    (day

    s)

    Real-time estimate, dRT(t) Ground truth, dGT(t)Event

    Figure 10: Event detection shown in real-time. Data was cropped at differentend dates (up to seven months after the event), and the process was applied tothis data. Shown are the time series for one, two and three months after theevent in San Juan respectively.

    14

  • 2013 2014 2015 2016 20170.0

    0.5

    1.0Posts up to 2017-10-05

    2013 2014 2015 2016 20170.0

    0.5

    1.0Posts up to 2017-11-02

    2013 2014 2015 2016 2017Date

    0.0

    0.5

    1.0Posts up to 2017-11-30

    Sep Oct Nov Dec Jan2018

    Feb Mar

    Date

    0

    10

    20

    30

    40

    50

    60

    70

    Down

    time

    (day

    s)

    Real-time estimate, dRT(t) Ground truth, dGT(t)Event

    Figure 11: Event detection shown in real-time. Data was cropped at differentend dates (up to seven months after the event), and the process was applied tothis data. Shown are the time series for one, two and three months after theevent in Juchitán de Zaragoza respectively.

    15

  • List of Tables1 Downtime (days) reported in four areas in Kathmandu, Nepal,

    from the surveys taken by De Luca et al. [2] . . . . . . . . . . . . 172 Detected events in Kathmandu, Nepal . . . . . . . . . . . . . . . 183 Detected events in San Juan, Puerto Rico. . . . . . . . . . . . . . 194 Detected events in Juchitán de Zaragoza, Mexico. . . . . . . . . . 205 Overall downtime reported over the whole of Kathmandu, Nepal,

    with different filters on the businesses (by the number of poststhey have made, and the daily average posting rate). Number ofbusinesses that meet the criterion are listed in brackets for eachreported downtime. . . . . . . . . . . . . . . . . . . . . . . . . . 21

    6 Overall downtime reported over the whole of San Juan, PuertoRico with different filters on the businesses (by the number ofposts they have made, and the daily average posting rate). Numberof businesses that meet the criterion are listed in brackets for eachreported downtime. . . . . . . . . . . . . . . . . . . . . . . . . . 22

    7 Overall downtime reported over the whole of Juchitán de Zaragoza,Mexico, with different filters on the businesses (by the number ofposts they have made, and the daily average posting rate). Num-ber of businesses that meet the criterion are listed in brackets foreach reported downtime. . . . . . . . . . . . . . . . . . . . . . . 23

    16

  • Mean downtime Standard deviation Number of surveys

    Area A 29 44 27Area B 15 20 31Area C 61 88 25Area D 78 62 10All regions 41 62 94

    Table 1: Downtime (days) reported in four areas in Kathmandu, Nepal, from thesurveys taken by De Luca et al. [2]

    17

  • Start date End date Duration (days) Event

    2013-10-13 2013-10-31 19 Dashain 2013 (5th Oct - 17th Oct)2013-11-06 2013-11-27 22 Tihar 2013 (1st Nov - 5th Nov)2014-01-07 2014-01-20 14 Unknown2014-10-03 2014-10-14 12 Dashain 2014 (24th Sep - 7th Oct)2015-04-27 2015-06-15 50 Gorkha Earthquake (25th April)2015-10-21 2015-11-02 13 Dashain 2015 (13th Oct - 26th Oct)2016-10-10 2016-10-25 16 Dashain 2016 (1st Oct - 15th Oct)2017-05-04 2017-05-21 18 Unknown2017-09-28 2017-10-14 17 Dashain 2017 (20th Sep - 4th Oct)2017-11-26 2017-12-18 23 Unknown2018-02-04 2018-02-11 8 Unknown

    Table 2: Detected events in Kathmandu, Nepal

    18

  • Start date End date Duration (days) Event

    2014-12-28 2015-01-12 16 Christmas / New Year 20152015-12-29 2016-01-11 14 Christmas / New Year 20162016-12-30 2017-01-09 11 Christmas / New Year 20172017-09-21 2018-01-16 118 Hurricane Maria (20th Sep)

    Table 3: Detected events in San Juan, Puerto Rico.

    19

  • Start date End date Duration (days) Event

    2015-05-26 2015-06-09 15 Unknown2016-11-01 2016-11-09 9 All Saints’ Day / Day of the Dead (1/2 Nov)2017-09-10 2017-10-31 52 M8.2 Chiapas Earthquake (7th Sep)2017-11-02 2017-11-13 12 All Saints’ Day / Day of the Dead (1/2 Nov)2017-11-23 2017-11-30 8 Revolution Day (20th Nov)

    Table 4: Detected events in Juchitán de Zaragoza, Mexico.

    20

  • # posts/Daily post rate 0 1/7 1/2 5/7 1

    1 50(10386) 48(4080) 47(1625) 34(1216) 37(900)100 48(2523) 47(1758) 35(526) 35(297) 51(161)200 47(1391) 47(1288) 36(448) 34(259) 51(147)300 47(952) 47(952) 47(398) 34(236) 51(138)400 46(686) 46(686) 35(363) 34(220) 37(130)

    Table 5: Overall downtime reported over the whole of Kathmandu, Nepal, withdifferent filters on the businesses (by the number of posts they have made, andthe daily average posting rate). Number of businesses that meet the criterion arelisted in brackets for each reported downtime.

    21

  • # posts/Daily post rate 0 1/7 1/2 5/7 1

    1 118(8459) 120(4833) 113(2276) 112(1649) 111(1140)100 120(3462) 120(2916) 113(1292) 112(824) 111(485)200 117(2412) 117(2314) 117(1183) 112(765) 111(457)300 117(1840) 117(1840) 116(1088) 112(724) 111(436)400 117(1513) 117(1513) 113(1021) 112(691) 111(422)

    Table 6: Overall downtime reported over the whole of San Juan, Puerto Ricowith different filters on the businesses (by the number of posts they have made,and the daily average posting rate). Number of businesses that meet the criterionare listed in brackets for each reported downtime.

    22

  • # posts/Daily post rate 0 1/7 1/2 5/7 1

    1 52(552) 48(287) 46(139) 57(104) 46(84)100 48(128) 48(105) 48(34) 48(24) 43(15)200 39(73) 40(71) 44(30) 46(21) 43(13)300 45(49) 45(49) 46(26) 45(18) 80(12)400 10(34) 10(34) 10(23) 47(16) 43(11)

    Table 7: Overall downtime reported over the whole of Juchitán de Zaragoza,Mexico, with different filters on the businesses (by the number of posts they havemade, and the daily average posting rate). Number of businesses that meet thecriterion are listed in brackets for each reported downtime.

    23