Flood Modelling for Cities using Cloud Computing

FINAL REPORT

Vassilis Glenis, Vedrana Kutija, Stephen McGough, Simon Woodman, Chris Kilsby

Summary

Assessment of pluvial flood risk is particularly difficult because it is sensitive to the spatial-

temporal characteristics of rainfall, topography of the terrain and surface flow processes

influenced by buildings and other man-made features. “CityCat” is an urban flood modelling,

analysis and visualisation tool which uses very accurate and computationally efficient

solutions for free surface flow equations. In this pilot project a Cloud Computing compatible

version of “CityCat” was developed and for the first time applied for estimating spatial and

temporal flood risk at a city-scale. The use of Cloud Computing enabled modelling of

flooding of larger domains (up to 1100km2) with much higher resolution (up to 16.000.000

computational cells) than it has been done previously. This project has demonstrated that

the use of Cloud Computing can enable efficient and detailed modelling of flooding at a city

or even regional scale, with high resolution, using standard terrain and rainfall data and

powerful Cloud enabled software which does not impose limitations on the computational

domain size.

Background

The risk from pluvial flooding, where intense direct rainfall overwhelms urban drainage

systems, in the UK cities is considerable and as a result of 2005 and 2007 floods, local

authorities are now required to develop Surface Water Management Plans (SWMPs)

However, assessment of pluvial flood risk is particularly difficult because it is sensitive to the

spatial-temporal characteristics of rainfall, topography of the terrain, local runoff and

surface flow processes influenced by buildings and other man-made features and the

performance of urban drainage systems.

Conventional assessments of urban flood risk are generally carried out at relatively small

scales using commercial software resulting in very restricted coverage in space and number

of design storms (Hunter et al., 2008). Alternatively, simplified codes may be used for city

scales (Neal et al., 2009) .The use of fully detailed numerical codes for larger areas is in its

infancy and the requirement for HPC or cluster facilities means it is limited to institutional

platforms such as Condor grids which are restrictive in terms of power as well as, crucially,

access for non-institutional users in industry. On the other hand, Cloud Computing (CC)

offers (apparently) huge amount of processing power to anyone. Also, Cloud Computing has

been successfully used within the business community allowing on-demand access to this

power at a (relatively) cheap price. The motivation for this project was exploration of

suitability of Cloud Computing for the assessment of city-scale flood risk

“CityCat” is an urban flood modelling, analysis and visualisation tool which uses state of the

art numerical solutions for flow equations in a user-friendly visual environment. Its

numerical solutions of the 2D free surface flow equations are very accurate and

computationally efficient but it has been recognised that users would benefit from further

enhancement of the computational power brought in by Cloud Computing as that will

enable modelling of larger domains, extended simulation periods and/or different future

climate scenarios.

Objectives

A meaningful assessment of city-scale flood risk requires a very large number of simulations

on large domains that cannot be achieved by existing institutional HPC or cluster facilities.

Therefore in this pilot study we developed for the first time a truly city-scale application of

the hydrodynamic model “CityCat” for estimating spatial and temporal flood risk using

Cloud Computing.

This has been achieved through the following objectives:

1. Porting the existing state-of-the-art hydrodynamic model ”CityCat” from desktop to

Cloud

2. Selecting and using only readily available data sets for “CityCat” simulations so that

this kind of flood risk assessment can be easily applied nationwide

3. Generating a large number of design storms with different return periods and

durations as rainfall input for simulations

4. Applying “CityCat” to different large domains (4km2 to 1100 km2) for a variety of

extreme storm events.

Methodology and results

1. Porting ”CityCat” from desktop to Cloud

Prior to this project, “CityCat” was written in Delphi and compiled only under Windows.

Also, CityCat’s Graphical User Interface (GUI) was used for preparation and visualisation of

results. Therefore in order to run “CityCat” on the Cloud the following changes were made:

a. The numerical engine was separated from the GUI and input files were used for setting up the model. Also, the variables: water depth, velocity in the x direction and velocity in the y direction were saved at predetermined time step intervals.

b. A new version of “CityCat” was developed and compiled under Linux because if the Windows version was used then the Windows OS would need to be installed at each PC. That would have increased the cost and due to our limited resources it would in fact have reduced the number of runs we could do.

In this project we adopted a high throughput model of computation on the Cloud in which a

Condor (http://research.cs.wisc.edu/condor/) cluster of nodes were deployed as a set of virtual

machines instances on the Amazon Cloud. Each instance was a standard Ubuntu Linux image

with the addition of the Condor deployment configured to use the large scratch space

provided with these images. A set of parameter sweep jobs were deployed by modifying the

original source code such that each job could be instantiated by passing a single integer

number as part of the command line arguments to the program. This caused the correct

configuration files to be selected. A simple script was used to wrap each job and would first

decompress the files needed for each run before executing the main program and then

compressing the results back up before returning the results to a central Condor computer

on the Cloud. These files were then staged back to computers within Newcastle University.

2. Standard Datasets needed for “CityCat” simulations “CityCat” uses standard datasets for simulations. For the topography the digital terrain model (DTM) is used (see Fig. 1) and the numerical grid is generated automatically using the cell sizes of the DTM.

In addition to the terrain data, “CityCAT” uses the buildings layer from the OS- MasterMap (see Figure2) in order to exclude the buildings footprint from the computational grid. The cells which are removed from the computational grid are characterised as building and depending on different options/properties, the water captured on the roof of each building is either directly distributed to the neighbouring cells or slowly released and distributed to the neighbouring cells if a green roof is introduced. Exclusion of the buildings from the computational domain (Fig.3) improves the ability of the model to capture realistically the flow patterns in urban areas while the different options for the roof

drainage enable assessment of the different adaptation techniques for flood risk reduction. In most of the simulations within this project the buildings were cut-out of the computational domain and the rainfall falling on the roofs was directly distributed to the neighbouring cells. Hydrodynamic simulations are driven by boundary conditions which are “sources” of water in the model. Usually, these are time dependent functions which describe rainfall over the domain or water entering the domain from an external source. External sources of water can either introduce some volume of water in the domain (for example from a burst pipe) or they can force the water level at the boundary of the

Figure 2 -An example of Master Map coverage. Solid representation

Figure 1 - Terrain given by a Digital Terrain Model (DTM)

domain to follow a certain condition (e.g. river water level or tide). In this project most of the simulations were driven by rainfall as most of urban flooding is of that origin. However, two examples of forced water level boundary condition were also tested to explore the

model’s ability to simulate such complex flows.

3. Rainfall events A set of storm events, for different return periods and different storm durations, have been

created following the standard procedure from the Flood Estimation Handbook (FEH) and a

summer profile rainstorm located at Newcastle was used. Modelling different return periods

is necessary if we want to assess different level of risk as a rainfall event with a return

period of n years has probability of 1/n to happen in any given year. The storms with higher

return period are less likely to happen but they are larger in magnitude so should they

happen the flooding would be more likely. Different rainfall durations are used because, it is

not clear which storm duration would be critical for a given situation. The shorter duration

rainfall events usually have larger rainfall intensity but the overall rainfall volume is not very

large, whereas rainfall events with larger duration have larger volumes of rainwater.

Susceptibility to flooding of any particular area is determined by its topography and other

features and depends on the combination of these two factors (rain intensity and total

volume of rain) in a complex way. Therefore, being able to model an extensive range of

rainfall events with different durations, is one

of the clear advantages of the use of Cloud

Computing in comparison with the standard

engineering practice where one storm of a

particular duration has to be chosen as a

critical one.

Altogether, 36 rainfall events were created

and used in simulations. The events covered

6 different return periods (2, 10, 20, 50, 100

and 200 years) and 6 different rainfall

durations (15 minutes, 30 minutes, 1 hour, 2

hours, 3 hours and 6 hours). The duration

0

1

2

3

4

5

6

7

8

9

10

0 5 10 15 20 25 30 35 40 45 50 55 60

Rai

nfa

ll (m

m)

Time (mins)

Figure 3 -Rainfall event no. 21 (duration 60min return period 50yrs)

Figure 3 -A detail of an example “CityCat” computational

domain without buildings

and the return period of each storm event are shown in Table 1 while an example of one

rainfall event is given in Figure 4.

Table 1. Rainfall events

Rainfall event

number

Return period (years)

Duration (min)

Rainfall event

number

Return period (years)

Duration (min)

Rainfall event

(years)

Return period

number

Duration (min)

1 2 15 13 20 15 25 100 15 2 2 30 14 20 30 26 100 30 3 2 60 15 20 60 27 100 60 4 2 120 16 20 120 28 100 120 5 2 180 17 20 180 29 100 180 6 2 360 18 20 360 30 100 360 7 10 15 19 50 15 31 200 15 8 10 30 20 50 30 32 200 30 9 10 60 21 50 60 33 200 60

10 10 120 22 50 120 34 200 120 11 10 180 23 50 180 35 200 180 12 10 360 24 50 360 36 200 360

4. Application of CityCat to different domains Within this project “CityCat” has been applied to large areas and for extensive duration of

simulations to test its limits and to be able to ascertain the benefits of the use of Cloud

Computing.

Three different domains, ranging in size from 4km2 to 1100km2 were used, with even the

smallest of them being much larger than the domains used in current engineering practice.

Additionally, for one of the domains, 4 different grid sizes were used which resulted in very

different model sizes. Most of the models were then run in parallel for multiple rainfall

events. All these simulations required very different memory, CPU effort and total run time

and benefitted in different ways from Cloud Computing. In Table 2, a summary of all the

runs performed on the Cloud is given and more detailed analysis for each simulation is

presented below.

Table2. Summary of all the simulations

Domain Area Cell size

Number of cells

Boundary conditions

Event duration Number of runs

1 Newcastle city centre

4km2 2m 1.000.000 Rainfall events

1-36 see rainfall

events, Table 1 36

2 Newcastle city centre

4km2 1m 4.000.000 Rainfall events

1-36 see rainfall

events, Table 1 36

3 Newcastle city centre

4km2 0.5m 16.000.000 Rainfall events

1-36 see rainfall

events, Table 1 36

4 Newcastle city centre

4km2 2m 1.000.000 Hypothetical

flood wave 2 hrs 1

5 Whole Newcastle City Council area

120km2 4m 7.500.000 Rainfall events

1-36 see rainfall

events, Table 1 36

6 Thames estuary ~1100km2 15m ~5.000.000 Tidal surge

water level 33 hrs

and 21 hrs 2

All these simulations, and some of them with multiple runs, produced a huge amount of

results which need to be converted into maps and videos in order to become meaningful to

end users. The development of flood risk maps requires the results of different rainfall

events to be compared in order to identify the most critical ones. Within this pilot project,

there was not enough time to carry out all this analysis due to the huge amount of results

that was generated. Therefore, within this project, just some of the results were turned into

maps and analysed, however, the rest of the results will be used in further studies. This

opened up the question if Cloud Computing could be used for the analysis and presentation

of results in flood risk studies and this will be explored in the future.

Domain 1 – Newcastle city centre – simulations 1 to 4 in Table2

A domain of 4km2 including the city centre of Newcastle was chosen as the main domain

area for this project. When this domain is

compared with the other domains used in this

project it might seem small but it is in reality much

bigger that the areas usually modelled in current

engineering practice, especially when the grid size

is taken into account. A large consultancy project

would typically model a domain of 1 km2 with a

grid of 5m which gives 40.000 cells while our

simulations 1, 2 and 3 have 1.000.000, 4.000.000

and 16.000.000 cells respectively. These

simulations could not have been done with most

software used for flood simulation as there are

usually software limitations to the number of cells

in the domain. However, “CityCat” does not have

any limits on the size of the domain but we still could not have run it on our university

facilities due to the memory limitations of our PCs. Use of PCs with large memory via Cloud

Computing (see table 3) enabled us to run these simulations. Additionally, using Cloud

Computing we were able to run all 36 rainfall events (see table1) in parallel either on one or

more PC instances.

Table 3 – Instances of PCs used in simulations 1,2 and 3

Simulation

Number of cells

Cell size

Required memory

PC Instances used Number of jobs submitted per instance

1 1.000.000 2m 3GB Standard instances – 7.5GB 2

2 4.000.000 1m 11GB High-Memory Instances – 68.4 GB 5

3 16.000.000 0.5m 40GB High-Memory Instances – 68.4 GB 1

The motivation to use different cell sizes for the same domain was not only to test the limits

of the software and the use of Cloud but also to investigate how different cell sizes influence

the results. See Figure 6.

Figure 4 - Domain1 - Newcastle City Centre- buildings layer from the MasterMap

There are numerical reasons why one could expect different results when using different

cell sizes but in the case of “CityCat” the main difference comes from the way the buildings

are cut-out of the domain. Larger cell sizes could produce unrealistic situations in the

models where buildings that

are not connected in reality

would appear connected in

the model and that could

completely distort the flow

pattern. From the previous

investigations, it has been

concluded that the grid sizes

of 2m or smaller, are not

prone to large errors of that

type. In this study even

smaller cell sizes were used

to check if they bring any

further improvements. The

initial conclusion is that the

smaller cell sizes capture

better the topography and

improve the model results

but unless some small

passageways are completely

cut-off when the buildings

are cut-out, the overall result

does not seem to be significantly improved with the use of smaller cell size grid. However, if

one is interested in changes of flood risk due to man-made interventions like rising of

c) time= 1 hour 20 min

a) time = 15 min

d) time = 2 hours

a) time =15 min

Figure 7. Flow in Newcastle City Centre induced by a fictitious constant

water level at the north boundary of the domain.

Figure 5 - Influence of the grid size on results - up left- cell size of 1m and up right- cell size of 2m

pavements or building of walls, then models based on very small grid sizes (i.e. 0.5m) are

beneficial as they give a more precise picture of the change in flow conditions.

In simulation 4 the same domain as in simulation 1 was used with different boundary

condition in order to test the capability of the model to handle extreme flood conditions

from other sources. This was achieved by introducing a hypothetical flood wave entering

the north boundary of the domain. (See Figure 7.)

The results of this model are completely fictitious but they show that the model is able to

capture the flow patterns and calculate the water depth and the velocities in extreme flow

conditions due to external sources (e.g. fluvial or tidal flooding).

Domain 2 – Whole Newcastle City Council area – simulation 5 in Table2

The second domain is the whole area

of Newcastle City Council which covers

approximately 120km2. We have

chosen this area to demonstrate that

by using Cloud Computing and

“CityCat” local authorities would be

able to model such big areas for

Surface Water Management Plans. In

a way it is a move towards producing

flood models at a scale larger than a

city and moving towards a region.

Newcastle City council area is a good

example of this as it has urban,

suburban and rural areas. See figure 8.

Due to the fact that the area of this domain is

large, a 4m cell size was used which resulted in

7.5million cells. Any further reduction in the cell

size was considered unfeasible at that stage of

the project.

On the enlarged detail in figure 9, it can be seen

that cutting-out of buildings, formed continuous

barriers within the domain which have a

significant influence on the flow patterns.

However, these are mainly dettached and semi

detached dwellings and although a lot of them have been extended and almost form rows

of terraced houses, there are still a lot of gaps between them which are not captured well

Figure 9 – Detail from -suburban areas

Figure8 - Flood map for the whole area of Newcastle City

Council for the rain event 27 at 60min

by the model. This is a consequence of the 4m grid and this points out to our previous

conclusion that the 2m grid provides much more realistic flow patterns when the buildings

are cut-out.

Domain 3 – Thames estuary – simulation 6 in Table2

The domain of the final simulation was the Thames estuary with the area of ~1100km2. Due

to the domain being so large the grid size was chosen to be 15m and buildings were not cut-

out. CityCat does not impose limitations on the size of the domain, however, larger or

higher resolution domains were not used due to the limited duration of the project. The

propagation of the tidal surge upstream along the Thames was modelled to see if “CityCat”

could be used in flood risk studies for the Thames estuary including London.

The boundary condition for this simulation was the tidal water level given as a function of

time and placed at the east boundary of the domain. Due to the fact that we did not have

the bathymetry of the river we decided not to include any discharge coming down the river.

Two different runs were performed using two different tidal boundary conditions.

Time = 27 hours

Time = 33 hours

Figure 10 – Water depth in the Thames estuary due to the extreme storm surge (1000 years return period)

The obtained results look encouraging and furthermore it was demonstrated that “CityCat”

running on the Cloud can handle such huge domains for events of prolonged duration and

could be used in the future for flood risk studies of large areas such as the Thames estuary.

Conclusions

Although “CityCat” does not impose any intrinsic limitations on the size of the modelled

domain, prior to this project, hardware limitations prevented modelling of very large

domains. The use of Cloud Computing enabled modelling of flooding in cities on a much

larger scale; with much higher resolution and longer duration of events than it is current

available in engineering practice. Domains of up to 1100km2 and 16.000.000 computational

cells were modelled, and it has been demonstrated that even much larger ones can be used.

Another major advantage of using Cloud computing was the ability to run models of the

same domain, with different rainfall inputs, all at the same time. For the majority of the

simulations 36 different rainfall events were run in parallel. This number can be much larger

if a flood risk analysis under climate change is undertaken. This would have been possible,

however, within this pilot project there was no scope to develop an efficient way to analyse

and visualize the already vast amount of the results which have been generated with 36

events. Based on this experience, the automatic visualisation and analysis of the results

using Cloud Computing has been identified as a potential area for further exploration.

In this project it was demonstrated that the use of “CityCat” on the Cloud can perform high

resolution large scale modelling of flooding that can be used for the assessment of city-scale

flood risk under climate change using only readily available data and producing results much

faster than all other currently available models and computing methods in general

engineering practice.