quality assurance checklist (quack) - reporting · showcase? is there a procedure in place for a...

Version: 2.0

Copernicus Climate Change Service – Global Impacts

Quality Assurance Checklist (QUACK) - reporting

What is QUACK:

This tool is developed for the showcases of in C3S_422_Lot1_SMHI to promote high quality in each showcase

For a set of quality indicators it offers you structured questions in combination with guidance material

How to apply QUACK*:

For each quality indicator you will find assessment questions. Please state if they apply for your showcase.

For each assessment question; you will find supporting material under: http://climateservice-global.eu/quality-assurance/

Give a short description about how you treated the quality indicators

Wherever possible – provide any documents, links, scripts, etc. that support your performance

Note: Please save the file name as: Organization_Country_QUACK_reporting (e.g SMHI_Sweden_QUACK_reporting.docx)

*in case you have any questions contact: [email protected]

http://climateservice-global.eu/quality-assurance/

Copernicus Climate Change Service - C3S_422_Lot1_SMHI

Version: 2.0

reporting Version: 2.0

Showcase name: Adapting the city to heavier rainfall and increased flood risk

Short description of a showcase: In small and medium-sized rivers managed by the Tokyo Metropolitan Government, it is necessary to prevent flooding damage up to a rainfall of 60 mm/hour, to keep the risk of house inundation at maximum 1/20, and to ensure the safety of life for the excess rainfall. There is concern that the risk of flooding will increase as the magnitude and frequency of heavy rain will increase. Flood risk scenarios and decision support is required to maintain flood safety and we use Kanda River basin in this showcase.

Organization: Tokyo Metropolitan University Reporter name & email: Hideo Amaguchi ([email protected]) Last update and Place: 25/01/2019, Tokyo-Japan


Version: 2.0 1

Dimension: Input data

Criterion: Scientific & methodological quality

Quality indicator

Applied

Short description

Supporting documents

Comments

Yes

No

Transparency

Are all data sources referenced? Please provide which category (observation, simulation, etc) of data you used and the source.

Example: Forcing data: Precipitation , Station data, 1 h, 24 stations, national weather service

Simulation: GCM data, 50 km resolution; ensemble of five models; RCP 4.5 and 8.5, provided by C3S_422_Lot1_SMHI

x

Rainfall observations: - 1-h rainfall from one gauge since 1900 - 10-min rainfall from several gauges since 1992 Simulated forcing data (RCM): - The NHRCM (Non-Hydrostatic Regional Climate Model (developed by JMA's Meteorological Research Institute (MRI)) were downloaded by Japanese DIAS since there were not enough time and spatial resolution in the CDS. Temporal resolution: 10 min, spatial resolution: 5 km. Time periods: 1980-1999, 2016-2035, 2076-2095. GCM data for temporal downscaling:

NHRCM: www.data.jma.go.jp/cpdinfo/GWP/Vol8/index.html NHRCM: http://search.diasjp.net/en/dataset/JMA_GWP GCMs:

GFDL-ESM2M, HadGEM2-ES and NorESM1-M are available in the Climate Data Store catalogue

MIROC5 and MRI-CGCM3, are available in the Dataset Search and Discovery catalogue

http://www.data.jma.go.jp/cpdinfo/GWP/Vol8/index.html

http://www.data.jma.go.jp/cpdinfo/GWP/Vol8/index.html

http://search.diasjp.net/en/dataset/JMA_GWP

http://search.diasjp.net/en/dataset/JMA_GWP

http://search.diasjp.net/finder?lang=en

http://search.diasjp.net/finder?lang=en


Version: 2.0

2

- Five model projections: GFDL-ESM2M, HadGEM2-ES, MIROC5, MRI-CGCM3, NorESM1-M Temporal resolution: 1 day, spatial resolution: 50 km. We use data from several grid cells in the Tokyo area.

Do the metadata follow international standards?

X

As far as we know, they do.

Appropriateness

Are the used data appropriate for the case study in terms of spatial and temporal scale?

x

The NHRCM projection represents the type of high-resolution RCM projection needed for climate change impact assessment of intense rainfall and urban flooding. There is generally a mismatch between the resolution in the CDS data (1 day; 50 km) and the much high resolutions needed for the urban flood risk assessment. Therefore we use the NHRCM projection with resolutions 10 min and 5 km. Then, by aggregating the NHRCM projection to the CDS resolutions, and compare extremes at that resolution, we conclude that NHRCM represents a rather average projection with


Version: 2.0

3

respect to future changes in rainfall extremes under RCP8.5. We believe this is as appropriate as we can do it.

Completeness

Is the technical consistency checked (no outliers, no gaps in time series, etc.)?

x

- Checked unit, calendar, missing value, metadata if correct.

- Checked timesteps per netcdf file and variables for dimension, shape and size.

Do you have all available data sets considered?

x

Yes.


Version: 2.0

4

Dimension: Processing


Quality indicator

Applied

Short description


Comments

Yes

No

Transparency

Are the processing steps (i.e. scripts) well documented and reproducible?

x

Processing steps (i.e. scripts) are documented and reproducible.

Are the applied methods well documented and follow scientific standards (i.e. peer reviewed)?

x

Statistical (IDF) rainfall analysis: - The IDF analysis follows established procedures for extreme value analyses (e.g. Chow et al., 1988). The Gumbel distribution is fitted to annual maxima for different durations. Flood modelling: - Flood modelling is performed using the Tokyo Storm Runoff model (Amaguchi et al., 2012). Downscaling of changes in daily rainfall

IDF analysis: Chow, V. T., Maidment, D. R. & Mays, L. W. 1988 Applied Hydrology. McGraw-Hill, New York. Flood modelling: Amaguchi, H., Kawamura, A., Olsson, J., and T. Takasaki (2012) Development and testing of a


Version: 2.0

5

extremes: - We are in C3S_422_Lot1_SMHI testing a new method for temporally downscaling future changes in rainfall extremes (with uncertainty), not yet published.

distributed urban storm runoff event model with a vector-based catchment delineation, J. Hydrol., 420–421, 205-215, doi:10.1016/j.jhydrol.2011.12.003.

Appropriateness

Are the used methods, for instance bias-adjustment, appropriate for the case study?

x

Accuracy/Robustness

Are the uncertainties assessed systematically in a standard manner?

X

Uncertainty assessment is a bit difficult in this case as we are using only one high-resolution RCM projection. But by downscaling changes in daily rainfall extremes from a GCM ensemble we aim at generating some support for characterizing the uncertainty in the RCM projection. We do not perform any systematic or sophisticated uncertainty assessment, because 1/ we have a rather small ensemble and 2/ we do not have the resources for it. But we do present e.g. ensemble spread in a basic way.


Version: 2.0

6

Is the case study based on different global or regional climate model simulations? In case that not all simulations of an ensemble are used, is the selection well explained and justified?

x

Case study is based on one high-resolution RCM simulations and five GCM simulations. More simulations are available but because of limitations in project resources we cannot handle anymore. The selection of the five GCMs is based on historical performance assessment as well as practical considerations.

Is the showcase using climate emission scenarios i.e. a low, medium and high scenario? (Please specify the name of scenarios used in the showcase).

x

The NHRCM projection is based on the medium SRES scenario A1B and the GCM projections are based on the high RCP8.5 scenario.

Validation

Is validation possible for the showcase? Is there a procedure in place for a validation against independent data?

x

Valdiation of the NHRCM projection is possible and there is a procedure in place for a validation against independent data. The agreement with historically observed precipitation is good.

Reflectivity

Is the scientific consistency among multiple data sets and their findings well

X

We have not had the resources for any analysis of or comparison with results from other similar studies.


Version: 2.0

7

documented? (i.e. comparing your results to the existing peer reviewed studies in the region)

Criterion: Practical Relevance

Suitability for target group

Is the relevance for the client assured?

x

The relevance is assured by having the client continuously involved in the design of the experiments and in the analyses of the results.


Version: 2.0

8

Dimension: Output


Quality indicator

Applied

Short description


Comments

Yes

No

Transparency

Are the produced results provided with metadata?

X

We have not provided any metadata, as the data/results will not be open access.

Are the limits of provided information disclosed?

X

The limitations of the provided information is made clear in the final report.

Consistency

About the visualization of your output: Do the graphics and tables of the presentation match the captions and explanations?

X


Version: 2.0

9

Reliability

Is the confidence of the data presented (e.g. the spread of the whole ensemble)?

X

Yes, as far as it is possible (see further above).

Criterion: Practical Relevance

Lucidity/Clarity

Is the presentation of your output clear and does It follow visualization standards?

X

Do the pictures, and maps used match the explanation i.e. story?

X

Rights of Use

Are the rights of dissemination clarified (i.e. open access)?

X

The data/results produced will, at least for the time being, not be open access but restricted for use by TMU and TMG.


Version: 2.0

10

Usability

Is the output intuitively presented and freely accessible?

X

Yes, in the Interactive Atlas.

Criterion: Review Process Review by third party

Are the results of the showcases reviewed externally or internally? To what level the review process was taken.

X

We have not had the resources for any proper reviewing, only implicitly done in the exchange/communication between the purveyor (TMU) and the contact person (SMHI).


Version: 2.0

11

Dimension: Outcome

Criterion: Satisfaction

Quality indicator

Applied

Short description


Comments

Yes

No

Usefulness

Does the product help for problem solving of your client?

X

This remains to be seen in the continued interactions between the purveyor (TMU) and the client (TMG), but as the results/product are based on regular and mutual discussion during the C3S project it is our hope that an added value for the decision process has been provided.

quality assurance checklist (quack) - reporting · showcase? is there a procedure in place for a...

Documents