Discovery of synoptic patterns of climate variability and change using data mining and high performance computing

Climate Wikience: A Summary of Results

DevelopersZapletin E.A., Melnykov A.I.

Donetsk National Technical University

Ukraine

Ideas, Architect, Slides Rodriges Zalipynis R.A.

P.I.Prof. Averin G.V.

SpeakerProf. Bashkov E.A.

The Wikience Concept

Let’s “create” new analysts!

Usually domain experts have a solid background on the subject. They will perform very well without computer scientists, even if equipped with much simpler tools. They know what to seek and how to seek. Many active experts are currently uninvolved, especially operational professionals.

WikienceOriginates from Wiki + Science

Wikience – international pronunciation rather than wiki science

Wiki science – already reserved – is the study of how the phenomenon of wikis, and free, open collaborative works in general (like free software and open source), are growing, changing and adapting.http://en.wikiversity.org/wiki/Wiki_science

Reanalysis archivesInterpolate all available data by statistical and numerical methods on regular latitude-longitude grid

Pioneer, released in 1996

NCEP/NCAR Reanalysis 1

1949 – present

2.5 x 2.5 grid

Applications

From simple warming trends to validation of climate models

Reanalysis archives• NCEP/NCAR 2

• ERA-40, ERA-15

• ERA-Interim

• JRA-25

• NOAA-CIRES 20CR

• NASA MERRA

• NCEP CFSR

Over 9 000 publications reference

Kalnay et al., The NCEP/NCAR 40-year reanalysis project, BAMS, 77, 437-470, 1996.

Visualization is vital for data understanding and perceptionIs it?

University of Melbourne (Australia)

Visualization is vital for data understanding and perceptionIs it?

National Oceanic and Atmospheric Administration (NOAA, USA)

Visualization is vital for data understanding and perceptionIs it?

Improving the Accuracy of Mapping Cyclone Numbers and Frequencies, 2010 by O.Zolina and S. Gulev (Russia)

Wikience

4D isopleths• 1st in the world

• real-time

• closed polygons (GIS)

• all TBs of reanalysis data on-line

KEY GRAPHICS

FOR CLIMATE

RESEARCH

Vast amounts of data (TBs)

1996 NCEP/NCAR ~ 4 TB

2010 NCEP CFSRR ~ 100 TB

Real-time technology for thousands of users

How to maintain all available data on-line?

Scientific formats

Convert data to new representation?

Need to keep metadata intact

Invent new schemas to import metadata into?

3D visualization technology

No open algorithms for isopleths, are they?

Obstacles for wide data accessibility

ChronosServer

Turns vast amounts of existing data into actionable intelligence

Rodriges Zalipynis R.A. ChronosServer: real-time access to “native” multi-terabyte retrospective data warehouse by thousands of concurrent clients. Informatics, cybernetics and computer engineering, pp. 151–161. Vol. 14 (188) – Donetsk: DonNTU — 2011. – 391 p.

Atmospheric major components: cyclones

Areas of low pressure

(simple definition)

Seen as several closed isobars nested in each other

Climate is changing along with cyclones

Cyclone importance

• heat transition from tropics to the poles

• define local weather

• local winds

• severe weather events

• pollution situation

key drivers of climate change

How to research cyclones? – TrackInitial

In a day

Sharing “active” research data4D Cyclone paths• 1st in the world

• real-time

Others can:• verify tracks• interactively analyze• use in own research

Remote Sensing: Terra MODIS• 36 spectral channels

• Daily frequency

• Global coverage

MODIS L3 product

• Global 1º×1º grid• Over 600 datasets

(clouds, water vapor, aerosol properties)

http://www.universetoday.com/84961/terra-satellite/http://modis-atmos.gsfc.nasa.gov/tools_level3browse.html

Terra MODIS: Cloud Optical Thickness

Tropical Cyclones

IBTrACS

International Best Track Archive for Climate Stewardship

• 1842 – present

• multiple characteristics

• the most completeAugust 23 – 30, 2005280 km/h highest wind

1,833 fatalities

$108 billion damage

Hurricane Katrina

just a picture

IBTrACS:Hurricane Katrina

NOT just a picture

Hurricane Katrina (IBTrACS) and Cloud Optical Thickness (MODIS L3)

August 28, 2005

Near peak strength

Analysis toolsSimply select & watch

Acknowledgements

This work was supported by Award No. UKM1-2973-DO-09 of the U.S. Civilian Research & Development Foundation (CRDF). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of CRDF.

DNTU Computer Cluster• Hybrid GPU-CPU

• 32 CPU cores

• 2 688 GPU cores, CUDA

• 7 Teraflops

• 35 TB HDD

• 1 Gbit/sec local network

• 1 Gbit/sec Internet band, optic fiber

• 10 KW power supply

Donetsk National Technical University

Global Models

Databases in knowledge domains

Wikience

Data mining

Donetsk National Technical University

Development of human potential

Database fusion:

World bankWorld bank http://data.worldbank.org

UN Development ProgramUN Development Programhttp://www.hdr.undp.org

Economical ForumEconomical Forum http://www.weforum.org

Data mining:Multidimensional scalingMultidimensional scaling

Donetsk National Technical University

Planet Biodiversity

Database fusion:

((Databases of animal and Databases of animal and plantsplants))

GIS-models for areals of biological species spread

((150 - 200 150 - 200 thousandsthousands of main of main animal and plant speciesanimal and plant species))

Climate reanalysis archives

Donetsk National Technical University

Data mining: world countries development indicators

Fertility Tuberculosis rate

Children's mortality

Internet usersGovernmental debt service

Population with age before15

Donetsk National Technical University

Data mining: world countries development - energy

CountryCountryDevelopment Development

indexindexCountry’s Country’s

raterateCountry’s rate Country’s rate according to according to development ratedevelopment rate

NorwayNorway 6313863138 22 66

USAUSA 3504335043 1010 1616

GermanyGermany 1804518045 2222 5353

PolandPoland 91559155 5050 4949

RussianRussian 1565815658 3333 2222

UkraineUkraine 91089108 5151 4242

ChinaChina 35003500 9191 6565

IndiaIndia 13851385 115115 9595

NigeriaNigeria 305305 138138 126126

Donetsk National Technical University

ЕЕА and DNTU: System for ecological monitoring of Donetsk region – English version

Discovery of synoptic patterns of climate variability and change using data mining and high performance computing

wikience.donntu.edu.ua Contact address: rodriges@csm.donntu.edu.ua

Climate Wikience: A Summary of Results

Please, reference materials taken from this presentation as

Rodriges Zalipynis R.A., Averin G.V., Zapletin E.A., Melnykov A.I., Bashkov E.A., Climate Wikience: A Summary of Results, Minneapolis, USA, 2012. Web: wikience.donntu.edu.ua/rodriges/publications/rodriges_CRDF_2012.pptx