how deep learning could predict weather events

42
NVIDIA GPU Technology Conference 2018 Copyright© 2018 Sa-Kwang Song, KISTI How Deep Learning Could Predict Weather Events Seongchan Kim, Ph.D. / Seunkyun Hong On behalf of Sa-Kwang Song, Ph.D. Research Data Platform Center GTC 2018

Upload: others

Post on 08-Apr-2022

4 views

Category:

Documents


0 download

TRANSCRIPT

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

How Deep Learning Could Predict Weather Events

Seongchan Kim PhD Seunkyun Hong

On behalf of Sa-Kwang Song PhD

Research Data Platform Center

GTC 2018

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 2

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data

bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 3

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data

bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 4

Vision vs Meteorology Tasks

Typhoon

Classification

Typhoon

Location WeatherClimate Event Detection

Or Segmentation

httpschaosmailgithubiodeeplearning20161022intro-to-deep-learning-for-computer-vision

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5

Similarities vs Differences

Phrabat 2017(CI 2017)

bull Similaritiesbull Tasks are similar

bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning

bull Differencesbull Unique attributes of Weather Data

bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6

Challenge Multi-Variate Data

COMS Satellite Himawari 8 satellite

WRF Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Numerical Model Results

Satellite

Radar

BAIPAS

ImageSensor Data

Prediction Model based on DL

Convolutional Neural Networks

Recurrent Neural Networks

Sensors

Typhoon Track

Surge LevelFlood Level

8

Prediction of Weather Events

Bigdata amp AI based Prediction and Analysis Platform

Disaster Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9

DeepRain Precipitation Prediction

Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation

bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving

speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 2

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data

bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 3

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data

bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 4

Vision vs Meteorology Tasks

Typhoon

Classification

Typhoon

Location WeatherClimate Event Detection

Or Segmentation

httpschaosmailgithubiodeeplearning20161022intro-to-deep-learning-for-computer-vision

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5

Similarities vs Differences

Phrabat 2017(CI 2017)

bull Similaritiesbull Tasks are similar

bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning

bull Differencesbull Unique attributes of Weather Data

bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6

Challenge Multi-Variate Data

COMS Satellite Himawari 8 satellite

WRF Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Numerical Model Results

Satellite

Radar

BAIPAS

ImageSensor Data

Prediction Model based on DL

Convolutional Neural Networks

Recurrent Neural Networks

Sensors

Typhoon Track

Surge LevelFlood Level

8

Prediction of Weather Events

Bigdata amp AI based Prediction and Analysis Platform

Disaster Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9

DeepRain Precipitation Prediction

Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation

bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving

speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 3

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data

bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 4

Vision vs Meteorology Tasks

Typhoon

Classification

Typhoon

Location WeatherClimate Event Detection

Or Segmentation

httpschaosmailgithubiodeeplearning20161022intro-to-deep-learning-for-computer-vision

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5

Similarities vs Differences

Phrabat 2017(CI 2017)

bull Similaritiesbull Tasks are similar

bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning

bull Differencesbull Unique attributes of Weather Data

bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6

Challenge Multi-Variate Data

COMS Satellite Himawari 8 satellite

WRF Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Numerical Model Results

Satellite

Radar

BAIPAS

ImageSensor Data

Prediction Model based on DL

Convolutional Neural Networks

Recurrent Neural Networks

Sensors

Typhoon Track

Surge LevelFlood Level

8

Prediction of Weather Events

Bigdata amp AI based Prediction and Analysis Platform

Disaster Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9

DeepRain Precipitation Prediction

Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation

bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving

speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 4

Vision vs Meteorology Tasks

Typhoon

Classification

Typhoon

Location WeatherClimate Event Detection

Or Segmentation

httpschaosmailgithubiodeeplearning20161022intro-to-deep-learning-for-computer-vision

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5

Similarities vs Differences

Phrabat 2017(CI 2017)

bull Similaritiesbull Tasks are similar

bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning

bull Differencesbull Unique attributes of Weather Data

bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6

Challenge Multi-Variate Data

COMS Satellite Himawari 8 satellite

WRF Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Numerical Model Results

Satellite

Radar

BAIPAS

ImageSensor Data

Prediction Model based on DL

Convolutional Neural Networks

Recurrent Neural Networks

Sensors

Typhoon Track

Surge LevelFlood Level

8

Prediction of Weather Events

Bigdata amp AI based Prediction and Analysis Platform

Disaster Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9

DeepRain Precipitation Prediction

Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation

bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving

speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5

Similarities vs Differences

Phrabat 2017(CI 2017)

bull Similaritiesbull Tasks are similar

bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning

bull Differencesbull Unique attributes of Weather Data

bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6

Challenge Multi-Variate Data

COMS Satellite Himawari 8 satellite

WRF Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Numerical Model Results

Satellite

Radar

BAIPAS

ImageSensor Data

Prediction Model based on DL

Convolutional Neural Networks

Recurrent Neural Networks

Sensors

Typhoon Track

Surge LevelFlood Level

8

Prediction of Weather Events

Bigdata amp AI based Prediction and Analysis Platform

Disaster Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9

DeepRain Precipitation Prediction

Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation

bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving

speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6

Challenge Multi-Variate Data

COMS Satellite Himawari 8 satellite

WRF Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Numerical Model Results

Satellite

Radar

BAIPAS

ImageSensor Data

Prediction Model based on DL

Convolutional Neural Networks

Recurrent Neural Networks

Sensors

Typhoon Track

Surge LevelFlood Level

8

Prediction of Weather Events

Bigdata amp AI based Prediction and Analysis Platform

Disaster Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9

DeepRain Precipitation Prediction

Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation

bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving

speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Numerical Model Results

Satellite

Radar

BAIPAS

ImageSensor Data

Prediction Model based on DL

Convolutional Neural Networks

Recurrent Neural Networks

Sensors

Typhoon Track

Surge LevelFlood Level

8

Prediction of Weather Events

Bigdata amp AI based Prediction and Analysis Platform

Disaster Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9

DeepRain Precipitation Prediction

Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation

bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving

speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Numerical Model Results

Satellite

Radar

BAIPAS

ImageSensor Data

Prediction Model based on DL

Convolutional Neural Networks

Recurrent Neural Networks

Sensors

Typhoon Track

Surge LevelFlood Level

8

Prediction of Weather Events

Bigdata amp AI based Prediction and Analysis Platform

Disaster Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9

DeepRain Precipitation Prediction

Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation

bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving

speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9

DeepRain Precipitation Prediction

Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation

bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving

speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10

DeepRain Precipitation Prediction

6 mins 15 = 90 mins

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596

11

DeepRain Radar Data

bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)

bull Radar reflection values 101101 numerical values (dBZ) of representing each cell

bull Precipitation amount Total amount of rainfall in target site (5050 area from center)

bull Normalization Anonymization

101101km2

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

(101 101 4)

ConvLSTM

h0

ConvLSTM

ConvLSTM

helliphellip

119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899

_X _X_119894119904119905119886119905119890

_119871119878119879119872_119874

1 2

_O _O _O

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_X15

119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)

12

DeepRain Precipitation Prediction

httpradarkmagokrlectureradardataflowdo

Height 1

Height 2

Height 3

Height 4

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13

DeepRain Experimental Results

1 3 5 7 9 11 13 15 17 19 21 23 25 27

0

5

10

15

20

25

30

Epoch

Loss

(RM

SE)

FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)

bull convLSTM shows better learning performance than FC-LSTM

bull Test result with Testsetbull Epoch 5

bull With two-stacked we achieved 230 performance increase than LR

bull Furthermore it is lower 217 than FC-LSTM

bull Because FC-LSTM lost spatial information

Model RMSE Drop Ratio

Linear Regression 1469 -DeepRain FC-LSTM 1446 16

DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14

Seq-to-Seq DeepRain Prediction of Next Seq Image

The last N input images as input sequence Prediction output sequence

tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15

Seq-to-Seq DeepRain Conv2Deconv RNNLSTM

bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16

DeepRain Deep Learning Models

bull Input and state transformation models1) Vanilla fully-connected models

bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM

2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)

3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17

DeepRain Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18

DeepRain Conv2Deconv-ConvLSTM

rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073

The last 10 input images

Model Predicted images

Ground truth images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data

bull GlobeNet Typhoon Track Prediction amp Autoencoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction

Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)

20

Motivation

Bolaben Wind (UVW) Bolaben DBz Bolaben Flow

Visualization (Vapor)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of

input data

bull Ensemble-like techniques learning from five differently conditioned WRF results

Ensemble Forecasting

21

Purpose

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull 10 typhoons drifting in close proximity to the Korea peninsula

No

ID(YYNN)

Name Period DurationSimulation Durations

Num of Simulation

1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500

9 days 3 hours 4 days 6 hours 25

2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600

3 days 15 hours 3 days 18 hours 15

3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800

4 days 3 hours 3 days 18 hours 15

4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200

9 days 19 hours 6 days 18 hours 27

5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500

3 days 18 hours 3 days 6 hours 13

6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000

11 days 15 hours 7 days 18 hours 35

7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600

8 days 15 hours 6 days 18 hours 27

8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900

7 days 4 days 18 hours 19

9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600

12 days 9 hours 5 days 18 hours 23

10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500

5 days 12 hours 3 days 6 hours 13

Sum 212

22

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

Name Description Dimension Data Dim

T perturbation potential temperature

(5 29 192 192) 4 dimension(time height width length)

P perturbation pressure

QVAPOR Water vapor mixing ratio

SST Skin sea surface temperature(5 1 192 192)

OLR TOA outgoing long wave

bull Selected variables bull 5 variables deemed to be the most significant for cyclone

tracking

bull four-dimensional bull Time height width and length

bull represented in 3-D spatial grids

23

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Simulation Settingbull 10 typhoons 1060 predictions

bull For example Rusa 4 days 6 hours

bull Simulation every 6 hour cyclically

bull Five different initial conditions (models)

bull 212 5 = 1060bull Whole WRF file size 22 TB

bull Data split

of instances TF Record Size

Training set 600 150 Gb

Validation set 200 50 Gb

Testing set 200 50 Gb

24

Data

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

WRF data

Conv2DLSTM

h0 Conv2DLSTM

Conv2D LSTM

helliphellip

_X _X

_119867

_119894119904119905119886119905119890

_119871119878119879119872_119874

0 1

_119871119878119879119872_119874 _119871119878119879119872_119874

_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878

_119867 _119867

_X4

Prediction every 6 hours for next 24 hours

0000 0600 2400

Best track(Ground Truth)

(lat0 long0)

_O0 _O1 _O4

(lat1 long2) (lat4 long4)

Time

119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12

119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89

25

Model

Prediction(lat0 long0) (lat1 long2) (lat4 long4)

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

bull Learning

0

5

10

15

20

25

30

35

40

45

50

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91

RM

SE

Epoch

26

Model RMSE

FC-LSTM 14724DeepTC ConvLSTM 1388

Result

- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against

K40m for 100 epoch

bull Test Result

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images

bull DeepTC Tropical Cyclone Prediction using Numerical ModelData

bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28

GlobeNet Typhoon Track Prediction

Observation Data

COMS-1MI (KMA)

Himawari-8 images (16ch)

Prediction of Typhoon Track

Deep Learning Prediction Models

HIMAWARI-8AHI (JMA)

Convolutional Neural Nets (CNNs)

Long shortTerm Memory (LSTM)

Index Conf Lat Long hPa NSWE

1 09999 279748 1259390 972 N

2 09361 133122 837975 1013 NS

3 0 0 0 0 X

4 0 0 0 0 X

5 0 0 0 0 X

6 0 0 0 0 X

Prediction Results

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29

GlobeNet Data

bull Input Satellite Data

bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)

bull Period 201104~201612

bull Label Best Track Data

bull RSMC-Tokyo Best Track

bull 6 years 2011~2016 (152 typhoon cases)

Typhoon TrackCOMS-1

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30

GlobeNet Typhoon Location Detection

Input Satellite Images

Typhoon Location Tracking

Conv2D based Model

Inception based Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31

GlobeNet Experiments

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32

GlobeNet Test Error

Distance(km)

- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter

30-65km

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33

GlobeNet Autoencoder

bull To build a pre-trained model

bull COMS-1

bull 4 channelIR1 IR2 SWIR WV

bull Size 1655 TB (201104~2017 187083 scenes)

bull HIMAWARI-8 AHIbull On-going

bull Size hundreds of TBs

bull Conv-Deconv Neural Network

Encoder(Convolution Step)

Decoder(Deconvolution

Step)

LatentVector

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34

GlobeNet Encoding Step

Convolution Step

Layer 1 Layer2 Layer3 Layer4 Layer5Input

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35

GlobeNet Decoding Step

Deconvolution Step

Layer4 Layer3 Layer2 Layer1Layer5

Prediction

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36

GlobeNet Autoencoder

Input Images

Output Images

Conv-Deconv Model

Skip-connection Model

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37

GlobeNet with Skip-connection Decoder

Output Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38

Recent Convolutional AE without Skip-connection

Output Images

Input Images

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39

Application of Skip-connection Conv AE PSIque

bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure

bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)

bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx

Encoder (Conv)5-layers ConvNet for Feature Extraction

Encoder (LSTM)Encoding SpatiotemporalChanges Through Time

Decoder (LSTM)Decoding SpatiotemporalChanges Through Time

Decoder (DeConv)5-layers DeConvNet for Image Restoration

SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder

Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40

Outline

bull Introduction Vision vs Meteorology tasks

bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder

bull DeepRain Prediction of Precipitation amp Next Step of Images

bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data

bull Conclusion

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41

Conclusion

bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data

bull DeepTC does using numerical model data

bull GlobeNet predicts meteorological phenomenon by analyzing satellite images

bull Distributed Deep Learning Platform is necessary

bull Number of associated challenges

bull Open to collaboration

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI

NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI