how deep learning could predict weather events
TRANSCRIPT
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
How Deep Learning Could Predict Weather Events
Seongchan Kim PhD Seunkyun Hong
On behalf of Sa-Kwang Song PhD
Research Data Platform Center
GTC 2018
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 2
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data
bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 3
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data
bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 4
Vision vs Meteorology Tasks
Typhoon
Classification
Typhoon
Location WeatherClimate Event Detection
Or Segmentation
httpschaosmailgithubiodeeplearning20161022intro-to-deep-learning-for-computer-vision
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5
Similarities vs Differences
Phrabat 2017(CI 2017)
bull Similaritiesbull Tasks are similar
bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning
bull Differencesbull Unique attributes of Weather Data
bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6
Challenge Multi-Variate Data
COMS Satellite Himawari 8 satellite
WRF Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Numerical Model Results
Satellite
Radar
BAIPAS
ImageSensor Data
Prediction Model based on DL
Convolutional Neural Networks
Recurrent Neural Networks
Sensors
Typhoon Track
Surge LevelFlood Level
8
Prediction of Weather Events
Bigdata amp AI based Prediction and Analysis Platform
Disaster Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9
DeepRain Precipitation Prediction
Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation
bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving
speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 2
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data
bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 3
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data
bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 4
Vision vs Meteorology Tasks
Typhoon
Classification
Typhoon
Location WeatherClimate Event Detection
Or Segmentation
httpschaosmailgithubiodeeplearning20161022intro-to-deep-learning-for-computer-vision
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5
Similarities vs Differences
Phrabat 2017(CI 2017)
bull Similaritiesbull Tasks are similar
bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning
bull Differencesbull Unique attributes of Weather Data
bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6
Challenge Multi-Variate Data
COMS Satellite Himawari 8 satellite
WRF Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Numerical Model Results
Satellite
Radar
BAIPAS
ImageSensor Data
Prediction Model based on DL
Convolutional Neural Networks
Recurrent Neural Networks
Sensors
Typhoon Track
Surge LevelFlood Level
8
Prediction of Weather Events
Bigdata amp AI based Prediction and Analysis Platform
Disaster Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9
DeepRain Precipitation Prediction
Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation
bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving
speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 3
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingNumerical Model Data
bull GlobeNet Tropical Cyclone Trajectory Prediction usingSatellite Images amp AutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 4
Vision vs Meteorology Tasks
Typhoon
Classification
Typhoon
Location WeatherClimate Event Detection
Or Segmentation
httpschaosmailgithubiodeeplearning20161022intro-to-deep-learning-for-computer-vision
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5
Similarities vs Differences
Phrabat 2017(CI 2017)
bull Similaritiesbull Tasks are similar
bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning
bull Differencesbull Unique attributes of Weather Data
bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6
Challenge Multi-Variate Data
COMS Satellite Himawari 8 satellite
WRF Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Numerical Model Results
Satellite
Radar
BAIPAS
ImageSensor Data
Prediction Model based on DL
Convolutional Neural Networks
Recurrent Neural Networks
Sensors
Typhoon Track
Surge LevelFlood Level
8
Prediction of Weather Events
Bigdata amp AI based Prediction and Analysis Platform
Disaster Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9
DeepRain Precipitation Prediction
Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation
bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving
speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 4
Vision vs Meteorology Tasks
Typhoon
Classification
Typhoon
Location WeatherClimate Event Detection
Or Segmentation
httpschaosmailgithubiodeeplearning20161022intro-to-deep-learning-for-computer-vision
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5
Similarities vs Differences
Phrabat 2017(CI 2017)
bull Similaritiesbull Tasks are similar
bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning
bull Differencesbull Unique attributes of Weather Data
bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6
Challenge Multi-Variate Data
COMS Satellite Himawari 8 satellite
WRF Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Numerical Model Results
Satellite
Radar
BAIPAS
ImageSensor Data
Prediction Model based on DL
Convolutional Neural Networks
Recurrent Neural Networks
Sensors
Typhoon Track
Surge LevelFlood Level
8
Prediction of Weather Events
Bigdata amp AI based Prediction and Analysis Platform
Disaster Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9
DeepRain Precipitation Prediction
Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation
bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving
speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 5
Similarities vs Differences
Phrabat 2017(CI 2017)
bull Similaritiesbull Tasks are similar
bull Classification Localization Detection Segmentationbull Clustering Regressionbull Representation Learning
bull Differencesbull Unique attributes of Weather Data
bull Multi-channelMulti-variatebull Different Spatio-temporal scalesbull Double precision floating pointbull Underlying statistics are likely differentbull Large amount of data compared to general Vision tasks
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6
Challenge Multi-Variate Data
COMS Satellite Himawari 8 satellite
WRF Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Numerical Model Results
Satellite
Radar
BAIPAS
ImageSensor Data
Prediction Model based on DL
Convolutional Neural Networks
Recurrent Neural Networks
Sensors
Typhoon Track
Surge LevelFlood Level
8
Prediction of Weather Events
Bigdata amp AI based Prediction and Analysis Platform
Disaster Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9
DeepRain Precipitation Prediction
Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation
bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving
speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 6
Challenge Multi-Variate Data
COMS Satellite Himawari 8 satellite
WRF Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Numerical Model Results
Satellite
Radar
BAIPAS
ImageSensor Data
Prediction Model based on DL
Convolutional Neural Networks
Recurrent Neural Networks
Sensors
Typhoon Track
Surge LevelFlood Level
8
Prediction of Weather Events
Bigdata amp AI based Prediction and Analysis Platform
Disaster Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9
DeepRain Precipitation Prediction
Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation
bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving
speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 7
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Numerical Model Results
Satellite
Radar
BAIPAS
ImageSensor Data
Prediction Model based on DL
Convolutional Neural Networks
Recurrent Neural Networks
Sensors
Typhoon Track
Surge LevelFlood Level
8
Prediction of Weather Events
Bigdata amp AI based Prediction and Analysis Platform
Disaster Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9
DeepRain Precipitation Prediction
Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation
bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving
speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Numerical Model Results
Satellite
Radar
BAIPAS
ImageSensor Data
Prediction Model based on DL
Convolutional Neural Networks
Recurrent Neural Networks
Sensors
Typhoon Track
Surge LevelFlood Level
8
Prediction of Weather Events
Bigdata amp AI based Prediction and Analysis Platform
Disaster Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9
DeepRain Precipitation Prediction
Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation
bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving
speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 9
DeepRain Precipitation Prediction
Weather Radar CenterhttpradarkmagokrlectureradardataflowdoMethods used to measure precipitation
bull Weather Radar Databull refers to data represented by a radar image that is composed using the moving
speed direction and strength of a signal transmitted by a radar transmitter into the atmosphere and received after it has collided with water vapor or the like
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 10
DeepRain Precipitation Prediction
6 mins 15 = 90 mins
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Shenzhen Meteorological Bureau-Alibab ldquoShort-Term Quantitative Precipitation Forecastingrdquo httpstianchialiyuncomcompetitioninformationhtmspm=517610006756782jsxLyXampraceId=231596
11
DeepRain Radar Data
bull Research-use data by Shenzhen Meteorological Agencybull Modeling specific areas in Shenzhen as a grid pattern(101101km2)
bull Radar reflection values 101101 numerical values (dBZ) of representing each cell
bull Precipitation amount Total amount of rainfall in target site (5050 area from center)
bull Normalization Anonymization
101101km2
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
(101 101 4)
ConvLSTM
h0
ConvLSTM
ConvLSTM
helliphellip
119865119894119899119886119897 119875119903119890119889119894119888119905119894119900119899
_X _X_119894119904119905119886119905119890
_119871119878119879119872_119874
1 2
_O _O _O
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_X15
119868119899119901119906119905 119868119899119901119906119905 119868119899119901119906119905(101 101 4) (101 101 4)
12
DeepRain Precipitation Prediction
httpradarkmagokrlectureradardataflowdo
Height 1
Height 2
Height 3
Height 4
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 13
DeepRain Experimental Results
1 3 5 7 9 11 13 15 17 19 21 23 25 27
0
5
10
15
20
25
30
Epoch
Loss
(RM
SE)
FC-LSTM(GDO0001) FC-LSTM(Adam0001)convLSTM(Adam 0001) convLSTM(Adam00012-stacked)
bull convLSTM shows better learning performance than FC-LSTM
bull Test result with Testsetbull Epoch 5
bull With two-stacked we achieved 230 performance increase than LR
bull Furthermore it is lower 217 than FC-LSTM
bull Because FC-LSTM lost spatial information
Model RMSE Drop Ratio
Linear Regression 1469 -DeepRain FC-LSTM 1446 16
DeepRain convLSTM 1151 216DeepRain convLSTM(2-stacked) 1131 230
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 14
Seq-to-Seq DeepRain Prediction of Next Seq Image
The last N input images as input sequence Prediction output sequence
tt-1t-2t-3t-4t-5t-6t-7t-8t-9 t+1 t+2 t+3 t+4 t+5
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 15
Seq-to-Seq DeepRain Conv2Deconv RNNLSTM
bull Model Sequence-to-Sequence Modelbull Input rarr encoderbull decoder Output
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 16
DeepRain Deep Learning Models
bull Input and state transformation models1) Vanilla fully-connected models
bull Vanilla FullyConnected-RNNbull Vanilla FullyConnected-LSTM
2) Convolution to Deconvolution (Conv2Deconv) modelsbull Convoluted RNNrsquos states (Conv2Deconv-ConvRNN)bull Convoluted LSTMrsquos states and memory cells(Conv2Deconv-ConvLSTM)
3) Full-size convolution(Input images are fed directly to the convolutional networks)bull Convoluted RNNrsquos states (Fullsize-ConvRNN)bull Convoluted LSTMrsquos states and memory cells (Fullsize-ConvLSTM)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 17
DeepRain Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 18
DeepRain Conv2Deconv-ConvLSTM
rarr Note predicted images are blurred- Testing mean-cost=1316 mean-rmse=2242- Testing best-cost=671 best-rmse=1073
The last 10 input images
Model Predicted images
Ground truth images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 19
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC Tropical Cyclone Trajectory Prediction usingTraditional Numerical Model Data
bull GlobeNet Typhoon Track Prediction amp Autoencoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Today meteorologists rely on numerical models to predict wind speed precipitation air pressure and other factors that indicate the path and intensity of a hurricane over its lifetimebull WRF (Weather Research and Forecasting model) MPAS (Model for Prediction
Across Scales) UM (Unified Model) and CAM5 (Community Atmosphere Model ver 50)
20
Motivation
Bolaben Wind (UVW) Bolaben DBz Bolaben Flow
Visualization (Vapor)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Predicting typhoon track predict using data from a numerical model (WRF) simulation results with Deep Neural Networkbull ConvLSTM Convolutional LSTM (shi et al 2015) that learn spatial features of
input data
bull Ensemble-like techniques learning from five differently conditioned WRF results
Ensemble Forecasting
21
Purpose
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull 10 typhoons drifting in close proximity to the Korea peninsula
No
ID(YYNN)
Name Period DurationSimulation Durations
Num of Simulation
1 0215 루사(RUSA)20020823 0900 ~ 20020901 1500
9 days 3 hours 4 days 6 hours 25
2 0314 매미(MAEMI)20030906 1500 ~ 20030914 0600
3 days 15 hours 3 days 18 hours 15
3 0415 메기(MEGI)20040816 1500 ~ 20040820 1800
4 days 3 hours 3 days 18 hours 15
4 0603 에위니아(EWINIAR)20060701 0300 ~ 20060710 2200
9 days 19 hours 6 days 18 hours 27
5 1004 뎬무(DIANMU)20100808 2100 ~ 20100812 1500
3 days 18 hours 3 days 6 hours 13
6 1214 덴빈(TEMBIN)20120819 0900 ~ 20120831 0000
11 days 15 hours 7 days 18 hours 35
7 1215 볼라벤(BOLAVEN)20120820 1500 ~ 20120829 0600
8 days 15 hours 6 days 18 hours 27
8 1216 산바(SANBA)20120911 0900 ~ 20120918 0900
7 days 4 days 18 hours 19
9 1509 찬홈(CHAN-HOM)20150630 2100 ~ 20150713 0600
12 days 9 hours 5 days 18 hours 23
10 1618 차바(CHABA)20160928 0300 ~ 20161002 1500
5 days 12 hours 3 days 6 hours 13
Sum 212
22
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
Name Description Dimension Data Dim
T perturbation potential temperature
(5 29 192 192) 4 dimension(time height width length)
P perturbation pressure
QVAPOR Water vapor mixing ratio
SST Skin sea surface temperature(5 1 192 192)
OLR TOA outgoing long wave
bull Selected variables bull 5 variables deemed to be the most significant for cyclone
tracking
bull four-dimensional bull Time height width and length
bull represented in 3-D spatial grids
23
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Simulation Settingbull 10 typhoons 1060 predictions
bull For example Rusa 4 days 6 hours
bull Simulation every 6 hour cyclically
bull Five different initial conditions (models)
bull 212 5 = 1060bull Whole WRF file size 22 TB
bull Data split
of instances TF Record Size
Training set 600 150 Gb
Validation set 200 50 Gb
Testing set 200 50 Gb
24
Data
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
WRF data
Conv2DLSTM
h0 Conv2DLSTM
Conv2D LSTM
helliphellip
_X _X
_119867
_119894119904119905119886119905119890
_119871119878119879119872_119874
0 1
_119871119878119879119872_119874 _119871119878119879119872_119874
_119871119878119879119872_119878 _119871119878119879119872_119878 _119871119878119879119872_119878
_119867 _119867
_X4
Prediction every 6 hours for next 24 hours
0000 0600 2400
Best track(Ground Truth)
(lat0 long0)
_O0 _O1 _O4
(lat1 long2) (lat4 long4)
Time
119896119890119903119899119890119897 = 33119891119894119897119905119890119903119904 12
119904ℎ119886119901119890 = (239279)119888ℎ119886119899119899119890119897 = 89
25
Model
Prediction(lat0 long0) (lat1 long2) (lat4 long4)
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
bull Learning
0
5
10
15
20
25
30
35
40
45
50
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 67 70 73 76 79 82 85 88 91
RM
SE
Epoch
26
Model RMSE
FC-LSTM 14724DeepTC ConvLSTM 1388
Result
- K40m(x2) reduced 54 of training time against CPU - While P100(x2) reduced 56 of training time against
K40m for 100 epoch
bull Test Result
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 27
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull DeepRain Precipitation Prediction of amp Next Step of Images
bull DeepTC Tropical Cyclone Prediction using Numerical ModelData
bull GlobeNet Typhoon Track Prediction using Satellite Images ampAutoEncoder
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 28
GlobeNet Typhoon Track Prediction
Observation Data
COMS-1MI (KMA)
Himawari-8 images (16ch)
Prediction of Typhoon Track
Deep Learning Prediction Models
HIMAWARI-8AHI (JMA)
Convolutional Neural Nets (CNNs)
Long shortTerm Memory (LSTM)
Index Conf Lat Long hPa NSWE
1 09999 279748 1259390 972 N
2 09361 133122 837975 1013 NS
3 0 0 0 0 X
4 0 0 0 0 X
5 0 0 0 0 X
6 0 0 0 0 X
Prediction Results
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 29
GlobeNet Data
bull Input Satellite Data
bull COMS-1 Satellite 4 channel(IR1 IR2 SWIR WV)
bull Period 201104~201612
bull Label Best Track Data
bull RSMC-Tokyo Best Track
bull 6 years 2011~2016 (152 typhoon cases)
Typhoon TrackCOMS-1
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 30
GlobeNet Typhoon Location Detection
Input Satellite Images
Typhoon Location Tracking
Conv2D based Model
Inception based Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 31
GlobeNet Experiments
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 32
GlobeNet Test Error
Distance(km)
- Average Error(distance in km) 7453 km- Eye of typhoon 30-65km in diameter
30-65km
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 33
GlobeNet Autoencoder
bull To build a pre-trained model
bull COMS-1
bull 4 channelIR1 IR2 SWIR WV
bull Size 1655 TB (201104~2017 187083 scenes)
bull HIMAWARI-8 AHIbull On-going
bull Size hundreds of TBs
bull Conv-Deconv Neural Network
Encoder(Convolution Step)
Decoder(Deconvolution
Step)
LatentVector
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 34
GlobeNet Encoding Step
Convolution Step
Layer 1 Layer2 Layer3 Layer4 Layer5Input
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 35
GlobeNet Decoding Step
Deconvolution Step
Layer4 Layer3 Layer2 Layer1Layer5
Prediction
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 36
GlobeNet Autoencoder
Input Images
Output Images
Conv-Deconv Model
Skip-connection Model
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 37
GlobeNet with Skip-connection Decoder
Output Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 38
Recent Convolutional AE without Skip-connection
Output Images
Input Images
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 39
Application of Skip-connection Conv AE PSIque
bull Complex Deep Seq2Seq Autoencoder model based on Memory Network structure
bull Unified model structure compatible with various RNNCells (eg LSTM GRU ConvLSTM)
bull Encoder(Conv)-gtEncoder(LSTM)-gtDecoder(LSTM)-gtDecoder(DeConv) oplus SkipConx
Encoder (Conv)5-layers ConvNet for Feature Extraction
Encoder (LSTM)Encoding SpatiotemporalChanges Through Time
Decoder (LSTM)Decoding SpatiotemporalChanges Through Time
Decoder (DeConv)5-layers DeConvNet for Image Restoration
SkipConxSelective Symmetricbypassing skip connection from Encoder to Decoder
Latent Info (OutputState)Condensed Tensor representing Spatiotemporal Transition
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 40
Outline
bull Introduction Vision vs Meteorology tasks
bull Deep Learning for Weather Predictionbull GlobeNet Typhoon Track Prediction amp Autoencoder
bull DeepRain Prediction of Precipitation amp Next Step of Images
bull DeepTC GPU-Accelerated Trajectory Prediction for TropicalCyclone using Traditional Numerical Model Data
bull Conclusion
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI 41
Conclusion
bull Deep Learning can be applied effectively to understand meteorological phenomenonbull DeepRain does using radar reflectivity data
bull DeepTC does using numerical model data
bull GlobeNet predicts meteorological phenomenon by analyzing satellite images
bull Distributed Deep Learning Platform is necessary
bull Number of associated challenges
bull Open to collaboration
NVIDIA GPU Technology Conference 2018Copyrightcopy 2018 Sa-Kwang Song KISTI