Upload File

machine learning in wan research - internet2 · • convolutional neural networks recurrent neural...

  • Home
  • Documents
  • Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning
24
Machine Learning in WAN Research Mariam Kiran [email protected] Energy Sciences Network (ESnet) Lawrence Berkeley National Lab Oct 2017 Presented at Internet2 TechEx 2017

Upload: others

Post on 28-May-2020

14 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

[email protected]

EnergySciencesNetwork(ESnet)LawrenceBerkeleyNationalLab

Oct2017

PresentedatInternet2TechEx 2017

Page 2: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

Outline

• MLingeneral

• MLinnetworkresearch– LiteratureReviewofresearchfrom[2010- Sept2017]ofMLalgorithmsinWANs

– Commonareas,datainvolved,whatproblemssolved

• RoadAhead(unexploredareas)

Page 3: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

AI,ML,DL– What’stheDifference?

• Turing“CanMachinesThink”– TuringTest:Exhibithuman-likeintelligence• MachinelearningiscollectionofalgorithmsthatcanhelpachieveAI

• e.g spamfilters,HRhiring,etc• DeeplearningisoneoftheseMLtechniques

• RecentadvancesduetoGPUandHPCprocessing(previouslyveryslow,toomuchdata,needtrainingtowork)

• Mainlyforimageandspeechrecognition– commercialapps

CourtesyNvidia Blog

Page 4: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

AITree(exampletechniques)onlyasubsetareMLalgorithms

AI

Optimizationtechnique

Manymore….

Expertsystems

Fuzzysystems

NeuralNetworks

Evolutionaryalgorithms(Geneticalgorithms,evolutionarystrategies,etc)

Swarmintelligence(antcolony,etc)

Deepbeliefnetworks

Deepboltzman networks

Convolutionalnetworks

Stackedautoencoders

Networks:graphalgorithm(routing–shortestpath)

ML:Whereevertrainingor‘learning’onstatisticaldata

RandomForrest,Clustering,etc

Page 5: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

5

Algorithmschosendependingon- dataavailable- problembeingsolved- combiningmultipletechniques(some50%accuracy,others80%accuracy)

Page 6: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

Example:ChoosingAlgorithmsforProblems(e.g.DNNs)

Deepneuralnetwork

InputData Appliedfor Variants

Feedforwardneuralnetwork

Hierarchicaldatarepresentations

• Generalclassification• Clustering• Anomalyfinding• Featureextraction

• Deepbeliefnetworks(usesrestrictedboltzmanmachineforactivationfunction)

• Convolutionalneuralnetworks

Recurrentneuralnetwork

Sequentialdatarepresentation(i.e.timeseriesdata)

Sequentiallearning (whentimerelationshipexists)

Longshorttermmemory(LTSM)usedforspeechtranslation

• There are many variants of DNNs. Papers and researchers in each specific DNN.

• DeepMind used Deep Q-learning for Attari and Go• Action-pairs based on learned data.

Page 7: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

MultipleToolsAvailable(DLLibraries)

• Google’s DNN platform TensorFlow used to tag unlabeled videos, recognize images with 70% accuracy and predict Gmail replies

• Scikit-learn good for learning, python library• Mostly used in image analysis

• HPC innovation: analyze massive data sets, quick training • Model and data parallelism to reduce the training time

Toolkit Language Use Processing capabilityCaffe C++ Images and video Distributed

(HPC, GPU)TensorFlow Python Images, regression, video, text, speech Distributed

(HPC, GPU)Theano Python Images Distributed

(HPC, GPU)Torch Lua Images and speech Distributed

(HPC, GPU)

Page 8: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

BringingitbacktoNetworks…(Reviewingpaperssince2010)

Page 9: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

MachinelearningUsecases(IETFforums)

• NetworkSecurity– Normalandoutlierbehaviorsintraffic

• Changeorpredictpossiblebehavior– This<QoS value>willcausethis<eventY>withprobability<P>

• Bugdetection– Softwareorhardwarefaults

• WANpathoptimization– Anticipatecongestion– Diverttraffictoalternatepaths

Page 10: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

ConductedaSystematicLiteratureReview

• Step1:Identifyresearchquestions

• Step2:Identifyasearchstring– “Wideareanetworks”AND(estimateORpredict)AND(learningOR‘‘datamining’’OR‘‘artificialintelligence’’OR‘‘patternrecognition’’ORregressionORclassificationORoptimization)

• Step3:Identifyrelevantlibraries,journals,papers– IEEEXplore,ACMDigitalLibrary,ScienceDirect,WebofScience,EICompendex,andGoogleScholar

Step1:Researchquestions

Step2:Searchstrategy

Step3:Studyselectioncriteria

Step3:Quality

assessment

Relevantpapers

Page 11: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

But…toomanypapersfound

• Spacewastoolarge:

• WANarecompletesystems

• Havemultiplelayers(e.g.seepicture)

• MultipleWANproblems

• Solution“Letsorganizetheresultsbasedon”:

• Createcategoriesofsimilarproblems

• ExploreMLandnon-MLsolutions

• Whichdatasetswereused

Page 12: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

CategorizingsimilarProblems

Usertrafficdata Usertraffic(directedflows)

12

WANTopology(trafficengineering)

(flow-level,trafficprediction,adaptation,pathoptimization,linkfailure)Infrastructuretrafficdata

(Packet-level,queues,TCP,UDP)

Infrastructure-levelmodifications(Switches,deployment,etc)

Page 13: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

MachinelearningapproachesinWAN

networks

2)TopologyEngineering

Trafficprediction

Trafficadaptation

Pathoptimization

Faultfinding

Multipledatacenter

connectivity

4)Infrastructureoptimization

1)Usertrafficoptimization

3)Packetleveloptimizations

TCPspecificproblems

Controllerplacements

Scheduling,congestion

Switchconfigurations

Note:SDNrelatedin(2,3,4)

Actual‘Actions’ontheWAN

Page 14: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

Results

Page 15: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

RelevantPapers:Statistics

IEEEExplore

ACMpub

ScienceDirect

WebofScience

#188

• Removeduplications

• Applyselectioncriteria

• Searchadditionalrelevancethroughreferences

• Removesurveys

• Applyqualityassessment

#3

#10

#532

#25

#223

Note:Googlescholargavemanyirrelevantresultsandisnotregardedasagoodpublicationsearchtool.

Page 16: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

Results– peryear(1)

• RiseofMLtechniquesin2017(WorkshopsatSigComm,HotNets,etc)

0

5

10

15

20

25

30

2010 2011 2012 2013 2014 2015 2016 2017

ML Non-ML

No.ofpapers

Page 17: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

Results– percategory(2)

• Non-MLstilllargelyfavored– problemsolving

• MostMLtechniquesareusedforclassification(oftraffic)andprediction(failures)– TechniquescoupledwithOpenFlow:Performclassificationandconfigurepackets

• SometoolsareenhancedbyMLembeddingfordecisionmaking:– Trafficawarenessandsecurityproblems– Formingtopologies,optimumpathfinding– Improvepathutilizationsdependingonarrivingtraffic

0

10

20

30

40

50

60

UserTraffic TrafficEngineering Packet-levelimprovements

Optimizinginfrastructure

ML Non-ML

No.ofpapers

Page 18: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

Techniquesused

Cat1:Usertrafficanalysis

Cat2:Trafficengineering

Cat3:Packetoptimization

Cat4:Optimizeinfrastructure

ML NaïveBayestheorem,decisiontrees,SVM,RandomForest,ANN

Regression andclassificationtechniques

SVR,decisiontrees, naïve-bayes

Regressionandclassificationtechniques

Non-ML Rule-basedlearning,statisticalanalysistechniques

Graphopt–mincost,greedysearch,SPF

Fairnesscomputations,pathfindinggametheory,Markovmodels,simulations

Simulation,greedyalgorithmsforresourceallocation

Classification,Regression

Page 19: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

Cat1:Usertrafficanalysis

Cat2:Trafficengineering

Cat3:Packetoptimization

Cat4:Optimizeinfrastructure

Usecases • Intrusiondetection

• Trafficprofiling

Classifyflowstoformoptimumtopologies

Pathperformance

Optimumconnectionsbetweendatacenters

Classification X X X X

Regression X X X

Clustering XDimensionreduction

Anomalydetection

X X

Featurelearning

Couplingwithdevices

XDemousingsimulations

Page 20: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

DataInvolved

• Rangefrompacketdata,pathproperties,IPaddresses,QoS,TCP/UDPtraces,etc…

– E.g.Google’sB4optimizestopologytoSDWAN(basedondemand,packetloss,utilization)

Usecases Focus DatasetusedCategory 3:Packet-leveloptimization

VMresources Fairnessschemes,MTTF,MTTR,Netflow

Category 4:Infrastructureoptimization

Flowtables,controllerplacements

No.ofjobsrunning,VMdata,CPUusage,Applicationdata

Page 21: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

RoadAhead…

Page 22: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

LostofAreasstillUnder-developed• Mostlygraphoptimizationproblems(MLislessapplied)

• Identifywhatwewanttoachievealongthepipeline:

Understanding(Classification) Prediction Action

Linkwithdevices(SDN,NFV,etc),butwhatarethe‘knobs’wecanalter?

Short-termgoals Long-termgoalsWhichMLalgorithmtouse?• No‘One-Pill-Solution’• Fast versusOK response,dataused

Toomanydatasets• Costoftrainingmodels• Dynamicenvironments

Dimension reductionandfeaturelearning Notonlydeeplearningbutother‘distributed’MLapproaches

Andthelistgoeson….

Page 23: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

MLforDistributedNetworks

ReinforcementLearningAgent

State s

DeepNeuralNetwork

parameterθ

policyπθ(s,a)

Takeactiona

• Whatifwedon’thaveacentralcloudorHPCtotrain?• Localizedlearningversusgloballearning• Learningindynamicenvironments(e.g.changingtrafficdemands)• MLresearchfocusesongamestrategies.Wedon’thavesimilar

“strategies”innetworks

• Learn,Try,Fail,Learn,Try,Succeed!

DQN

Page 24: Machine Learning in WAN Research - Internet2 · • Convolutional neural networks Recurrent neural network Sequential data representation (i.e. time series data) Sequential learning

ConclusionsandContact

• AIshowssomepromise

• Networks+ML+HPC+(complexworkflows)

• Opendatasetsforresearch

• Combiningtechniques(andalgos)toadvanceresearchinexplored:– Newareasinnetworkandperhapsevenmore

Thankyou!

<[email protected]>

FundedunderDOEPanoramaProject(2017-2019),DOEASCR(2017-2022)


Regional Ocean Wave Height Prediction using Sequential ... · Regional Ocean Wave Height Prediction using Sequential Learning Neural Networks N. Krishna kumara,, R. Savithab, Abdullah

Max-margin sequential learning methods

Sequential & Temporally-Delayed Learningski.clps.brown.edu/cogsim/cogsim.8temporal.pdf · Sequential & Temporally-Delayed Learning 1. The Problem. 2. Sequential Learning & Context

Recurrent Neural Networks 1: Modelling sequential data · MLP Lecture 9 Recurrent Neural Networks 1: Modelling sequential data5. TDNNs in action tackle late reverberations, DNNs should

Sequential Decision Problems and Neural Networkspapers.nips.cc/paper/194-sequential-decision-problems...Sequential Decision Problems and Neural Networks 687 instruction about how to

Recurrent Neural Networks - TJ Machine Learning · Recurrent Neural Networks, or RNNs, are a type of neural network that uses sequential data within its model. RNNs aren’t a new

Non-Sequential Neural Network for Simultaneous, Consistent

Recurrent Neural Networks 1: Modelling sequential data · Recurrent Neural Networks 1: Modelling sequential data Steve Renals Machine Learning Practical | MLP Lecture 9 15 November

DISENTANGLING FROM SEQUENTIAL DATA · Advances in Neural Information Processing Systems. 2015., Chen, Xi, et al. "Infogan: Interpretable representation learning by information maximizing

Learning Sequential Decision Rules Using Simulation … · Learning Sequential Decision Rules Using Simulation ... Sequential decision tasks may be characterized by the following

Keywords: Machine learning, reinforcement learning, artificial neural … · Keywords: Machine learning, reinforcement learning, artificial neural network, mobile robot, control,

Sequential Decision Problems and Neural Networks · Sequential Decision Problems and Neural Networks 687 ... Sequential Decision Problems and Neural Networks 689 policy. This function

Sequential Tests for Large Scale Learning - staff.fnwi.uva.nl · Sequential Tests for Large Scale Learning ... sequential hypothesis tests to adaptively select such a subset of

neural network learning

Learning to Remember More with Less Memorization · Memory-augmented neural networks consisting of a neural controller and an external memory have shown potentials in long-term sequential

Machine Learning Deep Learning Neural nets

Seminar fur Statistik¨ Dynamic Neural Regression Modelstresp/papers/paper181.pdf · Seminar fur Statistik¨ ... We consider sequential or online learning in dynamic neural regression

A Neural Implementation of Wald's Sequential Probability ... · 2/18/2015  · Neuron Article A Neural Implementation of Wald’s Sequential Probability Ratio Test Shinichiro Kira,1,2,7,8

An fMRI Investigation of the Neural Bases of Sequential ...sites.bu.edu/guentherlab/files/2016/12/Bohland-poster.pdf · An fMRI Investigation of the Neural Bases of Sequential Organization

Recurrent Neural Networks - GitHub Pages · Recurrent Neural Networks Deep Learning –Lecture 5 Efstratios Gavves. Sequential Data So far, all tasks assumed stationary data Neither

CS536: Machine Learning Artificial Neural Networks Neural Networks

Memory Augmented Graph Neural Networks for Sequential ... · Memory Augmented Graph Neural Networks for Sequential Recommendation Chen Ma 1, Liheng Ma 1,3, Yingxue Zhang2, Jianing

Sequential Supervised Learning

Sequential Learning 1. WHAT IS SEQUENTIAL LEARNING? 2

Similar neural correlates for language and sequential ... · Similar neural correlates for language and sequential learning: Evidence from event-related brain potentials Morten H

Sequential Neural Processes - arxiv.org · Thus, in Neural Processes, unlike typical neural networks, learning a function is fast and uncertainty-aware while, unlike Gaussian processes,

Neural signature of fictive learning signals in a …camerer/PNAS_NeuralSigFctvLrngSgnl...Neural signature of fictive learning signals in a sequential investment task Terry Lohrenz*†,

Deep Learning For Sequential Pattern Recognition

Sequence Modeling: Recurrent and Recursive Netssrihari/CSE676/10.0 SequenceModeling.pdfRecurrent Neural Networks process sequential data •RNNs are a family of neural nets for sequential

Neural Optimizer Search with Reinforcement Learning - … · Neural Optimizer Search with Reinforcement Learning ... Neural Optimizer Search with Reinforcement Learning ... Our approach

Project-Team SequeL · Project-Team SequeL Keywords: Machine Learning, Statistical Learning, Sequential Learning, Sequential Decision Making, Inference Creation of the Project-Team:

Unsupervised Learning With Neural Nets Deep Learning and Neural Nets Spring 2015

Uncertainty in Global Sourcing Learning, sequential

A Hybrid Online Sequential Extreme Learning Machine with ...Index Terms—single-hidden layer feed-forward neural networks (SLFN), minimal resource allocation network (MRAN), extreme

Learning With Sequential Art