The ACTION Project Preliminary Results and Project Plan

Eiji Oki, Naoaki Yamanaka, Andrea Fumagalli and Malathi Veeraraghavan

JUNO PI Meeting

June 25, 2014

1

PI cover area

MV

EO

NY

AF

Data-Center +

Network

Model +

Optimized Method Optical Transport

Energy efficient

Experiment Design Application

Platform

Outline

• Background

• Objectives and work program organization

• Short presentations from the individual PIs

• Research plan for 36 months with milestones

• Collaboration plan

• Dissemination plan

3

Background • New enabling technologies

1. Elastic Optical Networks (Flex Grid) [a] • WSS, Bandwidth Variable Transceivers (BVT), Energy Efficient Ethernet (EEE)

2. Sub-wavelength circuits (OTN, DSON) 3. Dynamic circuit/virtual circuit (VC) technologies

• MPLS, VLAN, Ctrl. plane solutions (RSVP-TE, PCEP)

4. Software Defined Network (SDN)/OpenFlow

• Measurements show that – Internet links are underutilized [b] – Networks are not operated in an energy-efficient manner [c]

[a] Gerstel, O.; Jinno, M.; Lord, A.; Yoo, S.J.B., "Elastic optical networking: a new dawn for the optical layer?," IEEE Comm. Mag., February 2012

[b] Sushant Jain, et al., B4: experience with a globally-deployed software defined WAN, SIGCOMM '13. [c] Dennis Abts, et al. Energy proportional datacenter networks. SIGARCH Comput. Archit. News June 2010

4

Good reasons for operating today’s Internet links at low utilization (25-35%) Challenges of high-utilization operation of a network

1. Failure handling: Additional network load placed on links 2. Long-term growth: A provider upgrading the network in say 2012 designs

network to handle loads upto say 2017 – http://es.net/overview-of-the-network/network-maps/historical-network-maps/

3. High-speed file transfers: dedicated data-transfer nodes are designed to move data at high speeds relative to link capacity – e.g., ESnet 100G testbed has hosts that can each push data at 40 Gbps – with striped transfers across three hosts, can fill 100G links

4. Avoid packet losses: TCP throughput sensitive to losses 5. Allow sufficient headroom for poor planning (imperfect traffic forecasts) or

poor routing (imbalanced load)

5

Objectives • Develop an Applications Coordinating with Transport, IP,

and Optical Networks (ACTION) architecture – by integrating four enabling technologies – by operating links at higher utilization while meeting the five

challenges of high-utilization operation

• Why operate at high utilization: the network will need fewer powered-on links, and hence the network will consume smaller levels of energy (power) to move the same number of information bits (bits/sec) – analogy: flying a half-empty large airplane!

6

ACTION Project Overview Core provider networks

Metro/access provider networks

Campus networks

Campus networks

Metro/access provider networks

ACTION Management System

Track 3: Introduce 4 new technologies into datacenter networks

Tracks 1 and 2: Introduce 4 new technologies into core (also metro) networks

ACTION SDN controller

Parallel links

FlexWDM DSON

FlexGrid/OTN/DSON

IP/MPLS/VLAN

OF

OF

ACTION SDN controller

Datacenters

Track 4: Introduce 4 new technologies into campus networks

7

Hosts

IP/Eth/VLAN

Access link

c11

c12

c1u

cw1

cw2

cwu

Cro

ss-b

ar

cir

cuit

sw

itch

ADC

Receiver

DS

P 1

DS

P

ADC

Transmitter

w ADC

Receiver

DS

P w

DS

P

ADC

Transmitter

s1 s2 su

O/E 1

O/E

E/O

E/O

s1 s2 su

s1 s2 su

s1 s2 su

Track 0: ACTION PCE algorithms and design

Long-term Vision (with Current Focus) Broaden scope of dynamic circuit/VC services

8

Circuit/VC endpoints

IP routers Computers (endpoints may be closeby switches/routers)

Intradomain Inter-domain

Datacenters

Campuses

Intra-datacenter Inter-datacenter (intra- or inter-domain)

Inter-domain Intra-domain

Trigger to setup/release circuit comes from NMS/admins

Trigger to setup/release circuit comes from user applications

Track 1: semi-static Track 2: dynamic

Track 3: dynamic

Track 4:dynamic

distributed hadoop job scheduling

Work Program Organization • Track 0: ACTION SDN controller

– Controls both OpenFlow Ethernet switches/IP routers PLUS optical crossconnects

• Four application tracks (applications for dynamic circuit services) – Routers are circuit endpoints (aggregate traffic: ACTION Management System)

• Track 1: Virtual Topology Management: Leverage long-timescale variations (such as night/day traffic patterns) to power off or reduce link rates for energy savings while planning for failures

• Track 2: Link Self-Sizing : Analyze short-timescale variations by observing IP-network link-level traffic (via SNMP MIB reads) and then ask ACTION SDN controller to adjust rate of elastic optical paths (used to realize IP-layer links) whenever possible for energy savings

– Computers are circuit/VC “endpoints” (individual flows: application triggers from endpoints)

• Track 3: Hybrid Data-Center Networks: EON + applications (Hadoop scheduling)

• Track 4: Campus Networks: Router access links adjustment

9

Track 0: ACTION SDN Controller • New path computation algorithms

– Take into account Quality of Transmission (QOT) metrics – Consider energy consumption – Account for failures – Handle traffic fluctuations – Intra-domain path selection – Inter-domain path selection (East-West API) – Multi-layer path selection

• e.g., by coordinating with per-layer SDN controllers

• Architecture, design and prototyping – Reduction of circuit setup delay

Domain definition: a network owned and operated by one organization

10

ACTION SDN Controller

11

Core, metro, campus, datacenter networks ACTION SDN

controller

Parallel links

FlexWDM DSON

1GE increments

Power consumption factors • Router I/O BVT • FFT for DSC multiplexing • Optical amplifiers

OA

Circuit setup delay factors • Control plane • Optical devices (OA, WSS) • Tx/Rx re-synchronization

Quality of Transmission (QoT) • OSNR and BER • Traffic Independent PLIs • Traffic Dependent PLIs

WSS

OXC

Router or switch

Confidential

(Structure and function of ACTION SDN) • Quality of Transmission (QoT) • Circuit setup performance • Power consumption of NW

Track 1 (lead: Eiji) • Issued to be addressed

– Difficult to estimate exact traffic demand each edge nodes – Avoid frequent dynamic route changes according to traffic fluctuation

• Objective – Virtual Topology Management: Leverage long-timescale variations (such as

night/day traffic patterns) to power off or reduce link rates for energy savings while planning for failures • Account for failures • Handle traffic fluctuations • Utilize flexibility of elastic optical networks

• Application – Robust and stable energy-efficient network – Changeable traffic demand of IP and layer 2 networks

12

Virtual topology management for aggregate traffic

13

Action Management System

(virtual topology management) ACTION SDN

controller

Monitor aggregated traffic, but not exact traffic matrix, allowing traffic fluctuation - Daily traffic (day, night, …) - Monitoring point

- Incoming/outgoing traffic volume at edge node - Traffic volume passing through each link

Aggregated

link traffic

Determine robust and stable IP/Layer 2 routing Bandwidth allocation Link reinforcement (protection, bandwidth) in cooperated with elastic optical networks

Incoming/outgoing aggregated traffic

ACTION PCE

Confidential

(Aggregated traffic model) • Path Computation Engine • ACTION SDN • Monitor traffic • ACTION Management system

Work plan (Track 1) • Virtual topology optimization

framework developed with

– Suitable traffic demand model

– Failure model

– Multi-layer orchestration

• Simulator developed – Simulate energy saving effect in cooperation

with elastic optical network simulator

– Provide simulated for integrated demonstration

14

Pipe

model

Hose

model Range of

traffic matrix Narrow Wide

Energy

Efficiency

High

Low

Suitable

traffic

model

IP networks

Optical networks

API

Track 2 (lead: Naoaki) • Aggregate flows (aggregate link traffic: all flows): Next, consider aggregate traffic. In

today’s network (commercial and REN), utilization is around 30-35%. Say we decide to operate at 70% utilization by lowering the link rates (using Bandwidth Variable Transceivers). We start using a network management system (NMS) read SNMP MIBs at IP routers of all links and monitor these levels. We can set some thresholds to adjust link rates, e.g. (65, 75)%. In 30 sec or min.

• Objective: – Bring 2 new technologies to improve IP network performance using elastic optical technique

• Applications: – Improve link efficiency by leveraging SNMP MIB data – Change virtual link bandwidth to maintain a predefined link utilization, e.g. 70%

• Study issue 1: Speed of SNMP is enough? Or new NFV is needed? • Study issue 2: BVT and elastic network adjustment speed: P2P link vs. network path

15

Track 2 : Self-sized link by SNMP data

Today : link= fixed bandwidth fiber

SNMP MIB

16

Simple (starting pt) heuristic 1. read past 30 sec byte count 2. forecast that next 30sec byte count is going to be same

Router 1 Router 2

1

Action Management

System (link self-sizing)

2

4 4 OpenFlow message to increase/decrease spectrum allocation

3

ACTION SDN controller

ACTION PCE

1Gb/s Fixed

→Bottle neck link →Low utilization ACTION: self-sized virtual link

Elastic Optical

3. create a SDN-northbound request (endpoints: router1 and router2, rate) to send to SDN controller asking for BW modification of link; wait for response

Self-sized Link 100%

Confidential

(Link Self-sized technique) • Path Computation Engine • ACTION Manager • ACTION SDN Controller • Elastic Optical Network

Method and Advantages (Energy saving and higher throughput)

17

G

Traffic Monitor (SNMP-Mib) Forecast 30sec

SDN North band IF OpenFlow control

Bandwidth flexibility Power Scalability

Higher Throughput or high-link utilization

Step1

Step2

↓

Step3

Step4

↓

Elastic optical

↓

Advantages

↓

Self-sized link → Self-sized MiDORi Network

Confidential

(Method and Advantages) • Monitor-Control-Openflow switch • Power consumption results • IP throughput

Work Plan (Track 2) • Try to obtain real SNMP traces • Measure Energy Efficient Ethernet (EEE) power consumption and response

time for making rate changes (also make projections for future BVTs) • Simulate Internet2 or ESnet or SINET using ns2 or P2A simulator to

determine the optimal timescales and characterize energy savings tradeoffs (with and without elastic optical path modifications).

• Network structure by multiple parallel links or VON( Virtual optical network)

• Prototyping in Keio Univ. NW with test with VLAN switches on testbeds emulation and JGN using deeply programmable node – Test assumptions – Collect measurements

• Extended ideas to virtual optical network slice and/or inter datacenter

18

Track 3 (lead: MV)

• Objective: – Bring 4 new technologies to datacenter networks

• Applications: – Hadoop scheduler extended to co-schedule CPU

and network resources (trigger dynamic circuits)

– Filesystem writes of large files (host-application triggered dynamic circuits)

19

ACTION Track 3 Core

Aggregation

Edge

Pod 0 Pod 1 Pod 2 Pod 4

Hybrid OpenFlow packet/optical circuit networks OSCARS: On-Demand Secure Circuits and Advance Reservation System

FlexiGrid/ DSON optical switch

SDN controller/OSCARS ACTION Hadoop scheduler (integrated CPU + network)

Dyn. circuit oriented apps on servers

Power-on second NIC and Ethernet switch port only when required

Dynamic circuit triggered by scheduler or host apps

20

Confidential

(Data center model) • ACTION Hadoop Scheduler • SDN Controller • Optical Dynamic Circuit by apps.

Work Plan (Track 3)

• Prototype applications and test with VLAN switches on testbeds (e.g., ExoGENI, PRObE, Keio test-bed, CPqD test-bed) – Test assumptions

– Collect measurements

• If possible, obtain real traces from datacenters

• Run simulations or create analytical models of large-scale datacenter network to estimate energy savings

21

Track 4 (lead: Andrea)

• Objective: – Bring 4 new technologies to campus networks

• Applications: – Network administrator requests temporary

augmentation of access link capacity for special events on campus (e.g., football game)

– Host-application triggers dynamic circuits (e.g., moving human genome sequence rough or processed data)

22

Current Campus

23

Metro/access provider network

Campus networks

Fixed Ethernet rate, e.g. 1GE, 10GE

Hosts

IP/Eth/VLAN

Access link

Metro/access provider network

ACTION SDN controller

Port(s) that is (are) shared dynamically

FlexWDM DSON

Hosts

Access link

ACTION Track 4

ACTION SDN controller

Hosts

ACTION SDN controller

ACTION SDN controller

Port that is dynamically powered on and off

Genome app.

Campus networks

Football game

Save energy Improve utilization

Access links

Confidential

(Campus network and inter-campus) • Metro/access elastic network • Inter-Campus dynamic network

Work Plan (Track 4) • Prototype campus applications and test with VLAN switches on test-

beds (e.g., Keio test-bed) – Test assumptions – Collect measurements

• Define and implement simulation modules based on experimental data (e.g., CPqD test-bed) – PLI models for EON and DSON – Circuit setup delay models for optical components, physical control

plane and signaling – Energy consumption models

• Obtain real traffic traces from campuses • Run simulations and/or create analytical models of campus network

to estimate energy savings 24

Outline

• Background

• Objectives and work program organization

• Short presentations from the individual PIs

Research plan for 36 months with milestones

• Collaboration plan

• Dissemination plan

25

26

Y Track 0 Track 0 - QoT Track 1 Track 2 Track3 Track 4

0.5

ACTION Architecture&Design (NY,

EO, MV, AF) Non-coherent TI-PLIs

0.5

Estimate of power

consumption for key already

existing devices

Modeling and formulation for

IP/layer 2 switch networks

(EO,MV)

1 Intra-domain PCE (EO,AF)

OA: EDFA non-flat gain and

noise level from experimental

work

Test hypothesis of day-night

traffic patterns being different

(MV)

Analyze Internet2 SNMP MIB

traces: heuristic for rate-

adjustment duration (MV,NY)

1

Coherent TI-PLIs and TD-PLIs

and WSS equalization

Modeling and formulation for

IP+optical networks (EO,AF)

Obtain energy and response time

measurements for EEE and switch

port (NY)

1.5 Cross-layer PCE (EO,AF)

Designing & solving PCE for

IP/layer 2 and optical PCE

(EO,AF,NY) Modeling and formulation (EO)

ACTION Hadoop scheduler +

characterize comm. traffic (MV,NY)

1.5

PCE prototype for IP/layer 2

networks w/ EON interfaces

(EO,AF,NY)

Circuit setup latency models

for control plane and key

optical devises paper #1

Prototype Link Self Sizing

Emulation Module of ACTION

Mgmt System(NY,MV)

Energy and response time data for

Ethernet and optical switch

configuration (NY)

2

Prototype SDN Controller (IP and

layer 2) (NV,NY)

Prototype system demo (NY and

MV); Performance evaluation (EO)

Prototype Integrated system demo:

apps+scheduler+SDN controller

(MV and NY)

Analyze campus access NetFlow

records for alpha flows and SNMP

data (MV)

2

DSCM transmission and power

consumption models paper #2

DSCM Architecture and spectrum

allocation algorithms (AF,NY)

Applications redesigned to make

full use of circuits oriented

services (MV)

2.5

Inter-domain PCE RSA strategies

(MV,NY,EO, AF)

Contain circuit setup delays and

their impact on applications

(AF,MV)

Definition of APIs required

between application and SDN

controller (MV, NY, AF)

2.5

SDN controller East-West

interfaces paper #3

Define SDN controller East-West

SDN interfaces (AF, NY, MV)

3

Campus application-SDN

controller APIs

Prototype system demo (NY and

MV)

3 paper #4

Collaboration Plan • Have been meeting via WebEx (almost) every week since the start of the project • IP Agreement is finalized • Face-to-face meetings done: NOC14, ICC14 and PI meeting (June 2014)

– Plan by coordinating conference attendance – Also, hold dedicated meetings

• Dropbox folder: already under heavy use! • Joint platforms for simulations, software/shell scripts for running experiments,

software for data analysis • Support graduate student/PI visits to other labs for more immersive learning • Joint workshop

27

Dissemination Plan • http://venividiwiki.ee.virginia.edu/mediawiki/index.php/ACTION

• Papers: Journals and conferences – COIN paper

• Software: post on wiki site

• Data: store measurements from testbed, SNMP from Internet2, SINET in university library data stores

• Organize workshop or special session – IEEE/IEICE HPSR 2016 (TBD)

28