[ 1 ]

2018 TechEX, Orlando, FL

Presenters: Rob Vietzke, Ron Milford, Chris Wilkinson

[ 2 ]

NGI Requirements Process

NGI Team Activities to date

NGI Activities at TechEX 2018

Discussion / Next steps

TABLE OF CONTENTS

[ 3 ]

Next Generation Infrastructure Requirements Process

[ 4 ]

• Community discussion about shared future since 2016.– 5+ face-to-face meetings with community leaders– 12+ community leaders calls– Pilots/Proof of Concepts - optical, router slicing, cloud, etc.

• Detailed, phased, planning has begun…– “Open Line System” photonic-layer sharing beginning in 2019– Advanced investments in interconnect, management network in 2019– In 2019, complete more detailed planning for services, including service

models, layer 2/3 and automation/software for future delivery

NGI Background: Community Discussion Evolution

Guiding principles

Ecosystem approach- Focus on joint service delivery model - campus, regional, Internet2

Experimentation- Try stuff, short term commit- No impact on current production

service

Target research end users- Push service delivery edge close

to user

[ 5 ]

• Research Community: Support emerging science infrastructure requirements– Consider instrumented, integrated allocation of 100G+ capacity for research applications – in partnership with regionals

• Research Community: Deliver software-driven infrastructure– More easily integrate with E-2-E science workflows and support API’s/orchestration/automation for big science applications

• Campus & Regionals: Additional end-to-end and infrastructure sharing– Reduce duplicative investment in new capital as well as operating expenses

• Campuses & Regionals: Cloud Access Services– Continue to enable cloud-access including enhanced self-service portal support

• Regionals: Respond to requests to increase capacity offered per dollar– Raise capacity offered for core services: peering, research, cloud and general R&E

NGI Background: Five Community drivers for services:

[ 6 ]

• Shareable Optical Wavelength & Spectrum Services

• Programmable Packet Platform– Research Networks– Academic Enterprise

• Custom Platform Support & Build Your Own

NGI Service Requirements: Cross-Cutting Success Criteria:

• Support of Research• Automated & Programmable• Cloud & Peer Connected• Embedded Security• Regional/National Integration• Infrastructure Sharing• Enterprise Support• Global Reachability• Measurement Services• Research Data Services• Resiliency

• Economics & Scalability• Operations• Secure Management

[ 7 ]

9.41

9.63

7.85

6.10

6.72

7.89 10

.33

9.10

6.91

6.67

7.68 10

.46

17.3

617

.06

14.3

116

.14

21.0

122

.00

27.0

521

.97

17.3

515

.04

19.0

226

.40

28.3

527

.53

23.2

324

.98

22.2

628

.96

29.0

023

.70

22.8

022

.60

27.3

027

.00 44

.82

42.9

435

.45

40.4

045

.60

47.7

550

.23

40.7

733

.53

35.5

342

.95 61

.75

66.8

258

.54

51.5

653

.56

54.2

857

.30

47.4

949

.71

38.9

538

.24

49.4

6 72.7

580

.34

86.4

965

.93 81

.43 96

.22

91.3

310

0.66

82.9

866

.80

68.1

076

.14

105.

2813

9.10

108.

7882

.81 10

4.71

105.

7510

4.04 11

8.11

94.9

273

.77

76.9

9 97.0

0 114.

4914

2.45

142.

5612

4.11

150.

0714

9.87

150.

69 179.

7013

9.95

113.

3510

1.61

138.

50173.17

y = 9.3273e0.0312x

R² = 0.9189

0

50

100

150

200

250

300

350

400

450

500

10/1

/10

11/1

/10

12/1

/10

1/1/

112/

1/11

3/1/

114/

1/11

5/1/

116/

1/11

7/1/

118/

1/11

9/1/

11O

ct-1

0N

oc-1

1D

ec-1

2Ja

n-1

2Fe

b-12

Mar

-12

Apr-

12M

ay 1

2Ju

n-12

Jul-1

2Au

g-12

Sep-

12O

ct-1

2N

ov-1

2D

ec-1

2Ja

n-1

3Fe

b-13

Mar

-13

Apr-

13M

ay-1

3Ju

n-13

Jul-1

3Au

g-13

Sep-

13O

ct-1

3N

ov-1

3D

ec-1

3Ja

n-1

4Fe

b-14

Mar

-14

Apr-

14M

ay-1

4Ju

n-14

July

-14

Aug-

14Se

p-14

Oct

-14

Nov

-14

Dec

-14

Jan

-15

Feb-

15M

ar-1

5Ap

r-15

May

-15

Jun-

15JU

L-15

Aug-

15Se

pt 1

5O

ct 1

5N

ov 1

5D

ec 1

5Ja

n 1

6Fe

b 16

Mar

16

Apr 1

6M

ay 1

6Ju

n 16

Jul 1

6Au

g 16

Sept

16

Oct

16

Nov

16

Dec

16

Jan

17

Feb

17M

ar 1

7Ap

r 17

May

17

Jun

17Ju

ly 1

7Au

g 17

Sept

17

Oct

17

Nov

17

Dec

17

Jan

18

Feb

18M

ar 1

8Ap

r 18

May

18

Jun

18Ju

l 18

Aug

18Se

p 18

Oct

18

Nov

18

Dec

18

Jan

19

Feb

19M

ar '1

9Ap

r 19

May

19

Jun

19Ju

l 20

Aug

20Se

p 20

Oct

20

Nov

20

Dec

20

Jan

21

Feb

'21

Mar

21

Apr 2

1M

ay 2

1Ju

n 21

Jul 2

1Au

g 21

Sep

21O

ct 2

1N

ov 2

1D

ec 2

1Ja

n 2

2Fe

b 22

Mar

22

Apr 2

2M

ay 2

2Ju

n 22

Jul 2

2Au

g 22

Sep

22

Internet2 Carried Load in PetaBytes per Month Expon. (Internet2 Carried Load in PetaBytes per Month)

NGI Capacity Driver: updating scale economies

Oct

‘10

Oct

‘22

[ 8 ]

9.41

9.63

7.85

6.10

6.72

7.89

10.3

39.

106.

916.

677.

6810

.46

17.3

617

.06

14.3

116

.14

21.0

122

.00

27.0

521

.97

17.3

515

.04

19.0

226

.40

28.3

527

.53

23.2

324

.98

22.2

628

.96

29.0

023

.70

22.8

022

.60

27.3

027

.00

44.8

242

.94

35.4

540

.40

45.6

047

.75

50.2

340

.77

33.5

335

.53

42.9

561

.75

66.8

258

.54

51.5

653

.56

54.2

857

.30

47.4

949

.71

38.9

538

.24

49.4

672

.75

80.3

486

.49

65.9

381

.43

96.2

291

.33

100.

6682

.98

66.8

068

.10

76.1

410

5.28

139.

1010

8.78

82.8

110

4.71

105.

7510

4.04

118.

1194

.92

73.7

776

.99

97.0

011

4.49

142.

4514

2.56

124.

1115

0.07

149.

8715

0.69

179.

7013

9.95

113.

3510

1.61

138.

50

173.17

y = 9.3273e0.0312x

R² = 0.9189

0200400600800

1000120014001600180020002200240026002800

10/1

/10

11/1

/10

12/1

/10

1/1/

112/

1/11

3/1/

114/

1/11

5/1/

116/

1/11

7/1/

118/

1/11

9/1/

11O

ct-1

0N

oc-1

1D

ec-1

2Ja

n-1

2Fe

b-12

Mar

-12

Apr-

12M

ay 1

2Ju

n-12

Jul-1

2Au

g-12

Sep-

12O

ct-1

2N

ov-1

2D

ec-1

2Ja

n-1

3Fe

b-13

Mar

-13

Apr-

13M

ay-1

3Ju

n-13

Jul-1

3Au

g-13

Sep-

13O

ct-1

3N

ov-1

3D

ec-1

3Ja

n-1

4Fe

b-14

Mar

-14

Apr-

14M

ay-1

4Ju

n-14

July

-14

Aug-

14Se

p-14

Oct

-14

Nov

-14

Dec

-14

Jan

-15

Feb-

15M

ar-1

5Ap

r-15

May

-15

Jun-

15JU

L-15

Aug-

15Se

pt 1

5O

ct 1

5N

ov 1

5D

ec 1

5Ja

n 1

6Fe

b 16

Mar

16

Apr 1

6M

ay 1

6Ju

n 16

Jul 1

6Au

g 16

Sept

16

Oct

16

Nov

16

Dec

16

Jan

17

Feb

17M

ar 1

7Ap

r 17

May

17

Jun

17Ju

ly 1

7Au

g 17

Sept

17

Oct

17

Nov

17

Dec

17

Jan

18

Feb

18M

ar 1

8Ap

r 18

May

18

Jun

18Ju

l 18

Aug

18Se

p 18

Oct

18

Nov

18

Dec

18

Jan

19

Feb

19M

ar '1

9Ap

r 19

May

19

Jun

19Ju

l 20

Aug

20Se

p 20

Oct

20

Nov

20

Dec

20

Jan

21

Feb

'21

Mar

21

Apr 2

1M

ay 2

1Ju

n 21

Jul 2

1Au

g 21

Sep

21O

ct 2

1N

ov 2

1D

ec 2

1Ja

n 2

2Fe

b 22

Mar

22

Apr 2

2M

ay 2

2Ju

n 22

Jul 2

2Au

g 22

Sep

22O

ct 2

2N

ov 2

2D

ec 2

2Ja

n '2

3Fe

b 23

Mar

23

Apr 2

3M

ay 2

3Ju

n 23

Jul 2

3Au

g 23

Sept

23

Oct

23

Nov

23

Dec

23

Jan

24

Feb

24M

ar 2

4Ap

r 24

May

24

Jun

24Ju

l 24

Aug

24Se

pt 2

4O

ct 2

4N

ov 2

4D

ec 2

4Ja

n 2

5Fe

b 25

Mar

25

Apr 2

5M

ay 2

5Ap

r 25

May

25

Jun

25Ju

l 25

Aug

25Se

pt 2

5O

ct 2

5N

ov 2

5D

ec 2

5Ja

n 2

6Fe

b 26

Mar

26

Apr 2

6M

ay 2

6Ju

n 26

Jul 2

6

Internet2 Carried Load in PetaBytes per Month Expon. (Internet2 Carried Load in PetaBytes per Month)

NGI Capacity Driver: updating scale economies

Likely less accurate the farther in to the future we look, but still significant growth to anticipate.

Oct

‘10

July

‘26

[ 9 ]

47.4

80.3

104.

120.

6

265.

1

351.

8

575.

4 694.

5

1099

.6 1298

.9

1397

.4

R² = 0.9787

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

CY 0

8

CY 0

9

CY 1

0

CY 1

1

CY 1

2

CY 1

3

CY 1

4

CY 1

5

CY 1

6

CY 1

7

YTD

18

CY 1

9

CY 2

0

CY 2

1

CY 2

2

CY 2

3

Internet2 Network Total PetaBytes Carried Per Year (Calendar Year)

Peta Bytes per year Expon. (Peta Bytes per year)

[ 10 ]

Ø Following Global Summit DiscussionsØ Kicked off formal project with 3 areasØ Refocused to enable Agile approachØ Significant progress in many areas

Ø Optical InfrastructureØ Interconnect InfrastructureØ Automation & SoftwareØ Requirements & Business Model

Ø Board Approved Budget (and future forecast) Ø Enables Optical project, secure management network, IXP upgrades to start in 2019

10

Community Service Requirements

& Service Definitions

Automation &Software Planning

Member-Facing ServiceInternal

Operations

Infrastructure Planning

Optical, L2/3 Hardware Programs

Com

mun

ity

Colla

bora

tion Experim

ents

Pilots & Testbeds

Management& Governance

NGI Background: Where are we?

[ 11 ]

Continuing input and feedback on service and investment strategy

• Collaborate with the community to consider what the service delivery model for expanded bandwidth should be and how to deliver that capacity for science, cloud and traditional R&E applications

• Model scalability and sustainability of the technology choices and the service delivery models

NGI Background: Requirements & Service Models

Com

munity

Services

[ 12 ]

Underlying photonic transport system for community wavelength sharing

• Support >100G wavelengths (for capacity)

• Support >35gbd modulation rates (for distance)

• Enable multifaceted “sharing” opportunities with regional partners and community projects

• Drive out scaling costs / find efficiencies

NGI Background: Photonics Investment

• >50 ghz / 35 Gbpd• Disaggregated• Programmable• Efficient

[ 13 ]

Underlying system for community packet services

• Introduce more deeply programmable and

virtualizable network elements to our platform

• Allow us to offer an Nx100G bandwidth increase to

connectors and address demand for capacity

• Adjust approach and topology to increase value

delivered, reduce cost, & enable scale-up

– Includes downsizing many locations to smaller platforms

as well as scaling up core peering locations

– Also includes potential sharing with partners

NGI Background: Programmable Packet Platform

[ 14 ]

New opportunity to automate operations & service delivery; reduce time to service; enable sharing

• Automate internal processes & automate configurations for consistency, rapid delivery

• Add self-service & API features that reduce time to results and enable infrastructure sharing

• Update measurement, analytics and operational transparency tools

• Provide leading network security capabilities that enable and also protect science workflows

NGI Background: Software, Systems, Automation

[ 15 ]

• Our approach has changed away from a single plan, developed at the start and implemented over several years.– Data driven; timely and iterative decision making along a multi-year path

– Still looking for economies of scale on major platform decisions, but less tied to schedules

– Tends to enable more complex decisions and better efficiencies

• Early wins:– Cloud Connect Portal/OESS 2.0– Interconnection 100G platform and capacity increases– Increase in cloud exchange/peering capacity

Being Agile -- wins to date and insights on future timings

[ 16 ]

Next Generation Infrastructure Progress Update

[ 17 ]

What we’ve completed:• Conducted calls with 26 of 48 connectors and network members

• TR-CPS usage analysis / modeling A recommendation to increase the cap to 50G was reviewed.

• Began modelling options and assumptions for a detailed expense model/calculator

• Developing service offering models for discussions with the community in Tempe

• Spinning up an NAOPpag NGI Service Model Subcommittee

What’s Next?

• Create service descriptions and potential business models based upon community input from regional calls, Routed Services Futures WG input, researchers, CIO's etc.

• Working with the NGI subcommittee, prepare for service model discussion with regionals in Tempe

• Maintain an open dialogue with the community, and continuously share feedback with the project team

NGI Requirements Gathering & Business Model

[ 18 ]

NGI Software and AutomationWhat we’ve completed:• Documented initial scope and service requirements• Delivered Cloud Connect Portal Pilot / OESS 2.0• Internal Testbed: Research and test streaming telemetry & integrating automation tools• Arista Testbed: Research Flow detection, dynamic path alteration, and streaming telemetry

• Collaborated with Pacific Research Platform (PRP) and Open Science Grid (OSG) teams on kubernetes, federation, cache and automation to support new OSG nodes in the PRP pilot

• Developed PShooter, a web service (in-development) that works with perfSONAR’s pScheduler to automate performance problem troubleshooting along a path between two points

• Began to conceptualize & design a policy compliance tool

What’s Next?• Continue collaboration with PRP and OSG teams• Automation & telemetry -- configuration management• Vendor engagement & market survey• Software & Automation Testbed

[ 19 ]

NGI Infrastructure: Interconnection ProjectWhat we’ve completed:• Completed peering & TR-CPS baseline data collection & analysis in support of the cap increase proposal

• Evaluated multiple models and completed ROI / cost-benefit analysis of new hardware vs legacy platform investment across the 7 peering sites. A recommendation to deploy MX10003s in Ashburn & Chicago, and redeploy MX960s from Ashburn & Chicago to Seattle, Los Angeles, Dallas, San Jose and NYC was reviewed and approved

– Result: 63% reduction in CAPEX and 31% decrease in OPEX per 100G port

• Continued to invest in the TR-CPS service and coordinated with peers to continue necessary augments

• Collaborated with the Routed Services Futures WG and presented an initial draft of operating principles and approaches for peering

What’s Next?

• MX10003 deployment (Chicago, Ashburn) and MX960 Parts Reclamation and Redeployment

• Coordinate next steps on Router Services Futures Workgroup Report

[ 20 ]

Next Generation Infrastructure Optical Line System Deep Dive

[ 21 ]

Next Generation Infrastructure Service Improvement Announcements

[ 22 ]

[ 23 ]

TR-CPS Cap Increase to 50G

• Increase “cap” from 20G to 50G for TR-CPS on all 100G AL2/3S ports

• Initial analysis indicates this may lead to the need to increase capacity on several segments that are adjacent to peering points

• Will also alter current practice for when to implement capacity augments on large 100G LAG groups

• Coincides with project to install new equipment at key peering points to support 100G capacity upgrades

[ 24 ]

Cloud Connect Portal / OESS 2.0 Pilot

• Introducing Cloud Connect features & Layer 3

VRF’s in the OESS Portal

• Includes Amazon provisioning integration with Google and

Microsoft to follow

• Like Equinix, Megaport and others, campuses and regionals

can provision multicloud VRF services leveraging their R&E

networks

• Introducing “Phonebook” for non-network

engineer users and resources

• Begins to introduce a new “look and feel”

• Pilot open after TechX; feedback welcome

• Some data and provisioning coordination with

regionals likely needed

[ 25 ]

Next Generation Infrastructure Community Activities at Tech EX

Ø Progress on Experiments & Pilots reporting progress at Tech XØ Router Slicing Working Group ReportØ Future Routed Services Working Group ReportØ ”Voyager” Optical Pilot ReadoutØ Cloud Orchestration ExperimentsØ National & Eastern Research Platform Pilots

Ø DemonstrationsØ Cloud Connect Portal / OESS 2.0Ø Streaming TelemetryØ Flow Identification and pathingØ Performance Troubleshooting (pShooter)

Community Activities

26

[ 27 ]

Key Tech EX NGI Activities

NGI Project: Plans for Next Evolution of Internet2 Infrastructure (Vietzke, Wilkinson, Milford) - Now

NGI Ecosystem: Report on Workshops at University of Michigan (Boyd, Vimawala) – Today – 11:20

NGI Ecosystem: Community Experiments Toward the Next-Gen Infrastructure (Moore, Von Oehsen, Lehman, Longo)Today – 1:40

NGI Ecosystem: National Research Platform (NRP)(Moore, Deaton) – Today – 2:40

NGI Ecosystem: Report on the Router Virtualization Pilot (Kara, Nordmark, Wilkinson) – Weds – 8:30

NGI Ecosystem: Desired Future State of Quality of Service (Wallace) – Weds - 9:20

NGI Ecosystem: Routed Services Future Working Group Report (Deaton, Gallo, Schopis) – Weds - 10:20

NGI Project: OESS 2.0 / Cloud Connect Portal(Ragusa) – Weds 10:40

NGI Project: Interactive Session on Plans for Next Evolution of Internet2 Infrastructure(Roos/Vietzke) – Weds 2:00

NGI Project: Interactive Breakout Sessions(Roos/Vietzke) – Weds 2:30

NGI Project: Breakout Reports -(Teams) – Thurs AM

Tutorials & Hackathons

Network Automation WorkshopNetwork Automation Hackathon

Introduction to KubernetesAutomated perfSonar with Ansible

Routing Security

Standing Meetings:Network Connector Member BoF

NTAC Meeting (NGI Voyager Pilot - Owens)

Chalk Talk with Cloud ProvidersAmazon, Google, Microsoft 1-1’s

Demonstration Area:pShooter

Cloud Connect Portal / OESS 2.0Flow Detection, Prioritization & Telemetry

Analytics

Track Sessions

[ 28 ]

Next Generation Infrastructure Discussion / Next Steps