Internet2 Technology Update

Rick SummerhillChief Technology Officer, Internet2

rrsum@internet2.edu

Internet2 Fall Member Meeting9 October 2007San Diego, CA

Introduction

• This session will provide an introduction and summary of many of the technology investigations and developments underway in the Internet2 community

• Technology group on the Internet2 staff• Rick Summerhill, CTO• Eric Boyd, Deputy Technology Officer, concentrating on Network

Architecture and Performance• Ken Klingenstein, Senior Director, concentrating on Middleware

and Security• Matt Zekauskas, Senior Researcher, concentrating on Network

Research

• The session is not meant to include an exhaustive list of everything the community is examining, but rather describe the flavor of new technologies under investigation.

Collaborations • Almost all technology development is done through a

variety of collaborations• Collaborations with members, including campuses,

regional networks, and corporate members• Almost all of the development in middleware, for example, is

done through these types of collaborations.

• Collaborations with the international community or other national networks like ESnet• Much of the work on network performance or architecture

includes these types of collaborations.

• Collaborations with researchers in academia and corporate members• For example, network research

• All of these collaborations are essential to technology development at Internet2

Agenda

• Some Examples• Performance and Architecture• Network Research• Security and Middleware

Examples

• Here are a few examples to illustrate how new technologies are undertaken

• It is crucial that our community push the boundaries on new developments and investigate new ideas.

• In these first few examples, consider• IPv6•Hybrid Networking and the Dynamic Circuit

Network

IPv6• IPv6 has long been an area of emphasis for our community

• IPv6 will likely become very important in the near future given

recent ARIN discussions and announcements about the

exhaustion of the IPv4 address space

• The IPv6 initiative is essentially member driven in our community

• There is an IPv6 working group that meets regularly at the Joint

Techs meetings and there are hands-on workshops to support

deployment

• Many of our connectors and members have deployed IPv6 from a

network centric point of view.

IPv6 Deployment• Although many connectors have deployed IPv6, it is

difficult to gauge the deployment deep into the campus• IPv6 deployment as a network protocol is

fundamentally not difficult• Getting campuses and connectors to support IPv6 on

crucial applications, however, is often problematic• For example, mail servers, web servers, authentication

servers - supporting (and porting, in some cases) critical applications to IPv6 lags

• We encourage you to participate in the IPv6 working group to help set strategic direction for Internet2 in the future

Hybrid Networking• There has been tremendous interest from all communities

associated with Internet2 to examine services that utilize lower layers of the protocol stack along with IP at layer 3

• This has become known as “hybrid networking”• It is motivated by applications from the research and education

community that require greater capabilities• High bandwidth flows (for example, flows that come close to

saturating links in the shared IP backbone)• Flows with special requirements related to quality of service, for

example jitter requirements

• On the Internet2 network, this takes the following form of an IP network together with the Dynamic Services Network:

Nodes

The Dynamic Circuit Network

• A Network using protocols different from the normal IP protocols• A similar model as an IP network, but with different basic

elements - dedicated circuits rather than shared data flows

• Create Circuits (data paths) in seconds for periods of hours to days between hosts

• Hosts might be individual hosts or routers on the IP network

• Tremendous international collaboration on this project - GÉANT2, ESnet and Internet2

• Innovative work involving exchange of topology, path computation and scheduling and signaling using web services

• Demonstration of how this works in the first plenary session

Technology Update:Architecture and Performance

Eric Boyd

eboyd@internet2.edu

CI Components

Network

Performance Infrastructure / Tools

MiddlewareControl Plane

….

Bulk Transport

2-Way Interactive

Video

Real-Time Communications

Applications

Applications call on Network Cyberinfrastructure

….

…. ….Phoebus

Netw

ork C

yberinfrastructure

Measurement Nodes

Control Plane Nodes

Internet2 DCN and HOPI

10 Gigabit Ethernet

1 Gigabit Ethernet or SONET/SDH

OC192 SONET/SDH

I2 DCS: Ciena CoreDirector10 Gigabit Ethernet

1 Gigabit Ethernet

I2 HOPI: Force10 E600

10 Gigabit Ethernet

Internet2 DCN “Circuits”• Physical Connection:

• 1 or 10 Gigabit Ethernet• OC192 SONET

• Circuit Service:• Point to Point Ethernet (VLAN) Framed SONET Circuit• Point to Point SONET Circuit (future)• Bandwidth provisioning in 100 Mbps increments

• How do Clients Request?• Client must specify [VLAN ID|ANY ID|Untagged], SRC Address,

DST Address, Bandwidth• Request mechanism options are Web Service API, Web Page,

phone call, email• What is the definition of a Client?

• Anyone who connects to an ethernet or SONET port on an Ciena Core Director; could be RON, other wide area networks, domain specific applications

Internet2 DCN Circuit IntraDomain

•Source Address•Destination Address•Bandwidth•VLAN TAG (None | Any | Number)

•User Identification (certificate)•Schedule

Client A

Client B

Circuit Request

api

api

Ethernet Mapped SONET or

SONET Circuits

Dynamically Provisioned Dedicated Resource Path (“Circuit”)

Internet2 DCN Service

Internet2 IDC

•api can run on the client, or in a separate machine, or from a web browser

XML

USER API

Actual Network Path

To IDC

Internet2 DCN Circuit InterDomain

• No difference from a client (user) perspective for InterDomain vs IntraDomain

RON Dynamic Infrastructure Ethernet VLAN

RON Dynamic Infrastructure Ethernet VLAN

Internet2 DCS Ethernet Mapped SONET

1. Client Service Request2. Resource Scheduling 5. Service Instantiation (as a result of Signaling)

A. Abstracted topology exchange

AA

22

1

USER API

XML

Internet2 DCN Current Status

• DCN Infrastructure Deployed• DCN Control Plane deployed and under test

• available for use for early adopters

• General DCN availability planned for January 2008

• Instructions for those interested in using Internet2 DCN or in deploying their own dynamic network will be made available soon

c

Phoebus Current Status

• Developed at University of Delaware (Martin Swany)• Transport Middleware

• Configuration per route/host/user• UDT for inter-depot communication• Transparent operation (library, iptables)

• Simple file transfer tool (scp)• Transparently use Phoebus/Dynamic Circuits

• Leverage Control Plane• Allocate dynamic circuits across Oscars (DCN, others)

• Authentication and Authorization (currently primitive)• Future: Utilize Measurement Infrastructure

• Help find best routes, provide information about paths and achievable bandwidth

Internet2 Active Measurement Tools

• OWAMP (Latency)

• v3.0c (RFC 4645 version) available now

• Regular tests between all routers, and on-demand

• BWCTL (Throughput)

• v2.0 version under development

• Regular tests between all routers and on-demand

• NDT (User Diagnostic)

• v3.4.1 available now

• Latest version added better logging and error handling

• NPToolkit (Active Measurement Tool Package)

• v1.7 available now

• Knoppix Live-CD bootable system

Internet2 Passive Measurement Tools

• Circuit Status Service (E2EMON)

•v1.0

•Internet2 implementation of European tool

•Circuit Status service, Link Status service,

Topology service

• Netflow

•Anonymized, available to researchers

Internet2 Measurement Framework

• Why do we need an end-to-end measurement framework?• Most organizations can do monitoring and diagnostics of their own

network

• Networking is becoming an increasingly cross-domain effort

• Monitoring and diagnostics must also become a cross-domain effort• What is perfSONAR?

• A set of protocols and schemas for implementing a service-oriented architecture for sharing and controlling network performance tools

• A community of users and developers (Internet2, ESnet, GEANT2, and RNP)

• A set of software (the sample implementation)

Internet2 perfSONAR Current status• perfSONAR UI v0.9 available• Java release v2.1 available• perfSONAR-PS

•Perl versions of perfSONAR services written by Internet2, ESnet, FNAL, SLAC, and UDel

•Now Available: Micro-releases of Circuit Status Service, Link Status Service, Lookup Service, Topology Service, SNMP MA

•Under Development: Micro-releases of perfSONOBUOY, and PingER

•perfSONAR-PS bundle release planned for early ‘08

Technology Update:Network Research

Matt Zekauskas

matt@internet2.edu

Research Support in Internet2

• Research on the network• Learning from measurements• Ability to test new theories, protocols and

components

• Research using the network• All kinds, not just “network research”•Much tends to be “big science”, but it also

spans a wide range including new methods of interaction and learning

Philosophy

• Internet2 does not do network research per se, but seeks to facilitate and support research projects led by faculty at member institutions• Make accessible network resources readily

available to this community• Participate in research collaborations and

provide support for proposals• Integrate research findings into the evolution of

Internet2 network initiatives and services

Making Resources Available

• Primarily through Internet2 Observatory

• Two pieces•Measurements of Internet2 Network made

available• Measurements for operations• Measurements specifically for research

•Opportunity to collocate equipment where it makes sense to do so

Existing Measurement Capabilities

• One way latency, jitter, loss• IPv4 and IPv6 (“owamp”)

• Regular TCP throughput tests – ~1 Gbps• IPv4 and IPv6; On-demand available (“bwctl”)• ~10GE now also possible (Myricom and Dell 1950, must ask)

• SNMP• Octets, packets, errors; collected 1/min

• Flow data• Addresses anonymized by 0-ing the low order 11 bits

• Routing updates• Both IGP and BGP - Measurement device participates in both

• Router configuration• Visible Backbone – Collect 1/hr from all routers

• Dynamic updates• Syslog; also alarm generation (~nagios); polling via router proxy

Dataset Use

• Major consumption• Flows

• Most popular (but also one that must be asked for)

• Routes• Configuration

• Nick Feamster (while at MIT)• Dave Maltz (while at CMU)

• Papers in SIGCOMM, INFOCOM• Hard to track folks that just pull data off of web sites

Current Collocation

• VINI, a Planetlab followon• Will provide some sort of private network• Congruence with routed network useful

• 100x100: programmable network processors• Again, want private interconnect• More details in Research talk

• Phoebus• Break TCP sessions to allow hosts that are not

tuned or on flawed networks to effectively use wide-area network

• May also take advantage of circuits or non-TCP

Current Research Collaborations

• Ultralight (NSF)• Research support for upcoming LHC Physics data flows• Project led by Caltech

• 100x100 (NSF)• Focused on understanding the technical & economic requirements for

providing 100-Mbps connectivity to 100 million U.S. homes• Project led by CMU, Stanford and Rice

• Hybrid Multi-layer Network (DoE)• Look at interoperability issues with new dynamic circuit networks. Data

plane interoperability, control plane interoperability…• Project led by U New Mexico, USC ISI; includes ESnet and

UltraScienceNet

Other, More Ad-Hoc, Collaborations

• Buffer sizing project (Stanford):• Reduce buffers available to router interfaces

(software controlled)• Take an anonymized but correlated packet trace• Look for throughput and latency anomalies

• Rapid raw SNMP to test link capacity measurement programs

• Occasionally run programs on behalf of researchers on backbone machines

Small Grant Participation

• Network Measurement for International Connections• I’m PI, but work is done in close collaboration

with Matt Mathis (who also has a small grant) and the International Research Network Connection PIs.

• Research current state and propose solutions• Suggest common measurements• Identify areas for improvement• Work to establish a program-wide

measurement group

Futures

• Work with Research Advisory Council to determine futures

• Restart some focus on outreach and dialog that was begun under a different small grant on the use of Internet2 facilities for research

• Provide the best possible data from our network, and facilitate other opportunities that come our way

• Come see the Network Research update late this afternoon for more details on current activity

Technology Update:Security and Middleware

Ken Klingenstein

kjk@internet2.edu

Security

•REN-ISAC - http://ren-isac.net/•CSI2•Real time security exchanges•Google analytics

•Disaster Recovery•FWNA and eduRoam

Middleware Developments

• SAML and Shibboleth

• InCommon and international federations

• Collaboration management platforms

• NSF-Mellon Scientific and Scholarly Workflow

SAML and Shibboleth

• Shibboleth 1.3 widely deployed as federating software; openSAML widely used as a library

• Shibboleth 2.0 completes Shib/SAML integration; now in beta

• Missing pieces (e.g. personal attribute release) becoming evident and being addressed

• Google, MS, others now provide some financial support; service companies now available

InCommon

• Growing steadily now; 65 members and 1.3M user base

• Major applications include outsourced services, content providers, wiki and collaboration tools

• NIH and federal follies elsewhere

• Apple, Google and Microsoft in contract review

• InCommon Bronze and Silver now under discussion

Prague Meeting on Inter-federation

• 15-20 International R&E federations (5 continents) plus Liberty Alliance and a few others

• Prague, September 3• Lots of topics: Attribute mapping, Privacy

Policies, Dispute resolution, Financial considerations, Technical direction setting

• UK drafting an analysis of International Peering needs, opportunities, etc.

Peering Parameters

Parameters:

•LOA•Attribute mapping•Legal structures• Liability• Adjudication•Metadata

•VO Support•Economics•Privacy

Collaboration Management Platforms

• Management of collaboration a real impediment to collaboration, particularly with the growing variety of tools

• Goal is to develop a “platform” for handling the identity management aspects of many different collaboration tools• Platform includes a framework and model, specific running

code that implements the model, and applications that take advantage of the model

• This space presents possibilities of improving the overall unified UI as well as UI for specific applications and components.

COManage

• Leverages federated identity and the attribute ecosystem heavily

• Shib-enabled; uses Grouper to manage groups, Signet to manage privileges, Eddy for diagnostics

• Built completely on open protocols, using open source components

• Open and proprietary applications can be plumbed to work with it• Sympa, wikis, audioconferencing, sharepoint, calendaring are

comanageable, to varying degrees, now• Web-based file shares, rich wikis next…

Comanage dimensions of growth

• In the applications that can be driven by it• Collaboration and domain science prime areas• Largely a function of the application’s respect for

middleware

• In the areas being managed - diagnostics

• In the identities being managed

• In the coupling of autonomous and diverse instances

Upcoming Talks

• Middleware: The Big Picture Gets Bigger• Happening now, look at slides online

• Network Research Update• Tuesday, 4:30, Grand Hall

• Performance Update• Wednesday, 10:30 AM, Golden West

• Dynamic Circuit Network Update• Thursday, 8:45 AM, California Room

• General Session: Cyberinfrastructure: The Way Forward• Thursday, 10:15 AM, Grand Hall