ops forum 25.06.2010 ops/eop collaboration for eo connectivity


Agenda

Background/Context (AR)

OPS-ERO role

OPS/EOP collaboration

WAN Connectivity Activity (CS)

Evolution Activity History

Overview Of Current EO WAN Network & New EO WAN Services

Service Level Agreements & Key Performance Indicators

Network Migration Approach & Schedule

Evolution Activity Management & Challenges

EOP IP-VPN Evolution

Q&A (AR-CS)


Background / Context (1)

OPS-ERO’s multiple role at ESRIN: Provision of Corporate ICT services to ESRIN users. Provision of Technical services Management of evolution activities for ESRIN customers

EOP-G is the main OPS customer at ESRIN for Technical Information Systems, mainly on connectivity-related services and activities

OPS/EOP relationship OPS-ERO provides support to EOP on:

Operational Service Management (ODAD-NS, through OSP) Definition (SoW), negotiation and phase-in of new Operations &

Maintenance contract for the NET work package

OPS-ERO manages EO NW transformation, according to EOP needs and strategy

Relationship is regulated through SLA for corporate and technical services Two-Year Work Plan for evolution activities Agreed EAD and EAP for each activity


Background / Context (2)

Guidelines Prepare for an end-to-end LAN managed service Consolidate WAN connectivity in a unique commercial provider. Current

service is based in several academic providers (NRNs) and a commercial service (Comnet, by OBS)

Develop technical capability to satisfy new-mission requirements (high capacity, SLAs, near-real time)

Transformation Plan EO PDGS NW upgrade at ESRIN: B11 re-cabling and LAN infrastructure upgrade

(completed) EO WAN service Set-up (Interoute), migration (replacement of NRNs and

Comnet), and integration in O&M framework contract (On-going) Consolidation of EO FWs and rationalisation of legacy infrastructure EO LAN harmonisation in remote sites Tender for LAN managed service integrated in O&M contract (through “best

practices” mechanism) EO WAN connectivity upgrade to support new generation missions

(GSC/Sentinel) data circulation and dissemination

EO Connectivity Roadmap


EO WAN Upgrade Drivers & Objectives

Drivers High Availability Single Service Level Agreement Increase performance & reliability Network simplification (Design and Operations; M&C, Reporting)

Objectives

Consolidation of ODAD & COMNET Deployment of centralized Internet service Modular design Removal of obsolete elements Increase scalability factor Migration of existing Public IP Addresses space into ESA Addresses


EO WAN Upgrade Scope

EO Facilities

The WAN infrastructure at the following thirteen EO Facilities will be migrated to the new WAN provider:

Country Facility - LocationDLR - Neustrelitz DLR - Oberpfaffenhofen CNES - Toulouse IFREMER - BrestCNMCA -Pratica di Mare ESRIN - FrascatiASI - Matera

Norway KSAT -Svalbard/ Tromsoe Spain ESAC - MadridFinland Sodankylä Canada CCRS - Gatineau Sweden Kiruna Salmijarvi United Kindgdom Infoterra - Farnborough

I taly

Germany

France


Evolution Activity History 1/2

May-June 2009 RFI – Market Analysis

September 2009 RFI Results – EOP Management Decision

November 2009 WAN Restricted Tender Issued

December 2009 - March 2010 Formalized OPS-ERO EOP-GS activity -Evolution Activity Plan

January-March 2010 ESA-Interoute Negotiation

24th March 2010 Signed Contract ESA-Interoute

April 14th 2010 Evolution Activity Kick-Off Meeting


Evolution Activity History 2/2

Market Analysis

An RFI was issues in order to analyse the market maturity for the provision of commercial high capacity WANs with guaranteed SLA by a single network operator (as an alternative to multiple National Research and Education Networks ) at an affordable cost

Requested to the following providers: Telefónica British Telecom T-Systems Level 3 Communications AT&T Deutsche Telecom Telecom Italia Orange Business Services Global Crossing Colt Verizon Business Interoute

Answer received from: Telefónica, Global Crossing, Orange Business Services, Telecom Italia, Interoute

Interoute demonstrated to be a viable candidate for the provision of EOP connectivity services, Highly competitive on cost Compliant with requirements Based on State-of-the-art technologies and service approach


Interoute: The New EO WAN Provider 1/2

Interoute:

Is one of the Pan-European network carriers considered by Gartner

European company, controlled by the Sandoz Family foundation (Swiss)

Is competitively positioned to carriers able to deliver the services being sought (e.g. Global Crossing, Colt)


Interoute's Strengths

Interoute has an extensive fibre network in Europe based on a multiple 10G Ethernet backbone.

It has strengthened its service portfolio and delivery, including WAN optimization solutions and further enhancements to self-service automation in areas such as delivery and configuration.

Interoute offers strong processes for developing service components, and an extensive component library that enables a wide range of custom-made network-centric solutions

It has a strong portfolio of network-based communication applications such as site-to-site and public IP voice.

Cautions

In some smaller European markets, deep in-country coverage is still missing, affecting pricing and service levels for enterprises with requirements in these markets.

Interoute is focused on network-based solutions and has limited capability in the area of customer premises-based services, such as managed LAN or IP PBX.

(Source: Gartner, December 2008)

Interoute: The New EO WAN Provider 2/2


Overview Of Current EO WAN Network 1/5

Private VPNs over NRENs (ODAD)

ComNet



Country Facility - Location ODAD COMNETDLR - Neustrelitz Yes -DLR - Oberpfaffenhofen Yes YesCNES - Toulouse Yes -IFREMER - Brest - YesCNMCA -Pratica di Mare - YesESRIN - Frascati Yes YesASI - Matera Yes Yes

Norway KSAT -Svalbard/Tromsoe Yes -Spain ESAC - Madrid Yes -Finland Sodankylä Yes -Canada CCRS - Gatineau - YesSweden Kiruna Salmijarvi Yes YesUnited Kindgdom Infoterra - Farnborough Yes Yes

Germany

France

I taly

The ODAD & COMNET networks are deployed at the following EO Facilities:



COMNET Main Characteristics

ESACOM is the ESA wide IP-VPN, based on MPLS and end-to-end managed service

The EO COMNET is a subset of the ESACOM IP-VPN and consists of two IP-VPN communities. Full meshed ENVISAT IP-VPN community to interconnect ENVISAT networks among EO

centres Full meshed ERS-TPM IP-VPN community to interconnect ERS networks among EO centres

5 Classes of Service (CoS) for congestion control and traffic prioritization. Two classes are configured (D1 and D2; D3, RTvo, RTvi not configured)

SubVPN setup on CE routers in case of multiple IP-VPN communities configured within the same site

Guaranteed connectivity with commercial SLAs (Service Availability / MTTR), max Round Trip Delays, Packet Loss, etc.

Low bandwidth transfer rates i.e. 32Kbps – 512Kbps



High-speed IP-VPN over Internet among EO facilities

Access to EO servers within EO DMZs

IP addresses used within EO DMZs are owned by ISP

No SLA

Eight individual contracts and term and conditions to manage

Typical EO firewall infrastructure

National Research and Education Network (NREN) Internet Access Service

Country Facility - Location ISP MbpsDLR - Neustrelitz DFN (X-WIN) 34DLR - Oberpfaffenhofen DFN (X-WIN) 100

France CNES - Toulouse Renater \ Garonne 10ESRIN - Frascati GARR \ Sirti 160/ 196ASI - Matera GARR 50/ 100

Norway KSAT -Svalbard/ Tromsoe UNINETT 34

Spain ESAC - Madrid RedIRIS \ NeoSky 100

Finland Sodankylä TeliaSonera 100

Sweden Kiruna Salmijarvi SUNET \ IRF 100United Kindgdom Infoterra - Farnborough J ANET (UKERNA) 100

Italy

Germany



RAS service provided to the EO industrial community in support of the execution of their contractual tasks from their own companies’ premises

Secure IP network access from remote locations to the Payload Data Ground Segment (PDGS) systems connected within the EO networks

IPSec tunnels over the Internet: no guaranteed bandwidth / performance

Two types of secure network access to the EO PDGS systems are provided to EOP contractors:

LAN to LAN: contractors use systems connected to a trusted remote LAN

PC to LAN: contractors use PCs connected to the Internet, according to the specific security

policy (e.g. protocols, etc.)

Network Remote Access Services (RAS) Characteristics

New EO WAN Service 1/7

New EO WAN Services

IP-VPN Single Access & Dual Homed AccessMulti-ServiceQoS

Internet Central Redundant Central Firewall

Remote Access LAN-to-LAN Token Based

Distributed Denial of Service Protection DDoS Attack Detection & Mitigation

Single & Dual Homed IP-VPN

Offers single or dual connectivity to the Interoute MPLS backbone at high or low access speeds allowing the interconnection of multiple IP-VPN communities which will have the capability to run a range of features such as Quality Of Service (QoS), Multi-VPN and IPSLA for performance tracking.


ESA ESACOM MPLS VPN

ASI (Matera, IT) DLR Oberpfaffenhofen(Wessling, DE)

Infoterra Farnborough (Southwood, UK)

DLR Neustrelitz(Neusterlitz, DE)

Kiruna Esrange(Kiruna, SE)

ESRIN (Frascati, IT)Kiruna Salmijarvi(Kiruna, SE)

Ksat TTS (Svalbard/Tromsoe, NO)

CNES (Toulouse, FR)

ESAC (Villafranca del Castillo, ES)

CCRS (Gatineau, CA)

IFREMER (Brest, FR)

CNMCA (Pratica diMare, IT)

FMI (Sodankylä. FI)

34 Mbps Ethernet3

00

Mb

Multi-Service CPE

This functionality is available at each site and sites can be part as many of the Multi-VPN instances as required.

No Connectivity is permitted between VRF instances – Each instance is a completely isolated IPVPN

ENVISAT IP VPN

ERS-TPM

EOP Internal LAN

Local Internet Breakout

Internet


ESACOM MPLS/VPN Network

QoS

Interoute offer 4 Levels of QoS. The RP+M queue is reserved by Interoute for Management Traffic

Priority Queue: this is a low latency queue best suited to jitter sensitive apps like VoIP.

Interoute QoS implementation has the following features:

Reserves Bandwidth for each queue to pre-agreed levels Allows all queue (except the priority queue) to burst up to full line rate when line is not congested WRED to avoid congestion in TCP flows SLA related information shown in the Interoute Hub on a per queue basis

JitterRTTPacket Loss



Internet Central

Offers controlled and mediated public Internet access through a central Interconnect between the VPN and the public Internet. Internet Central is delivered as a central connection provided in two of Interoute's carrier neutral co-location facilities one in Paris and the other in Frankfurt. The Firewall will act as demarcation point between the public Internet and the private VPN network providing up to 1.2Gbps redundant Internet access.

Current Internet Setup Interoute Internet Central Setup

Remote Access Service

LAN-to-LAN Remote Access

Delivers Site based IPSec with multi-service functionality at all sites. This will be provided using VRF lite on each CPE with a separate physical LAN interface for each VRF. Each CPE will come with the capacity to deliver two distinct VPNs at each site.

Token Authentication Remote Access

The managed token authentication service allows remote users to access local resources via a two factor authentications process. Token Authentication is delivered via the Internet Central Firewall upon which IPSec tunnels terminate and that acts as the gateway between the public Internet and the EO IP-VPN MPLS.



Distributed Denial of Service Protection

The detection process works by analysing Netflow information on Interoute PE routers. This information is fed back to Arbor Netflow Collectors which are used to retain and analyse the Netflow stats, this part of the process also flags up anomalous or suspicious traffic flows to the Interoute NOC.

Once a flow has been detected and confirmed as malicious it is manually forwarded to Interoute traffic scrubbers, these are Cisco Guard devices capable of ‘cleaning’ traffic. This means that attack traffic can be dropped while genuine traffic will be forwarded back to its original destination.

New WAN Reference Architecture

DAP/DMZs

EO remotecentre 2 EO remote

centre 1

ERS/TPM226, 232, 228

GlobalInternet

Internal LANsENVISAT LANs

New redundantCPEs

ODAD-FW

MPLS IP VPN

New SP backbone

ESRIN

Two physical redundant links

Internal LANs

ODAD-FW

Internal LANs

ODAD-FW

ENVISATnetworks ERS/TPM

networks

DAP/DMZs

DAP/DMZs

ENVISATcommunity

ERS/TPMcommunity

Global Internet < > ODAD DMZs

ODAD VPN (traffic within SP backbone)

ENVISAT networks < > ENVISAT PDS / PACs

ERS/TPM networks < > ERS/TPM stations / PAFs

Internet Central

New CPE

WAN

Internet

Internal LANs

ERS/TPM

ENVISAT LANs

New WAN High Level Design

Interoute CPE provides simultaneous access to the EO IP-VPN and Internet

The Intranet interfaces of the CPE devices will be interconnected to the internal LAN

Internet interfaces will be directly connected to the firewall system

DMZ-to-DMZ and Internal-to-Internal traffic

will be no longer routed over the public Internet.

The IP bandwidth is equal or greater that what is in place today

High speed connectivity is available at all locations, however in order to deliver high BW requires in most cases an upgrade of the Interoute infrastructure.


SLA ID KPI SLA Metric TargetSLA-01 Yes Service desk response time ≤ 30 Minutes

Critical ≤ 1 DayNon Critical ≤ 2 Day

SLA-03 Yes Incident Notification & updates ≤ 30 MinutesCritical ≤ 4 HoursHigh ≤ 8 HoursStandard ≤ 5 DaysInternet Central = 100%Single access = 99.5%Dual access = 99.95%RAS = 99.95%DDoS = 99.9%Priority = 99.95%Critical = 99.9%Premium = 99.9%Standard = 99.0%Internet access 99.9%

SLA-07 Yes Round trip delay

SLA-08 Yes Packet jitter ≤ 5ms – 5 minute intervalRCA Critical ≤ 2 DaysRCA High ≤ 2 DaysRCA third party ≤ 5 DaysRCA third party update ≤ 2 Days

Monthly report = 100% SLA-10 No Planned maintenance notification 100%

SLA-11 No Proactive incidents detection 100%

SFO = 10Mbit/ sDatamat = 10Mbit/ sACS = 5Mbit/ s

SLA-13 Yes Change Contract Notice ≥ 90% within the agreed schedule≥ 90% input to written requests100% quarterly reports 100% General support

Yes Service availability

SLA-06 Yes Packet delivery

SLA-09 Yes Reporting

SLA-02 Yes Service request implementation

SLA-04 Yes Maximum time to repair

SLA-05

SLA 14 No Senior Network Architect

SLA-12 No RAS throughput

Service Level Agreements & Key Performance Indicators

Contractual SLAs & KPIs

The services provided by Interoute are regulated by fourteen SLAs and measured by ten KPIs

KPIs are measured on a monthly basis and associated with a penalty scheme based on service credits

Service credits can reach up to 100% of the total monthly charge for the affected site or service in the applicable monthly review period.


Evolution Activity Management 1/2

Evolution Activity Interface Structure

EOP-GS Customer representative

Five EOP-G Technical Officers

Thirteen EOP-G Domains/Sub-Domains

Thirteen Facilities

Two ODAD engineers

Two OPS-ERO engineers

Interoute PM/CSM

Interoute engineering

Lead Engineer

Technical Lead Engineer

ODAD Engineer

Customer Representative

Facility Coordinator

Interoute PM/CSM

Interoute Engineer

Facility Technical Officer

EOP Technical Support

Local Facilities

EOP Services


Evolution Activity Management 2/2

TTO WP600

EOP WAN Upgrade

EA Management WP100

Design WP200

Server IP Migration WP300

Interoute Management WP400

Site – Service Migration WP500

Network Design WP210

Migration Plan WP220

Evolution Activity Work Breakdown Structure

There are six major work packages defined in the Evolution Activity Plan WBS. The EAP describes in

detail all the WPs defining owner, resources, schedule, inputs, outputs and deliverables which are in total 42.

WP400 defines the interaction with the Interoute PMP and is subdivided into twenty sub work

packages


WAN Migration Approach

WAN Migration Approach

The deployment of the Interoute services at the EO facilities will be carried out in a phased approach. The migration steps will slightly differ between facilities however they will be sequential, one facility at a time. The deployment of the Internet Central will be the first service to go live and will be gradually utilized by the facilities that have been migrated.

Pre-Migration ActivitiesDeployment of IP overlay networkDMZ server IP address migration – Over 400 IP addresses to be changedSite SurveyNetwork to Network interface between OBS-InterouteODAD upgrade & reconfiguration

Migration ActivitiesInteroute IP-VPN deploymentInteroute RAS deploymentODAD reconfigurationODAD VPN service migrationESA facility/service acceptance - validationTransfer into Operations/SRR per facility/serviceFinal SRR & Contract transfer to O&M (Serco)


Schedule

Evolution Activity Schedule

A key element concerning the schedule is given by the lead time to deploy the access lines in each facility; it may vary between nine and twenty-five weeks. The lead time depends on the country where the EO facility is located and on the bandwidth that was requested, in some cases civil works will have to be carried out. The final SRR will be held in December 2010.


Evolution Activity Challenges

Main Challenges

Complex network infrastructure

Server IP address migration

Schedule – multiple implementation dependencies

Migration synchronisation between current WAN providers (nine) & ODAD/Interoute/O&M (Serco)

Interaction with EOP domains and services

Interaction with local facilities

Dismissal of COMNET and NREN current contracts

Network downtimes for migration activities


EOP WAN Service Evolution

DAP/DMZs

MPLS IP VPN

PDGSInternal LANs

GlobalInternet

PDGSInternal LANs

New redundantCPEs

New SP backbone

PDGSInternal LANs

DAP/DMZs

Internet Central

PDGSInternal LANs

PDGSInternal LANs

DAP/DMZs

InternetCentralDMZ

The WAN service described within the presentation is the first step of the EO transformation roadmap.New missions such as the Sentinels will produce terabytes of data on a daily basis and require near real time dissemination and high availability services. For these reasons the EO WAN will have to be able to evolve in order to accommodate these needs.

Future Improvements

Improve security, only two Internet access gateways

Reduce operational complexity: Reduce number of firewalls and distributed

systems, two per central location Remove legacy devices

THANK YOU

Jose Antonio Rodriguez Vazquez

Cristiano Silvagni

Questions

Backup Slides Start

ESACOMIP VPN

PROM129ESRIN

PMUC232DLR

PLPA017Maspalomas

PKRN002Salmijarvi

PFAB016Infoterra

PBRI002ASI

PQEZ020 CNMCA

PBES008IFREMER

PYOW013Gatineau

PKRN001ESRANGE

Country Facility - LocationGermany DLR - Oberpfaffenhofen France IFREMER - Brest

CNMCA -Pratica di Mare ESRIN - FrascatiASI - Matera

Canada CCRS - Gatineau Sweden Kiruna Salmijarvi United Kindgdom Infoterra - Farnborough

Italy

Red CE routers = multiple IP VPN community multiple LAN interface installed on CE and SubVPN configured to forbid communications across LAN interfaces.

LAN interface used as gateway for local Payload Data Ground Segment (PDGS) networks to reach other PDGS networks at different EO centres.

COMNET Characteristics

Overview Of Current EO WAN Network

“Green Networks” at remote sites belong to the same security class of the EOP Internal LAN.

“Blue Networks” at remote sites belong to the same security class of the EOP DMZ LAN.

LAN to LAN connectivity among Green networks and/or Blue networks across remote sites is forbidden.

The use of multi-homed systems either on the Green networks or on the Blue networks is not allowed.

The RAS is a modular service so that it can be available at any location.

ESRIN

Serco Frascati Office & Elsag-Datamat

ACS

RAS LAN-to-LAN service is provided at the following locations:

Overview Of Current EO WAN Network

Interoute MPLS Cloud

Internet Central Firewall

Internet

Optional DMZ

Internet Central

Untrusted Interface

Trusted Interface

Sites gain Internet access according to the ESA security policies through a centralised Internet Central Firewall (Cisco ASA)

All traffic to the Internet from the internal networks can be NAT’ed by the FW. DMZ traffic will not be NAT’ed

All inbound traffic towards each facility is filtered by the Internet Central Firewall according to the security policy.

New EO WAN Service

Site I Site IISite III

ESACOM MPLS/VPN Network

eBGP eBGP

Interoute can run on the LAN interface of the CPE

•OSPF

•EIGRP

•RIPv2

•HSRP

Interoute Managed CPE

Interoute PE Routers

Dual Homed Multi-Service IP-VPN

Using this solution both circuits mirror each other and in the event any failure on the primary bearer the secondary circuit takes over – This solution offers a 99.95% availability SLA

New EO WAN Service

EOP Site Current BW Mbps BW Increase Interoute BW Mbps Max Available BW Infrastructure Upgrade ASI (Matera, IT) 100.256 0.00% 100 1Gbps YesDLR Oberpfaffenhofen 100.192 49.71% 150 1Gbps YesInfoterra Farnborough 100.296 49.56% 150 1Gbps YesDLR Neustrelitz 34 0.00% 34 1Gbps YesESRIN 204.096 144.98% 500 1Gbps NoKiruna Salmijarvi 200.384 49.71% 300 1Gbps YesKsat TTS, Tromsoe 34 194.12% 100 1Gbps Yes

CNES ,Toulouse 100 0.00% 100 1Gbps YesESAC 300 0.00% 300 1Gbps NoCCRS, Gatineau, 0.128 1071.88% 1.5 1Gbps YesIFREMER 0.48 316.67% 2 100Mbps YesCNMCA,Pratica 0.192 941.67% 2 1Gbps YesSodankylä 90 11.11% 100 1Gbps Yes

IP Bandwidth Capabilities

The IP bandwidth that will be delivered at each facility in all cases is equal or greater that what is in place today. It is important to note that high speed connectivity is available at all locations, however in order to deliver high BW requires in most cases an upgrade of the Interoute infrastructure.

New WAN Reference Architecture

Backup Slides End

ops forum 25.06.2010 ops/eop collaboration for eo connectivity

Technology