network planning & design: an art or a science?

Network Planning and Design: Network Planning and Design: An Art or a Science?An Art or a Science?

Vishal Sharma, Ph.D.Metanoia, [email protected]://www.metanoia-inc.com

Metanoia, Inc.Critical Systems Thinking™

© Copyright 2007All Rights Reserved

Keynote Talk, SANOG 9, Colombo, Sri Lanka, 23 January 2007 2


Copyright 2007All Rights Reserved

Agenda

Overall network planning process

Network design -- control at different timescales

“Science” of network planning/design Architecture and design philosophy

Network planning functions by organization – ops, eng., & planning

“Art” and “science” of network planning/design Best practice examples – GBLX, Sprint, Reliance

Role of automated TE and planning tools

Conclusions

Overall Network Design Overall Network Design ProcessProcess





A View of the Overall Network Planning/Layout Process

SS7 SignalingNetwork Design

Optical Link Configuration

Bandwidth Requirements

Link Requirements

Switch NetworkDesign

ATM NetworkDesign

IP/MPLS NetworkDesign

SONET/SDHNetwork Design

Optical RingNetwork Design

Optical MeshNetwork Design

Traffic DemandGeneration/Estimation

SONET/SDHSynchronization

We are here

Note: In a combined multi-layer network design, this strict division may not hold




Network Design: Exercising Control over Different Timescales

Forecast traffic demandsTraffic requirements

Performance objectives

Traffic measurement dataObserved network statesForecast

Load

Load Uncertainties

Traffic Managementpkt. level processing,

routing control, congestionmanagement

Capacity Managementcapacity allocation, cap.planning, routing, design

management

Network Planning network dimensioning,modeling, perf. analysis,

what-if analysis

ms, sec, min

days, weeks

months, years

Controls(config/reconfig)

ActualLoad

Network

Datacollection

Network modeling, layout, &expansion process

The “Science” of Network The “Science” of Network PlanningPlanning





Network Design System: Architecture

Centralized or distributed?

Obtaining network topology &

utilization info?

Inputs

Their formats?

Interface with n/w elements?

Outputs

Their formats?

Installing routes/LSPs in n/w?

Path route computation

On-line, dynamic?

Off-line, global?

Combination of above?

Route computation trigger

User? Administrator?

New request(s)?

Scalability

# of links & nodes handled?

# of flows, LSPs, circuits?

# of constraints handled?




Network Design System: Philosophy

Control-centric

Long time scales

Large granularity flows

Verisimilitude-centric

Pkt. by pkt. sims. of elements & network

Traffic trace-based perf. simulations

Hybrids

Long time scale analysis for network perf.

Pkt. level sims. for element & flow performance




Key Carrier Network Design Functions by Organization

Operations Engineering/Architecture Planning

Network design + opt.

Flow analysis

Simulation &experimentation

Expansion planning

Multi-protocolmodeling

Survivability analysis

Economic analysis

Monitoring

Diagnostics

Validation

1

2

3

4

5

6

78

9

10

The “Art” of Network Planning The “Art” of Network Planning (and some “science”)(and some “science”)





Global Crossing IP Backbone Network

100,000 route miles 27 countries 250 major cities5 continents200+ POPs

Courtesy: Thomas Telkamp, GBLX




Global Crossing: Network Design Philosophy

Ensure there are no bottlenecks in normal state

On handling congestion

Prevent via MPLS-TE

Manage via Diffserv

Over-provisioning

Well traffic engineered network can handle all traffic

Can withstand failure of even the most critical link(s)

Avoid excessive complexity & features

Makes the network unreliable/unstable




Global Crossing’s Approach: Big Picture

WebServer

HR

DR BR

AR

CR

WR

DR

HR BR

AR

CR

WR

DR

HR BR

AR

CR

WR

EthernetSwitch

ModemBank

To other ISPs

To Customers

POP1

POP2

POP3

AR = Access Router

BR = Border Router

CR = Core Router

HR = Hosting Router

WR = WAN Router

DR = DSL Aggregation

OC-3/OC-12

OC-12/OC-48

OC-48/OC-192




Global Crossing’s LSP Layout and Traffic Routing Methodology

Region 1 Region 2

Region 3

Region 4

POP1POP3

POP4

POP5POP2

Full LSP Meshin Core

Core LSP betweenWRs in POPs 1 & 5

Source

Destination




SprintLinkTM IP Backbone Network

19+ countries

30+ major intl. cities5 continents(reach S. America as well)

400+ POPs

Courtesy: Jeff Chaltas Sprint Public Relations

Represents connectivity only (not to scale)

110,000+ route miles (common with Sprint LD network)




SprintLinkTM IP Design Philosophy

Large networks exhibit arch., design & eng. (ADE) non-linearities not seen at smaller scales

Even small things can & do cause huge effects (amplification)

More simultaneous events mean greater likelihood of interaction (coupling)

Simplicity Principle: simple n/wks are easier to operate & scale

Complexity prohibits efficient scaling, driving up CAPEX and OPEX!

Confine intelligence at edges

No state in the network core/backbone

Fastest forwarding of packets in core

Ensure packets encounter minimal queueing




SprintLinkTM Deployment StrategyL2 failure detection triggersswitchover before L3 converges

ZA

Parallel links 50% utilizationunder normal state

1

2

3

4

SONET framing forerror detection

LineCard

LineCard

SONETOverheadIP Data




Reliance Communications: India Network

80,000-100,000 route kms

4000 towns, 350, 000 villages

3+ p2p paths on metro rings

Ring + mesh design, 130+ rings

Source: FLAG Telecom Extranet




Reliance Network Design Philosophy Be integrated telecom provider: e2e ownership of entire architecture

Access systems, national fiber-optic backbone, international cable systems, Internet gateways, wireless base stations, and aggregation devices

80,000 km of high-capacity, pan-India fiber-optic digital network

Common infrastructure for wireline & wireless traffic Integrate packetized voice, video, data and circuit-switched voice on same

network

Ring architecture for physical redundancy (2+ disjoint paths)

SONET/SDH for framing and OAM capability (easier management)

MPLS LSPs for resilience (avoid SONET/SDH protection at PHY layer)

A Look at Planning ToolsA Look at Planning Tools(advancing the “science”)(advancing the “science”)





Canonical Structure of Automated Planning Tools

H/w DeviceModels

User Interface/GUI/Command Line

Simulate

Analyze

Design

InternalEngine

ModulesIGP

MPLS-TE

Protection

RerouteOutputs

Config.Files

Inputs

Inputs

(Topo, Cap., Failure)Analysis

Reports

Eco. AnalysisForecasting

NMS

EMS EMS

Vendor Data Extraction

EquipmentInventory

Network

Data ExtractionData Conversion

Element DataElement State/Stats

Measurements/Probes




OPNET SP Guru: Components & Features

ProcessInputs

Discovery via NMplatforms for topology

Simulate failure &overload to get:- Worst-case loading- % disrupted traffic- SLA violation

Config.Verification

FlowAnalysis

FailureAnalysis

NetworkDesign

NetworkSimulation

MPLSModule

Integrated topo.,config., util. datafrom VNE Server

Switch/routerconfig. filesTraffic & link util.

data from systems

Traffic matrix info. fromdata collection tools

- Family of design algos.- TE capability- Impact of new algos. ortechnologies, e.g. VPNs

- “What-if” scenarios- Reachability analysis- Arch. & cost for givenQos, delay, jitter, loss- Current n/w loading

- Distribution of trafficon network links

Network model: nodes, links,speeds, topology, metrics

Validate configs.Deploy validated config.

Export n/wconfig.

- Primary/sec. path layout- FRR backup tunnels

Network simulation and analysis




OPNET Solutions: Key Characteristics

NetDoctor for configuration & network operations analysis

VNE Server

Automated I/F to various network data components

Ability to build a complete network view

Hybrid simulation techniques

Provide balance between speed and resolution

Ability to add SP’s own rules, algorithms, modules

Facility to map actual IP addressing to internal network model




Cariden MATE: TE Process

DataCollection

DemandEstimation

RoutingOptimization

ChangeoverPlanning

User-definedConstraints

ChangeoverExecution

UserMonitoring

MATE’s TE System Components




Cariden MATE: IGP Traffic Engineering

Problems

Uneven link utilization

Heuristic/ad-hoc planning

Coarse capacity upgrade rules (e.g. at 50 or 75%)

Above 60% utilization expected in 6 mo!

Sample network with projected traffic growth in 6 months

Original network state




Cariden MATE: Routing Optimization

[Reproduced with permission: Cariden Technologies]

Objectives

Max. headroom on failure

Max. normal headroom

Minimize latency

Constraints

Fixed intra-site metrics

Symmetric weights

Latency bounds

Results

Max. link util. 89% 59%

Max. link util. on failure 110% 92%

All links brought to below 60% utilization

under normal conditions




Cariden MATE: Resilience Capabilities

Before (worst case) After (worst case)

>> 95% utilization

Max. utilization under 92%




Cariden MATE: Key Characteristics

Demand estimation & characterization Estimate p2p demands from aggregate node/intf. demands & routing

Estimate effective b/w per queueing class to meet QoS for demands

Robust routing changeovers Sequence of moves to transition network from one routing/LSP pattern to

another via a series of “make-before-break” operations

IGP metric-tuning based optimization in IP networks

Practical BGP simulations – peering, load balancing

Fully cross-platform – supports client/server or client-only model




A Perspective on Some Current Carrier Requirements

Accurate planning models

IP n/w planning with peering

Usability and consistency

Obtain precise traffic matrices

Good interface with monitoring tools

Intelligent heuristics

Extensible architecture

Application-level performance monitoring

Multicasting for multi-media services support




Concluding Remarks Designing and operating networks methodically will be crucial for

South-Asian carriers with growing networks

Services, not size, will dictate the sophistication needed

“Small” carriers offering same advanced services as “big” ones So, a systematic approach is must for sustained competitiveness

A “science” of network design can be used to … Identify potential problems, experiment with solns, verify designs

An “art” will always be there … Dictated by unique local or evolutionary needs

Tools and expertise exist to help operate world-class networks!

network planning & design: an art or a science?

Documents

india network

network simulation

network perf

network corebackbone

network unreliableunstable

network design system

art of network planning

sprint ld network