juniper vmx a2b internet - more-ip eventswitching hardware (pfe) nic ram ssd 12x1ge 2x10ge 1ge......

ImplementationJuniperNetworksvMX atA2BInternetBy ErikBais– A2BInternet

|MORE-IP2017presentation 2016Page|2

Whatisourbusiness??

oRegistrationofIPaddressesandASnumbers

o IPTransitinvariousDutchdatacenters

o Internet(Fiber)Access&DatacenterNetworkServices

o24*7MonitoringandmanagementofBGPinfrastructure.

o SpecializedconsultancyforISPrelatedtopicslikevendorselections,networkdesign&implementation.


Currently inthe following Dutchdatacenters


Shortintro

oA2BInternetisaDutchnetworkprovider.§ Providingdatacenterconnectivityandinternetaccessonfiber.

oWeimplementedtheJuniperNetworksvMX solutionrecently§ http://newsroom.juniper.net/press-releases/a2b-internet-deploys-juniper-networks-vmx-as-the-first-virtual-network-function--nyse-jnpr-11g134000-001

oButthe realquestioniswhy govirtual??…


Casestudy online


Previoussetup

oA2BInternethasalways beenan ExtremeNetworksshop…

oWenoticed that the BGPconverge took too longwith the current DFZsize

o Some updates(especially onthe AMS-IXpeering switch)took waytoo long..§ Slowupdatesofannouncing some prefixes..§ Orevensloweraccepting certain routes..§ Droppingpeers under highBGPload....(AARGGHHH!!)§ And properfiltering,madethings evenworse …

o Limitation wasinasinglethreaded BGPprocess onadual-core CPU


Let’shavealookatthevMX


vMX ProductOverview

VCPVFP

Physical NICs Management traffic

Guest VM (Linux) Guest VM (FreeBSD)

Hypervisor: KVM, ESXi

Cores Memory

Bridge / vSwitch

Physical layerPCI P

ass

thro

ugh

SR-IO

V

Virt

IO

Virtual Control Plane (VCP)• JUNOS hosted in a VM. Offers all the capabilities

available in JUNOS• Management remains the same as physical MX• SMP capable

Virtual Forwarding Plane (VFP)• Virtualized Trio software forwarding plane. Feature

parity with physical MX. Utilizes Intel DPDK libraries• Multi-threaded SMP implementation allows for

elasticity• SR-IOV capable for high throughput • Can be hosted in VM or bare-metal

Orchestration• vMX instance can be orchestrated through OpenStack

Kilo HEAT templates• Package comes with scripts to launch vMX instance


Architectural Difference with Shipping NFX250-S2

RIOT VMXT

SwitchingHardware(CrossconnectNIC) NIC RAM SSD

12x1GE 2x10GE 1GE2x10GE(internal)

...

External

X86CPU

LinuxHostOSVFP(PFE) KVMHypervisor

L2_TVPBSDJunos(JCP)

VirtualMachine

JunosDeviceManager (JDM)Container

LinuxBridge

SwitchingHardware(PFE) NIC RAM SSD

12x1GE 2x10GE 1GE

...

External

LinuxHostOS

X86CPU

VirtualControlPlane(VCP)

VirtualMachine

vMX on NFX Native NFX

SameHW

DifferentSW

ApproachvMX

DCPFE LCMD

BCMD

LCMD

KVMHypervisor

LinuxBridge

VNF#2

VNF#3

VNF#N

…vSRX2.0

VNF#1

2x10GE(internal)


OurUsedHWkitlist

oWeselectedHPasourvendorforthevMX setup.

oTheusedkitperbox:§ 1*HPProliant DL360gen92xE5-2650v4,64GB,2xPSU§ 2*HP560SFP+10GbePCIe Intelbased82599§ 2*HP240GBSSDHotplug 2.5inchSFF

o InShort..Enoughcore’s,enoughmemory,stickwiththe‘tested/recommendedNIC’s‘andsomeSSD’sforquickerbootingifneeded..


VMXlicenses…

oThevMX licensesthatweusearetheAdvanceversion..10G

oThereare1Gband5Gbversionsaswell..

oWedidn’tneedL3VPNor4Mil.routes..(yet)


Implementation

oYouneedtofollowtheJuniperimplementationguide..

oYes..RTFM!!..§ http://forums.juniper.net/t5/Day-One-Books/Day-One-vMX-Up-and-Running/ba-p/289129

oTheimplementationisquitepickyinkernelandlibraryversions.

o StartwiththerecommendedUbuntuversion..(notthelatest)

o Skipany idea ofrunningthis onVmWare ..Use Ubuntu+KVM… <period>


/home/vmx/vmxlite/config/vmx.conf

o SelecttherightimagestouseinKVMforVMX..

§ #Configurationonthehostside- managementinterface,VMimagesetc.§ HOST:§ identifier :vmx1 #Maximum6characters§ host-management-interface:em1§ routing-engine-image :"/home/vmx/vmxlite/images/junos-vmx-x86-64-16.1R3.10.qcow2"§ routing-engine-hdd :"/home/vmx/vmxlite/images/vmxhdd.img"§ forwarding-engine-image :"/home/vmx/vmxlite/images/vFPC-20161019.img"


/home/vmx/vmxlite/config/vmx.confo #vREVMparameterso CONTROL_PLANE:o vcpus :2o memory-mb :4096 #<=- 4Gbisbetterthan2Gb.2Gbworks.o console_port:2211o

o interfaces :o - type :statico ipaddr :<privateIP>o macaddr :"0A:00:DD:B0:DE:0E"


/home/vmx/vmxlite/config/vmx.conf

o #vPFEVMparameterso FORWARDING_PLANE:o memory-mb :24576o vcpus :22o console_port:2212o device-type:sriov #<=- YouwantandNEEDSR-IOV...o

o interfaces :o - type :statico ipaddr :<privateIP>o macaddr :"0A:00:DD:B0:DE:10” #<=- StaticMAC’s...Beware!!


WhatisSR-IOV?AndwhydoIwantthis?

o Single-rootinput/outputvirtualization

oSR-IOV isa networkinterface thatallowstheisolationofthe PCIExpress resourcesformanageabilityandperformancereasons.AsinglephysicalPCIExpresscanbesharedona virtualenvironment usingtheSR-IOVspecification.


SR-IOVallowsfordedicatedaccesstotheNICbytheVM


SR-IOVtakeaway…


WhatisSR-IOV?AndwhydoIwantthis?

o Single-rootinput/outputvirtualization

oSR-IOV isa networkinterface thatallowstheisolationofthe PCIExpress resourcesformanageabilityandperformancereasons.AsinglephysicalPCIExpresscanbesharedona virtualenvironment usingtheSR-IOVspecification.

ohttps://www.youtube.com/watch?v=hRHsk8Nycdg - IntelSR-IOVExplanation


Performanceo Weusea10Gfortransitand2*10Gtoourinternalnetworkpertransitbox.

o PCIExpresscando:

§ Source:Intel- http://www.intel.com/content/www/us/en/support/network-and-i-o/ethernet-products/000005811.html

PCIExpressImplementation EncodedDataRate UnencodedDataRate

x1 5Gb/sec 4 Gb/sec(0.5GB/sec) x4 20 Gb/sec 16 Gb/sec(2GB/sec) x8 40 Gb/sec 32 Gb/sec(4GB/sec) x16 80 Gb/sec 64 Gb/sec (8GB/sec)

Theoretical Maximum Bus Throughput:•PCI Express* (PCIe*) Theoretical Bi-Directional Bus Throughput.


TheIntelNIC

o Specifications:

§ HostInterface§ n PCIe BaseSpecification2.0(2.5GT/s)or(5GT/s)§ n Buswidth— x1,x2,x4,x8

oOurnetworkcardshave2*10GbSFP+..Andpercard40Gbps theoreticalthroughput..

o2NIC’sperbox..Whichleavesenoughroomforline-rateperformance…


Intothenetwork…oWestartedbymigratingIPv6ontothevMX’s beforeIPv4.

o OnceIPv6wasrunningwithoutanyissues....Wescheduledav4migrationintovMX.

o Transitsmigrationwasreallysimple..§ ThevMX’s feellikeyouareworkingonanactualJuniperMX.

oMigratingallthepeersonthepeeringbox,wasabitmorework.MostlyduetolegacypeerswithMD5orspecificroute-mapsforcertainpeers.

o Afullv4BGPtableloadisdonewithin4seconds!!§ NoneedtoprogramASICS/TCAM..Allroutesareusablewhenloaded..


Currentmaxbandwidthsincethemigration:

oAMS-IX10Glink:8.2GboTransit10Gblink:6.3Gb

oCPUusageAMS-IXrouter:


Futurepath?

o Itisalwayspossibletopastetheexactsameconfig intoaJuniperMX240orbigger...

oYoucanstarthere… and your development(automation)can be ported into anyother Junos environment.

oNextversions ofvMX will supportQSFP’s and 100GbNIC’s …

oAnd if you don’t likeit oroutgrow the setup,the hw can be re-used for other tasks …


Questions?

juniper vmx a2b internet - more-ip eventswitching hardware (pfe) nic ram ssd 12x1ge 2x10ge 1ge......

Documents