setting up and testing an nfv ecosystem | qualitest group

Setting Up and Testing an NFV Ecosystem

2

Agenda

Setting up NFV Ecosystem Testing NFV Ecosystem

Summary & Questions Use case for NFV TestingUse case for NFV Ecosystem setup

Introduction

3

Agenda



Introduction

4

Introductions

Bryon LowenVice-President Telco

Benny Sand NFV Strategy Lead

Mark AbolafiaVice President, SDN Practice

5

About QualiTest Group

| World’s 2nd Largest Independent Testing Company

| QualiTest was recognized as the only visionary pure play QA and testing Service Company in the “ Magic Quadrant”

6

About Datavision

| Datavision provides network engineering and software development expertise to build customer-driven SDN-based use cases:| Software Defined Networking controllers and Service Orchestration

solutions| Compute and Network Virtualization for datacenter/WAN

environments| Network Functions Virtualization

| Company Highlights:| Network Integration and Consultancy, established 2004| Experience in Architecting, Designing, Realizing Software Defined

Networking infrastructure| Leader in SDN and NFV Systems Integration| Clientele consists of Tier 1 Telecom Carriers, Large Enterprises| Strong ecosystem of technology partners| 130+ Networking Engineers and Software Development Professionals

7



Introduction

8

The NFV Ecosystem: What is it?

| The “NFV Ecosystem” is series of technical and business partnerships you can structure to minimize your time to adopt the technologies, products and processes to implement SDN/NFV Infrastructure

| The ecosystem is something that you can organize and build based on your business requirements and the Use Cases that you develop for investigating, testing, prototyping and implementing Software Define Networking in across your enterprise.

| It also needs to feed into, and draw from, your Line of Business, HR and Engineering groups| Re-Training/Supplemental Training of existing staff| Sourcing the right people| Ensuring the business (and IT/Engineering) understand the ramifications of adopting the technology

9

The NFV Ecosystem: Where do you Start?Analysis and Business

Case Development

What to Virtualize?/Business

Value

Identify Dependencies for Use-Case

Implementation

Develop Migration Plan

Develop ROI Metrics based on Migration

Plan

10

The NFV Ecosystem: Tips and Techniques| Much of this technology is “new”. new Companies, new products, new

approaches| Advantages:

| Leverages the most advanced technologies and techniques| Typically small startups – Agile – willing to do whatever it takes for adoption to

Inaugural clients, early adopters| Advantageous commercial terms| Avoids Vendor lock-in

| Disadvantages| Typically small start ups (!). Will they be around two years from now?

Financials/staying power| Sometime field resource constrained – can they help you analyze/implement as

effectively.| Typically “One-Horse” to offer in a “point solution” versus portfolio (not

insurmountable, but differs from larger, broader technology firms)

The NFV Ecosystem: Tips and Techniques

| Start simple – define a single use case| Avoid Analysis paralysis| Keep focus – try not to utilize more than 2 vendor solutions at a time to

execute Use Case.| Ensure Buy-in from the business.

11

12

The NFV Ecosystem: Tips and Techniques (Cont)

| Build the Proof Of Concept| Not just to “show it works” - but design it so things break to expose

weaknesses in aspects of the operation or skill sets| Fail Fast – fix, repeat.| Be prepared to look at Workflows and Business Processes along

the way – This technology is truly transformative from both a network infrastructure standpoint and a back-office operations standpoint.

13



Introduction

15

QualiTest NFV validation process

QualiTest aims to create a unified end-to-end approach to service assurance and network

performance

NFV Management Orchestration Infrastructure Migration

16


Assure full compliance to ETSI ISG Standards.

•Define NFV Testing process•Test Requirements•NFV Test Environment•Risk Assessment•Roles and Responsibilities•Team Skills and Training•Test Inputs / Basis

Assure full compliance to NFV requirements for any network equipment provider and streamline the ability of vendors and users to be able to implement compatible NFV solutions.

Test Environment: simulation tools, network analytics, QoS and QoE probes.Automated Certification Process for each VNF before it is introduced to the network

Test management and Orchestration prior to VNF migration, including each component (e.g. Orchestrator, Vnfm, VIM).

Test the VNF architecture and implementation to assure smooth transition.

NFV Test Policy

ETSI NFV ISG Compliance

(NFV-Co)

Selecting Mano Vendors

(NFV-Ven)

NFV Staging Environment

(NFV-SE)

MANO Testing (MANO-T)

VNF Migration (VNFM-T)

17


Assure full compliance to ESTI ISG Standards

ETSI ISG Standards Test procedure

Assuring compliance aims to find the deviations from standards• MANO• VNF/NE

• Analyse ISG current standards • Prepare compliance requirements • Analyse each NFV component / int • Define gap points and analyse • Repeat steps 1-6 for each VNF and NE

NFV Test Policy

ESTI NFV ISG Compliance

(NFV-Co)


(NFV-Ven)

NFV Test Environment

(NFV-TE)



18


Assure full compliance to NFV requirements for any network equipment provider and streamline the ability of vendors and users to be able to implement compatible NFV solutions.

NFV Test Policy


(NFV-Co)


(NFV-Ven)


(NFV-TE)



ETSI specifications on Network Functions Virtualization

• NFV Infrastructure; Hypervisor Domain• NFV Resiliency Requirements• NFV Architectural Framework• NFV Terminology for Main Concepts in NFV• NFV Infrastructure; Compute Domain• NFV Infrastructure; Network Domain• NFV Service Quality Metrics• NFV Management and Orchestration

• NFV Virtual Network Functions Architecture• NFV Security; Security and Trust Guidance• NFV Performance & Portability Best

Practices• NFV Proofs of Concept; Framework• NFV Infrastructure; Methodology to

describe Interfaces and Abstractions• NFV Security; Problem Statement• NFV Use Cases• NFV Virtualization Requirements

19


Build a comprehensive Test Environment, including Control plane and Data plane workload simulation tools, network analytics, QoS and QoE monitoring probes.

NFV Test Policy


(NFV-Co)


(NFV-Ven)


(NFV-TE)



NFV provides great flexibility:

• Build parallel test environments

• Scale up and down compute resources

• Copy any existing environment

Environment build guidelines:

• Management and Orchestration should be similar to the production

• NS and VNF Instantiation should be similar to production environment

• Same level of resources should be allocated.

Test Appliances:

• Simulate workload traffic on the client / server

• Simulate both data plane & control plane traffic

• Measure key metrics data & control plane

Virtual Appliances vs. Physical Test

Appliances:• Virtual test appliances

are more cost effective • Physical test appliances

are recommended for highest levels of data-plane performance or microsecond-level timing accuracy

20


Test management and Orchestration prior to VNF migration, including each component (e.g. Orchestrator, Vnfm, VIM).

NFV Test Policy


(NFV-Co)


(NFV-Ven)


(NFV-TE)



The first step is to test NFV Management and Orchestration Architectural Framework

NFV Orchestrator

VNF Manager

(VIM) and NFV Infrastructure (NFVI)

VIM– VNF Manager (Vi-Vnfm)

Orchestrator – VIM (Or-Vi)

The VNF is deployed as one or more Virtual Machines; src ETSI GS NFV 002 v1.2.1

21


Test the VNF architecture and implementation to assure smooth transition.

NFV Test Policy


(NFV-Co)


(NFV-Ven)


(NFV-TE)



Static testing VNF-Mano Integration

Conformance testing VNF Internal functionality

Scalability/Performance End-to-end/system test

22



Introduction

23

NFV Ecosystem Setup| Use Case(s): Automate Cloud Infrastructure-As-A-Service and MPLS VPN

Offers| New Service Offering by a Telco| Manual processes in place to provision MPLS VPN| No existing automation in place to provide Customer Self-Service Provisioning for

either Service Offer.| Business Needs:

| Provide new IaaS services| Automate Provisioning internally for both IaaS and MPLS VPN| Need to roll out “quickly”| Need to be able to be agile with respect to Customer Self-Service, Internal resource

management

24

The NFV Ecosystem: Where do you Start?

• Ecosystem also created• Internal constituencies brought into project; i.e. HR, business,

IT/Engineering

• System Design specifics identified and architected• Vendor solutions evaluated and selected• Integration “glue” defined, initial development

• Run and “break” against criteria, internal operations• Reiterative process to integrate learnings in to refined

architecture/solution• Final vendor product(s) selected/integrated

• Develop, Integrate, Test, Test, Test.

Use Cases Defined

System Requirements

Defined

System Design Document Created

(vendors, etc)

PoC/Demo Built/Tested Against

Criteria

Design/Build Production System

25



Introduction

Example:MMS VNF Test Strategy

Multimedia Messaging Service (MMS) is similar to the Short Message Service (SMS) in that a message is passed between an Originating Party (OP) and a Destination Party or Parties (DP).

The core Network Element is the Multimedia Messaging Service Centre (MMSC) which acts as a central store and forward server between the OP and DP as shown below.

26

Test Phases

28

Factory Acceptance Testing

• Compliance testing for each VNF introduced to the network:

NFVI –test facility integration•Ensure full validation against NFVI in labs

VNF compliance testing:•VNF Compliance Criteria – functional criteria

End 2 end testing:•Test service end2end to ensure full integration

FAT

NFVI

VNF

E2E

DevOps Automation

29

| Every layer of testing should be automated

| The concept of executing all regression test cases for a test pass is soon becoming obsolete.

| Strategy around testing new features need to be formalized and interim builds can be supplied to QA

Test strategy goals

| Build an NFV Ecosystem

| Drive Quality and governance

| Continuous integration model

| Reduce cost of integration

30

31



Introduction

32

Summary

| NFV – Testing challenges deep interdependencies and lightning fast technology introductions hence it becomes the quality gatekeeper for nfv

| If an operator is to find benefits in the new technology, they primarily want reliable services, at the right price, supported by strong SLAs.

| The move to a software-centric architecture means becoming more software focused – skills pivot

| Network DevOps is the future in NFV| Start with the business (process, skills, need) and the

Use Case| Develop your PoC criteria and execution plan| Fail fast, reiterate| Execute with narrow selection of vendor(s)

33

Q&A

www.QualiTestGroup.com

setting up and testing an nfv ecosystem | qualitest group

Technology